# I love you so jevving much — Diogo Almeida, TypeSafe AI

Diogo Almeida, TypeSafe AI

[Talk page](https://ai.engineer/talks/Cyt3LU8TyNc-i-love-you-so-jevving-much) · [Video](https://www.youtube.com/watch?v=Cyt3LU8TyNc)

## I love you so jevving much

A model can write a research report that Diogo Almeida would struggle to match and still give an answer he would hesitate to use. His case for Jev starts at the moment a program has to act on that answer, with consequences for agent memory, security checks and type systems.

### Why is accounting harder to automate than a research report?

[0:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1s)

If difficulty alone explained what AI could do, routine bookkeeping ought to come well before prize-level mathematics. Yet someone watching the benchmarks sees extraordinary performance on hard reasoning problems, while someone looking at everyday work sees people still doing the accounting. [Diogo Almeida](https://www.linkedin.com/in/diogomda), cofounder of TypeSafe AI and a contributor to ChatGPT, enjoys coding agents and chatbots. What bothers him is how readily success on the first kind of task gets treated as a forecast for the second.

Who gets to inspect the answer before anyone depends on it? Take deep research. Almeida thinks the system can produce a better report than he can. Now suppose you have to pick whose answer to use, in advance, and the only thing that counts is whether it is correct. He thinks you would pick him. A human reader can take what is useful from an impressive report and investigate a doubtful claim. A program handed a decision needs to know what to do next.

A drive-through makes the distinction concrete. The interaction can please the customer and still yield the wrong order. Automation requires a decision the ordering software can use, after which ordinary code can carry on. Almeida traces the mismatch to a technique he helped develop, reinforcement learning from human feedback (RLHF). Human preferences supply a training signal for the kind of response a person wants to receive. That signal works remarkably well for assistance. It rewards operational correctness only to the extent that the person giving feedback can recognize it.

Tasks with cheaply verifiable answers are a partial exception. A system can run on its own and defer inspection until the end. Training with verifiable rewards also gives a model a signal tied to success on the task, although Almeida finds models trained that way difficult to interact with until they receive human-feedback training. Background work poses its own test. The decision has to hold up after the person leaves.

![The left column bears a red stamp reading “Human in the loop” and “Assistance.” The right stamp reads “Machine only ideally” and “Automation.”](https://ai.engineer/api/talk-essay-media/20b2a9a760d023551e19023136c6af712f8c899ae1bb379430584ba6b357053e)

Almeida draws the line at who judges the result. DeepResearch can help a person who checks the answer. Automating a drive-thru means the system has to get the order right on its own.

### What should a machine-native model practice?

[5:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=339s)

**Jev** is [TypeSafe AI](https://docs.typesafe.ai/introduction)’s attempt to train a model whose decisions another program can use. Its interface is only part of that bet. Almeida starts with a loose reading of Sutton’s bitter lesson, that methods able to use more compute tend to win over clever algorithms. He puts two decisions ahead of buying compute. Where will the training data come from? And what, exactly, will the model practice doing with it?

Chess hides both decisions. The rules tell you what counts as a win. Spending compute to play out games produces more experience under those same rules. It is easy in that setting to credit compute for the improvement, because the game supplies a well-defined task and turns computation into data. With language pretraining, researchers had to choose what a model should learn from a finite supply of text.

Masked language modeling asks the model to recover a missing piece of text. Next-token prediction asks it to continue a prefix. The latter has a useful consequence at inference time. Append the prediction to the prefix and the model has another instance of the task it practiced. It can keep generating a string. Getting better at filling in a blank serves a different purpose, even when both objectives use the same text.

How much can changing the task buy? In the early RLHF work Almeida recalls, researchers first needed data about which responses people preferred. With that feedback, even a small model using simple training algorithms made a striking gain on instruction following. Almeida expresses the size of the gain by estimating how far researchers would have had to scale pretraining alone to match it. The extrapolation concerns one capability. The experimental result was that a small model got markedly better at following a request once its training signal rewarded that behavior.

Jev takes its cue from that experiment. If another piece of software consumes the answer, dependable decisions on a narrow task are worth training for directly. Almeida credits the choice of task for Jev’s behavior. Its particular training objective and data recipe remain unspecified.

![The pyramid’s data tier has an arrow labeled “human feedback / verifiable environments.” An arrow labeled “RLHF / RLVR” points to the base tier, “doing the right task.”](https://ai.engineer/api/talk-essay-media/53de4025740328bb3f694c3f6e98e59dadb5d764714fbc207d20e43af69b4fc3)

Almeida puts task choice and training data at the base of the LLM pyramid, below compute and algorithms. Human feedback and verifiable environments supply the data. RLHF and RLVR use it to train the model on the intended task.

### The call an application can depend on

[9:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=568s)

Imagine getting Einstein 95% of the time and Mr. Bean the other 5%. In Almeida’s analogy, the trouble is that the caller cannot tell who answered this time. A person who occasionally needs a brilliant idea can check it afterward. Software that acts on every answer inherits every mistake. Almeida attributes developers’ enthusiasm for Jev to its dependability on a focused task. Speed, price and a convenient interface alone would leave the caller with the same predicament.

A coding agent can generate software just in time and make it cheaper to write. Once delivered, the application can still behave much as it did a decade ago. A chatbot alongside it goes a step further by interacting with the running application, while leaving costly decisions to the user. What Almeida calls *smart software* puts judgment inside the application’s own execution loop. The program encounters a case, calls the model for the part that needs judgment, then uses the result to continue. At that handoff, a more dependable model changes which decisions the product can take on.

### What the blue LED suggests about a fourth logic gate

[12:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=740s)

Red and green LEDs made good indicators and clocks. Adding blue sounds like a 50% increase in the choice of colors. The interesting change came from what engineers could do with that third color. White light and full-color displays became possible, putting LEDs to work in products that had little in common with an indicator lamp.

Software already builds elaborate behavior out of binary operations and comparisons such as greater than, less than and equal. Almeida imagines a conceptual fourth logic gate that lets code ask a question about meaning. Consider an email address. A programmer can spell out a pattern for a Gmail address in a regular expression. Asking whether an address *looks real* expresses a different condition, one a person can often judge without being able to enumerate the rules. If a model could return that judgment cheaply, quickly and dependably, existing code could branch on it. A new kind of question becomes available wherever the program already makes decisions.

Which combinations would matter is hard to predict from the primitive alone. Almeida calls the work of trying them *software research*. His precedent is the early web, where people could build a strange site simply because the tools allowed it. Enough people experimenting independently found uses that would have been hard to justify in advance.

### Agent memory beyond one KV cache

[15:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=924s)

A program with a model in its inner loop still needs decomposition. Break the task into stages and give each stage a result that can be checked. Then the next stage can use that result without reconstructing everything the previous stage did. State needs the same care. If every stage relies on whatever happens to be in a single model context, the program has effectively acquired global state.

There is an efficiency incentive to build agents that way. The key-value (KV) cache retains attention keys and values for the context the model has already processed. Extending the same history lets it reuse that work. Rearranging the context or sending work to an agent with a different history can sacrifice some of that reuse. An agent architecture can look sensible as software and prove expensive as a sequence of model calls.

Consider two kinds of lost work in a coding agent. Across sessions, the system may have the entire history on hand yet lack a good way to select which earlier discovery matters to a new call. Across concurrent agents, several workers may each spend time rediscovering the same pattern in a codebase. The first case calls for selective lookup over stored experience. The second calls for shared state that lets one worker benefit from another’s discovery.

Routing work to subagents raises both questions at once. Which agent should get the task, and what must it know to get started? A clean division of responsibilities can still duplicate expensive context processing. Cache-aware routing, shared discoveries and efficient retrieval are open engineering problems in Almeida’s account. Their payoff would be a division of labor that saves work in the running system.

![Bullets ask how the KV cache affects memory scaling, state sharing and subagent routing. Two pie charts compare expected and actual agent costs, with output largest in the first and cache reads largest in the second.](https://ai.engineer/api/talk-essay-media/7ad7ef4dbaeb97b93805e232ce9dd9c39c102964ccaa5db5ff89dfd359c929f1)

Almeida’s chart puts rereading cached context at 1.5% of the cost developers expect and 67% of the cost at a 100% cache hit rate. He asks why coding agents have to rediscover the same patterns in a codebase when they could share state.

### Where a program needs to know what someone meant

[17:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1049s)

The most speculative direction begins with the principal–agent problem. Someone delegates a job to a representative whose incentives may differ from their own. Citizens and elected officials are Almeida’s example. Writing explicit rules can make enforcement fairer, but a representative can satisfy the letter of a rule while defeating its purpose. Software runs into a related difficulty when the condition a programmer can encode captures only part of what they meant. Could an application check the intended property where a decision actually happens?

- **Real-time security.** Recognizing that an action is suspicious may take less intelligence than carrying it out. A defender still needs the judgment early enough to intervene, at a cost it can afford on routine actions, with enough reliability to act on the result. A model that spots the problem on retrospective inspection meets only part of that requirement.
- **Semantic types.** A developer wants to declare locally that a property must hold. Convincing yourself that it holds can require reading down a chain of dependencies. Almeida imagines a Jev-like model in the lint checker that evaluates the semantic property where it is declared. How dependable such a check could be remains open.
- **Voice interfaces.** The old ambition was to tell software what you wanted and have it infer the intended action. Earlier attempts ran into the sheer number of rules someone had to hard-code. A spoken request becomes a different sort of interface if the software can interpret what the user means after it hears the words.

### Who pays for slop, and who gets to play?

[19:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1184s)

Unreliable AI already has a use wherever somebody else bears the cost of a mistake. A company may hesitate to let a model make a decision that could lose it money, yet feel comfortable sending out material nobody at the company has read. The recipient inherits the job of sorting, checking and discarding it. That asymmetry is Almeida’s explanation for slop. Reading and filtering before sending puts the work back on the producer. A reliable filter on incoming material could also give the recipient some relief.

How much play can you get out of an account-deletion prompt? Almeida describes a Bad UI Battles entry that turns that familiar prompt into a game. The joke works because everyone knows how straightforward the interaction ordinarily ought to be. He likes these exercises because the maker cares about the result, even when the intended result is a horrible interface. Give people room to pursue enough odd ideas and one of them may uncover a possibility worth keeping.

Engineers carry a mental list of things they do by hand because they cannot see how to express the decision in code. Almeida’s closing invitation is to revisit that list. If a program could use a judgment about meaning as readily as it uses a comparison, the question for some of those jobs would become how much engineering the automation takes. His parting objection to benchmarks is on behalf of the engineer with time to consult only a score. That engineer needs to know whether the next call is dependable enough to build on. Making a semantic decision as reliable as a logic gate is still the ambition.

## Key takeaways

- For an automated decision, ask how the caller can tell when an answer is safe to use. Occasional brilliance helps much more when a person can inspect the result.
- Judge a training task by the operation it makes possible afterward. Repeated next-token prediction generates strings. Human preference data made a small model better at following instructions. Jev pursues the corresponding training problem for decisions consumed by code.
- Keeping a coding agent’s history solves only the storage problem. A later call needs a way to retrieve what matters, concurrent agents need a way to share discoveries, and the cost of processing their context can determine whether dividing the work pays off.

## Slides

### Platinum Sponsors

The opening slide credits six platinum sponsors. AWS, Devin, Databricks and Exa appear across the top row, with Factory and Sonar centered below.

[0:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=8s) [Under “Platinum Sponsors,” AWS, Devin, Databricks and Exa appear in the top row, with Factory and Sonar below.](https://assets.ai.engineer/sites/aiecode2025/sha256/fd6fca8e00f5fbcf673afb823d450ecf66215865e6d1dca543afc5d3090da660.webp)

PLATINUM SPONSORS
aws   Devin   databricks   exa
FACTORY   Sonar

### The Jev! Guy

Diogo Almeida introduces himself with a mock election landslide. In a bright pink coat, he spreads his arms in front of a map that gives Jev all 538 electoral votes. His work on ChatGPT, RLHF and InstructGPT gets a footnote.

[0:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=24s) [A man in a bright pink coat spreads his arms in front of a U.S. electoral map showing all 538 votes for Jev. A footnote also mentions ChatGPT, RLHF and InstructGPT.](https://assets.ai.engineer/sites/aiecode2025/sha256/94d0ae08bd53d49bb6ac9ea4dc00a4c343e617cb76dd987225537fed248346de.webp)

the Jev! guy

Jev 538
Undecided 0

Map labels:
WA 12, OR 7, NV 6, ID 4, MT 4
ND 3, SD [illegible], NE [illegible], MN 10, IA 6, MO 10
WI 10, MI 16, IL 20, IN 11, OH 18
KY 8, TN 11, WV 5, VA 13, NC 16, SC [illegible], FL 29
PA 20, NY 29, VT 3, NH [illegible], ME 4
[illegible]4 NJ, 3 DE, 10 MD, 3 DC
AK 3, HI 4

Split Electoral Votes
ME 3   1   ⓘ
NE 3   1   1

*also ChatGPT/RLHF/InstructGPT/etc.
TypeSafe AI

Interface text:
270 TO WIN
Generated Map
Share Map

### Where Is All the Automation?

A puzzled man holds out one hand as if asking where all the automation went. Almeida asks why AI can do so many impressive things and still be so little help with other tasks.

[0:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=36s) [On a pink slide, a puzzled man holds out one hand. The meme reads “WTF is going on in AI” above him and “and where is all the automation” below.](https://assets.ai.engineer/sites/aiecode2025/sha256/0a66c790d0cf4c4fdd375006152728d439e1ecce1d8cf97c53eabd51c3fcca1f.webp)

WTF IS GOING ON IN AI

AND WHERE IS
ALL THE AUTOMATION

TypeSafe AI

### My Relationship to ChatGPT

Diogo Almeida puts himself where ChatGPT’s makers and critics overlap. He helped make ChatGPT, loves using coding agents and chatbots, and speaks up about their limits.

[1:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=68s) [A green “Made ChatGPT” circle contains the OpenAI logo. A red “Hates on ChatGPT” circle contains portraits labeled Yann LeCun and Gary Marcus. Their overlap reads “Me.”](https://assets.ai.engineer/sites/aiecode2025/sha256/2451ebcba1d07256ab1ff74d77e675e705e3efce516bc1a00bc96d3057088b6a.webp)

MY RELATIONSHIP TO CHATGPT

MADE CHATGPT
OpenAI

ME

HATES ON CHATGPT
Yann LeCun
Gary Marcus

### Today’s AI Frontier Is “Jagged”

Three tasks that seem equally difficult sit on the same dashed arc. The blue curve marks the edge of what AI can do. One task falls within reach, one sits at the limit, and one lies beyond it.

[2:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=157s) [A dashed arc represents tasks of seemingly equal difficulty. A jagged blue curve crosses it, and three diamonds mark tasks inside, on and outside the AI capability frontier.](https://assets.ai.engineer/sites/aiecode2025/sha256/ee41a173094728b1d37514758970384a1e164edbbbed1b5c2faa530534da021d.webp)

In other words: today’s AI frontier is “jagged”

“We suggest that the capabilities of AI create a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI.”
- Dell'Acqua, McFowland, Mollick et al. 2023

seemingly equal-difficulty tasks
the jagged frontier

task inside the frontier
task on the frontier
task outside the frontier

diagram adapted from Ethan Mollick
CSET
cset.georgetown.edu

### The Sane View of AI — What Explains This Line?

Customer service appears on both sides of Almeida’s line. When AI has no decisions to make, he groups it with coding agents. Customer service more broadly sits alongside accounting and other work that requires reliability.

[2:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=175s) [Two lists headed “Too good to be true” and “Too bad to be useful” place coding agents on the left and accounting on the right. A red arrow points to the divider and asks “What explains this line?”](https://assets.ai.engineer/sites/aiecode2025/sha256/140a01b1b43bdcd4951b6ca77fc91359b3512561972e631450ace24b3ae25c8e.webp)

The Sane View of AI

Too good to be true
• Instruction Following
• ChatGPT
• Copilots
• Coding agents / Claude Code
• NotebookLM
• DeepResearch
• Voice agents
• Customer Service w/o decisions
• Slop / Spam

Too bad to be useful
• Customer Service
• Drive Thrus
• Accounting / taxes
• QA
• Insurance
• Data entry
• Anything requiring reliability

What explains this line?

### The Sane View of AI — Assistance and Automation

Almeida draws the line at who judges the result. DeepResearch can help a person who checks the answer. Automating a drive-thru means the system has to get the order right on its own.

[3:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=217s) [The left column bears a red stamp reading “Human in the loop” and “Assistance.” The right stamp reads “Machine only ideally” and “Automation.”](https://assets.ai.engineer/sites/aiecode2025/sha256/20b2a9a760d023551e19023136c6af712f8c899ae1bb379430584ba6b357053e.webp)

The Sane View of AI

Too good to be true
• Instruction Following
• ChatGPT
• Copilots
• Coding agents / Claude Code
• NotebookLM
• DeepResearch
• Voice agents
• Customer Service w/o decisions
• Slop / Spam

HUMAN IN THE LOOP
ASSISTANCE

Too bad to be useful
• Customer Service
• Drive Thrus
• Accounting / taxes
• QA
• Insurance
• Data entry
• Anything requiring reliability

MACHINE ONLY IDEALLY
AUTOMATION

### Why Do All LLMs Require a Human-in-the-Loop?

The cartoon shows optimization trying to fit humans into the loop. Almeida argues that RLHF puts them there by design. Even when an easy-to-verify task runs on its own, a person still checks the result.

[4:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=275s) [A cartoon figure labeled “Optimization” tries to fit a block labeled “Humans” into a shape sorter labeled “The Loop.” The caption asks why all LLMs require a human in the loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/41980f3f982f51eaf56af0cbc23d37bb83e77b736b0d4992d39702f4fe8ed95a.webp)

WHY DO ALL LLMS REQUIRE A HUMAN-IN-THE-LOOP

OPTIMIZATION
HUMANS
THE LOOP

### WTF is going on in AI?

Almeida argues that 100% of LLMs are optimized for assistance and 0% for automation. That training gap explains why he trusts them to answer questions but doubts they can work unattended. He guesses automation is worth more.

[4:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=293s) [A numbered list claims that 100% of LLMs are optimized for assistance and 0% for automation. It guesses that automation is more valuable, then ends with “???” and “Profit.”](https://assets.ai.engineer/sites/aiecode2025/sha256/d6817d7bb4336063d0bda3311622c7b0b2ee1bb1fdcf381a46f815ab6b509beb.webp)

WTF is going on in AI?

1. 100% of LLMs are optimized for assistance (RLHF)
2. LLMs are too good to be true at assistance
3. 0% of LLMs are optimized for automation (RL??)
4. LLMs are too bad to be useful at automation
5. Guess: $(automation) > $(assistance)
6. ???
7. Profit

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

### Viva la Jevolucion

A vintage revolutionary poster spells JEV in the same hot pink as the figure’s mustache and fur coat. Almeida uses the pun to introduce Jev’s approach to training AI for automation.

[5:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=337s) [The poster reads “VIVA LA JEVOLUCION,” with “JEV” in pink. Below it, a figure in a sombrero holds a rifle and has a bright pink mustache and coat.](https://assets.ai.engineer/sites/aiecode2025/sha256/eb554b53e422445b666f77dd383217fbe33c16166a445d2bd474882ff8de0499.webp)

VIVA LA
JEVOLUCION

TypeSafe AI

### Forking the Tree of Research

Almeida traces one path from pre-trained language models through RLHF chat models to RLVR reasoning models. He proposes a second branch directly from pre-training: RLCD, which trains models to make calibrated decisions.

[5:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=357s) [A diagram runs from LM to LLM, then forks. The upper path connects RLHF chat models to RLVR reasoning models. The lower path leads to a green box labeled “RLCD — Reinforcement Learning for Calibrated Decisions.”](https://assets.ai.engineer/sites/aiecode2025/sha256/e289b5df9fa0cdf24be7ef46dbe81b6c0d653f4d23d5d590acd0bd72a0079f58.webp)

Forking the Tree of Research

LM
BERTs
Low Intelligence
GPT

LLM
Pre-Trained Models
High Intelligence
GPT-3
LLaMa

RLHF
Chat Models
Easy Assistance
Chat GPT
Claude
Instruct GPT
Grok
Deep Seek

RLVR
Reasoning models
Hard Assistance
o1
o3

RLCD
Reinforcement Learning for Calibrated Decisions

### The Bitterest Lesson

Almeida argues that compute matters more than algorithms and data matters more than compute. Then he adds a wider base beneath all three: doing the right task matters most.

[6:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=390s) [A pink pyramid has three labeled tiers. Data forms the broad base, compute sits above it, and algorithms occupy the tip.](https://assets.ai.engineer/sites/aiecode2025/sha256/d2574d8b76cda40e762c92fa89b555bceb46c33c5c93fb91b1cf8222beee59ca.webp)

A Bitterer Lesson

algorithms
compute
data

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

[6:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=399s) [A pink pyramid has four tiers. From base to tip they read “doing the right task,” “data,” “compute” and “algorithms.”](https://assets.ai.engineer/sites/aiecode2025/sha256/0ad5522385f2b30562e6aef94d2b91b2b9638b2e0472746dc069322954f590ef.webp)

The Bitterest Lesson

algorithms
compute
data
doing the right task

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

### Classical reinforcement learning games

In games like chess, the rules define what counts as success. The algorithm can generate its own training data by playing out games. That’s how compute supplies the data tier of the pyramid.

[7:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=423s) [On the four-tier pyramid, an arrow labeled “derived by compute” points to “data.” A second arrow labeled “defined by rules” points to “doing the right task.”](https://assets.ai.engineer/sites/aiecode2025/sha256/34999643cf9e2c1bd3440dba82f2986909704a8a8c3ef598bd1f2997292c954d.webp)

This it true for classical RL games

algorithms
compute
data ← derived by compute
doing the right task ← defined by rules

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

AI Engineer
Code Summi

### This is extra true for making LLMs useful

Almeida puts task choice and training data at the base of the LLM pyramid, below compute and algorithms. Human feedback and verifiable environments supply the data. RLHF and RLVR use it to train the model on the intended task.

[8:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=490s) [The pyramid’s data tier has an arrow labeled “human feedback / verifiable environments.” An arrow labeled “RLHF / RLVR” points to the base tier, “doing the right task.”](https://assets.ai.engineer/sites/aiecode2025/sha256/53de4025740328bb3f694c3f6e98e59dadb5d764714fbc207d20e43af69b4fc3.webp)

This is extra true for making LLMs useful

algorithms
compute
data ← human feedback / verifiable environments
doing the right task ← RLHF / RLVR

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

AI Engineer
Code Summi

### Jev’s secret: reliable intelligence

Almeida says developers prefer Jev to copycats because his team optimized it to do a narrow task reliably. He argues that benchmarks miss the question developers face when they automate a task: can they count on the model every time?

[9:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=557s) [Bullets connect Jev’s optimization goal with developer preference and question the value of benchmarks. Beside them, a gray T-shirt reads “jev ne sais quois.”](https://assets.ai.engineer/sites/aiecode2025/sha256/b7e39d2b64c34cfad30b63d419c7e13d3c32c13b38b962ba4bb2902c10e33d03.webp)

Jev’s secret: reliable intelligence

• you get what you optimize for
• this is why benchmarks don’t matter
• this is why people love the model over copycats
• this is why there was a “ChatGPT for developers” moment
• NOT: speed/cost/interface

jev ne sais quois
typesafe

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

### What is Jev for the world?

Almeida puts Jev at the base of a stack of smart software. Developers can program it to automate work in the background. He predicts a SaaS boom that brings the energy of the early internet back to software engineering.

[10:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=632s) [Bullets describe Jev as programmable AI that developers can run in the background for automation. A drawing labeled “smart software” shows stacked blocks, with a TypeSafe AI arrow pointing to a narrow support near the base.](https://assets.ai.engineer/sites/aiecode2025/sha256/3243d93c7a3052c9718adbbe20e16ea75d7f69772d10c50ad4d856881b3490c8.webp)

What is Jev for the world?

• AI that is
  ○ made for automation
  ○ programmable
  ○ made to run in the background
  ○ for devs!
• software is so over → we’re so back
• SaaSpocalype → SaaSapalooza
• SWE == permanent underclass → golden age of SWEs: back to early internet energy!!!

smart software
TypeSafe AI

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder  TypeSafe AI

### Traditional, just-in-time and smart software

Traditional software follows fixed branches, one ending in an SOS. The just-in-time sequence ends at a question mark above an overwhelmed brain. Almeida’s smart software puts AI at decision points, with a money bag at the end of every path.

[10:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=656s) [The traditional flowchart branches to three money bags and an SOS robot. The just-in-time sequence ends in a question mark above an overwhelmed cartoon brain. The smart flowchart places brains at decision points and leads to four money bags.](https://assets.ai.engineer/sites/aiecode2025/sha256/92b6d20b6c2aa47c430fb6f5b528b73b09bcc78438ce7a6abd2fe3c7c53c9203.webp)

Traditional Software
$   $   $   SOS

Just-in-time Software
?
SOS
SOS

Smart Software
$   $   $   $

### People are jevving

Almeida reports that 30+% of the Fortune 500 use Jev, usage reached 1 trillion tokens per day by the first weekend, and Jev has already saved millions of dollars. The project gallery ranges from playing Catan to formatting spreadsheets.

[11:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=692s) [Beside a gallery of project thumbnails, the slide reports Jev use by 30+% of the Fortune 500, one trillion tokens per day by the first weekend, and millions of dollars saved.](https://assets.ai.engineer/sites/aiecode2025/sha256/152ff624ef5b380ab322bcb8516fb32c74c42ac28bfda508bde70d3935d3ed41.webp)

PEOPLE ARE JEVVING

• 30+% of fortune 500 using jev
• 1T tokens/day by the first weekend
• $Ms already saved
• SO MUCH CREATIVITY !!!

TypeSafe AI

AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

Interface text:
All projects
Featured

Jev plays Catan
wanted to see what happens when a few Jevs play Catan so I forked...after a while the Jevs just refuse to negotiate...
Sachin

Ask Jev
Ask Jev anything.
Ask Jev anything. Give it a try at [illegible] answer. It will judge. Let's see if we can get to 1 million questions. @typesafeai...
Wayne Sutton

A jargon-detecting pigeon
I put Jev inside a pigeon. it has one job: find jargon in your writing and take a dump on it.
Prateek K. Keshari

JevForm
introducing JevForm, a form that dynamically branches and chooses what to ask next using @typesafeai's...
Tamir

traffic-guard
Pre-flight bot and attack detection gate for web traffic
Baked traffic-guard: an offline bot gate for reverse proxies with zero dependencies. Uses wire-order heade...
Hemanth.HM

TC39 Atlas
TC39 Proposal Atlas
For years I wanted to classify @TC39 proposals to spot common patterns. Using @typesafeai's Jev, I evaluated...
Hemanth.HM

Find in page by meaning
ctrl-f has never understood a single query. It matches strings. This one scores meaning: every sentence on th...
Jiayao Yu

rtrvr.ai with Jev
Make every site work for you.
Jev is now live on @rtrvrai, our free browser agent. No API key needed. With Jev enabled, our recorded tasks took...
Bhavani Kalisetty

Jev Search
Built Jev Search -- free and open source, powered by @typesafeai Jev and @search1api_dev search API.
Ask in plain language. Jev picks where to search and ranks what comes back.
Try it now:
fatwang2

pkg-gate
So, I baked pkg-gate with @typesafeai! npm lifecycle scripts can run arbitrary code on install. pkg-gate: a ~100ms...
Hemanth.HM

jev-codes
Hey Everyone, Just Built : Audit your git diff against YAML coding-standards packs using TypeSafe's Jev model, fro...
Kushal Agarwal

On-demand UI flow capture
I've wanted this for ages and it's finally possible now with Jev by @typesafeai. Imagine Mobbin, but it grabs fresh flo...
omar

Jev Chess Master
Introducing Jev Chess Master, a realtime online chess game leveraging the power of pure System One AI...
Leo Lou

A Jev code-review gate
Jev is your code review that costs nothing and never talks I spent a week trying to get a large model to gate its...
Kunal Jain

Semantic spreadsheet formatting
jev semantically formatting spreadsheet instantly should we put it in @tryshortcutai ?

A faster browser agent
Breaking: Browser Use + Jev = Ultrafast [illegible] Findings: Flights took 7s and cost only $0.0039 [illegible] > new action space every...
Gregor Zunic

Magic Jev Ball
I made a Magic Jev Ball for code reviews [illegible] Click it on any GitHub PR and ask: “should I approve this?” It checks...

Typeahead UI
“Typeahead UI” with Jev by @typesafeai If you watch closely it already knows what to update *before...
mikegee

### AIE secret

Almeida argues that LLMs have narrowed how engineers think about software. Too many ideas stop at swapping out the model. He wants engineers to rethink where intelligence belongs inside the software itself.

[11:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=702s) [A siren and a shushing face flank the title. Three bullets say the field is “jevving” too little, describe dependence on LLMs, and end with “RETHINKING SOFTWARE” in bold.](https://assets.ai.engineer/sites/aiecode2025/sha256/ab7a266e427ebb2e2acfc9651282bf041e2f24f7fe85c1ea05fdb02257f8f20b.webp)

🚨 AIE secret 🤫

• the field isn’t jevving hard enough
• people still have LLM stockholm syndrome
• a lot of ideas are just replacing LLMs instead of RETHINKING SOFTWARE

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder  TypeSafe AI

### Infinitely replicable lego blocks

Almeida draws AI blocks alongside AND, OR and NOT gates. He proposes that software could combine ordinary logic with judgments about meaning, such as whether an email address looks real.

[13:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=790s) [A six-panel diagram pairs AND, OR and NOT logic gates with brain-shaped blocks labeled NOUL, SCORE and CHOICE. Below it, the caption reads “Infinitely replicable lego blocks,” with an arrow pointing to “software.”](https://assets.ai.engineer/sites/aiecode2025/sha256/583a8003bb1d59da3c302c7197f8232566fc2085c06a114b7c162d2ee9991494.webp)

Infinitely replicable lego blocks
→ software

AND: A, B → Y
OR: A, B → Y
NOT: A → Y
NOUL: D, Q → Prob_TRUE, Prob_FALSE
SCORE: D, Q
CHOICE: D, Q → Prob₀, Prob₁, Prob₂, ⋮

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

### Theme: real software engineering

Almeida breaks an AI task into stages an engineer can verify independently. To decide what state each stage needs, he asks what information a person would need to do that part of the job.

[15:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=928s) [Four bullets describe breaking a task down, verifying each stage, managing the state each stage needs and engineering around the task. A retro-style poster shows a frustrated computer user.](https://assets.ai.engineer/sites/aiecode2025/sha256/c2475167c83bc8d2d753ca6ee6871e72678779d167d39800e2646ed7c49c9142.webp)

Theme: real software engineering

• breaking it down (decomposition)
• thinking about stages that can be verified (encapsulation)
• designing what a smart person would need to do that stage (state management)
• engineering around the task!!!!

This is not your vibecoder’s AI.

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

### Agents beyond the KV cache

Almeida’s chart puts rereading cached context at 1.5% of the cost developers expect and 67% of the cost at a 100% cache hit rate. He asks why coding agents have to rediscover the same patterns in a codebase when they could share state.

[15:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=958s) [Bullets ask how the KV cache affects memory scaling, state sharing and subagent routing. Two pie charts compare expected and actual agent costs, with output largest in the first and cache reads largest in the second.](https://assets.ai.engineer/sites/aiecode2025/sha256/7ad7ef4dbaeb97b93805e232ce9dd9c39c102964ccaa5db5ff89dfd359c929f1.webp)

Agents beyond the KV cache

• so much tyranny of agent design comes from the KV cache!
• why is memory O(N)? instead of log(N)?
• why is state not shared b/w agents?
• why can’t non-naive routing/subagents work?

(KV) Cache Rules Everything Around Me
TL;DR: The biggest cost to devs of an agent is re-reading its own context, which costs the provider close to nothing to serve.
DIOGO
SEP 09, 2026

Background
Pop quiz: what’s the most important cost when running an agent?

What devs think agents cost
Input 18.8%
Output 75%
Cache read 1.5%
Cache write 4.7%

What agents actually cost
Input 12%
Output 18%
Cache read 67%
Cache write 3%

Cache write = the extra 25% write premium · Actual costs assume a 100% cache hit rate

TypeSafe AI

AI Engineer
Code Summit
PRESENTED BY
Google DeepMind

I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

Interface text:
59   5   6   Share

### Principal–agent problem

A principal hires an agent to act on their behalf. Each has interests of their own, as citizens and elected officials illustrate. Almeida argues that rules leave room for loopholes when their wording misses their purpose.

[17:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1046s) [A Wikipedia excerpt describes how interests can diverge when an agent acts for a principal. A diagram shows the principal hiring the agent, the agent performing for the principal and each pursuing self-interest.](https://assets.ai.engineer/sites/aiecode2025/sha256/36c744c94ec0d86ca075ccfd8074b943d8de270ec770564b3b07dbba0a0a13f8.webp)

Principal–agent problem
From Wikipedia, the free encyclopedia

The principal–agent problem (often abbreviated agency problem) refers to the conflict in interests and priorities that arises when one person or entity (the "agent") takes actions on behalf of another person or entity (the "principal").[1] The problem worsens when there is a greater discrepancy of interests and information between the principal and agent, as well as when the principal lacks the means to punish the agent.[2] The deviation of the agent's actions from the principal's interest is called "agency cost".[3]

Common examples of this relationship include corporate management (agent) and shareholders (principal), elected officials (agent) and citizens (principal), or brokers (agent) and markets (buyers and sellers, principals).[4] In all these cases, the principal has to be concerned with whether the agent is acting in the best interest of the principal. Principal-agent models typically either examine moral hazard (hidden actions) or adverse selection (hidden information).[5]

Hires
Principal
Self Interest
Agent
Self Interest
Performs
Basic idea of principal–agent problem (agency theory)

Interface text:
24 languages
Article   Talk
Read   Edit   View history
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages) [show]

### Realtime Intelligent Security

A dashed line traces an escape path from a robot’s sandbox through a message board to Hugging Face. Almeida argues that catching the escape as it happens requires security checks fast, cheap, and reliable enough to run in real time.

[18:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1098s) [A drawing titled “The Escape Path” shows a robot in a sandbox, a message board and a building labeled “Hugging Face,” connected by a dashed orange line.](https://assets.ai.engineer/sites/aiecode2025/sha256/72b1a16fc8fb09c0abd8fa85aad4c38cba32de6aac3f6638f4e4c745df2cb514.webp)

Realtime Intelligent Security

THE ESCAPE PATH
SANDBOX
MESSAGE BOARD
HUGGING FACE

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI
AI Engineer
Code Summi

### Generative UI

A shushing face is the slide’s only bullet. Almeida leaves generative UI to the speakers who will cover it later.

[18:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1126s) [The heading “Generative UI” appears on a black slide above a single bullet containing a shushing-face emoji.](https://assets.ai.engineer/sites/aiecode2025/sha256/df95102b47b97e846b2e89f463a67da963f2fb6dabd3b6e740fdc91eb116906f.webp)

Generative UI
• 🤫

TypeSafe AI

Interface text:
AI Engineer
Code Summit
PRESENTED BY
Google DeepMind
I love you so jevving much
Diogo Almeida / Co-founder   TypeSafe AI

## Full transcript

[0:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1s) [music]

[0:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=12s) I'm Diego Almeida and this talk is I

[0:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=16s) love you so jeving much. Um, Glass

[0:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=19s) Animals reference. We're not just weird.

[0:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=21s) Um, like I said, I'm Diego Almeida. I'm

[0:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=24s) known as the Jev guy. Sometimes Jevy

[0:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=27s) Pinkman. um also co-created chat GPT,

[0:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=30s) RLHF and instruct GPT, but I assume that

[0:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=33s) for the crowd today, you all know who I

[0:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=35s) am, at least mostly. [snorts] Um I have

[0:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=39s) like a bunch of AI specific stuff I want

[0:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=41s) to get to, but I like going into my

[0:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=43s) favorite topics in AI. First one is what

[0:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=46s) the [&nbsp;__&nbsp;] is actually going on and where

[0:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=48s) is all the automation. If you've heard

[0:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=50s) me say this before, I'm sorry, but it's

[0:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=52s) such an important thing to beat into the

[0:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=54s) field because people still do not get

[0:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=56s) it. And I need to yell this until

[0:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=58s) everyone really understands what the

[1:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=60s) [&nbsp;__&nbsp;] is going on. And if you're an AI,

[1:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=62s) you should have a really good answer

[1:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=63s) about why AI can do so many awesome

[1:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=66s) things and yet still be so useless for a

[1:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=68s) bunch of other ones. My relationship to

[1:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=71s) chat GPT is somewhat unique. One of the

[1:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=73s) few people who hates on chat GPT who

[1:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=76s) made it. I don't hate chat GPT to be

[1:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=79s) clear. I love using agents. I love using

[1:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=81s) coding agents. I love chat bots. They're

[1:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=83s) just quite limited. And in particular, I

[1:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=85s) think it's a little tragic how much

[1:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=87s) they've bifurcated the field into

[1:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=89s) overpromise, underdeliver. Really,

[1:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=92s) really terrible stuff for the world in

[1:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=93s) my opinion. And I like to talk about

[1:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=95s) like the two cults in AI and how we can

[1:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=98s) resolve the cognitive dissonance that

[1:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=100s) everyone I think should be having unless

[1:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=103s) they have a very good answer. There's

[1:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=105s) cult one, which is, you know, the cult

[1:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=107s) of like EAS. They look at all the

[1:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=109s) benchmarks and they say like not only

[1:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=110s) are LMS surpassing human level on

[1:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=113s) basically every benchmark they have been

[1:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=115s) for two years we've been announcing AGI

[1:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=118s) three months away for two years now even

[2:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=120s) since like the 03 days do people

[2:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=121s) remember that no one even care cares

[2:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=123s) about that model anymore and it was that

[2:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=125s) that was like peak internal to openai

[2:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=127s) AGI hype and on the other hand you look

[2:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=130s) at the nonbenchmarks and you look at the

[2:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=132s) reality and you see the rest of the

[2:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=134s) world especially if you guys go outside

[2:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=136s) and leave San Francisco a lot of the

[2:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=138s) world is like really unchanged despite

[2:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=140s) AI being so [&nbsp;__&nbsp;] cool. And I think AI

[2:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=143s) is transformative. It's it's a rare

[2:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=145s) point of view to actually think AI is

[2:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=146s) cool and transformative and also like

[2:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=149s) underwhelming the world right now. And

[2:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=152s) the tricky part is this is the same

[2:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=153s) field and we are all seeing the same

[2:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=155s) facts. So what gives you know why is AI

[2:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=159s) jagged so to speak like why can it be so

[2:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=161s) good at some things and so not good at

[2:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=163s) other things and what is the aam's razor

[2:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=166s) explanation for that. Um I will be

[2:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=169s) blasting through my slides fast. I'm

[2:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=171s) sorry maybe I could slow down for people

[2:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=172s) taking pictures. Um but uh I'm noticing

[2:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=176s) that uh this is going to go fast is all

[2:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=178s) I'm saying. So I think that we as a

[3:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=180s) field should have an aams razor

[3:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=182s) explanation for why are there some

[3:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=184s) things that are just not just good they

[3:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=187s) are unbelievably good superhuman like

[3:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=189s) millennium price problems in math but on

[3:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=191s) the other hand they're not just bad

[3:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=193s) they're so useless they can't even be

[3:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=195s) used to touch accounting that sounds way

[3:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=197s) easier than million prize problems in

[3:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=198s) math right I could do the latter I

[3:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=200s) definitely could not do the former and I

[3:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=201s) was a mathlete so what gives here what

[3:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=204s) is the reason for this and I normally

[3:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=207s) love this as an interactive thing, but I

[3:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=209s) will instead just tell you my answer.

[3:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=210s) Sorry for the power dynamic. Um,

[3:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=212s) everything on the left side is

[3:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=214s) intrinsically a human in the loop task.

[3:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=216s) Like deep research, I could not do

[3:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=218s) better than deep research. The models

[3:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=220s) are so [&nbsp;__&nbsp;] good at that. But if gun

[3:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=222s) to my head or gun to your head, you had

[3:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=225s) to pick between me and deep research and

[3:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=226s) the answer had to be correct, not well

[3:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=228s) written, you'd probably pick me every

[3:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=230s) time. And that's the distinction between

[3:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=233s) like optimizing for automation which

[3:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=235s) tends to be a lot easier and a lot

[3:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=236s) simpler yet machine AI keeps failing at

[3:59](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=239s) this every year there's like an infamous

[4:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=241s) failure of AI failing to even automate

[4:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=243s) drive-throughs and that should be kind

[4:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=246s) of shocking to people and my claim is

[4:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=249s) actually for these tasks pleasing a

[4:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=251s) human is nothing to do with doing the

[4:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=252s) task. What you want to do is basically

[4:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=254s) be an abstraction away. You want the

[4:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=257s) decision to be made and then you want to

[4:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=258s) get back into doing automation and code

[4:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=261s) is the language of automation.

[4:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=262s) Therefore, you need to integrate with

[4:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=263s) code. This is a lot but I will summarize

[4:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=265s) it again. I get the irony that uh you

[4:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=268s) know I co-created RHF and I do believe

[4:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=270s) RHF is part of the problem and RHF in

[4:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=273s) particular is why all AI today does

[4:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=277s) require a human in the loop with an

[4:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=278s) asterisk unless the task is like very

[4:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=280s) trivially verifiable and can run on its

[4:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=282s) own but still requires a human in the

[4:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=284s) loop to verify it afterwards. It is

[4:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=286s) because we literally put it in. That is

[4:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=288s) what the HF is in RLHF. And my claim is

[4:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=291s) that there's a better way. You know, my

[4:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=294s) summary of the state is 100% of LMS

[4:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=296s) today are optimized for assistance. You

[4:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=298s) might talk about RLVR, but the RLVR

[5:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=300s) models are very bad to interact with if

[5:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=302s) they've not been RHFed. The result is LM

[5:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=305s) become too good to be true at

[5:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=306s) assistance. They are just so [&nbsp;__&nbsp;]

[5:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=308s) good. Like you should never pick me over

[5:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=310s) chat GPT for your chat GPT queries. But

[5:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=313s) for actually doing like you know run in

[5:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=316s) the background cognitive work you should

[5:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=318s) not trust LMS at all because they've not

[5:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=320s) been optimized for that which is why

[5:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=321s) they're so bad at the tasks of

[5:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=323s) automation and I would also guess that

[5:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=325s) automation is a lot more valuable than

[5:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=326s) assistants because you can do way cooler

[5:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=329s) deeper I'm going to get into this later

[5:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=330s) but I think there's going to be so much

[5:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=331s) stuff that we can do and we are just

[5:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=333s) scratching the surface of that right now

[5:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=335s) and that's the tragic thing about AI uh

[5:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=337s) the state of AI for me

[5:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=339s) let's talk about the evolution Um, I I

[5:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=343s) can't tell if this is appropriate, but

[5:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=344s) it's too late. Um, oh god, don't take

[5:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=348s) pictures of this one. Um, cool. Oh, I'm

[5:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=351s) getting a thumbs up. Yeah. Uh,

[5:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=353s) excellent. So, we're we instead of doing

[5:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=355s) RHF, we just have to do something else.

[5:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=357s) We need to think from first principles.

[5:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=358s) How do we optimize for machine native

[6:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=360s) AI? You guys hopefully know the

[6:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=361s) interface of Jev and that is what we are

[6:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=363s) doing. more important than optimizing

[6:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=365s) for its interface, we are uh rethinking

[6:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=368s) how AI should be done because this is

[6:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=370s) what we did again for Arlf for Sutton's

[6:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=374s) bitter lesson. I'm going to loosely

[6:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=376s) paraphrase to make my point easier to

[6:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=377s) say is that scaling um and compute

[6:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=381s) matters more than algorithms. This is

[6:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=383s) why people become like scaling pill

[6:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=384s) jironis and I would argue that there's

[6:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=387s) an even bitter lesson than that which is

[6:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=389s) data always matters more than compute.

[6:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=391s) We are learning that quite a bit in the

[6:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=393s) field now. But it has been obvious since

[6:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=395s) the beginning of time. I'll tell you

[6:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=396s) when the this is the one you should be

[6:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=398s) photographing. I would argue that the

[6:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=400s) the bitterest lesson is doing the right

[6:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=403s) task. And doing the right task is like a

[6:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=405s) weird thing to think about especially

[6:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=406s) for ML people because they like

[6:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=408s) algorithms. They want number goes down

[6:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=411s) or up or whichever is good direction.

[6:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=412s) They don't want to think about the real

[6:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=413s) world. But this is always how you make

[6:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=416s) gigantic gains in performance. This is

[6:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=417s) how the big breakthroughs happen. 100%

[7:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=420s) of the time it comes from this and this

[7:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=422s) is true for classical RL games. This is

[7:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=424s) why Sutton had his bitter lesson it

[7:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=426s) which is data is derived by compute when

[7:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=429s) you're doing rollouts like there's

[7:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=430s) there's why would you need off policy

[7:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=432s) data when playing chess or something

[7:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=433s) like that and doing the right task is

[7:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=436s) defined by the rules. So actually when

[7:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=437s) you're doing RL you don't even need to

[7:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=439s) think about the right task for

[7:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=441s) pre-training we actually as a field had

[7:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=443s) to do a lot of thinking about the right

[7:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=445s) task. I don't know if people remember

[7:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=446s) the birth days with um masked language

[7:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=449s) modeling, but it took a while and it was

[7:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=451s) very unclear to figure out that next pro

[7:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=453s) token prediction was super [&nbsp;__&nbsp;] cool.

[7:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=456s) Not because it was better at masked

[7:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=457s) language modeling. You know, the

[7:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=459s) objectives on their own aren't special.

[7:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=461s) It's about what you can do with them

[7:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=462s) afterwards. And it turns out next token

[7:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=464s) prediction is incredible at generating

[7:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=466s) strings, which is what these pre-trained

[7:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=467s) models were so good at. So no level of

[7:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=470s) mass language modeling even if you had

[7:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=471s) today's compute and algorithms would

[7:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=473s) make it as interesting as GPT3 was

[7:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=475s) because it was just doing a different

[7:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=476s) task and data you know the internet is

[8:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=480s) obviously a giant bottleneck now we're

[8:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=481s) like you know burning books or whatever

[8:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=482s) in order to get more data but data

[8:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=484s) obviously matters a lot more than

[8:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=486s) compute and people keep like

[8:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=487s) rediscovering this and figuring this

[8:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=488s) out. It's extra true for making LLMs

[8:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=491s) useful. um before RHF even existed the

[8:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=495s) data for RHF didn't exist and you could

[8:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=498s) like it was night and day I'll just show

[8:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=500s) you the next slide this is the probably

[8:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=501s) the relevant part this is what this was

[8:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=502s) the work in the RHF paper and the high

[8:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=505s) level was that even with the dumbest

[8:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=507s) algorithms in the world even with a GPT2

[8:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=510s) sized model you got something like GPT9

[8:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=513s) level performance like real GPT9 not

[8:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=515s) today's like fake GPT numbers like the

[8:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=518s) numbers used to mean like a 100x scaling

[8:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=520s) and effective compute for pre-training

[8:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=521s) and they've given up on that because it

[8:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=523s) just can't scale anymore. And still even

[8:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=525s) with that it would take something like

[8:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=526s) GPT7 to9 to match GPT to the model from

[8:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=530s) like 2018 or whatever it was at the task

[8:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=533s) of instruction following. That is kind

[8:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=535s) of wild because you think big LMS are so

[8:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=538s) important for this and they are but that

[9:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=540s) is not the most important thing compared

[9:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=542s) to even the gigantic jump of doing the

[9:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=545s) right task. And spoilers this is what we

[9:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=547s) did for Jeff, right? like we made this

[9:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=548s) gigantic leap in the field that no one

[9:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=550s) realized could be coming because we were

[9:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=552s) optimizing for a task and very few

[9:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=554s) people think about the right task. Um

[9:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=557s) this is why Jev has the Jebniqua. Um

[9:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=561s) that's a really smart bun I think. Um

[9:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=566s) little bit of a pity laugh, but thank

[9:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=568s) you. Um you get what you optimize for. I

[9:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=571s) I'm looking for a better word than

[9:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=572s) reliable. So if you guys have

[9:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=573s) suggestions, let me know. But there's

[9:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=575s) there's this reliability of intelligence

[9:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=577s) that's really missing in LLMs. I often

[9:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=579s) call it you get Einstein 95% of the

[9:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=582s) time, but Mr. Bean 5% of the time. And

[9:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=585s) if you can't sell which is which, how

[9:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=586s) the [&nbsp;__&nbsp;] are you going to automate

[9:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=588s) anything? And instead instead of like

[9:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=590s) focusing on like sometimes occasional

[9:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=592s) smarts that you can then verify, what if

[9:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=593s) you focused on extreme extreme narrow

[9:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=596s) focused reliability so you can get that

[9:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=598s) smarts every time? And this doesn't show

[10:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=600s) up on benchmarks. It is the reason why

[10:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=602s) people love Jev. And this is also I

[10:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=604s) realized in hindsight we obviously

[10:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=605s) didn't prepare for a chat GPT moment for

[10:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=607s) developers but it happened because

[10:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=609s) people in the AI space just overpromise

[10:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=612s) underdel so [&nbsp;__&nbsp;] much that having a

[10:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=615s) model that actually don't do that was a

[10:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=617s) surprise and it is not the speed cost or

[10:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=619s) even interface. This is something that

[10:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=622s) you know my team tells me I shouldn't be

[10:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=623s) telling people so that we can have a

[10:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=625s) long-term advantage and I just care too

[10:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=626s) much like it's reliable intelligence.

[10:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=628s) make sick models, please. Even if we,

[10:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=630s) you know, if it doesn't involve our

[10:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=632s) models. Um, we see that Jev as like a

[10:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=635s) foundation for smart software. I'm going

[10:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=637s) to skip all over this, but I think

[10:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=638s) software is going to be so [&nbsp;__&nbsp;] back.

[10:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=640s) I think the narrative of SAS apocalypse

[10:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=642s) is obviously not going to happen, but I

[10:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=645s) think that even calling it a Sasa Palooa

[10:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=648s) is like an understatement, and I will

[10:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=649s) get into that a bit, and you guys

[10:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=651s) ideally are the perfect crowd for this.

[10:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=652s) I'm so excited for this part because

[10:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=653s) that's the part that's new to me. Um, I

[10:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=656s) want to make the distinction between

[10:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=658s) smart software and just in time

[11:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=660s) software. The the crazy part is AI is so

[11:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=663s) transformative and yet somehow despite

[11:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=666s) AI being so awesome, software has

[11:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=668s) essentially unchanged in the last decade

[11:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=670s) except being cheaper to write. Isn't

[11:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=673s) that weird? Isn't the it weird that the

[11:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=675s) best we can do in software is having a

[11:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=677s) chatbot to the side that sometimes

[11:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=679s) interacts with a software but cannot

[11:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=680s) take expensive decisions for you. That

[11:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=682s) is kind of like a a weird thing if AI

[11:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=684s) was really that transformative. And I

[11:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=686s) think the problem is we're focusing too

[11:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=688s) much on coding agents and not enough

[11:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=690s) about putting the intelligence into the

[11:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=692s) inner loop, which I'm going to get more

[11:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=693s) into a bit. And people are jeving.

[11:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=695s) There's been tons of successes. We don't

[11:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=697s) need to flex this. There's so many cool

[11:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=699s) things. But my AIE secret is going to be

[11:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=703s) that the field isn't jebbing hard

[11:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=705s) enough. And this is what Swix said that

[11:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=706s) I can make a call to action for you guys

[11:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=708s) that's unique. And hopefully I have

[11:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=709s) enough time for this. I think the world

[11:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=711s) still has LLM Stockholm syndrome. We are

[11:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=713s) still thinking from a really LLM pill

[11:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=715s) place which is I think tragic you know

[11:59](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=719s) like I I I love the software saying the

[12:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=721s) tools you use change the way you think

[12:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=723s) and I think that it's changed the way we

[12:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=724s) think way too much and we're not really

[12:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=727s) rethinking software like I think the

[12:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=729s) field needs. I think that we could just

[12:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=730s) do be doing so much more. Hopefully I

[12:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=732s) can try to do my best for my remaining

[12:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=734s) time to convince you guys of that. Um so

[12:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=737s) let's Jev to the future. Um

[12:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=740s) I an analogy I love uh bear with me is

[12:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=743s) about LEDs. Before there was the blue

[12:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=746s) LED, there was red and green LEDs. These

[12:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=748s) were really limited in capability and

[12:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=750s) they were used for like simple

[12:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=751s) indicators and you know like clocks and

[12:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=753s) stuff like that. And when the blue LED

[12:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=756s) was made discovered, I actually don't

[12:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=758s) really know how things outside of

[12:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=759s) computer science work. Um, but when it

[12:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=762s) existed, I guess, um, it enabled not

[12:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=765s) just like 50% more use cases, which what

[12:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=767s) you'd expect about like changing colors

[12:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=769s) from two to three, it enabled so many

[12:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=771s) more. Like now LEDs could be used for

[12:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=773s) white light. It could be used for like

[12:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=774s) the displays that we all use right now.

[12:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=776s) And it created a revolution in lighting.

[12:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=778s) And it's it's it's unfathomable how much

[13:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=781s) bigger the capabilities of LEDs were

[13:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=784s) just ex, you know, adding a new

[13:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=786s) primitive that composes together with a

[13:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=788s) old primitive. So you can see where I'm

[13:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=789s) going here maybe. And I think it's a

[13:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=792s) similar story for software. Software is

[13:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=794s) so [&nbsp;__&nbsp;] incredible. Ignore AI for the

[13:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=796s) last, you know, half decade. But

[13:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=798s) software has done so many insanely cool

[13:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=801s) things. We've done a lot of stuff, you

[13:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=803s) know, like do I love using SAS? Not

[13:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=806s) really. But does it solve problems? Abso

[13:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=809s) [&nbsp;__&nbsp;] lutely. And it's incredible how

[13:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=811s) much we've done with three

[13:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=812s) [clears throat] fundamental logic gates.

[13:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=813s) You know like we are doing this with

[13:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=815s) like simple binary operations that can

[13:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=817s) do things like greater than less than

[13:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=819s) equal. It's kind of nuts how much

[13:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=821s) software has been built from such simple

[13:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=823s) primitives. And my claim is that what we

[13:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=827s) are doing with AI now when we're making

[13:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=829s) our intelligence machine native this is

[13:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=831s) kind of conceptually adding a fourth

[13:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=833s) semantic logic aid. What if we can

[13:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=836s) program with common sense with intention

[13:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=838s) with human understanding? Like do we

[14:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=840s) still need reexes with like crazy things

[14:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=842s) or can we just say like we want a Gmail

[14:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=844s) email address or a real looking one?

[14:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=846s) There's so much that we can do that

[14:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=847s) we're going to need to push the you know

[14:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=849s) push the frontiers of cheap, smart, fast

[14:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=852s) in order to get done. But I think that

[14:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=854s) it's going to be like so many things

[14:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=855s) here and I think just like the blue LED

[14:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=857s) it's going to be way bigger than just

[14:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=859s) increasing the representational ability

[14:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=861s) of software by 33% because it's adding

[14:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=864s) effectively another kind of sort of

[14:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=866s) logic gate. And this sci-fi is the stuff

[14:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=868s) that I think will be impossible to

[14:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=870s) predict just like who could predict how

[14:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=872s) powerful the internet would have been

[14:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=874s) because it requires it's like much more

[14:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=875s) grassroots and decentralized and it

[14:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=877s) requires people to be empowered to do

[14:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=879s) insane stuff. Um I call this software

[14:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=882s) research. Um that's not a joke. Um but

[14:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=886s) uh I I think that there's going to be

[14:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=888s) like an explosion of what software can

[14:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=891s) do and we will want people to be

[14:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=893s) exploring those boundaries of software.

[14:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=895s) Not necessarily because it's valuable,

[14:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=897s) but because they can because we have

[14:59](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=899s) like new frontiers to explore and this

[15:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=901s) is what the early internet was like.

[15:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=903s) Should all of those websites existed?

[15:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=905s) Probably not. But like it was very cool

[15:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=908s) and we found a lot of awesome stuff from

[15:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=909s) that. And I think we we, you know, we

[15:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=911s) have another opportunity for that. So we

[15:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=913s) should not just keep living in this

[15:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=915s) little, you know, fish tank that we are

[15:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=917s) the victims of open anthropic on

[15:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=919s) whatever they ship and we just have to

[15:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=920s) be like LM rappers. So I'm going to

[15:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=923s) pitch software research as a direction

[15:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=924s) here. The theme is going to be real

[15:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=927s) software engineering, not just vibe

[15:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=929s) coding. There's it's going to be like

[15:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=931s) the kind of stuff that everyone has

[15:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=932s) known if you've been a software

[15:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=933s) engineer. Decomposition, breaking tasks

[15:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=936s) down. We need to think about stages,

[15:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=938s) ideally ones that can be verified so we

[15:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=939s) have real encapsulation. And we need to

[15:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=942s) actually do state management instead of

[15:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=944s) assuming everything is global state

[15:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=945s) within a KV cache. And the theme is just

[15:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=947s) going to be like real [&nbsp;__&nbsp;]

[15:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=949s) engineering by people who care a lot

[15:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=951s) about the end result. It doesn't have to

[15:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=953s) be valuable. Ideally, it's sick as [&nbsp;__&nbsp;]

[15:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=955s) Um, and you know, we should be pushing

[15:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=957s) the limits.

[15:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=958s) I'm going to go into a couple of like my

[16:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=960s) favorite ideas and I want to figure out

[16:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=962s) how to encourage these in the world. So,

[16:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=964s) this is my first step at this. I'm just

[16:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=966s) going to talk about it. We have raised a

[16:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=968s) bunch of money. So, I would like to

[16:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=969s) figure out how to fund this, but we

[16:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=971s) don't have the operations help to do

[16:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=972s) that. But here are just some ideas out

[16:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=974s) there. Um, I think agents are they're

[16:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=978s) very cool, but like they're they're so

[16:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=981s) inflexible in their design because of

[16:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=983s) the tyranny of the KV cache. I don't

[16:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=984s) have the time for this if this timer is

[16:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=986s) to be believed. But believe me that

[16:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=988s) there's so much that people have tried

[16:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=990s) to do but failed at because it ends up

[16:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=991s) becoming inefficient because it fights

[16:33](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=993s) against the KV cache. But I think

[16:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=995s) there's a lot more to do. Why is memory

[16:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=997s) o, right? Like why can't we have all of

[16:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1000s) our history of coding agents always

[16:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1002s) included in every new call? That is how

[16:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1005s) continual learning works, right? And we

[16:47](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1007s) have the memory available. It's just

[16:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1008s) that we don't have the smarts to map

[16:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1010s) produce over the memory to find out like

[16:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1012s) what you should what you should

[16:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1014s) remember, what you shouldn't. Um, I

[16:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1016s) think there's many ways to do login or

[16:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1018s) even better lookups. Why when you're

[17:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1020s) making many agents at once, why can't

[17:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1021s) they not share state? Why do they need

[17:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1023s) to rediscover the same patterns of the

[17:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1025s) codebase over and over and over again?

[17:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1027s) And why can't non-naive routing and sub

[17:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1029s) aents work? Like even naive routing

[17:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1030s) doesn't work, but like why can't we do

[17:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1032s) these things? And why does like the you

[17:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1034s) know the the big labs keep going back

[17:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1035s) and forth on whether or not sub aents

[17:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1037s) work. Um I think that like from an

[17:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1039s) engineering perspective there's elegance

[17:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1041s) to it but they fight against the KV

[17:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1043s) cache and I think that if we design

[17:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1044s) agents around that we could do so much

[17:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1046s) more. This one's the the probably the

[17:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1049s) most vague of my directions but I feel

[17:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1051s) very strongly about it. Um the world is

[17:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1054s) full of principal agent problems. This

[17:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1056s) is when you have like you know ulterior

[17:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1057s) motives between this is not a coding

[17:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1059s) agent but like any agent. The the

[17:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1061s) classic example is citizens versus

[17:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1063s) elected officials. Now, I don't want to

[17:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1064s) make a too political claim here, but

[17:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1066s) like there's misalignment in all levels

[17:49](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1069s) of society. And this results in like

[17:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1071s) waste, fraud, bad things, and it it

[17:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1074s) results in for a society that wants

[17:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1076s) fairness. You need to overemphasize the

[17:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1078s) letter over the spirit, which then

[18:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1080s) results in loopholes and play uh

[18:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1082s) payto-in dynamics in the world. And I

[18:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1084s) think the root of that problem is we

[18:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1086s) can't encode the spirit. Just like with

[18:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1088s) software, we're constrained to encoding

[18:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1090s) bits instead of what we actually meant

[18:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1092s) in that situation. And I think so many

[18:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1094s) organizations, society could be

[18:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1096s) transformed by really encoding the

[18:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1097s) spirit all over the place. Obviously,

[18:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1100s) real-time intelligence security, can we

[18:23](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1103s) prevent the OpenAI hugging face sandbox,

[18:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1105s) blah blah blah. Um, why did why couldn't

[18:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1107s) they prevent it? It is not because they

[18:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1109s) didn't have the intelligence. I would

[18:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1111s) argue it's a lot easier to figure out

[18:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1112s) they're doing a naughty thing than to

[18:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1114s) actually do the naughty thing. It is

[18:35](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1115s) because it was not fast, cheap, and

[18:37](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1117s) reliable enough to actually run real

[18:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1120s) time. And what happens when every

[18:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1122s) corporation has their own real-time

[18:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1124s) security that's actually smart? How cool

[18:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1126s) would that [&nbsp;__&nbsp;] be? I can't talk

[18:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1128s) about generative UIs because people are

[18:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1130s) going to be talking about it afterwards.

[18:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1132s) I don't want to steal their thunder, but

[18:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1133s) it's cool as [&nbsp;__&nbsp;] Trust me. Um,

[18:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1136s) like semantic dependent typing. Why

[18:59](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1139s) can't we have like a type system that

[19:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1142s) actually is semantic, right? like why

[19:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1144s) can't we like declare what local

[19:07](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1147s) properties we want so we don't have to

[19:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1149s) read a chain of dependencies deep down

[19:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1151s) these all should be possible and imagine

[19:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1153s) if you could like run a jevlike model in

[19:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1156s) your lint checker to check if you're

[19:17](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1157s) it's all type safe and it could you know

[19:20](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1160s) uphold semantic properties it would be

[19:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1162s) so [&nbsp;__&nbsp;] cool there's there was a

[19:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1165s) great Twitter video on um like a voice

[19:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1168s) interface in general I think new

[19:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1170s) interfaces are so exciting this is to me

[19:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1172s) the return of like this going to show by

[19:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1174s) agent software like do what I mean. This

[19:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1176s) used to be like a popular thing but it

[19:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1178s) never really worked because you it was

[19:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1180s) it was too many hard-coded rules to

[19:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1181s) really figure out what I mean. Um

[19:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1184s) there's stopping slop obviously a big

[19:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1186s) theme of here but like we definitely

[19:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1188s) believe this. This is my props. I have

[19:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1190s) to we have a stop geni baseball bat for

[19:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1193s) type safe. It is very very awesome. I

[19:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1196s) want to talk about the root of slop. Um

[19:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1198s) the the root of slop is that AI is

[20:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1200s) unreliable. Therefore, you as a company

[20:02](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1202s) or you as you know like someone who's

[20:04](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1204s) doing stuff, it can't be used for

[20:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1206s) decisions that costs you stuff. But it's

[20:08](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1208s) actually totally fine to push the costs

[20:10](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1210s) into other people, which is horrible.

[20:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1211s) You know, externalized costs and then

[20:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1213s) this is what ends up becoming slop. You

[20:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1215s) know, if people actually read the stuff

[20:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1216s) and filter it and like care about the

[20:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1218s) quality, it becomes not slop. But now

[20:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1221s) the world is full of this because

[20:22](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1222s) there's an asymmetric advantage for

[20:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1224s) people who are like bad actors and don't

[20:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1226s) give a [&nbsp;__&nbsp;] So, I think that we can

[20:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1228s) save the world from at least inbound

[20:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1229s) hell among many other things and that

[20:31](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1231s) would be very cool. Um, and even

[20:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1234s) something I think is super sick is

[20:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1236s) cursed software. Like why if you guys

[20:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1239s) ever seen the subreddit bad UI battles,

[20:41](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1241s) it is so [&nbsp;__&nbsp;] cool. Even though they

[20:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1243s) make the horrible interfaces like this

[20:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1245s) whole game of like would you like to

[20:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1246s) delete your account becomes you know a

[20:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1248s) random thing. And and I actually think

[20:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1251s) doing things because we can is what

[20:53](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1253s) software engineering is about because

[20:55](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1255s) who knows maybe 1% of these ideas

[20:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1257s) actually lead to something super

[20:59](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1259s) awesome. And I think like really

[21:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1261s) cultivating even bad ideas is kind of

[21:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1263s) what being like this you know playful

[21:05](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1265s) hacker is all about and generally just

[21:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1269s) automating everything. This is what the

[21:11](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1271s) followup this is this is a slop image

[21:13](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1273s) but uh sorry logo is a little bit wrong

[21:15](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1275s) but don't stare at that. That's a little

[21:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1276s) weird. Um, this is what a lot of the

[21:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1279s) follow-up talks are going to be about.

[21:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1281s) Thank you. Um, and if you're a software

[21:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1284s) engineer, you almost certainly think in

[21:26](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1286s) the world of bits. If you know anything

[21:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1288s) about software, uh, and you've

[21:29](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1289s) interacted with software, you know that

[21:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1292s) there are things software can do and

[21:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1294s) software cannot do because that is just

[21:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1296s) only obvious, right? Like there's some

[21:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1298s) things that you know you need to do

[21:40](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1300s) because that is not compatible with

[21:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1302s) software. But what if those constraints

[21:44](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1304s) were no longer true? Maybe not today,

[21:46](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1306s) but maybe in like six months or a year.

[21:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1308s) What couldn't we automate? And that's

[21:50](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1310s) kind of where I want to leave this whole

[21:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1311s) thing because I think there is very

[21:54](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1314s) likely to be a future where it's not a

[21:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1316s) question of whether or not we can

[21:57](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1317s) automate something. It's going to be a

[21:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1318s) question of like how much work is it,

[22:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1320s) which is how it is right now for the

[22:01](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1321s) stuff that can be automated. And that

[22:03](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1323s) would be cool as [&nbsp;__&nbsp;] So, hell yeah.

[22:06](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1326s) Thank you. Um,

[22:09](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1329s) I'm super overtime, but thank you to the

[22:12](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1332s) early believers. You know, we had no

[22:14](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1334s) idea so many people would resonate with

[22:16](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1336s) this. We thought we were the insane

[22:18](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1338s) ones, but it turns out like we were

[22:19](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1339s) actually the majority of the world who

[22:21](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1341s) realized AI could be so much more. Thank

[22:24](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1344s) you to like the people who are like

[22:25](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1345s) really realistic about reliability.

[22:27](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1347s) Everyone knows I hate benchmarks and a

[22:28](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1348s) lot of other people are really screaming

[22:30](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1350s) out like benchmarks [&nbsp;__&nbsp;] suck.

[22:32](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1352s) They're actually they hurt the people

[22:34](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1354s) who do not have time for anything other

[22:36](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1356s) than benchmarks most. It's tragic. And

[22:38](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1358s) for the people who are pushing the

[22:39](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1359s) limits of software, like [&nbsp;__&nbsp;] yes. and

[22:42](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1362s) we want to be there with you helping you

[22:43](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1363s) all do that. The the dreamers and the

[22:45](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1365s) people who don't want to be limited by

[22:48](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1368s) everything that has happened so far, we

[22:51](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1371s) want to really fight for you and we want

[22:52](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1372s) this to be as reliable as a fourth logic

[22:56](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1376s) age and that is a very high bar and we

[22:58](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1378s) have like a lot of work to do. So, thank

[23:00](https://www.youtube.com/watch?v=Cyt3LU8TyNc&t=1380s) you. Thank you.
