# Agents Made Code Free. Maintaining It Didn't Get Cheaper. - Dmitry Buykin, Maersk Dmitry Buykin, Maersk [Talk page](https://ai.engineer/talks/oeknLUsN_GE-agents-made-code-free-maintaining-it-didnt) · [Video](https://www.youtube.com/watch?v=oeknLUsN_GE) ## Agents Made Code Free. Maintaining It Didn't Get Cheaper. At Maersk, one agent writes the instructions another follows in production. A request to change the amount on an approved order shows what it takes to keep the team lead in the approval path as both code and instructions change. ### The rewrite lost useful pieces. The sprawl grew back. [0:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=0s) Dmitry Buykin’s first attempt to get an AI-coded project under control earned the Thanos analogy. He deleted tests and unused boilerplate and rewrote the whole project. Some useful pieces went with it. Then the agents kept writing, and the mess grew back. The practical limit was how much change people could check. Once the project crossed that line, even a cleanup required someone to work out, in one pass, everything that deserved to survive. Buykin’s next attempt spread that judgment over time. Small weekly sweeps would handle deletions, and people using the system would have a way to correct it while they could still inspect the task that went wrong. ### Who decides what should happen to an approved order? [1:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=94s) Someone wants to change the amount on an order that already has approval. A coding agent helped write the instructions for handling the request. A second agent follows them in production. The system runs in 110 countries with about 10,000 scenarios described in prose. The unit of change is a *slice*, one process in one country. By default, a correction to the order process stays in the slice where the problem occurred. Buykin estimated that prose made up about three quarters of the system. To find out how well the tests protected it, the team borrowed a technique from mutation testing. They deliberately broke a sentence and watched for a failing test. Almost none turned red. A coding agent could rewrite those instructions easily, but the passing tests gave little guidance about which meaning the rewrite had to preserve. Even when the wrapper around a model consists entirely of code, its agent instructions and tests still express what people intend the system to do. Consider an instruction to check with the team. A business expert knows what decision the team makes. A developer can inspect traces and logs with a coding agent to work out how much of that step the connected systems can perform. For the order change, the team lead kept the decision. The developers could connect the data for the limit, and they had their coding agents record those connections so the runtime could check them. One document serves both readers. Agents read precisely specified blocks. Experts read the surrounding explanation, including a plain description of what the code does, and sign off. Keeping one copy avoids having to reconcile two accounts of the same rule. When a task goes wrong, an expert reviews it with the information the production agent saw. The expert’s note gives the coding agent a change to make, and the original task gives it a case to run again. A group of experts participates from the start because no one person knows all the rules. ### When a repeated path through prose becomes a tool [4:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=296s) The team records whether each production outcome was accepted or rejected and uses that history to direct expert attention. How could the same order be acceptable in one place and too large in another? The prose set the limit at 1,500. The code applied a different limit. Each could apply its own rule successfully, so the disagreement raised no error. Wording can change behavior in subtler ways, too. An instruction forbidding a particular tool call tells the model that the tool exists. The team built a linter for prose and had authors lint each changed file and record the result in a ledger. Two copies of a rule could still drift apart, so the team also watched what Buykin calls a *walk*, the path an agent actually takes through the instructions. When agents kept taking the same path, the team packaged its steps into one workflow tool. The first tools made things worse. The prose still named the steps inside the tool, so agents ran those steps again after the workflow had performed them. Removing the step-by-step prose and leaving the decisions fixed the duplication. The team now moves a walk into code when it repeats *and* agrees across countries. A path that still varies stays in prose. Who makes the decision inside a workflow? Code prepares the facts. Experts own a decision table describing the allowed choices. The model picks a matching row, and code carries out the decision. If no row fits, a person takes over. That handoff gives the team a way to handle a new case without asking the model to invent a rule. Connecting many small model decisions through workflows was a proposed next step. Some rules moved the other way. The team took hundreds of lines of expressions out of code and put them into prose because the experts responsible for the decisions couldn’t read their implementation. A regular expression could still spot a critical signal, and a growing collection of such patterns could live in configuration. Where the answer varied, the person who understood the rule needed to be able to read and correct it. How does an expert’s rule reach an order decision? Two inputs meet at the model. Code supplies facts about this task and experts supply the possible decisions. A missing match changes who handles the task. - Task facts — Code prepares the data needed for this decision. - Decision table — Experts own the rows that describe the choices. - Model looks for a row — Match the task facts to an available decision. - Code acts — Carry out the selected decision. - A person takes over — Handle a case the table does not cover. - Task facts → Model looks for a row (Prepared facts) - Decision table → Model looks for a row (Available rows) - Model looks for a row → Code acts (Matching row selected) - Model looks for a row → A person takes over (No row fits) ### What the model sees and who can write to the order [7:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=448s) Given a choice of what to load, the model loaded everything. It saw prices and invented a pricing check. Nobody had asked for one. The team moved context selection into the runtime. For the order change, the runtime supplied three facts from the order and kept the full record itself. With fewer irrelevant facts to act on, the model made fewer wrong moves. For a larger set of data, the model passes a *handle*, a short name for data the runtime already holds. The model can refer to the set without carrying the whole set in its context. The runtime checks the context for consistency and validates the handle before a write. Giving the model enough information to choose a row and allowing it to change an order are separate engineering decisions. What happens if a write succeeds but looks like a failure to the agent? The team checks changes during development, and only code can open the write path at runtime. Buykin counts five checks before the write and a sixth to establish whether it landed. An apparent failure can prompt the agent to retry. A journal ensures the action takes effect only once. Keeping that protection in code lets the team revise the prose many times a day. ### Finding the changes that need a person [9:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=564s) Buykin initially read every change. That was how the system picked up his habits, and how he became the person every change waited on. Eventually the volume outgrew what he could read. He was burning out. His earliest warning came when he was tired and couldn’t explain what had changed. The dashboard could stay green through all of that. He took the loss of his ability to explain a change as reason to stop and fix the review process. One way to direct scarce attention was to estimate the reach of a change before making it. Before building a proposed check, the team replayed its rule over several thousand past sessions. It would have fired on less than 1% of them. That figure measured how much past traffic the check would have touched. Judging whether it was right for the affected sessions still took case review. Buykin watched for jumps in the numbers to tell him where to read. The team waits for an isolated signal to recur, even when an expert raised it. A cluster across slices warrants investigating the rule they share. Otherwise a coding agent can keep adding conditions until today’s examples pass. Removing those conditions can lower the eval score. The failures that reappear give the team a list of cases its rules still need to handle. An expert’s note about a failing task becomes an eval, a case that checks for that failure. An agent then searches sibling slices for the same behavior. The search can be broad because it gathers evidence. The edit stays narrow because markets and local laws differ on purpose. Only siblings where the failure is confirmed receive the fix. ### Getting feedback into the next agent session [11:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=713s) With 10 to 20 feedback loops open at a time, Buykin had become the message bus between users and agents. Giving each user a screen to evaluate a task removed a handoff. Each loop writes to a ledger, so an investigation can continue without waiting for the architect to relay a comment or remember an unfinished case. Agent sessions end, too. The team keeps two kinds of memory in files. - **What happened.** The history preserves events such as an issue being reported and investigated. - **What is true now.** The current account gives the next session its starting point, including what the investigation established. Each issue needs evidence connecting it to its root cause. Without that link, the next agent can act on a mistaken explanation, and a check has nothing concrete to count when it looks for the problem again. ### Replaying the order after a refactor [12:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=774s) Checking took almost nine tenths of the agent’s time on a change and a growing share of Buykin’s own week. The team ranked replay on real tasks as its strongest evidence. Before seeing the results, they wrote down what would prove their expectation wrong. They also made each new check fail once on purpose, to establish that it could detect the failure it was meant to catch. What became of the team lead’s approval after a refactor? Every condition and every decision-table row represents a branch. The team wanted to account for each old branch exactly once in the new implementation. The branch for amounts over the limit had disappeared. The order would have gone through without the team lead. To compare behavior, the team freezes a set of real tasks and runs the old and new versions against the same inputs. For an over-limit order, the outcome to compare is whether the request still reaches the team lead. If none of the frozen tasks takes a particular branch, the team records that branch as having no evidence. Agreement on the rest of the set cannot establish what happens there. A second agent reviews the change with a checklist. It works without the authoring agent’s reasoning, so the author’s explanation cannot steer it toward agreement. Over six months this independent review grew to cover most of Buykin’s checking. Batch completion still needed its own accounting. Agents sometimes finished most of a batch and quietly skipped the remainder, especially when their context was full. In one backlog, most items looked covered. The ledger showed that only about a quarter had enough evidence to close. ### Retiring a variant means retiring its test [15:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=905s) What became of the variants of one process over a year? Some branches merged back together. Others continued to diverge, and some dead branches still looked alive. The history of a split or a merger could explain how a variant got there. Deciding whether to keep it took evidence from actual tasks. The team introduced gauges for regrowth. A ratchet puts a ceiling on a tracked count. Once the team lowers the count, it cannot rise again. Two gauges were running at the time of the talk, with six more planned. Each week the team identifies behavior that no real task reaches and records the name and evidence for each candidate before removing it. That record is what Buykin lacked during the original rewrite. A test could keep a dead variant alive. As long as a test mentioned an option, coding agents treated it as behavior they had to preserve. Now the test goes with the variant when the team retires it. With these small weekly removals, Buykin had yet to need another wholesale rewrite. ### Whose judgment does the repository remember? [16:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1015s) After the system went into production, Buykin recognized his own engineering principles in its design. Its skills and memory carried his biases along with his habits. He put a personal twist on Conway’s observation about systems reflecting the organizations that build them. This system reflected whoever corrected it. Involving other experts changed whose judgment the next session would inherit. Which question would your repository struggle to answer? Buykin closes with three for each of his six Thanos stones. *Time* asks how much change people can check. *Space* asks where a change may spread. *Reality* asks what evidence establishes that a rule or change is correct. *Mind* asks what the model sees. *Power* asks what can actually change the system. *Soul* asks who supplies judgment and what it costs them. [18 repository-audit questions](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1039s) Buykin was already burning out when he learned to work this way. His advice to the person a team’s changes are waiting on is to pick one question the repository can’t answer yet and start there, while you can still read every change. ## Key takeaways - When a repeated procedure becomes a tool, remove its internal steps from the instructions. Leaving both in place caused agents to perform the work twice. - Use an expert’s correction to search related country slices. Apply the fix only where the same failure is confirmed, since local differences may be intentional. - Compare old and new behavior on a frozen set of real tasks. Record which branches the tasks never exercise. A passing replay says little about an approval path that no order in the set took. ## Slides ### The Thanos Dilemma An orderly grid dissolves into fragments at its upper right. Dmitry Buykin uses it to illustrate a maintenance problem: coding agents make it cheap to add code and even easier for disorder to accumulate. [0:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=0s) [The subtitle reads “Maintaining large AI-coded codebases.” Beside a grid dissolving into fragments, the slide reads “Agents made code cheap. They made entropy cheaper.”](https://assets.ai.engineer/sites/aiecode2025/sha256/9ee441b2476f3999a6cf9f6935ff3c593db007a4c165cbc997df2c3aa3f3463e.webp) The Thanos Dilemma Maintaining large AI-coded codebases Agents made code cheap. They made entropy cheaper. Dmitry Buykin · Maersk AIE CODE SUMMIT · SAN FRANCISCO · NOVEMBER 2026 AI Engineer Code Summit ### When the universe got crowded, Thanos snapped. So did I. Half the crowded grid disappears in the snap. The red outlines show what a cut can cost. When Buykin cleaned up one AI-coded project, he deleted tests and unused boilerplate, but he also lost useful code. [0:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=23s) [An arrow labeled “half of everything” connects a crowded rollout grid to a sparse grid after the snap. Red dashed outlines mark places where a cut without evidence can break something people use.](https://assets.ai.engineer/sites/aiecode2025/sha256/a1f468cfac233ea514dbed666f57567af46510e800c6efdd612d0029fd62dcea.webp) When the universe got crowded, Thanos snapped. So did I. PROLOGUE · THE SNAP After the rollout keep everything: drown in chaos snap half of everything After the snap cut without evidence: break what people use AI Engineer Code Summit ### Since the snap, we keep entropy under a breach line. The rewrite cut entropy sharply, but the agents kept adding code. Weekly sweeps bring entropy back down as it builds. When changes pile up faster than anyone can check them, a small sweep is due that same week. [0:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=54s) [Entropy rises above a dashed breach line and drops sharply at the snap. Weekly sweeps produce a sawtooth pattern below the line, with one circled breach calling for a small sweep that same week.](https://assets.ai.engineer/sites/aiecode2025/sha256/10bbbe28b5bff2d34c37856648294ac02a730d46b40234fb3fddbdb99762ac11.webp) Since the snap, we keep entropy under a breach line. TIME PROLOGUE · HOW DO WE STAY UNDER THE BREACH LINE? Y-axis: entropy X-axis: time → the snap breach line weekly sweeps breach → a small sweep, the same week six stones Time Mind Space Power Reality Soul ### The rules people carry Buykin used to think tests decided what was right. “Check with the team.” leaves a question open: do you ask a person for the rule or consult a tool? [1:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=92s) [Beneath “What does ‘right’ depend on?”, the instruction “Check with the team.” leads to a question mark. Dashed paths branch from it to icons labeled “a person” and “a tool.”](https://assets.ai.engineer/sites/aiecode2025/sha256/ef491bd701b47d74f4c875ce41c07002d6dd8dd6890e9f31a27ec45b83602e88.webp) The rules people carry What does “right” depend on? SPACE CHAPTER 1 “Check with the team.” ? a person { } a tool ### Changing the amount on an approved order A coding agent writes the prose instructions. A task agent follows them in production when someone requests a change to an approved order whose amount of 1,250 already exceeds its limit of 1,000. [1:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=106s) [A coding agent writes prose instructions that a task agent runs. A second arrow feeds the task agent an approved order showing an amount of 1,250 and a limit of 1,000.](https://assets.ai.engineer/sites/aiecode2025/sha256/e62371fcb8d5669a7470a2494f3771831bc4075c2e3133626c72824ede9026c2.webp) One task: change the amount on an approved order. SPACE CHAPTER 1 · THE RULES PEOPLE CARRY coding agent writes the prose prose instructions task agent runs the prose amount 1,250 limit 1,000 · approved ### A code skeleton filled with prose instructions The router sends each task to one process in one country. By default, a change stays in that slice. The agent follows prose instructions within the code, controls check the result before a write, and experts feed back corrections weekly. [2:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=123s) [A router directs the task to a slice for one process and one country. A blue code grid holds gold prose instructions for one agent run. The flow passes through controls to a write step, and experts feed weekly corrections back into the grid.](https://assets.ai.engineer/sites/aiecode2025/sha256/2330d9521d15ad3f13d822adc82b21333b677ac3ed59d448feec344a6e002679.webp) Our system: a code skeleton, filled with prose instructions. SPACE CHAPTER 1 · WHERE MAY A CHANGE TRAVEL? task amount 1,250 limit 1,000 · approved router slice one process · one country one agent run prose code controls write experts weekly corrections ### A quarter code, three quarters prose. Both need engineering. Buykin estimates that prose makes up three quarters of the system. His team adapted mutation testing by deliberately breaking one instruction at a time. For 10 of 11 broken sentences, every test still passed. [2:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=145s) [A bar divides the system into one quarter code and three quarters prose. Arrows link TDD to mutation tests and BDD to scenario replays. Ten of eleven deliberately broken sentences passed every test.](https://assets.ai.engineer/sites/aiecode2025/sha256/142d464d637d55e00b559629e7c30a92a972583ebb309d9c2841b285b464037e.webp) A quarter code, three quarters prose. Both need engineering. REALITY CHAPTER 1 · THE RULES PEOPLE CARRY code ¼ prose ¾ TDD → mutation tests BDD → scenario replays 10 of 11 broken sentences passed every test ### A rewrite is easy. Knowing what must survive is the hard part. After a rewrite, green checks show that the code still behaves as its tests expect. For prose, only readers can judge whether the new wording preserves the intended meaning. [2:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=170s) [The question “still right?” points to two panels. Green checkmarks represent automated checks on rewritten code. People beside lines of prose represent the readers who judge whether the meaning survives.](https://assets.ai.engineer/sites/aiecode2025/sha256/00a39c2e8f2f2629340efed9c5cc7d330a9e5a9825d00913cd8524f0584511e6.webp) A rewrite is easy. Knowing what must survive is the hard part. REALITY SOUL CHAPTER 1 · THE RULES PEOPLE CARRY still right? code + tests a machine says still right prose only people can tell ### Developers hold what experts can’t see. Business experts see rules and exceptions at the tip of the iceberg. Below the waterline, developers trace how systems connect. The agent stops at a contract. Developers have coding agents record those links so the runtime can check them. [3:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=223s) [Rules and exceptions sit above the waterline. Below it, a developer’s lines reach into a network labeled contracts, wire formats, legacy systems and deploy order. An agent’s line stops at a contract.](https://assets.ai.engineer/sites/aiecode2025/sha256/4c3dd1261b2863ed41b91e950cf2d0dbef422514ed2114d2b210d8fd0b3ac994.webp) Developers hold what experts can’t see. CHAPTER 1 · THE RULES PEOPLE CARRY SPACE POWER SOUL Experts see this rules exceptions developer Developers hold the links contracts wire formats legacy systems deploy order stopped at a contract ### One document, two readers. Experts sign the rule in plain language. Beside it, the agent reads a locked table that maps a change within the limit to confirmation and an over-limit change to team lead review. A separate copy could drift out of sync. [4:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=245s) [One document pairs prose that experts read and sign with locked blocks that agents read. Both specify confirmation within the limit and team lead review above it. A separate copy is stamped “drifts.”](https://assets.ai.engineer/sites/aiecode2025/sha256/9792846465fa710a18d132ec019d81144dbdf695344152900186b0dfae2244ac.webp) One document, two readers. MIND SOUL CHAPTER 1 · THE RULES PEOPLE CARRY For the expert Purpose When a requester asks to change an approved amount, we check the limit before we answer. Applies when the new amount is known. Does not apply to a cancellation. What it means over the limit, a team lead decides. Within it, we confirm. experts read the text, and sign For the agent ```tool check_limit(task) ``` | limit_status | next | | WITHIN | CONFIRM | | OVER | LEAD_REVIEW | ```reply title="confirm" Your change is confirmed … ``` agents read the frozen blocks a second copy drifts ### The expert’s screen is our main feedback channel. In the example, 1,250 exceeds the 1,000 limit. An expert flags the draft reply because a team lead must decide. A coding agent makes the change and tests it on the same task. The note closes with evidence of deployment. [4:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=263s) [An illustrative screen pairs an over-limit request with a three-step trace. A blocking note on the draft reply says the team lead decides. Arrows below follow the note through a change, a test on the same task and closure with deployment evidence.](https://assets.ai.engineer/sites/aiecode2025/sha256/0f4e6365623beb30a727aabd339e2e5b648f82e62525438b5991658894eb9fe7.webp) The expert’s screen is our main feedback channel. REALITY SOUL CHAPTER 1 · THE RULES PEOPLE CARRY illustrative screen REQUEST amount 1,250 limit 1,000 · approved answer reply TRACE read_order() check_limit() draft_reply() “draft_reply()” BLOCKING team lead decides note → change → tested on this task → closed with deploy evidence ### The paths that repeat A dashed path follows the route several maze variants share. Two question marks along it ask where experts still need to step in. The team tracks which outcomes experts accept or reject to see where their judgment matters. [4:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=295s) [Overlapping maze outlines show several variants. A dashed teal route labeled “a shared walk” crosses them, with two question marks along the route beneath the question “Where is judgment still necessary?”](https://assets.ai.engineer/sites/aiecode2025/sha256/1fe63c1005dfdaed580f3f193edef781487130ed25e76c3ca53b02ff7cb96259.webp) The paths that repeat Where is judgment still necessary? REALITY CHAPTER 2 variants ? ? a shared walk ### At scale, prose becomes a soup. The bowl of scattered instructions hides a conflict. The same order for 1,250 falls within the prose limit of 1,500 and exceeds the code limit of 1,000. The order still shows as approved, and nothing raises an error. [5:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=318s) [Callouts around a bowl of scattered lines identify an unreachable gate, drifting copies, a hidden comment and a prohibition. An amount of 1,250 falls within the prose limit of 1,500 and exceeds the code limit of 1,000.](https://assets.ai.engineer/sites/aiecode2025/sha256/a1c711b9de20f8bb424b8a09ee7efd8114ad96b1e050eb7ed5546ef0dee2c799.webp) At scale, prose becomes a soup. REALITY CHAPTER 2 · IS IT TRUE? limit: prose 1,500 · code 1,000 unreachable gate drifting copies hidden comment “Do not call X” amount 1,250 limit 1,000 · approved prose: within code: over ### Make prose parseable first. Then lint it like code. A paragraph becomes a typed status lookup, a reply template and a decision table. The table tells the agent when to send a hold note and when to update, then confirm. Authors lint each changed file. [5:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=345s) [Colored arrows connect phrases in a paragraph to a typed tool call, a reply template and a decision table. The table sends a hold note for blocked status and calls for an update followed by confirmation for ready status. A check mark indicates linting per changed file.](https://assets.ai.engineer/sites/aiecode2025/sha256/efaf374d691e4fd15765eefadfb5cecdf32cb133ddf5ae4c7ccf91ea7d7d8fee.webp) Make prose parseable first. Then lint it like code. REALITY CHAPTER 2 · THE PATHS THAT REPEAT Free prose If the request looks urgent, use the lookup tool, reply with the usual note, but first check the status. Do not call the old status tool. compile typed blocks ✓ ```tool lookup_status(id={task.id}, preset="status") templates ✓ ```template title="status-update" Your request is {status}. … decision tables ✓ decision table status = blocked → send hold note status = ready → update, then confirm ✓ linted per changed file ### Encode repeated walks as workflows. The overlapping blue paths show agents following the same steps through written instructions. One workflow tool takes over those steps, and the prose keeps only the decisions at the two yellow forks. [5:57](https://www.youtube.com/watch?v=oeknLUsN_GE&t=357s) [Two mazes appear side by side. Overlapping blue traces show agents repeatedly following prose on the left. A single green workflow on the right marks two decisions in yellow where the route branches.](https://assets.ai.engineer/sites/aiecode2025/sha256/41fc22c434cd88ff08cfb30302c0622bd870f748ddcaa316bf8c7195fb062323.webp) Encode repeated walks as workflows. MIND CHAPTER 2 · THE PATHS THAT REPEAT Before: agents walk the prose task crystallize After: one workflow tool decisions stay at the forks ### A walk becomes code only through a gate that can say stop. A walk is the path an agent takes through written instructions. If agents repeatedly follow the same steps and those steps agree across countries, the team turns the path into one workflow tool. Otherwise, it stays in prose. [6:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=375s) [A diamond asks whether a walk repeats and agrees across variants. The yes branch leads to a workflow tool with prepare, decide and execute steps. The no branch keeps the walk in prose.](https://assets.ai.engineer/sites/aiecode2025/sha256/d371a90827ce1a64db01bd04ed2a0cc8bdaa26023535e930891b17f2d0189f43.webp) A walk becomes code only through a gate that can say stop. REALITY CHAPTER 2 · THE PATHS THAT REPEAT prose walk gate repeats? agrees across variants? no: stays prose code yes: workflow tool Prepare, decide, execute ### What varies stays in prose, inside a frame developers own. Code prepares the facts, and the model picks a row from the experts’ decision table. Since 1,250 exceeds the 1,000 limit, a team lead decides before code carries out the decision. If no row fits, a person takes over. [6:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=397s) [An amount of 1,250 exceeds a limit of 1,000. The highlighted row routes the decision to a team lead before execution. The other rows call for confirmation within the limit and refer unmatched cases to a person.](https://assets.ai.engineer/sites/aiecode2025/sha256/e0bf25aa33d68dad38f1a2adce49d5d3f15762a6e033454cf64344427b3a001c.webp) What varies stays in prose, inside a frame developers own. POWER CHAPTER 2 · THE PATHS THAT REPEAT Prepare amount 1,250 limit 1,000 · approved experts decide over the limit → team lead decides within the limit → confirm no row → a person execute ### Facts move into code. Judgment moves out of it. A repeated prose step becomes check_limit(), so code enforces the hard stop. A regex spots requests to change an amount. Experts can review the prose instructions that say how to handle those requests. [7:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=422s) [A blue arrow leads from numbered prose steps to a limit-check function for repeated steps and hard stops. A yellow arrow leads from an amount-change detector in code to prose instructions that experts can review.](https://assets.ai.engineer/sites/aiecode2025/sha256/309d29572a09b76b2e7aac9fd5963f61931f8cf6a58e615ce68e10430dec678f.webp) Facts move into code. Judgment moves out of it. CHAPTER 2 · THE PATHS THAT REPEAT REALITY POWER CODE check_limit() repeated steps, hard stops /chang.*amount/i a detector; many become config PROSE 1 2 3 asks to change an amount → amount instructions judgment experts can review ### A decision becomes an action The task agent has a record marked approved even though its amount of 1,250 exceeds the 1,000 limit. The question mark before the database asks whether that approval is enough to let the write go through. [7:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=445s) [A task agent has a record marked approved with an amount of 1,250 and a limit of 1,000. A question mark stands at the boundary before the database write.](https://assets.ai.engineer/sites/aiecode2025/sha256/fac957bd2109c1bd0ceafec93e3190fa6b963b8898bcd27f288ee53cb8e052fd.webp) A decision becomes an action What does it mean to stay in control? MIND POWER CHAPTER 3 task agent amount 1,250 limit 1,000 · approved ? the write ### When the model chooses what to load, it invents work. The model loaded every field from four API calls, though it needed only a few to decide. Seeing prices in those responses led it to invent a pricing check. [7:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=460s) [Four read steps each call one API. Their raw responses fill a context panel where only a small portion is marked as needed. The model loads all fields and acts on everything.](https://assets.ai.engineer/sites/aiecode2025/sha256/50a95137d51c94183c2f0ace1229c2a95ba0f60afb85a3d9bd65556e9a4a3752.webp) When the model chooses what to load, it invents work. MIND CHAPTER 3 · WHAT DOES THE MODEL KNOW? Simple prose 1. read A 2. read B 3. read C 4. read D 5. then decide Thin tools one API one API one API one API Raw responses Context what’s needed fields: ["All"] loads everything acts on everything Whatever is in context becomes a reason to act. ### The model sees a few facts. The runtime keeps the full record. The runtime gives the model three facts about the order. The amount is 1,250, the limit is 1,000, and the status is approved. The model selects one row. The runtime uses that selection and the full record to execute the action. [8:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=482s) [Preparation code sends the amount, limit and approval status to the model. The model selects one row. Its selection and the runtime’s full record both feed into execution.](https://assets.ai.engineer/sites/aiecode2025/sha256/d830f128bd0a40b5ed064a1aa336c8df240a9fedb253d853213f1a7d962ee89f.webp) The model sees a few facts. The runtime keeps the full record. MIND POWER CHAPTER 3 · A DECISION BECOMES AN ACTION Raw response prepare (code) amount 1,250 limit 1,000 · approved decides one row full record: hundreds of fields execute ### Control the change before release and at the write. Contracts, tests and replays are in place, but the release gate still needs manual work. At runtime, the system limits which tools can act and guards against applying a change twice. Consent for the exact change and readback are partial. [8:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=496s) [Before release, contracts, tests and replays are marked built. The partly manual release gate is marked partial. At the write, allowed tools and a single-apply journal are built. Consent for the exact change and readback are partial.](https://assets.ai.engineer/sites/aiecode2025/sha256/6d3079c8cc23a1462bc0fd660276f92f8ce40be3ff01397f590414be7aae1c7d.webp) Control the change before release and at the write. POWER CHAPTER 3 · WHAT MAY IT CHANGE? ● built ○ partial BEFORE RELEASE ● contracts ● tests ● replays ○ release gate (partly manual) AT THE WRITE ● allowed tools ○ consent for this exact change ● single-apply journal ○ readback ### The codebase had a breach line. So did I. Buykin read every change his agents made at first. As their output crossed the dashed line marking what one person could read, the orange wedge of unread changes grew. He was burning out and stopped reading every change. [9:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=579s) [A rising blue curve for changes made by agents crosses a dashed line marking what one person can read. Orange hatching marks the changes above that limit.](https://assets.ai.engineer/sites/aiecode2025/sha256/afc91e2fe7dc442881cb10b789e3a2c03d0a7bc2c1ce3dbb9184cf38c3266246.webp) The codebase had a breach line. So did I. SOUL CHAPTER 4 · WHO JUDGES, AND WHAT DOES IT COST? changes my agents made what one person can read changes I could no longer read ### My own sanity is the gauge that fires first. Even when all four dashboard gauges are green, Buykin has an early warning: he’s tired and can’t explain what changed. He stops adding parallel work, makes the changes explainable, then restores handoffs. [10:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=608s) [Four gauges labeled “all green” sit beside a person icon ringed in red above “can’t explain what changed.” Arrows connect three actions: stop adding parallel work, make changes explainable, and restore handoffs.](https://assets.ai.engineer/sites/aiecode2025/sha256/e7374894b67ca0c5f2751d056c7689d854ac1501e2df02e10a0cfa564a046144.webp) My own sanity is the gauge that fires first. TIME SOUL CHAPTER 4 · BEYOND ONE PERSON’S ATTENTION all green “can’t explain what changed” when it fires: stop adding parallel work make changes explainable restore handoffs ### When changes outran my attention, I watched the derivatives. Buykin watches for jumps in four signals to decide which changes to read. To estimate one proposed check’s blast radius, his team replayed its rule over several thousand past sessions. It would have fired on less than 1% of them. [10:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=638s) [Four numbered locations on a flow map correspond to four signals, each with a small trend line: rate per domain, boundary leaks, contract diffs, and a query for blast radius.](https://assets.ai.engineer/sites/aiecode2025/sha256/ba6266660c317fe05430a3760e013a019cf7469b4af0a910005cf767294adea4.webp) When changes outran my attention, I watched the derivatives. CHAPTER 4 · BEYOND ONE PERSON’S ATTENTION TIME SPACE SOUL 1 2 3 4 what I watch 1 rate per domain 2 boundary leaks 3 contract diffs 4 blast radius: a query Friction is the signal. ## Full transcript [0:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=0s) Hi, I'm Dimmitri. In this session, I [0:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=3s) Hi, I'm Dimmitri. In this session, I [0:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=3s) Hi, I'm Dimmitri. In this session, I will show how I work on maintaining [0:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=5s) will show how I work on maintaining [0:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=5s) will show how I work on maintaining large AI coding projects. Coding agents [0:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=8s) large AI coding projects. Coding agents [0:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=8s) large AI coding projects. Coding agents made code cheap. They made entropy even [0:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=12s) made code cheap. They made entropy even [0:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=12s) made code cheap. They made entropy even cheaper. In the Marvel movie, Thanos [0:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=15s) cheaper. In the Marvel movie, Thanos [0:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=15s) cheaper. In the Marvel movie, Thanos watched his home planet Titan collapse [0:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=18s) watched his home planet Titan collapse [0:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=18s) watched his home planet Titan collapse under its own growth. So, when the [0:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=20s) under its own growth. So, when the [0:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=20s) under its own growth. So, when the universe got crowded, he raised half of [0:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=23s) universe got crowded, he raised half of [0:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=23s) universe got crowded, he raised half of everything. This talk is about the one [0:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=26s) everything. This talk is about the one [0:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=26s) everything. This talk is about the one time I did the same in one project. Our [0:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=30s) time I did the same in one project. Our [0:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=30s) time I did the same in one project. Our code was growing the way Titan did. When [0:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=32s) code was growing the way Titan did. When [0:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=32s) code was growing the way Titan did. When I found what my agents had done, I [0:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=36s) I found what my agents had done, I [0:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=36s) I found what my agents had done, I deleted tests and boil the plate code [0:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=40s) deleted tests and boil the plate code [0:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=40s) deleted tests and boil the plate code nobody used. I had to rewrite the whole [0:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=43s) nobody used. I had to rewrite the whole [0:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=43s) nobody used. I had to rewrite the whole project and I lost some useful pieces. [0:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=46s) project and I lost some useful pieces. [0:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=46s) project and I lost some useful pieces. And that's the dilemma. Drown and chaos [0:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=49s) And that's the dilemma. Drown and chaos [0:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=49s) And that's the dilemma. Drown and chaos or break what people use. And here we'll [0:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=53s) or break what people use. And here we'll [0:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=53s) or break what people use. And here we'll build the third option for that project. [0:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=56s) build the third option for that project. [0:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=56s) build the third option for that project. The rewrite didn't hold. The agents kept [0:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=59s) The rewrite didn't hold. The agents kept [1:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=60s) The rewrite didn't hold. The agents kept writing and the mess grew back. So I [1:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=63s) writing and the mess grew back. So I [1:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=63s) writing and the mess grew back. So I drew a line above it. There is more [1:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=66s) drew a line above it. There is more [1:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=66s) drew a line above it. There is more change than anyone can check. The third [1:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=69s) change than anyone can check. The third [1:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=69s) change than anyone can check. The third option is to stay under it with small [1:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=72s) option is to stay under it with small [1:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=72s) option is to stay under it with small sweeps every week and feedback loops [1:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=75s) sweeps every week and feedback loops [1:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=75s) sweeps every week and feedback loops built into the system. Each chapter of [1:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=77s) built into the system. Each chapter of [1:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=77s) built into the system. Each chapter of this talk is one thing I had to learn to [1:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=79s) this talk is one thing I had to learn to [1:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=79s) this talk is one thing I had to learn to get there. Thanos needed six stones for [1:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=82s) get there. Thanos needed six stones for [1:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=82s) get there. Thanos needed six stones for one snap. Ours are six questions and [1:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=86s) one snap. Ours are six questions and [1:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=86s) one snap. Ours are six questions and they light up in the corner as we go. [1:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=89s) they light up in the corner as we go. [1:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=89s) they light up in the corner as we go. The first one is time. How do we stay [1:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=92s) The first one is time. How do we stay [1:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=92s) The first one is time. How do we stay under the line? [1:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=94s) under the line? [1:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=94s) under the line? Chapter one. To stay under the line. I [1:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=97s) Chapter one. To stay under the line. I [1:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=97s) Chapter one. To stay under the line. I first had to know what's right means [1:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=99s) first had to know what's right means [1:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=99s) first had to know what's right means here. I used to think that tests decide [1:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=102s) here. I used to think that tests decide [1:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=102s) here. I used to think that tests decide that. So what does right actually depend [1:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=106s) that. So what does right actually depend [1:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=106s) that. So what does right actually depend on? [1:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=108s) on? [1:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=108s) on? Let's keep one task in mind. Someone [1:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=110s) Let's keep one task in mind. Someone [1:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=110s) Let's keep one task in mind. Someone wants to change the amount on an order [1:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=113s) wants to change the amount on an order [1:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=113s) wants to change the amount on an order we already approved. Two kinds of agents [1:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=116s) we already approved. Two kinds of agents [1:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=116s) we already approved. Two kinds of agents meet here. One wrote the instructions [1:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=119s) meet here. One wrote the instructions [1:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=119s) meet here. One wrote the instructions and the other follows them in [2:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=121s) and the other follows them in [2:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=121s) and the other follows them in production. [2:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=124s) production. [2:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=124s) production. We run this in 110 countries with about [2:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=127s) We run this in 110 countries with about [2:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=127s) We run this in 110 countries with about 10,000 scenarios of pros instructions. [2:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=131s) 10,000 scenarios of pros instructions. [2:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=131s) 10,000 scenarios of pros instructions. Every task lands in a slice. One process [2:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=134s) Every task lands in a slice. One process [2:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=134s) Every task lands in a slice. One process in one country. There's a second stone [2:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=137s) in one country. There's a second stone [2:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=138s) in one country. There's a second stone space. It asks where a change is allowed [2:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=141s) space. It asks where a change is allowed [2:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=141s) space. It asks where a change is allowed to go. By default, it stays inside its [2:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=145s) to go. By default, it stays inside its [2:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=145s) to go. By default, it stays inside its slice. [2:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=147s) slice. [2:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=147s) slice. Most of this system is pros about three [2:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=149s) Most of this system is pros about three [2:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=149s) Most of this system is pros about three quarters by my estimate. So pros needs [2:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=152s) quarters by my estimate. So pros needs [2:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=152s) quarters by my estimate. So pros needs engineering too. At start we borrowed [2:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=155s) engineering too. At start we borrowed [2:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=155s) engineering too. At start we borrowed mutation testing. You break one sentence [2:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=157s) mutation testing. You break one sentence [2:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=157s) mutation testing. You break one sentence on purpose and you watch for red test. [2:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=161s) on purpose and you watch for red test. [2:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=161s) on purpose and you watch for red test. Here almost none turned red. And if your [2:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=164s) Here almost none turned red. And if your [2:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=164s) Here almost none turned red. And if your wrapper is all code, your agent [2:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=167s) wrapper is all code, your agent [2:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=167s) wrapper is all code, your agent instructions and your tests are your [2:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=169s) instructions and your tests are your [2:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=169s) instructions and your tests are your pros. With agents, a rewrite is easy. [2:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=173s) pros. With agents, a rewrite is easy. [2:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=174s) pros. With agents, a rewrite is easy. The hard part is knowing what has to [2:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=176s) The hard part is knowing what has to [2:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=176s) The hard part is knowing what has to survive it. For code, tests can tell [2:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=179s) survive it. For code, tests can tell [2:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=179s) survive it. For code, tests can tell you. For pros, only people can. Agents [3:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=182s) you. For pros, only people can. Agents [3:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=182s) you. For pros, only people can. Agents can't build a complex system on their [3:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=185s) can't build a complex system on their [3:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=185s) can't build a complex system on their own. They fix what is easy to see and [3:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=188s) own. They fix what is easy to see and [3:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=188s) own. They fix what is easy to see and break consistency on the way. And people [3:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=191s) break consistency on the way. And people [3:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=191s) break consistency on the way. And people also get tired. So we spend their [3:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=193s) also get tired. So we spend their [3:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=193s) also get tired. So we spend their judgment only where it's needed. When [3:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=197s) judgment only where it's needed. When [3:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=197s) judgment only where it's needed. When business asked check with the team, it [3:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=200s) business asked check with the team, it [3:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=200s) business asked check with the team, it hides a real question. Is this a human [3:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=203s) hides a real question. Is this a human [3:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=203s) hides a real question. Is this a human step or can we automate it? Experts [3:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=205s) step or can we automate it? Experts [3:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=205s) step or can we automate it? Experts can't answer that. They don't see how [3:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=208s) can't answer that. They don't see how [3:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=208s) can't answer that. They don't see how the systems connect inside. So [3:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=210s) the systems connect inside. So [3:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=210s) the systems connect inside. So developers dig through the traces and [3:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=213s) developers dig through the traces and [3:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=213s) developers dig through the traces and logs with their coding agents and they [3:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=215s) logs with their coding agents and they [3:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=215s) logs with their coding agents and they decide which it is. In our task, the [3:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=218s) decide which it is. In our task, the [3:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=218s) decide which it is. In our task, the team leads decision stays human. The [3:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=221s) team leads decision stays human. The [3:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=221s) team leads decision stays human. The limit was data we could connect. [3:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=225s) limit was data we could connect. [3:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=225s) limit was data we could connect. Business experts see the rules. [3:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=227s) Business experts see the rules. [3:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=227s) Business experts see the rules. Developers see what's under the water. [3:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=230s) Developers see what's under the water. [3:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=230s) Developers see what's under the water. And that's where most bugs leave. They [3:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=233s) And that's where most bugs leave. They [3:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=233s) And that's where most bugs leave. They have their coding agents write those [3:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=235s) have their coding agents write those [3:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=235s) have their coding agents write those links down so the runtime can check [3:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=238s) links down so the runtime can check [3:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=238s) links down so the runtime can check them. Our guide for experts says it [4:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=240s) them. Our guide for experts says it [4:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=240s) them. Our guide for experts says it simply business tell us what we work out [4:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=244s) simply business tell us what we work out [4:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=244s) simply business tell us what we work out how. One document to readers. Agents [4:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=249s) how. One document to readers. Agents [4:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=249s) how. One document to readers. Agents read the blogs. So their blocks have to [4:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=251s) read the blogs. So their blocks have to [4:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=251s) read the blogs. So their blocks have to be exact. Experts read the text around [4:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=254s) be exact. Experts read the text around [4:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=254s) be exact. Experts read the text around them and they sign it. We keep one copy, [4:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=257s) them and they sign it. We keep one copy, [4:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=257s) them and they sign it. We keep one copy, a second one with drift and works both [4:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=260s) a second one with drift and works both [4:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=260s) a second one with drift and works both ways. Our code needs a plain description [4:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=262s) ways. Our code needs a plain description [4:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=262s) ways. Our code needs a plain description for experts too. Experts can only fix [4:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=266s) for experts too. Experts can only fix [4:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=266s) for experts too. Experts can only fix what they can see. Each of them reviews [4:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=269s) what they can see. Each of them reviews [4:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=269s) what they can see. Each of them reviews a task exactly the way the agent saw it. [4:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=273s) a task exactly the way the agent saw it. [4:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=273s) a task exactly the way the agent saw it. The node becomes the next change made by [4:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=275s) The node becomes the next change made by [4:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=275s) The node becomes the next change made by a coding agent and test it on the same [4:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=278s) a coding agent and test it on the same [4:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=278s) a coding agent and test it on the same task. So right depends on people. No [4:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=282s) task. So right depends on people. No [4:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=282s) task. So right depends on people. No single person knows it. A group of [4:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=285s) single person knows it. A group of [4:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=285s) single person knows it. A group of experts decides and that's why we bring [4:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=288s) experts decides and that's why we bring [4:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=288s) experts decides and that's why we bring them in from the start. You'll see the [4:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=291s) them in from the start. You'll see the [4:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=291s) them in from the start. You'll see the soul stone here a lot. We take it in [4:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=294s) soul stone here a lot. We take it in [4:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=294s) soul stone here a lot. We take it in chapter 4. [4:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=296s) chapter 4. [4:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=296s) chapter 4. Chapter two. Experts correct one task at [4:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=299s) Chapter two. Experts correct one task at [4:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=299s) Chapter two. Experts correct one task at a time while agents run the same pros [5:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=302s) a time while agents run the same pros [5:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=302s) a time while agents run the same pros all day. We track every outcome accepted [5:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=306s) all day. We track every outcome accepted [5:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=306s) all day. We track every outcome accepted or rejected. The accuracy shows us where [5:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=310s) or rejected. The accuracy shows us where [5:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=310s) or rejected. The accuracy shows us where people still need to look. I used to [5:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=313s) people still need to look. I used to [5:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=313s) people still need to look. I used to think the goal was to move everything [5:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=315s) think the goal was to move everything [5:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=315s) think the goal was to move everything into code. So where do we still need [5:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=318s) into code. So where do we still need [5:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=318s) into code. So where do we still need judgment? Here is our task again. The [5:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=321s) judgment? Here is our task again. The [5:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=321s) judgment? Here is our task again. The pro says the limit is 1,500. The code [5:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=325s) pro says the limit is 1,500. The code [5:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=325s) pro says the limit is 1,500. The code says a,000. The same order is fine in [5:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=328s) says a,000. The same order is fine in [5:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=328s) says a,000. The same order is fine in one place and too high in the other and [5:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=331s) one place and too high in the other and [5:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=331s) one place and too high in the other and nothing throws an error. Even a line [5:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=334s) nothing throws an error. Even a line [5:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=334s) nothing throws an error. Even a line like do not call X teaches the model [5:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=337s) like do not call X teaches the model [5:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=337s) like do not call X teaches the model that X exists. That's the third stone [5:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=341s) that X exists. That's the third stone [5:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=341s) that X exists. That's the third stone reality. Is it true or does it only look [5:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=344s) reality. Is it true or does it only look [5:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=344s) reality. Is it true or does it only look true for the model? Pros has no [5:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=347s) true for the model? Pros has no [5:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=347s) true for the model? Pros has no compiler. So we built one. Then the [5:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=350s) compiler. So we built one. Then the [5:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=350s) compiler. So we built one. Then the author lints every changed file into [5:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=353s) author lints every changed file into [5:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=353s) author lints every changed file into ledger just like code. [5:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=358s) Lind can't stop copies drifting aparts. [6:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=362s) Lind can't stop copies drifting aparts. [6:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=362s) Lind can't stop copies drifting aparts. So we watched the walks. A walk is the [6:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=364s) So we watched the walks. A walk is the [6:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=364s) So we watched the walks. A walk is the path one agent takes through the pros. [6:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=367s) path one agent takes through the pros. [6:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=367s) path one agent takes through the pros. When agents keep walking in the same [6:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=369s) When agents keep walking in the same [6:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=369s) When agents keep walking in the same path, we turn it into one workflow tool [6:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=372s) path, we turn it into one workflow tool [6:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=372s) path, we turn it into one workflow tool and we delete the pros that walked it. [6:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=376s) and we delete the pros that walked it. [6:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=376s) and we delete the pros that walked it. Our first workflow of tools made things [6:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=379s) Our first workflow of tools made things [6:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=379s) Our first workflow of tools made things worse. The pros still named the steps [6:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=381s) worse. The pros still named the steps [6:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=382s) worse. The pros still named the steps inside them. So agents run those steps [6:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=385s) inside them. So agents run those steps [6:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=385s) inside them. So agents run those steps again. It worked once. The pros kept [6:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=387s) again. It worked once. The pros kept [6:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=387s) again. It worked once. The pros kept only the decisions. Now a walk becomes [6:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=391s) only the decisions. Now a walk becomes [6:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=391s) only the decisions. Now a walk becomes code only when it repeats and agrees [6:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=394s) code only when it repeats and agrees [6:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=394s) code only when it repeats and agrees across countries otherwise it stays [6:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=397s) across countries otherwise it stays [6:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=397s) across countries otherwise it stays pros. What still varies stays in pros [6:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=401s) pros. What still varies stays in pros [6:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=401s) pros. What still varies stays in pros inside the frame. Code prepares the [6:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=404s) inside the frame. Code prepares the [6:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=404s) inside the frame. Code prepares the facts and carries out the decision. [6:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=407s) facts and carries out the decision. [6:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=407s) facts and carries out the decision. Experts own the decision table and the [6:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=409s) Experts own the decision table and the [6:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=410s) Experts own the decision table and the model only picks a row. If no row fits, [6:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=413s) model only picks a row. If no row fits, [6:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=414s) model only picks a row. If no row fits, a person takes over. We haven't built [6:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=416s) a person takes over. We haven't built [6:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=416s) a person takes over. We haven't built the next step yet. The model will take [6:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=418s) the next step yet. The model will take [6:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=418s) the next step yet. The model will take the small decisions and workflows will [7:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=421s) the small decisions and workflows will [7:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=421s) the small decisions and workflows will connect them. This boundary moves both [7:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=424s) connect them. This boundary moves both [7:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=424s) connect them. This boundary moves both ways. We moved hundreds of lines of [7:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=427s) ways. We moved hundreds of lines of [7:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=427s) ways. We moved hundreds of lines of expressions out of code and into pros [7:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=431s) expressions out of code and into pros [7:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=431s) expressions out of code and into pros because experts couldn't read them. A [7:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=434s) because experts couldn't read them. A [7:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=434s) because experts couldn't read them. A reax is still fine for spotting a [7:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=435s) reax is still fine for spotting a [7:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=435s) reax is still fine for spotting a critical signal. When there are many, [7:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=438s) critical signal. When there are many, [7:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=438s) critical signal. When there are many, they move into config. Judgment stays [7:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=441s) they move into config. Judgment stays [7:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=441s) they move into config. Judgment stays wherever the answer varies in a place an [7:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=444s) wherever the answer varies in a place an [7:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=444s) wherever the answer varies in a place an expert can read. Chapter three. Now the [7:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=448s) expert can read. Chapter three. Now the [7:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=448s) expert can read. Chapter three. Now the model decides and its decision becomes [7:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=451s) model decides and its decision becomes [7:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=451s) model decides and its decision becomes an action. I used to think control meant [7:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=455s) an action. I used to think control meant [7:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=455s) an action. I used to think control meant permissions on the agent. So what does [7:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=457s) permissions on the agent. So what does [7:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=457s) permissions on the agent. So what does it mean to stay in control? [7:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=461s) it mean to stay in control? [7:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=461s) it mean to stay in control? Control starts with what the model sees. [7:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=464s) Control starts with what the model sees. [7:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=464s) Control starts with what the model sees. That's what the fourth stone mind. What [7:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=467s) That's what the fourth stone mind. What [7:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=467s) That's what the fourth stone mind. What does the model know? When we let it [7:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=469s) does the model know? When we let it [7:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=469s) does the model know? When we let it choose what to load, it loaded [7:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=472s) choose what to load, it loaded [7:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=472s) choose what to load, it loaded everything. Then it invented a pricing [7:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=475s) everything. Then it invented a pricing [7:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=475s) everything. Then it invented a pricing check just because prices were visible. [7:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=478s) check just because prices were visible. [7:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=478s) check just because prices were visible. Whatever sits in its context becomes a [8:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=481s) Whatever sits in its context becomes a [8:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=481s) Whatever sits in its context becomes a reason to act. The runtime decides [8:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=484s) reason to act. The runtime decides [8:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=484s) reason to act. The runtime decides what's the model sees for our task. [8:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=487s) what's the model sees for our task. [8:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=487s) what's the model sees for our task. That's three facts from the order. The [8:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=489s) That's three facts from the order. The [8:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=489s) That's three facts from the order. The full record stays with the runtime and [8:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=492s) full record stays with the runtime and [8:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=492s) full record stays with the runtime and the model makes fewer wrong moves. [8:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=497s) Here how we minimize context blo for a [8:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=500s) Here how we minimize context blo for a [8:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=500s) Here how we minimize context blo for a big set. The model passes a handle, a [8:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=502s) big set. The model passes a handle, a [8:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=502s) big set. The model passes a handle, a short name for date that the runtime [8:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=504s) short name for date that the runtime [8:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=504s) short name for date that the runtime already holds. The runtime checks it [8:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=507s) already holds. The runtime checks it [8:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=507s) already holds. The runtime checks it before any write. It keeps the context [8:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=509s) before any write. It keeps the context [8:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=509s) before any write. It keeps the context small and still enough for the decision. [8:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=512s) small and still enough for the decision. [8:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=512s) small and still enough for the decision. At the same time, it uh validates [8:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=515s) At the same time, it uh validates [8:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=515s) At the same time, it uh validates context for consistency. A model with [8:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=519s) context for consistency. A model with [8:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=519s) context for consistency. A model with the right facts can still do damage. [8:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=521s) the right facts can still do damage. [8:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=521s) the right facts can still do damage. That's the fifth stone, power. What made [8:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=525s) That's the fifth stone, power. What made [8:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=525s) That's the fifth stone, power. What made change? We control a change twice. [8:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=527s) change? We control a change twice. [8:47](https://www.youtube.com/watch?v=oeknLUsN_GE&t=527s) change? We control a change twice. during development and again at the [8:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=530s) during development and again at the [8:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=530s) during development and again at the right at runtime only code opens the [8:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=533s) right at runtime only code opens the [8:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=533s) right at runtime only code opens the right path so we can change pros many [8:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=536s) right path so we can change pros many [8:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=536s) right path so we can change pros many times a day the open rings are the parts [8:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=539s) times a day the open rings are the parts [8:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=539s) times a day the open rings are the parts that still depend on people let's zoom [9:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=542s) that still depend on people let's zoom [9:02](https://www.youtube.com/watch?v=oeknLUsN_GE&t=542s) that still depend on people let's zoom in and run right five walls stand in [9:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=546s) in and run right five walls stand in [9:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=546s) in and run right five walls stand in front of it and the sixth checks that it [9:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=548s) front of it and the sixth checks that it [9:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=548s) front of it and the sixth checks that it landed if a right looks like it failed [9:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=552s) landed if a right looks like it failed [9:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=552s) landed if a right looks like it failed an agent retries and the journal makes [9:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=555s) an agent retries and the journal makes [9:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=555s) an agent retries and the journal makes sure it runs only once means much more [9:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=558s) sure it runs only once means much more [9:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=558s) sure it runs only once means much more than permissions on the agent. [9:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=564s) Chapter 4. Most days I glide over the [9:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=567s) Chapter 4. Most days I glide over the [9:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=567s) Chapter 4. Most days I glide over the system. From up here, everything looks [9:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=570s) system. From up here, everything looks [9:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=570s) system. From up here, everything looks fine, so I don't go down into every [9:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=572s) fine, so I don't go down into every [9:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=572s) fine, so I don't go down into every change my agents made. Then the surface [9:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=576s) change my agents made. Then the surface [9:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=576s) change my agents made. Then the surface gets rough. How do we know we're losing [9:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=579s) gets rough. How do we know we're losing [9:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=579s) gets rough. How do we know we're losing control? [9:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=581s) control? [9:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=581s) control? All those controls protect the system. [9:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=583s) All those controls protect the system. [9:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=583s) All those controls protect the system. Nothing protected my attention. At the [9:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=586s) Nothing protected my attention. At the [9:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=586s) Nothing protected my attention. At the start, I read every change and that's [9:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=588s) start, I read every change and that's [9:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=588s) start, I read every change and that's how the system picked up my habits. Then [9:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=592s) how the system picked up my habits. Then [9:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=592s) how the system picked up my habits. Then the volume grew past what one person can [9:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=595s) the volume grew past what one person can [9:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=595s) the volume grew past what one person can read. I was burning out and I stopped [9:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=598s) read. I was burning out and I stopped [9:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=598s) read. I was burning out and I stopped reading every change. That was the price [10:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=600s) reading every change. That was the price [10:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=600s) reading every change. That was the price of the last stone. So, who judges and [10:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=604s) of the last stone. So, who judges and [10:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=604s) of the last stone. So, who judges and what does it cost? With it, the gauntlet [10:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=607s) what does it cost? With it, the gauntlet [10:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=607s) what does it cost? With it, the gauntlet is complete. Every tech lead I know has [10:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=610s) is complete. Every tech lead I know has [10:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=610s) is complete. Every tech lead I know has been here. The one person every change [10:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=613s) been here. The one person every change [10:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=613s) been here. The one person every change waits on. agents just get you there in [10:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=616s) waits on. agents just get you there in [10:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=616s) waits on. agents just get you there in weeks instead of years. I needed a way [10:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=619s) weeks instead of years. I needed a way [10:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=619s) weeks instead of years. I needed a way of working that doesn't run through one [10:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=622s) of working that doesn't run through one [10:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=622s) of working that doesn't run through one head. And the first gauge was me. When [10:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=625s) head. And the first gauge was me. When [10:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=625s) head. And the first gauge was me. When I'm tired and I can't explain what [10:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=627s) I'm tired and I can't explain what [10:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=627s) I'm tired and I can't explain what changed, that's the signal. The [10:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=630s) changed, that's the signal. The [10:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=630s) changed, that's the signal. The dashboard can be green while that gauge [10:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=633s) dashboard can be green while that gauge [10:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=633s) dashboard can be green while that gauge is firing. So, I stop and fix the [10:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=636s) is firing. So, I stop and fix the [10:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=636s) is firing. So, I stop and fix the process first. I don't wait for a [10:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=638s) process first. I don't wait for a [10:38](https://www.youtube.com/watch?v=oeknLUsN_GE&t=638s) process first. I don't wait for a number. I couldn't read every div [10:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=641s) number. I couldn't read every div [10:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=641s) number. I couldn't read every div anymore. So I built instruments that [10:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=643s) anymore. So I built instruments that [10:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=644s) anymore. So I built instruments that watch how things change and their [10:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=646s) watch how things change and their [10:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=646s) watch how things change and their derivatives. One of them is blast [10:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=649s) derivatives. One of them is blast [10:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=649s) derivatives. One of them is blast radius. Before we built one new check, [10:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=652s) radius. Before we built one new check, [10:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=652s) radius. Before we built one new check, we replayed its rule over several [10:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=654s) we replayed its rule over several [10:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=654s) we replayed its rule over several thousand past sessions. It would have [10:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=656s) thousand past sessions. It would have [10:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=656s) thousand past sessions. It would have fired on less than 1%. Where the numbers [11:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=660s) fired on less than 1%. Where the numbers [11:00](https://www.youtube.com/watch?v=oeknLUsN_GE&t=660s) fired on less than 1%. Where the numbers jump, I go down and read. When something [11:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=664s) jump, I go down and read. When something [11:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=664s) jump, I go down and read. When something looks wrong, we wait. One signal waits [11:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=666s) looks wrong, we wait. One signal waits [11:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=666s) looks wrong, we wait. One signal waits until it comes back, even when it comes [11:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=669s) until it comes back, even when it comes [11:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=669s) until it comes back, even when it comes from an expert. A cluster across slices [11:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=672s) from an expert. A cluster across slices [11:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=672s) from an expert. A cluster across slices is different. Then we fix the concept [11:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=675s) is different. Then we fix the concept [11:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=675s) is different. Then we fix the concept behind it once. Left alone, an agent [11:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=678s) behind it once. Left alone, an agent [11:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=678s) behind it once. Left alone, an agent keeps adding conditions until today's [11:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=680s) keeps adding conditions until today's [11:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=680s) keeps adding conditions until today's tasks pass. The eval score drops when [11:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=683s) tasks pass. The eval score drops when [11:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=683s) tasks pass. The eval score drops when you take them out and you get an honest [11:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=686s) you take them out and you get an honest [11:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=686s) you take them out and you get an honest list of gaps. [11:31](https://www.youtube.com/watch?v=oeknLUsN_GE&t=691s) One red slice is rarely alone. The [11:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=694s) One red slice is rarely alone. The [11:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=694s) One red slice is rarely alone. The experts note becomes an eval and an [11:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=696s) experts note becomes an eval and an [11:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=696s) experts note becomes an eval and an agent searches the siblings for the same [11:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=699s) agent searches the siblings for the same [11:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=699s) agent searches the siblings for the same failure. Siblings differ on purpose by [11:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=702s) failure. Siblings differ on purpose by [11:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=702s) failure. Siblings differ on purpose by market and by local law. So we sweep [11:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=705s) market and by local law. So we sweep [11:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=705s) market and by local law. So we sweep them and the fix goes only where the [11:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=708s) them and the fix goes only where the [11:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=708s) them and the fix goes only where the failure is confirmed. [11:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=713s) At any time 10 to 20 of these loops are [11:57](https://www.youtube.com/watch?v=oeknLUsN_GE&t=717s) At any time 10 to 20 of these loops are [11:57](https://www.youtube.com/watch?v=oeknLUsN_GE&t=717s) At any time 10 to 20 of these loops are open. At one point I had become the [11:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=719s) open. At one point I had become the [11:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=719s) open. At one point I had become the message bus between users and my agents. [12:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=723s) message bus between users and my agents. [12:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=723s) message bus between users and my agents. At this scale, every user has to be an [12:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=726s) At this scale, every user has to be an [12:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=726s) At this scale, every user has to be an evaluator with a screen to give feedback [12:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=728s) evaluator with a screen to give feedback [12:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=728s) evaluator with a screen to give feedback without me. My job as the architect is [12:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=732s) without me. My job as the architect is [12:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=732s) without me. My job as the architect is to build those loops into the system. [12:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=735s) to build those loops into the system. [12:15](https://www.youtube.com/watch?v=oeknLUsN_GE&t=735s) to build those loops into the system. Each loop writes to a ledger. So nothing [12:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=737s) Each loop writes to a ledger. So nothing [12:18](https://www.youtube.com/watch?v=oeknLUsN_GE&t=738s) Each loop writes to a ledger. So nothing waits on my memory and nothing waits on [12:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=740s) waits on my memory and nothing waits on [12:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=740s) waits on my memory and nothing waits on me. [12:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=744s) Agents forget everything when a session [12:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=746s) Agents forget everything when a session [12:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=746s) Agents forget everything when a session ends. So the memory has to leave in [12:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=749s) ends. So the memory has to leave in [12:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=749s) ends. So the memory has to leave in files. We keep two kinds. what happened [12:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=752s) files. We keep two kinds. what happened [12:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=752s) files. We keep two kinds. what happened and what is true now. The next session [12:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=755s) and what is true now. The next session [12:35](https://www.youtube.com/watch?v=oeknLUsN_GE&t=755s) and what is true now. The next session starts from what is true now. [12:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=760s) Every issue has to connect to its root [12:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=763s) Every issue has to connect to its root [12:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=763s) Every issue has to connect to its root cause. Without that, the agent decides [12:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=766s) cause. Without that, the agent decides [12:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=766s) cause. Without that, the agent decides on wrong facts and the check has nothing [12:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=769s) on wrong facts and the check has nothing [12:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=769s) on wrong facts and the check has nothing to count. So we don't act without [12:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=771s) to count. So we don't act without [12:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=771s) to count. So we don't act without evidence. [12:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=774s) evidence. [12:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=774s) evidence. Reality comes back. In chapter two, it [12:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=778s) Reality comes back. In chapter two, it [12:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=778s) Reality comes back. In chapter two, it asked if the pros is true. Now it asks [13:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=781s) asked if the pros is true. Now it asks [13:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=781s) asked if the pros is true. Now it asks if the change is I used to trust green. [13:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=785s) if the change is I used to trust green. [13:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=785s) if the change is I used to trust green. So what would make us change our mind? [13:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=788s) So what would make us change our mind? [13:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=788s) So what would make us change our mind? Green was tempting because checking is [13:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=791s) Green was tempting because checking is [13:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=791s) Green was tempting because checking is where the times goes. My agent spend [13:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=793s) where the times goes. My agent spend [13:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=794s) where the times goes. My agent spend almost 9/10en of that time on a change [13:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=797s) almost 9/10en of that time on a change [13:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=797s) almost 9/10en of that time on a change just checking it and checking took a [13:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=799s) just checking it and checking took a [13:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=799s) just checking it and checking took a bigger share of my own week too. [13:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=802s) bigger share of my own week too. [13:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=802s) bigger share of my own week too. Verification speed is the limit. [13:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=807s) We rank evidence by what it proves. A [13:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=810s) We rank evidence by what it proves. A [13:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=810s) We rank evidence by what it proves. A replay on real tasks sits at the top. [13:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=813s) replay on real tasks sits at the top. [13:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=814s) replay on real tasks sits at the top. Before the data arrives, we write down [13:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=816s) Before the data arrives, we write down [13:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=816s) Before the data arrives, we write down what would prove us wrong. And every new [13:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=819s) what would prove us wrong. And every new [13:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=819s) what would prove us wrong. And every new check has to fail once on purpose before [13:41](https://www.youtube.com/watch?v=oeknLUsN_GE&t=821s) check has to fail once on purpose before [13:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=822s) check has to fail once on purpose before we trust it. [13:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=826s) AI refactors fail quietly. Every [13:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=830s) AI refactors fail quietly. Every [13:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=830s) AI refactors fail quietly. Every condition and every decision table row [13:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=833s) condition and every decision table row [13:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=833s) condition and every decision table row is a branch. And after a refactor, each [13:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=836s) is a branch. And after a refactor, each [13:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=836s) is a branch. And after a refactor, each one has to land exactly once. Here the [13:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=839s) one has to land exactly once. Here the [13:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=839s) one has to land exactly once. Here the branch for amounts over the limit got [14:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=841s) branch for amounts over the limit got [14:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=841s) branch for amounts over the limit got lost. Our [snorts] order would have gone [14:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=844s) lost. Our [snorts] order would have gone [14:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=844s) lost. Our [snorts] order would have gone through without the team lead. Real [14:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=847s) through without the team lead. Real [14:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=847s) through without the team lead. Real tasks decide what we test. We freeze [14:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=851s) tasks decide what we test. We freeze [14:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=851s) tasks decide what we test. We freeze them as a set and replay the old version [14:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=854s) them as a set and replay the old version [14:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=854s) them as a set and replay the old version against the new one. If no real task [14:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=857s) against the new one. If no real task [14:17](https://www.youtube.com/watch?v=oeknLUsN_GE&t=857s) against the new one. If no real task takes a branch, we call that no [14:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=859s) takes a branch, we call that no [14:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=859s) takes a branch, we call that no evidence. [14:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=861s) evidence. [14:21](https://www.youtube.com/watch?v=oeknLUsN_GE&t=861s) evidence. You can't trust one agent to judge its [14:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=864s) You can't trust one agent to judge its [14:24](https://www.youtube.com/watch?v=oeknLUsN_GE&t=864s) You can't trust one agent to judge its own work. A second agent checks it with [14:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=867s) own work. A second agent checks it with [14:28](https://www.youtube.com/watch?v=oeknLUsN_GE&t=868s) own work. A second agent checks it with a checklist and without the author's [14:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=870s) a checklist and without the author's [14:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=870s) a checklist and without the author's reasoning. It can't be talked into [14:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=872s) reasoning. It can't be talked into [14:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=872s) reasoning. It can't be talked into agreeing. Over 6 months, it grew to most [14:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=876s) agreeing. Over 6 months, it grew to most [14:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=876s) agreeing. Over 6 months, it grew to most of my checking. [14:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=879s) of my checking. [14:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=879s) of my checking. Coding agents finish most of a batch and [14:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=883s) Coding agents finish most of a batch and [14:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=883s) Coding agents finish most of a batch and quietly skipped the rest, especially [14:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=885s) quietly skipped the rest, especially [14:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=885s) quietly skipped the rest, especially when their context is full. The ledger [14:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=888s) when their context is full. The ledger [14:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=888s) when their context is full. The ledger shows what was skipped in one backlog. [14:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=891s) shows what was skipped in one backlog. [14:51](https://www.youtube.com/watch?v=oeknLUsN_GE&t=891s) shows what was skipped in one backlog. Most items looks covered, but only about [14:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=894s) Most items looks covered, but only about [14:54](https://www.youtube.com/watch?v=oeknLUsN_GE&t=894s) Most items looks covered, but only about a quarter could really be closed. So now [14:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=898s) a quarter could really be closed. So now [14:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=898s) a quarter could really be closed. So now evidence we can count is what changes [15:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=901s) evidence we can count is what changes [15:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=901s) evidence we can count is what changes our mind. And this is the third option [15:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=905s) our mind. And this is the third option [15:05](https://www.youtube.com/watch?v=oeknLUsN_GE&t=905s) our mind. And this is the third option from the start of the talk. I used to [15:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=908s) from the start of the talk. I used to [15:08](https://www.youtube.com/watch?v=oeknLUsN_GE&t=908s) from the start of the talk. I used to think there were two choices. Keep [15:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=911s) think there were two choices. Keep [15:11](https://www.youtube.com/watch?v=oeknLUsN_GE&t=911s) think there were two choices. Keep everything or snap. So what is worth [15:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=914s) everything or snap. So what is worth [15:14](https://www.youtube.com/watch?v=oeknLUsN_GE&t=914s) everything or snap. So what is worth carrying forward? [15:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=916s) carrying forward? [15:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=916s) carrying forward? This one process over a year. Some [15:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=919s) This one process over a year. Some [15:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=919s) This one process over a year. Some branches merged back, some still drift, [15:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=922s) branches merged back, some still drift, [15:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=922s) branches merged back, some still drift, and some are dead but still look alive. [15:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=925s) and some are dead but still look alive. [15:25](https://www.youtube.com/watch?v=oeknLUsN_GE&t=925s) and some are dead but still look alive. Which ones deserve to survive? [15:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=930s) A fix is only a moment. These forces [15:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=933s) A fix is only a moment. These forces [15:33](https://www.youtube.com/watch?v=oeknLUsN_GE&t=933s) A fix is only a moment. These forces keep pushing right after it. You've met [15:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=936s) keep pushing right after it. You've met [15:36](https://www.youtube.com/watch?v=oeknLUsN_GE&t=936s) keep pushing right after it. You've met most of them already. This chapter deals [15:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=939s) most of them already. This chapter deals [15:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=939s) most of them already. This chapter deals with regrowth and branching. [15:44](https://www.youtube.com/watch?v=oeknLUsN_GE&t=944s) The first gauge was me back in chapter [15:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=946s) The first gauge was me back in chapter [15:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=946s) The first gauge was me back in chapter 4. These are the gauges for the code. [15:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=950s) 4. These are the gauges for the code. [15:50](https://www.youtube.com/watch?v=oeknLUsN_GE&t=950s) 4. These are the gauges for the code. Time gets its answers here. The bridge [15:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=953s) Time gets its answers here. The bridge [15:53](https://www.youtube.com/watch?v=oeknLUsN_GE&t=953s) Time gets its answers here. The bridge line comes back as a number. A ratchet [15:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=956s) line comes back as a number. A ratchet [15:56](https://www.youtube.com/watch?v=oeknLUsN_GE&t=956s) line comes back as a number. A ratchet means the count can only go down. Two [15:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=958s) means the count can only go down. Two [15:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=958s) means the count can only go down. Two gauges run today and six more are next. [16:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=964s) gauges run today and six more are next. [16:04](https://www.youtube.com/watch?v=oeknLUsN_GE&t=964s) gauges run today and six more are next. Every week real use decides what stays. [16:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=967s) Every week real use decides what stays. [16:07](https://www.youtube.com/watch?v=oeknLUsN_GE&t=967s) Every week real use decides what stays. Anything no real task reaches gets a [16:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=970s) Anything no real task reaches gets a [16:10](https://www.youtube.com/watch?v=oeknLUsN_GE&t=970s) Anything no real task reaches gets a name and its evidence before it goes. [16:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=973s) name and its evidence before it goes. [16:13](https://www.youtube.com/watch?v=oeknLUsN_GE&t=973s) name and its evidence before it goes. Test the evidence I didn't have when I [16:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=976s) Test the evidence I didn't have when I [16:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=976s) Test the evidence I didn't have when I snapped. Tests kept that variants alive. [16:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=980s) snapped. Tests kept that variants alive. [16:20](https://www.youtube.com/watch?v=oeknLUsN_GE&t=980s) snapped. Tests kept that variants alive. Agents kept every option a test still [16:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=983s) Agents kept every option a test still [16:23](https://www.youtube.com/watch?v=oeknLUsN_GE&t=983s) Agents kept every option a test still mentioned. Now retired variant takes its [16:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=987s) mentioned. Now retired variant takes its [16:27](https://www.youtube.com/watch?v=oeknLUsN_GE&t=987s) mentioned. Now retired variant takes its test with it. Here's the whole method as [16:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=990s) test with it. Here's the whole method as [16:30](https://www.youtube.com/watch?v=oeknLUsN_GE&t=990s) test with it. Here's the whole method as one week. This is the third option as a [16:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=994s) one week. This is the third option as a [16:34](https://www.youtube.com/watch?v=oeknLUsN_GE&t=994s) one week. This is the third option as a process. [16:39](https://www.youtube.com/watch?v=oeknLUsN_GE&t=999s) together. That's the full gauntlet. [16:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1002s) together. That's the full gauntlet. [16:42](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1002s) together. That's the full gauntlet. Thanos used his for one snap. We use [16:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1005s) Thanos used his for one snap. We use [16:45](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1005s) Thanos used his for one snap. We use ours every week in small doses with [16:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1008s) ours every week in small doses with [16:48](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1008s) ours every week in small doses with evidence. I haven't had to snap again. [16:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1015s) Soul had a second price and I only saw [16:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1018s) Soul had a second price and I only saw [16:58](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1018s) Soul had a second price and I only saw it later. After we deliver it to [17:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1021s) it later. After we deliver it to [17:01](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1021s) it later. After we deliver it to production, I found my own engineering [17:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1023s) production, I found my own engineering [17:03](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1023s) production, I found my own engineering principles built into the systems [17:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1026s) principles built into the systems [17:06](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1026s) principles built into the systems design. Its skills and its memory carry [17:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1029s) design. Its skills and its memory carry [17:09](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1029s) design. Its skills and its memory carry my habits and my biases. Conway said the [17:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1032s) my habits and my biases. Conway said the [17:12](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1032s) my habits and my biases. Conway said the system copies the orc chart. Ours copies [17:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1036s) system copies the orc chart. Ours copies [17:16](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1036s) system copies the orc chart. Ours copies whoever corrects it. Here are 18 [17:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1039s) whoever corrects it. Here are 18 [17:19](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1039s) whoever corrects it. Here are 18 questions, three for each stone. I'm [17:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1042s) questions, three for each stone. I'm [17:22](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1042s) questions, three for each stone. I'm using them when working on my projects. [17:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1046s) using them when working on my projects. [17:26](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1046s) using them when working on my projects. Pause here and pick one your own [17:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1049s) Pause here and pick one your own [17:29](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1049s) Pause here and pick one your own repository can't answer yet. That's [17:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1052s) repository can't answer yet. That's [17:32](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1052s) repository can't answer yet. That's where to start. [17:37](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1057s) I learned to use this process when I was [17:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1060s) I learned to use this process when I was [17:40](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1060s) I learned to use this process when I was already burning out. If you're the one [17:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1063s) already burning out. If you're the one [17:43](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1063s) already burning out. If you're the one your team's changes weight on today, [17:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1066s) your team's changes weight on today, [17:46](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1066s) your team's changes weight on today, this is for you. Use your stones while [17:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1069s) this is for you. Use your stones while [17:49](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1069s) this is for you. Use your stones while you can still read every change. In the [17:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1072s) you can still read every change. In the [17:52](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1072s) you can still read every change. In the end, Thanos retires to a garden. Maybe [17:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1075s) end, Thanos retires to a garden. Maybe [17:55](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1075s) end, Thanos retires to a garden. Maybe he should start using them daily. Thank [17:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1079s) he should start using them daily. Thank [17:59](https://www.youtube.com/watch?v=oeknLUsN_GE&t=1079s) he should start using them daily. Thank you.