Making Codebases "Agent-Ready"

What would it take to make an engineering organization capable of using autonomous software systems successfully? For Eno Reyes at Factory, bringing autonomy to software engineering requires work on the organization as well as the agent. That makes agent readiness a useful goal regardless of which coding, review, or documentation tools a team chooses.

The starting point is automation through verification. Reyes introduces it through Andrej Karpathy’s Software 2.0 framing; Karpathy’s Verifiability develops the closely related distinction. Traditionally, a programmer specifies an algorithm: given an input, perform these operations to produce an output. With a verifiable objective, a system can instead search through candidate solutions and check whether they satisfy it. Reyes connects that possibility to frontier-model post-training on tasks whose outcomes can be checked. The practical question becomes whether you can specify the objective and recognize a successful solution.

This works especially well when checking an answer is easier than discovering it. The discussion of Jason’s verification-asymmetry post matches Jason Wei’s Asymmetry of verification and verifier’s rule. The P-versus-NP analogy provides intuition about finding versus checking solutions, rather than a complexity-theoretic result about engineering work. A useful verifier has several properties:

The last property matters because a candidate need not be simply right or wrong. Reyes illustrates graded correctness with 30%, 70%, and 100%; these are examples of feedback signals, not evaluation results.

Software development already has a substantial verification infrastructure. Reyes attributes the progress of software agents partly to decades of investment in automated checks. Unit tests validate small pieces of behavior; end-to-end and QA tests exercise larger workflows. Browserbase and computer-use agents are examples of tools expanding what can be checked in complex visual and frontend changes.

Documentation and code quality provide additional checks. An OpenAPI specification makes an HTTP API’s interface explicit and gives tooling a description it can validate, although validating that description does not prove the implementation behaves correctly. Formatters and linters similarly turn some conventions into repeatable decisions.

The more demanding question is whether those checks express the team’s actual expectations. A linter that catches syntax problems may still accept code a senior engineer would reject. A test suite may pass while an agent introduces an unwanted implementation pattern. Agent readiness requires making more of those quality judgments executable. Reyes proposes a higher bar: checks that reject unwanted generated code and accept code meeting the team’s standards. This is a direction for improving validators, not a guarantee that linting can capture every senior engineer’s judgment.

People often compensate for incomplete automation without making that work visible. Reyes uses 50–60% test coverage, supplemented by manual testing, as an illustrative level an organization might tolerate. He also imagines a flaky build that fails every third run. An experienced developer may know when to retry, what to inspect manually, and which apparent success still deserves scrutiny. Those judgments let work continue even when the automated feedback is incomplete or noisy.

Reyes invokes organizations with more than 44,000 engineers to illustrate how tolerated validation gaps can persist at scale; he does not identify a measured organization. Such gaps need not prevent a human organization from functioning, but they constrain what agents can do across coding, review, documentation, and testing. Stronger validation gives agents more dependable evidence about whether their work is acceptable and gives the organization more confidence in using them.

Rigorous validation changes how a task can be organized. The familiar sequence is to understand the problem, design a solution, write code, and test it. With agents, the work becomes an explicit loop:

1. Specify the desired behavior and the constraints a solution must satisfy.
2. Generate a candidate implementation.
3. Verify it using automated checks and human judgment.
4. Iterate on the implementation or clarify the specification using what the checks reveal.

Human intuition remains part of verification. The shift is to make the acceptance conditions available before, and throughout, generation.

Specification and planning modes in coding tools support this workflow. Reyes points to Factory’s Droid and its Specification Mode, alongside other tools and IDEs organized around planning before implementation. These interfaces are useful when they help connect what should be built with how its correctness will be assessed.

That changes the procurement question. Reyes contrasts spending 45 days comparing tools for a hypothetical 10% advantage on SWE-bench with improving the practices that help every coding agent succeed. Those figures are rhetorical examples, not a reported tool evaluation. Shared validation improvements can benefit multiple tools, leaving teams room to choose tools their developers prefer.

Parallel agents make verification more consequential. Before assigning several agents work at once, an organization needs a way to assess whether each pull request achieves its purpose and is unlikely to break production. A large modernization project adds another requirement: decomposing the project into subtasks that can be executed and assessed separately. If the organization cannot reliably evaluate one change, generating many changes increases the unresolved review burden.

The basic unit is a task with a clear outcome, an intended approach, and a validation method. Reyes proposes nearly 100% success on such single tasks as a prerequisite for scaling to more complex agent workflows, not as an observed benchmark result. Decomposition and parallelism depend on that foundation.

Code review illustrates the same dependency from another direction. A review agent needs documentation of what the team expects. Agents may improve at finding tests, discovering lint commands, searching the repository, and working without explicit pointers. But finding an existing rule is different from supplying a rule the organization has never articulated. Better search cannot reliably recover missing acceptance criteria.

Developers therefore remain responsible for shaping the environment in which software is built. They set constraints, build automations, and progressively encode more of the organization’s engineering preferences. Much of this work can begin without buying another tool. Reyes mentions Factory’s assistance, assessment tools, and ROI analytics, but the underlying questions are direct: which checks exist, and how much useful judgment do they contain?

The readiness assessment returns to the talk’s eight verification pillars. Begin with concrete questions: is there a linter, and how strong are its rules? Are there instructions for agents? The broadly supported AGENTS.md format provides a place to make project guidance explicit. Merely having a file or a command is not the endpoint; the assessment concerns whether the agent can discover and use the practices that matter.

Differences between developers can reveal what is missing. If tool-usage visibility shows that senior developers get good results from an agent while junior developers struggle, investigate the practices experienced developers supply implicitly. A junior developer’s difficulty may expose a niche convention or missing validation rule, rather than a lack of skill with the tool. Systematically improving those checks makes the knowledge available beyond the people who already possess it.

Reyes illustrates the protective role of validation by contrasting Google or Meta with a still-large organization of roughly 2,000 engineers. In his example, a new graduate with little context can make YouTube’s boundary slightly rounder without taking the service down for roughly a billion users. The point is the infrastructure surrounding a small change: extensive checks can protect deployment even when its author lacks deep organizational context. Coding agents can now help identify gaps in that infrastructure and propose fixes.

An agent can inspect lint rules for places where the repository is insufficiently opinionated. It can also generate tests. Reyes quotes Factory engineer Alvin: “A slop test is better than no test.” His defense of that deliberately provocative advice has a condition: the initial test should pass when changes are correct and reasonably match the intended specification. It gives people something concrete to improve and future agents a pattern to follow.

For example, suppose a TypeScript project requires account names to be trimmed and lowercased, with empty names rejected. A small test can turn that previously informal rule into an explicit acceptance condition:

import assert from "node:assert/strict";
import test from "node:test";
import { normalizeAccountName } from "./account-name.js";

test("account names follow the project convention", () => {
  assert.equal(normalizeAccountName("  ALICE  "), "alice");
  assert.equal(normalizeAccountName("alice"), "alice");
  assert.throws(() => normalizeAccountName("   "));
});

This initial test does not settle every naming policy. It makes a few intended behaviors inspectable, so a developer or agent can extend the specification and its checks together. Existing tests also communicate how the repository expects new tests to be written.

That creates a reinforcing developer-experience loop. Better agents can improve the environment; the improved environment helps agents produce better work; the time saved can fund further improvements. The benefit extends across coding agents and review tools because they draw on the same repository standards and validation infrastructure.

The investment decision consequently expands beyond staffing. A request for ten more people is Reyes’s example of the familiar input to an engineering project. Investing in the environment offers another way to increase capacity: make the existing people, additional hires, and agents more effective at producing changes that can be trusted. Because agents can repeatedly apply those improvements, a better check can influence many subsequent tasks.

The agent still has to use the environment. A practical selection criterion is whether it proactively looks for linters, tests, and other validation criteria. Infrastructure outside the agent product matters only to the extent that the workflow takes advantage of it. Once an engineer’s preferences are expressed in reusable checks, an agent can apply them repeatedly; measurement and systematic improvement can extend that engineer’s influence across the organization.

The closing example makes the intended destination concrete:

1. A customer reports an issue.
2. A bug ticket is filed and picked up by a coding agent.
3. The agent implements a change and presents feedback to a developer.
4. The developer reviews the result and approves it.
5. The code is merged and deployed to production.

Reyes envisions this issue-to-production loop taking one to two hours; he does not report a measured deployment result. Although he describes an autonomous flow, the example explicitly retains human approval.

Reyes argues that this workflow was technically feasible at the time of the talk and that organizational validation, rather than agent capability, was the limiting factor. His prediction is that investment in these foundations could produce 5×–7× gains rather than 1.5×–2× gains, and put adopters in the top 1–5% for engineering velocity. These are forecasts, not measured productivity results. The organizational commitment is the substantive requirement: AI will not automatically supply missing engineering standards. Teams have to choose which outcomes they will accept, encode the checks that support those choices, and invest in making the resulting feedback dependable.