How Anthropic Merges 30,000 Code Changes a Day - Manish Kapur, Sonar & Addy Osmani, Anthropic

Almost everyone raises a hand when Sonar’s Manish Kapur asks whose coding agent opened a pull request in the past week. Far fewer keep their hands up when he asks who reviewed every line before merging. That gap sets up his conversation with Addy Osmani, who works on the developer experience for Claude Code at Anthropic: how can verification keep pace with software that agents produce?

Kapur introduces the delivery problem with two figures he attributes to CircleCI’s analysis of 28 million CI workflows: throughput rose about 59%, while three in ten attempts to merge code into production failed. What does faster output buy if changes still cannot get through delivery? The comparison separates the volume of work moving through a system from its ability to accept that work successfully.

Osmani’s phrase for a software factory is “loops at scale.” It shifts attention from an individual agent writing an implementation to the repeated work around it: planning, building, checking and shipping. Scaling that process requires deciding where people still exercise judgment and what evidence lets automation proceed.

The factory metaphor describes a spectrum of human involvement:

• Lights-on factory: Agents repeatedly help with planning, building, checking and shipping. People apply judgment and taste within that process.
• Dark factory: Agents scope, build and check work with much less intervention. People concentrate on intent and enter execution when exceptional cases require them.

The question is how much of the process a team can trust to run without continuous human attention.

Kapur provocatively calls AI coding a solved problem, then identifies the harder constraint: 76% of developers generating AI code say it arrives at a speed they cannot verify. He also cites coding-agent adoption at 85% of organizations. These are adoption and reported-capacity claims; the practical tension is already clear without treating code generation as universally correct.

Confidence needs something the agent can check. Tests, benchmarks and specifications give it criteria for judging whether its implementation is correct and complete. Agents can help create those checks more cheaply, but teams still need to refine them. A task with a usable verification method lets the agent inspect its own result and correct failures before handing the work back.

Kapur brings the scale into focus: Anthropic has grown from roughly 2,500 merged code changes a day to 30,000. Osmani places the earlier figure in May. He also describes a recent refresh of claude.ai involving about 3,000 pull requests from engineers, product designers and other functions, with a reported threefold performance improvement. These are distinct quantities: daily merged changes across Anthropic and pull requests for a particular refresh. The performance claim does not specify a metric here.

Claude performs a first-pass review, with tools that let it try the result like a user. For a browser experience, that can include opening the application, exercising it through end-to-end tests, running linting and using other necessary checks. Review therefore includes observing behavior, rather than relying solely on a plausible reading of the diff.

When review exposes a bug or missing functionality, edits trigger hooks that study the diffs and support rapid correction of the PR. Osmani describes Anthropic using its own code-review tooling to automate as much of this work as possible. The useful feedback is tied to changed code: an edit becomes another opportunity to inspect whether the correction closes the gap.

Team judgment also needs a form the reviewer can consult. Shared files such as CLAUDE.md and AGENTS.md can record best practices and the team’s expectations. Those instructions supply review criteria; people remain responsible for checking the agent’s evidence. For a UI change, Osmani wants browser testing and perhaps a video showing the agent clicking through the experience. If mobile and desktop both matter, screenshots or videos should cover those experiences.

How do team standards, executable checks and human inspection connect? The diagram separates the information supplied to review from the evidence returned to a person. A browser recording makes the exercised interaction inspectable; linting supplies a different signal. Neither replaces the team’s decision about whether the evidence supports the intended experience.

Sonar’s forthcoming State of Code survey, described here as covering more than 1,000 developers, returns to the capacity problem. Kapur reports that 84% say AI gets features and fixes into production faster, alongside larger and more complex pull requests and more code reaching production without review. What does the survey slide reveal about those changes occurring together? Faster delivery can increase the amount of material that verification must handle, especially in regulated or mission-critical applications.

Osmani expects further gains from both model quality and better verification practices. But “verify the work” leaves the implementer with another question: which checks actually establish confidence? He names end-to-end tests, property tests and visual verification tests as checks Anthropic relies on. They offer different ways to examine behavior, and the choice matters when the goal is eventually to merge changes automatically.

Kapur reports software-factory use at 5% of companies, with a projected rise to 34%, or 39% in financial services. A question from the room immediately exposes a complication: what counts as a software factory? Osmani returns to repeated loops and the degree of human involvement. Kapur describes an assembly line beginning with intent and specifications, then automating development stages at scale. The percentages depend on that definition; they do not establish that those organizations already operate unattended systems.

The path toward a darker factory depends on the agent generating verification that earns confidence. Human involvement can shrink as that evidence becomes dependable. Most teams in this discussion still occupy the spectrum between assisted execution and autonomous operation.

Sonar’s proposed approach puts verification inside each kind of recurring work: the agent’s development loop, the CI verification loop and ongoing code maintenance. These are places to apply checks throughout the system, rather than a requirement to defer all judgment until a final review.

Kapur divides the checks by the failures they target:

• Deterministic analysis: Checks for bugs and security issues, including SQL injection and buffer overflows.
• Agentic review: Examines logic and intent issues that deterministic analysis is less suited to identify.
• Software supply-chain security: Looks for known vulnerabilities, malware and malicious packages.

The layers complement one another because each examines a different class of problem. A favorable agent review does not settle whether a dependency is malicious, just as a deterministic check does not establish that an implementation follows the intended product behavior.

Verification of code does not answer every operational question. Osmani describes companies that allowed teams to experiment with agents, then discovered they had hundreds or thousands of them without a clear view of their activity. Those agents could touch systems, access data and potentially deploy to production. Agent identity and telemetry become necessary for understanding which agent acted and what happened.

Three concerns follow: control, permissions and the sandbox. Permissions determine what an agent may access or do; the sandbox must keep execution within its intended bounds. Kapur calls Sonar’s approach “guide rails”: directing agents along an allowed path and preventing them from going out of bounds. These controls constrain actions while review and tests examine the resulting software.

The lightning round begins with a request for one-sentence answers. “Can these be long sentences?” Osmani asks. His forecast for most teams by the end of 2027 is similarly qualified: “light factories with some dark corners.” He expects pockets of autonomy rather than everyone operating an entirely dark factory.

Both participants favor labeling agent-written PRs. For Osmani, labeling Claude as the author is compatible with caring about the outcome: whether the shipped code and features are right for users. The label can also help identify additional constraints or checks. Kapur favors keeping labels for at least the foreseeable six to nine months while confidence develops, and points to auditability for financial customers.

Authorship and ownership remain separate records. Anthropic still labels who owns a change. The person who starts a Claude run stays accountable for the result and must fix it if something goes wrong. Naming the agent explains where the implementation came from; naming the owner tells the organization who will answer for it.

The skill to build depends on the role:

• Engineering leaders: Identify the gap between current testing and the evidence that would make the team comfortable letting an agent sign off on a merge.
• Junior developers: Gain experience with different kinds of tests, including end-to-end and property tests, so verification becomes something concrete they can design and assess.

When is an agent-opened PR safe to merge? Osmani’s answer is confidence that things will not go wrong, or that the risks have been mitigated as much as possible. That is a judgment about the work and its consequences, rather than a universal test-count threshold. It brings the discussion back to the evidence people need before they accept a change.

Kapur closes with “Fast to write is not fast to ship.” Teams need to look at what actually gets merged, verify the output and keep agents within their sandbox before changes reach production. His prescription is independent automated verification inside every factory loop. Autonomy is earned by making the output trustworthy enough to operate with less intervention.