Rebuilding CI for AI-Scale Commit Volume — Gautam Korlam, Sonar

Uber had “agent-scale commit volume” before coding agents: thousands of daily changes entering a giant monorepo with long builds. Gautam Korlam, formerly responsible for build and merge infrastructure at Uber and later a Gitar co-founder who joined Sonar, opens with the resulting dilemma. Serializing every merge protected the main branch, but he describes a backlog that could take twenty days to clear. Allowing everyone to merge independently made the iOS monorepo frequently red. A change could pass CI on its own and still fail when combined with another change.

SubmitQueue addressed the interaction between changes. It tested likely combinations of incoming commits in parallel, rather than waiting to test and merge each change serially. A PR implicated in a failing combination could be demoted from the queue; its author would fix it, rebase and resubmit. Changes reached trunk only after passing the relevant combination tests. The useful question for the queue illustration is therefore: which combined code state was tested before a change became eligible to merge?

This spends extra testing work to reduce waiting while protecting trunk. Testing likely combinations also matters: the account does not establish an exhaustive search over every possible subset of commits. The mechanism is speculative testing of combinations that may land, followed by admission or removal based on those results. Korlam says SubmitQueue remained in use at Uber at the time of the recording.

Merge queues solve a correctness problem, but their capacity still matters. CI and review systems built around human working rhythms now receive bursts of PRs from developers running agents and from background workflows. Those producers do not necessarily wait for the previous change to clear. More code enters the system while testing and review remain centralized, so work accumulates at those stages.

Korlam reports that Gitar’s own commit rate had increased almost tenfold over roughly two years with the same small team. He also describes approximately 25% month-over-month growth in the PR volume Gitar handled for review, CI fixes and related work. These are observations from his team and platform; they do not separate increased activity by existing users from customer growth, or establish a corresponding increase in delivered value.

The proposed “software factory”—agents implement specifications while humans handle specifications and review—still depends on several constrained services:

• Build and test: Compilation failures require rebuilds. Integration failures and tests that cannot be reproduced locally can demand repeated CI runs, sometimes taking hours.
• Review and approval: Without a useful automated first pass, human reviewers become the queue. Heavy volume also creates pressure to rubber-stamp changes rather than examine them.
• Decision records: Logs expire, while an approval stamp may remain without explaining what was actually checked. That makes later auditing difficult even if the change originally followed an acceptable process.

On Gitar’s platform, at least 13% of PRs had one or more failures in required CI checks. Each affected PR needed another round trip: a developer or agent reads the logs, identifies the cause, changes the code, pushes a commit and waits for CI again. A failure percentage therefore understates the operational work if it is read merely as a count of red checks. The expensive unit is the complete repair attempt.

Review can make those delays serial. Human reviewers often wait for green CI before examining a PR, so they begin only after the initial repair work finishes. Their comments can then cause another edit that breaks CI again. Faster code generation does little to shorten that path if each correction still waits through a long build before the next decision.

Several interventions act on different parts of that cost:

• Shorten CI runs: Parallel execution, sharding and caching reduce the wait for each repair attempt. Korlam also proposes moving some expensive checks after merge; doing so speeds the merge path while leaving those checks to detect problems later.
• Return review signals sooner: Multiple review passes can give quick feedback first, with deeper scrutiny reserved for higher-risk changes. Lower-risk changes may qualify for automated approval and merge.
• Run suitable checks earlier: Moving a check into the local development process can make its results cheaper to reuse and place the failure beside the agent’s existing task context. Some regression or production checks cannot move there, so the opportunity depends on the check.

Korlam illustrates the limits of technical checks with an account from Uber: a feature-flag change allowed drivers onto the platform without the proper checks, contributing, in his telling, to the loss of its London operating license. The response required tracing changes that affected London and identifying the controls applied to them. He connects that process to winning the license back.

The mechanism begins with a question that compilation cannot answer: does this change affect an activity subject to a particular rule? If it does, the development process needs to apply the corresponding controls and record their enforcement. A passing build alone cannot establish that an agent followed a regulatory requirement. Policy enforcement and its evidence must accompany the code through the merge process.

Gitar’s own CI initially took around ten minutes. For a team pushing more changes, that was fast enough to function and slow enough to become irritating. Its Rust build work focused first on caching outputs and reusing them, then on reducing compiler churn so that compilation results were invalidated less often. Sharding and removing other overheads contributed additional reductions.

The reported result was CI taking less than a minute for incremental steps after the cold run had warmed the system. That condition is essential: the claim concerns subsequent work that can reuse prior outputs, rather than every clean build. Caching helps when unchanged work remains reusable; reducing unnecessary invalidation preserves that advantage across edits.

Faster CI then exposes the next wait. At Uber, Korlam recalls that review accounted for the vast majority of the time in the P90 measure of PRs reaching production. Automated review still creates findings that someone must resolve. An agent can receive that context locally or repair the PR in CI, leaving humans to examine what remains after the automated first pass. Counting review comments without arranging their resolution would leave this bottleneck in place.

The CI optimization itself produced a useful review example. Gitar was adopting nextest to run tests in parallel. Database tests reused a shared database, avoiding the cost of setting it up repeatedly. But missing guards allowed cross-contamination when CI became busy: concurrent tests could interfere through shared database state and become flaky. The intended saving in setup work introduced a risk under increased concurrency.

What did the review catch before this reached main? Korlam reports that Gitar identified the problem on the PR, suggested a correction, committed it and added the engineer as a co-author. The account does not specify the guard or database-isolation technique, so it supports the failure mechanism and the pre-merge intervention without supplying an implementation recipe. The example makes the review’s value concrete: it caught a concurrency hazard in a change whose purpose was to make tests faster.

Approval is another place to avoid sending every change through the same scarce human queue. Gitar supports built-in criteria and additional plain-English rules that gate approval and merge. Korlam’s candidates for automation include documentation, routine dependency updates that do not break APIs, and low-risk internal tooling changes. Eligibility depends on the change meeting the policy; a dependency label alone does not establish low risk.

The approval example asks a concrete question: which branch should this PR take?

• Meets the low-risk criteria: Documentation or a qualifying dependency change can receive automated approval.
• Touches sensitive code: Changes involving payments or authentication may require an additional human approver.
• Changes beyond the criteria: A new edit that no longer qualifies must escalate to another approver.

Across those cases, the agent that authored the change must not approve its own work. Automatic approval still needs a separation between producing the change and accepting it.

A branch name does not permanently identify the code that passed CI. In Korlam’s next example, CI passes, a rerun encounters a flaky failure, and a force-pushed repair moves the branch to different code. Over time, CI artifacts also expire. Without a record of the tested commit, the failure and its resolution, a later auditor cannot reconstruct what was accepted merely by looking at the branch’s current state.

Agent trajectories and outcomes after each push provide part of that history. Stored on the PR or another lasting system of record, they can preserve what happened after temporary logs disappear. The control Korlam emphasizes is independent testing and approval: the record should establish who accepted the change and that the acceptance did not come from its author.

The pipeline comparison asks why a failed run with a documented exception can be more acceptable than a passing run with an incomplete history. In the first case, an independent approver recognizes a known failure mode, records why it is acceptable and saves that decision with the change that lands. In the other, a green result lacks the recorded checks or reasoning needed to justify acceptance. This is a distinction about evidence and policy: a documented exception must explain the failure and the reason for permitting the merge.

The ending turns auditability into specific records a team can retain. Gitar’s examples cover several different questions:

• Origin: Which coding agent produced the PR, which session created it and who initiated it? Was the trigger an automated workflow or an engineer’s prompt?
• Test evidence: Which commit received the trusted CI result? Preserve the test report alongside the status.
• Finding resolution: Was each finding fixed, dismissed or left for follow-up?
• Approval independence: Who approved the change, and how were they separate from the authoring bot and its initiating context?
• Overrides: If a red build was accepted or a coverage check skipped, name the exception and explain why it was acceptable.

Those records must describe the commit that merges. A small edit on top of an approved version still changes the code being accepted, so Korlam calls for checks to run again on each change. Recording who prompted an agent also remains useful alongside commit authorship: the origin of the work and the identity permitted to approve it are separate facts that the system needs to retain.

The proposed software factory gains capacity at several points: faster CI shortens repair waits; suitable local checks reduce downstream test load; automated review and low-risk approval reduce the work reaching humans. After every approval and merge, the system records what happened. That history serves both accountability and future work: as more changes move through the factory, earlier failures, fixes and exceptions remain available as context rather than disappearing with the agent session or CI logs.