AI-Generated Code Is Already Competing With Human Code — Daksh Gupta, Greptile

An agent opening a hundred pull requests a day makes an impressive story. It also raises a practical question: can a company with real customers safely use those changes? Daksh Gupta, co-founder of Greptile, approaches that question with skepticism earned from programming before autonomous agents. Producing code quickly and producing code a business can merge are different accomplishments.

Greptile already sits between a proposed change and its acceptance. Its agents inspect changed files and related code to look for bugs. They can also install dependencies, start the application in a sandbox, mock inputs, and click through the running app. That position provides both code-review observations and a reason to care about whether the author is a person or an agent.

The shift in coding tools explains why the question has become urgent. Gupta traces a progression from tab completion, through multifile editing in 2024, to agents in 2025 that could take a task and create an entire pull request. Each step expands the unit of work delegated to software: first a continuation, then coordinated edits, then a proposed contribution ready for review. The remaining question is whether that contribution holds up in an existing enterprise codebase.

Greptile reviews more than a million pull requests a month across thousands of companies, usually enterprises or businesses with established products and customers. Before comparing quality, Gupta had to divide that stream into likely agent-written and human-written changes. The first attempt—looking for agent names in GitHub author fields—identified fewer than 1% of pull requests.

That result prompted a wider search for traces left by the tools:

• Author fields: An agent named as the author provides a direct metadata signal.
• Co-author footers: PR descriptions may credit Claude, Cursor, or another tool even when the author field does not.
• Branch prefixes: Agent-created branch names provide another indication that a tool participated in producing the pull request.

Consider the classification change implied by the footer check. A PR whose author field names no agent would escape the first detector. If its description credits Claude, the expanded detector now has evidence of agent involvement. Nothing about the code has changed; the observable change is in how the PR is classified. Gupta takes an agent writing the description or branch name as a reasonable sign that it also wrote much of the implementation. That remains an inference: these signals do not establish full autonomy, and an unmarked PR need not be entirely human-written.

With the combined signals, roughly a quarter of reviewed PRs appeared largely or entirely AI-generated. Looking back over the preceding twelve months, Gupta found fewer than 1% showing such evidence early in that period. He describes the adoption curve as continuous: individual model releases did not stand out as obvious discontinuities. The striking observation is the spread of the workflow through companies, rather than a single release-day jump.

Authorship answers who likely produced a change. Quality requires a separate definition. Gupta starts with reverts: a change that a team subsequently undoes is a useful candidate for a bad PR. He tracks GitHub’s revert branch naming pattern, which includes the original PR number and name, to connect an undo operation to its source change.

The reported rates are about one revert per thousand Codex PRs, about 3.5 per thousand Devin PRs, and about 2.5 per thousand human PRs. Humans sit between the two agents rather than clearly outperforming both. These are observational comparisons using reverts and automated review findings as quality proxies; they do not establish equivalent task difficulty or general superiority.

One plausible explanation was task allocation. People might give agents small, well-scoped changes while retaining complicated work themselves. More complicated PRs expose more opportunities for failure, so a raw revert comparison could penalize humans for doing harder jobs. Gupta therefore examines PR size alongside reverts. He reports that the size comparison did not provide strong evidence that human PRs were better. This addresses the proposed size explanation, although size alone cannot capture everything that makes a task difficult.

The next signal is the severity of issues Greptile flags. P0, P1, and P2 are the severity labels used in the reported review findings; the talk does not define their thresholds. Counting findings in these categories asks whether similar revert rates conceal different amounts of problematic code caught before acceptance. Three of the four tested agents produced fewer flagged P0 issues than humans. Across the severity categories, Gupta again reports broadly comparable quality.

Finally, review rounds measure the work needed to make a proposal mergeable. In the workflow described here, an agent reads review comments, addresses them, and commits a revision to the same branch. A higher-quality starting point should, in principle, need fewer cycles. Tracking the rounds between opening and merging a PR produced another comparison with little reported difference between human and agent contributions.

A similar total number of problems can hide a different mix of problems. Greptile makes about four comments per PR on average, giving Gupta several million comments across the preceding months. Searching that corpus for phrases such as SQL injection and N+1 query lets him compare specific failure categories across authors.

The comparison sets human frequency to 1× separately for each error category:

• SQL injection: Claude’s reported relative frequency is about 1.5× the human baseline.
• Authentication bypass: Devin’s reported relative frequency is about half the human baseline.

The ratios compare each agent with humans within a category. They do not say that SQL injection and authentication bypass occur equally often, or that one agent is uniformly safer. Because the analysis searches review-comment language, these are patterns in flagged findings rather than independently confirmed incident rates.

This changes the useful question from “Are agents bad at coding?” to which failures a particular tool tends to introduce. The aggregate comparisons weakened Gupta’s initial skepticism about enterprise use; the category comparisons preserve a reason to inspect the resulting code carefully. Agents can make meaningful contributions while retaining distinctive weaknesses.

Once agents can contribute useful code, the next constraint is the capacity to validate it. Gupta describes a large gap between typical Greptile users and the most prolific users, whose monthly output reaches hundreds or thousands of changes. Exact activity figures are uncertain because the recording and its description use different units and percentile figures. The operational concern is the same: manual review, testing, and external QA must keep pace with a much faster stream of proposed changes.

Greptile’s response starts with what must be established between a PR and a safe deployment. Gupta proposes three questions:

• Present behavior: Does the change violate the application’s user contract—the behavior users should be able to rely on?
• Future risk: Does the change make a later violation of that contract more likely?
• Author intent: Does the implementation accomplish what its author meant to change?

The questions separate three ways a change can disappoint. It can break existing behavior now, leave behind a risk that becomes a failure later, or preserve existing behavior while failing to deliver the requested improvement. This gives validation a purpose beyond accumulating review comments: establish whether the proposed change is safe and whether it does the intended job.

How can validation connect a changed file to what users actually experience? Greptile combines two avenues described across the talk. Code analysis follows the change into related files, extending inspection beyond the diff. Sandbox execution installs dependencies, starts the app, supplies mocked inputs, and lets browser agents interact with it. One avenue examines the implementation and its relationships; the other looks for failures in running behavior.

The flow below answers where a proposed change becomes evidence for a revision. Findings from code inspection and browser interaction become review comments. The coding agent can then address those comments and commit a new version to the PR branch. The observable change is a revised proposal, and further review determines whether more work is needed before merge.

Gupta expects this combination to discover many issues and increase confidence in merging. The talk supplies the approach rather than a measured coverage guarantee, and it does not spell out how each of the three validation questions is conclusively answered. Its closing adoption signal is that almost a fifth of the PRs Greptile reviews are merged without human review or human testing. Greptile wants that share to grow while maintaining code quality. The ambition is to make validation scale with code production, so generating a useful change does not simply move an unmanageable workload onto reviewers.