The State of AI Code Quality: Hype vs. Reality

Three cloud outages in a few weeks, at companies publicly embracing AI-generated code: are the two related? Itamar Friedman, Qodo’s co-founder and CEO, opens with that question and immediately admits he does not know. The companies’ stated AI contributions—10%, 30%, or 50% of code—establish the scale of adoption, not the cause of an outage. Qodo’s name stands for quality of development; the problem here is how to retain that quality while code becomes easier to produce.

Friedman describes roughly 60% of developers saying a quarter of their code is generated or shaped by AI, and 15% saying the share exceeds 80%. The published Qodo report uses a different first figure: 65% report at least a quarter of their code being AI-influenced. Either way, AI is participating in enough development that checking its output becomes a central engineering task.

That leads to a more concrete example: the Claude Code security-review prompt. It assigns the reviewer a senior security engineer role, then excludes denial-of-service issues. The public prompt currently retains that exclusion within a deliberately scoped review intended to emphasize actionable findings and reduce false positives; it does not establish the exact historical command version shown here.

The exclusion makes the operational lesson clear: a convincing reviewer persona does not define complete review coverage. A tool can follow its instructions and still leave a category of risk unchecked. Friedman jokes about a possible connection to the outages, but supplies no causal evidence. His practical demand is rigor: teams need to know what their quality checks cover, what they exclude, and where the remaining checks happen.

The same gap appears in Cursor and Copilot rules. Developers invest in instructions because better guidance improves generation. But when Friedman asks whether those rules are completely followed, the responses tend toward partial compliance. There are two questions to answer: did the generator follow the rule, and did it follow the rule deeply enough to satisfy the intended technical standard? Merely having a rules file answers neither.

To examine the gap, Friedman draws on reports from Qodo, Sonar, and Faros. He describes evidence spanning thousands of developers, millions of pull requests, and about a billion lines of code, without assigning each population to a particular report. These are different kinds of evidence: developer surveys describe perceptions and reported behavior, while repository analysis and engineering telemetry describe observed code or workflow activity. They should not be treated as a single experiment.

Sonar contributes a useful perspective because its checks predate AI coding and address software quality beyond AI-specific concerns. Large-scale inspection can reveal security, reliability, and maintainability problems regardless of who—or what—wrote the code. The next step is to break quality into dimensions that teams can actually inspect and improve.

Autocomplete improves the act of writing code. Agentic generation—Friedman’s Gen 2.0—can perform larger pieces of development work and raises that productivity ceiling. Both benefit from investment in instructions and rules. But faster generation alone eventually runs into work elsewhere in the development process.

The next layer is agentic quality workflows: AI operating across review, testing, and other quality processes, including those outside the IDE. Friedman emphasizes organizations with roughly a hundred developers or more, where many such workflows already exist. His final layer adds learning: standards, rules, and workflows must change as the organization learns what quality requires. The graph is a conceptual progression from better generation to an improving quality system, not a measured productivity curve.

The distinction matters when discussing promised 2× or 10× gains. Friedman invokes McKinsey and Stanford as part of the surrounding productivity hype, but the target he cares about is the entire software development lifecycle. Multiplying code-writing speed does not establish the same multiplier for delivering and operating software.

Adoption is already broad. Friedman cites one report putting daily or weekly AI developer-tool use at 82%. He then gives tool-count figures of 59% and 20%, describing them as more than three and more than five tools; the published Qodo report defines those thresholds as three or more and five or more. Cursor, Copilot, Codex, Claude Code, and Lovable all contribute to this expanding set of code-producing tools. His prediction is that developers will reach ten such tools within two or three years.

Friedman attributes half of usage to teams with fewer than ten developers, then describes adoption spreading into enterprises. His broader cross-report summary places weekly-to-monthly use between 82% and 92%; that combines different reporting windows rather than measuring one uniform adoption rate.

In some cases, Friedman reports a 3× improvement in code-writing productivity. Yet 67% of the developers he says were asked expressed serious concerns about AI-generated or AI-influenced code. The missing piece is not necessarily another generator: respondents also describe lacking a framework for defining and measuring quality.

The throughput figures expose the next bottleneck. Friedman summarizes reports as about 20% more completed tasks, 97% more opened PRs, and 90% more review time. The published Faros analysis makes several consequential distinctions:

These are observational comparisons between teams’ periods of lower and higher AI adoption. Merged PRs are not opened PRs, and longer elapsed review time does not necessarily mean an equivalent increase in reviewer labor. The findings nevertheless locate a practical constraint: producing changes faster can increase the queue awaiting review.

Even if bugs per line remain unchanged, more code means more total bugs. Friedman explicitly does not need to claim a higher defect rate for that argument to work:

His personal illustration is generating roughly a thousand lines with Claude Code in five minutes, compared with previously spending hours writing ten careful lines. The reviewer still has to understand the behavior and consequences of the larger change.

Generation is especially powerful for greenfield projects and proofs of concept. Once software serves millions of clients, moves money, or supports transportation, the obligations expand. Code integrity, governance, review standards, testing, and reliability remain necessary even when producing the implementation becomes cheap.

One way to map quality is by lifecycle stage: planning, development, review, testing, and deployment. Each stage introduces different questions, and checking quality is only one part of the review process. This is a useful starting inventory, not an exhaustive lifecycle model.

A second dimension separates defects in the artifact from weaknesses in how the organization handles it:

• Code-level quality: Security and inefficiency matter even when a program produces the expected functional result.
• Process-level quality: Teams must learn and own the code, verify it, handle porting, and maintain guardrails and standards. If AI-generated code contributes to an outage, assigning responsibility to the AI does not replace an accountable team.

These dimensions overlap: a process can prevent a code defect, while investigating a defect can reveal a missing process.

Friedman reports 42% more development time spent resolving issues and bugs, alongside a 35% project-delay figure. He acknowledges that the questions framed the discussion around quality problems and their impact, which can bias responses. He also cites reports of threefold security incidents and connects that to threefold code-writing output: his explanation is a volume-and-correlation argument, not evidence that each generated line is three times as dangerous.

Testing is the first intervention. Friedman describes heavy AI use in testing as doubling trust in AI-generated code. The published Qodo finding is narrower: confidence in tests is 61% versus 27% for the compared groups. That is a self-reported association, but it points toward a useful workflow: apply AI to checking behavior as well as producing implementations.

Code review can also enforce process requirements. For example, a review workflow can block a PR whose test coverage falls below a specified threshold. The gate connects a requirement about testing to the decision about whether code may advance. This is why review can address both code-level and process-level concerns: it inspects the change while enforcing conditions on how the change was prepared.

Friedman reports double the quality gain among AI-review users and a 47% improvement in code-writing productivity. The published report’s doubled-quality comparison is specifically among respondents whose productivity stayed unchanged: 36% of AI-review users reported quality improvement, versus 17% of nonusers. That subgroup result should not be read as a general doubling of code quality.

Qodo scans about a million PRs a month, Friedman says, and 17% of a million-PR sample contained high-severity issues. The published report corroborates the 17% figure and defines high severity as scores of 9–10, but does not establish that sample size. Friedman also says the before-and-after AI comparison is not yet available: most customers already use generated code, so historical scanning would be needed to establish a baseline. The observed issue prevalence therefore does not show how much AI changed it.

Both generation and review depend on the context supplied to the model. A reviewer that sees a diff but lacks the surrounding code, team norms, or project structure may miss why an apparently reasonable change is wrong for this system. Friedman describes distrust of the model’s context as a major source of distrust in its output.

The figures here differ across the spoken presentation, slide, and published report. Friedman says context is distrusted 80% of the time and receives 33% of requested-improvement choices; the displayed slide shows 88% and 33%, while the published Qodo report assigns context 26% of improvement votes. The consistent engineering concern is the scope of what the model knows, rather than a single interchangeable percentage.

Friedman calls context a central investment at Qodo. When its context engine is connected, he says approximately 60% of generation and review tools’ MCP calls go to a context MCP. He also describes 8% of review context usage as coming from standards and best-practice files. The published report’s 8% instead refers to review suggestions about best practices; it is not a measurement of context-file usage. The distinction does not change the design requirement: organizational standards need to be available alongside source code.

Friedman cites Jensen Huang presenting Qodo’s context-engine architecture at a GTC keynote and says NVIDIA examined that engine. His emphasis is on the infrastructure underlying generation, review, and testing, rather than any one of those interfaces. Whether a team builds or buys that infrastructure, it needs sustained investment. Useful context includes code, version history, PR history, and organizational logs—not just the latest branch. Those records preserve decisions and constraints that the current source tree alone may not explain.

Automated quality gates can use the same parallel and background-agent capabilities now applied to generation. Friedman pairs those gates with intelligent code review, testing, and living documentation. Documentation matters here as an input that must remain useful as the system changes, not simply as another artifact to generate once.

The proposed architecture centers on two artifacts—specification and code—and the infrastructure needed to improve and validate them:

The sandbox is essential to the execution side of this picture: agents need an environment where they can perform checks, not merely produce commentary about code.

Quality is the competitive advantage; AI is one tool for producing it. That shifts the scope from code generation to the whole software and product development lifecycle. Friedman points to reports of reduced vulnerabilities and faster review, while acknowledging the additional burden generated code has introduced. He also says test coverage can triple in a month, depending on the project. These are reasons to invest in the surrounding workflow, not a return guaranteed by purchasing a tool.

The closing Qodo demonstration makes the architecture concrete with a small rule: “I don’t like nested ifs.” It resembles an instruction a developer might already place in Cursor. Qodo then uses project context to build good and bad examples, constructs a workflow to detect the issue, and tracks acceptance over time so the team can inspect and adjust the standard.

For a concrete TypeScript illustration, consider a rule that prefers guard clauses over nested conditionals. The proposed transformation preserves the return values while flattening the control flow:

type Account = {
  active: boolean;
  canDeploy: boolean;
};

// Before review
function mayDeployBefore(account: Account | null): boolean {
  if (account !== null) {
    if (account.active) {
      if (account.canDeploy) {
        return true;
      }
    }
  }
  return false;
}

// Proposed review suggestion
function mayDeployAfter(account: Account | null): boolean {
  if (account === null) return false;
  if (!account.active) return false;
  return account.canDeploy;
}

The important event is the PR review: the generator may have produced nested conditionals despite already having a rule against them. A separate review workflow detects the violation and proposes a change using the contextual good and bad examples. The suggestion still awaits a decision; detecting a violation and applying a fix are different operations.

The demonstrated feedback loop continues beyond that suggestion:

1. Run checks for individual rules, including through the CLI.
2. Surface the nested-if finding and the associated suggestion; Friedman also describes a graph for inspecting the results.
3. Record whether the developer accepts or rejects the suggestion.
4. Use those responses to adapt the standard and its enforcement.

Qodo can also propose standards learned from the project, so each rule need not begin as a manually written preference. This closes the gap raised earlier: instead of assuming that a rules file is followed, the team gets detection, visible responses, and a way to revise the rule.

The progression is from generating code, to delegating generation, to applying AI across the lifecycle—including the work that determines whether a change should ship. Friedman’s closing 2× return is an aspiration contingent on that investment. The useful system is not the tool alone, but the quality workflow the team builds, measures, and keeps improving around it.