AI Engineer Code 2025
The State of AI Code Quality: Hype vs. Reality
Read the talk
AI Code Quality Beyond the Generation Bottleneck
Faster code generation shifts work into review, testing, and ownership. Itamar Friedman examines the evidence and demonstrates how coding rules can become measurable, adaptive quality workflows.
From a talk by Itamar Friedman
Before you start: Familiarity with pull requests, automated tests, and AI coding assistants will help; MCP is a protocol through which AI tools can access external context and capabilities.
What does a security review leave out?
Three cloud outages in a few weeks, at companies publicly embracing AI-generated code: are the two related? Itamar Friedman, Qodo’s co-founder and CEO, opens with that question and immediately admits he does not know. The companies’ stated AI contributions—10%, 30%, or 50% of code—establish the scale of adoption, not the cause of an outage. Qodo’s name stands for quality of development; the problem here is how to retain that quality while code becomes easier to produce.
Friedman describes roughly 60% of developers saying a quarter of their code is generated or shaped by AI, and 15% saying the share exceeds 80%. The published Qodo report uses a different first figure: 65% report at least a quarter of their code being AI-influenced. Either way, AI is participating in enough development that checking its output becomes a central engineering task.
That leads to a more concrete example: the Claude Code security-review prompt. It assigns the reviewer a senior security engineer role, then excludes denial-of-service issues. The public prompt currently retains that exclusion within a deliberately scoped review intended to emphasize actionable findings and reduce false positives; it does not establish the exact historical command version shown here.
The exclusion makes the operational lesson clear: a convincing reviewer persona does not define complete review coverage. A tool can follow its instructions and still leave a category of risk unchecked. Friedman jokes about a possible connection to the outages, but supplies no causal evidence. His practical demand is rigor: teams need to know what their quality checks cover, what they exclude, and where the remaining checks happen.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A rule is an instruction, not a guarantee
The same gap appears in Cursor and Copilot rules. Developers invest in instructions because better guidance improves generation. But when Friedman asks whether those rules are completely followed, the responses tend toward partial compliance. There are two questions to answer: did the generator follow the rule, and did it follow the rule deeply enough to satisfy the intended technical standard? Merely having a rules file answers neither.
To examine the gap, Friedman draws on reports from Qodo, Sonar, and Faros. He describes evidence spanning thousands of developers, millions of pull requests, and about a billion lines of code, without assigning each population to a particular report. These are different kinds of evidence: developer surveys describe perceptions and reported behavior, while repository analysis and engineering telemetry describe observed code or workflow activity. They should not be treated as a single experiment.
Sonar contributes a useful perspective because its checks predate AI coding and address software quality beyond AI-specific concerns. Large-scale inspection can reveal security, reliability, and maintainability problems regardless of who—or what—wrote the code. The next step is to break quality into dimensions that teams can actually inspect and improve.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Where generation stops improving the whole system
Autocomplete improves the act of writing code. Agentic generation—Friedman’s Gen 2.0—can perform larger pieces of development work and raises that productivity ceiling. Both benefit from investment in instructions and rules. But faster generation alone eventually runs into work elsewhere in the development process.
The next layer is agentic quality workflows: AI operating across review, testing, and other quality processes, including those outside the IDE. Friedman emphasizes organizations with roughly a hundred developers or more, where many such workflows already exist. His final layer adds learning: standards, rules, and workflows must change as the organization learns what quality requires. The graph is a conceptual progression from better generation to an improving quality system, not a measured productivity curve.
The distinction matters when discussing promised 2× or 10× gains. Friedman invokes McKinsey and Stanford as part of the surrounding productivity hype, but the target he cares about is the entire software development lifecycle. Multiplying code-writing speed does not establish the same multiplier for delivering and operating software.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
More tools, more output, unresolved quality
Adoption is already broad. Friedman cites one report putting daily or weekly AI developer-tool use at 82%. He then gives tool-count figures of 59% and 20%, describing them as more than three and more than five tools; the published Qodo report defines those thresholds as three or more and five or more. Cursor, Copilot, Codex, Claude Code, and Lovable all contribute to this expanding set of code-producing tools. His prediction is that developers will reach ten such tools within two or three years.
Friedman attributes half of usage to teams with fewer than ten developers, then describes adoption spreading into enterprises. His broader cross-report summary places weekly-to-monthly use between 82% and 92%; that combines different reporting windows rather than measuring one uniform adoption rate.
In some cases, Friedman reports a 3× improvement in code-writing productivity. Yet 67% of the developers he says were asked expressed serious concerns about AI-generated or AI-influenced code. The missing piece is not necessarily another generator: respondents also describe lacking a framework for defining and measuring quality.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The work moves into review
The throughput figures expose the next bottleneck. Friedman summarizes reports as about 20% more completed tasks, 97% more opened PRs, and 90% more review time. The published Faros analysis makes several consequential distinctions:
| Measure | Published Faros finding |
|---|---|
| Completed tasks | 21% more |
| Merged PRs | 98% more |
| PR review time | 91% longer |
These are observational comparisons between teams’ periods of lower and higher AI adoption. Merged PRs are not opened PRs, and longer elapsed review time does not necessarily mean an equivalent increase in reviewer labor. The findings nevertheless locate a practical constraint: producing changes faster can increase the queue awaiting review.
Even if bugs per line remain unchanged, more code means more total bugs. Friedman explicitly does not need to claim a higher defect rate for that argument to work:
His personal illustration is generating roughly a thousand lines with Claude Code in five minutes, compared with previously spending hours writing ten careful lines. The reviewer still has to understand the behavior and consequences of the larger change.
Generation is especially powerful for greenfield projects and proofs of concept. Once software serves millions of clients, moves money, or supports transportation, the obligations expand. Code integrity, governance, review standards, testing, and reliability remain necessary even when producing the implementation becomes cheap.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Quality belongs to code and to the process around it
One way to map quality is by lifecycle stage: planning, development, review, testing, and deployment. Each stage introduces different questions, and checking quality is only one part of the review process. This is a useful starting inventory, not an exhaustive lifecycle model.
A second dimension separates defects in the artifact from weaknesses in how the organization handles it:
- Code-level quality: Security and inefficiency matter even when a program produces the expected functional result.
- Process-level quality: Teams must learn and own the code, verify it, handle porting, and maintain guardrails and standards. If AI-generated code contributes to an outage, assigning responsibility to the AI does not replace an accountable team.
These dimensions overlap: a process can prevent a code defect, while investigating a defect can reveal a missing process.
Friedman reports 42% more development time spent resolving issues and bugs, alongside a 35% project-delay figure. He acknowledges that the questions framed the discussion around quality problems and their impact, which can bias responses. He also cites reports of threefold security incidents and connects that to threefold code-writing output: his explanation is a volume-and-correlation argument, not evidence that each generated line is three times as dangerous.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use testing and review to enforce requirements
Testing is the first intervention. Friedman describes heavy AI use in testing as doubling trust in AI-generated code. The published Qodo finding is narrower: confidence in tests is 61% versus 27% for the compared groups. That is a self-reported association, but it points toward a useful workflow: apply AI to checking behavior as well as producing implementations.
Code review can also enforce process requirements. For example, a review workflow can block a PR whose test coverage falls below a specified threshold. The gate connects a requirement about testing to the decision about whether code may advance. This is why review can address both code-level and process-level concerns: it inspects the change while enforcing conditions on how the change was prepared.
Friedman reports double the quality gain among AI-review users and a 47% improvement in code-writing productivity. The published report’s doubled-quality comparison is specifically among respondents whose productivity stayed unchanged: 36% of AI-review users reported quality improvement, versus 17% of nonusers. That subgroup result should not be read as a general doubling of code quality.
Qodo scans about a million PRs a month, Friedman says, and 17% of a million-PR sample contained high-severity issues. The published report corroborates the 17% figure and defines high severity as scores of 9–10, but does not establish that sample size. Friedman also says the before-and-after AI comparison is not yet available: most customers already use generated code, so historical scanning would be needed to establish a baseline. The observed issue prevalence therefore does not show how much AI changed it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Context includes the reasons behind the code
Both generation and review depend on the context supplied to the model. A reviewer that sees a diff but lacks the surrounding code, team norms, or project structure may miss why an apparently reasonable change is wrong for this system. Friedman describes distrust of the model’s context as a major source of distrust in its output.
The figures here differ across the spoken presentation, slide, and published report. Friedman says context is distrusted 80% of the time and receives 33% of requested-improvement choices; the displayed slide shows 88% and 33%, while the published Qodo report assigns context 26% of improvement votes. The consistent engineering concern is the scope of what the model knows, rather than a single interchangeable percentage.
Friedman calls context a central investment at Qodo. When its context engine is connected, he says approximately 60% of generation and review tools’ MCP calls go to a context MCP. He also describes 8% of review context usage as coming from standards and best-practice files. The published report’s 8% instead refers to review suggestions about best practices; it is not a measurement of context-file usage. The distinction does not change the design requirement: organizational standards need to be available alongside source code.
Friedman cites Jensen Huang presenting Qodo’s context-engine architecture at a GTC keynote and says NVIDIA examined that engine. His emphasis is on the infrastructure underlying generation, review, and testing, rather than any one of those interfaces. Whether a team builds or buys that infrastructure, it needs sustained investment. Useful context includes code, version history, PR history, and organizational logs—not just the latest branch. Those records preserve decisions and constraints that the current source tree alone may not explain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give quality agents artifacts, tools, and safe execution
Automated quality gates can use the same parallel and background-agent capabilities now applied to generation. Friedman pairs those gates with intelligent code review, testing, and living documentation. Documentation matters here as an input that must remain useful as the system changes, not simply as another artifact to generate once.
The proposed architecture centers on two artifacts—specification and code—and the infrastructure needed to improve and validate them:
| Component | Role |
|---|---|
| Specification and code | Describe intent and implement behavior |
| Parallel agents | Improve both and translate specifications into code |
| Tests | Express executable specifications |
| Context engine | Supply a software-development database |
| Quality and verification tools | Expose checks, especially through MCP |
| Stable, secured sandboxes | Run agents and validation workflows |
The sandbox is essential to the execution side of this picture: agents need an environment where they can perform checks, not merely produce commentary about code.
Quality is the competitive advantage; AI is one tool for producing it. That shifts the scope from code generation to the whole software and product development lifecycle. Friedman points to reports of reduced vulnerabilities and faster review, while acknowledging the additional burden generated code has introduced. He also says test coverage can triple in a month, depending on the project. These are reasons to invest in the surrounding workflow, not a return guaranteed by purchasing a tool.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn a nested-if preference into a learning workflow
The closing Qodo demonstration makes the architecture concrete with a small rule: “I don’t like nested ifs.” It resembles an instruction a developer might already place in Cursor. Qodo then uses project context to build good and bad examples, constructs a workflow to detect the issue, and tracks acceptance over time so the team can inspect and adjust the standard.
For a concrete TypeScript illustration, consider a rule that prefers guard clauses over nested conditionals. The proposed transformation preserves the return values while flattening the control flow:
typescript
type Account = {
active: boolean;
canDeploy: boolean;
};
// Before review
function mayDeployBefore(account: Account | null): boolean {
if (account !== null) {
if (account.active) {
if (account.canDeploy) {
return true;
}
}
}
return false;
}
// Proposed review suggestion
function mayDeployAfter(account: Account | null): boolean {
if (account === null) return false;
if (!account.active) return false;
return account.canDeploy;
}
The important event is the PR review: the generator may have produced nested conditionals despite already having a rule against them. A separate review workflow detects the violation and proposes a change using the contextual good and bad examples. The suggestion still awaits a decision; detecting a violation and applying a fix are different operations.
The demonstrated feedback loop continues beyond that suggestion:
- Run checks for individual rules, including through the CLI.
- Surface the nested-if finding and the associated suggestion; Friedman also describes a graph for inspecting the results.
- Record whether the developer accepts or rejects the suggestion.
- Use those responses to adapt the standard and its enforcement.
Qodo can also propose standards learned from the project, so each rule need not begin as a manually written preference. This closes the gap raised earlier: instead of assuming that a rules file is followed, the team gets detection, visible responses, and a way to revise the rule.
The progression is from generating code, to delegating generation, to applying AI across the lifecycle—including the work that determines whether a change should ship. Friedman’s closing 2× return is an aspiration contingent on that investment. The useful system is not the tool alone, but the quality workflow the team builds, measures, and keeps improving around it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Qodo’s survey of developer adoption, perceived quality, confidence, testing, review, and context.
June 2025 telemetry analysis linking greater AI adoption with increased throughput and longer review times.
Public security-review prompt specifying review scope, confidence thresholds, and excluded vulnerability categories.
An introduction to Qodo’s context engine for understanding large enterprise codebases.
Further reading
- Sonar: The State of CodeArticle
A four-part analysis of reliability, security, maintainability, and language-specific issues across 7.9 billion lines of code.
Anthropic’s GitHub Action for reviewing pull-request changes for security vulnerabilities.
Updates since the talk
The February 2026 beta announcement describes discovering, enforcing, measuring, and maintaining organizational coding standards.
Read the complete timestamped transcript
- 0:00
[on-hold music] I'm really excited being here.
- 0:22
So many-- so much pragmatic and insight and suggestions. I was sitting there, uh, just, just before. So I'm Itamar Friedman, the CEO and co-founder of Qodo. Qodo stands for quality of development, and I'm gonna share, uh, our reports and other companies' reports about state of AI code quality.
- 0:39
Uh, you know, trying to, uh, talk about the hype versus reality, which was, uh, like, one of the, uh, points that were discussed here quite a lot, which is awesome.
- 0:49
So in the last three weeks, four weeks, we saw like, three outages in the clouds, unfortunately, right? And these are f- coming from companies that really care about moving fast, right?
- 1:00
They're, they're, they're saying themself that they're using AI to generate code ten percent, thirty percent, fifty percent. At the same time, they care about quality. So how did that happen?
- 1:10
And is it, is it related? I don't know, but let's have some... Uh, I'm gonna share some guess. So by the way, [REDACTED:age] percent of developers say that the, like, quarter of their code is either generated by AI or in f- in f- like, uh, uh, shaped by AI, and fifteen percent say that even more than eighty,
- 1:27
eighty percent of their code, uh, is basically generated or sh- or shaped by AI. Now, people are using AI to do vibe coding, but actually they're even doing it for vibe checking, vibe reviewing.
- 1:42
This is the command of Clau-- this is the prompt for the command of Claude Code for security review. It was hyped like two months ago. Do you know what I'm talking about?
- 1:52
Now it says there, I don't know if you see it, uh, "You are a senior security engineer." Good. And then like somewhere there, uh, down the line, it says, "Please exclude denial of service."
- 2:04
Don't, don't, uh, catch denial of service issues. Maybe that's part of the, [chuckles] part of the reason like we're, we're having, uh, cloud outages. Probably not just that, but you get the point.
- 2:15
Like, we need to be rigorous about how we deal with quality. It's not just like vibe quality or, or so, like we're doing vibe coding sometimes. Uh, let's go to another example, okay?
- 2:27
Cursor, I, I guess like... or, or Copilot, most of you use rules, right? We're gonna talk about it. You invest in code generation. After a while, you understand if you invest, you'll get more out of it.
- 2:38
And, uh, we, we asked like a bunch of, of developers, and I'm asking you as well, think, think for a second for all the developers there in, in the audience, like, when you write Cursor rules or Copilot rules, et cetera, do you feel they're completely followed or it's like mostly followed?
- 2:54
Do you know how much they are followed? And what extent are they followed? It's rigorously, like how technical deep they're, they're being followed. So the... what we get back, like the answer from what you see here on the screen is mostly like B, C, and D.
- 3:07
They are followed, but they're not completely followed. Okay? So that means like we are generating code, trying to push it to the standards, but it not necessarily still like getting to the quality we wanted.
- 3:19
I'm gonna share a bit more statistics and, and information and some insight from three reports. One done by Qodo, uh, another by... done by Sonar, another by Pharos, and all of them are, are focused on code, code quality, review, et cetera.
- 3:35
The sample size is thousands of developers, in some cases even more, millions of pull request, and, and a billion of, of lines, lines of code that were, uh, uh, were being checked.
- 3:46
Like, for example, if you think about, uh, Sonar, this is a company, yeah, that like coming from pre-AI, but they see code at scale, and you-- they're doing like a lot of, uh, checks in code that are not necessarily AI-focused, but are necessary in order to check, uh, your, your software from all possible direction, and that's why
- 4:09
they're scaling and the scale of the code that they're seeing is, is immense. Okay? So for example, we took information from, from their report, and eventually my purpose here is to break down the different dimension of what, uh, code quality means and give you some-- share some stats and, and insights.
- 4:25
I want to start with the end. [chuckles] Okay? This is the takeaway I want you all, all, like to take from, from the next thirteen minutes that I have. We started with code generation.
- 4:38
We, like out of the box, use it for autocomplete, et cetera, and you invest in it, and you can get more out of it. But there is a glass ceiling for how much productivity you can get from code generation.
- 4:49
And then we move to the agentic code generation, right? Uh, let's call it Gen two point oh. And that's a higher glass ceiling. It could do much more productivity, and especially if you invest in it, for example, rules, et cetera.
- 5:02
Then with AI breaking outside of the IDE, we can start using AI also for code-- for Agentic quality workflows. It could be inside the IDE, but the, the truth is that if you think about all the workflows you have in your organization, especially if you're more than one hundred developers or so, you probably have a lot of
- 5:23
workflow that you... are related to quality that you need to auto-automate. And that's where you start like, uh, breaking through the glass ceiling of, of productivity if you invest in it.
- 5:33
And finally, I, I claim that you need those agentic workflows, keep learning, and we might touch a little bit of that like l-later, later on. Okay? Like, because quality is something dynamic, so you'll only finally break, break the glass ceiling if, if you really have those quality workflows and rules and standard being, being dynamic.
- 5:54
And then, then you will see the promised two X, let alone the ten X that you were promised the hype and, and you, you heard from McKinsey and from Stanford, you're not getting that.
- 6:03
I don't need to tell you the two X, ten X for the entire software development, uh, lifecycle.
- 6:09
So a bit about more about, uh, the market adoption. Uh, one of the report say that, uh, 82% of, uh, adoption already for AI, uh, dev tools are being used daily or weekly.
- 6:21
Uh, some people, at 60 per- 60%, fifty-nine report that they're using more than three, and 20% saying they're using more than five code generation tools. If you think about it for a second, uh, don't only take like Cursor, Copilot, Codex, uh, Claude Code, et cetera, sorry from insulting anyone in the, in the...
- 6:37
That I forgot their tool, but there's also the lovable et cetera. They also generate code, and by the way, you're gonna get to 10. I'm-- Count on me, you're gonna get to 10 tools in two, three years that generate code for you, okay?
- 6:49
Come to talk to me about it later, I-I'll try to convince you. And, and the thing is that it-it's coming from bottom up, like 50% of the usage is coming from less than 10 of...
- 6:58
A team that are less than 10 developers, but it is propagating also to the enterprise. Again, I'm sure you know, I mean, talk-- propagating to the enterprise at, at scale, like not just like five developers.
- 7:08
In the last year, we're seeing like more and more enterprise using co-code generation. Uh, so, uh, and if like, um, an average w-within reports, we saw 82 to 92% using weekly to a monthly, uh, code generation tools.
- 7:23
And in some cases, maybe extreme, maybe not, we're gonna talk about it, we saw 3X productivity boost in writing code, okay? But that doesn't mean that if you have, uh, 3X productivity in writing code, that you actually guarantee any quality like I presented before.
- 7:40
So actually, 67% of the developer that we asked, asked have serious quality concerns about all the AI generated-- all the generated code, uh, uh, like, uh, code generated by AI or influenced by AI.
- 7:54
And they're claiming that are missing the framework, how to deal with quality, how to measure quality. It's a big question. What is quality? I'm gonna talk about it in the next few slides, okay?
- 8:04
Think about it for a second before I bre-break it down. What, what is quality? Um, so what we're actually saying that the crisis with vibe coding, uh, viable coding, we're seeing it shifting and, and evolving, is that you're getting like more task being done, like 20 some report, 20% more task, you know, velocity.
- 8:24
And like ninety-seven or more percent or so of PR is being opened. And eventually it takes more time to review PR, like 90% more time to review PR. And by the way, like there's a lot of statistics about AI generating code.
- 8:39
A-at least there's not less amount of bugs per line of code. I'm not claiming that there are more, but even if there's not less bugs per line of code, you have much more bugs because there are much more PRs, much more code being generated, et cetera, right?
- 8:53
So that, that's a problem for the reviewer. So it's somebody surprised it takes more time to review these, especially in the age of agents, right? When five minutes calling to Claude Code, I have one thousand line of code after five minutes.
- 9:05
Once upon a time, it took me like hours to write ten proper line of code, right? Now, let's zoom out for a second. Code generation is magnificent, okay? Like it, it's a game changer when you're talking about greenfield.
- 9:18
You saw, uh, people talk about it a few slides, a few, uh, other minutes before me. Uh, it, it revolutionized how we do p- proof of concept, uh, project, et cetera.
- 9:28
But when you're dealing with heavy-duty software, then you, you like it or not, we are dealing with a lot of things when, uh, w-when you serve millions of clients, you have financial transactions.
- 9:40
When you're doing transportation, you're dealing with code integrity, if you like code governance, uh, review standards, testing, reliability, et cetera. That's what we need to, uh, uh, to deal with.
- 9:51
Now, let's break that under the surface part of the glacier into two dimensions. This is one dimension. You can look on the qua-quality issues, uh, in-- throughout the software development lifecycle, like planning and then development, writing code, code review.
- 10:08
Code review is a bit of a process, but like what you're s- like checking quality, that's part of the process of code review. Testing, which is another part of, of quality, and, and deployment, and I know I didn't cover the entire like, uh, uh, software development lifecycle, but just to give an example.
- 10:25
And each one of them like possess, like introduce new problems that are coming because you're using more and more AI-generated code. Um, now another dimension to look at it is actually code-level problems and process-level problems, okay?
- 10:40
I'm not, I'm not opening the, you know, lists of functional, just opening the list of non-functional. You're talking about security, inefficiency that are not necessarily, uh, functional. Use-- I'll show you some statistics about that.
- 10:55
And then process level is, for example, learning. Hey, if you will have
- 11:02
a, a, a bad outage because of AI-generated code, who is responsible? Is it the AI or, or the team that own that, okay? Like you need to learn and own the code eventually.
- 11:13
That's a process that n-needs to be done. Verification, porting, guardrails, standards, uh, et cetera. So, so, so all of those issues, when they're introduced to thousands of developer that we ask them, do you think like actually AI helped to reduce with those problems or, or actually made m-more like more challenging?
- 11:34
Forty-two, uh, people reported that they spend forty-two more of the development time on solving issues, on fixing bugs, et cetera, and, and they saw thirty-five, uh, uh, percent project delays.
- 11:48
We're talking about, we're talking about m-maybe gains, they're talking about like delays. Okay, there's some bias. We told them, we talked about problem with quality and what's the impact, e-et cetera.
- 11:58
Um, but that's what they, they sh- they present, uh, to... When they, they answer, uh, when, when you're talking about like when you're mess- using AI code-- AI-generated code.
- 12:08
And we see reports, uh, some of the reports talking about three X more security incide- incidents. By the way, it makes sense. You remember we had a slide saying three X more writing code, [chuckles] so three X more security incidents, like the same amount of line of code, the same amount of, uh, uh, problems, the correlation.
- 12:23
So what to do with that? Like, I talk about problems and problems and problems. Okay. Help, help me deal with it. Uh, like, let's, let's spend a few minutes on, on that.
- 12:32
So one, one suspect, of course, is testing. And actually, really interesting, we ask a couple of question about testing, and one really relevant saying is that people said that when they heavily use AI to...
- 12:46
on testing, use AI to do testing, they actually double their trust in the AI-generated code. Okay, that's one thing. The ne- next suspect to help us with the quality is code review.
- 12:59
What really interesting about code review, that it's a process that helps almost with all the process level and the code level, like, issues. For example, you can set your AI code review tool to tell you, uh, block this PR if it doesn't cover certain level of test coverage.
- 13:17
So through the PR, you take care of the testing process problem, okay? So code, like code review with AI is actually one of, one of the major things you, you, you can do.
- 13:29
And people that are, uh, developers that are using AI code review tool, they're saying that they're seeing, they're seeing double the quality gain. And they're saying that actually it's- it helps them to, uh, uh, im-impr-impr-improve forty-seven percent in productivity of writing code, okay?
- 13:47
Now, a bit statistics from our own, uh, AI code review tool. We scan a million of PR's a month, and we took one mill- million of those PR's, and we noticed that seventeen percent include like high-severity issues.
- 14:01
By the way, we're now analyzing, uh, before and after using AI. I don't have that statistics yet. But we're noticing since we're starting, uh, most of the companies we serve, they use AI-generated code.
- 14:11
So that's why, uh, I don't have before. We need to go scan backwards. Uh, and that's like a, a really big, a big number. Another thing I wanted to talk to you like about, uh, when you're trying to improve on quality is, is the foundation of having the right context that is brought to the, uh, code generation
- 14:30
tool, that is brought to the AI code review tool. Better context, better quality across the board, wherever you're using AI. Uh, so when we ask developers when, when you ha- when you don't trust AI-generated code, like you remember like [REDACTED:age]-seven percent, uh, said, uh, like are really worried about that.
- 14:48
They said eighty, eighty percent of the time, they don't trust the context that the LLM have, okay? And, and, and, uh, when we ask developers, "What would you like to be improved in your AI-generated code, in your AI code review tool?"
- 15:02
They said the number one was context. It was number one at thirty-three percent. They can choose like among many things to, to improve. So context is extremely important. I can tell you that at Qodo, one of our technology moats, uh, is, is around context, and when you connect our context engine, we're seeing it as the number one
- 15:20
tool that is being used. Like [REDACTED:age] percent of code generator or code review tools, [REDACTED:age] percent of their calls to an MCP would be to a context MCP, okay?
- 15:31
And just to tell you, the context doesn't necessarily need to include only your code. It could also include context to your standards, to your best practices. We're seeing in our AI code review that eight percent of the context usage, usage is actually from files that are related to standards and, and best practices, et cetera.
- 15:49
Okay, I have to, uh, CEO of Qodo, like marketing will be mad at me if I don't brag a little bit, right? So this is, uh, our kind of like our architecture of our context engine being presented by Jensen on GTC keynotes.
- 16:01
And he-- notice he didn't talk about our
- 16:04
co-co- code review capabilities, about our testing capabilities. He talked about our context engine that NVIDIA checked because there is a realization that AI quality, AI-generated, whatever review testing will come from bringing the right context.
- 16:17
So invest in that. You need to, to build your context, buy a solution and invest in it, build your solution, uh, et cetera. And the context needs to include code, uh, uh, versioning, peer history, uh, organization logs, et cetera.
- 16:33
That's where all the context sits. It's not just in the last branch of your code base, okay? So I'm, I'm zooming out, starting to talk about like recommendations and, uh, and like, uh, the takeaways.
- 16:46
So what, what, what's next? So automated g- uh, quality gateways, invest in that. People talked throughout the morning about parallel agents. You know what I'm talking about? Like background agents.
- 16:58
You can use a lot of those like tools and capabilities to build, build your quality gates. Uh, use intelligent code review testing, and you need a li- living and breathing like documentation and, and...
- 17:12
What documentation means is, is a story by itself. Uh, I'm not gonna double-click on it. And, and this is how I present for three years now, and I think I'm gonna go all the way until age of [REDACTED:age] with this slide, of how I think the future of software development looks like, okay?
- 17:31
So basically, you have your specification and you have your code, right? And you have multiple agents, parallel agents, that are helping you to improve your spec, write your spec, improve your code, transfer, transfer from your spec to your, to your code, uh, uh, make tests which are executable specs, right?
- 17:51
Uh, and, and then you're gonna have your context engine, the software development database, and you will build your tools, especially MCPs around quality and verification. And you'll make sure you have environments Stable, secured sandboxes where those agents can run and, and run validation and quality, uh, workflows.
- 18:11
So don't, don't forget, like the path forward is quality is your competitive edge over your [laughs] competition. AI is a tool. It's not, it's not a solution, okay? And don't like only think about code generation as the only thing.
- 18:27
Look on the entire SDLC or product development lifecycle. I saw one of the, uh, uh, people talk to, uh, uh, on s- speakers, and it- iterate with everything we talked about today.
- 18:39
I have t- uh, I want to tell you that you will gain value from it. We're seeing in the reports, people seeing like security vulnerability being reduced, faster code review.
- 18:49
You... we just got a hit on that because of AI-generated code, and test coverage in a month can, can triple, uh, depends on, on the project, et cetera. With, with the last minute, I wanna show like a really small piece of what you can do with Qodo.
- 19:03
Uh, you can go into Qodo and define your own rule. For example, almost the same rule you'll put in Cursor of, "I don't like nested ifs," if this is a problem that you have.
- 19:14
But then Qodo will look on your context, build the good example, the bad example, and then start giving, like building a workflow that is specifically to catch that issue and give you statistics over time when it's being accepted and when not, so you can adjust that rule and really know and have visibility to, to your standards, okay?
- 19:37
So when an, a PR is written with a few ifs and else, although it was written with Cursor Copilot that had a rule do not do nested ifs, et cetera, then eventually when you open a PR, you'll get, uh, Qodo, uh, uh, catching that and giving a suggestion according to the good and the bad example.
- 19:56
Qodo will also make a graph, give you a CLI checks, like check each one of the rules, and eventually tell you the nest- nested if, and then will record and learn what you did or did not do with that suggestion in order to adapt the standard and, uh, of the, of the quality.
- 20:13
Um, there were also automated like suggestion. You don't need to write your own. It learns your, your s- your, uh, standards and quality and offer that to you. And that's it.
- 20:22
I'm, I'm really, really excited about like breaking the glass ceiling, [laughs] okay, with what we did with code generation and then agentic code generation. Now we're turning into the era of putting AI into work and through the entire SDLC.
- 20:37
The most important part is related to quality. You would need to invest in that. It's not out of the box, okay? And then you would see eventually the promised 2x te- [laughs] that, that, that probably, uh, promised to the CEO or something like that once they give you the budget for, for the relevant tools.
- 20:55
Thank you so much. [upbeat music]