AI Engineer World's Fair 2026
Guide, Verify, Solve: The Engineering Discipline Agentic Development Demands
Read the talk
Guide, Verify, Solve: Engineering Agentic Development
Agentic development needs more than faster code generation: task-specific context, independent verification, and repair loops must connect to enforceable release criteria.
From a talk by Anirban Chatterjee
Before you start: Familiarity with coding agents, pull requests, static analysis, and CI/CD will help you follow the development and verification loops.
When faster generation stops producing faster development
What makes AI-assisted development repeatable enough to move from experiments into software that other people depend on? The transition resembles an earlier shift in cloud computing: powerful technology becomes broadly useful when engineers make it consistent, scalable, and trustworthy. Anirban Chatterjee, a former software engineer who introduces himself as working in product marketing at Sonar, places safety and trust at the center of that transition.
The opening evidence is a study of Cursor adoption in GitHub projects. Chatterjee describes a temporary productivity increase that disappears around the three-month mark, alongside a persistent increase in static-analysis warnings and code complexity. The corresponding paper, Speed at the Cost of Quality, makes the measurement more precise: repository velocity means commits and lines added, and the gains dissipate after two months. Configuration files serve as a proxy for Cursor adoption; they do not establish which tool authored each change.
SonarQube supplied the study’s warning and cognitive-complexity measurements. Warnings identify potential issues, not necessarily confirmed defects, but their persistence matters: a burst of additional code can leave a larger maintenance burden after the initial velocity gain has faded. Chatterjee proposes that this burden helps explain the slowdown. That is a plausible mechanism, rather than a causal conclusion established by the account presented in the talk.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Verification debt grows with criticality
A short-lived internal experiment can tolerate a relatively small gap between generated code and acceptable code. Perhaps its only user is its author; perhaps a small team will discard it next week. The requirements change when the same development approach serves a large, frequently changing codebase, many users, and adversaries actively trying to break the application.
Verification debt is the work required to close the gap between the agent’s default output and the quality the application needs. Engineers must close that gap before shipping. The slide’s rising required-quality curve and flat agent-quality line illustrate this relationship conceptually; they are not measured quality scores.
Model improvements do not remove the underlying problem. Chatterjee mentions wanting to try the newly available Fable, but emphasizes that even strong models remain error-prone. Errors that are tolerable during exploration can have serious organizational consequences in production.
Missing context creates another source of failure. An agent may not know about changes elsewhere in the repository, the business objective behind a feature, or a decision made in a meeting two weeks earlier. The engineer carries those constraints into the task; the model only has the context supplied to it. A locally reasonable implementation can therefore miss the actual objective.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Choose models by their quality profiles
Models also make different kinds of mistakes. Chatterjee describes the Sonar LLM Leaderboard as evaluating models on approximately 4,000 coding tasks. Its evaluation considers correctness, task-solving rate, complexity, maintainability, reliability, and security. A single success score cannot express all of those properties.
The displayed comparison pairs Claude Sonnet 4.6 with Claude Opus 4.6. Chatterjee’s interpretation is that Sonnet performs well on correctness, task solving, and reliability, while Opus may be useful when maintainability, security, or lower complexity matters more. Switching models to manage token spending can therefore change the quality profile of the resulting code. These are the profiles discussed in the recording; the leaderboard changes as new evaluations arrive.
Model selection and verification solve different problems. Selecting a model can improve the starting point for a particular task, but the selected model can still produce code that should not ship. Verification remains part of the loop regardless of which model generates the change.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Human approval needs an independent backstop
Reading the generated code is an obvious verification step, but confidence in an assistant can undermine the reviewer’s independence. Chatterjee introduces Wharton research in which participants could use an AI assistant that was instructed to give confidently incorrect answers on some trials. In Study 1 of Thinking—Fast, Slow, and Artificial, participants followed the advice on 92.7% of AI-accurate trials and 79.8% of AI-faulty trials, conditional on using the chatbot. The tasks were adapted Cognitive Reflection Test questions, not code reviews.
The code-review connection is Chatterjee’s extrapolation: several agents can generate changes simultaneously, while a human still has to understand the combined application and ship on a deadline. Higher volume and integration work create pressure to rubber-stamp plausible output. Automated checks provide a backstop that does not depend entirely on the reviewer resisting that pressure.
There is also a distinction between understanding a function and understanding the software around it. A correctly written function can behave predictably under its assumptions. Requirements, external dependencies, interactions with other code, and unexpected user behavior make whole-system behavior harder to establish. Chatterjee frames this as the difference between provable code and software that breaks in novel ways. Automated verification controls some of the risk introduced when developers delegate more of the implementation; it does not make the surrounding system disappear.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Independent checks inside a repair cycle
The first requirement is zero-trust verification: apply the same comprehensive checking regime whether a human, one model, or another model wrote the code. The verifier should use a different methodology from the generator. Algorithmic checks provide an auditable, explainable, repeatable basis for deciding whether the change satisfies a rule, instead of asking the generating model to endorse its own work.
The second requirement is multilayer verification. Computational analysis and LLM-driven review contribute different forms of scrutiny. The demand for independence does not exclude LLM review; it means that LLM review cannot be the only basis for trust. One or two techniques will not uncover every problem in software.
Sonar calls its framework AC/DC, the agent-centric development cycle. Chatterjee introduces verification first because he sees it as the fastest phase to implement now: a multilayer review spanning quality, security, and compliance. He then places guidance before it and remediation after it.
The resulting execution order is:
- Guide: Supply the context, guardrails, and constraints the agent needs before it writes code.
- Verify: Evaluate the generated change against the required criteria.
- Solve: Give the agent access to the tools and findings needed to repair detected problems, then repeat the loop.
The useful unit of work is therefore not just a generated patch. It is a patch that has passed through a feedback cycle in which detected issues can be corrected.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A shared rulebook across teams and tools
The organizational reasons for adopting this cycle extend beyond finding individual bugs:
- Consistent verification: Apply a standard rulebook across projects, teams, and coding tools.
- Effective AI use: Manage token spending and match models to work they handle well.
- Earlier security detection: Find security issues before they reach production. Chatterjee points to the possibility of exploitation on the day a CVE is announced as a reason to shorten that window.
- Provable compliance: Preserve an audit trail showing that required checks ran consistently, especially in regulated environments.
SonarQube supplies computational verification across syntax, data flow, architecture, and control flow. Chatterjee describes broad language coverage and agent-facing integration points that expose those checks directly to coding tools. He then introduces Gitar, the San Mateo company Sonar had recently acquired, as the complementary AI code-review component.
Gitar’s role extends into CI workflow automation: it can identify issues with LLM review and block changes that fail the required quality standard. Chatterjee describes optional capabilities to write fixes, approve them, and merge pull requests automatically. In the default behavior he presents, it surfaces issues and starts a dialogue instead. Teams enable additional automation as they gain confidence; automatic merging is not the starting assumption.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
New code and existing debt need different entry points
Sonar Vortex brings guidance and verification into the agent’s inner development loop. It provides constraints before generation and checks while the agent works, so the same agent can find and fix issues before continuing. The SonarQube Remediation Agent starts from a different input: existing issues in the backlog, technical debt, or legacy code. Its purpose is to let repair work proceed while developers focus on new functionality.
Chatterjee presents both capabilities as generally available during the week of the talk. The Remediation Agent’s published launch announcement describes a staged rollout, so that announcement should not be read as establishing one universal availability date for both products. The functional distinction is more durable:
| Capability | Starting point | Work performed |
|---|---|---|
| Sonar Vortex | An active coding task | Guide generation, verify, and repair in the inner loop |
| Remediation Agent | Existing issues and technical debt | Repair accumulated problems in an existing codebase |
Both apply verification to agent work, but at different points in the codebase’s life.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Encode the requirements before starting the loop
The cycle operates at two levels: the inner loop where an agent develops a change, and the outer CI/CD loop where that change is reviewed and shipped. Before either begins, the team needs explicit specifications. These include the desired architecture and architectural boundaries, acceptable coding patterns and syntax, allowed and prohibited dependencies, and logging, observability, and tracing practices.
Specifications describe how the software should be built; acceptance criteria determine what may reach production. Security, quality, and maintainability thresholds must be written down and encoded. Sonar supplies defaults, but Chatterjee emphasizes adjusting them to the organization’s requirements.
For example, a repository could make part of its guidance concrete with a small JSON document:
json
{
"architecture": {
"databaseAccess": "repository layer only"
},
"dependencies": {
"allowed": ["existing project dependencies"],
"newPackages": "require approval"
},
"observability": {
"logs": "structured",
"requestTracing": "preserve request identifiers"
},
"releaseCriteria": {
"security": "pass configured security gate",
"quality": "pass configured quality gate",
"maintainability": "pass configured maintainability gate"
}
}
These illustrative policy values express the kinds of constraints discussed here; this is not a Sonar configuration schema. Guidance must be supplied to the agent, while release criteria must also be enforced by the verification workflow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Verify inside the agent loop and again at the PR
At task initiation, the agent needs the relevant slice of those specifications and the codebase. Sending the entire repository into its context window can lead to unnecessary exploration and token spending. Chatterjee describes Vortex as selecting context for the assigned work so the agent can begin with the constraints that matter to that change.
After generation begins, the agent can call verification and receive an issue list for the code it is writing. It can then repair those issues immediately. This placement matters because an unfixed defect can become an assumption in subsequent development: later agent loops may build additional code around it. Repairing the problem near its introduction prevents that propagation.
The outer loop begins when the work enters formal pull-request review. Gitar supplies LLM-driven review alongside SonarQube’s computational analysis of the code in the PR. Findings cover quality, security, and maintainability; the configured quality gate determines whether the change may advance.
If the PR fails that gate, a Fix Agent can address the findings. In the flow presented, passing the gate permits progression through testing, building, and deployment. The two verification points have complementary scopes:
| Loop | Scope | Purpose |
|---|---|---|
| Inner agent loop | Code being developed | Find and repair issues before later work depends on them |
| Outer CI/CD loop | The proposed PR change | Enforce release criteria during formal review |
Inner-loop verification does not replace the outer release gate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The Cursor demonstration: context, findings, repair
The recorded demonstration puts the inner loop into Cursor. At the start of the assigned task, Cursor calls the Sonar Vortex context tool to obtain information about the code it is about to change. It then writes an initial implementation and calls verification to obtain findings. The integration lets the agent respond to those findings inside the same task rather than handing them off as a separate review chore.
Chatterjee also names Claude Code, Codex, and Antigravity integrations. Their invocation behavior should not be inferred from this Cursor example: the separate Claude Code setup guide requires prior CI analysis and notes that context retrieval may need an explicit request.
Near the end of the demo, Chatterjee points out a flagged issue and describes the next steps: fix it, run analysis again, and do not proceed until verification passes. The presentation moves on before establishing a completed passing result. The demonstrated mechanism is the feedback path—an issue becomes input to the agent’s next repair action, and the repaired code must face another verification pass.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bounded autonomy changes the developer’s job
Chatterjee closes by positioning governance and verification as infrastructure for tackling larger problems with coding agents. He refers to customer data showing greater success with AI tools but skips the detailed results. His practical adoption point is that teams may already have access to some of the required tooling.
Sonar reports more than seven million developers using its tools and close to 750 billion lines of code analyzed across its solutions each day. Those are vendor-reported scale figures, not measurements of the effectiveness of this particular workflow. Chatterjee also cites Gartner Magic Quadrant leader status without specifying a category or year.
The operational destination is bounded autonomy: agents have freedom to generate code within centralized constraints, and independent verification determines whether their work is acceptable. Context guides the work, checks expose mistakes, and agents receive the tools and authority to repair those mistakes.
That shifts developer responsibility toward orchestration. Engineers need tools to design the context frameworks and processes in which agents operate. At the organizational level, Chatterjee recommends one independent, multilayer verification platform used consistently across projects, teams, developers, and coding tools. The purpose is to prevent separate tool silos from creating blind spots: teams can vary how they generate code while retaining a shared, enforceable standard for what they ship.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A matched-repository study of Cursor adoption, development velocity, static-analysis warnings, and code complexity.
Shaw and Nave's experiments on reasoning with an optional AI assistant, including acceptance of confidently incorrect advice.
An interactive leaderboard for coding-task pass rates, complexity, and static-analysis findings.
Sonar's announcement of its acquisition of the AI code-review platform Gitar.
The launch announcement describes automated repairs, verification of fixes, and developer oversight through pull requests.
Further reading
A setup walkthrough covering project context, verification hooks, repeated repairs, and CI prerequisites.
Read the complete timestamped transcript
- 0:00
[upbeat music] [audience cheering] All right. Thank you.
- 0:18
That's very helpful. My name is Anirban Chatterjee. I do product marketing at Sonar. I'm really excited to be talking to this group today. It's actually my first time here at this conference, um, and so I've been having a blast, uh, along with the rest of my team here, uh, meeting a whole bunch of AI engineers as well
- 0:33
as leaders and, you know, influencers and founders. Uh, there's a lot going on in this space. I think this year there's really been a turning point from experimentation to engineering, and that makes me-- that warms my heart very deeply because I started my career many, many, many, many, many years ago as a software engineer, uh, writing code
- 0:55
for, for servers, if you can believe it. Um, and I think there's a turning point that's happening right now where we're starting to add the capabilities that we need to add to these systems in order to make them repeatable, make them scalable, make them consistent, much in the way we were doing with cloud computing not too long
- 1:14
ago in order to expand the access that IT technology gave to small businesses and other innovators. I think AI is gonna do the same thing for software development going forward.
- 1:24
But in order to do that, in order to get there, we need to start adding safety and trust to these systems so that they can be used more widely across a wide variety of use cases so that we can build new things and solve bigger problems.
- 1:37
And how we get there is what we're gonna talk about today. And, uh, for those of you who were in, uh, Tarek's keynote yesterday, uh, he presented some of this data, and I'm gonna talk about it a little bit deeper today.
- 1:46
So there was a study that Carnegie Mellon did, um, where they actually looked at projects that were posted on GitHub, and they were able to use the metadata to support-- to sort them into projects where, uh, there was just traditional tools that were being used and projects where an AI tool was used to write the code, and
- 2:03
in this case it was Cursor, although it could have been any AI tool. And what they found was interesting. They found that there was in fact a temporary spike in productivity, um, but it lasted about three months and then it went back down.
- 2:16
And the reason for that, we think, is because there was also a persistent increase in static analysis warnings and code complexity. They were actually using SonarQube, uh, to actually collect the data on this, and they saw that there was a persistent increase in these types of issues that went beyond the three-month mark and persisted well into the
- 2:33
future. And so it's these types of issues that end up actually slowing developers down even more and, and making... And this is what makes it a challenge to deliver high-quality code using AI tools.
- 2:46
The reason for this is that there's a differing need for quality depending on the criticality of the application, right? If you're experimenting, if you're playing around, if you're just one person building things to see what's possible, um, it's an internal non-critical application with just a few users.
- 3:00
Maybe it's just you, maybe it's a small team. Maybe it's such a short-lived project that it's not gonna last very long. The gap between the quality that you're getting from the AI tool and the quality you need from the application is quite small, right?
- 3:12
And so you can l-- And you can live with, uh, that gap. But as you move to higher levels of criticality, as you run into situations where you're supporting many, many users, it's a larger code base with many lines of code and many changes happening across that code base all the time.
- 3:27
Uh, you have many, many users. Some of them could be adversarial users actively trying to break your software, right? And so in those cases, the quality level you need is quite a bit higher than the quality level you're getting by default from these AI tools.
- 3:40
And that's where this verification debt comes in. That's where you have to bring the humans in, bring your software engineers in to try to close that gap and make sure that the quality level is brought up to an acceptable level before you ship that code into production.
- 3:55
So why is this happening? Why is this gap actually occurring? We know these models are excellent. They're getting better and better all the time. I'm really excited to start playing with Fable now that that's out to see what levels of, um, of code we can get out of Fable going forward.
- 4:08
But we do know that because of the technology, because of the way that these models are built, they will still make mistakes. They will still have quality issues. They are still somewhat error-prone.
- 4:17
And if you let these errors go into production code, you could have a catastrophic effect to your organization. They're also missing context, right? They only know what you tell it.
- 4:26
They don't know the broader things that are happening, uh, elsewhere in the code base. They don't know what's happening with your business. They don't know what happened in the meeting you had with somebody else two weeks ago that's gonna influence the code you're writing today.
- 4:37
They don't have all the context that you have as an engineer, right? And so they don't always know your objectives the way you do, and that is gonna also cause gaps between what you need from the software and the way it's built.
- 4:51
We also know that models are diverse. No two models are the same, and they have diverse quality issues. And we actually want to explore this. And so we actually have a leaderboard that you can go to on our website right now.
- 5:00
It's called the LLM Leaderboard. And what we do is we take all of the major new models that come out, and we evaluate them. We give them, you know, four thousand or so coding tasks, and we evaluate them using all of the metrics that SonarQube uses to evaluate code, right?
- 5:13
We look at their correctness, complexity, um, the way-- the rate at which they're solving the tasks we assign them, and then our classic things: maintainability, reliability, and security. And we're able to kind of graph all of these models across these different axes and show you where models perform well and where they have room to improve.
- 5:29
And what you're looking at on the screen right now is actually Claude Opus 4.6 and Claude Sonnet 4.6. If you're a Claude customer, you might be toggling between these two models to control your token burn rates.
- 5:39
And you'll see that, uh, Cl-Claude Sonnet is actually quite good from a correctness standpoint, from a-- from solving tasks, uh, and from a reliability standpoint. But if you're requiring higher levels of maintainability or higher levels of security, if you're trying to get a lower complexity out of your code, you might benefit from switching to Opus, uh, for
- 5:55
tasks like that. And so we run these kinds of analyses across a lot of different models, and you're always able to go to our website to get the latest, uh, analyses that we run.
- 6:03
I think we're actually doing the latest Claude and, and OpenAI models pretty soon. But this kind of data is helpful, right? 'Cause it tells you what-- where models are good and where models are not good.
- 6:12
And it also serves to put some salient on the fact that you still need to be vigilant with these models, right? None of these models are ever going to be perfect.
- 6:19
You're always going to have some kind of need for verification in the loop to make sure, um, that the code that you're getting is the code you actually want to ship.
- 6:27
Now classically, that verification can be human verification. It can be you, uh, your own eyes reading the code, your own intellect reviewing the code to make sure that is successful.
- 6:36
But we know based on experience and now based on research that human review can also be compromised. This is a study that was done earlier this year by Wharton, and they actually gave quite a lot of human participants tasks to complete, and they gave those human participants the use of an AI, uh, tool to complete those tasks.
- 6:53
But unbeknownst to those participants, the AI was told t- to confidently lie to these participants some of the time. And what they found in the data is that while participants did follow the AI advice ninety-two point seven percent of the time when the AI was correct, they unfortunately also listened to the AI nearly eighty percent of the
- 7:11
time when the AI was wrong. This is almost surely happening in code review as well, right? Especially when there's higher amounts of code being written, when there's multiple agents writing code simultaneously, when you now have to bring all those pieces together into a single software application.
- 7:26
The load is just too great. There's, there's only so many hours in the day, right? And you still have to ship something. And so there's a lot of rubber stamping that I'm sure is happening, uh, in all your organizations.
- 7:35
It's happening everywhere, and so we need to backstop that somehow with an automated verification tool. What can we do about it, right? A- as Tarek was talking about yesterday, uh, you know, I think all of us sort of-- when we got involved with software, one of the things that we found most attractive about it is that code
- 7:53
is, is quite... You know, once you write code properly, it's gonna run the same way every single time, and there's a certain level of comfort in that, right? There, there's a certain level of comfort in knowing that if I write this function the right way, it is gonna work this way every single time.
- 8:06
And there's, there's a clarity that comes to that, and there's, there's a certain confidence you get out of being able to build something that you know is gonna work well for every user going forward.
- 8:15
But we also know that code's written by humans, humans have requirements, and those requirements and externalities have impact in how this code functions. And as you add more and more and more code to the application, they can interact in unpredictable ways.
- 8:29
As you now allow users to use those applications, those users can do all kinds of things you didn't expect. And so software is not provable in the same way that code is provable.
- 8:37
Software can break in interesting and novel ways, and as you're using AI to write more and more software to solve bigger and bigger problems, um, you're going to run into these limitations more and more often.
- 8:49
And so having automatic verification as part of this process is an important part of the solution. It's going to help you control some of the risks that you're introducing by maybe releasing some of the control you have over the code that's actually being written.
- 9:04
And so we believe that verification is going to be a key enabler and a, and a key unblocker for all of the amazing things that we're gonna be able to achieve with, with, uh, AI-driven software development going forward.
- 9:16
And so when we say verification, what do we mean, right? We think there's two core elements, uh, to successful automated verification when it comes to AI coding. One is that it needs to be zero trust.
- 9:26
What do we mean by that? Zero trust in this context basically means that the code could really have come from anywhere. Uh, it could still be written by a human.
- 9:32
It could be written by an AI. As I just showed you a few slides ago, different AIs will write code in different ways, and you're not going to want to be-- You're not gonna wanna use that same AI to, to validate the code because you're gonna want a diversity of, of tools being used to make sure that
- 9:48
you're catching all the different issues that can happen. And so no matter where the code is coming from, you want to have a s- a similar comprehensive regime to verify that code that works the same no matter how that code was written, right?
- 10:00
Uses a different methodology to review the code than was used to write the code, uh, is completely auditable, completely explainable, so you can prove that verification was run the same way every single time, and it's algorithmic and repeatable and consistent no matter how you run it.
- 10:14
It also needs to be multilayered. You need to have multiple ways, uh, or multiple techniques being used, multiple approaches being used to review the code that is being generated, right?
- 10:23
Um, because you're never gonna be able to find every single problem that can occur in software by just using one or two methods. You need to use computational review.
- 10:31
You also need to use LLM-driven u- driven review and everything else in between, right?
- 10:38
You heard a little bit about agen-- Um, um, we've been hearing a lot about agentic loops this week. Uh, and Tarek, uh, talked, uh, yesterday about our framework for agentic loops.
- 10:46
We call it AC/DC or agentic-- agent-centric development cycle. And there's three phases in, in the AC/DC that we, that we like to talk about. The, the easiest one-- By easiest I mean the, the fastest one to implement now, the one that many of you are probably already on a path to implementing, is the verification step, which is
- 11:02
front-- which is top center, right? This is the most important piece that allows you to, to write code in these agentic loops in a way that is gonna be easily shippable.
- 11:11
Uh, it needs to be multilayer. It needs to be reasoning based, and it needs to cut across quality issues, security issues, and compliance issues to make sure that you're shipping quality that you can stand behind, right?
- 11:22
Um, before the verification step, there's a guidance step, and what guide allows you to do is provide guardrails and context and constraints to make sure that the agent has everything it needs up front to write better code, uh, the first time, right?
- 11:36
And then finally, after verification, you need to solve the issues that come up, right? And that's where the solve state come. That's where you can remediate any issues that are found in the code.
- 11:44
You-- Hopefully, you're allowing the agent to have the agency to do so itself by providing access to the tools it needs to, to find the issues and fix them itself, and then just repeat the loop.
- 11:53
And so these, these agentic loops with verification at the core is how you can get to shipping quality software using AI agents.
- 12:02
And there are prior-- Th- there's different reasons why you would wanna do this. And when we talk to customers, and we've talked to a lot of customers about this, uh, the driving functions that are forcing them to adopt verification across all of their AI coding processes are, are very similar, right?
- 12:17
They wanna make sure that AI code is verified consistently. They don't wanna have different methods of verification applying to different projects or different teams. They wanna have a standard rule book that applies everywhere no matter what tool is being used.
- 12:28
They also wanna make sure they're using their AI tools effectively. Right. Some of that-- A, a big part of this is token efficiency or just efficiency in general, but also it's making sure that the tools are being used for the things they're being designed to do in ways that we know they're good at doing, right?
- 12:42
So we talk a lot about token efficiency, and we talk a lot about making sure that the right models are used for the right projects, and so on. Finally, and third, catching issues from a security standpoint as early as possible in the development cycle.
- 12:54
Shifting left on security issues has been very important for a number of years now, and now that AI is writing more and more code, catching security issues up front is extremely critical, especially now that we're in a world where CVEs are announced and then immediately exploited, uh, by bad actors almost often the same day.
- 13:12
And so you need to make sure that your code is as hardened as possible from those types of issues creeping into production. And finally, maintaining compliance. Many of you, I'm sure, work in a regulated industry, uh, and for those types of situations where you need to be able to prove that verification is run constantly and consistently across
- 13:27
the board, maintaining an audit trail that allows you to prove that is extremely important.
- 13:32
We have a number of solutions that, that help with that. Uh, SonarQube has been around for quite a while. There are probably quite a few of you that are already SonarQube users.
- 13:40
It is a zero trust multilayer verification platform that works across syntax issues, data flow issues, architectural issues, and control flow issues. And it works across basically any language you would be using.
- 13:50
Uh, we have a lot of deep hooks that I'm gonna take you through in a moment that allow agents to have first party access to the SonarQube verification, so they can more effectively write high-quality code.
- 14:00
And we also just recently, and by recently, I mean just a few weeks ago, acquired a company called Gitar, uh, based right here in San Mateo, and they do AI code review.
- 14:08
Uh, and more than that, they actually pro-- uh, build a full... They build and automate the full CI workflow so that you can not only, uh, find issues using an L-- using our LLM, uh, approach, but you can also automatically block if those issues cause a quality issue that you wouldn't wanna push forward.
- 14:26
It can write fixes, and it can approve those fixes and merge those PRs completely automatically if you want it to. Now, we'd wanna earn that trust. It doesn't happen that way by default.
- 14:33
Usually, uh, the-- Well, by default, it'll just find the issues and show them to you and enter a dialogue with you, so you can have those issues fixed. But as you use it more and more and gain confidence, you can turn on more and more features and completely automate, uh, the PR review workflow if you like using
- 14:48
Gitar. So, uh, the other big news, uh, from earlier this week, and Tarek alluded to this, uh, during his talk yesterday, is that we also announced a new agentic loop capability, uh, with a product called Sonar Vortex.
- 15:00
And Sonar Vortex, uh, I'm gonna, I'm gonna show another flowchart that shows what it does. But basically, it is providing your a-agents with tools in the inner loop, in the agentic loop to not only get constraints and guardrails up front to write better code but also run verification as it's writing code in real time, so it can
- 15:17
find and fix the issues that are being created. And then we also released the Remediation Agent, and the Remediation Agent allows you to tackle backlog issues, uh, and, and take down your tech debt at a scale that you might not have the bandwidth to do now with human developers, right?
- 15:30
You can basically take your, your tech debt or your older issues, your legacy code, point them at Remediation Agent, and it can then improve your code base almost in the background while you focus on the, on the innovation work at the front end that you're working on now.
- 15:44
And those are both, uh, GA as of this week. Now, uh, there was a very, uh, detailed chart that was shown during the keynote yesterday that I'm gonna kinda break down for you and really explain what the different pieces of this chart mean.
- 15:56
This is kind of how we see the AC/DC applying not only to the inner agentic loops but also the outer, uh, CI/CD loops, uh, that we're all working in to develop code.
- 16:06
Before you start either of those roo-loops, though, it is really important to have a sense of the specifications of what you actually are going to want to accomplish with the software that's being written.
- 16:17
This is, uh, this is your architectural constraints. This is the way-- your desired architecture for the software. This is your coding standards and your coding patterns that are acceptable.
- 16:25
These are the list of dependencies that you are and are not allowed to use. These are your coding standards and, and syntax standards that you obey in your organization.
- 16:33
This could be your, your logging practices or your observability and tracing practices. All of that goes into your specs. Um, and then you also need to define what your quality criteria are, right?
- 16:44
And we have quality criteria that we ship with that you can use by default, or you can adjust them as it makes sense for your organization. But this is like what are the levels of security, of quality, of maintainability that you're willing to accept, uh, in your code that you're pushing into production.
- 16:57
You need to basically write that down and encode it. And now you're ready to start using LLM coding agents to write code. And whenever you're initiating a coding task with an, with an L- with an agent, uh, one of the first things that we can help, uh, is providing context and constraints so that the agent starts from
- 17:14
the ground floor with an understanding of the code base and the guardrails that is relevant to it in that moment. You have to manage the context window of the agent.
- 17:21
You can't just throw your entire code base at the agent up front. It's gonna spend a lo- spend a lot of time thrashing and exploring and burning tokens while it's doing it.
- 17:28
We, uh, are able to efficiently provide just the context that it needs based on the work that it's being given so that it can get to work writing productive code very quickly.
- 17:37
It then generates the source code, as you can see, and we also provide in-loop verification to the agent. As it's writing, it can call into us and provide and, and get a list of issues that we are finding in real-time in the code that's being written.
- 17:50
And the great thing about that is those issues can then be fixed immediately by the agent, so they don't propagate into future agentic loops that are gonna run in order to fully build out the software project that you're doing.
- 18:00
And this is being enabled by Sonar Vortex as of this week.
- 18:04
Once all of the in-inner loops have run, you, you reach a point later on when you have to start entering the formal review and, and shipping process for the code, and this is the CI/CD process, right?
- 18:13
And so there's a PR flow that gets initiated that I'm sure we're all familiar with. Uh, Gitar can live in that flow. SonarQube also lives in that flow in order to run a broad automated review of all of the code that is in the PR, uh, and a, and an automated verification that actually returns issues across quality,
- 18:29
security, and maintainability. Uh, there's a superhuman review that is LLM-driven by Gitar, and there's a computational review that is run by SonarQube that actually assigns grades for all three of those things and won't allow the PR to go past into production unless it gets a passing grade across that criteria.
- 18:45
So if there are issues that come up, you can actually use a Fix Agent, uh, to fix all the issues that are discovered there. And then once you're actually able to pass that quality gate, that is when you're able to g- proceed in the test and build and deploying the application.
- 18:58
That verification needs to run in both the inner agentic loop and also in the outer loop, uh, for CI/CD.
- 19:05
I'm running low on time, so I'm gonna hope this video completes. This is basically a video demo of the inner loop, the agentic loop. It's a, it's a demo of Sonar Vortex basically.
- 19:13
And what you've just seen happen is we've given it a task, and in starting that task, this is, this is actually Cursor that's running right now. Cursor called into our Sonar Vortex context tool to get some context upfront to, to g- give it an understanding of the code that it's working in so it knows how to, um,
- 19:30
how to write the code. It is now writing the code, and once it's completed, uh, the initial write, it's going to call into our verification process to get a list of issues that it finds.
- 19:40
And then if it-- and if issues get provided, it will actually fix those issues immediately in the inner loop. This is all happening automatically through an integration that we have directly with Cursor.
- 19:50
We have similar integrations with, uh, with Cloud Code, with Codex, with Anygravity, with basically any major AI coding tool that you would have.
- 19:59
You can sort of get the gist. It, it flagged an issue. It's gonna plan-- it's gonna fix it immediately, and it's gonna run, uh, the analysis again. And then it will not proceed until it actually is able to get a passing grade from us on the verification pass.
- 20:11
I'm gonna move past this 'cause I'm out of time now. [clapping]
- 20:16
So, uh, if you take one thing away from this presentation, it's that, you know, we believe very strongly, we're very convicted about this, that a governance and verification regime is a-- engine is extremely critical to unlock the next level of success that we need to be able to get from AI coding tools so we can solve bigger
- 20:33
and bigger problems. And we know based on our data, uh, that Sonar customers and Sonar users are able to get higher levels of success from AI coding tools. I can-- I'm not gonna go through all of these now 'cause I'm out of time, but you can stop by the-- our big red booth downstairs, and we'll be happy
- 20:48
to talk to you about any of these. Um, but, you know, the good news is, I think many of you probably have access to some of this stuff already.
- 20:55
Uh, SonarQube is one of the most widely adopted verification tools in existence today. We have over seven million developers around the world using us, and we analyze close to seven hundred and fifty billion lines of code across our solutions every single day.
- 21:06
And if there are a few of you in the room who care about Gartner at all, it's nice to know that we're a Gartner Market, uh, Magic Quadrant leader as well.
- 21:12
So, uh, final slide. Key takeaways, right? What are the things that we wanna walk away from this? First, you know, establish some bounded autonomy guidelines for your AI agents.
- 21:22
Give them the freedom to generate code, but also make sure that you're enforcing a centralized scheme of verification and constraints. Implement the AC/DC, right? Give them context, verify that they're doing the right work with independent metrics, and then use agents to solve their own mistakes.
- 21:36
Empower them to do that. Make sure your developers are equipped with the orchestration tools that they need in order to design the context frameworks and the processes that are gonna enable them to be successful and use AI most effectively.
- 21:48
And finally, standardize on a single independent multilayered verification platform that is used consistently across all projects, across all teams, across all developers, and all AI coding tools, so that you can eliminate any blind spots that might be created by using silos of tools across your organization.
- 22:05
Um, that's all from me. Uh, you can learn more by... Oh, it's off the screen now. But, uh, you can come down to our big red booth downstairs, and we'd be happy to talk to you.
- 22:12
Thank you very much. [clapping] [outro music]