AI Writes More PRs. Who Validates Them? — Ali-Reza Adl-Tabatabai, Sonar

Let's get started. Um, good afternoon, everyone. I'm Ali, and, uh, I'm one of the founders of Gitar.ai and the previously the CEO up until about a month ago, where we were acquired by Sonar, and now we are part of Sonar. And you've heard a lot in the past few days about the importance of verification in the age of AI. And so today, I'm gonna focus my talk on code and

verifying code. So in m- in almost all software teams, there is this centralized validation phase that is basically CI plus code review. And as you scale your engineering, this plays-- this phase plays a very critical part, uh, especially in an AI world. Uh, first, it enforces all your quality gates centrally. So these are security, compliance, code quality gates. It's on the critical path of getting your

changes into production, so the faster is always the better. But as you scale engineering, this becomes kind of a challenge. As, as you have longer running builds and tasks, you have asynchronous team review flows across sites and geographies, and things just get slower naturally as you grow. Um, this is also a phase where you end up spending a lot of developer infrastructure dollars. So there's a lot of expensive tools here that need to be integrated and

scaled up as you, um, scale your organization. And it all runs on very expensive infrastructure. Uh, for these reasons, it's oftentimes the main focus of your platform organization team or your platform team. Um, but any kind of failure in this phase is very expensive. It introduces... Failures introduce an outer loop that are typically measured in hours and days. So

failures are expensive. They costly. They really slow down your velocity. Uh, you lose developer time, and that's compounded by disruptions to your flow state. Uh, it's also very expensive to optimize. So you've got a lot of fragmented systems that have been integrated together with bespoke configuration and scripting, and you have a usually headcount restricted, constrained platform organization trying to keep it all up and running for you. And, uh, trying to address that all with

AI is also kind of complex because it requires complex workflow orchestration, uh, involving various inter-- asynchronous interaction, team workflows, integrations across tools, and so on.

In fact, AI kinda makes the problem even worse. Uh, you've got a lot of PRs now being generated, more PRs than before, uh, thanks to AI, and bigger PRs, uh, all of which can have defects. And so your costs in your development life cycle are kinda shifting to the right. You have a lot more code to review. You, you can now potential for a lot more failures in production. Uh, and so your choices are kind of tough here. Either you slow down and ask developers to review every PR very carefully,

or you rubber-stamp PRs and you risk incidents in production. Either way, you've got

less happy developers, lower sentiment, and much less productivity than, than you'd expect. So what's the answer here? Of course, it's more AI. It's, uh, an agentic approach to validation, which is essentially what we built at Gitar, um, that we're now also shipping, uh, as part of Sonar. And basically, what we've got here is an agent that fully automates all the validation steps, and as outcomes, it delivers

for you green PRs that are ready to merge or merge automatically. Now, how does this work? Well, first, when you create your PR, it reviews your PR and posts issues and inline comments. No surprise there. But our goal has been to really make this experience as noise-free as possible, both in the way we handle the UX, as well as posting issues that are real and really matter. You can customize your review with your own rules, and you can add your own custom checks, and you can trigger your own

custom workflow automations. And then you can also configure the system to block merging if there are any unresolved code review findings.

It also analyzes CI failures and posts a summary of the root causes of those failures, and in particular, it's really good at catching flaky tests, and you can even set up rules that automatically retry flaky tests if you want. That's a very popular feature. You can also automatically fix any code review issues or any CI failures, any kind of, uh, failures that happen in CI. It can fix it for you as well as address any of the issues that it raises. You can either ask it to specifically

fix an issue, or more interestingly, you can set it up to automatically loop until all the fish-- issues have been fixed in your PR, and you have a green PR ready to merge. And finally, you can automatically approve and merge PRs that are green. You can define rules and conditions that control when, uh, PRs get automatically approved by the agents and merged automatically.

So we're seeing actually a very interesting trend in our users. There's a clear trajectory towards fully automated PRs. Uh, as users use the system, as they build trust in the accuracy of the AI, what we see is they work towards full automation. First, they get reviews, and they're happy with... They build trust in the accuracy of the reviews. They see that these reviews are not only accurate, but they're raising really important issues. So then they give teeth to the agent by

allowing it to start blocking the PRs, and then they start using the autofix capabilities to automatically turn PRs green and fix all the issues that were raised by code review or any CI failures that come up. And then where the real interesting leap is beginning to happen now is users setting up rules that define when the agent automatically approves and merges PRs. So over time, as they build trust in the

system, they unlock more levels of automation, and with the additional levels of automation and with the trust, they're getting more value out of the system. Now because the system processes every single PR using AI, it's also able to uncover new types of insights. For example, uh, one of the insights that are ve- is very useful to platform teams is a categorization of the top CI failures. Where are you seeing failures in your CI system?

Where are your developers running into failures? Are these flakiness issues that you can addre- that you need to address by improving the reliability of the tasks? Or are they infrastructure issues that you need to address? Or are there other kinds of failures that they're running into? Very, very useful for platform teams. We also have... Another example of what else we have in terms of AI-enabled insights are categorization of PRs. What exactly is happening across your engineering team in

terms of the PRs that are being landed? This is extremely useful for leadership teams. For example, what amount of PRs are going to feature development versus fixing issues, chores, refactors, and so on. Now, under the hood, if you look at the system, there's a number of components that constitute Gitar. There are three interesting pieces here. One is the control plane that acts as this orchestration layer for handling PR workflows. And

then there's an agent runtime or, or harness, we built our own harness here, that handles multi-agent execution, context management, memory management, um, tool calling, all the integrations, et cetera. And having this, having our own agent harness has really allowed us to optimize specifically for the validation use case, and it's allowed us to optimize for token cost, precision, and coverage of the agent itself. So that allows us

to essentially guarantee outcomes at a fixed PR price while giving really good quality results. We also have an LLM proxy that handles the routing across different models as well as things like fa- failover. Now that we're a part of Sonar, we've been integrating Gitar with SonarQube. There's a ton of opportunity here in combining an agentic approach to validation with program analysis. So combining

agents with things like, um, taint analysis, control flow analysis, data flow analysis, software composition analysis. So all the traditional approaches combined with agents. We think there's a ton of opportunity here, and that together you can get much better precision, much better coverage at a really great price point. So the overall results of the value that we're seeing in, in all of this is much better quality, much better

security, especially with AI-generated code, faster delivery times through all the automation going from PRs straight to approved and merged PRs, much higher productivity and sentiment by developers, so less time spent on actually reviewing code and worrying about what's happening in production with more confidence, as well as sentiment because now you... Much better sentiment be- now because you have a lot of automation in place and less grunt work. And finally, the to- the features that we've built for

platform teams gives them a lot more leverage to be able to fix issues and to add customizations and see insights in this critical part of the software development life cycle.

To wrap up, PR validation is really critical, even more so in AI. But AI turns validation into a friction point, and Gitar, what we've built, uh, allows you to automate this step, fully automate PR validation, go from created PRs all the way through merge, completely limiting this friction, and helps you keep up with the rate at which you're generating code from AI. And finally, the combination of

using a- agents, uh, doing agentic validation plus an algorithmic program analysis, it will give you something that, that I think is gonna be much better than either alone. If you wanna talk more and you wanna see more, if you'd like to see a demo, come and visit us at booth number P7 right down, right down the hall there. Thank you.