Can LLMs generate Enterprise Quality Code? — Prasenjit Sarkar, Sonar

You give a coding agent instructions in English, let it generate an implementation, and then review the result. What would make that code trustworthy enough to maintain in an enterprise codebase? Prasenjit Sarkar opens with this shift in responsibility, invoking Andrej Karpathy’s description of English becoming a programming language: the human increasingly specifies the work and evaluates what the agent produces.

The tooling progression runs from VS Code and JetBrains to Cursor, Windsurf and Antigravity, and then to agents such as Codex, Claude, Devin and Gemini CLI. Sarkar cites The Pragmatic Engineer’s AI tooling survey, published in March 2026: 55% of its reader-survey respondents reported regularly using AI agents. That is a measure of the survey’s respondents, not all developers. As agents take on more implementation work, the review questions remain concrete: is the output maintainable, secure and readable?

Model vendors emphasize pass rates on HumanEval, MBPP and SWE-bench. These evaluations provide evidence about satisfying tested requirements, but passing tests does not establish enterprise readiness. Security, real-world reliability, architectural fit, engineering discipline, maintainability and accumulated technical debt require additional scrutiny. A solution can meet its immediate functional requirements while leaving expensive work for the people who must operate and change it.

Sarkar describes Sonar’s evaluation as covering more than 4,444 distinct Java programming assignments from an open-source dataset, with generated solutions analyzed using SonarQube Enterprise. The additional analysis examines defects and engineering quality rather than stopping at test success. The distinction between the dataset and the evaluation system matters: the results were made public; Sarkar corrects his initial description of the analysis as open source.

In the five-model comparison, Sarkar reports an 84.17% pass rate for Gemini 3.1 Pro High, attributing it to SWE-bench. The recording does not establish how that score relates to Sonar’s Java assignment evaluation, so it should not be read as the pass rate on the Java corpus. For that corpus, he reports 307,000 generated lines, a cyclomatic-complexity value of 234, 614 bugs per million lines and 210 security issues per million lines. The complexity value has no stated aggregation or normalization, limiting its usefulness in isolation.

Sarkar then identifies Claude Sonnet 4.6 as having the highest security-issue density in the displayed comparison, correcting an initial reference to Gemini. He reports 300 security issues per million lines and 627,000 generated lines for the same assignments. He associates GPT-5.4 and GPT-5.4 Pro High with about 1.2 million generated lines, without clearly separating the two variants’ totals.

These are Sarkar’s reported volumes for the same roughly 4,444 Java assignments. The comparison exposes another dimension of model selection: a model can generate much more code to address the same workload. More output gives reviewers and maintainers more material to inspect; it is not, by itself, evidence of a better solution.

Sarkar offers several explanations for the mixed results, beginning with the code used to train models:

• Mixed-quality examples: training data combines sound implementations with weaker code from open source and other sources.
• Insecure patterns: vulnerable examples can be learned alongside secure ones.
• Hidden logic errors: subtle mistakes in training code can contribute to generated implementations that fail or misbehave.

These are proposed explanations for the observed quality problems; the evaluation described here does not isolate their individual causal contributions.

The model adds another source of variation. Probabilistic generation can produce different implementations—and different amounts of code—from repeated prompts. Without sufficient project context, an agent also lacks the company’s data conventions, codebase structure and architectural constraints. Sarkar closes this diagnosis with explainability: when the system produces a poor implementation, understanding why it chose that implementation can be difficult, which complicates diagnosis and improvement.

The Sonar LLM Leaderboard makes the evaluation results available for comparison. Sarkar reports coverage of more than 53 models and versions, including different thinking configurations such as Gemini 3 Pro High and Gemini 3 Pro. Those configurations matter: model selection concerns the particular version and operating mode being evaluated, not merely the model family.

In the snapshot he presents, Gemini 3.1 Pro High leads on the reported pass rate; Sarkar describes its entry as evaluated on February 19. Alongside pass rate, he examines issue density, generated lines, cyclomatic complexity and cognitive complexity. The leaderboard is updated as models arrive, and its individual model pages provide more detail for deciding which tradeoffs suit a particular architecture.

Sarkar describes all five displayed models as exceeding 80% functional correctness. Once several candidates perform well on tested functionality, the surrounding quality measures become especially useful: they help distinguish implementations that a single accuracy ranking would otherwise place close together.

Cyclomatic complexity concerns control-flow paths: decisions and loops create alternative routes through a function. Sarkar explains it through if/else, for and while. Cognitive complexity, which he describes as a Sonar proprietary measure, asks a different question: how difficult is the code for a person to read, understand and maintain?

Consider a small Java example that totals positive values. The nested version makes the reader keep the enclosing conditions in mind; the guard-clause version handles exceptional paths first and leaves the main operation less deeply nested.

public final class PositiveTotals {
    public static int nested(int[] values) {
        int total = 0;
        if (values != null) {
            for (int value : values) {
                if (value > 0) {
                    total += value;
                }
            }
        }
        return total;
    }

    public static int withGuards(int[] values) {
        if (values == null) {
            return 0;
        }

        int total = 0;
        for (int value : values) {
            if (value <= 0) {
                continue;
            }
            total += value;
        }
        return total;
    }
}

Both methods return 5 for new int[] {3, -2, 0, 2}. They retain similar decision structure, but arrange it differently for the reader. That distinction is why counting branches alone cannot describe every aspect of maintainability.

Sarkar reports roughly one million generated lines for GPT-5.2 High, versus fewer than 250,000 for the older model he calls GPT-4.0, across the roughly 4,400-plus Java assignments. The earlier Sonar report names GPT-4o, so the spoken label should not be treated as a distinct, firmly identified model. His broader observation is that newer models in this comparison produce more code and can also produce higher cyclomatic and cognitive complexity. The maintenance question is therefore not just how many lines an agent writes, but how much branching and reasoning those lines impose on the next developer.

Sarkar reports increasing total bugs per model, while describing the bugs and security issues in more mature models as subtler. He credits reinforcement learning with addressing previously observed problems, but says newer output can contain finer-grained mistakes that are harder for a human reviewer to detect.

At the same time, he reports decreasing total vulnerabilities per model. These observations concern different measures: total bugs and total vulnerabilities need not move together, and fewer vulnerabilities do not mean that the remaining ones are easy to find. In his account, the types of vulnerability are shifting as the models improve. Verification must therefore keep adapting instead of checking only for familiar mistakes.

What should a development process do when the generated code does not meet its engineering standards? Sarkar introduces Sonar’s agent-centric development cycle, or ACDC, through three intervention stages: guide, verify and solve. These stages operate across an inner loop around code generation and an outer loop around committed changes and review.

The guide stage addresses inputs before asking the model to produce better output:

• Sonar Sweep: Sarkar describes cleaning problematic training data at its source, so insecure or defective examples are not simply carried forward into generation. The product was in private beta.
• Sonar Context Augmentation: supplies relevant project context to the coding agent so its output can better fit the existing codebase. Sarkar describes this broadly as giving the LLM the entire codebase’s context; the product’s selective guidance should not be understood as literally inserting every file into every prompt.

Context Augmentation was already announced in open beta by the time of the talk, distinct from Sweep’s private-beta status. The underlying mechanism is to improve both the examples a model learns from and the project information available when it generates code.

The verify stage brings SonarQube analysis into the agent’s working loop. Sarkar presents SonarQube Agentic Analysis as an open-beta capability available through MCP to compatible tools such as Claude, Codex and Gemini CLI. Here, analysis during the agent’s run means inspecting the generated code before it is committed—not waiting for the application to execute in production.

Sarkar claims 1–5 seconds for this pre-commit analysis, compared with 1–5 minutes for CI; he does not specify workload conditions. The workflow uses the earlier feedback to repair code while the agent is still working on it:

1. Generate the implementation in the coding agent.
2. Request analysis before committing or opening a pull request.
3. Return the reported issues to the agent and have it repair them.
4. Commit the changes and submit the pull request.
5. Run the pull-request analysis as well.

Earlier feedback supplements the later gate. The pre-commit loop does not remove PR analysis; it gives the agent an opportunity to resolve findings before they reach that stage.

The solve stage handles issues that still reach the pull request. If SonarQube finds problems and the quality gate fails, Sarkar describes using the SonarQube Remediation Agent to request fixes for the issues in that PR. The agent was presented as being in open beta.

The same remediation approach extends to existing technical debt. In the SonarQube dashboard, a developer selects issues and assigns them to the agent. For this backlog workflow, Sarkar describes creating one pull request per selected issue. These are proposed changes: developers review them, approve those they accept, and merge them.

Before returning a fix, the remediation agent runs the proposed change through analysis and compilation again. Sarkar says it discards fixes that produce issues, framing this verification loop as protection against regressions. Analysis and compilation can reject detected problems, but they do not establish the absence of every behavioral regression; developer review remains part of the process. The mechanism is a checked proposal, followed by human acceptance—not an automatic merge based solely on the model’s confidence.

Sarkar closes with the integration reach of the guide, verify and solve portfolio. He claims support for programming languages and frameworks, alongside broad DevOps integration, IDE partnerships and marketplace availability. These connections place guidance, analysis and remediation inside the tools through which code is generated, submitted and reviewed. Trust in generated code comes from that surrounding engineering process: relevant context before generation, explicit checks on the result, and reviewed repairs when those checks find problems.