How to Build Quality Gates into Agentic Coding Workflows — Nnenna Ndukwe, Qodo AI

An AI-generated change can go wrong before there is any code to review. Planning can miss a constraint; implementation can violate a convention; testing can overlook a failure mode. Nnenna Ndukwe, developer relations lead at Qodo, opens with that full development lifecycle because the practical question is personal: when can a developer say, “I can put my name on that”?

The first gate is defining what quality means for the team and writing that definition somewhere an agent can retrieve it. Standards may live inside a repository or in a shared context management layer. Either way, they need to reach the interfaces where work happens: the CLI, IDE and CI workflows. A standard that only exists in a developer’s head cannot reliably guide an autonomous coding session.

The workshop’s organizing decision is to reuse those standards during planning, code generation and review. Rules describe the expectations; skills guide the agent’s work. Carrying both through the workflow gives later checks the same reference point as the original implementation request.

SignalPay keeps the application deliberately narrow: a self-contained FastAPI payments API with no frontend to build. Its repository includes AGENTS.md, a hidden planning directory, rules, skills, documentation, application code, tests, Semgrep checks and CI verification. After cloning the workshop repository, the local setup uses make setup and make run. Getting the existing app running establishes a starting point before the agent changes it.

The quality checkpoints have different jobs:

• Standards selection. Identify which payment rules apply to the requested change.
• Planning and tests. Describe expected behavior, choose implementation skills and use test-driven and behavior-driven development to guide the work.
• Local verification. Run Ruff, Pyright, Bandit, pytest and Semgrep, with conventional-commit checks also included in the workflow.
• Pull request review and remediation. Inspect Qodo’s findings, document fixes or deferrals, and rerun verification after changing the code.

Deterministic tools and AI review contribute different kinds of evidence. The local tools check defined properties of the candidate; AI review examines the change using relevant rules and skills. Passing the local run therefore advances the change to another checkpoint rather than completing the whole process.

For the shared-context part, Qodo connects to the fork through its GitHub Cloud integration and ingests repository rules and skills for later review. Ndukwe recommends granting access only to the workshop fork. The demonstrated setup also creates a portal API key and configures it locally through .qodo/config.json; Qodo skills can be installed separately, while workshop skills are already present in the repository. The purpose of the connection is continuity: review should receive the expectations used during planning.

Participants choose either a refund workflow or capture retry handling. Both are small payment mutations where plausible code can still introduce consequential errors. Ndukwe ultimately chooses the refund path. Before implementation, Codex reads AGENTS.md and the rules, selects relevant local rule IDs and explains why they apply. It also discovers that the repository’s workshop skills have not been installed and runs the installer, which copies committed skill folders into the agent’s skills directory.

The planning request asks for two artifacts before code is written:

• High-level implementation plan. State the problem, constraints, scope, behavior scenarios, applicable rules, skill routing, verification gates, failure and recovery rules, and commit plan.
• Build session execution plan. Turn that intended behavior into an ordered implementation session, including the test plan and an explicit handoff to the first implementation skill and its test-driven development prompt.

For the refund task, Codex selects the repository’s planning skill and payment idempotency skill, then inspects the current application behavior in read-only plan mode.

What makes the execution template useful beyond a task checklist? It records the starting branch and current verification process, names the commands that determine success or failure, and specifies the implementation handoff. Completion notes then record which commands actually ran and what was deferred or changed. This keeps the intended procedure and the session’s actual outcome together, including decisions that might otherwise disappear into chat history.

The generated refund plan contains the summary, associated repository rules, both planning layers and assumptions. Only then does implementation begin. The handoff matters because “implement the feature” now refers to a defined behavior, a set of constraints and a verification procedure rather than an isolated request.

While the refund implementation runs, the discussion turns to skill selection. Python projects may need typing, Pydantic or Pyright guidance; implementation audits may examine abstractions, names, dead code, errors, boundaries and drift. These are different review concerns, so the useful combination depends on the repository’s language, framework and task.

Local code review is another opportunity to address small problems before another developer sees the pull request. Ndukwe recommends it regardless of whether the reviewer is built into Codex, Claude Code or another tool. She reports that a separate, deliberately large change of over 1,900 lines produced one reliability-bug finding in Qodo’s automated review after using this workflow. That is an encouraging experience, but the number of findings does not establish how many defects remained undetected.

Reusable context has several scopes: a planning template can apply across repositories, broad engineering standards can span languages, and Python-specific skills can serve the repositories that use Python. Qodo’s rules portal also separates category from severity. Accessibility, correctness, maintainability, architecture and security describe the concern; severity determines how strongly the organization wants to respond.

Scoping rules introduces an organizational choice. The demonstrated product supports organization and repository scopes. An audience member asks about assigning rules through code ownership because different teams own projects end to end. Ndukwe does not confirm that capability and offers to verify it with the product team. Repository scope is the supported answer given in the workshop; ownership-based routing remains unresolved.

How does a convention become a rule? Explicit rules already written in a repository’s rules directory can be ingested into the portal. Inferred conventions take a different route: Qodo analyzes the codebase and proposes rules that were never explicitly written down. Those suggestions do not automatically become active rules.

That distinction preserves a human decision between observing a pattern and requiring it. Existing code supplies evidence of a convention, while a person decides whether the convention should govern future changes.

What can the reviewer inspect before accepting a suggestion? The editable rule includes good and bad code examples, a category and warning levels. A person can revise those details, accept the suggestion and activate it. Examples make the intended distinction concrete for agents; category and severity describe how the organization wants violations handled.

Selecting an active rule for a particular task is another operation. In this small workshop, most or all local rules relate to the two available payment workflows. More generally, Ndukwe describes Qodo’s get-rules skill as calling the backend and using an agentic retrieval-augmented generation process to filter for relevance. Having a rule available and selecting it for a given change are separate steps.

The implementation opens a draft pull request titled “Add refund workflow with quality gates.” Ndukwe marks it ready for review so Qodo can analyze it without merging. The PR receives a generated description and change diagram. She also returns to the explicit verification prompt: Codex has already run verification because of the repository setup, but the intended process calls for checking after implementation and local review.

While analysis runs, an audience question challenges the value of generic skills such as “write DRY code.” Ingesting a skill does not demonstrate that it improves results. Ndukwe explicitly says Qodo is not evaluating skill quality and recommends Tessl for skill evaluations, including effectiveness and security concerns. The workshop therefore separates using skills from measuring whether they help.

An earlier PR supplies a concrete rule violation while the live review is pending. The organization requires docstrings on Python functions, and Qodo identifies a maintainability violation as “action required.” That urgency comes from the organization’s configured severity. A team could treat the same convention as advisory or require it before merging.

The audience presses on the word deterministic. Two mechanisms need separating:

• Rule selection and detection. AI review determines which rules are relevant and whether the change violates them. A relevant organizational rule can be missed.
• Merge enforcement. Once an action-required or high-severity issue is identified, configuration can block the merge until it is fixed and Qodo verifies it again.

A reliable response to a detected finding does not guarantee that every applicable rule was checked. Ndukwe acknowledges this limitation and describes stronger determinism in rule application as a desired capability, particularly for regulated customers.

Verification after remediation also needs an explicit instruction. Asked whether reruns happen automatically, Ndukwe says the agent’s behavior can vary and recommends prompting it to rerun. A passing result belongs to the code that was checked; a subsequent fix creates another candidate that needs checking.

The live review returns five invoked skills and six findings: one bug, three rule violations and two skill insights. The concrete issue discussed is refund idempotency: the implementation uses a process-local dictionary to store and replay refund results. Why inspect the code beside the finding? The storage choice explains the concern. A dictionary can retain results within that process, but it is not persistent storage; losing the process loses the replay information on which that implementation depends.

The finding connects the relevant code with rule and file evidence. Remediation can start from the supplied agent prompt for an individual issue or from the Qodo PR resolver skill. An audience member repeats the central failure case: fixing one issue can break something else. Automated remediation therefore needs verification again, rather than treating an applied edit as a completed fix.

The resolver runs against pull request number four and retrieves the review findings. It offers two levels of control: review each issue individually and plan its fix with the agent, or auto-fix all. Ndukwe chooses all six for the workshop. That choice speeds the demonstration while leaving individual review available when a developer wants to decide how each finding should be addressed.

The resolver’s stated next steps include verification, commits and a summary posted back to the PR. Commit formatting has its own automated check. The PR update is intended to record the changes and mark resolved issues, connecting the original findings to the remediation rather than leaving the explanation only in the agent session.

The closing questions clarify what the review system handles. For parallel workflows that touch the same files, Ndukwe uses Git worktrees; Qodo itself does not coordinate those edits. Local code review remains a recommended checkpoint, though she says she skipped it in this demonstration because of time. The Qodo-specific local review workflow is described as forthcoming and would call its backend, so running a review from a local interface should not be confused with wholly local processing.

Repository context also extends beyond the patch. Qodo temporarily clones the repository on its backend to understand the broader code around the diff. Later, in response to a question about code remaining outside a public cloud environment, Ndukwe says Qodo offers on-premises deployments. These are separate deployment considerations: where the review is initiated, what code it reads and where that processing runs.

The reusable starting point is the workshop’s structure: ask an agent which verification steps would help an existing repository, then make rules, skills and engineering standards available at the stages that need them. For long sessions, Ndukwe generally lets context compact and reports little degradation in her recent experience; when needed, she starts a new session and carries selected context across. That is a working preference, while the plans and completion notes provide an explicit place to retain the task’s decisions.

The final fix summary arrives before the workshop ends, and Ndukwe reports that all issues were resolved. What changed in the resolver’s final summary is the useful result to inspect here: the six review findings have reached a reported resolution. The recording does not explain the replacement storage design or individual fixes, so that result supports completion of the remediation exercise rather than a claim about the refund implementation’s production readiness.