Separate the policy from the running system
A policy says a change must be approved before release, or a customer must be verified before an account opens. The organization then has a running system that is supposed to carry out that rule. Keeping the two distinct is Evan Kormos's starting point for deciding where an agent belongs.
Policies change at particular moments. The operating system keeps moving: people process requests, products evolve, and technology changes around them. Kormos describes policy as institutional intent, often expressed through words such as must, may, and should. RFC 2119, one of his closing references, provides a familiar vocabulary for those requirement levels.
Agents cut across people, process, and technology. They work alongside employees, perform parts of a process, and depend on technical systems. The useful unit for this discussion is the process: what work an agent receives, what conditions must be satisfied before that work advances, and how anyone knows its output is good enough.
Mapping the workflow makes those questions concrete. Some steps have to occur in sequence; others can run independently. Agents can make parallel reviews economical where coordinating them used to cost too much. The opportunity is to add that capacity while retaining the checks required before work proceeds.
A forward-moving workflow can contain an iterative agent
A directed acyclic graph, or DAG, represents steps and dependencies without a path looping back on itself. Apache Airflow is Kormos's example of a tool that expresses these workflows in Python. The same idea can describe commercial lending, listing a token in a region, or delivering software through regulated controls.
Inside one of those steps, the work may need to repeat. Consider code review. Someone drafts a change, checks it, and uses the feedback to refine it until the result can move on. That inner process contains a cycle even though the outer workflow progresses forward.
The inner review loop at 58:20. The check decides whether to leave the loop. More iterations are useful only if that check reliably distinguishes work that can pass from work that needs repair.
Kormos calls the inner structure a directed cyclic graph and points to LangGraph as a tool for managing its state. People and agents can collaborate inside that loop. The surrounding process still determines which checks are required before the next stage begins.
Exceptions need a route too. A change might be canceled or fail a security requirement. The workflow should make that outcome visible without requiring its designer to predict every possible failure in advance.
Faster review can move the queue somewhere else
Improving one step changes the load on the others. If agents review code faster, design or security scanning may become the slow part. A team needs to observe the whole process to see where the queue moved.
The software-delivery example makes those dependencies visible: a change request fans out into code review and a security scan, then the work joins at release approval. Exceptions have a separate route.
Source frame at 1:00:00. Adding more agent capacity to the review loop cannot remove the requirement for the security result. It may instead make that requirement's waiting time more obvious.
Kormos puts operational visibility ahead of the convenience of authoring a workflow. Metrics across the process show where additional parallel capacity would help and whether the required controls still hold as volume changes.
Input quality also affects the inner loop's cost. An agent needs trusted context, including where its work sits in the larger DAG. Giving it the workflow state helps connect the current task with the process around it. Longer-horizon goals make this context and the cost of repeated attempts more consequential.
The check needs a standard of good work
The hard part of the loop is deciding what good enough means. Repeating draft, check, and refine does not guarantee improvement if the check cannot recognize it.
Traditional software already has confidence mechanisms such as unit tests run during delivery. They catch cases where a change breaks known behavior. Agents add a different evaluation problem: the same request may produce different answers, and more than one answer may be acceptable. Comparing output strings alone cannot capture that quality.
Kormos proposes representative datasets, explicit standards, and repeated evaluations. Looking at many outputs reveals behavior that a single pleasing example cannot. Traces record what happened during execution; evaluations assess whether the resulting work meets the standard. Together they complement the tests around the deterministic software.
Domain experts help define and refine that standard. An AI engineer can build the pipeline, but deciding whether a credit memo or risk review is good enough also requires the people who understand the work. The evaluation must reflect their judgment before its score can justify handing over more responsibility.
Choose a node, then earn the reduction in supervision
Return to commercial lending. Customer verification and financial analysis can run in parallel before a credit memo, risk review, and decision. Each stage offers a different opportunity for agent assistance. Letting an agent drive the stage requires a stronger case.
Kormos asks four practical questions:
- Can policy check the result? There must be a usable standard for deciding whether the output complies.
- Can the work run in a bounded loop? Iteration needs limits rather than an open-ended instruction to keep trying.
- Can the execution be traced? The organization needs to inspect how the result was produced.
- Can a human take over? An exception needs a path to someone who can resolve it.
A node that satisfies those conditions is a better candidate for automation. Agent assistance can still be useful elsewhere. In his lending example, the final decision remains human-driven because it checks the work against policy.
The rollout begins with supervision. People verify outputs, subject-matter experts refine the dataset, and the team checks whether its evaluation captures the right quality standard. Only after measured confidence reaches the policy's threshold does Kormos move toward sampling: people review a slice while systematic evaluation covers what they do not see.
Source frame at 1:07:10. The threshold on this explanatory slide illustrates a policy decision; it is not a universal score at which an agent becomes safe to operate.
After a further confidence threshold, people can concentrate on exceptions while continuing to improve the dataset and assess quality. Their role changes instead of disappearing. Fixed requirements govern the outer workflow, adaptive work happens inside its stages, and evidence determines how much supervision each stage needs.