Building Reliable Agentic Systems

A code review follows a relatively constrained workflow. A request to migrate a codebase or implement a feature leaves far more decisions open. Factory’s Droids span that range: review, documentation, testing, refactors, migrations, and feature work. In Eno Reyes’s account, ReviewDroid and DocumentationDroid operate with more guardrails, while CodeDroid attempts coding tasks expressed in natural language. Those differences call for different cognitive architectures rather than one universal agent loop.

Three capabilities organize the engineering problem:

• Planning: Choose one or more future actions.
• Decision-making: Apply criteria and a decision procedure when the available choices are broad.
• Environmental grounding: Read from and write to an external environment.

The third capability makes the first two operational. An agent’s plan and decisions must remain connected to the environment in which its actions actually take effect.

Even migrating a small part of a codebase can involve hundreds of steps. Across a long sequence, an agent can drift: its reasoning changes, and later decisions stop fitting the assumptions behind earlier ones. Factory borrows an idea from control systems and robotics that Reyes calls a pseudo Kalman filter. The mechanism described here is passing intermediate reasoning between plan steps; the name is an analogy, not a specification of a mathematical Kalman filter.

Carrying that reasoning forward gives each decision access to the rationale accumulated so far. How the system modifies or shares it can become sophisticated, but the immediate benefit is continuity: individual decisions converge toward consistent reasoning. The corresponding danger is error propagation. If an early conclusion is wrong, preserving it can make subsequent steps consistently wrong. Reasoning continuity therefore improves coherence without establishing correctness.

Sub-task decomposition gives the system smaller decisions to make. Reyes places it in the lineage of approaches such as ReAct, which interleaves reasoning and actions while allowing plans to change as information arrives. That connection does not require a fixed hierarchical decomposition. Factory experiments with the structure and resolution of sub-tasks: finer steps clarify the action space and give the system more precise control.

The trade-off is the number of decisions introduced. Breaking every operation into tiny tasks can leave the model with more choices to get right, even if each action is narrowly defined.

The useful resolution is the one that makes the workflow easier to execute reliably, not simply the one with the most detailed plan.

Model predictive control supplies another planning idea: evaluate sub-task outcomes against the current state, then adapt the remaining plan using feedback received during execution. This matters when humans continue developing software while an agent runs a long workflow. The environment can change underneath an otherwise coherent plan.

The practical sequence is to observe what happened, update the trajectory information, and replan from the updated state. Reyes emphasizes that useful improvements do not require implementing the full Bayesian or predictive-model machinery associated with control theory. Keeping feedback and planning current is itself valuable.

Simple agents often start with explicit plan criteria. As builders pursue more general behavior, they may remove those constraints. Factory’s experience suggests retaining useful definitions of successful plan structures and acceptable initial states. These give the system a more reliable starting point before a long trajectory unfolds.

Criteria can enter the system through several mechanisms:

• Instruction tuning: Teach the model the desired planning behavior.
• Few-shot examples: Show concrete examples of acceptable plans in the prompt.
• Hard-coded validation: Check plans against explicit domain rules.

All three aim to reduce errors and keep trajectories successful for longer. The cost is that explicit logic takes work to build and becomes difficult to scale.

Domain constraints are part of delivering a working agent. Reyes connects this choice to lessons from symbolic AI: when the goal is to solve a customer’s concrete problem, successful generalization is not the only design objective. Factory deliberately uses explicit criteria and hard-coded logic where they help deliver useful behavior today.

At a decision point, the system does not have to accept its first model output. Consensus approaches include self-consistency, prompt ensembles, and cluster sampling. They generate multiple candidates and use a selection or aggregation mechanism to choose among them. Factory found this useful for producing consistent, high-quality decisions; the benefit depends on selecting useful samples, not merely increasing their number. More runtime inference costs more, and serial calls can increase waiting time.

A separate approach makes the decision criteria explicit. Reyes groups together checklists, chain-of-thought reasoning, analogical prompting, and Chain of Density. These methods are not interchangeable: the matched Chain of Density work concerns summarization, rather than establishing a general decision procedure. The practical idea here is to give the model a clearer structure for deciding.

Consider choosing left or right. A checklist can describe what makes a left choice reasonable and what makes a right choice reasonable. The model can then compare the situation with those criteria instead of making an unconstrained choice. This becomes harder as the domain broadens or the action space grows: a comprehensive checklist may no longer be practical, motivating exploration of more flexible reasoning approaches.

Fine-tuning becomes attractive once there is data for a specific decision. Reyes sketches a direct workflow:

1. Select an open-source model.
2. Generate training examples using a high-quality model.
3. Have team members validate those examples.
4. Fine-tune the selected model for the target decision.

This is an option for a well-defined decision problem, rather than a prerequisite for every agent.

Training introduces expense and ties the system to the quality of the resulting specialized model. A system that can sample from interchangeable general models can more easily benefit when a stronger model becomes available. Fine-tuning gives up some of that automatic improvement, but Reyes finds it particularly useful for decisions that are out of distribution for the base model.

Simulation adds a way to inspect possible consequences before selecting a path. Its feasibility depends on the domain. Software is favorable because code can be executed, and candidate execution trajectories can also be reasoned through. Factory uses both real execution and LLM-imagined execution paths when exploring decisions; an imagined result is not an observed execution outcome.

Once several paths are available, search can help determine which to pursue. Reyes cites Language Agent Tree Search and describes its relationship to Monte Carlo tree search informally, with a qualification that his sketch is not exact. The published method does integrate Monte Carlo tree search with language-model evaluation and reflection. The engineering question is how to use the simulation results to choose the next move.

These techniques need an evaluation of the decisions the system is supposed to make. Apply a technique, inspect whether those decisions improve, weigh its costs and drawbacks, and adapt. Neither extra sampling nor a more elaborate search procedure substitutes for measuring the behavior that matters in the application.

Environmental grounding begins with tools, or what Reyes calls AI-computer interfaces. Existing tool ecosystems provide useful starting points: calculators, Python execution, and open-source implementations of artifact-style experiences associated with Claude. Reyes mentions LangChain and sandboxed execution in this context; a Python execution tool still needs an actual isolation boundary to constitute a sandbox.

Custom interfaces become useful when the available tools do not match a recurring workflow. Consider the sequence Reyes describes:

1. Run a calculation.
2. Pass its output to another system.
3. Read and parse that system’s logs.
4. Transform the parsed result.

If this sequence is predictable and frequently needed, asking the agent to reconstruct it every time adds unnecessary decisions. Package the operation as a tool and give the model an interface through which it can invoke the combined workflow.

Software development already has an enormous collection of capable tools. Their interfaces, however, may be designed for a person using a CLI or navigating to a definition in an editor. An agent needs both a way to invoke those capabilities and a way to interpret their outputs. Adapting those interfaces is a substantial part of Factory’s work: the underlying developer capability may already exist, while its usable interface for an LLM does not.

A tool returning data does not complete the feedback loop. CI/CD logs illustrate the gap: they can contain test results, debug statements, and large amounts of information irrelevant to the next decision. If the agent will need that output, design an explicit processing path for it. That path may use a tool, programmed decisions, or a dedicated LLM interpretation step.

For the log example, extract the failing tests and provide a brief explanation of what the rest of the logs contain. The result gives the model the failures it needs to act on while retaining context about the surrounding output. This is a deliberate interface between environmental feedback and the next decision, rather than an expectation that the agent will always find the important evidence in raw logs.

The same principle applies to feedback generated inside the agent. When it reflects on an error trajectory, process that trajectory into meaningful information before passing it back to the model. Reyes cautions that responding to an instruction to criticize itself is not the same as reliably diagnosing its own actions. A useful reflection loop needs informative feedback, not just a request for self-criticism.

Agents need opportunities to gather context about the problem, often near the beginning of a workflow, though not exclusively there. Long-context models make it possible to retain more of that information. Reyes’s examples from the period are Gemini Pro 1.5 and Sonnet 3.5. But the capacity to keep adding context does not tell the system when to stop exploring and start solving the problem.

Bounded exploration requires evaluation. Factory found it difficult to determine the right exploration duration in advance. The agent needs time to collect data, judge relevance, and establish a useful starting state, but eventually it must act. The talk supplies no universal exploration budget; the balance depends on evaluating how context gathering affects the subsequent trajectory.

Human participation is also an interface design problem. Decide where in the workflow the system should ask a person to intervene or provide guidance. Those interaction points determine when the agent can proceed independently and when a human helps it recover or choose a direction.

Reyes suggests that careful UX and interaction design for human guidance can move reliability from 30–40% to 90–100%. These are illustrative figures, not a specified benchmark: the talk does not define the task population, reliability metric, sample size, or intervention protocol. The claimed improvement includes human participation, so it should not be read as autonomous agent performance.

That leaves a product decision as well as a technical one. Human guidance can make a workflow more dependable, while reducing how much of it is fully autonomous. Designing a reliable agent includes choosing that balance explicitly.