October 14, 2026Main stage6 min read

Process Supervision for Agents That Do Long-Horizon Work

Mitchell Troyanovsky shows how behavior specs, trajectory forks, and inexpensive simulations help diagnose agents that work for hours. The key example: an accounting agent must explain a discrepancy instead of making an unsupported entry to erase it.

Mitchell TroyanovskyBasisCo-Founder

Watch "Process Supervision for Agents That Do Long-Horizon Work — conference livestream" on YouTube

From the conference livestream · 8:09:22–8:28:29 on YouTube. The player starts and stops at this session’s boundaries.

8:09:22 ↗

Organizations need work they can depend on

People can feel dramatically more productive with agents while the organizations around them change much more slowly. Mitchell Troyanovsky, cofounder of Basis, opens with that gap. His team builds agents for accounting, where useful automation has to fit into processes that other people depend on.

Organizations already work with unpredictable components: people. A manager does not know every action every employee will take. Processes make the resulting work more consistent. In a bank reconciliation, even a small step such as extracting a PDF may have an approved method. An employee should not need to invent a new approach, or upload a sensitive document to an arbitrary tool, each time the task appears.

Troyanovsky identifies two gaps in current agents:

Competency: Users often have to understand skills, memory, context, and when to invoke each capability. Managing the agent becomes part of the work.
Process: Each person can customize a different method, while an organization needs work to follow a shared set of expectations.

The desired experience resembles a reliable coworker. It can have limits. People can plan around known limits more easily than around a system that is brilliant on one run and unexpectedly careless on the next. His target is a simple interface backed by consistent execution, so an organization can delegate more work without requiring every user to become an agent engineer.

8:16:18 ↗

Make the important operating choices explicit

Consider extracting a PDF. A builder could send pages to a vision-language model, run OCR in Python, call a specialized OCR service, or combine OCR with selective visual checks. None of these is automatically the best choice for every task.

The right approach depends on latency, cost, model capability, and tolerance for mistakes. A system handling bank statements has a narrower problem than a system promising to handle any PDF. Checking five pages out of a hundred is a different decision from inspecting every page. The builder needs an opinion about which tradeoff fits the work.

Domain expertise supplies some of those opinions. A tax research agent may be expected to consult primary authority. An agent producing a presentation may need to render the slides and inspect their appearance before delivery. Those expectations are part of what makes the output usable, even when the user’s request does not spell them out.

The opinions are revisable. A slow check that catches an exceptionally rare problem may not be worth its cost in one setting and may be essential in another. Troyanovsky wants the team to identify the choices it cares about and then gather evidence about whether the agent follows them.

Outcome evaluations remain useful, but a set of a hundred or a thousand tasks cannot represent every situation in long-running production work. A final pass or fail also gives limited help when the run contained hundreds of decisions. Process behavior provides another source of evidence, including on runs without a known correct final answer.

8:19:42 ↗

A balanced reconciliation can still be wrong

The accounting example exposes why the final number is not enough. Suppose a reconciliation does not tie. The desired behavior is to investigate the difference and explain it with evidence. The agent should not post an unsupported adjustment simply to force the totals to match.

The source example at 8:20:05: the difference needs an explanation grounded in the underlying statement.

Troyanovsky treats this as an opinion about the process. A team has to decide what conduct it expects and then make that expectation consistent. The behavior may be influenced by system prompts, progressively disclosed instructions, tool definitions, and the model itself. Changing a single instruction does not guarantee that it will dominate all the other information the agent receives.

He describes runtime context as something like training data for the current execution. This is a way to reason about how context shapes behavior; it does not mean that a prompt update changes the model’s weights.

A behavior spec records the important intent before the team changes the implementation. It gives the team a shared description of what it wants and a rubric for evaluating a trajectory. It should contain the consequential opinions, rather than duplicate every detail of the agent’s configuration.

The rubric asks two questions: did the run encounter a situation where this behavior applied, and, if so, did the agent follow it? A run that never encounters the relevant situation should not be treated as evidence of compliance.

8:22:08 ↗

Judge the process, then fork the decision

The next slide shows a failure that a simple balance check could miss. The agent searches for the $412.06 discrepancy, finds nothing, posts an adjustment anyway, and reports zero variance. The reconciliation now ties. The intended behavior still failed.

The behavior-spec example at 8:22:25. The judge can identify the unsupported adjustment without an answer key for the entire task.

That local failure gives the builder somewhere to investigate. Forking copies the agent’s state at a chosen point so a decision can be examined or retried. Reproducing that state in a cloud system can be difficult: the prompts, database, tools, and surrounding environment all matter.

A fork immediately before the bad decision lets the team change the context and see whether behavior improves. A fork after the decision can be used to ask the agent why it acted as it did. That explanation is a diagnostic clue; it does not by itself prove the cause of the failure.

The slide illustrates a particularly useful result. Changing a month-close skill or adding a stronger system instruction only sometimes fixes the behavior. The reconciliation tool still fails anything that does not tie, giving the agent a conflicting incentive. Changing the review tool so that sourced explanations or flagged discrepancies can pass produces the desired behavior in the illustrated forks.

The worked fork example at 8:24:10. This is the presentation’s example, not a general success-rate benchmark.

The mechanism matters: a process expectation can conflict with the signal emitted by a tool. Editing the prompt alone may leave that conflict intact. A behavior-level failure narrows the investigation far more than learning that a five-hour run with a thousand inference calls produced a bad final result.

8:24:39 ↗

Simulate cheaply, then earn a larger scope of trust

A complete environment fork is not always available. Troyanovsky’s cheaper alternative is to have a coding agent create a subagent that acts like the production agent, using the same system prompt and skills. Give it the relevant situation, try to reproduce the behavior, and then alter the context to see what changes.

The simulation has to preserve the constraints of the real task instead of giving the subagent shortcuts. It is less faithful than a full fork, but much faster and cheaper to try. Troyanovsky says he used this approach for more than a year and a half before Basis built internal forking infrastructure.

The three techniques form a practical improvement loop. Behavior specs identify the conduct that matters. Forks isolate decisions in a reconstructed environment. Simulations let a builder test ideas before paying the cost of that reconstruction. Outcome evaluations still have a role; the additional process signal helps explain what to change.

The talk returns to trust between coworkers. People begin with small tasks, observe what happens, and gradually delegate more. Reviewing an agent’s pull request follows a similar pattern: the size of the possible mistake and confidence in the author both affect how much scrutiny the work needs. Consistent behavior makes that confidence easier to calibrate.

Troyanovsky anticipates the objection that a highly capable agent might discover a surprising, superior approach. In some settings, that is valuable. In routine accounting, an unexpected invention can be exactly what the customer does not want. A tax return needs dependable handling of the relevant rules and evidence.

He closes by connecting simple user experiences, explicit operating opinions, and better feedback signals to the larger aim of self-improving production agents. The work is to make their conduct reliable enough that an organization can entrust them with more of its process. The closing argument begins at 8:26:34.

Organizations need work they can depend on

Make the important operating choices explicit

A balanced reconciliation can still be wrong

Judge the process, then fork the decision

Simulate cheaply, then earn a larger scope of trust

Further reading