The maturity phases of running evals

Building a generative AI proof of concept is one thing; gaining enough confidence to put it in front of real users is another. Phil Hetzel encountered that gap while consulting: his customers were prolific at creating prototypes, but much less successful at bringing them into production. He describes a path through KPMG and Slalom, where he led the global Databricks business unit, before adopting Braintrust and eventually joining it to lead solutions engineering. That experience frames the practical question behind evaluation maturity: what evidence does an increasingly capable agent need before you can trust it with real work?

Evaluation and observability form a continuous confidence loop. In Braintrust’s framing, evals build confidence before deployment; observability maintains it afterward. They address the same agent-quality problem from different points in the lifecycle, and the infrastructure supporting them must evolve as the underlying technology changes.

Quality means that an agent behaves as expected when it encounters real users and real usage. The risks extend beyond an incorrect answer: an unkind or unhelpful response can damage a brand, excessive resource consumption can make the system expensive, and inappropriate behavior can create compliance or legal exposure. Evals also support improvement. For each change to an agent, they provide evidence about whether the application got better and by how much.

Begin with the failures that matter. A builder or subject matter expert can identify where an agent goes wrong and construct evaluations around those cases. Hetzel contrasts this with an exhaustive unit-testing mindset: the point is not that unit tests literally cover every possibility, but that enumerating every possible agent failure is an unbounded task. Trying to do so leaves little time to ship.

Evaluation results can be useful without being perfect. An LLM-as-a-judge evaluator may produce nondeterministic judgments, yet still provide directional evidence that a change is helping. That makes the evaluation’s structure especially important: you need to know what ran, which examples exercised it and what the scores actually assessed.

An example starts the workflow; the task produces behavior to inspect; the scoring functions assess that behavior. Keeping these responsibilities separate makes it possible to change the agent while retaining a useful basis for comparison.

The maturity framework follows increasing complexity rather than a set of certification gates. Its four areas are getting started, measuring to manage, accounting for complexity and advanced evaluation techniques. As agents gain capabilities, they gain additional ways to fail, and their evaluations must expand accordingly. These are overlapping areas on a continuum, not rigid stages every team must complete in the same way.

Manual inspection is a reasonable starting point if it leaves a record. Hetzel’s starter example is to run roughly ten inputs through an agent and inspect the outputs. This is an illustrative batch, not a statistically validated sample-size requirement. The reviewer can be the builder, but a subject matter expert is preferable when recognizing a good response requires domain knowledge.

For each response, collect two things:

1. A thumbs-up or thumbs-down judgment.
2. A written explanation of that judgment.

The explanation is the reusable part of the expert’s knowledge. A negative rating tells you that something failed; its justification tells you what the reviewer noticed and why it mattered. Capturing that reasoning creates material for an automated evaluator later, instead of leaving the quality standard inside the reviewer’s head.

An audience member describes working without evaluation infrastructure: run an agent on their data, inspect the result, create a prompt and repeat. The immediate improvement is to preserve what that loop teaches you. A trace comes in, someone judges it, and the justification becomes part of the record that can inform a future judge.

The review interface matters too. Hetzel shows Braintrust’s human annotation view and describes custom annotation views that users can generate and tailor. A generic trace dump may not match how an expert understands the work. Present traces in a form reviewers recognize so they can evaluate them appropriately and contribute useful feedback.

Once explanations accumulate, use them to identify recurring failure modes. Hetzel suggests analyzing the reasons behind negative ratings with tools such as Cursor, Claude Code or Codex. The goal is to derive explicit descriptions of what goes wrong, then automate assessments that previously depended on a small number of experts.

An LLM judge can scale assessments that require interpretation, but assigning a model the role of judge does not make its decisions trustworthy. Its outputs also need evaluation. For objective failure modes, ordinary code may be a better fit: an agent that exceeds a tool-call budget or consumes too many tokens can be checked deterministically.

A small TypeScript scorer can express those two budget checks directly. The limits are inputs because the acceptable budget depends on the application:

type Usage = {
  toolCalls: number;
  totalTokens: number;
};

type Limits = {
  maxToolCalls: number;
  maxTotalTokens: number;
};

function scoreUsage(usage: Usage, limits: Limits) {
  return {
    toolCallBudget: Number(usage.toolCalls <= limits.maxToolCalls),
    tokenBudget: Number(usage.totalTokens <= limits.maxTotalTokens),
  };
}

Each score is 1 when the corresponding budget is respected and 0 when it is exceeded. Keeping them separate preserves which failure occurred; neither score attempts to decide whether the answer was helpful.

At this point, the evaluation dataset should include production traces—or at least traces from user acceptance testing. The purpose is to exercise the workloads the agent actually encounters. Think of evaluation as rerunning production: capture real examples so offline experiments can test changes against real usage.

That creates an improvement loop:

1. Observe: capture agent traces in production.
2. Analyze: use human review or automated tooling to understand failures.
3. Evaluate: bring those examples into an offline experiment and rerun them.
4. Improve: use the results to guide changes to the agent.

The same production behavior that exposes a weakness becomes evidence for deciding what to change. This connects observability to evaluation and gives the evaluation process a role beyond blocking regressions.

LLM judge scoring extends this loop beyond the throughput of human-only review. The scoring-function example here illustrates the next step after collecting expert judgments: make the assessment repeatable across more examples, while retaining the quality criteria extracted from those judgments.

Once an agent interacts with external systems, evaluating only its final response becomes insufficient. Hetzel distinguishes two useful categories of tools:

Both can improve an agent’s usefulness, but each external interaction adds opportunities for failure. The distinction directs attention toward what a call contributes to the model’s context and what it does to the surrounding system.

The unit of inspection expands from the output to the entire agent trace. Capture the sequence of steps, inspect how the agent arrived at its result, and target evaluations at individual tool or MCP calls where necessary. A final answer alone cannot expose every problem in the path that produced it; trace-level visibility supplies the intermediate behavior that those checks need.

External systems create two distinct replay problems. First, an offline evaluation must represent the state those systems were in when the original input arrived. Their current state may be different, and the offline environment may not have access to them at all. Second, replay must not overwrite production data while exercising the agent’s create, update or delete operations. Hetzel describes this as an incompletely solved problem, with approaches that approximate production rather than guarantee a perfect reconstruction.

Mock APIs can provide a place to exercise interactions against a representation of external state. Hetzel then proposes treating the trace as a container for much richer context than conventional application tracing often carries. Store relevant system state alongside the original execution and inject it into the task during evaluation. That can encapsulate part of the replay environment inside the example itself and reduce the separate test infrastructure required. His description of potentially very large traces is a design premise here, not a demonstrated storage-capacity guarantee.

A further option is to query the external system as it existed at the original execution time. For example, a vector database might support timestamp or version queries that recover the retrieval state associated with a captured input. This depends on the database’s capabilities, configuration and retained history; it is not a universal vector-search feature. Where available, it addresses a specific replay mismatch: today’s retrieval results may not be the results the agent could see when the example was recorded.

The prepared material closes with two emerging patterns. Topic modeling across production traces could help uncover recurring failure modes automatically. Coding agents such as Claude Code, paired with an evaluation provider’s CLI, could automate more of the evaluation workflow. These are directions for further development in the talk, not demonstrated implementations. The accompanying slide also lists simulated user interactions, without developing that third idea in the spoken explanation.

The closing audience question brings the discussion back to the reliability of scoring: should evaluation platforms favor deterministic graders, or embrace LLM judges? Hetzel’s answer starts with the nature of the criterion. Some aspects of agent behavior are subjective, so an LLM judge can be appropriate. The requirement is to evaluate that judge extensively against what a human would decide in the same circumstances.

A concrete way to do that is to constrain the judge to discrete decisions and create a ground-truth dataset of human judgments. Those labels make the judge’s decisions easier to compare with the intended standard. Discrete output is a property of this chosen rubric, not an inherent property of every LLM judge. Deterministic checks still handle objective conditions; subjective judgments gain credibility through demonstrated alignment with human decisions.