AI Engineer World's Fair 2026
Your Agent Evolved. Your Evals Didn't.
Read the talk
Your Agent Evolved. Your Evals Didn’t.
As agents move from single model calls to tool loops, memory, and code execution, evaluations must expand from answer quality to the behavior of the whole system.
From a talk by Ameya Bhatawdekar
Before you start: Familiarity with model prompts, tool calls, and basic retrieval-augmented generation will help; no evaluation framework experience is required.
Why a better model can require a different system
Why is an AI demo easy to build, while maintaining its quality in production is so difficult? A chatbot, agent, or batch-processing application can change substantially after its first working version: prompts become pipelines, orchestration grows, and eventually the application needs a rewrite. Ameya Bhatawdekar, Braintrust’s Field CTO, frames this as a moving-target problem. Models change, users discover different ways to use the application, and the underlying data changes too.
Better tool calling, longer usable context, code generation for sandboxed execution, and practical memory systems can each change what an application is capable of doing. These are sometimes large enough advances to require re-platforming, rather than another round of prompt tuning.
Simply replacing the model may leave those capabilities inaccessible. If the old model struggled to choose tools, the surrounding software probably made those choices for it. A stronger model placed inside the same constraints still cannot exercise its new judgment. Restructuring the application can unlock that capability, but it also introduces places where behavior can go wrong. Architecture follows model capabilities; evaluations must follow architecture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with an SRE agent—and the simplest evaluation
Consider an SRE agent that can read system information and take operational actions. Its tools include deployment rollback, escalation to a human, and paging. The distinction between read tools and write tools matters: the system can change the environment, not merely describe it. Bhatawdekar uses this agent to ground a series of notional evaluations as the architecture evolves.
The starting architecture is one input, one model call, and one output. Here, the final answer is a reasonable unit of evaluation. Does it answer accurately? Are its claims factual? Does it invent information or rely on stale training knowledge rather than the information relevant to the task?
A golden dataset supplies representative inputs and expected behavior. Scorers encode what a good answer looks like and evaluate the model’s responses against that definition. This is a useful baseline precisely because the architecture is narrow: there is no retrieval stage, tool execution, or orchestration to inspect independently.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A chain creates intermediate failure points
Retrieval-augmented generation adds work before the answer. A typical chain follows a fixed procedure:
- Parse the user’s input to extract information needed for retrieval.
- Retrieve relevant information.
- Assemble that information into context.
- Ask the model to reason over the context and synthesize an answer.
The answer is now the product of several stages, each of which can affect correctness.
| Stage | Failure to evaluate |
|---|---|
| Parsing | Extracting the wrong information |
| Retrieval | Returning the wrong context |
| Context construction | Overloading the model with material it cannot use effectively |
| Generation | Failing to reason correctly over the supplied context |
A larger advertised context window does not itself establish that a model can reason well over everything placed inside it. Bhatawdekar recalls early systems in which context stuffing impaired performance even as window sizes increased. Intermediate evaluations make a bad final answer actionable: they help distinguish a retrieval problem from a reasoning problem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Letting the model choose the next step
A fixed chain performs the same kind of sequence for every request. ReAct offers a different structure: interleave reasoning and action, allowing the model to decide what to do next. The paper appeared in 2022; Bhatawdekar places its growing popularity among application builders in 2023 and early 2024.
The loop is straightforward: choose a tool call, inspect its return, reason about the new information, and select the next step. Continue until the user’s intent is satisfied or the iteration budget is exhausted. Instead of requiring a developer to predefine the entire route, the model can organize the work around the request and the observations it receives.
The early models made that flexibility difficult to use reliably. They supplied incorrect tool arguments, selected the wrong tools, made reasoning mistakes, and struggled to preserve useful context over longer executions. An appealing orchestration pattern still depended on capabilities the model did not consistently possess.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Workflow graphs buy predictability with explicit control
When the model could not reliably control execution, teams moved control into software. A graph or state machine encoded planning, orchestration, and execution; models performed narrower tasks inside individual nodes. This made behavior more predictable because the surrounding program constrained how the system could proceed.
Those constraints also defined the system’s supported distribution of requests. An intent that fit the graph could work well. An intent requiring a different sequence could expose a gap in the orchestration. Teams responded by adding branches and special cases, increasing the graph’s complexity as they expanded its coverage.
Evaluation consequently expanded beyond the answer and the individual model call:
- Branch behavior: Does the system choose the appropriate path and behave consistently along it?
- Inter-node contracts: Does one node produce information the next node can correctly consume?
- Classifier accuracy: Does an early classification send the request down the right workflow?
- Node behavior: Does each bounded model task work on its intended inputs?
- Retries and orchestration: Do retry loops and the overall execution behave correctly?
These checks supplement earlier evaluations. Moving decisions into deterministic software does not remove the need to evaluate those decisions or their interaction with model outputs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The loop returns, and evaluation becomes a distribution
Bhatawdekar places widespread production use of graph frameworks in late 2024 and early 2025. In his account, Anthropic and OpenAI releases in mid-to-late 2025 then substantially improved tool calling, orchestration, long-horizon planning, and the ability to recognize mistakes and course-correct. This is an architectural assessment rather than a model-specific benchmark: he identifies no model versions or measured success threshold.
Graphs built to compensate for earlier weaknesses could still constrain those stronger models and retain their own brittleness. The model-driven loop became practical again. Yet successful execution did not imply a fixed path: repeated runs on the same input could take substantially different trajectories while arriving at the correct answer.
A single successful trace is insufficient evidence of repeatable behavior. Run the same evaluation multiple times and inspect the distribution of results. Bhatawdekar proposes repeated trials without specifying a fixed value of k or an uncertainty calculation; the required number depends on the reliability question being investigated.
| Metric | Success condition across k attempts | Question |
|---|---|---|
pass@k | At least one attempt succeeds | Can the agent solve the task? |
pass^k | Every attempt succeeds | Can it solve the task consistently? |
The talk’s verbal description of pass^k as how many trials succeed needs a precise distinction. In the original τ-bench definition, it is the probability that all k independent, identically distributed trials succeed, averaged across tasks—not the count or fraction of successful trials. That definition dates to 2024; these metrics become useful here because the architecture makes repeated execution especially consequential.
For a single task with independent trials and success probability p, the distinction is:
More attempts increase the chance of seeing at least one success, while requiring every attempt to succeed becomes more demanding. A high pass@k therefore does not establish dependable repeated performance.
A small TypeScript helper can keep those questions separate when inspecting one batch of graded SRE-agent runs. Each boolean represents whether a run met the evaluation’s success criteria; the helper summarizes the batch, rather than estimating a population probability from it.
typescript
type BatchResult = {
attempts: number;
successes: number;
successFraction: number;
anySucceeded: boolean;
allSucceeded: boolean;
};
function summarizeTrials(passed: readonly boolean[]): BatchResult {
if (passed.length === 0) {
throw new Error("At least one graded trial is required");
}
const successes = passed.filter(Boolean).length;
return {
attempts: passed.length,
successes,
successFraction: successes / passed.length,
anySucceeded: successes > 0,
allSucceeded: successes === passed.length,
};
}
Keeping the success fraction, any-success condition, and all-success condition distinct prevents a capability result from being mistaken for a reliability result. The traces behind those outcomes then help locate the sources of failure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The agent becomes a product system
The next expansion surrounds the loop with persistent capabilities. Memory systems store and retrieve information within a session and across sessions, allowing later runs to benefit from earlier ones. Sandboxes provide an execution environment for model-generated code. The behavior being evaluated now depends on more than the current prompt and the model’s immediate tool choices.
Model Context Protocol connections and skill directories add extensibility. Skills augment the agent through symbolic instructions, expanding what it can do without changing model weights. Memory, execution environments, and these extensions all add surfaces where the product can fail. An evaluation suite inherited unchanged from the previous generation covers only part of the resulting system.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Production must keep changing the evaluation suite
Across these architectural generations, evaluations remain a durable description of intended behavior. Their implementation and coverage must change, but they preserve a useful contract: new capabilities should not break the experiences that already work.
That contract needs a continuous connection to actual usage. Production data supplies cases for evaluations; those evaluations guide improvements; the improved system generates new production evidence. Bhatawdekar reports that many teams endorse this flywheel but leave their suites largely static. Even without an architectural change, a suite can become less representative as users and their requests evolve.
The harder requirement is discovering failures the existing suite does not know how to look for. Collecting another example of a known error improves coverage within an established category. Discovering a new category changes the definition of what must be evaluated. Architectural changes make this second kind of discovery especially important: a new component or execution pattern can fail in ways the previous definitions of good behavior never anticipated.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Finding missing categories with Topics
Braintrust combines evaluations, observability, and production insights to feed new cases back to the development team. Its Topics feature adds clustering as a way to find patterns in production data, including failure categories a team has not already encoded in a scorer or guardrail.
Bhatawdekar describes this as cluster analysis over all production data. The operational scope is more specific: Braintrust’s later engineering account describes clustering sampled, extracted facets and then classifying incoming data. The mechanism supports discovering patterns; it does not guarantee discovery of every failure. The useful result is a surfaced category that prompts the team to investigate behavior its existing checks missed.
Those observations can become evaluation cases, expanding the suite beyond new examples of already-known failures. After an architectural change, this gives teams a route from unexpected production behavior to a dataset that explicitly exercises the new failure mode.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make evaluation part of how the system changes
Bhatawdekar expects further model capability unlocks and, with them, more substantial changes to agent architectures. The operational requirement is to make those changes without losing existing capabilities: the system should improve at new tasks while continuing to perform the work users already depend on.
That requires evaluation discipline supported by tools, automation, and an everyday production-feedback workflow. The same process must accommodate both small improvements and changes large enough to reshape the system. Evaluations earn their durability by continuing to describe—and test—the behavior that matters as the implementation underneath them changes.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The original research on interleaving model reasoning with actions and observations.
Instructions for finding patterns in production traces and turning selected examples into evaluation datasets.
Official introduction to connecting AI applications with external tools and data through MCP.
Further reading
A tool-and-user interaction benchmark introducing pass^k to measure success across every repeated trial.
Practical guidance on graders, outcomes, regression coverage, and repeated-trial metrics.
- Inside Braintrust TopicsArticle
An engineering account of trace preprocessing, facet extraction, embeddings, clustering, and continuous classification.
An explanation of packaging instructions and supporting resources into reusable agent skills.
Read the complete timestamped transcript
- 0:00
[on-hold music] Hello, everyone. My name is Ameya Bhatawdekar, and I am the Field CTO at Braintrust.
- 0:18
Uh, Braintrust is a evaluation and observability platform that helps AI teams build and improve their AI with confidence.
- 0:28
So [clears throat] I'm sure all of you, if not, you know, I, I'm sure everyone here has built some application over the last couple of years that has a model at the center of it, right?
- 0:41
Some sort of a chatbot or a AI agent or some system that's doing batch processing using AI at the heart of it.
- 0:52
And I'm sure all of you, over that time span, have done significant, uh, changes to that application. You have either rewritten that application entirely, or you have, like, done some pretty complex surgery on your application, and the way it looks now compared to how it looked when you started is likely very, very different.
- 1:14
And I think everyone's probably, uh, experienced the same pattern, which is, like, how building a demo with AI is really easy, but making it production quality is really hard.
- 1:26
The same way when you're evolving your AI application and making significant changes to it, it can be very, very challenging, right? And, uh, the challenge is not because you built it the wrong way.
- 1:40
The challenge is because the system around you is evolving and changing so dynamically, so rapidly. You know, the models are changing. The way your users use your application changes.
- 1:50
The data that your application works with changes, and all of those things require you to continually make changes to your applications.
- 2:01
And so if you look at, you know, the rate at which the models have evolved over the last couple of years, it's truly astonishing. Like, every few months, there's a new release, and that locks...
- 2:13
unlocks a, you know, a ton of new capabilities, a ton of new features that were not present in the previous generation of the models, right? We have started seeing, like, models that got really good at working with tools, models getting really good at handling very long context.
- 2:31
Uh, we started seeing models generate code that can be reliably and safely executed in, uh, sandboxes. We've seen memory systems becoming very sophisticated and practical. And so each of those was not a minor upgrade.
- 2:48
It was not an incremental change to the previous state of art. It was a step function change, right? And so now we are moving from this era of, like, re- uh, sort of iterating on improving our applications to re-platforming our applications because everything is changing so dramatically.
- 3:07
So [clears throat] why can't you just drop in a new model and als- expect your system to work? Um, well, the models, uh, the previous system that you built was built with some assumptions around the existing limitations and the constraints that the models had, right?
- 3:29
Your previous systems were built to account for the fact that your models weren't really as good at tool calling, for example. And so your system implemented a bunch of logic to make it work with those limitations.
- 3:41
And so when you drop in a new model, right, uh, you are not able to tap into the new capabilities, the new state-of-the-art, without really restructuring your systems in a pretty dramatic way, right?
- 3:55
And so in order to capture that kind of capability, the new unlock, you have to re-architect.
- 4:04
And so as you re-architect, right, um, what happens is, uh, models evolve, so you've got to go in and change your application architecture, do a lot of work on, on getting it to work with the new models.
- 4:17
But that means that you also now have to update your evals, the way you ensure that your system is going to operate reliably, right? Because every new, um, uh, unlock is potentially also giving you new surface area where things can go wrong.
- 4:36
And so your evals now have to adapt and evolve to your new architecture. And so, you know, architecture follows model updates, and your evals have to follow your architecture.
- 4:48
So as I talk through the various generations of the AI systems architectures and how you do, uh, you know, what that architecture is and how the evals have to evolve with those architectural changes, I wanna ground it in a real example.
- 5:04
And so what I want to talk about is, um, on the subsequent slides, I'll share a bunch of notional evals, but I want them to be grounded in a real agent.
- 5:13
In this case, we are gonna look at this SRE agent. The SRE agent is able to not only read information, but it's able to take actions on and update systems, so it can, you know, roll back a deployment or, uh, escalate it to a human, page someone.
- 5:29
Uh, so it has access to read tools and write tools. So let's see how this system would have evolved through the various generations of AI architectures.
- 5:40
So let's start with the simplest case, right? This is how a lot of AI applications started about three years ago. This is a single prompt, a single model call.
- 5:50
You have one input, one model call, one output. And so the focus of evaluations was on the final answer quality, right? Did you get the correct answer in terms of, uh, accuracy and factuality?
- 6:06
Um, or did the, uh, model hallucinate something? Did it make up stuff, or did it reference, uh, old non, uh, the, the previous knowledge that it had been trained on and not the latest, uh, information related to that subject?
- 6:22
Um, so in this case, um, you were really lo-- focusing primarily on the final answer. That was your unit of evaluation. And so the approach was you would put together a golden dataset.
- 6:34
You will create a bunch of various scores that we're looking at, um, encoding your definition of what good looks like that then you could evaluate the answers against. And this was great.
- 6:45
This was a good way to get started. It was narrow because there's no tool calling, there's no orchestration, there's no, uh, retrieval, no other steps. It's just a simple call to the model.
- 6:56
Uh, but the next iteration of this was the chain. This is where you started doing a set of steps before you actually made the model call, right? Uh, the typical RAG application looked like it took the user input, it parsed some information from the user output input.
- 7:13
It then used that to retrieve information, then generate the context, and then hand it over to the model. And then the model synthesizes reasons on that information, synthesizes an answer, and you've evaluated the answer.
- 7:27
But there's a mu- number of other places where things could go wrong. Yeah. Your parser could extract the wrong information. It could retrieve the wrong context. The model could struggle with the context.
- 7:39
Like in the early days, even though the model windows were the-- context window sizes were increasing, the models struggled to, um, reason over large context. So context stuffing could be an issue for the model performance.
- 7:51
And so now you had multiple s- uh, areas of failure, and so you needed to eval all of these steps in order to figure out like where things went wrong and how to act upon it.
- 8:05
But this was kind of very, um, what I would call very limited. Like it did things a very specific way all the time, right? And so in late, mid, late 2023, early '24, the ReAct paper became really popular.
- 8:22
And so folks were looking at building, um, model, um, in the loop, running a model in a loop where, where it could, uh, reason and act, uh, in a stepwise way.
- 8:34
So the model could make tool calls. It could then understand what the tool calls returned, uh, reason on that data, and then figure out what the next step was so that it could then continue to run this in a loop till the user intent was finally satisfied, or the model ran out of the iteration budget, right?
- 8:55
And so this was great because it now gives you, gives the model, the AI system, a lot more flexibility. It's not pinned down to operating in a very specific workflow.
- 9:06
It now is able to reason on the various intents, and it's able to self-organize, self-orchestrate, and complete the user tasks. Unfortunately, the models of that era were not as robust as they needed to be.
- 9:19
So, you know, models struggled with tool callings. They got the arguments wrong. The models struggled with orchestration, so they called the wrong tools. The models still had challenges with reasoning.
- 9:31
They weren't necessarily doing a great job of, you know, dealing with long context. So you had things like context collapse. And so while the idea was like really, really exciting, um, it fell short of delivering on the actual promise.
- 9:46
And so what does-- what do you do when your model can't be controlled, right? You take the control, and you bake that control into the system that you're building around the model.
- 9:56
And so teams started moving towards these kind of workflow graphs, right? Um, they started building the orchestration and the execution and planning logic into the, the system itself, either as a graph or as a state machine.
- 10:15
And so you took control of the orchestration while you allowed the models to operate at the node level. And that way you got a lot more, uh, reliability and predictability in how your AI was going to operate across those various intents.
- 10:35
But then the problem is you are now building a system that is designed to work for a specific set of intents for a specific types of use cases. And as you start hand-- you know, the system starts interacting with, with instances that are outside that distribution, the s- system starts struggling with that, right?
- 10:55
You expect, um, you know, a certain set of applications or, uh, user interactions to work well because they can be fulfilled by the orchestration that you have designed. But when your, the user intent needs to be, it requires other things to happen beyond what's specified in the orchestration, the system can start, um, you know, breaking at the
- 11:16
seams. And, uh, in order to do that, folks were now building a lot more complexity into their orchestration logic. And so you're building these special, uh, branches and way you handle special intents in the complex graph that describe your system.
- 11:33
And so what that means is like you had now a ton of different surfaces for failure. So you now had to deal with, uh, you know, uh, dealing with, uh, branch consistency and branching logic failures.
- 11:47
You had to deal with things like the contracts between the nodes not working out well. Uh, you had to deal with the limitations of, uh, the nodes that were, you know, built for a- ...
- 12:01
specific set of use cases. So, you know, there were classifier nodes, for example, and they could make mistakes. And so you could now have a significant amount of, um, you know, areas where you could, uh-- where the system could fail.
- 12:17
And so your evals now have to not only look at, uh, you know, the overall orchestration, but they now have to... you have to have node-level evals. You have to, uh, make sure that you have evals for testing your branching logic or, uh, uh, you know, how you do retry loops.
- 12:41
There's a lot of complex behaviors of the system that now need to be evaluated
- 12:47
in addition to all the other things that you were evaluating before.
- 12:54
So [clears throat] the graphs are kind of popular, like, in, in late '24, early '25, and so a lot of systems were now implemented using certain frameworks, and they were now in production.
- 13:08
Uh, but then, um, Anthropic and OpenAI launched some amazing new model capabilities mid to late '25. And what that was like tool calling became extremely reliable. We started, uh, seeing, uh, much better orchestration control.
- 13:27
Uh, the models were able to plan a lot more effectively, accurately. They were able to manage long-horizon tasks. They were able to do a much better job of introspecting and course-correcting.
- 13:39
And so, like, as things went a little off track, the models were able to, you know, understand that and bring the execution back on track. And so what that meant was a lot of these, uh,
- 13:52
uh, graph-based systems were not able to take advantage of these new capabilities. They were still running into some of those, like, brittleness issues that the new model state of art had unlocked.
- 14:06
And so, um, we started looking at building out, um, the ReAct loop again. That's, that started working. And so now you had this new AI systems that could effectively, reliably work in the loop.
- 14:22
They could make those tool calls. They could figure out the next step, and then they could, uh, essentially go in and, um, fulfill the user intent. But the way they worked was very-- It had a high degree of variance.
- 14:36
So every trajectory for the same input, if you ran it a couple of times, you would see, you know, dramatically different trajectories while yielding the right answer. And so now there's a lot of variance that you have to deal with.
- 14:47
So now instead of just focusing on a specific eval, the unit of eval was no longer just one eval. Now you're looking at doing, uh, an analysis of the distribution of the evals.
- 14:59
You're taking the same eval, you're running it multiple times, you're running it k times, and you're ensuring that, uh, you get a statistically relevant signal from that eval. So now new metrics like, uh, pass@k and pass raised to k or pass wedge k, these were the new metrics that certainly started to make a lot of sense.
- 15:21
Pass@k is like if you take the same that eval and you run it k times, that it-- does it succeed at least once? And that is a measure of its capability.
- 15:31
And pass wedge k is like if you run that eval multiple times, how many times of those k instances does it run successfully? That's a measure of its, uh, reliability.
- 15:41
And so now you can understand whether your system with a high pass@k, uh, you know, is reliable by seeing how it, you know, by measuring the pa-pass wedge k metric, for example.
- 15:55
So this gives you a lot more, um, you know, um, understanding of, like, how your system is working, what the failure sources are, and how you work on those.
- 16:06
Right? And then more recently, what we've seen is, um, there's a big shift from it's... your system is not just a model running in a loop, right? It becomes a product system.
- 16:17
It's that there's a model in the loop that's augmented by a lot of peripheral components. You know, you have a memory system that is able to provide robust memory storage and memory, um, retrieval capabilities, uh, within a session, cross sessions.
- 16:34
Uh, models can tap into this memory to, you know, improve upon their runs in subsequent instances by learning from previous runs, for example. You've got robust code-execution sandboxes now.
- 16:47
And so you can run model-generated code reliably, robustly on, uh, uh, during, uh, execution. You've got, um, MCP and skill, uh, directories that the model can now tap into, and you can, uh, you know, weave in extensibility.
- 17:04
You now have things like a skills repository or a skills systems that can be used to continually augment the, the, the capabilities of models through, you know, symbolic instructions.
- 17:17
And so, uh, now, you know, like, uh, these systems are getting pretty complex, and as a result, uh, you know, if you are continuing to, to use the evals from the previous generation, you're gonna get sort of a partial coverage of your system.
- 17:36
You're not going to see, uh, how your system is fragile in ways
- 17:43
because of the unlock-- because of the new surface that you have, uh, you know, uh, unlocked in your new system.
- 17:49
So what that means is, um, just reflecting back on the pattern is, like, you know, all of these model innovations resulted in, in, you know, corresponding shift in the architectures.
- 18:04
And so, so you've seen these waves of architecture, and then what's needed is, like, your evals to be congru-congruent with that architecture, right? Uh, because ultimately it's the evals that are sort of your durable asset that describe how your system is supposed to work.
- 18:21
And as you go through these generational shifts, that's a good way to ensure that, you know, your system, your user, uh, users experience your system in a way that things that were working are not broken, but it's unlocked a bunch of new capability.
- 18:35
And so everyone's seen this, you know, diagram of this flywheel. Everyone's sort of, like, bought into it conceptually, right? The idea of harvesting data from production to inform your evals so that your evals are reflective of the real world.
- 18:50
I think that all makes sense, right? And, and this is the way that, you know, teams that are doing a great job at building and shipping and improving their AI systems, they, they, they follow this workflow pretty religiously.
- 19:03
Um, so I've talked to a lot of teams, and I think while there is a general acceptance that, yeah, you need to run that workflow, um, in practice, a lot of teams don't do that.
- 19:12
Their evals are somewhat static. And even if you're not changing your AI agent architecture, you're, you know, by not really being disciplined about running that, that workflow, that flywheel, you are now getting stagnant evals that are not being as effective in helping you measure and improve the quality of your AI.
- 19:34
And especially as you go through this generational shift, it's really important that you need a mechanism to not only harvest data from production in a way that shows you failures that you are looking out for because you have defined what good looks like as part of your evals, but you also want something to shine a light on
- 19:56
the new failure types, right? The system is gonna fail in new and novel ways, in ways that you might not have anticipated, and you now need to start harvesting that data in a meaningful way.
- 20:08
And you want to do this, again, as, as part of the flywheel. And so this is where you need systems to come in and, uh, shine a light on things that are broken in ways that you had anticipated, but also broken in a way-- in ways that you had not anticipated.
- 20:26
And this is really important. So I'm gonna quickly talk a little bit about, like, how we do this in Braintrust. So Braintrust provides all the components that you need to run this flywheel.
- 20:37
We've got evals. We've got observability. We have ways in which you can get insights from your production data to harvest, uh, new eval cases that you can then pass off to the, to the team that they can then use to help climb and improve your AI system.
- 20:56
But Topics is a really cool feature. What Topics does, it does a cluster analysis on all of your production data. And so the idea over here is now you're able to find new categories of failure that you had not anticipated.
- 21:13
So your system is now able to look at all what's going on in production, and it's able to now start surfacing these new failure modes that tell you, "Here's a new, new failure, uh, you know, um, uh, situation that you hadn't thought about," and you didn't have any guardrails in place, or you didn't have any evals in
- 21:32
place. And so now it's really easy for teams to expand the set of their evals to now cover those kind of new failures. And so this is, this is a pretty exciting, uh, capability in Braintrust that enables these teams to continually not only get new failure examples for known failure modes, but more
- 21:57
importantly, as they make these systemic architectural changes, they're able to also understand the new ways in which your system is gonna fail and build out effective datasets from production data.
- 22:11
So I think the takeaway for today's talk is that
- 22:16
the models will keep on changing. Uh, I, I don't think we are gonna see any slowdown. I don't think we have hit a plateau yet. I think there are lots of unlocks that are coming down, um, this road.
- 22:28
Um, and as a result, you will be making significant changes to your AI systems. You know, you'll be doing a lot of surgery on your AI agents in the coming months, years.
- 22:42
And so it's really important that you have a robust workflow system in place to ensure that as you make those changes, as you incorporate these new models into your systems, that your systems continue to get better at doing new things, but also continue to work well for the things that they were doing before.
- 23:06
And so building out, like, a robust evals discipline, uh, with the right tools and the right automation and the right systems becomes paramount to manage these generational changes. And so ultimately, what you want is, um, to really, uh, index on that flywheel and make it part of your workflow so that, uh, you know,
- 23:31
the ability to improve incrementally when the changes in the system are incremental, and the ability to improve your system in a s-- in, in sort of a step function way are both supported by your evals.
- 23:48
So with that, I will say thank you. [audience applauding] [outro music]