AI Engineer World's Fair 2025
Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents
Read the talk
Building Agents That Improve—and Evaluators That Keep Up
Agent evaluation starts with individual tool calls, expands to trajectories and conversations, and closes the loop by checking whether the evaluators themselves are right.
From a talk by Aparna Dhinakaran
Before you start: You should be familiar with an LLM agent calling tools and using conversation context to decide what to do next.
When a spreadsheet cannot tell you what improved
A team swaps prompts in an Excel sheet, compares a few model outputs, and ships the version that feels better. Meanwhile, the model and tool definitions may also be changing. Which change actually helped, and on which examples? Without a systematic comparison, even a promising result is difficult to turn into a repeatable development process.
The problem also involves who can participate. Product managers and subject-matter experts need a way to contribute judgments to the same evaluation process. Aparna Dhinakaran, an Arize cofounder, frames agent development around making those judgments systematic. Once an agent reaches production, the question becomes more specific: which sub-agent or tool call consistently produces poor responses, and what needs to change there?
Evaluation needs to follow the scope of the behavior being tested. A tool call needs checks on its selection and inputs. A trajectory needs checks across the sequence of calls. A conversation needs checks across turns, where earlier context should inform later answers. Improving the agent also requires improving the evaluations that identify its failures.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate tool selection from argument correctness
An agent with several tools must first decide which action fits the current situation. That decision depends on the user's request, previous conversation context, and actions already taken. After choosing a tool, it must extract the arguments from that context. These are separate opportunities for failure.
| Check | Question |
|---|---|
| Tool selection | Is this the appropriate tool for the next action? |
| Argument correctness | Do its inputs match the request, context, and required parameters? |
A correct tool name does not establish that the call is correct. Keeping the two judgments separate makes the result actionable: a routing problem and an input-extraction problem require different fixes.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with the paths that need attention
The demonstration moves into Arize's own copilot, an agent that helps users investigate application performance, troubleshoot problems, and find possible improvements. Arize records traces of those user questions and evaluates the interactions using its own tooling. Those traces make the steps behind a response available for inspection.
Before opening an individual trace, Dhinakaran looks across the paths the agent can take. The copilot uses an orchestrator-worker pattern: a top-level planner chooses where to send a request, and tools perform the next actions. A tool's output can lead to another router or orchestrator that chooses a subsequent tool. The resulting response may therefore depend on several layers of decisions.
At the planner level, the useful question is where those paths diverge in quality. Dhinakaran describes the displayed copilot search Q&A results as roughly half correct and half incorrect. The recording does not supply a sample size, collection period, model, or evaluator configuration for that approximate result. It does provide a direction for investigation: search deserves attention before paths whose evaluations look healthier.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Follow a failed answer down to its arguments
The aggregate view narrows the search; the trace supplies the diagnosis. Dhinakaran follows a concrete drill-down procedure:
- Select the search Q&A correctness evaluation.
- Filter to examples marked incorrect.
- Open a failing trace and inspect its whole-trace evaluations.
- Inspect the individual tool calls, separating function selection from argument correctness.
The trace can contain both a failed overall answer and a successful local decision. That distinction is what makes the next inspection useful.
In the selected example, the function-call evaluation is correct, but the arguments are marked incorrect. The evaluator's explanation says the arguments do not align with the required parameters. Dhinakaran identifies the likely problem as supplying the wrong arguments for the conversation context, despite selecting the appropriate tool. The demonstration reaches a diagnosis; it does not show an implemented correction.
A small Python fixture illustrates why the two results should remain separate. Here, a request to search for dashboard duplication selects search, but supplies a query about monitor alerts. Exact comparison is sufficient for this fixture; the real diagnostic question is whether the arguments express the user's intended request.
python
request = "Find documentation on duplicating a dashboard."
expected = {
"name": "search",
"arguments": {"query": "duplicate a dashboard"},
}
actual = {
"name": "search",
"arguments": {"query": "configure monitor alerts"},
}
checks = {
"tool_selection_correct": actual["name"] == expected["name"],
"arguments_correct": actual["arguments"] == expected["arguments"],
}
assert checks == {
"tool_selection_correct": True,
"arguments_correct": False,
}
A single failure label would obscure the useful fact that selection succeeded. Separate results preserve that information while directing the investigation toward argument construction.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Correct calls can still form a bad trajectory
The copilot's search path invokes multiple sub-tools. Evaluating each call independently therefore leaves another question unanswered: did the agent call them in the right order? Trajectory evaluation examines the sequence across an entire trace, rather than treating every action as an isolated decision.
For repeated instances of a task, Dhinakaran asks whether the agent consistently completes the necessary actions and converges on an appropriate number of steps. A run that veers into a different order can spend more tokens on the same request, or produce an incorrect output because the actions no longer form a valid sequence. The concern is both completion quality and the work required to get there.
The displayed tool-calling-order evaluation is marked incorrect. That adds a distinct failure category to the earlier argument problem: the individual actions may need attention, but so may their arrangement. No target step count or measured token savings are supplied. A trace-level judgment still covers only one interaction, which leads to the next scope of evaluation: the conversation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Carry context across the conversation
An example with three human-agent back-and-forths introduces behavior that a single trace cannot fully describe. Does the agent maintain a consistent tone? Does it ask the same question repeatedly, despite having already received the answer? Repetition can expose a failure to use earlier interactions, even when an individual response appears reasonable on its own.
The core requirement is to use context from the previous N−1 turns when answering turn N. That makes session evaluation broader than checking each answer separately:
- Tone consistency: Does the agent maintain an appropriate tone across turns?
- Context retention: Does it use information supplied earlier instead of asking for it again?
- Conversation-wide correctness: Does it correctly answer the questions throughout the interaction?
Dhinakaran then opens a conversation from another project to inspect both answer correctness and whether earlier context has been missed. Session evaluations complement the tool and trace checks by testing what survives between interactions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Improve the evaluator alongside the agent
Tool, trajectory, and session evaluations all feed the application-improvement loop. Evaluate the agent, identify where it fails, annotate or refine the outputs, and use those examples to improve its prompt. The evaluations determine which cases enter that process, so their quality directly affects what the team chooses to fix.
An evaluator can be the component that is wrong. With an LLM acting as a judge, its prompt should not remain static while the application evolves. Inspect the evaluation outcomes themselves: an application response marked incorrect may have been acceptable, and the judge may have misclassified it. Treating every negative evaluation as an application defect would send the improvement process after the wrong target.
The evaluator needs its own iteration process: identify mistaken judgments, improve the evaluation prompt, and build and continually refine a golden dataset. The two loops operate together:
| Loop | Evidence to inspect | What changes |
|---|---|---|
| Application | Failed outputs and corrected examples | Application prompt |
| Evaluator | Mistaken judgments and reference labels | Evaluation prompt and golden dataset |
The self-improving stack presented here is a workflow of repeated evaluation and refinement, not a demonstrated autonomous optimizer. Its effectiveness depends on maintaining both the agent's behavior and the quality of the judgments used to improve it.
To try these practices in an application, Dhinakaran points to Arize Phoenix, which she describes as completely open source. For present-day reuse, its current Elastic License 2.0 restricts offering substantial functionality as a hosted or managed service. She also points to Arize AX for running evaluations on your own data.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Source code and setup instructions for tracing agents, evaluating outputs and running dataset-based experiments.
Further reading
Arize's December 2024 account of developing Copilot through tracing, skill-specific evaluations, datasets and experiments.
Updates since the talk
- Phoenix and Arize AXDocumentation
Current documentation explaining the two products and their deployment and operational differences.
An August 2025 tutorial on grouping traces into conversations and evaluating session correctness, frustration and goal achievement.
A March 2026 Phoenix tutorial on tool selection, argument correctness and refining application and evaluator prompts together.
Read the complete timestamped transcript
- 0:02
Hey everyone, my name's Aparna. I'm one of the founders of Arize, and today we're gonna talk about agent evaluation. At Arize, we build development tools to help teams build agents and take them all the way to production.
- 0:14
We focus on everything from evaluation to observability, monitoring, and in tracing your application so you can see every single step that your application took. Uh, let me tell you a little bit about why we got into this, and then I'll jump into, uh, some concrete tips about how we think about evaluating agents.
- 0:34
First, building agents is incredibly hard. There is a lot of iteration that goes on at the prompt level, at the model level, at iterating on the different tool call definitions.
- 0:44
And for a lot of teams, this is what their experience looks like. They're in an Excel sheet, they're swapping out different prompts. They're trying to figure out, did one prompt give them a better output than another prompt?
- 1:00
Often a lot of this is just based off of how it felt on a couple different examples, and then they go live into production with that prompt. Part of this is that it's, it's pretty difficult to actually systematically track where is this new prompt doing better than your previous prompts?
- 1:18
Where is this model doing better? Um, and it's hard to actually include other people, especially if you have your product managers or your SMEs actually a part of this iterative evaluation-driven process to how you actually want to think about improving your application.
- 1:35
Um, and so it's hard to consistently figure out what makes your agent better, and it doesn't get easier once you actually deploy into production. It's pretty hard to understand, well, where's the bottlenecks in my application?
- 1:48
Is there a specific sub-agent or tool call that is kinda, kind of consistently creating, you know, poor responses? How do I wanna actually identify those bottlenecks? And then what do I actually need to do to go fix it? [clears throat]
- 2:04
And so today I'm gonna be diving into a little bit of the different components of how I think about agent evaluations. We're gonna talk about some of the most common components, which is evaluating at the tool call level, um, taking that one step further, going all the way to the trajectory and looking at did it actually, um,
- 2:22
maybe across an entire trace or, or, you know, did it actually call, for example, all the tool calls in the right order? We're gonna then not only look across a single trace, but then across, um, multi-turn conversations because these, the- these interactions are no longer just kind of a single-turn experience.
- 2:39
They're often multi-turn, keeping track of what happened in the previous, um, interaction and keeping that in mind as context for the next, um, eh, next turn of the conversation.
- 2:49
Uh, so we're gonna talk a little bit about that. And then I'm gonna jump into kind of a, a, you know, approach that we've been really excited about, which is how do we get these agents to self-improve?
- 2:58
And that starts with not just thinking about the agent improving, but also your evals consistently improving. So with that, let's jump in. I'm gonna do a little bit of slides, and then I'm gonna h- jump into actually a, a real example so you guys can actually see it in, in practice.
- 3:13
So first we're gonna talk a little bit about tool calling evals. Um, anyone who's building agents is probably writing a lot of different tools, um, and making sure that your agent has access to call all these different tools, uh, depending on the action it needs to take.
- 3:27
And pretty consistently, it's your, your agent needs to probably make the decision of what's the right tool call to call in this specific scenario? I have, you know, potentially contacts from a previous part of a conversation or previous actions it's taken, and what do I actually need to call in order to kinda next continue whatever's happening in
- 3:47
that, in that interaction? Um, so not only do you have to pick the right tool call, but you also have to figure out from that conversation or, or context, what's the right arguments to pass into that tool call?
- 3:59
Um, and so it's pretty important to actually evaluate, did it call the right tool, um, and did it pass the right arguments into that tool call itself? And I'm gonna go into a little bit of depth of this and show you how we think about evals, uh, actually from, from a product perspective.
- 4:15
Um, this is the Arize product. You can see here I'm actually, uh, tracing and looking at the traces of our own actually copilot here. So this is our own copilot and our own, um, agent that actually...
- 4:27
think about it almost like an insights, um, uh, a tool where teams can come in and ask all sorts of different questions about whether their application is doing well or not doing well, and use this to actually troubleshoot their, their application and, and suggest improvements.
- 4:43
And, you know, like any great product, we actually dogfood our own tooling, and so these are actually the traces of different questions that users have asked us. Um, and we actually evaluate, uh, these interactions so we can understand where our copilot's doing really well and where it's actually not doing well.
- 5:00
Um, one thing that we, we actually like to look a lot at is not just kind of the individual traces, but actually starting at a little bit more of a higher level view where we can look across all the different paths that, you know, all the different, you know, trajectories that our agent actually, um, can, can go
- 5:20
down. So in our case, this is actually the architecture of our agent. You can see here we, we follow a little bit of an orchestration worker-type pattern where there's, at the very high level, a, a planner that decides, based off of the information, um, what to go down, and there's all sorts of different tools that it can
- 5:38
then call. And sometimes, depending on the output of those tools, it might need to call even another, um, router or orchestrator to figure out, um, you know, what next tool call to actually call.
- 5:49
And so there's kind of multiple levels to this to actually make sure that ultimately we respond to the user in kind of a, a, a good, a good way.
- 5:58
Um, with this specific agent of ours, um- You can see here that, you know, for me, what I really think of a lot about is, well, at the planner level, the, the very beginning, uh, is, y- you know, all-- across all the different paths that this agent could go, where is it kinda doing really well, and are
- 6:17
there any bottlenecks in performance? And as I can look through some of this, it's, you know, there's evals around questions that, that are just related to, [clears throat] you know, generic questions that the user's asking.
- 6:28
Um, it looks like we're actually not doing so well on search. I can see here we're, we're almost pretty consistently doing, you know, it's about half and half of times we're getting it correct as we're getting it incorrect, which is not that great.
- 6:40
So this is probably an area that I would dive into and look at kind of the bottlenecks, uh, of, you know, where we're not doing so well when the user's asking search questions.
- 6:50
Um, and it looks like there's other questions that we're actually doing, doing pretty good on. So this type of high-level view first is just giving me a view of all the different paths that my agent can go down and really kind of pinpointing to me what I should go focus on specifically.
- 7:05
So now when I go look inside of my traces, I can actually, uh, start with something like the
- 7:11
Q&A correctness and look at something like, well, I, I should probably care about, um, [clears throat] in this case it was specifically the search Q&A correctness. So what I should probably go look at is for search Q&A correctness, when it's incorrect, let me go take a look at some of the examples of that and try to understand where...
- 7:32
what I'm doing wrong here. And so in this case, when I'm looking at these, I can actually now drill and go into specific, um, traces. And at this level, I have evals across the entire trace.
- 7:45
I have evals on the individual tool calls. Um, and at the tool call level, I also have, um, you know, not only did it call the right tool call, it says the function call is correct, but I also have evals on did it actually pass all the right arguments.
- 8:02
In this case, it looks like that's where it's going wrong. It says, "Therefore, the arguments do not align with the required parameters, leading to the conclusion that the arguments are incorrect."
- 8:10
So I probably have an issue here where even though I'm calling, it looks like the right tool call, um, out of all the different ones that I have, it looks like maybe I'm not passing in the right arguments inside of my tool call based on the context of the conversation.
- 8:24
So that's something that I should go fix. Um, so this is kind of the first big one that, that we think a lot about. Um, the next one that's pretty interesting is also, you know, for a lot of these, it's not just a single tool call that's made, it's m- many actually.
- 8:40
And you can kind of see that when, when I was showing you the way that our architecture is built. There's actually a lot of different... Even within the Q&A correctness, uh, when we go down the search correctness path, there's actually a lot of different sub-tools that are even called here.
- 8:54
So it's pretty important to not only get, uh, y- you know, if it's individually calling a single tool correctly, but also is it getting the order of the tools that it's supposed to call correctly.
- 9:04
And that's really what, as, as we think about trajectory eval starts to, uh, starts to... it starts to become about is really is it calling tool calls in the right order?
- 9:15
Um, if across a series of steps as needed for an agent to complete a task, is it consistently calling and, and executing them in the same set of steps or...
- 9:28
and, and eventually converge on, you know, X number of steps to complete the section, or does it sometimes veer off and call it in a different order and therefore require me to, A, have to spend a lot of tokens in order to do the same ask, um, and then, B, is it kind of messing up and providing
- 9:46
wrong kind of outputs because of that? And so we recommend teams to actually drill in and look at, you know, across not just an individual kind of a tool call, but actually looking across, um, an entire trace and looking at the order of the tool calls to see if that's actually done well, and then evaluating kind of
- 10:05
is overall, in this case, I have it incorrect. And so in this case, is it actually consistently getting the tool calling order correctness, uh, uh, correct? Um, the next step here is, well, you can look across a single individual trace or interaction, but a lot of these interactions we're seeing with agents actually ends up being multi-turn.
- 10:25
So in this case, I have like a three back and forths between a human and an agent, and there's a lot of interesting questions you can actually ask at this stage.
- 10:33
You can ask questions like, is the agent consistent in tone? Um, is the agent maybe asking the same question and again and again, um, in which case it's not really learning anything from the previous interactions that it's had with the human?
- 10:48
Um, and, and part of that is really does it keep track of context from the previous kind of N minus one turns in order to be able to answer the Nth, uh, turn of that conversation really well?
- 10:59
And so these are all the types of questions to think about when you have something that, that is multi-turn. And I'll actually show you an example from another project here where I do have kind of some of that multi-turn interaction.
- 11:13
Um, this is maybe one where I have kind of this back and forth with, with an agent. And what I care about here across this entire conversation is, did I actually correctly answer all the questions?
- 11:26
Did I actually make sure I kept context so that I wasn't kind of missing context that was made earlier in the conversation? And so deeply recommend folks to actually think about session evaluations as part of their evaluating of agents.
- 11:40
Um, and then lastly, I'll kind of go through this, and, um, I think this will be kind of a good spot for us to, um, deep dive into, which is
- 11:50
we spend a lot of time, even just now on the tool calling trajectory session, a lot talking about how to think about evaluating the agent or the application prompt.
- 12:01
And this is really important. I mean, I... we could spend a whole deep dive on this itself, but- It's really important to evaluate it correctly, identify where does it go wrong, so that you can annotate or, you know, refine those outputs and use those to actually improve your existing prompt.
- 12:20
And I think a lot of teams actually totally get this and are doing this all the time to improve the agent prompt. Um, but what's really important is that the evals that you're actually using, just kind of the crux of how you identify those failure cases, end up becoming crucial to calling out what you need to improve.
- 12:40
And you don't want those evals to remain static, the prompts for those evals if you're using LLM-as-a-judge. And so there really is kind of another loop going on here, which is about improving the evals and the eval prompts.
- 12:54
And part of this is you're collecting, um, consistently checking the, the eval outcomes as well to make sure it's not just the application that got it wrong, but it could have been the eval that, you know, miscorrectly labeled it as wrong.
- 13:10
Um, and start to identify where the eval itself might need some improvements. And similar to the process you did for the agent application, do a workflow where you're iterating on the eval prompts, um, you know, building up a coding dataset, consistently refining it.
- 13:28
And there really are kind of two iterative loops kind of going on at the same time, one for the agent evaluations, one for, um, your, your agent application prompts, and then one for your eval prompts.
- 13:40
And as you think about this, both of them kind of go hand-in-hand to actually create a really good product experience, um, for teams. Um, there's a lot more in here that, you know, I, I think we, we can dive into, but, um, hopefully this gives you a little bit of a primer about how to think about agent
- 13:58
evaluations. And, uh, check out Arize Phoenix. It's a completely open source, um, product that you can use to learn a lot about what we just went through and test it out in your own applications.
- 14:10
Um, you can check out Arize X if you wanna, um, think about how to run a lot of these evaluations on your own data. Um, so feel free to check it out, and hopefully you guys got something out of this.
- 14:23
Thanks everyone for the time.