AI Engineer Summit 2025
Keynote: Why people think "agent" is a buzzword but it isn't
Read the talk
Building agents that can handle complexity, tools, and context
An agent’s useful reach depends on more than its model: task length, tool contracts, and memory determine whether it can turn a request into reliable action.
From a talk by Chip Huyen
Before you start: Basic familiarity with language models, API calls, and context windows will help; no agent-framework experience is required.
A chessboard, a browser, and a filesystem
A chess agent perceives a board and acts by making moves. That simple example captures the definition Chip Huyen takes from Stuart Russell and Peter Norvig: an agent perceives its environment and acts on it. The term predates today’s language models; the useful engineering question is what the system can observe and what it is allowed to change.
For ChatGPT, browsing makes the internet part of the environment, alongside capabilities such as calculator use and text and image generation. For SWE-agent, the environment is a computer with a terminal and filesystem. Its actions include navigating a repository, searching, viewing files, and editing them.
The environment constrains the actions, and new actions expand the environment. A game agent cannot perform moves the game does not permit. Conversely, giving a model a browser adds access to an environment it could not previously inspect. Tool selection therefore defines the agent’s reach, not merely its convenience.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Tools extend capabilities and bring AI into work
Tools first address limitations of the underlying model. A training cutoff need not prevent an application from answering questions about recent events: news APIs, weather APIs, and browsing can supply fresh information. A calculator similarly provides a direct route to numerical computation without requiring the language model to learn every calculation reliably.
Tools can also extend the modalities an application handles. A text-only language model cannot directly consume an image, but an image-captioning model can turn that image into text. The language model then uses the caption to produce a response. The combined system accepts images even though its language component still operates on text.
The larger opportunity is to put the model inside an existing workflow. Access to an inbox, Slack, a calendar, or a code editor lets an agent work where the relevant information and actions already live. Users no longer have to move every task into a separate AI interface. This is why actions matter beyond compensating for model weaknesses: they connect inference to the work someone wants completed.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A small error rate becomes a large task failure rate
If those capabilities are so useful, why are agents not already doing everything? The first obstacle is complexity. A company might tolerate a task failure rate of one or two percent, but that does not mean a model with a two-percent error rate at each step can reliably complete a long workflow.
Huyen’s example puts ten-step failure at roughly 18%. The calculation can be made explicit by assuming independent errors, a constant per-step failure probability, and a task that fails whenever any step fails, without recovery:
Under those assumptions, a 2% per-step error rate produces approximately 18.3% task failure over ten steps and 86.7% over one hundred steps. These are conditional calculations, not measured agent results. They explain why a locally acceptable error rate can become unacceptable across a workflow.
This creates a difficult product tradeoff. Simple tasks often do not need an agent, and may offer less economic value. The more attractive tasks frequently require several dependent operations—the very structure that makes them harder to finish reliably.
Consider Huyen’s apparently simple business question: how many people bought products from Company X last week? The proposed workflow has four conceptual stages:
- Fetch Company X’s product list.
- Retrieve last week’s order count for each product.
- Sum the counts.
- Generate a response.
The second stage can contain many individual calls. There is also a semantic problem to resolve before treating the answer as correct: summing per-product order counts does not establish the number of distinct buyers, and an order containing multiple products may appear in more than one count. The workflow illustrates decomposition, but answering the original question requires the right counting and deduplication rules.
Among the agent use cases Huyen had seen, consistent success on tasks requiring more than five steps was very rare. That observation motivates the next question: how do you establish the complexity your own agent can handle before giving it consequential work?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Measure difficulty before choosing the workload
Required step count is one useful measure of complexity. Huyen describes a synthetic planning benchmark she was developing, where generated tasks let her control that count and examine how performance changes. In Huyen’s then-unpublished synthetic planning benchmark, most tested models could solve tasks requiring at most five steps, and most failed after ten steps. The result is her historical report; the talk does not supply the full dataset, prompts, or sampling settings needed to reproduce it.
A related pattern appears in the 2021 paper Evaluating Large Language Models Trained on Code. That study evaluates generated-code correctness as string-processing instructions combine more operations; it is analogous evidence, rather than a test of autonomous tool execution. Huyen’s two-operation example is to lowercase a string and then remove half its characters. The following Python function makes the composition concrete, choosing to retain the first half:
python
def lowercase_first_half(text: str) -> str:
lowered = text.lower()
return lowered[:len(lowered) // 2]
result = lowercase_first_half("ABCDefgh")
assert result == "abcd"
Each instruction is simple in isolation. Success requires satisfying both, in order. The study’s pass rates fall rapidly as more operations are composed; Huyen distinguishes that pattern from the absolute scores of the older models, which she expects to have improved.
Her planning results also show progress. DeepSeek-R1, Gemini 2.0 Flash Thinking, and o1-preview move the success curves upward, solving more complex tasks than the other models in her comparison. She did not test o1 or o3, because she lacked access when she ran the experiment. Her aggregate task-count comparison separates those reasoning models from Sonnet 3.5, Gemini 2.0 Pro, and GPT-4o, without giving exact counts in the spoken explanation.
Step count is not the only way to define difficulty. ZebraLogic studies logic-grid puzzles and uses Z3 solver conflicts as a complexity measure alongside search-space size. Those conflicts describe the solver’s work, not the language model’s reasoning steps; puzzle success requires the whole grid to be correct. Huyen highlights the same qualitative relationship: model success falls rapidly as conflict counts rise. The broader lesson is to measure difficulty in terms that fit the task, then evaluate performance across that range.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Decompose the task or spend more compute
Once the operating limit is known, there are several ways to work within it or extend it:
- Decompose the task. If the workload usually takes five or six steps but the agent reliably handles only three, split it into two smaller subtasks. Assign work at a granularity the agent can complete.
- Increase test-time compute. Give the model more inference computation, either through more thinking tokens or through multiple candidate outputs. For a math problem, Huyen suggests generating ten solutions and selecting the most common answer.
- Use a stronger model. Huyen frames investment in training larger, stronger models as train-time compute scaling, in contrast to spending more computation on each inference.
These approaches address different parts of the problem: decomposition reduces the scope of an individual assignment, while additional compute seeks to improve the model’s ability to solve it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Natural language must become an exact API call
Tool use is a translation problem: a person gives an instruction in natural language, but the environment expects particular function calls. Given a customer email and a request to create an order, an agent might first extract a customer ID from the email address, then extract an order ID from the email contents, and finally call the function that creates the order. Both sides of this translation can fail: the request may be ambiguous, and the API may be badly designed or documented.
Huyen illustrates the mismatch with two tools. Written as identifiers, fetch_top_products(start_date, end_date, number_of_products) retrieves a ranked product list, while fetch_product_info supplies information such as price. Now consider a request to find best-selling products under $10.
| Required information | Specified by the request? |
|---|---|
| Price ceiling | Yes: under $10 |
| Start date | No |
| End date | No |
| Number of products | No |
The model may recognize the right tool without having enough information to call it correctly. Yesterday’s best sellers, last week’s, and last month’s answer different questions. Valid argument types do not resolve missing intent.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Document what results mean and how failures recover
An agent cannot use an interface reliably if its contract is unclear. At minimum, the documentation needs to describe the function’s purpose, parameter types and meanings, error codes, and expected return values. This is more than supplying a function name and a JSON shape: the agent needs enough information to decide when a call is appropriate and what its result establishes.
Errors need operational meaning too. An opaque error code does little to guide the next step. Explain the likely cause and what the agent should do when it encounters that condition. Otherwise, the tool reports failure without helping the planner recover.
Successful returns can be equally ambiguous. What does a returned value of 1 mean? Without that interpretation, the agent may be unable to decide whether to continue, retry, or take a different action. Huyen reports that an unnamed company saw one of its biggest agent improvements after documenting how to interpret function results. Return-value semantics are part of the planning interface. They determine what the agent can infer from a completed call.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Do not assume human tool habits are optimal
Good tool design also requires recognizing that humans and agents may use the same environment differently. A person working in Salesforce may prefer its website, while an agent may be better served by direct API calls that avoid interpreting visual cues. Similarly, a person commonly performs work sequentially, whereas an agent system can issue independent operations in parallel.
Browsing and summarizing hundreds or thousands of websites illustrates the difference. Huyen recalls the tedium of visiting thousands of sites while researching her book. An agent workflow can distribute independent retrieval and summarization work across concurrent calls. The point is the available execution strategy, not a measured throughput claim.
That difference matters when creating training examples. A human annotator’s sequence of actions may not be the best sequence for an AI system. Supervised fine-tuning teaches the model to imitate the demonstrated behavior; reinforcement learning can let it discover other strategies through trial and error. An effective agent policy need not reproduce the way a person navigates the task.
Alongside good documentation, Huyen recommends narrow, well-defined tools. An acquaintance at an unnamed large search company reported using only three or four small tools; for that team’s tasks, the agents did not work with more than five tools. This is a task-specific experience that motivates testing the tool inventory, rather than treating every additional capability as an automatic improvement.
On the language side, query rewriting and intent classification can help establish what the user wants. When the missing information remains consequential, the agent should ask. A request for five best-selling products under $10 now supplies the product count, but still lacks a date range. Asking whether the user means yesterday or last week is preferable to silently choosing yesterday or last year.
Another direction is to train specialized action models for particular environments. Coding models and tools such as VS Code provide an existing example of specialization; similarly complex enterprise environments may benefit from models trained around their own APIs and tasks. Huyen presents this as a general opportunity, rather than a confirmed initiative at a particular company.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Every capability adds information to manage
The third challenge is context. Even before adding agent behavior, an application may have lengthy system instructions governing permitted requests, security, and tone, plus user instructions and examples. Agents add more information on top of that baseline.
Tool documentation grows with the number of tools. Tool outputs accumulate with execution steps. Reasoning about those outputs—what to do next, whether a plan is sensible, whether to execute it—adds further tokens as the task becomes more complex. The information needed to continue the task can grow much faster than the original user request suggests.
Domain information can be large before execution even begins. In a text-to-SQL application with a thousand tables, the model must first identify which tables are relevant. Supplying all thousand schemas to support that choice can itself create a substantial context burden. The challenge is not simply remembering the conversation; it is retaining enough of the environment’s structure to act correctly.
Planning strength does not automatically solve this problem. Huyen describes planning as output-heavy, because it produces reasoning tokens, while long-context processing is input-heavy. In her personal benchmarks, models that performed well on planning were not necessarily the models that performed well on long context, and vice versa. Model selection therefore needs to test both demands together for the intended workload.
Nominal context capacity is not the same as effective context use. Huyen illustrates the distinction with a hypothetical model that accepts a million tokens but becomes unreliable beyond a few thousand; those numbers are not a measured threshold for an identified model. Her practical method is to evaluate when adding more documentation begins to produce hallucinations or invented details. Information that exceeds the model’s effective working context needs another place to persist.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put information where it is needed
Treat the active context as short-term memory: it should hold information relevant to the task at hand. External databases and storage provide long-term memory, as they do in retrieval-augmented generation. The model does not need every stored fact in its active context at every moment.
For an agent working through a ten-step task, the outputs of all ten steps may not fit together. Earlier outputs can be saved to external files once they are no longer immediately relevant, then retrieved when a later step needs them. This preserves the information without requiring it to remain continuously present in the prompt. Storage and retrieval become part of how the agent carries work forward.
A third layer is internal knowledge: information already learned by the model. Huyen suggests including knowledge that is essential across many tasks in training data or fine-tuning, rather than repeatedly spending context tokens to supply it.
| Memory layer | What belongs there | How it becomes available |
|---|---|---|
| Active context | Information needed now | Present in the current input |
| External storage | Retained facts and earlier outputs | Retrieved when needed |
| Internal knowledge | Knowledge useful across tasks | Learned during training or fine-tuning |
This makes memory design a question of placement and timing: what must be present now, what can be retrieved later, and what should the model already know?
The engineering work connects across these boundaries. Task decomposition limits what an agent must complete at once; precise tool contracts make each action interpretable; a memory system keeps the information needed to continue available. Huyen closes by inviting discussion of her ongoing planning benchmark—the measurement problem that determines how much work an agent can responsibly take on in the first place.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Chip Huyen's AI Engineering excerpt explains agent tools, planning, reflection and failure evaluation.
Research on interfaces that let language-model agents navigate repositories, edit files and execute programs.
The 2021 Codex paper studies declining correctness as synthetic docstrings combine more string-processing operations.
Original preprint evaluating logic-grid puzzle accuracy against search-space size and solver conflict counts.
Further reading
Official source repository and setup entry point for experimenting with SWE-agent.
Read the complete timestamped transcript
- 0:00
Hello, my name is Chip. I started an AI infrastructure startup a few years ago, and after selling it last year, I have been happily unemployed. Before that, I was with NVIDIA, Snorkel AI, and also taught a, a couple of courses at Stanford.
- 0:16
I have, uh, my, uh... for today, I want to talk about the challenges in building agents or why people think agent is a buzzword and why I think that it's not.
- 0:26
I have been wanting to come to, like, the AI Engineering Summit for a long time, but Swyx, like, never invited me until very recently. I shared the session of agents, uh, from my book, AI Engineering.
- 0:38
It's a very long session. It's, like, 8,000 words, and people seemed to like it, so Swyx invited me here. And, um,
- 0:45
I actually, like, prepared, like, another talk for, uh, for, for the summit. But then after watching a lot of talks yesterday, I realized that people have covered a lot of ground, so I create a new talk hoping to cover, like, newer, like, more exciting topics.
- 1:02
So this is a new talk created especially for this conference, and I hope you like it.
- 1:09
I heard that if you are to give an agent talk today, you're obligated to define what an agent is. Um, I know that a lot of people think, uh, oh, there are a lot of talk about, like, agents is just hype, but I don't think so.
- 1:22
I think there's a lot of, like, exciting use cases for agent, but I guess I'm, like, preaching to the choir. So agent is not a new term. Uh, when I was working on my book, uh, I decided to, like, look at a lot of books from, AI books from the '80s and the '90s, and try to understand,
- 1:39
like, how people define agent back then. And a definition that's, like, really, uh, resonate with me, uh, is from the book by, um, by Stuart Russell and Peter Norvig.
- 1:49
So they defined an agent as anything that can perceive the environment and that acts on the environment. So let's say that you have an agent that play chess, that the chess board is its environment and its actions are the chess moves.
- 2:04
Um, ChatGPT, right? ChatGPT can interact with the internet, so the internet is its environment, and it can do actions like web browsings. It can also use calculator. It can also, like, generate text and images.
- 2:18
So one of the most popular use cases of agent nowadays is, like, coding agent, and here is from the paper, like, SWE- um, SWE-agent paper. And as you can see here, the environment for SWE-agent is a computer with terminal and file system, and the list of actions it can perform includes navigate repo, search files, view files, edit
- 2:40
files. So the environments, like, determines the kind of actions that the model can perform. So if you're in a game, if an agent is in a game, it can only perform the action that the game allows.
- 2:53
At the same time, giving the model more actions can also help expand its environment. So if you give the model the ability to browse the web, then now the internet becomes its environment.
- 3:05
There are many reasons why, like, we would want to give a model access to actions. So first, actions can help address a model limitations. So all the models have the cutoff date, and that make it, like, pretty hard to, like, answer questions that require new informations.
- 3:22
By giving the models access to, like, newer APIs, su- such as, um, news or weather, web browsers, the model now can get, like, relevant recent information to answer questions.
- 3:34
A very common limitations that people discover very early on with AI is that AI is pretty bad with math. So instead of trying to train a model to, like, be really, really good with numbers, you can simply, like, give the model access to a calculator.
- 3:49
Another thing is a very exciting use case that it can turn a text-only or image-only models into a multi-model model by giving it access, uh, to tools, um, or actions.
- 4:00
So for example, um, given a language model, right? A language model can only process text and output text. So if you want a model to, like, also be able to process image, you can...
- 4:11
I can give you access to, like, say, an image captioning model. So, like, given an image, it can, like, use this tool to generate captions, and then use a caption to generate a response.
- 4:20
Now the model can process both text and image. It's very cool. But something that's, like, even more cool, so I think it's, like, why agents are so exciting, is that, like, actions allow you to embed models into the workflow.
- 4:34
So now, for example, you can give the model access to your inbox, uh, your Slack, your calendars, or the code editors so that the models... so that you can use a model into digital workflow instead of having to open, say, like, a, a web browsers so that you can use AI.
- 4:51
So when I'm talking about, like, agent, uh, people will always ask me, like, "Okay. If agents are so cool, so why aren't everyone... why, why isn't everyone using it?
- 5:01
Like, tell me." Like, everyone asking me, like, what, what would be, like... give me one good use cases of, like, agent. So why isn't everyone using it? It's because, like, doing agents is, like, really, really hard.
- 5:13
So for the rest of the talk, I will cover, like, a few reason why, like, doing agents is so hard. So we're gonna start with the curse of complexity.
- 5:23
So we know that, like, task failure rate increases as the task complexity increases. This is true not, not just for AI, but also for human as well. Like, if you're given more complex task, we will be more likely to fail.
- 5:36
So let's say that you're building an application for your company, and you're okay with a failure rate of, like, say, um, 1 or 2%, right? And the model makes, like, makes mistake, like, 2% of the time for one step.
- 5:50
So over 10 steps, the mistakes, the model can make mistake about, like, 80%, 18% of the time, and that is a lot. That might be unacceptable. And, like, if you increase your number of steps to, like, 100 steps, the model become, like, almost, like, worthless.
- 6:04
Like, it could make mistake most of the time.
- 6:08
For a lot of agent use cases, um, a lot of u- a- agent use cases are pretty complex and might require mu- multiple steps to solve, uh, to solve them.
- 6:18
So it's not that you don't want to use agents for, like, simple tasks, but, like, simple, simple tasks just don't need, don't usually need agent to do. And also simple tasks might have, like, lower economic value, so, like, they are less exciting for agent to solve.
- 6:37
Uh, let's, let's go through, like, a very, very simple example. Like, let's say that you want to ask the agent to, like, how many people bought product from, like, Company X last week?
- 6:47
So this is a very, very simple query, but the agent might need to break it down in, like, in several, in several steps. Like, it might first get the product list of the Company X, and then for each product in this list, it will want to get the number of, like, order for, like, last week.
- 7:03
And then given all this number co- um, order counts, it would have to sum it up. And then given this numbers, it has to generate a response to the users.
- 7:11
So even with this very, very simple query, very simple task, there were, like, like, four steps. And the more steps there are, like, the more complex queries, even, uh, the higher number of steps, and the more likely the agent is going to fail.
- 7:25
So in, in the, uh, a vast majority of, um, agent use cases I'm seeing right now, um, it's very, very rare to see them, like, consistently being to solve tasks that involve, like, more than five steps.
- 7:39
I do believe that enabling agent to handle more complexity would unlock many, many new use cases.
- 7:46
Um, so this is tricky questions. Um, how to know what complexity does your agent, your agent can solve? Because you want to give agent, um, the, the task that, like, has a right level of complexity it can solve so that it doesn't fail and, like, cause, like, catastrophic, uh, business failure.
- 8:06
So to... So, um, different kind of tasks, different use cases have different definitions of complexity. A very, a very, very common way to define complexity is by the number of steps needed to solve the task.
- 8:18
So this is by, um, this is, like, a synthetic planning benchmark that I'm working on, and I'm hoping to, like, publish very soon. Um, so I use, I use synthetic data set, uh, synthetic benchmark because it allows me to, like, control the level of complexity to study a model behavior.
- 8:36
So now I can ask the model, like, generate, like, tasks that require, like, five steps to solve. So, so, so with that, um, um, so in, in my benchmark, most models perf- don't perform quite well.
- 8:49
Like, most model can only solve, um, tasks, like, that have, um, at most, like, five... That require at most, uh, five steps. And after ten steps, um, most models fail.
- 9:01
And this is, like, consistent with another studies that I have seen. Uh, it's an older study now. It was, um, from, like, twenty-twenty one. So it's in, uh, it's in coding.
- 9:10
So the results, so the actual pass rates for the task for models must have increased a lot by, by now. However, the, the, the learning, the insight, uh, is still, like, I think still very lev- relevant.
- 9:25
So in this, um, in this paper, they try to construct, like, different docstrings, and then they ask the model, the agent, to, uh, to the model to generate code based on the docstring.
- 9:37
So they, they count the complexities. They consi- they measure complexity, uh, of the task based on, like, how many steps needed in the docstring. So for example, like, for this task, like, first you want to con- you ask the model to con- write code to convert the string into lowercase, and then you ask the model to write
- 9:55
code to remove half of the characters in the string. So, like, this c- are considered, like, two building blocks or, like, two steps in the docstring. And they found out the same result, like, um, as, uh, as I did, is that the success rates, the pass rate, like, decrease rapidly as the number of steps increase.
- 10:16
Um, but the good news is that, like, with newer models, um, they are get- actually getting a lot better with planning. So here in the same, uh, results, you can see this, like, here's this three very nice curve that come from, like, DeepSeek R1, Gemini 2.0 Flash Thinking, and o1-preview.
- 10:35
I didn't test on, like, o1 and o3 because I didn't have access to this model when I run this test. And you can see this, like, the curves are being pushing upward.
- 10:45
Like, the models, the newer models are able to solve, like, tasks with more complexity. And I do believe that this is gonna increase over time, allowing us to, like, using agent for more practical, complex, real-world task.
- 10:58
Um, so here's another, uh, result from my benchmark. So as you can see here, it shows the number of tasks that each model was able to solve. And in overall, you can see there's, like, there's a pretty big difference between, like, newer reasoning models such as o1-preview, DeepSeek R1, and Gemini Flash Thinking, and non-reasoning models just like,
- 11:19
uh, Sonnet 3.5, Gemini 2.0 Pro, or, like, GPT-4o.
- 11:27
Different use cases might define, uh, different, um, the complexity differently. So here is a paper from Zebra Logics. This, like, just came out, like, just last month, in which, uh...
- 11:38
It's a, it's a logic task. So they define each problem complexity by its number of, like, Z3 conflicts. So you can see this, like, by the... They also got the same result.
- 11:49
Like, the model success rates, like, decrease rapidly as the number of, um, as the number of Z3 conflicts increase.
- 11:59
So I think there are several tips to get the agent to handle more complexity. First, we might want to break tasks into sub-tasks that agent can solve. So you don't want to give an agent a task more than it can handle.
- 12:11
So let's say that a task, uh, your tasks, like, consistently require something like five or six steps to solve, and the agent can maybe like solve it, or can do at most like three steps, then you might want to break the task into like two sub-task.
- 12:26
Another way to like have the model deal with more complexity is do like test time compute scaling. Uh, so I, I... Test time compute, uh, but I think that in the last few years people have been talking a lot about test time compute scaling, so it's one of the very, one of the new or very exciting concepts
- 12:43
that gave rise to like reasoning models, and I'm very excited about it. So the idea is that like you can have, you can give the model more compute during inference so, so that, um, so that it can either generate like, uh, using more, um, more thinking tokens, so it can think more.
- 13:01
Or it can also, it can, it can also use the compute budget to generate more, more output. So for example, given a math problem, it can maybe like output 10 different samples, 10 different solutions, and then pick the one that like the model, like most of this out- pick the ones that's most common.
- 13:21
Like most, um, that's what the model thinks output most of the time. So yeah, so it's test time compute scaling. You can also use stronger models. So using stronger models can also call like, uh, train time, train time compute scaling because now you need to invest more compute into like training bigger model.
- 13:42
Okay, so we finished the first challenge, which is like the curse of complexity. The next part we're talking about is challenge related to tool use. So tool use is basically like natural language, uh, to API translations.
- 13:55
And what does this mean? So a lot of time for agent, right, we have humans using agent, and the human gives the agent instruction in natural language. So for example, an agent or a human might give the agent a task like, "Hey, given this customer email, create an order."
- 14:12
So the agent, um, will need to translate that into like, uh, functions that can perform this task. So it might first need to call a function to extract the customer ID from the e-mail address, and then it might call another function to extract the order ID from the content of the email.
- 14:28
And then given this customer ID and the order, you would need to actually create the order. So now you can see that it can trans- it need to translate from this natural language to just a set of API calls.
- 14:40
The challenge with this is that the challenge comes from both sides of the, of the translations. So for natural language can be like extremely ambiguous, and at the same time on API side, you can have very bad AB- API and very bad documentations.
- 14:55
So let's go into the first example of like ambigu- ambiguous natural language. Consider this, uh, agent with access to very, very simple functions, like fetch top products and fetch product info.
- 15:07
So fetch product info can return you like the product price. So let's say like the, say that the fetch top products take in like three argument, like start date, end date, and number of products, right?
- 15:18
And then the user has this query, find best-selling products under $10. So now, uh, the, the agent would need, know that it need to call the fetch top products, but what would the start date be?
- 15:30
What would the number of product be? Like, how many products should it query? And what start date, what end date should it be? Like, would it be like from like, does the user want best-selling products from like yesterday, from last week, or from last month?
- 15:43
So this is very ambiguous. Okay, so now we talk about like very, very bad API or bad documentations. In my coding career, I have been like pretty like fortunate or unfortunate to have seen like really, really, really bad comments.
- 15:59
Um, so as an engineer myself, uh, I know that like people don't usually like writing documentations. And if you can't explain the function to the, uh, to, to, to the agent, it's gonna be really, really hard for the agent to know how to use this right.
- 16:18
So I do think just like when you, when, when you give an agent, like access you a tool, you will need to provide necessary documentation. As a list, you should need to explain like what the function does, what parameters it take in, like what is the types of parameter, what does the parameter stand for.
- 16:35
You also need to show like err- different error codes for the, for the, for the functions, and also like expected like returned values. And the more details, the better.
- 16:46
And that's not all. Because like with error code, right, you don't just want like, okay, this model returns this error like step 99. Like it doesn't mean much for the model.
- 16:56
You might want to like explain to the agent like, okay, this error is usually caused by this. And if you enter this error, if you encounter this error, maybe this is how you should address this.
- 17:06
And one of the, uh, one company told me that like one of the very, very, one of the biggest improvement they got for their agents is after they explained and add to the documentations like how to interpret, uh, returned values of the functions.
- 17:22
So let's say that a function like return the value of like one, like what does that mean? So if you help the model interpret the result, the model can, the agent can actually be able to perform, like call the functions like a lot better and then be able to plan a lot better.
- 17:38
Um, another very important thing to think about is that like, um, it's very... If tool use for agents can be like counter-intuitive for us, because humans and AIs like have fundamentally different ways of like using tools.
- 17:52
So first of all, like, um, humans and AI have, have different like preference. So humans might prefer working with like visual thing, like with GUIs, whereas like AI might work better with like APIs.
- 18:03
So like if you ask a human to use Salesforce, they might go to Salesforce website. But if you like assign a task for AI, it will like, it would perform much better, like not having to deal with a lot of visual cue and just like calling the straightforward API instead.
- 18:17
And also like humans and AI operate in different way. Like Humans like, at, at least for me, uh, I find impossible to perform multiple tasks at once. So I would perform, like, different step by sequentially, whereas AI can perform tasks in, like, parallel.
- 18:32
So first of all, um, if you need agent to perform, like to browse, um, if you need to, like, browse 100 websites, um, it could be, like, very, very boring for humans.
- 18:44
Like hu- I did that for my book. Like, I browsed, like, thousand of websites, but it was not fun at all. However, for AI, like browsing 100 of website or, like, 1,000 of websites, extremely easy.
- 18:55
You can just send out, like, open like, uh, like query, like this thousand of websites and get back the summaries, and it's pretty straightforward. So that is actually a challenge for, like, training or, um, creating examples for the models to do, to, to do planning or tool use.
- 19:12
Because given a task, what the human annotator does might not be optimal for AI. So that's, that's, that's the reason why the reinforcement learning is so exciting because with supervised fine-tooling, like, uh, you are teaching AI to, like, clone human behaviors, which might not be optimal for AI.
- 19:32
Whereas with reinforcement learning, like you let the mo- the model figure it out, like with try and error, and it might find ways to do it that is optimal for AI.
- 19:43
So there are several tips, like how to make agent better at tool use. So the first is that you should create like very, very good documentations. Like, with everything, not just the function descriptions, parameters, but like error, like return values like we just talked about.
- 19:59
You should also, like, give agents, like, very narrow and well-defined functions. So I just caught up with a friend, um, working for a very big company. I, I wouldn't say the name, but if you say the search engine, you probably know what it is.
- 20:11
And he was saying that like for, for their use cases, uh, they give their agents like only three or four very narrow and small well-defined tools. For the agents, like they...
- 20:22
For their tasks, like the agents just did not work at all with like more than five tools.
- 20:27
You should also, like, uh, because of, like, the ambiguity of natural languages, it can help the models understand the task or the user query better by using techniques like query rewriting or using like intent classific- classifier to help, like, classify the user intent.
- 20:44
You can also, like, instruct or you should definitely instruct your agent to ask for clarifications when it's unsure of what users want. So for example, like if user asks like five, um, five best-selling products until, like, under $10, it can, like, make some random guess, like to fetch product from yesterday or from last year.
- 21:04
Or it can also like ask user like, "Hey, do you want top product, best-selling products from yesterday or from last week?" Um, you can also... One, one, one pretty exciting or interesting, uh, direction I'm seeing is that, like a lot of companies are building specialized action models for specific types of queries and APIs.
- 21:24
So we already have like specialized model, action models for coding, right? Now we have the model trained specially for, let's say, like VS Code or like for coding. Um, so why not have like specialized action models for different environment as well?
- 21:39
So for example, I've seen like a, a Salesforce might be interested in like building... Maybe I shouldn't say like Salesforce, but say general. Like different companies with very complex, like, ecosystem might want to like train action model for their environment.
- 21:54
Okay, two down, because of course the complexity is a tool use issue with natural language and API translations. The last one is context. And it's really funny because we have been talking about context for a long time, like first for RAG and now for agents, we still talk about context.
- 22:09
So models, like AI, has always requires a lot of informations. So before agents, like a model has already had to work, like system instructions, which can be pretty long if you, like, really want the model, really want to...
- 22:24
want the application to perform well and secure. So you might want to instruct the, the model to like what kind of queries it should respond to, what kind of query it should not respond to, what kind of tone it should carry.
- 22:35
And there are also like user instructions and like examples. But with agents, like you see a lot more, um, informations. So first you might need to pass documentations about the tools to the agent.
- 22:47
And the more tools there are, the more documentation will be needed. Um, of course, like after you call a tool, there can be tool outputs. That's a model we need to keep track of as well, and this will grow with more like execution steps.
- 23:02
And, um, after getting back like a tool outputs, right? The model may need to reason, like, "Okay, now I got this result. What do I do next?" Or like after a model generate a plan, the, the agent might also want to reason like, "Hey, is this plan like reasonable?
- 23:15
Should I execute it?" And all these reasoning tokens like take a lot of ti- take a lot of like, um, take a lot of input tokens, and this also grows with more complex tasks.
- 23:26
So like the informations that an agent can work with, like can grow very, very, very quickly. Um, and I haven't even mentioned like other kind of like information, such as...
- 23:35
such as like table schema for tasks like text to SQL. Let's say that you want to do like a text to SQL task, right? And you're not just... You, you don't have...
- 23:43
You, you have not just one table, but like 1,000 of tables. So when, when you translate a SQL query, you might need to figure out like what table to apply the SQL query to.
- 23:53
And for the model to be able to pick the right table, you might need to pass in like all the table schemas. And if you have like 1,000 like table schemas, that can be a lot of informations for the model to process.
- 24:07
So, so one thing that like I have experienced, like when I was working with agents that I would love to have more research on is like, um, how to make a model that works well with both planning and long context.
- 24:19
Because in my experience, like some models that are good with planning- I'm not necessarily the models that work with long context. And the reason is that, like, plannings are like reasoning, usually like, um, require a lot of reasonings.
- 24:31
Like, it require a lot of... generate a lot of thinking of reasoning tokens. So this kind of task are like output heavy, whereas for long context it's like input heavy.
- 24:41
And I have, in my benchmark, uh, my personal benchmark, I see in this like, um, models that perform well on my long context benchmarks don't perform as well on the planning benchmark and vice versa.
- 24:53
Um, so okay, so we've talked about, like, an agent, like, has to deal with a lot of informations, and that information might not fit inside a model's, like, efficient context.
- 25:02
So I want to highlight the word, like, efficient here, because a model might have very long context, but then it might not use that context, like, efficiently. So, like, a model might be able to fit in, like, a million tokens, but, like, if you give it anything more than, like, three thousand tokens, it might get really, really,
- 25:20
really funky and, like, hallucinate all the time. So at least, like, in my, in my personal experience, um, I have, like... Uh, yeah, so, so I've, like, done a lot, a lot of, uh, a lot of, like, um, benchmark evaluations just to see, like, at what point of the, of my documentation does the model start, like, hallucinate
- 25:38
and making up things. Um, so, so, like, if you can't fit all your information into the model context, like efficient context, you might need to realize on, like, other form of, like, information persistence or information, uh, storage.
- 25:54
So context, you can think of it as like a short-term memory. Like, you should use this for, like, uh... It should, it, it should be used to store information relevant to the task at hand.
- 26:03
And then you can also supplement it with, like, long-term memory. For example, like external databases or storage. Um, and it's very common with use case like RAG, right? Like, so if you connect a, a model to your external databases, then you're connecting it to, like, um, long-term memory.
- 26:20
So, um, you can also, like... In the case of agent, right, um, you can, like, store less immediate relevant informations in, like, external, external file. So let's say that your task requires, like, ten steps.
- 26:35
So maybe, and, and the output from this, all these ten steps, like, doesn't... don't quite fit into the context. So you might want to store the output of the first few steps into external file, and then we retrieve the output, like, when necessary.
- 26:50
And of course, like, there's also, like... So we have short-term memory, uh, long-term memory, and another level of, like, memory system is internal, internal knowledge, which is, like, the knowledge that the model already has.
- 27:02
So if you have some informations of models, like, that is essential for, like, the model to perform, like, multiple tasks, you might want to include that in the training data and fine-tune the model on it so that, so that the model can just use this as part of internal knowledge instead of, like, having to waste, like, context
- 27:18
tokens. Okay, so that is pretty much for today. Uh, so I think we talk about, like, um, what is a, what is an agent, uh, different challenges to building agents, including, um, like, first including, um, trying to, like, get the model to handle the right, the task of right complexity.
- 27:39
And we talk about tips, like how to make the model handle more complexity. We talk about tool use challenges of, like, how to translate between natural language and API.
- 27:47
And we talk about, like, how to help get model to, like, handle longer context with, like, a memory system. So thank you so much, everyone. Um, I do have a website, and if you have any questions or if you want to talk about the agent planning benchmark I'm working on, feel free to reach out.
- 28:04
Bye.