AI Engineer Code 2025
From Stateless Nightmares to Durable Agents
Read the talk
From Stateless Nightmares to Durable Agents
A game of twenty questions exposes the restart problem in long-running agents. Temporal preserves completed work, while Python keeps control of the research workflow.
From a talk by Samuel Colvin
Before you start: Familiarity with Python async functions, LLM tool calls, and basic process failures will help; prior Temporal experience is not required.
When restarting becomes the problem
What happens when an agent dies after doing useful work? For a single question and response, starting again may be acceptable. For a longer workflow, restarting means paying for completed computation again—and making the user wait through it again. Samuel Colvin explores that boundary with Pydantic AI, Temporal, Pydantic Logfire, and Pydantic Evals. At the time of the recording, Pydantic AI supports Temporal and DBOS, with roughly five pull requests proposing other execution or orchestration backends.
Deep research is the motivating workload: enough time and computation accumulate that losing progress becomes expensive. Colvin tentatively identifies Temporal as infrastructure behind OpenAI’s deep research; that attribution is not established here. His own demonstration starts smaller, with a game whose intermediate work is easy to follow, before building a research pipeline.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Two agents try to identify a potato
The game has two agents. An answerer knows the secret object; a questioner must discover it. Rather than restricting replies to yes or no, the answerer can give qualified responses such as kind of or completely wrong. The extra information helps shorten otherwise slow games. The answerer uses Claude Haiku 3.5, and the secret—potato—is added to its context.
The questioner receives instructions for playing and asks questions through ask_question. That tool runs the answerer agent and returns its response, so one agent’s tool call becomes another agent’s inference request. Colvin points to the public companion examples, then still in a pull request he intends to merge after the demonstration.
One run narrows the object through fruit versus vegetable, rules out fruit, asks whether it is orange, and eventually reaches potato. Other runs wander. Colvin reports that the game sometimes takes around fifty steps. If an endpoint fails or Kubernetes terminates the process during scaling, rerunning the ordinary script starts the game from scratch. Every completed exchange disappears from the new run’s working history.
The toy has the same dependency structure as deep research. Each answer informs the next question; a search or retrieval step plays the role of the answerer. Colvin likens it to asking the troll at the bottom of the garden for the next riddle on a quest. Replace that intermediary with web search or RAG, and preserving the chain of discoveries becomes a practical infrastructure requirement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Record external work, replay the control flow
The durable version wraps both agents in TemporalAgent, which behaves like a Pydantic AI agent and subclasses AbstractAgent. For simplicity, this version puts potato directly into the answerer’s system prompt rather than passing it through context. The agent logic otherwise stays much the same. This is the recording’s API: the current Temporal integration guide recommends TemporalDurability and deprecates the wrapper approach.
Temporal separates workflows from activities. Workflow code must be deterministic so it can reproduce the same control flow during replay. External I/O and nondeterministic operations belong outside that replayable logic, in activities. Temporal records activity inputs and results; when it reconstructs a workflow, it supplies the recorded answers instead of repeating completed external operations.
| Component | Responsibility | During replay |
|---|---|---|
| Workflow | Branches, sequencing, orchestration | Runs again deterministically |
| Completed activity | External operation with a recorded result | Returns the recorded result |
TemporalAgent | Turns model and tool I/O into activities | Preserves the agent-facing interface |
The wrapper makes the activity boundaries largely implicit, including the tool call that invokes the second agent. Colvin criticizes OpenAI’s then-current integration for lacking tool activities. The narrower distinction matters: a dated Temporal support example documents explicit activity_as_tool wrapping, so the blanket claim should not be read as an absence of any tool-activity support. Automatic wrapping and explicit adapters are different integration choices.
The runtime has three roles: a server stores execution history, a worker runs workflow code and activities, and a client starts or waits for workflows. Colvin connects to a separate local open-source Temporal server through localhost; he can reset this demo server to clear its state. He presents Temporal Cloud as a production deployment option. The launch call is execute_workflow, supplied with the workflow and its inputs. This version of twenty questions has no launch inputs because the answer is already in the prompt.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retry a failure, then kill the process
The first durable run immediately raises an exception. Colvin has deliberately made the tool fail with a 20% probability. Temporal retries, but the demonstration then appears to stop progressing; the cause is not established. He resets the local server and reruns with a 10% injected failure probability. This time, questions continue despite occasional errors. These are configured fault-injection probabilities, not observed production failure rates.
Retries alone could be implemented without Temporal. The stronger test is to terminate the agent process while it is working and preserve the existing workflow. Colvin kills the process, then opens Pydantic Logfire. Its trace shows the Claude requests and the nested activity running the answerer. Recovery proceeds through the workflow’s identity:
- Open the top-level workflow-start trace and copy its workflow ID.
- Pass that ID to the demo script’s resume option.
- Restart the worker and wait for that existing workflow to finish.
The explicit ID is a convenience for this launch script: running it normally would start another workflow alongside the unfinished one. It is not resume logic added throughout the agents.
The restarted run quickly reaches question six and continues. Colvin reports roughly five-millisecond responses for the replayed LLM calls. Those responses come from recorded activity results, not fresh inference. The workflow executes its ordinary procedural logic, consumes the saved answers, and moves forward until it reaches work that still needs external I/O. Recovery replays the program while reusing completed work. It does not ask the model to reconstruct its previous reasoning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A durable agent can still be wrong
The recovered questioner keeps running, but its reasoning is not improving: it starts exploring whether the object might be a salad bowl. That leads into a separate question—how should the models be compared? Colvin uses Pydantic Evals for twenty questions, showing GPT-4.1, Gemini, and Claude Sonnet 4.5. The displayed results include assertion outcomes, average cost, speed, and question count.
Gemini initially appears cheaper and faster, reaching an answer in fewer questions. Colvin subsequently inspects its answers and discovers why: it invents an incorrect answer, and his evaluation was not checking correctness. The apparent advantage therefore does not establish better task performance. The current companion evaluation includes normalized answer-equality checking, but that does not repair the comparison displayed in the recording.
Meanwhile, Colvin leaves the durable game after forty-six steps without a correct identification of the potato. Persistence has protected the sequence of operations; it has not made those operations useful. For this task, correctness must constrain the interpretation of question count, latency, and cost.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build research from smaller agents
The next example replaces the game with a deep-research pipeline, built as a quick prototype the previous night. Colvin considers its output competitive with other research systems, though he does not present a quality evaluation. It begins with a structured plan: an executive summary describing the intended work, a bounded list of web-search steps, and analysis instructions. The five-step limit keeps the research from running indefinitely. A compact Python representation is:
python
from pydantic import BaseModel, Field
class DeepResearchPlan(BaseModel):
executive_summary: str
web_search_steps: list[str] = Field(max_length=5)
analysis_instructions: str
The schema gives the next stages concrete fields to consume rather than requiring them to parse a free-form planning essay.
This decomposition depends on what agent means. Colvin distinguishes three common uses:
- AI: An LLM calling tools in a loop.
- Engineering: A microservice.
- Business: Something that can replace a human.
He describes a shift away from assuming one tool-calling loop per microservice. Smaller agents become units of development that combine into a larger autonomous task. Here, deep research is the application assembled from those units.
| Agent | Job | Output or capability |
|---|---|---|
| Planner | Turn the request into a plan | A Pydantic model instance |
| Searcher | Investigate a planned step | Search tools and result text |
| Analyst | Synthesize the findings | Final research report |
The searcher uses Gemini Flash for speed; the analyst uses Claude Sonnet 4.5. Colvin keeps the orchestration in ordinary Python because this control flow does not require a graph. For persistence, he prefers the finer activity boundaries of durable execution over graph-level snapshotting. The analyst also has a tool for further web search, although he does not think it uses it in this example.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Plan, search in parallel, then synthesize
The orchestration has three phases. First, await the planner. Next, launch the planned searches concurrently using Python’s TaskGroup. After the group finishes, collect the search outputs. The concurrency portion can stay this small:
python
import asyncio
from collections.abc import Awaitable, Callable, Sequence
async def search_all(
steps: Sequence[str],
search: Callable[[str], Awaitable[str]],
) -> list[str]:
async with asyncio.TaskGroup() as group:
tasks = [group.create_task(search(step)) for step in steps]
return [task.result() for task in tasks]
The task group waits for the searches; the result list preserves their planned order. Colvin then uses format_as_xml to package the collected text into readable context for the analysis agent, which produces the final synthesis.
The request is a sales-research question: find hedge funds that write Python in London. Colvin launches the script with uv run and inspects its progress in Logfire. In the ordinary research run, Colvin observes a nine-second planning step. The trace then shows searches running concurrently, followed by the analysis agent once their results are available. Individual search traces expose the questions, generated queries, retrieved material from sites including Medium, and additional context supplied to the agent.
During that inspection, Colvin refers to ten parallel searches, although the plan shown earlier is capped at five search steps. The distinction is not explained, so the demonstrated bound should be understood as a limit on planned steps, without inferring an exact count of underlying search operations. While analysis is still running, Colvin observes eight cents of cumulative spend. That is an interim cost, not the price of a completed report. Killing this ordinary process would discard its progress and require another run from the beginning.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the Python, add durable boundaries
The Temporal research version follows the same pipeline, with one recording-specific substitution. Colvin reports a Vertex SDK incompatibility with Temporal and switches to OpenAI Responses plus Tavily instead of built-in search. He duplicates the example for readability, although the shared code could have been imported.
Each agent receives a TemporalAgent wrapper. The analysis activity also receives a one-hour timeout because synthesis had exceeded a shorter limit. Colvin is unsure of that earlier default, so the useful operational point is to size an activity timeout for its actual workload, not to infer a universal default from the demonstration.
Inside the workflow, the orchestration remains imperative Python. The same TaskGroup runs concurrent searches; Colvin also names asyncio.gather as an alternative. A hypothetical seven-day sleep illustrates another benefit: Temporal can manage a durable suspension for periodic work instead of requiring a process to remain alive throughout the wait. The final stage still receives the combined context and runs the analyst.
After registering the agents and observability integrations through plugins, the client again launches with execute_workflow. This time, the question is which Python agent framework is best for durable execution and type safety. The changed research prompt does not change the execution structure: plan, parallel searches, then analysis.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recovery stops at the unfinished activity
Logfire shows the workflow starting and searches beginning. Colvin then terminates the process around the transition into the final stage. Instead of submitting the research request again, he restarts the worker and waits on the same workflow. In a deployed system, another available worker can continue the stored workflow after the original process disappears.
In the recovered research run, Colvin observes a twenty-four-millisecond replayed planning step. Completed searches also return their stored results almost immediately. The analysis stage still has to execute: Temporal has no completed result to substitute for that activity.
| Work at interruption | Recovery behavior |
|---|---|
| Completed plan | Reuse its recorded result |
| Completed searches | Reuse their recorded results |
| Analysis without a completed result | Execute the activity |
Durability preserves completed activities, not arbitrary progress inside an activity. If an activity is interrupted before completion, it must run again from its beginning. Choosing the activity boundary therefore determines how much work a failure can force the application to repeat.
The report eventually completes and recommends Pydantic AI with Temporal. It includes an executive summary, links, and discussion of Pydantic AI, LangGraph, and Temporal alone. Colvin’s favorable assessment of the report and jokes about competing frameworks are promotional commentary, rather than comparative findings established by this run. What the demonstration does establish is the completion of research after recovery; a polished interface for presenting that research remains to be built.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The examples and the forthcoming gateway
Colvin closes by returning to the public pydantic-stack-demo repository and reiterating his plan to merge the durable-execution examples. The repository and on-screen QR code give viewers a route to the code behind the demonstrations.
He also previews Pydantic AI Gateway, still forthcoming in the recording, and invites requests for early access. His proposed capabilities include buying inference from major providers and most open-source models, enterprise self-hosting, and observability. This extends the stack beyond orchestrating and inspecting agent work to providing access to the models that perform it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Python examples for twenty questions, deep research, evaluations, and Temporal and DBOS execution.
Evaluation code measuring question count and answer correctness across a small object dataset.
Setup documentation for the observability platform used to inspect the demo's agent runs.
Updates since the talk
Current setup and execution guidance, including the capability API that supersedes the demonstrated agent wrapper.
Current Python integration examples, including exposing Temporal activities as agent tools.
- Pydantic AI Gateway launchArticle
Samuel Colvin's launch announcement explains provider access, spending controls, observability, and deployment options.
Read the complete timestamped transcript
- 0:00
Hi, I'm Samuel from Pydantic, and today I'm gonna give a demo of Pydantic AI, Temporal, and Pydantic Logfire. I'll also cover Pydantic Evals. So we have in Pydantic AI support for Temporal and DBOS, two durable execution frameworks.
- 0:16
We're actually adding a bunch more. I think we've had something like five pull requests to add other durable execution or like workflow orchestration backends. But at the moment it's, it's these two, and I think it's fair to say Temporal are like the big incumbent in this space, and they're, they're kind of, I guess, leaders, leaders in how
- 0:33
you do this. To, to demonstrate this simple example of like I go ask an LLM a question, it replies, mostly just works, and we don't need the durable execution component.
- 0:43
So, but once we get into longer-running workflows, that's where it really becomes a problem. In particular, where we've done enough compute that we don't wanna lose it, or we've spent enough time on that compute that we really don't wanna have to start again for the user.
- 0:56
That's what, for example, I, I think OpenAI, it's-- I think it's public that OpenAI use Temporal for their, um, deep research, and I think some of the other LLM deep research do the same thing.
- 1:06
So I'll, I'll start with a kind of toy example, and then I'll move on to a more deep research type example. In fact, a deep research example. But before we get into that, let me, let me run this example without.
- 1:16
So this is a, uh, two agents that play, um, twenty questions. Um, so instead of just a yes/no answer, they get to give a little bit more detail like, yes, kind of, not really, no, completely wrong.
- 1:28
Um, that was because I was getting bored of waiting for them to take ages to succeed on the... with the just a yes/no. So we have an answerer agent, which is runs a relatively small model, Haiku three point five, 'cause I didn't know four point five came out until about an hour ago.
- 1:41
And, and this basically takes a question and answers yes or well, with, with one of these answers, and it, it gets added into its context the, the, like, secret object that you're looking for, um, which in this example is a potato.
- 1:53
So and then we have the, um, questioner agent or the player agent has a bit more context on what it's gonna go and do. You don't need to read, read through all of this stuff.
- 2:03
This code is, i-is public now. It's a pull request, but I'll, I'll merge that afterwards, but you should get the idea. Um, and the way that the questioner agent gets to ask its questions is by calling a tool, ask question.
- 2:17
Inside that tool, we run the other agent, the answer agent, to basically decide the answer to this question, and then we respond, and it takes a little bit of time to run.
- 2:26
You can see in this case it succeeded pretty quickly. Sometimes it's amazing how even these very simple questions, even very intelligent LLMs get themselves completely confused and go down some weird track and, and, like, get very confused.
- 2:39
But you can see in the last run that it asked a bunch of questions, got down to like, "Is this a ve... fruit or vegetable? Is it a fruit?"
- 2:46
No. So it knew it was a vegetable. Is it orange? And it worked out the answer was, was potato. But obviously, and here it is running again, I don't know how many steps it's gonna take.
- 2:56
Sometimes it can take up to like fifty steps to get this question right. And obviously the problem is if this dies, either because we have some unreliable, uh, endpoint within our system or because we're running in the cloud and Kubernetes decides it wants to scale or whatever it might be, if we run this again, we obviously have
- 3:16
to start from scratch. That is problematic in this case, uh, but you can imagine as the tasks get longer and longer, just restarting it gets more and more problematic.
- 3:26
So, uh, um, and I think the other thing to say about this, this like, um, twenty questions example is, although it's pretty simple to understand and it feels like a toy, it is actually directly equivalent to a deep research case where effectively the agent is, is like going off on a quest to go and find an answer
- 3:43
to a question where it needs to an-- ask the like, you know, troll at the bottom of the garden the, the other, like, uh, question to the next riddle to get to the next endpoint, right?
- 3:52
Like deep research is effectively this twenty questions just with like web, like web search or RAG or whatever it might be is your intermediate steps. So let's turn that twenty questions into a durable agent.
- 4:06
Um, so this is mostly the same code. For simplicity, I've actually copied it here, but you see we have our answerer agent. We need to wrap it in this temporal agent, which gives us another thing that behaves like an agent, uh, like a Pydantic AI agent, so it's also a subclass of abstract agent.
- 4:22
We do the same with our questioner agent. To keep things simple, we aren't doing the same, um, stuff about passing around the answer in context. We've just hard-coded the answer into the system prompt here for the, for the answer agent.
- 4:36
But apart from these, like adding the temporal wrappers, you can, as I will show later, just, um, apply durable execution later. But here's where the, the temporal bit comes in, and I'm not a salesperson for Temporal, and although the underlying stuff they do is am-amazing, I do think some of their Python abstractions are kind of ugly.
- 4:54
But anyway, the, the principle of Temporal is that you have workflows and activities, and workflows need to be entirely deterministic, and activities then need to do anything that is non-deterministic, like I/O in particular.
- 5:09
So you can basically do anything inside a workflow other than I/O and calling random. And if you're, if you, if that's the case, then you have a deterministic system.
- 5:18
And what you can think of what Temporal is doing in the background as it's running that workflow, and it's basically recording the every activity that runs and both the, the inputs to that and the outputs.
- 5:30
And so if you want to rerun it from like from the beginning to a certain point, it can basically plug in those answers. And I'll show what that looks like.
- 5:37
So this is how we define our workflow. The activities here are implicit. The point is that this temporal agent takes care of turning all of the I/O that you need to do to call an LLM into activities in the background, including tool calls.
- 5:52
So OpenAI claim to have temporal support, but they don't support tool calls as activities, which to me makes it slightly a chocolate teapot. Like there's actually no point in having any of these things without, without tool calling or little, little point.
- 6:05
But We define our workflow like this. I think you can, for the most part, just copy-paste their, their definitions of how to do it. Here we have our play mechanism.
- 6:14
The point here is we're gonna, we're gonna connect to the Temporal server, which is what's gonna record the state of our task or our agent as it's executing and be able to resume and stuff.
- 6:24
I have Temporal running locally here. This is just the open source version of Temporal, which runs as a separate process, and I can restart that to kind of kill the state, and that's why we're connecting to localhost.
- 6:35
In production, you use Temporal's cloud. That's why they make so much money. Um, and here we, we run the worker. This is where we're actually gonna kick off our workflow.
- 6:44
In general, we just kick off our workflow with execute workflow. We plu- pass in the, the workflow that we want to run. We pass the inputs. There aren't any inputs in this case because we just start, so there aren't any here, and it will then go and run.
- 6:58
And so if I run this, we will... You will see it start to, to execute.
- 7:06
You'll see it running. The only couple of things to note, there's a couple of log messages. Ah, and we immediately have this exception broken, and that was because to simulate some system that's unreliable inside the tool, I added 20% of the time it's gonna, it's gonna break.
- 7:22
What you will see is that Temporal has immediately taken care of continuing after that. So even though this broke, it will continue to run, and I may have set 20% to be too high, um, because it's now failing all the time, but it's actually gonna continue and deal with those runtime errors and just continue to operate absolutely
- 7:40
fine. Let me-- However, I think I dialed up 20% too high just before, so I'm going to... I'm going to actually see if this is gonna continue to operate.
- 7:49
Obviously, when you give a demo, everything suddenly grinds to a halt. Someone recently said they hate demos where everything goes to plan, and I said, "You never need to worry about that with me."
- 7:57
Um, I don't know why that has actually ground to a complete halt. I don't know whether that's it just repeatedly failing.
- 8:06
Let me-- I'm gonna kill Temporal server here and restart it so that we don't have the state stored, and I will clear this and run it again. And you should now see it succeeding most of the time and failing ten percent of the time.
- 8:19
So yeah, you see it now asking questions and occasionally breaking, good timing, but continuing. Um, so that's one of the things Temporal does. It just does the, like, retry logic that is like, well, you could implement without Temporal, but they do it very nicely.
- 8:33
But there are more powerful things. So let's say this process that's in the middle of running gets killed by Kubernetes. Now, so we go across here, and we, we just, like, kill it.
- 8:42
The process gets killed. Now, what I didn't show you is I also instrumented this with Logfire. So if we look at our, our workflow, we can see exactly what was going on here.
- 8:52
Um, and we, we can see what's going on here. We have the different calls to Claude, and then inside that we have-- we're running the activity, which is then running the, um, other agent.
- 9:02
But in particular, if we come to the top-level start for the workflow, I can take the, the workflow ID, and if we come back over to code here, you'll see I had some, some code in here to basically allow me to continue with a given, uh, resume ID and to continue a workflow.
- 9:17
Now, for the most part, you wouldn't have to do this. This is just for the sake of the demo. If I just reran the script again, it would kick off this workflow again, and it would run the two in parallel.
- 9:26
That would look really confusing. So instead of doing that, I'm, I'm specifically hanging on a particular workflow to finish so you can see what's going on. So if I r- if I run my script again, but I give it the workflow that was ongoing, now you see it's already whizzed forward to question six, and it's continuing to
- 9:44
operate. So we've got it to basically resume without having to add any resume code anywhere in our actual agent code. We just set up Temporal, and it works. And you can see exactly what's happened if you, um, look at Logfire.
- 9:57
What you will see is that, that whole, that first bunch of, um, calls to the LLM responded in like five milliseconds. So these were not actually sent to the LLM.
- 10:07
Temporal just returned the result, the kind of cached result that it already had for each of these cases. So we're able to effectively zoom forward to the point where it then continues to, to call the LLM.
- 10:19
It's, it's as if you've gone through everywhere that you're doing I/O, and you've set up, uh, caching on each individual call so that you can run your code. Uh, I see some people nodding, which is making me feel a bit better about explaining this.
- 10:30
But we don't have to do the inference. We don't have to wait the time. We can basically run our workflow codes. It's generally very fast because it's no I/O, it's just procedural, and it will just keep getting results instantly until it gets to the point where it needs to, needs to continue.
- 10:44
And you see in this case it's got itself completely confused, and it's off, um, wondering about whether this thing is a salad bowl. So you see how sometimes the LLM does well, sometimes it does, does terribly.
- 10:55
Um, I'm gonna... Actually I'll leave that running to see whether it... You see it's g- it knows it's related to food, but it's got itself really confused. Um, I will just say as, as it might interest you, just before this, I was k- wondering how the different models would perform.
- 11:08
And so I, I ran some evals with Pydantic Evals on these different cases, and you can see here we have... It's a bit hard to read on the screen, but, uh, GPT-4.1, Gemini, and Claude Sonnet 4.5.
- 11:23
And you can see the different assertion for each case, whether they passed or failed here. And you can see the, the average cost. You can see Gemini was way, way cheaper, way, way faster.
- 11:36
And somewhere we should have, if we look at an individual case, we have a, a, a metric for how many steps it took to succeed. I think maybe we have to scroll over.
- 11:45
Yeah, question count. You can see here Gemini was way quicker each time. I discovered subsequently having, having checked the results that actually the reason Gemini is way faster and answers much more quickly is it just invents an answer that's wrong, and I wasn't checking it.
- 11:58
So this is not perfect yet, but like, it's, uh, the, the ev-- There's, it's definitely an interesting case for evals and seeing and, like, working out which model is actually better because they're definitely not particularly good at it by default.
- 12:08
But yeah, in my naive case, Gemini did way better, but that's not representative. Anyway, I'm gonna leave that 'cause it's got forty-six steps in, and it's still failing to work out that that thing's a potato.
- 12:19
I, I can show the evals case if you want, but I think it might be more interesting to look at a Deep Research case, which is a kind of more meaningful example of where you would run durable execution, um, and also doing stuff in parallel, which is also one of the things that, like, just works out of
- 12:34
the box with Temporal without you having to write any, any code. So this is my very quick last night hours attempt at building Deep Research. I honestly think it's as good as lots of the actual Deep Research systems.
- 12:47
So we have-- We define our plan for Deep Research, and this is, this is effectively our Deep Research plan. So it has an executive summary, what you would effectively pump out to the user about what I'm gonna go and do.
- 13:00
Then we have a list of web search steps. We maximum-- We have a maximum of five here, so it doesn't take forever. And then we have analysis instructions. And the point is that, like, I mean, I think this is one of the big change in AI this year, answering a bit the question I had on the other
- 13:14
Zoom. I think we've moved from thinking that, like, agents in the sense that... So there were t-three definitions of agents. There is the, like, AI definition, which is LLMs to-calling tools in a loop.
- 13:26
There is the tech definition, which is a microservice. And then there is the business definition, which is something that can replace a human. Um, ignoring the business definition for a minute, if you think about the AI and the, like, engineering definitions, we thought at the beginning of this year you would have one AI agent, one, uh, LLM
- 13:44
calling tools in the loop within each microservice. I think we've moved more and more to think that the agents are actually the kind of quantum of development. They are the, the, the microtasks that are doing--
- 13:56
that you build up to, to form a, like, what most people would think of as an agent, something that actually goes and autonomously completes a task. And so our Deep Research agent is actually made up of multiple agents.
- 14:05
So we have this plan agent, which goes off with a prompt, and it returns an instance. Structured data extraction gives you an instance of this Pydantic model, which is your plan to run it.
- 14:15
Then you have the search agent, which has access, in this case, to search tool, or I'll show using Tavily in the other case, um, which is using a, a faster model, Gemini Flash, uh, in this case.
- 14:26
And then you're using, in this case, I'm using Claude Sonnet 4.5 for the final analysis stage. So I suppose this is a bit what people talk about when they're leaning towards graphs.
- 14:36
I haven't built this in a graph, although I could, because it doesn't need a graph. It's not complex enough to need a graph. And durable execution is a way better way of getting snapshotting, but, like, much more granular support for, for durable execution.
- 14:48
We added a tool that allowed the analysis agent to do a bit more web search if it really wanted to. I don't think it uses it. But this is the actual Deep Research code.
- 14:57
So you can see how concise it is. We run the plan agent. We get back our plan. We run in parallel all of the search agents. So we're just using a task group from Python to run all of these.
- 15:10
We get those results, which will all be the, the text results of the different bits of search. We use format as XML to basically smash all of that into a massive lump of reasonably readable data for the analysis agent.
- 15:21
And then we go off and run the agent. And we run the kind of question that I'm asking AI relatively re-regularly for sales, which is find me a list of hedge funds that write Python in London.
- 15:30
And if I go and run this, uh, UV run Deep Research,
- 15:39
we'll see it starting to churn away. We can see it in the terminal with Logfire, but we can also
- 15:48
come over here to Logfire. Let me clear that. Um, go to the bottom here, and we can see this, this run here. As it's going on, it's, it's run the plan step in nine seconds, and you can see all of the search steps going on in parallel.
- 16:04
Once they've finished, it will start. You can see the analysis agent has just started. We can look at the individual searches, so you get a pretty good idea of what happened, the question it got asked, uh, the queries it decided to run, bunch of data from Medium, different sites, structured data, and then, like, the agent also bangs
- 16:25
in quite a lot of context, right? So this is each individual one of our, like, ten parallel searches, and now the analysis is gonna go and run with all of that input.
- 16:35
You can see so far we've sent-spent eight cents on this particular run. We'll see what it gets to by the time it finishes. But obviously, the problem with this is if I kill this now, it's just gonna die, and I'd have to restart from the beginning if I wanted to, to run it again.
- 16:50
So while that churns away, let me start introducing you to the durable execution example. Spoiler, it's gonna be pretty similar. I discovered last night that there's a bug with the Vertex SDK that means that you can't use it with Temporal right now.
- 17:06
So I swapped out. Uh, I think we should fix that, or at least I'll be whinging at, uh, DeepMind today to go and fix that. So I've switched it out to OpenAI responses here, and I'm using Tavily instead of the built-in search.
- 17:20
Yeah, but other than that, this is all pretty similar code. I could have probably imported the code from the other module. I just decided to duplicate it just to keep things easy.
- 17:30
But you see again, we do the same thing. We wrap, uh, our agents in Temporal agent. This analysis one can take more than, I think, whatever the default activity duration is 'cause it's a long, basically build up a, a summary.
- 17:42
So I give it a, I think it was taking longer than two minutes or whatever the default is, so I just gave it an hour so it was not gonna fail.
- 17:49
Um, and then the, the-- for me, the most powerful bit is everything here inside my workflow looks exactly the same. I don't have to do any crazy stuff to do parallelism.
- 17:59
I just use a task group exactly the same. I could use async I/O gather. It's all just imperative Python code as you would be used to. I could have a-- If I wanted to run this periodically, I could sleep for seven days in here.
- 18:13
Temporal would take care of pausing everything. Again, I'm not here to be a Temporal salesperson. I don't love everything about what they do, but it's a pretty powerful way of thinking about code, where you don't have to do all the infra stuff.
- 18:24
And then ultimately, again smash all my context into the last agent and run it. And again, there's a bit of plugin stuff. I have to plug in log... Have add some plugins, add the agents as plugins.
- 18:35
But again, my code to actually go and s- kick it off is just execute workflow, simple as that. And I asked it here slightly more controversial question of what's the best Python agent framework to use for durable execution and type safety?
- 18:48
And we will pray to God it gives the right answer when we run it in front of everyone. If I go and kick that off and run this
- 18:56
again, we should see it. If we come over here, we should see it running in Logfire. You can see we have the stuff related to kicking off the agent.
- 19:05
It's kicking off the workflow, excuse me, here, and we have the searches beginning to happen, happen. But the, the powerful bit here is, again, imagine that we're halfway through running all these searches.
- 19:16
We're about to start the final step, and something comes along and kills the process. And by... In general, you'd have to go and completely restart this process and run your deep, deep research all over again.
- 19:29
With Temporal, it will just go and rerun that workflow automatically. In this case, I'm restarting it and just running that one workflow. But y- in general, it would just automatically go and be restarted, and on the, the next time the Kubernetes comes up, the workflow will run as it would've done before, but it will get answers to
- 19:47
each individual question basically instantly. And so if it's not gonna fail for me, which it seems to be... There we are. It's started again. You see our plan took twenty-four milliseconds.
- 20:01
Search all took no time at all in the grand scheme of things because it just got the result back from Temporal immediately. And then the analysis, that was the, the, the task we needed to, that we haven't run yet, obviously, that needs to go and start again 'cause that's an activity, and you can't...
- 20:16
Activities obviously have to run again from scratch. And so once that finishes, I think it does take quite a long time. Maybe I can show the previous output, or did we not get to displaying the previous output?
- 20:29
Did it ironically actually fail the time before? But hopefully once this finishes, we should be able to see, uh, its analysis, which, you know, I think is on a par with what I see from the other Deep Research things.
- 20:40
Obviously, there will be some, there's some UI work to do to display this in a nice deep, Deep Research interface. There we are. It's completed, and it has primary recommendation is Pydantic AI with Temporal.
- 20:51
So it, it, it did what I hoped it would do, and you see it's given a reasonable report here of like the relative trade-offs of the other inferior agent frameworks, and it should have done an executive summary at the beginning with, with links.
- 21:04
Yeah. So it said Pydantic AI, LangGraph, obviously if you love snapshotting or writing unsafe code, type unsafe code, Temporal on its own, which makes sense. Yeah. So there's a, there's the summary.
- 21:15
That is the main stuff I had to show. I will merge the, the durable execution stuff in here. So go here. And I just... Other thing I wanna just say quickly while I have, I can't work out how to post a comment, but like you'll find it on Pydantic if you, if you, if you look for it.
- 21:29
Um, oh, I have f- I can do that. If anyone wants to take a picture of that QR code. The other thing I just wanted to mention, we're about to announce, uh, Pydantic AI Gateway.
- 21:38
So if anyone wants to try it early, let us know. Um, but yeah, that platform will be landing soon. You'll be able to use Pydantic AI Gateway directly to buy inference from any of the big models or most of the open source models and self-hosting for enterprise, all the observability stuff.
- 21:54
But I, I'll, I'll save you the full spiel, but that's coming soon. I think some of you will find it interesting. That's it. Thanks so much for watching. If you wanna learn more about Pydantic AI, Pydantic AI Gateway, or Pydantic Logfire, please scan these QR codes.
- 22:07
If you have any feedback, uh, please come and talk to us. Thanks so much for listening.