AI Engineer World's Fair 2026
Your agent architecture has a half-life of 6 months
Read the talk
Your agent architecture has a half-life of 6 months
Dan Farrelly explains how separating execution, context, and compute can let an agent change models and tools without rebuilding its reliability machinery. A scheduled health-check and triage system makes the design concrete, including how to evaluate what happens after the agent finishes.
From a talk by Dan Farrelly
At a glance
Ideas worth remembering
Separate execution, context, and compute so a change to a model, prompt, or sandbox does not force a rewrite of recovery and coordination.
A long run needs durable state outside its worker, flexible invocation, and a trace spanning triggers, model calls, tools, and infrastructure failures.
The health-check example has two independent schedules: a thirty-minute assessment conditionally launches triage, while a weekly reviewer reads execution history to assess the system.
Evaluation can wait for downstream events—such as an engineering PR or saved research—and attach those outcomes to the session that produced them.
Which parts of your agent survive a rewrite?
A new model arrives. A framework changes. A new tool-calling standard or agent pattern becomes attractive. Suddenly, the architecture you shipped six months ago no longer fits. Dan Farrelly, CTO and cofounder of Inngest, opens with a practical question: which parts of that system still run as originally written, and did they survive by design?
The question applies whether you adopted a framework, wrote a custom harness, or accumulated an architecture through successive changes. A diagram of components and connecting lines can describe the current implementation without explaining which responsibilities should remain stable. The talk instead develops a model of conceptual layers: a way to decide where a change belongs before it spreads through the system.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Execution, context, and compute change at different rates
Farrelly divides an agent into three layers, using a deliberately simple analogy:
- Execution—the brain: flow, state, durability, and retries. This layer manages how work proceeds and survives interruptions.
- Context—the knowledge: models, prompts, tools, and memory. This is the layer he expects to change most often.
- Compute—the hands: sandboxes, runtimes, and automated browsers. These provide environments in which the agent can act.
The title’s “half-life” borrows the idea of decay from science. Here it is an architectural metaphor, rather than a measured replacement rate: Farrelly estimates that prompts may last weeks, models months, and well-designed execution years. The useful distinction is that these responsibilities age at different speeds. A prompt revision should not force a rewrite of the machinery that preserves completed work.
Coupling lets the shorter half-life spread. Orchestration can disappear inside a chain or framework; state can end up inside a sandbox; retry handling can become entangled with prompt logic. Each choice makes replacing one piece require changes elsewhere. A framework may make the first version feel magical, while a custom implementation may take weeks or months, but neither approach automatically creates useful separation. Abstractions can be missing, too high-level, or merged together.
Execution is the place to invest in stable primitives. Its job is to run code reliably and manage how, when, or whether each piece of work completes, independently of the infrastructure hosting it. For an agent, that includes planning, calling a model, running code, invoking a subagent, looping, retrying, and coordinating. The intended result is straightforward: swap the model, context, or sandbox while retaining the execution layer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resume at step thirty-eight and trace the whole session
Resumability gives that separation an immediate purpose. Suppose a request fails at step thirty-eight of a long agent run. The system should retry, wait if necessary, and continue from there. Restarting from the beginning can waste tokens, money, time, and work the agent has already completed. A failed model or tool call should not erase the useful progress that preceded it.
For a three-hour run to survive the loss of its worker, its recovery state must live outside that worker. Memory and worker-local disk cannot be the sole source of truth. Farrelly calls for durable, external state. Manual checkpoints or a log that reconstructs state can address recovery, but they also create a design risk: checkpointing and restoration logic can leak into the rest of the harness, making unrelated parts harder to change. The talk establishes this requirement without specifying a persistence or replay algorithm.
Invocation needs its own flexibility. Different entry points and timing requirements should compose within the same execution machinery:
- Entry points: cron schedules, events, APIs, and human participation.
- Delegation: subagents and dynamically chosen workflows.
- Timing: synchronous calls, asynchronous work, and delayed invocation.
Without these primitives, the harness starts absorbing queues, workers, polling, backoff, and scheduling. The agent logic becomes responsible for infrastructure concerns every time a new pattern needs them.
Observability must follow the entire session. Model and tool calls are only part of the path: a database error, permission problem, trigger, or performance issue can determine what actually happens. A trace that starts at the trigger and follows work through the stack makes those causes available for debugging. It also provides the history needed to improve the agent; an incomplete trace leaves both developers and evaluators guessing about the missing work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A sandbox executes work; the run must outlive it
Sandboxes let agents execute code, browse, manipulate files, and work on disk. Farrelly treats them as ephemeral compute environments and warns against making sandbox snapshots the mechanism that keeps an agent run alive. If the sandbox owns the run’s state, losing that environment leaves the system trying to reconstruct its progress. A separate execution layer supplies the sandbox’s context, sequence, and durability while the sandbox does the immediate work.
This becomes more important as work moves into the background. A background agent has no person sitting in a request-response exchange waiting for the next message. It may run for minutes or hours and make hundreds of calls. Farrelly uses a possible two hundred tool calls to illustrate the exposure to failures, rather than giving a measured failure rate. Recovery and tracing have to span the process—or multiple processes—because there may be no interactive user present to notice where it stopped.
A loop adds another requirement: the system repeatedly assesses its current state against goals or criteria and decides what to do next. It may run continuously or on a schedule. That requires scheduling, delegation, reliable execution, and inspectable history. Emerging background agents, dynamic workflows, and autonomous loops can also combine, so their execution needs to be inspectable by both humans and agents. The architecture must support the combination rather than assuming one fixed conversational pattern.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A thirty-minute health check delegates deeper investigation
The code example makes the layers concrete with a health-check system. Every thirty minutes, a cron-triggered function collects high-level system metrics and passes them to an LLM for a health assessment. If the assessment indicates a problem, the function can invoke a triage agent for the individual service. The useful code distinction is where that conditional invocation sits: scheduling starts the check, context informs the assessment, and the assessment determines whether deeper work runs.
typescript
export const infraHealthCheck = inngest.createFunction(
{ id: "infra-health-check" },
{ cron: "*/30 * * * *" }, // Every 30 minutes
async ({ step }) => {
const metrics = await step.run("fetch-service-metrics", async () => {
return await fetchServiceMetrics(); // error rates, latency, memory, CPU
});
const assessment = await step.run("assess-health", async () => {
return await callLLM(`Given these service metrics, classify overall system health
as "normal", "degraded", or "critical". Explain your reasoning.
Metrics: ${JSON.stringify(metrics)}`);
});
if (assessment.status === "degraded" || assessment.status === "critical") {
await step.invoke("triage-incident", {
function: incidentTriage,
data: { metrics, assessment, services: assessment.affectedServices },
});
}
}
);
The triage agent then gathers more context and detailed metrics. It passes that material to the LLM and enters an investigation loop, calling tools to seek the root cause. The work may take a minute or several minutes. It may need a sandbox, a cloned codebase, or commit analysis. These are possible investigation activities in the example, rather than a demonstrated incident with a reported root cause.
The conditional branch creates an operational question: did triage run at all? Its timing depends on the health assessment, and its duration depends on the investigation. Observing only the scheduled check would miss the delegated work; observing only model calls would miss why the investigation started. The execution history needs to connect the trigger, decision, and subsequent work so the system can be inspected later.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The weekly reviewer reads execution history
A third function reviews the system every week. It looks at the history of the health checks and triage work to evaluate whether the system performs as intended. Its questions are specific: do the prompts need adjustment? Are the metrics useful? Does the agent have the data it needs? Is it doing anything, or is it overreacting? The thirty-minute check and weekly review have separate schedules; the reviewer does not need to run immediately after every investigation.
How can a periodic check, conditional investigation, and independent weekly review fit together? The relationship below separates the trigger that launches triage from the history that informs review. This matters because reading past runs is an information dependency, not a requirement that each triage run invoke the reviewer.
The reviewer must understand orchestration as well as model output. It needs logs showing what executed, what the agent chose to execute, which subagents it fanned out to, and which workflows it called. With that history, it can analyze performance and either make a change or return recommendations about where the system underperformed. This is why execution awareness is part of evaluation: a reviewer cannot assess delegated work from the final answer alone.
The example stays deliberately small: three functions for checking, investigating, and reviewing. Their separate responsibilities make it possible to change the prompts, metrics, or investigation behavior without treating the entire system as one indivisible agent. Farrelly acknowledges that production becomes more complex; the example supplies a structure for that complexity, rather than a complete production implementation.
Two schedules, conditional triage, and a shared history
Read the diagram as text
- Every thirty minutes. Control — Cron starts the health check.
- Health check. Execution — Collect high-level metrics and ask an LLM whether the service is healthy.
- Conditional triage. Execution — Gather detailed context and investigate through model and tool calls; sandbox work may be needed.
- Execution history. Information — Logs expose what ran, delegated subagents, and invoked workflows.
- Every week. Control — An independent schedule starts review.
- Reviewer. Execution — Evaluate activity, overreaction, prompts, metrics, and missing data; make changes or return recommendations.
- Every thirty minutes → Health check: Control: scheduled invocation.
- Health check → Conditional triage: Control: invoke if unhealthy.
- Health check → Execution history: Information: record what happened.
- Conditional triage → Execution history: Information: executed investigation and delegation.
- Every week → Reviewer: Control: scheduled invocation.
- Execution history → Reviewer: Information: history and logs for evaluation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Attach later outcomes to the run that produced them
The execution layer also offers a place to connect evaluation data. It sits between user input and the work performed throughout the system; user feedback, actions, and session results can flow through it. Connecting those signals to the actual execution makes it possible to score what the agent did, rather than evaluating an output detached from the path that produced it.
Inngest is Farrelly’s product implementation of this execution layer. He describes durable steps, event triggers, scheduling, agent-to-agent coordination, and full session traces, with no infrastructure for the application team to manage. The intended integration lets the team bring its own model, framework, tools, sandbox, or browser. These are the capabilities presented in the talk; the code example does not compare providers or establish performance results.
Scoring can happen after execution, when the inputs, outputs, and full trace are available. It can also wait for additional events and attach them to the same session. This uses the same delayed-work and event-coordination primitives as the agent itself. The important timing change is that an evaluation need not end when the agent returns an answer: later activity may reveal whether the work was useful.
The ending gives concrete outcome signals:
- Triage: did the investigation lead the engineering team to act, such as opening a PR?
- Research: was the research saved, and was the report useful?
An engineering action is a possible positive signal, not proof that the diagnosis was correct. Farrelly’s point is that these downstream events can support outcome-based scores when they are connected to the originating session, extending evaluation beyond a thumbs-up or thumbs-down response.
The final advice is to build the harness with an explicit understanding of its layers. Models, prompts, and tools will continue changing quickly. Execution primitives that preserve progress, coordinate work, and connect traces to outcomes give those changes somewhere to land without repeatedly rebuilding the whole agent.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:00
[upbeat music] Going good? Oh, all right. There we go.
- 0:16
All right. Hello, everyone. All right, just checking voice. All right, good. All right, well, uh, my name's Dan, and I'm here to talk to you about how your agent architecture has a half-life of six months.
- 0:33
But first, who am I? I'm Dan. I'm the CTO and co-founder at Inngest. And why should you listen to me? Well, first, I lead an amazing team over at Inngest.
- 0:45
We, we build a system that reliably executes anything from, uh, agents to workflows to context pipelines, whatever you want.
- 0:56
I'm deep building AI infra every day, and on top of that, I'm also building agents. I'm not just pontificating up here in theory about how you should build them.
- 1:07
So speaking of building agents, if you've been building agents for more than six months, you've likely rewritten something, maybe more than once. A new model, a new framework or framework version, a new tool calling standard, a new pattern.
- 1:25
Suddenly, your architecture doesn't fit. It's not a complaint. It's just the reality, right? Things move faster than ever. You can see if you've been to any of the, the sessions or keynotes.
- 1:39
So I want you to think about what you shipped six months ago and how much of it still runs the way that you originally wrote it. Some parts of your code likely survived, but did they survive by accident, or did you design it that way?
- 1:59
Did they survive by design? How did you actually architect your agent? Did you architect it at all? It's okay. Uh, did you use a framework? Did you custom roll something?
- 2:14
Was the architecture an intentional design that you really thought through or just kind of evolved into what you have now? These are all various ways of where people are these days.
- 2:26
So a lot of folks have talked about harness architecture, building harnesses. But mostly, I think a lot of people draw diagrams, and they talk about specific components, draw just nice lines.
- 2:40
It looks very similar, uh, like, you know, kind of simple. But I wanna talk about the conceptual layers when you're building a system like this. This is maybe the mental model, not specific components.
- 2:54
So in my opinion, there are three discrete layers. First, the execution layer. I think of this as the brain. It's where flow, state, durability, retries happen.
- 3:09
Then there's the context layer. This is the knowledge. Right? This is models, prompts, tools, memory. This is the layer that changes the most.
- 3:22
And then there's compute. There are, um-- This is the hands, right? This is sandboxes, runtimes, browsers that you're automating.
- 3:32
And I think these layers are important to consider for the following reason. It's your half-life. And what is half-life, right? It's a scientific term, and it's a scientific term for the time that it takes for your-- for something to decay by half.
- 3:51
And I think that your architecture has a half-life also.
- 3:55
So prompts last weeks, if you're lucky, maybe, maybe a single week. Uh, the models that you use, months, again, if you're lucky.
- 4:04
Uh, but I think that execution can last years if you do it right. So the problem is that I think that most teams couple everything together. And what happens then is that one layer's half-life kinda leaks and drags the other components down.
- 4:21
You're building-- It's technical debt by another name.
- 4:26
So my thesis is think in layers. Decouple them.
- 4:33
So let's talk about what I mean. Teams run into a lot of issues with the approaches that they choose.
- 4:40
You might choose a framework. It might feel super magical to get going. You might grab a pre-built harness from one of the frontier labs or wherever, and-- or you might custom roll the entire system yourself, and it might take months or weeks or whatever.
- 4:55
But I think in a lot of these situations, the abstractions either are not there at all, or they're too high level, or the layers merge. You know, like orchestration is buried deep inside the chain or inside the framework, and you don't know what the heck's happening.
- 5:11
Uh, state might be kept in a sandbox. Uh, retries get tangled with prompts, the prompt logic. And what's hard is you can't re-- you can't, like, swap any of these things out without rewriting almost everything.
- 5:26
So I think you need to embrace the change, know that things are going to change, know that things are moving fast. But I think also with that is I wanna focus on a layer that I think is the stable layer here, where you can invest and think about, get your abstractions right, so you don't need to rewrite
- 5:45
a large component every six months. So I'm talking about the execution layer. I don't think enough people talk about this, and I define it as, uh, the execution layer as being the system responsible for running your code reliably.
- 6:02
Managing how, when, or whether each piece of work completes. And that's independent of the infrastructure that it's on.
- 6:11
So what does this look like in an agent architecture?
- 6:15
So the execution layer manages the full life cycle. You know, plan, call a model, run code, invoke a sub-agent, loop, retry, coordinate. You can swap the model, swap the context, swap the sandbox, and the execution layer should be able to remain the same.
- 6:35
It's just the, the fundamentals. And that's how you, I think, decouple and you build the system that it can e-evolve for change. So let's talk about what the execution layer has to do.
- 6:47
First, resumability. Your execution layer must enable your system to pick up after failures without restarting from the beginning. You know, agents are handling longer running tasks than ever before, so they must be able to resume when an LLM call fails, a tool call fails, everything flakes out.
- 7:09
We all know. So if you have a failed request step thirty-eight,
- 7:16
you should be able to retry and wait, continue onwards instead of having to go from the beginning and you're gonna lose tokens, costs, time, work that your agent might have completed.
- 7:28
So for this, this to work, a three-hour run cannot hold state in memory or in disk. The state must live outside of the worker. So this means that the state must be durable and external.
- 7:43
So without it, you might cobble together maybe some manual checkpointing. You might come up with a system that has, like, a log-based approach where you're gonna be, like, hydrating the state back if you have to pick up where you left off.
- 7:57
But I think a lot of those abstractions start leaking into the other layers of your harness, right? It gets harder to change.
- 8:04
So next, a key aspect of the execution layer is that it must enable you to convinal-- combine a lot of invocation patterns. You're gonna need crons. You're gonna need to trigger things with events.
- 8:17
You're gonna need APIs, human-in-the-loop. Sub-agents or dy- or dynamic workflows also must be possible. And you're gonna need to be able to invoke these things synchronously, asynchronously, delaying invocation.
- 8:33
And I think the key here is that flexible execution and orchestration primitives will enable you to build the pattern or the system, the architecture that you actually need. So if not, I think, again, your harness logic starts absorbing other concepts like queues, workers, polling, back-off, scheduling, and now you end up kind of a mess,
- 8:59
bad abstractions. So lastly, execution needs to provide observability across your entire session, not just the LLM calls and the tool calls, but database errors, permissions issues, um, triggers, performance.
- 9:19
So the full session trace across your entire run is essential. So if you can't see the entirety of a trace from the trigger through the whole stack, it's really hard to debug it, let alone improve your agent and keep evolving it.
- 9:37
So what about sandboxes? Sandboxes are s-so hot. So agents need sandboxes. We've kind of, like, settled upon that at this point in time. They need to execute code. They need to browse.
- 9:51
They need to manipulate files. They might work on disk. But a sandbox is ephemeral and stateless by design. So using it for durability with snapshots or something in state, I think is an anti-pattern.
- 10:05
I think it's a, a difficult thing where the state gets lost, and you're kind of trying to peel-- pull the pieces back together.
- 10:13
So I think when you have the execution layer separate, the execution layer is what gives the sandbox its context, its sequence, its durability. So I think of the sandbox as the hands and execution as the brain.
- 10:29
So, so what's next? Right? What are the next six months of architectures that you're gonna need to handle?
- 10:38
So if you're paying attention, you're joining a lot of these sessions, you'll see what architectures are emerging and what each approach brings new engineering requirements. You know, you have background agents,
- 10:52
dynamic workflows, autonomous loops, agent factories, whatever you wanna call it, all these emerging trends that we're seeing the last couple days, they're all long-running. They're asynchronous. They're delegated.
- 11:09
That means that you need to be able to observe them all, uh, down to the core. Right? They need to be inspectable by human and by an agent. So the patterns, as you see, they-- as these things emerge, they must be mixed and combined together.
- 11:26
So what does this all have in common? I think that execution is fundamental to building any of these systems. So let's just take one example here, background agents, right?
- 11:38
They're not request response. There's no person just waiting there for, for the, the, the chatbot to respond. So it ca-- might run for minutes or hours. It's going to have maybe hundreds of calls, uh, tool...
- 11:50
You know, maybe two hundred tool calls. You're gonna probably guarantee to have at least one failure in that. So you can't even debug your background agent that's running asynchronously without the right infrastructure, without the observability, without everything that's going on in that process or multiple processes.
- 12:12
And another example that we have is, is loop architectures, right? And I think we're talking about /loop commands and whatnot and coding agents, but I think where we're taking this is how are you building, um, actual systems in your products that employ these, these, these approaches?
- 12:30
So, like, what is a loop? Right? A loop is a system that basically just runs continuously or on a schedule, and it's assessing the state of the system against the, the goals that you set or the criteria that you set, and determines what to do next.
- 12:45
So you're gonna need crons, sub-agent, delegation. History needs to be inspectable, and of course, it needs to be reliable because you don't know when things are running or how the system is continuous to-- continuing to evolve.
- 13:00
So the frameworks of three months ago were not designed to handle this, right? Like, you're going to need to design these systems yourself, so I think you're gonna need a proper execution layer.
- 13:12
So let's look at just some examples of code. It's small on here, but in general, let's just look at how you might put it together.
- 13:20
In your loop, you're gonna need some sort of cron. Maybe that cron here is some sort of health check system, runs every thirty minutes, pulls down some key high-level system metrics, and it looks...
- 13:32
You pass to an LLM, you say, "Is this healthy? Is this looking okay? Should we do more?" And is it, is healthy or not? What should we do? And if it isn't healthy, maybe you just invoke a, a triage agent that goes and looks at that individual service and goes and digs more.
- 13:47
This is pretty simple looking, right? Now you have this triage agent itself. You've just triggered it. This should maybe pull some more context, pull some detailed metrics. It's just context.
- 14:00
And then you're passing to the LLM to start that investigation. Putting it in a loop, you're calling some tools, trying to get to the root cause. This may run for a minute.
- 14:09
This ra-- may run for multiple minutes. It needs to spin up a sandbox, uh, maybe clone code, maybe analyze commits. It's gonna be doing a lot of work here, and you don't know when this is gonna run.
- 14:20
It might run, it might not. How do you know? How do you observe that system later?
- 14:24
And then to complete the loop, there-- you may have a reviewer function, right? Like, are we-- Is the system actually performing as expected, right? Like, it's gonna run every week and look at the history of what just happened and evaluate how the triage system is working.
- 14:43
Do we need to adjust the prompts? Do we need to do better metrics? Does this have the data that it needs? Is it doing anything at all? Is it overreacting?
- 14:51
So it needs to be execution aware, and I think it needs to be execution aware or orchestration aware because it needs to be able to pull the logs of the system and understand, all right, what was executed?
- 15:00
What did this agent choose to execute? What sub-agents did it fan out? What, uh, workflows did it call upon? And then it needs to be able to analyze that and understand what happened and come back to you, either make the change itself or tell you, "This is where we performed or underperformed.
- 15:18
This is what we can do, how we can improve the system overall." So this completes this improving loop. It's three functions. It's pretty simple by design, and it's flexible to iterate on.
- 15:29
I know things get always a lot more complex in production, but I think about-- when, when I think you think about the layers this way, it makes things a little bit, little bit easier to think about building such a complex system that feels like it's, you know, something that you might not be able to reach right yet.
- 15:45
So in the spirit of the reviewing functions, what do you think about measuring your agent, how your agent is performing? You know, I think what's interesting is this execution layer sits between what your user's input is and what's happening throughout the system.
- 16:02
So user feedback, actions, um, and the results of the sessions all flow through this execution layer. And I think this makes it, uh, an ideal place because it becomes the hub for observability, and it allows you to be able to score and understand what your agent is doing.
- 16:22
So to iterate on your application, you kinda need all this data connected, and it needs to be aware of how your code is actually executing, right? So I think the, the linking of this is, is extremely important.
- 16:33
So this is what we built Inngest for. We're Durable execution for AI agents. We're the execution layer.
- 16:42
You can plug in any context layer, bring a model, framework, tool, any compute layer, bring whatever sandbox you want, bring whatever browser you want, get durable steps, small primitives, right?
- 16:55
Uh, event triggers, scheduling, agent-to-agent coordination, full session traces, no infra to manage.
- 17:06
And I think, like I just mentioned with scores, we also think that, that, that the middle of the execution layer when your agent is running is a key place to instrument and score your agents.
- 17:18
When you have access to the data that you need that's all flowing through there, or you've just executed something, it's a perfect place to run something after and defer some scoring for, uh, with the whole trace information, maybe the inputs and the outputs.
- 17:33
So again, that's just execution, orchestration, delayed tasks, defers. Um, and since there's also data flowing through, you can also wait for additional events and attach them to the sessions that you're building and un-un-understand did this u- did this actually works, right?
- 17:53
If you're, if you're running this triage, um, was this triage successful? Did it result in an action by the engineering team? Then that means it probably was a positive result.
- 18:02
Instead of a thumbs up, thumbs down, it's like, did we open the PR, right? If it's a research agent, was this research saved? Was it a good report? That is these, these things that are events that you should be able to attach, and when you have a system that can connect all these pieces, I think it's really
- 18:18
makes doing a lot of those things, um, like creating outcome-based scores a lot easier.
- 18:25
So to wrap up, build your harness. I want you to understand the layers of the architecture. We have to embrace the fast pace, pace of change. And I think if you can get your execution layer right and think about the right primitives, everything else can quickly evolve around it.
- 18:44
And the next six months, the next three months will all be a lot easier for you, especially as everything continues to change. You all are here. So thanks for listening.
- 18:54
I'm Dan. Um, come and find me at the Inngest booth. We're right on the other side of this wall. We're very bright and orange. So thanks everyone. Appreciate it. [upbeat music]