AI Engineer World's Fair 2025
Events are the Wrong Abstraction for Your AI Agents
Read the talk
Events Are the Wrong Abstraction for Your AI Agents
An agent needs to survive failed calls, lost processes, and long waits. Durable execution puts that control flow at the center while moving event machinery into the platform.
From a talk by Mason Egger
Before you start: Familiarity with message queues, service APIs, and basic asynchronous programming will help you follow the architecture comparisons.
Choosing the center
Put Earth at the center of a diagram of planetary motion and the paths become elaborate loops. Put the Sun at the center and those same motions become much easier to understand. Mason Egger, who works at Temporal, opens with this comparison to ask a software design question: what belongs at the center of the system?
In Egger’s historical framing, the Copernican shift helped open the way to new discoveries about nature and gravity. His architectural point depends on the choice of reference frame, not on declaring one projection useless. Both can describe motion; Earth is a useful reference for a moon trajectory, while the Sun makes planetary trajectories easier to reason about. The best abstraction makes the problem you are solving simpler to express.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The event-driven agent baseline
Making AI software available and scalable brings familiar infrastructure problems into an unfamiliar application. Egger compares scaling agents with scaling microservices: the model does not remove the distributed-systems problems engineers have spent decades addressing. One established response is event-driven architecture. The definition he gets from Claude describes components producing and consuming events, processing asynchronously, and reacting to state changes rather than making direct method calls.
The agent diagram turns that definition into a recognizable system:
| Component | Responsibility |
|---|---|
| Cron jobs | Clean up inactive chat sessions |
| Message bus | Carry events between participating components |
| LLM orchestration and tools | Perform the agent’s work |
| Dead-letter queue | Retain failed tasks that cannot be reprocessed |
This is a workable design. But tracing its boxes raises a different question: how much implements the application’s purpose, and how much exists to prevent the application from breaking?
Egger speculates that hundreds of thousands of production agents use this architecture; he provides no measured adoption count. His objection is not that it cannot work. It is that events have become the organizing center, while the core logic is surrounded by machinery for managing their delivery and failure.
Egger recalls an SRE incident in which roughly 100 lines of simple application code brought down a large travel enterprise because of mismanaged queues. The anecdote illustrates the imbalance: a small amount of business logic can inherit a much larger operational failure surface from its coordination infrastructure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Contracts, control flow, and state become scattered
The first cost is an unclear application contract. Egger argues that adopting events often sacrifices the explicit, documented interfaces developers expect from APIs. He acknowledges AsyncAPI, but describes it as documenting produced and consumed message formats. That description is narrower than the specification: AsyncAPI 3.0 also covers operations, channels, servers, security, and replies. The architectural concern remains useful, however: documenting individual interactions does not by itself make the application’s overall control flow easy to follow.
The next cost is reconstructing that control flow. In the extreme Egger describes, business logic is fragmented across thousands of services. Debugging no longer means opening the file containing a procedure. It means searching the codebase for an event name, finding its producers and consumers, and reconstructing who triggered what, when, and why. If the relationships remain opaque, an engineer may have to run the system just to discover how it fails. Egger likens that to crashing a car into a wall to learn whether it breaks.
Each participating service then becomes an ad hoc state machine, often with a local database or cache to remember its progress. Egger jokes that enthusiasm for implementing state machines fades after being paged for one. The operational problem is concrete: without a transaction joining message handling to the corresponding state update, the system can record one part of a transition without recording the other.
Those inconsistent intermediate states create opportunities for race conditions and difficult overnight incidents. Recovery becomes more application code—or manual work pushed onto customer-success teams, who reset the system or roll back a database after it crashes. The queues have not eliminated coordination; they have distributed responsibility for implementing and repairing it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Runtime independence is not design independence
This leads to Egger’s most provocative claim: event-driven systems can be tightly coupled. The distinction is between runtime coupling and design coupling.
| Dimension | Question |
|---|---|
| Runtime independence | Can other services keep running when one fails? |
| Design independence | Can one service change without breaking its consumers? |
An event bus can help with the first without guaranteeing the second. A consumer need not be running at the instant a producer emits an event, but its code can still depend on the producer’s event structure and meaning.
Egger illustrates the confusion with a fictional refactor: an engineer makes every local variable global, then celebrates that new code can read those variables without asking anyone. Access has become easier, but dependencies have become harder to see. Open-ended event consumption can create the same problem.
A producer changes an event format and breaks a consumer it never knew existed—perhaps through a dependency several services downstream. Once teams have experienced that failure, they become reluctant to change the event at all. Runtime flexibility has produced a design dependency that discourages iteration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Execution that outlives its process
The proposed change of center is durable execution: express what the application should do, and give the platform responsibility for preserving its progress through failures. Egger calls this “crash-proof execution.” The promise is that a process failure need not become a failure of the application’s entire operation. He develops that promise through four characteristics.
First, preserve application state. An ordinary process crash loses in-memory variables. Developers compensate by writing state into Redis, a database, or another store, then implementing reconstruction themselves. Egger describes durable execution as automatically preserving variables, calls, inputs, outputs, and returns. For Temporal, the mechanism is more precise than a snapshot of arbitrary process memory: its Workflow Execution documentation describes deterministic replay against a recorded Event History, allowing workflow state to be reconstructed.
Second, virtualize execution. A logical execution no longer belongs to one process on one machine. If the process fails, another can recover progress from the recorded history and continue. The developer can reason about a continuing workflow even though different processes may perform its work over time.
Third, let execution outlive a short request window. A thirty-day wait becomes a reasonable operation because it does not require keeping the original process alive. A durable timer records the wait; a crash does not require starting the full interval again. For example, the wait can be expressed in a Python Temporal workflow as:
python
from datetime import timedelta
from temporalio import workflow
@workflow.defn
class ThirtyDayWait:
@workflow.run
async def run(self) -> str:
await workflow.sleep(timedelta(days=30))
return "Wait complete"
The significant operation is workflow.sleep: the workflow waits durably instead of relying on a process-local timer to survive for a month. If execution infrastructure is unavailable when the wait ends, work can continue after it recovers.
Fourth, put reliability in software rather than specialized hardware. Egger contrasts this approach with expensive fault-tolerant machines that support hot-swappable CPUs and memory. He points to a Raspberry Pi experiment as an illustration. The company’s Temporal in Space account describes a Raspberry Pi 5 Worker carried into the stratosphere by a weather balloon, with recovery after connectivity returned. That is the concrete experiment behind his outer-space shorthand: ordinary hardware participating in an execution whose progress survives interruption.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put the workflow at the center of the agent
The revised architecture keeps the chat UI, LLM calls, and tools, but places a durable workflow between them. Failures such as an LLM outage or rate limiting become retryable operations managed by the execution platform. If a worker crashes during the operation, the workflow can reconstruct its state and continue.
Egger describes calls retrying until they succeed. In Temporal, that behavior belongs to Activities under a retry policy, rather than to every ordinary function call. Activity retries can stop because of configured attempt limits, timeouts, or non-retryable failures; Workflow Executions do not retry by default. Automatic retry removes repetitive recovery machinery, but does not guarantee that an unavailable dependency will eventually succeed.
The workflow can also arrange longer-term state storage for auditing, either at a predefined point or when the execution closes. In the completed diagram, the chat UI connects to the central Durable Workflow, with Tools, an LLM API, a Durable Timer, and chat-state storage around it. Automatic retries and LLM response validation appear at the relevant connections. The simplification is in the developer-facing control flow: the application’s steps are central, while queue and event management recede into the platform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Different languages, shared execution machinery
Temporal provides this execution model, with an open-source server released under the MIT license. Egger presents it as a way for developers to concentrate on business logic rather than managing the surrounding queues.
At talk time, Egger lists seven SDK languages: Go, Python, TypeScript, Ruby, .NET, Java, and PHP. This is a historical inventory, not a claim that all seven had the same maturity: the Ruby SDK release history places Ruby in public preview at the June 2025 conference, with general availability announced that October. Egger also teases another SDK later that year without naming it.
The language interfaces can participate in a polyglot system. Egger’s example is Go code invoking work written in Ruby by supplying a function name and input parameters. The consequential reveal follows immediately: events still exist underneath. Durable execution changes the interface developers use to coordinate work; it does not make distributed communication disappear.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Moving complexity into the platform
Egger places this move in a history of programming abstractions reaching back to the 1950s. Progress often comes from moving machinery out of application code and into a language or runtime. His examples trace that shift:
| Abstraction | Machinery it helps developers stop managing directly |
|---|---|
| FORTRAN’s mathematical operations | Assembly and manual register manipulation |
| ALGOL 60 and Pascal’s structured control flow | Jumps and goto statements |
| Lisp’s garbage collection | Manual memory management |
| Simula and Smalltalk’s objects | Lower-level organization of state and behavior |
These are the milestones in his analogy: developers gain a more useful vocabulary for the work they intend to express.
Durable execution makes a similar proposal for distributed applications. Events and queues remain necessary implementation machinery, but the platform takes responsibility for coordinating them. The application can express a sequence of work, a retryable operation, or a long wait without spreading that sequence across independent event handlers.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The network boundary and the crash challenge
The closing Scooby-Doo meme removes an AI disguise to reveal distributed systems underneath. Its practical point is the network boundary: once an agent calls an external model or tool, it depends on another process and a connection that can fail. Naming the application an agent does not remove that responsibility.
Egger ends by inviting attendees to the Temporal expo demo and challenging them to turn off his computer or laptop. He promises that the durable agent will keep running; the recording ends with the invitation, rather than showing the shutdown and recovery. The challenge makes the intended abstraction tangible: the lifetime of the agent’s work should not be the lifetime of the machine currently executing it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Specification for documenting message-driven APIs, including operations, channels, messages, and security.
MIT-licensed Temporal server source with setup instructions and links to SDKs.
Account of a Raspberry Pi weather-balloon experiment testing workflow recovery after lost connectivity.
Further reading
- How Temporal Workflow Execution worksDocumentation
Explains Event History, replay, commands, and timers behind durable workflows.
Documents retry defaults, backoff, attempt limits, non-retryable failures, and timeout constraints.
Mason Egger's step-by-step tutorial builds an agent with event search, flight search, and invoice tools.
Updates since the talk
- Temporal Ruby SDK release historyDocumentation
Tracks Ruby SDK milestones, including public preview in May 2025 and general availability in October 2025.
Read the transcript and explore slides
Loading transcript and slides…