Two Roads to Durable Agents: Replay vs. Snapshot — Eric Allam, Co-founder, Trigger.dev

Your agent works on your laptop. An outer loop handles user turns, while an inner LLM loop carries out the work within each turn. Now move it onto a production server. It needs to perform long-running, meaningful work, maintain continuity across turns and code changes, and recover when something fails. Preserving its progress becomes an infrastructure requirement.

Allam frames his work on Trigger.dev around this problem. What assumptions change when a backend hosts an ongoing agent rather than a request handler or a bounded workflow? His explanation starts with the infrastructure agents inherit.

Allam begins his condensed history with CGI in 1993. An HTTP request arrives, the server forks a process, and request data enters that process. The program does its work, writes a response to standard output, and exits. The process does not remain around to remember what happened for the next request.

PHP and the LAMP stack reused processes, but retained the same basic application model: request + database state → response. A subsequent request performs the computation again, using its own inputs and whatever state the database now contains. Process reuse changes how the computation runs without making the process itself the durable home of application state.

This is the shared-nothing architecture Allam emphasizes: the compute layer is stateless, while meaningful durable state lives in the database. He traces the same architectural tradition through Rails, Node.js and serverless systems. A backend can handle another request without needing the previous request's machine to survive.

Applications soon needed work outside that request-and-database lifecycle: send an email, charge a credit card, resize an image. Individual asynchronous tasks became sequences of side effects. Allam's processOrder example makes the failure concrete: fetchOrder and chargeCard succeed, but sendReceipt fails. Calling processOrder again starts the sequence over, risking another charge before it reaches the receipt. The diagram shows why retrying a function is different from retrying only its unfinished work.

Workflow and durable execution engines address this by wrapping side effects in recorded steps. In Allam's replay model, recovery works like this:

1. Execute the charge step and durably record its successful result.
2. Attempt to send the receipt; that operation fails.
3. Restart the workflow. At the charge step, return the saved result instead of charging again.
4. Reach the unfinished receipt step and retry it.

The successful charge must have a saved completion record. If a payment succeeds externally but that record is lost, replay alone cannot establish exactly-once charging; payment-side idempotency still matters.

Replay builds durability on top of stateless compute. Its execution history also supplies an audit trail, and resuming at a recorded point supports deliberate waiting as well as failure recovery. A workflow can pause for an external event, such as a human taking an action, and continue afterward.

The programming constraints follow from the recovery mechanism. Side effects must sit inside recorded steps, while code outside those steps must satisfy the engine's determinism requirements. Re-executing the workflow needs to encounter a sequence compatible with its existing history. Deploying changed code can therefore create a versioning problem: the new control flow may no longer agree with the old replay journal.

Early LLM applications fit comfortably into this workflow structure. A model could classify some text as one step in a sequence the application had already determined. Tool calling reverses the orchestration relationship: instead of code deciding when to invoke the model, the model can decide which code to invoke next. The agent loop makes the LLM an orchestrator.

Applying replay to that loop means turning each LLM call and each tool call into a journaled step. On resumption, the function executes from the top, consuming recorded results for completed steps until it reaches new work. Even one user turn can produce multiple entries, because the model may call tools and continue working before returning a response.

As users keep interacting, the history grows. A particular replay implementation may limit the number of entries, the size of their contents, or both. Allam's concern is the pressure an increasingly long-lived agent places on those limits: every additional turn adds more history for the system to retain and handle during recovery.

The distinction becomes more consequential as agents perform useful work over longer periods. A transaction or multi-step workflow has an intended beginning and end. An agent is a session, and that session may continue for as long as its user wants to keep working. Completing a predetermined sequence and sustaining an ongoing working relationship place different demands on the backend.

Allam separates an agent into two kinds of valuable state. The first is its context: system messages, user messages, tool calls, tool results and assistant responses. This is the history of what went into and came out of the LLM, and preserving it is fundamental to continuing the agent's work.

The second is execution state. An increasingly capable agent wants the resources available on a laptop: files it can write, memory it can use and subprocesses it can create. These are valuable too, but they are different objects from messages and tool results. The two kinds of state can have separate durability mechanisms.

For context, the mechanism is an append-only log. A database, object storage or a distributed filesystem can hold it durably. Once that history exists independently of the running machine, an upgraded agent harness can consume the same context, and a replacement machine can recover the conversation after a crash. Append-only storage also provides a scalable persistence primitive. The interaction history survives without requiring the original version of the harness to remain the only code capable of using it.

Now consider the execution environment the agent has already prepared. It has cloned a GitHub repository, installed packages, loaded datasets into memory and started a development server. A sandbox may be running in a subprocess. Saving the conversation does not preserve that live environment: a tool result describing an installation is not the installed package, and a message about a server is not the running process.

The agent now needs to wait for the next user message. The agent.ts slide makes that pause explicit with wait.for with type argument Message and the identifier "user-message". The control flow has reached a waiting point, but the machine still contains the working state that the next turn will need.

Keeping the machine running throughout an indefinite pause retains that state, but also retains the compute expense. Allam proposes snapshot and restore: capture the machine's execution state, save the snapshot to disk, shut down the running machine, and restore it when another message arrives. If the user goes to lunch, the agent's prepared environment need not keep consuming live compute throughout the break. The cost shifts from keeping the machine live to retaining its saved state.

Combining a durable context log with execution snapshots gives the system different recovery paths for different failures. Allam jokes about the supposedly impossible event of an LLM provider failing, then considers a request that cannot be retried for fifteen minutes.

A healthy machine waiting on that provider needs a different response from a machine that has itself become unusable.

This leads Allam toward a backend model in which compute itself contains state worth preserving. Snapshotting is not a new idea, however. His historical example is an IBM mainframe from 1966: expensive jobs ran for hours, so programmers inserted checkpoints to avoid repeating all that work after a failure.

He then turns to CRIU, developed beginning in 2011 for checkpointing and restoring processes from userspace. In his explanation, a small helper is injected into a process to capture its state and then removed. The checkpointing can be transparent to the application, which does not need to implement its own checkpoint protocol, and it integrates with container runtimes. Allam reports that Trigger.dev had performed millions of snapshot restores after shipping its CRIU implementation in 2024.

Covering the environment agents wanted to use proved harder. Allam describes three constraints encountered in Trigger.dev's integration:

• Subprocesses. Workloads involving FFmpeg, Chrome and other processes were difficult to preserve with its process-oriented checkpointing approach.
• Files. The integration's file capture depended on files being open at snapshot time, leaving a gap for agents working across a filesystem.
• Registries. Container compatibility brought image push and pull operations into the system, adding latency to checkpointing and restoration.

These practical constraints pushed the team toward a broader unit of preservation.

Allam describes moving to Firecracker microVMs to capture the environment at the virtual-machine boundary. Guest memory and VM state become available for restoration together, rather than treating an individual application process as the unit of preservation. The upstream snapshot contract gives this whole-machine description a precise boundary: the integrator must preserve the backing disk files, and network connections need not survive a restore.

Capturing a machine creates a size problem. In Allam's example, a VM configured with 512 MB of memory produces a naive 512 MB memory snapshot. Much of that capacity may not contain useful application data, yet the snapshot still creates storage and network-transfer costs.

The implementation uses seekable compression, allowing it to access compressed portions of the snapshot without decompressing the entire memory image first. Restoration is demand-driven: when execution needs a memory page, the system retrieves and decompresses the relevant portion. The program can continue without eagerly restoring every page. With compression and additional snapshot layering, he reports reducing that example to about 14 MB. Allam mentions the layering without detailing its design; compression strength provides a knob for trading snapshot size against performance.

After that compressed account of the engineering, Allam jokes about how much work remains hidden behind it. For this implementation, Allam reports snapshots taking slightly under a second and restores taking a few hundred milliseconds. He presents those timings in a comparison with the earlier CRIU implementation; the hardware and workload behind the comparison are unspecified.

The closing demonstration packages this work into FCRun, also called F-Run in the talk. Allam describes a Docker-like CLI for running containers inside Firecracker VMs and snapshotting and restoring them. Familiar container-oriented operations expose the underlying machinery without requiring the user to manage each checkpointing mechanism separately.

He demonstrates launching Alpine, taking a snapshot of a running VM and forking a VM. These operations make the execution environment itself something that can be started, preserved and branched. An agent's prepared machine becomes reusable state, rather than something that must always be reconstructed by repeating its prior actions.

The next benchmark concerns TTI, which Allam describes as the time until a VM is ready to interact with the internet. For the demonstrated FCRun benchmark, Allam reports approximately 15,000 VM starts per minute. That figure measures aggregate startup throughput, not the readiness latency of an individual VM. His accompanying analogy to roughly 30 frames per second captures the speed of the demonstration; it is not a conversion of starts per minute into a per-VM timing.

At talk time, FCRun was intended to power Trigger.dev's future compute layer and was not yet open source, though Allam said a release was coming. He returns to the original agent loop with two complementary mechanisms: a durable context log for continuity across harness versions and recovery, and execution snapshots for preserving working state across turns and waits. Together, they support the shift he argues agents require: stateful compute whose working state can outlive a running machine.