From fork() to Fleet: Designing an Agent Sandbox Cloud — Abhishek Bhardwaj, OpenAI

What is 3 + 3? How many Rs are in strawberry? Early ChatGPT could produce fluent answers while remaining unreliable on questions whose answers were straightforward to check. Abhishek Bhardwaj, who works on OpenAI’s reinforcement-learning and agent infrastructure, uses this contrast to begin a first-principles design of sandbox infrastructure. It follows his earlier talk, How to Build an AI Sandbox from Scratch.

Bhardwaj’s intuition is that familiar arithmetic appeared frequently in training data, while explicit counts of letters in particular words did not. The useful engineering response is to give the model a way to calculate and check an answer. These questions can become small executable tasks:

print(3 + 3)
print("strawberry".count("r"))

Code execution supplies an observable result that a grader can evaluate. It also gives the model a way to improve through repeated attempts on tasks with verifiable rewards.

The training loop has several distinct responsibilities:

1. The training framework supplies a task to the model.
2. The model emits a response that may request code execution.
3. The harness parses that request and runs the code through a tool runtime.
4. A grader evaluates the result.
5. The training loop uses the reward to update the model’s weights.

This trains two behaviors together: deciding when to use a tool, and generating code that actually solves the task. The harness and runtime are therefore part of the learning system, not merely conveniences added after training.

At inference time, the weight-update loop disappears, but the harness still has to parse tool requests, execute them, and return observations. A model trained to solve problems through code cannot exercise that capability unless the product supplies an execution environment.

That environment might be a laptop running Codex, or a cloud node serving Codex Web or ChatGPT. In either case, generated code must be treated as untrusted code. Damage need not come from deliberate malice: an overzealous agent might try to obtain root privileges to complete its task. A kernel exploit could compromise the machine, and on a shared cloud node an escape could expose another user’s data. A sandbox must let useful work proceed while containing these behaviors.

Long-running agents make the cloud case especially compelling. Bhardwaj points to OpenClaw and a picture of a laptop with its lid held open so its agents will not sleep. Users renting Hetzner VPSs or hosted Mac Minis are already seeking an environment that stays available independently of their personal computer. His expectation is that persistent cloud agents will make that improvised laptop arrangement unnecessary.

Research and product workloads share the need for security and reliability, but emphasize different performance goals:

A rollout is one attempt at a task; research needs many attempts in parallel. Product users experience each delay directly. In both settings, runtime failures waste work already paid for in GPU-generated tokens. These requirements lead to three design problems: a secure runtime on one node, persistence for the agent’s disk, and orchestration across a fleet.

On Linux, a thread is the smallest unit of execution. When a userspace program needs a privileged resource, it asks the kernel through system calls, including interfaces such as ioctl. In the processor model used here, userspace executes in ring 3, while the kernel executes in ring 0. A system call transfers control into the privileged kernel context.

Becoming root and exploiting the kernel are different escalations. Root is a highly privileged user identity, but root’s userspace code still runs in ring 3. It may nevertheless gain access to secrets such as SSH keys and encrypted-data credentials. A kernel exploit can go further, allowing execution in ring 0 and access to other processes’ memory. That is the kind of incident Bhardwaj describes as a potential newspaper headline.

The simplest tool runtime is an API server that receives a harness request, calls fork(), and uses exec() to run the requested program. This gives native process execution without an additional isolation layer, but the child can still exercise the host kernel’s attack surface. It can also exhaust the node: a tool that repeatedly forks processes can prevent unrelated tool calls from running. Security isolation and resource isolation are separate requirements, and raw fork/exec supplies neither boundary by itself.

Linux containers build on two mechanisms: namespaces isolate resource views, and cgroups constrain resource consumption. A PID namespace can make processes appear inside a container as PIDs 1, 2, and 3, while the host sees those same processes under ordinary host PIDs. A mount namespace allows a filesystem to be mounted at a path inside the container without changing what the host sees at that path. Network namespaces apply the same separation of views to networking.

A private view does not limit consumption. Cgroups add CPU and memory limits so that a runaway workload cannot consume the entire node’s resources. Together, namespaces and cgroups address much of the noisy-neighbor problem. The processes remain native host processes, however, and their system calls still reach the shared host kernel. A successful escape through that boundary can compromise other sandboxes.

Seccomp reduces the exposed kernel surface by filtering which system calls a process may invoke and which arguments those calls may use. The operational difficulty is predicting an open-ended agent’s future behavior. A restrictive filter may block a legitimate tool, leaving users waiting for the runtime operator to revise the policy. Containers and syscall filtering provide meaningful protection, but the question remains: can the workload reach less of the host kernel while retaining broad Linux functionality?

gVisor moves much of the Linux API implementation into userspace. Its Go-written Sentry acts as an application kernel, implementing facilities such as process management. A companion daemon, the Gofer, handles filesystem access. Workload system calls are intercepted and serviced through this layer rather than being passed directly to the host kernel as ordinary application calls.

An exploit in that application-kernel implementation initially lands in userspace, not automatically in host kernel mode. Bhardwaj frames the remaining risk as an exploit chain: first compromise the Sentry or Gofer, then find a way through its restricted interface to compromise the host kernel. The extra boundary makes the attack harder, but the underlying host still matters. Increasing model capability at finding bugs and chaining exploits motivates asking for another kind of separation.

The next goal is to contain a workload even if it obtains root or compromises its own kernel. Hardware virtualization supplies a separate processor execution context. In the Intel terminology used in the talk, the guest kernel executes in ring 0 in VMX non-root mode, while the host kernel and hypervisor execute in VMX root mode. Guest ring 0 grants control inside the guest; it does not itself grant host control. Transitions between guest and host contexts carry a performance cost.

A virtual machine monitor, or VMM, manages this environment from the host. QEMU is a familiar example. In the simplified startup path, the VMM configures the guest kernel, root filesystem, and memory, then calls Linux’s KVM interface through /dev/kvm to run the guest. Inside, the guest sees a separate Linux system with block and network devices. Access to those virtual devices can transfer control back to host-side device backends that service the request.

Paravirtualization makes that communication more efficient by using guest drivers that know they are running in a VM. With virtio, the guest and host communicate through a virtualization-aware device interface. In the illustrated arrangement, devices appear through PCI, and device interactions can cause exits into host processing. Bhardwaj describes the host-side execution thread as appearing blocked while the CPU runs the guest, then resuming VMM work when guest execution exits. The application still uses ordinary Linux interfaces; the virtualization machinery sits below them.

A hardware boundary still leaves host-side software to defend. QEMU supports many architectures and devices, and its C device implementations have historically supplied targets for VM escapes. Bhardwaj’s team at Google worked on crosvm to run Linux VMs on Chromebooks with a narrower device set and a Rust implementation. Memory safety reduces one class of implementation risk; a smaller supported surface reduces the amount of code that must be defended.

Device-level jailing adds another boundary. A block backend can receive access to storage resources without network access; a network backend can receive networking permissions without access to block resources. Compromising one device should not automatically grant the permissions of every other device. This is a concrete use of least privilege inside the VMM architecture.

The micro in microVM refers here to the reduced monitor footprint, device scope, and boot overhead, not to a restriction on what the guest can do. Firecracker began from crosvm and serves AWS serverless workloads. That lineage predates the talk’s suggested 2023 turning point: AWS publicly announced Firecracker in 2018. Cloud Hypervisor is another Rust VMM, with a more general scope and contributions from multiple organizations.

The control path is a sequence of API operations:

1. The harness launches a cloud-hypervisor process.
2. The process exposes an API over a Unix domain socket.
3. The harness creates a VM configuration, supplying the root filesystem, kernel, CPU allocation, and memory.
4. It starts the VM, causing the VMM to enter the KVM execution path.

The talk describes this conceptually as create, then start. Current Cloud Hypervisor documentation names those operations vm.create and vm.boot. With the API socket available and the kernel and root disk prepared, that sequence looks like:

curl --fail --unix-socket /tmp/agent-vmm.sock \
  -X PUT http://localhost/api/v1/vm.create \
  -H 'Content-Type: application/json' \
  -d '{
    "cpus": {"boot_vcpus": 2},
    "memory": {"size": 1073741824},
    "payload": {
      "kernel": "/srv/agent/vmlinux",
      "cmdline": "console=ttyS0 root=/dev/vda rw"
    },
    "disks": [{"path": "/srv/agent/rootfs.img"}]
  }'

curl --fail --unix-socket /tmp/agent-vmm.sock \
  -X PUT http://localhost/api/v1/vm.boot

The resource sizes and paths above make the configuration concrete; the essential mechanism is the separation between defining the VM and booting it.

Once the guest is running, the harness also needs a control channel into it. In the described sandbox design, guest PID 1 exposes an API server for requests such as saving state or coordinating device operations. Host and guest can communicate through vsock, a socket mechanism designed for that boundary, or through the node’s IP networking stack. The VMM API controls the machine; the guest API coordinates work inside it.

MicroVM isolation is strong, but VM escapes remain possible through vulnerabilities in KVM or device handling. Granular device jails and seccomp policies add defenses around that remaining surface. The operational costs appear in several places:

• Execution overhead: guest/host transitions are expensive relative to staying within one execution context.
• Memory sharing: a balloon driver asks the guest to return memory, making reclamation cooperative and reactive rather than immediately available to the host.
• GPU access: virtio-gpu provides graphics-oriented interfaces, while direct device access introduces assignment and sharing constraints. The talk’s single-sandbox limitation applies to whole-device passthrough; VFIO mediated devices can expose separate instances when the hardware and vendor driver support them.

These are workload and capacity-planning decisions as well as runtime implementation details.

Optimize performance around the security boundary you need. Bhardwaj prefers this approach because a performance problem can often be mitigated with systems techniques, while a security breach can permanently damage trust. His recurring experience is that teams move through containers, gVisor, or V8-based environments before discovering that their agents need a complete Linux machine. His advice to founders is to start with microVMs and investigate alternatives if that choice fails their requirements. The promised avoidance of years of grief is practical encouragement, not a migration estimate.

Imagine a laptop that loses your work whenever you close it. An ephemeral cloud sandbox creates a similar experience for an agent. The next requirement is disk persistence—distinct from preserving live process memory. A microVM’s disk may be a host file or another attached block device, and it can accumulate installed packages, presentations, and entire repositories. Losing that state after a node failure or an interrupted model run wastes both user work and the computation that produced it.

Periodic checkpoints let the system recreate a sandbox’s saved disk state on another node after a node or cluster failure. The same mechanism supports intentional movement during cluster upgrades or node A/B tests. Persistence therefore improves fleet reliability and flexibility, rather than merely adding a storage feature.

Bhardwaj reports a personal Codex goal-mode run of roughly three days. As tasks stretch across longer periods, checkpoints give the harness a place from which to recover instead of repeating all preceding work. His example is an anecdote about task duration, without a specified workload or success criterion.

Saved states also enable branching exploration. A harness can checkpoint a workspace, try a solution, backtrack, and branch again—an execution substrate for Monte Carlo tree search. Multi-day rollouts can then explore alternatives without destroying their common starting point. Bhardwaj connects this possibility to ambitions such as drug discovery, but the enabling requirement is immediate and concrete: reliable primitives for saving and restoring the environment.

At ChatGPT or Codex scale, repeatedly uploading a full disk would make frequent checkpointing costly and slow. Incremental snapshots save changes since an earlier snapshot. Saving must be cheap enough for a harness to use routinely, and restoring must be fast enough to serve as ordinary sandbox creation.

Several independent choices shape the storage contract:

Saving whole changed files can amplify writes when only a small part of a large file changes. Block-level snapshots can preserve the smaller change. The overall flow remains straightforward: identify changes, compress them, upload them, then download and reapply them during restoration.

Linux exposes a disk as logical blocks numbered from zero onward. The filesystem uses metadata such as inodes to map file offsets onto those blocks; device firmware maps logical blocks onto physical sectors or pages. Snapshotting at the block layer works below the file abstraction, preserving the blocks that encode both file data and filesystem state.

That abstraction also affects VM I/O. A shared-folder interface can require host service for many filesystem operations. Giving the guest a block device lets its own filesystem and caches handle more work internally, reaching the host when block access is required. Bhardwaj favors the latter path for reducing exits and retaining guest-side caching.

In an always-on design, the VM sees an image as a block device while a distributed backing layer carries writes toward cloud storage. In an explicit design, the harness calls a Save API. That API packages the changes since the preceding snapshot and returns a snapshot ID. Restoring the ID means resolving its lineage, fetching the required artifacts, and applying them on the destination node before recreating the microVM.

A copy-on-write filesystem such as reflink-enabled XFS can create a writable copy of an image while initially sharing its data blocks. Bhardwaj describes the initial copy as near-zero latency, without supplying a benchmark. The mechanism avoids copying all the data up front: new storage is required when a write separates the writable copy from the shared original.

Start with base.image, create a writable copy, and attach that copy to the VM. For a concrete teaching example, suppose logical blocks 0, 1, and 2 contain A, B, and C. The writable image initially shares all three. Writing B2 to its logical block 1 leaves the base at A, B, C, while the writable image becomes A, B2, C; blocks 0 and 2 remain shared. This is the state separation that makes an incremental artifact possible.

In the reference design, FIEMAP supplies extent information used to identify ranges for the snapshot artifact. FIEMAP itself returns file extent mappings, not a ready-made difference between snapshots. An implementation still needs to establish which ranges changed and define a consistent snapshot boundary. Once identified, those ranges are compressed and uploaded.

The Save API can return before its background upload completes, keeping the harness moving while storage work continues. That creates two distinct states: a snapshot has been acknowledged, and its artifacts are durably available for restoration elsewhere. The described API does not specify the transition between them, so a caller must not equate an early response with completed remote persistence. Restoration downloads the required diffs, applies their extents over the base image, and starts a microVM with the reconstructed disk state. This restores disk contents, not live process memory.

For always-on persistence, the goal is to give the guest familiar filesystem behavior while moving durability behind a block interface. Bhardwaj raises concerns about NFS and favors standard POSIX behavior for model workloads. The relevant Linux NFS caveat is specific: close-to-open consistency and cross-client caching can require coordination. That does not establish a blanket performance or POSIX verdict for every NFS configuration.

His alternative uses NBD, the Network Block Device interface, to present a disk backed by a storage system with tiered block caches. Writes go first to an in-cluster tier, which writes back to durable object storage such as GCS or S3. The guest runs its filesystem over the block abstraction, retaining its normal filesystem interfaces and caching behavior while the backing service manages remote storage.

The architectural opportunity is broader than keeping files between sessions. Fast snapshot and restore allow the harness to recover from failures and revisit earlier states during search. Storage becomes part of how an agent explores a problem, as well as how its work survives.

A secure runtime on one node still fails when that node disappears. To build a fleet, group nodes into clusters and distribute clusters across regions. A top-level control plane chooses a cluster using regional load and proximity to the ChatGPT service or harness. Within that cluster, a scheduler selects a node using load and health information, avoiding nodes that are failing. The design goal remains low latency and reliable execution; no particular orchestration framework is required to explain the hierarchy.

Startup offers three strategies:

• Warm pool: keep sandboxes ready and assign one when a request arrives.
• Memory-snapshot restore: recreate a prepared guest from saved memory on demand.
• Hybrid: serve from a warm pool and expand it using memory snapshots.

Bhardwaj says memory snapshots can support startup in milliseconds, without specifying hardware, image size, cache state, or measurement boundaries. Unlike the earlier disk snapshots, this technique saves guest memory to avoid repeating startup work. Warm pools exchange idle CPU and memory consumption for readiness; a hybrid uses snapshot restoration to replenish that capacity.

Persistence can also guide the scheduler’s placement decision. Consider the closing example: a restore needs a lineage of four snapshot layers. Nodes A and C have only some of those layers cached, while node B has all the required layers. A layer-aware scheduler gives B the highest score because it needs the fewest downloads. It uses the structure of the saved state to reduce the work required to recreate the sandbox.

Here the three design problems meet. The runtime supplies the isolation boundary, persistence makes the workspace recoverable, and orchestration places that workspace where it can become useful quickly. Snapshot locality turns storage knowledge into a scheduling advantage, completing the path from one untrusted tool call to a fleet of securely operated agent computers.