AI Engineer World's Fair 2026
I Run a Fleet of AI Agents Across Three Machines. Here's What Broke.
Read the talk
Running an Agent Fleet: From Six Contexts to Three Machines
Kyle Jaejun Lee’s agent fleet turns attention, state, approvals and machine placement into separate engineering problems—and exposes what remains unsolved between them.
From a talk by Kyle Jaejun Lee
Before you start: Familiarity with terminal sessions, Git and AI coding agents will help; no Kubernetes experience is required.
The first bottleneck was attention
A MacBook sleeps. Long-running coding jobs need somewhere that does not. Kyle Jaejun Lee’s everyday fleet spans that laptop and two headless, always-on Linux boxes, connected through one control plane. The MacBook handles heavy coding and personal projects; Linux A takes long-running coding tasks, while Linux B runs short-lived personal projects. He opens with the fleet running live: daily infrastructure across two operating systems.
Before distributing the work, though, Lee ran into a limit on one machine: his own attention. A few tmux panes became four, five, then six simultaneous agent contexts. Every additional agent gave him three jobs: scheduler, deciding who should do what; memory, remembering what each agent was doing; and reviewer, checking the results. Six active contexts were more than he could keep in his head.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate context, then persist it
How can a handful of executives oversee thousands of people? They do not retain everyone’s working context. Lee applied that separation to his agents, building a hierarchy with four actual entity types: CEO, VP, manager and worker. Each entity is an agent with its own scoped context and approval boundary. Context flows downward in the slices needed for each task; results flow upward, and Lee reviews what reaches the top.
Lee describes his supervisory context load falling from six contexts to one. That is his account of what he has to hold in mind, rather than a measure of fleet throughput. The hierarchy reduces the context reaching the human, but each agent still has a finite context window. The next question is where its state should live when that window fills.
His answer is a workspace on disk for every entity. The layout separates common obligations, machine-specific information and the work an individual agent needs to continue:
| Location or content | Purpose |
|---|---|
shared | Context every entity must honor |
machines | Machine-bound state |
| Entity workspace | The entity’s mission and current status |
| Handoff folder | Work product passed to the next agent or session |
The durable state lives in files. A model’s current conversation is no longer the only place that knows what the agent is doing.
That changes how Lee handles a full context window. He finds compaction slow and dislikes leaving retention decisions to a summary, so inside Claude he clears the context completely. The fresh context then reads the handoff and history files the agent wrote for itself and resumes from those records. Current Claude Code session guidance does allow focus instructions to steer compaction, although the summary remains lossy; that does not establish what Lee’s installed version supported. His preference is to make the continuity material explicit before resetting.
A context wipe—or even a machine crash—can therefore leave the written work available for recovery. This preserves recorded state and work product; it does not keep a running process alive through a crash. With continuity moved onto disk, the next problem is whether agents remain aligned with the intended plan.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make approval a blocking operation
Plans drift as they move through the hierarchy. Initially, Lee detected that drift by entering individual panes and inspecting the work. He replaced those visits with a review gateway: any layer that wants to act submits its plan and blocks. The plan is pending until he approves it; approval fires a hook that starts or resumes the work automatically. One web inbox becomes the control point.
The demonstrated cycle has three steps:
- An agent submits a plan to the gateway.
- Lee reviews and approves it.
- The approval hook resumes execution.
An infrastructure team inside the fleet built the gateway itself. Agents were now building the tools used to coordinate agents, and the arrangement worked well for Lee on one machine. Scaling the workload exposed a different set of limits.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Five failures under load
1. Orchestrators did the work themselves. An agent assigned to coordinate a task would start implementing it instead of dispatching it to a worker. Lee responded with a CLI harness and skills that invoke those CLIs. In his implementation, dispatch became the only available path for the orchestrator. The intended division of responsibility was enforced through the available operations.
2. Worker panes became unreadable. As managers received more tasks, they created more worker panes inside a single tmux window. Eventually, each pane was too small to inspect. Lee reports that even tmux capture-pane, his programmatic way of reading a pane, stopped returning meaningful content in this arrangement. The fleet had exhausted its display space as well as his ability to observe it; he does not describe a specific pane-layout fix.
3. Accumulated sessions exhausted memory. Lee’s Activity Monitor view shows overlapping process lists with repeated MCP entries. He describes Claude Code and MCP processes stacking up until swap was almost full and almost no free memory remained. The failure here is session accumulation consuming the host’s resources.
4. Git credentials crossed workspace boundaries. The desired mapping was simple: credential A belonged to workspace A, and credential B to workspace B. Instead, credentials collided and became associated with the wrong workspaces. Lee’s fix was a clean, fully separated environment for each workspace.
5. The laptop could disappear. Lee reports losing in-progress jobs when his MacBook lost power or dropped off the network. Network disconnection is a deployment-specific part of that account: tmux itself is designed to preserve sessions across an SSH disconnect, so losing access is not inherently the same as losing running processes. In a more decisive failure, Lee loaded the laptop with so many workers that he later opened its lid to find it had restarted. The in-flight execution was gone.
His first response was a fleet recovery command:
bash
overlord boot
The command brings the fleet back up using the state already stored in files. Recovery was possible because the work was not held solely in model contexts. But restartability could not make a laptop continuously available. Lee still needed somewhere else to run the work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move execution without losing context
Lee added the two always-on Linux machines, offloading long-running coding tasks to Linux A and short-lived personal projects to Linux B. That added resources and took long-running work off the laptop while retaining one control plane. It also created a new requirement: an agent on one machine needed the context produced on another.
File-backed state made the transfer possible with familiar tools:
- Commit the context files on the source machine.
- Push the commit through Git.
- Connect over SSH and use
tmux send-keysto tell the destination to pull. - Have the destination agent read the transferred files and continue the work.
Git carries the durable context; SSH and tmux provide the signal that prompts the other machine to retrieve it.
The first version allowed two machines to change the same directory. Conflicts followed, and supposedly shared context diverged. Lee then separated machine-specific state into per-machine directories and required pull requests for shared-state changes. The separation gives local state an explicit home, while the pull-request rule puts a review boundary around changes that affect everyone.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bring approvals and routing back together
Distributing execution also distributed the gateway: Lee now had an inbox on every machine. That recreated the attention problem the gateway was supposed to solve. He collapsed the inboxes into one main gateway, with every machine forwarding review requests over SSH. The gateway lives on an always-on Linux box because the sole point of control cannot depend on the MacBook staying awake.
Machine selection remained another burden. While developing across the fleet, Lee went looking for a feature and could not remember which machine he had built it on. He made Discord the single routing interface, with one bot for the MacBook, one for Linux A and one for Linux B. His phone became a remote control for the fleet. That unified the place from which he issued requests, though it did not yet eliminate the need to know where work belonged.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let agents declare requirements
The distributed fleet still leaves Lee with four open engineering problems:
- Consistency: keeping state coherent across machines.
- Tool access: abstracting tools that remain local to the Mac, including MCP servers and the browser.
- Credential handoff: securely moving access between instances.
- Resource management: deciding what should run where.
He still makes placement decisions himself. The contract he wants is for an agent to declare what it needs and let the system choose where it runs.
That suggests a boundary between agent orchestration and execution infrastructure. The logical hierarchy, orchestrator and review gate sit above compute, secrets and tools; a scheduler places the workloads underneath them. Lee identifies Kubernetes as the foundation for that direction. Its scheduler places Pods using resource requirements and constraints, which fits the requirement-based placement he wants.
This is a proposed architecture, not a completed migration. Lee intends to put existing infrastructure underneath his orchestration manager and keep the custom work focused on task orchestration, review flow and context management. Kubernetes supplies useful primitives, but tool portability and secure credential handling still need engineering. In particular, Kubernetes Secrets are stored unencrypted in etcd by default; encryption at rest and restricted access require configuration.
Lee’s closing assessment is that single-machine operation works for him, while the cross-machine system remains rough and under construction. The next step is to reuse established infrastructure below the agent layer while continuing to build the coordination above it. He ends by inviting other people running agent fleets to compare notes on those unresolved operational problems.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The project manual for terminal sessions, windows, panes, detaching and reattaching.
How Kubernetes assigns Pods to nodes using resource requirements, constraints, filtering and scoring.
Further reading
A bilingual account of hierarchical Claude Code orchestration using tmux, file handoffs and SSH.
Anthropic explains compaction, fresh sessions, written handoffs and separate subagent contexts.
Credential delivery to workloads and the encryption and access controls needed to protect Secrets.
Read the complete timestamped transcript
- 0:00
Hello, I'm Kyle. I run a fleet of AI coding agents across three machines every day, and this is the story of what broke, starting with me. This is my fleet.
- 0:09
Three machines I run every single day. The MacBook is where I do my heavy coding and my personal projects, and it sleeps because, well, it's a laptop. Then two Linux boxes, both headless and always on.
- 0:22
Linux A takes the long-running coding tasks. Linux B runs my short-lived personal side projects. Two operating systems, one control plane tying it all together. And to be clear, this isn't a demo I spun up for the talk.
- 0:35
This is what I actually use all day, every day. And here it is live, the whole thing running on one screen. Real agents, real work. This is a normal Tuesday for me.
- 0:45
So let's go back to the beginning because before any machine broke, I broke. When I started, it was just me and a terminal. A few tmux panes, a few agents, then a few more, and really fast, I'm sitting in front of four, five, six live contexts at the same time.
- 1:02
And here's what nobody warns you about. At that point, I'm not running agents anymore. I've become the scheduler, deciding who does what. I'm the memory, holding what every one of them is doing, and I'm the reviewer, checking all of it.
- 1:14
One human, three roles, six contexts. It does not scale. This. This is the mess that broke me. I just couldn't hold what six agents were doing at once. My own attention was the bottleneck.
- 1:28
So I asked myself kind of a strange question: how does a handful of executives run a company of thousands of people? They don't hold all of it in their heads.
- 1:36
They separate context. Each person only ever sees their own slice. And that was the unlock. What if my agents weren't a flat pile? What if they were an organization?
- 1:46
So that's what I built, a hierarchy: CEO, VP, manager, worker. And please hear me on this. These are real entity types in the system. It is not a cute metaphor.
- 1:55
Each one is its own agent with its own scoped context and its own approval boundary. Context flows down. Each layer only gets the slice it needs. Results flow back up, and I only review what reaches the very top.
- 2:08
So instead of holding six contexts in my head, I hold exactly one. Next problem, where does an agent's state actually live? Normally, it lives inside the model's context window, and that window fills up.
- 2:20
So I moved it out. Every entity gets its own workspace on disk. Shared context that everyone has to honor lives in shared. Machine-bound state lives under machines. And inside each workspace, the mission, the current status, and a handoff folder, the actual work product that gets passed along.
- 2:38
The state lives in files. It is not trapped inside one model. And this is the single most practical thing I learned all year. When that window fills up, the built-in move is to compact, summarize the history, make room.
- 2:52
I stopped doing it. It's slow, I can't choose what survives, and whatever it throws away is just gone. So instead, I don't compact, I reset. And by reset, I mean right inside Claude, I clear the context completely, and then it just reads back the handoff and the history files it wrote for itself and picks up exactly where
- 3:12
it left off. The context can get wiped, the machine can even crash, and the work still survives because it was never only in the model. But there was still one problem: alignment.
- 3:23
Plans flow down the hierarchy, but they drift, and I'd only catch it by walking into each pane by hand. So I built a review gateway. Any layer that wants to act submits its plan, and then it blocks.
- 3:35
It waits. Nothing runs until I approve. And the second I approve, a hook fires the work off automatically. One web inbox, one control point. I never walk into the work windows anymore.
- 3:45
Here's one full cycle. A plan comes in, I approve, the work resumes. That's the whole loop. And honestly, I didn't build this gateway by hand. An infra team inside the fleet built it.
- 3:57
Agents building the tools that run the agents. I think that's kind of the whole point. So on one machine, all of this works beautifully. And then it didn't. Then one machine wasn't enough.
- 4:08
Five things broke. Failure one, my agents kept doing the work themselves instead of dispatching it down to a worker. The orchestrator is supposed to delegate. Instead, it just rolls up its sleeves and does the task itself.
- 4:21
Wrong. So I forced its hand, a CLI harness with skills that call those CLIs, so dispatching becomes the only path it can take. Failure two, and this one's almost funny.
- 4:33
As I kept throwing more tasks at my managers, they kept spinning up more and more worker panes inside a single window until it was so crowded the panes were too small to read.
- 4:44
Even tmux capture-pane, the thing I use to read a pane programmatically, couldn't pull anything meaningful out of them anymore. I'd literally run out of room to see what my own fleet was doing.
- 4:55
Failure three, out of memory. Look at this. That's my activity monitor completely buried under Claude Code and MCP processes. Swap almost full. Basically, no free memory left. The sessions just stacked up until the machine couldn't breathe.
- 5:09
Failure four, Git credentials. The expectation is clean. Credential A to workspace A, credential B to workspace B, one to one. The reality, they collided, crossed over, bound to the wrong workspaces.
- 5:23
The fix was a clean, fully separated environment for each workspace. And failure five, the one that really hurt. The MacBook is a laptop, so the moment it loses power or drops off the network, every in-progress job dies with it.
- 5:38
And one time, I'd pushed so many workers onto it that the whole machine just gave out. I came back later, opened the lid, and it had already restarted itself.
- 5:48
Everything in flight, just gone. So the very first thing I did was build a boot command, one overlord boot, and the whole fleet comes straight back up because all the state was sitting in files.
- 5:59
But a machine that can just vanish on you like that, that's when I knew one machine wasn't going to cut it. Every single one of these pointed at the same answer.
- 6:07
So I added machines. Here's the split. The MacBook was carrying everything, heavy coding and my personal projects. So I offloaded it. Long-running coding tasks went to Linux A, short-lived personal projects went to Linux B, both always on.
- 6:21
More resources, the long-running work finally off the laptop, and still one control plane over all of it. But now I've got context on one machine that I need on another.
- 6:31
How do I move it? Git. I commit the context files and push. Then I poke the other machine with tmux send-keys over SSH and tell it to pull. It pulls, the agent there reads the files, and picks up exactly where the first one left off.
- 6:44
Of course, it wasn't that clean. When two machines point at the same directory and both change it, you get conflicts. The same context quietly diverged. So I separated it.
- 6:53
Per machine directories for machine-specific state, and the shared stuff only changes through a pull request. It's boring, but boring is what stops the two machines from silently disagreeing. Then the gateway came back to bite me.
- 7:07
I now had one on every machine, so I was checking multiple inboxes again, so I collapsed them. Every machine sends its review requests over SSH into one main gateway, and that main one lives on an always-on Linux box.
- 7:21
Because remember, the Mac sleeps. Your one point of control can't be a thing that falls asleep. And then my absolute favorite failure of the whole project. I was developing across all of these, bouncing back and forth, and at one point, I went looking for a feature I'd built, and I genuinely could not remember which machine I'd built
- 7:39
it on. And that's when it clicked. I need one place to route from. So Discord became my single router. One bot per machine, Mac, Linux A, Linux B. My phone is now the remote control for the entire fleet.
- 7:53
So where am I now? Honestly, with a pile of things I've not solved. Four of them, consistency across machines, abstracting away the local only tools that are still stuck on my Mac, the MCP servers, the browser, secure credential handoff between instances, and resource management.
- 8:09
I really want to stop being the one who decides what runs where. And when I lined those four up, I realized these aren't new questions at all. An agent should just declare what it needs, not where it runs.
- 8:20
Above it sits the orchestrator, the review gate, the logical hierarchy. Below it, compute, secrets, tools. A scheduler places it, and the machines just disappear underneath. These are the exact questions Kubernetes already answers.
- 8:35
So that's where I'm headed. I'm not going to reinvent compute, secrets, and tools. Kubernetes already nailed those. I'm stacking them underneath and building my orchestration manager on top, task orchestration, review flow, context management.
- 8:49
Reuse what exists, build the new part on top. And that's the honest state of it. One machine, I solved. Across machines, still rough, still building. If you're running agents at any kind of scale, I'd genuinely love to compare notes because this is the part nobody has figured out yet.
- 9:05
I'm on LinkedIn and on X. Links are in the description. Thank you so much.