Is the Agent Harness Dead? — Jediah Katz, SpaceXAI

Jediah Katz joined Cursor in early 2025 and described himself as its longest-tenured agent-harness engineer; he later moved to SpaceX following its acquisition of Cursor. He starts with a term that arrived after “prompt engineer” and “context engineer.” A context engineer assembles what the model should see for a particular request. A harness engineer owns a larger system: the software that wraps the model, steers it, preserves its work, and connects it to computers and third-party services. The model supplies learned capability; the harness determines how that capability enters the world.

The useful split is between two layers. Runtime controls where the core agent loop executes and how its state is stored so interrupted work can resume. This resembles conventional software engineering: processes, persistence, recovery, and execution environments. Context and orchestration decide which models run, when they run, and what they see. Prompts and tools are only the obvious pieces; memory, context compaction, and behavioral interventions also live here. Much of the “harness is dead” debate concerns this second layer, even though the runtime remains part of the same system.

The previous high-water mark was an expensive, long-running, multi-model harness that coordinated many agents and machines to build a browser from scratch. Katz says producing that demonstration even disrupted Cursor once or twice. Yet the industry soon moved in the opposite direction: system prompts shrank, guardrails disappeared, and some builders predicted that only reusable skills would remain. Cursor itself removed interventions that had compensated for model weaknesses.

Those deleted interventions were concrete. A small auxiliary model once applied edits because frontier models could not reliably modify files themselves. Coding-specific instructions taught agents how to explore a repository, persist on long tasks, run tests, and check their own work. As models improved, keeping all of that machinery could become redundant rather than helpful.

The provocative test is a harness that offers little more than a shell—plus, in Katz’s deliberately unsafe joke, access to every secret. Katz describes a UC Berkeley study that tested seven frontier models across several harnesses, including Pi, Codex, and Claude Code. On SWE-bench Lite, he reports no significant difference in scores attributable to harness choice, particularly for newer models. Separately, he says another study found that a dedicated machine-learning-engineering harness made little difference on MLE-bench. These are his descriptions of two distinct results; the supplied material does not include the papers or numerical results.

That is the limited sense in which part of the harness is dead. Repository exploration, persistent execution, test running, and edit application used to be product-side patches for missing “coding muscle memory.” Once those behaviors became native model capabilities, the patches stopped earning their complexity.

Katz calls the mechanism the data flywheel. A frontier harness first makes harder tasks feasible. More users can then attempt those tasks productively, producing more coding interactions that reach model labs. Models learn the recurring behaviors, so the next generation can perform them without the original prompts or guardrails. The harness can finally delete the intervention that created the training opportunity.

Deletion is therefore evidence that one turn of the flywheel succeeded, not evidence that the surrounding product has no value. Once a capability such as basic coding becomes commoditized inside the model, the harness can move outward to a harder domain or a broader part of the job. That final transition—from absorbed capability to a newly accessible frontier—restarts the cycle.

Grokbot changes the unit of interaction. Instead of treating each request as a one-off chat, it uses long-lived coordinator bots that direct fleets of worker agents. Katz compares the coordinator to a lawyer who owns strategy and deals directly with the client while delegating pieces of work to paralegals. The coordinator maintains the relationship and overall objective; workers execute delegated tasks.

This broader job reverses the simplification seen in coding agents. Grokbot has needed new prompts and guardrails because models have not yet internalized all the behaviors required of an autonomous colleague. The product is operating in unfamiliar domains, over longer periods, with more consequential actions. Old scaffolding disappeared from coding while new scaffolding appeared around knowledge work.

Four unresolved behaviors show why:

• Website workflows: Tasks such as producing slide decks or booking travel require reliable interaction with external systems.
• Communication: A long-running colleague must report progress without flooding the user.
• Acting versus asking: Sending email may be useful, but the agent needs to know when intent is clear enough to act and when approval is required.
• Memory: The system must learn how a particular person works over time.

Experienced coding-agent users may recognize all four problems, but a general colleague encounters them routinely rather than at the edges.

The harness currently makes those behaviors feasible by structuring coordination, inserting behavioral guidance, and constraining risky actions. If models later absorb these patterns, some of that scaffolding may disappear too. The next frontier can also remain within software engineering: senior engineers spend substantial time on work beyond writing code.

The software-factory example raises the abstraction again. A bug arrives in a team channel and an automation triages it. That triage can start a cloud agent to attempt a fix. The agent then produces before-and-after video evidence. After the change merges and begins deploying, another agent watches the rollout for production problems. No single model call owns the lifecycle; a “meta-harness” connects specialized agent runs to events and deployment state.

The example remains a direction rather than a solved autonomous development lifecycle. Its value is architectural: each stage has a distinct trigger and responsibility, while the outer harness coordinates work across them. Better coding models improve the fix attempt, but they do not by themselves build the event routing, deployment integration, or production watcher.

Some harness responsibilities cannot migrate into model weights. Infrastructure and runtime are the first. Agent tasks have stretched from minutes toward hours and days. A sleeping laptop cannot reliably host an autonomous colleague through that duration, so execution state and compute move to persistent cloud infrastructure. A smarter model cannot keep a powered-down process alive or recover state that was never stored.

Trust and security are the second. A shell plus a file containing every API key might perform well on a coding benchmark, but benchmark capability does not make that design acceptable. Production systems need permissions scoped to particular agents and people, plus enforced limits that prevent catastrophic actions or runaway spending. Instructions can influence behavior; the harness controls what credentials and actions are actually available.

Efficiency is the third. The studies that reportedly showed similar evaluation scores across harnesses also found meaningful differences in cost per task. Katz says Grokbot’s team adjusted cache hit rate, token efficiency, and model mix, reducing cost without a reported loss of quality. His wording about speed does not establish whether it improved. The talk gives no figures or evaluation method, and does not explain how the model mix was allocated.

Efficiency compounds with adoption. A small per-task difference matters little when an agent runs occasionally; it becomes a major product and infrastructure concern when agents handle a growing share of daily work. Equal benchmark scores therefore do not imply equivalent harnesses. Reliability, authority boundaries, and cost can diverge even when the final answer score does not.

The flywheel is accelerating, but product requirements are again moving ahead of model behavior. That means the near-term harness becomes messier before it becomes cleaner: teams add interventions quickly, then remove them when models catch up. The engineering requirement is reversibility. A harness should make features cheap to ship, observe, and unship rather than allowing each workaround to harden into permanent architecture.

One response is to expose less of the harness’s internal machinery as a user-facing contract. Hyrum’s Law captures the risk: anything observable may acquire users who depend on it. Hiding internals reduces change-management cost and permits faster iteration, but it also reduces visibility into what an agent is doing. That is a real product tradeoff, especially when users need to inspect consequential work.

Another response is to design reusable primitives rather than one-off integrations. The example is subscriptions: an agent may need to wake when someone posts in Slack, CI turns red on a pull request, or a particular time arrives. Instead of giving every event source its own bespoke integration, one framework and one tool let the agent subscribe to an event. New triggers can then reuse the same model-facing concept.

Temporary behavioral patches receive their own lifecycle. If Grokbot waits too long to respond, a deterministic guardrail can nudge it that the user is waiting. Katz calls these patches shims: they fix current model behavior without waiting for another checkpoint. The team tracks shims centrally and logs every firing. A bot watches that telemetry; when a shim stops firing, it opens a pull request proposing removal. The workaround is observable by design, so model progress can delete code instead of leaving fossilized guardrails behind.

The harness is also moving up a level of abstraction. Earlier systems composed individual LLM calls; Grokbot and software-factory designs compose agents as their building blocks. Each child agent can own its own context, tools, and execution loop, while the parent system coordinates assignments and lifecycle events. This does not eliminate lower-level model orchestration—it places another control layer above it.

Katz closes with a concrete forecast: within six months, 90% of agent work will run off the laptop. That is a prediction, not an observed result. Its rationale follows from the runtime argument: agents that work for hours or days need persistent compute, resumable state, and access that survives a user closing a lid.

So the answer depends on which harness is under discussion. The coding-specific prompts and guardrails that once supplied missing intelligence are disappearing as models absorb their lessons. The harness that opens new task frontiers, coordinates agents, preserves state, limits authority, and controls cost is becoming more important. Yesterday’s harness can die while the system around tomorrow’s agents expands.