Claude for long-horizon tasks

What product should you build around a model that can finish increasingly substantial work without your intervention? Lance Martin starts with a metaphor: Claude is a light source, and products are windows that let its capabilities through. As the light changes, the window needs to change too. A chat interface suited to short exchanges may not expose what a model can accomplish when left to work.

Martin recalls Opus 3 around 2024 as occupying roughly a 10–20-minute task-horizon regime. He attributes the measurement to METR, whose horizons describe human-expert task duration at a specified agent success probability—not elapsed agent runtime. In that regime, autocomplete and chat made sense: the human stayed close, steering after relatively small units of work.

Martin describes the subsequent synchronous coding-agent regime as roughly an hour of work. Products such as Claude Code could undertake larger tasks while running locally, where a developer could readily intervene. Early asynchronous experiences were less satisfying: the agent would leave, encounter an error, and return soon afterward. Delegating the work did little to remove the need for supervision. Longer task horizons make the asynchronous product surface more useful because the agent can accomplish more between interventions.

The API surface changes along with the product. Sending messages to a model, supplying an agent harness, and operating that harness are different responsibilities. Martin traces Anthropic’s progression through three surfaces:

The Messages API is enough to build an agent, but you must implement and deploy the machinery around the model. The Agent SDK supplies that machinery. Managed Agents, which Martin dates to an April release, also supplies its deployment infrastructure. The architecture that follows is relevant beyond this particular managed service: any agent working asynchronously for long periods faces similar problems.

The first Managed Agents design put the harness and its execution sandbox in one container. That made the container a shared failure boundary: when the harness or container died, the session disappeared with it. A long-running task could lose its accumulated work because one process failed. Colocating credentials created another concern. Giving an unattended agent access to secrets throughout a ten-hour run increases the consequences of unsafe behavior.

The replacement architecture separates three responsibilities:

• Brain: a stateless harness that coordinates work.
• Session: a durable, append-only event log that records the interaction.
• Hands: execution containers where tools perform work.

The harness talks to the session and reaches out to execution environments. It need not have only one pair of hands: Martin reports that Claude can coordinate multiple containers from a single harness.

The session must outlive the process doing the work. If a harness or sandbox fails, its loss no longer destroys the durable session history. Credentials are also separated from generated sandbox code. Martin describes a separate vault; the companion Managed Agents architecture article describes both resource-bound authentication and an external OAuth vault. The common security property is credential isolation from the sandbox, rather than a requirement that every credential live in one vault.

A durable session also changes context management. Martin connects this design to Recursive Language Models: context can exist outside the active model window as an object the model inspects. The shared idea is externalized, retrievable context; the RLM method additionally uses recursive model calls, so an append-only log alone is not the full method.

With naive compaction, the summary becomes the surviving account of what happened. Anything omitted may be unavailable later. With a persistent session, compaction changes the active working context while the original events remain available for retrieval. Here, immutable means existing history is not destructively rewritten; new events can still be appended.

A small Python example makes the distinction concrete. Suppose a tool result records a failed test, but the compacted summary preserves only that tests ran. Keep the event log separate from the working context so the omitted detail remains retrievable:

from dataclasses import dataclass

@dataclass(frozen=True)
class Event:
    id: int
    kind: str
    text: str

session = (
    Event(1, "request", "Fix the parser and run its tests."),
    Event(2, "tool_result", "test_empty_input failed: expected [], got None"),
)

working_context = "Parser changes attempted; tests ran."

# Retrieval restores detail without rewriting the original events.
retrieved = next(event for event in session if event.id == 2)
working_context += "\n" + retrieved.text

# New activity extends the session rather than replacing its history.
session = session + (
    Event(3, "note", "Next step: investigate empty-input handling."),
)

The tuple illustrates the separation; the deployed architecture needs durable storage for the session. The useful invariant is that a shorter working context does not imply a shorter historical record.

An agent that has spent a long time producing an answer is not necessarily well positioned to judge it. Its context contains the work, its assumptions, and the reasoning that led to the result. Asking it to grade itself in that same context can produce confabulation and other unreliable behavior. Martin’s remedy is to put verification in a separate context window, tuned specifically for critique.

The resulting loop has a clear division of labor:

1. The build agent produces or revises the work.
2. A verifier receives the result and a goal or rubric in an independent context.
3. Its feedback directs another build iteration when the result is insufficient.
4. The loop finishes when verification confirms the required result.

This separates the context used to create an artifact from the context used to assess it.

Martin names goal in Claude Code and outcomes in Managed Agents as product primitives for this pattern. In his description, both establish a measurable end state, use independent context to grade progress, and make successful verification the completion condition. These are the talk’s conceptual semantics, rather than a demonstrated API call signature.

Martin applies the pattern to OpenAI’s Parameter Golf, a constrained language-model training challenge. The task requires training a small model on eight H100 GPUs in under ten minutes. The published rules additionally impose a 16 MB artifact limit and evaluate FineWeb validation bits per byte. Martin describes the plotted quantity as loss-like: lower is better.

His experiment uses Managed Agents outcomes with Opus 4.7 and a frontier Mythos-class model. Martin’s specified outcome required exactly twenty iterations and satisfaction of the benchmark’s experimental criteria. The training-time limit applies to the challenge’s training run; twenty iterations describes his agent experiment’s stopping condition. He reports strong results from the frontier model with this loop, without giving numerical scores in the spoken explanation.

The important mechanism is where steering happens. Instead of requiring Martin to inspect every result and decide what to do next, the environment supplies feedback through the verifier. The model can use that feedback to correct its next attempt. A capable model paired with an explicit completion condition and an independent feedback loop can keep working asynchronously without requiring the human to supply every intermediate correction.

Long-running work also requires deciding what to remember. Martin introduces two memory processes through a human-memory analogy: fast experiential traces, associated with the hippocampus, and offline consolidation into longer-term cortical memory. Remembering today’s lunch illustrates a transient trace; retaining an important experience illustrates something worth consolidating. For agents, the corresponding distinction is between writing memory during a task and improving it afterward.

The memory tool itself can be simple: permission to write files in a memory directory. The harder question is what the model chooses to save. In Martin’s Claude Plays Pokémon examples, Claude Sonnet 3.5 writes weak tactical notes and makes limited progress. A later 4.6 model, whose family he does not specify, writes more strategic notes and progresses further. The change is in the model’s ability to use the substrate, not merely in the availability of storage.

He observes a similar pattern in a task he calls Continued Learning Bench: sequential question answering over a SQL database, with opportunities to write memory between steps. Performance improves across the models he tests. His interpretation is that higher-capacity models are better at distillation—choosing an abstraction that will help in a future session rather than recording only a fact useful in the current one. Memory quality depends on whether an experience has been converted into reusable guidance.

Writing memory during a task has a weakness: the agent may save something incorrect, or something locally useful that does not generalize. Martin calls the out-of-band process for consolidating and improving that store dreaming. It gives memory a chance to be revised outside the immediate pressure of solving the current task.

The Pokémon example makes the failure concrete. Claude writes an incorrect memory about location, subsequently mislocalizes itself, and falls through a trapdoor. Persistent memory then carries the same mistake into another attempt. Martin reports trapdoor falls in five out of five replicates using the raw memory store. With dreaming, he reports that the location error is corrected and the agent avoids the trap. The side-by-side game screenshots show the resulting route contrast: a downward arrow under RAW MEMORY and an upward arrow under MEMORY + DREAMING.

The progress traces show why simply adding memory is insufficient. Upward movement represents progress to later levels; downward movement represents backtracking.

Raw memory can make an error repeatable. Its persistence is useful only if the stored guidance is useful.

Dreaming examines the memory store alongside prior traces or sessions, looking for errors and revising the stored guidance. The agent has more evidence available than the isolated moment in which it originally wrote the note. Without such correction, an in-band mistake can become a lasting instruction to repeat the wrong action.

The next shift concerns who the harness serves. Claude Tag appears through Slack, but the interface is only the visible entry point. Martin contrasts the system beneath it with a personal Claude Code installation: one developer’s machine, global context, connectors, and accumulated configuration. An organizational harness is shared by everyone who uses it.

That shared harness has its own identity and credentials rather than borrowing the identity of one particular user. It can work with organizational context, which makes several activities possible:

• Check prior work: inspect colleagues’ findings before beginning another experiment.
• Deduplicate research: recognize when an answer or investigation already exists.
• Investigate internally: combine context beyond one person’s local setup.
• Start with a working configuration: give a new employee access to an established harness immediately.

Martin contrasts that last capability with the weeks or months a person might otherwise spend configuring a personal harness and its connectors. The shared system makes a developed working environment available across the organization.

This is a larger product change than adding another chat endpoint. The harness can persist across users and support longer asynchronous work, while its usefulness no longer depends entirely on each employee independently assembling the same context and integrations.

An agent with organizational context need not wait for every question. Martin expects asynchronous agents to become increasingly proactive: a user can configure the system to watch that context and alert them when something relevant emerges. Locally scoped agents typically respond to direct steering; a persistent organizational agent can also initiate a useful interaction.

The other emerging interaction is multiplayer steering. Many people can direct the same harness concurrently. That changes the unit of interaction from a private conversation with a personal agent to a shared system coordinating work across people. Martin presents proactivity and multiplayer use as directions for asynchronous-agent UX.

The first audience question asks why frontier models lead lighter or open-source models on long-horizon tasks, and how long that gap might last. Martin recalls Codex 5.6 and Mythos-class or strong OpenAI models as reaching a twelve-plus-hour METR task-horizon regime. He does not know why non-frontier models lag and offers no timeline for how long the gap will persist.

His more concrete answer concerns the surrounding agent product. During a long unattended run, memory must encode the user’s working preferences—including when to ask for help. Security must address prompt injection. The architecture must separate brain and hands and tolerate execution failures. Infrastructure must keep the work operating. Model capability is necessary, but these other systems determine whether that capability can be used reliably.

Frontier labs’ investment across memory, security, architecture, and infrastructure is Martin’s possible explanation for the product gap. Managed Agents is intended to package those considerations so every application developer does not have to reconstruct them independently.

Why favor files for memory rather than a database? Martin’s answer is that files are not essential. What matters is a highly programmable substrate with simple primitives the model can use to write and manage its own memory. A database can provide that too.

The troublesome choice, in his experience, is prescribing the memory’s conceptual structure in advance: these are the categories, and these are the types of memories the model must save. That restriction can exist in a filesystem just as easily as in a database. Martin reports worse performance with overly prescriptive memory schemas and frames this as a Bitter Lesson observation: increasingly capable models can learn how to organize useful memory rather than relying on categories a developer intuited ahead of time.

Let the model structure and maintain its memory. The closing exchange makes the storage distinction explicit: expose a general medium as a tool for memory management. The important freedom is the ability to organize and revise what is stored, not a requirement to use files instead of database records.

The final question returns to dreaming: after inspecting sessions and updating the memory store, how do you know the new memories are correct? Martin points to evaluation. He characterizes the Pokémon result as an anecdotal example of a correction, then reports that other evaluations also show improvements from dreaming. The visible progress chart supplies the concrete example discussed here: memory with dreaming climbs toward Cerulean, while raw memory drops toward Pallet.

The decision still belongs to the application’s evaluations. A plausible rewrite of memory is not enough; it must improve the subsequent work that matters to the user. Measure whether that improvement is worth the offline compute used to produce it.