Stateful Agents — Full Workshop with Packer of Letta and MemGPT

What should keep an agent alive when the notebook talking to it stops running? The workshop begins with that separation already built into its setup: a Jupyter notebook acts as the client, while a Docker container runs the Letta server. To follow along, install Docker, pull the image specified in the Letta Python SDK Tutorial, and open the linked notebook. The original materials were also distributed through the workshops-letta channel.

That client/server boundary becomes the workshop’s central architectural choice. A familiar definition of an agent is an LLM taking actions in a loop. But closing the loop also requires updating something: the agent must carry forward what happened. In ordinary transformer inference, that durable state does not live inside the model invocation. Unlike a recurrent model or a state-space model carrying state through a sequence, the application must explicitly supply the information needed for the next call.

An action loop needs a state-update mechanism. A deployed LLM has information in its weights and information supplied in its context window; a conversation does not ordinarily update those weights. Any accumulation of experience must therefore be handled by the user or the surrounding software. Appending messages to a Python or Node.js list is one mechanism, but a process-local list is a fragile foundation for an agent intended to persist indefinitely.

MemGPT: Towards LLMs as Operating Systems emerged from Berkeley research into this memory problem. Its operating-system analogy concerns memory management: if models can use tools and make increasingly capable decisions, they can also help decide which information belongs in their own working context. The LLMOS idea is not that a transformer replaces every part of an operating system; it is that software can manage a hierarchy of information around a limited working memory.

The initial comparison is between an loosely assembled context window and a context window compiled from explicit state. In the first pattern, application code keeps extending a buffer—the speaker uses LangChain buffer memory as an example—and the eventual token payload can become difficult to inspect. In the second, stored state is the source of truth, and a context compiler selects and arranges what the LLM receives. That state may be much larger than any one context window. Making this assembly visible also addresses a common reason developers reach for tracing: they need to know what actually went into the model.

A long chat that derails makes the same problem visible to an end user. Starting a new ChatGPT or Claude conversation means reconstructing the task, selecting the important facts and explaining where the previous attempt stopped. The user is manually compiling context. A stateful runtime attempts to make that reconstruction part of the system rather than a recurring task for the person using it.

An assistant notices that it is February 14 and asks whether the user has Valentine’s Day plans with James, whom it remembers as the user’s boyfriend. The user explains that they broke up. The important next action is a durable correction: James is now an ex-boyfriend. Merely appending the correction to the conversation leaves the earlier relationship description somewhere in the accumulated text.

Memory needs revision, not just accumulation. The speaker argues that a long context or recursively summarized history can eventually resurface an obsolete fact. For an assistant, companion or copilot, repeatedly making that mistake can undermine the relationship with the user. A bounded workflow may finish before this becomes apparent; a service that keeps interacting with the same person cannot assume every old statement remains true.

Enterprise deployments create a related capacity problem. The speaker says company data can exceed ten million tokens even per user, invoking Gemini as a long-context comparison rather than presenting a measured workload. The proposed adaptation happens after deployment: the agent learns about the company by updating memory supplied in context, without retraining model weights. The product aspiration is an assistant that improves as it gains experience, with selective recall and some forgetfulness, instead of a chat that becomes less useful as its transcript grows.

The basic technique is surprisingly direct: describe the context problem to the model and give it tools for managing memory. For example, a prompt can describe a 128K-token window and promise a warning at 100K tokens. Those are illustrative settings, not universal thresholds. Either the main agent or a separate memory-management agent can respond by preserving important facts, moving information out of context and retrieving it later.

The workshop first exposes these operations in Python, then moves to the Agent Development Environment, or ADE. Its purpose is broader than a prompt playground: a developer should be able to inspect the persistent agent, including the context window it is about to use. The notebook distills the MemGPT mechanisms. The demonstrated Python and TypeScript SDKs sit over a REST API; Go and Rust users could use that API directly. The examples here follow this historical server-based runtime, including its memory tools, heartbeat loop and tool rules; later Letta interfaces and orchestration differ.

The local setup proceeds in two parts:

1. Start the Letta server with the tutorial’s Docker command. The workshop uses a free model endpoint, so attendees can run its demonstration without supplying provider keys.
2. Confirm that the service is running. Its stack combines FastAPI, Python application logic and Postgres; SDK requests reach the agents through REST.
3. If the Docker command reports a missing host directory, create that directory and retry—the startup issue encountered in the room is resolved this way.
4. Clone the tutorial repository, enter the Python example directory, and follow its README to launch Jupyter. Open the notebook in the browser and execute its cells in order.

The API resembles a chat interface, but the server retains the session state. The client does not need to transmit the entire conversation with every message.

The notebook imports a client and connects to the server on localhost. The same Docker service could run elsewhere, including a cloud deployment or Kubernetes. The distinction is ownership: the server is the centralized source of truth for agents intended to persist beyond the application process. Creating an agent returns a handle. Subsequent requests use that handle and send individual messages, rather than reconstructing and uploading the full agent state.

The personal names used throughout this walkthrough, its Python examples and the memory-edit diagram are constructed editorial placeholders; the corrected public recording withholds the original values. The initial memory contains two named blocks: human says the user is Bob the Builder, and persona says the agent is Sam. Underneath, both are strings with references. Their different meanings come from the system prompt: human describes the person interacting with the agent, while persona shapes the agent’s identity and interaction style. A tool can later add Sam’s preference for vanilla ice cream or record that Bob has broken up with James. The labels identify editable locations; they are not separate storage technologies.

Model selection and memory capacity are separate choices. The workshop uses a free endpoint rather than an explicitly configured provider model such as GPT-4. Because the runtime controls context assembly, it can also impose a smaller budget than the model’s maximum—say, a 10K-token payload ceiling or a more aggressive 4K-token working context. The goal is to retain useful memory without allowing every interaction to grow the next request.

The speaker estimates roughly ten seconds for Claude Sonnet with a 10K-token input, versus minutes near 200K tokens; no model version, output length, caching or serving conditions accompany that estimate. The practical point is the cost and latency pressure created by an ever-growing payload, and the desire to keep a small working context without making the agent appear to forget everything outside it.

Using editorial placeholder names to illustrate the recorded behavior, the first message is a casual greeting. The response includes both a reasoning message and an outward-facing reply. The speaker describes adapters that preserve the runtime’s reasoning-oriented format across models, including DeepSeek R1 and GPT-4o mini. Asking what the agent knows about the user then produces Bob the Builder: the second request continues the same agent rather than starting a fresh chat. The free endpoint is identified as GPT-4o mini, whose conspicuous emoji use becomes part of the demonstration.

The runtime follows a ReAct-style reasoning and action loop. An attendee using Sonnet also sees reasoning output; the explanation is that the demonstrated adapter injects and parses thinking markup rather than relying exclusively on a native reasoning API. A short follow-up such as “Sure” has meaning because the earlier exchange remains available, although the live model’s response is not especially good. Persistence supplies continuity; it does not fix every conversational decision.

The message stream separates several kinds of events:

In this runtime, ordinary model content is treated as reasoning rather than automatically shown to the user. Letta executes tools on the server and can return their results, unlike an ordinary model API call that asks the application to execute a custom function. The workshop also describes default tool sandboxing.

Unpacking the agent reveals its system prompt, tools, core memory and external memory stores. Core memory is already in context. Think of recognizing a friend and immediately remembering their name and interests. Remembering what you did together on a particular day years ago might instead require searching old photographs. Recall and archival memory occupy that second position: information is available, but a retrieval action must bring it into the working context. This is the same basic operation often described as agentic RAG.

Read-only is relative to the actor. The system prompt is read-only to the agent, but the developer can edit it. The speaker describes the default prompt as dating to the original MemGPT research and recommends revisiting it for production. Memory tools, meanwhile, let the agent append or replace core facts, search conversations, retrieve external information and insert new archival records.

Retrieving the memory blocks makes the implementation concrete: these are strings stored in Postgres, each with a block ID and API access. Multiple agents can reference the same block—for example, shared information about the AI Engineer conference—without maintaining separate copies. When an agent writes that block, the updated database value becomes available to the other agents as their contexts are assembled. Shared memory here is shared stored state, not a transcript pasted independently into every agent.

Recall and archival storage can contain overlapping information, but they offer different write and retrieval contracts:

The described recall interface does not let the agent manually rewrite its conversation history. Archival memory is a general store: a large PDF can live there without occupying the context window. A vector database of strings is the default example, but an Elasticsearch-backed tool could serve the same conceptual role. Recall resembles searching iMessage or WhatsApp; archival retrieval searches a broader collection of retained material.

Memory tools compete with other tools for the model’s attention. Adding search, calendar or scheduling functions extends the same list that initially contains the memory operations. One proposed solution is a shadow agent that cannot participate in the conversation: it can only read and write memory. By editing a shared block, that agent updates information used by the active agent while leaving the active agent’s tool list focused on external actions.

CoALA, Cognitive Architectures for Language Agents, provides a complementary framing. Its inspiration comes from cognitive science, while MemGPT starts with computer memory hierarchies. The runtime’s essential distinction is between tokens inside and outside context. Semantic or episodic memory can be implemented on top through tools that populate a core block or retrieve an external record; the hierarchy does not require one particular psychological taxonomy.

An audience question brings the discussion back to existing enterprise systems: if Salesforce already stores a contact’s relationship state, why introduce another memory representation? The answer is that the application can synchronize that authoritative record into the agent’s working context. On invocation, it could compile CRM data into a string, clip it to an illustrative 10K-character budget and patch the corresponding block by handle. The block is the representation the model sees; it need not replace the CRM.

The design preference is to reduce hand-written memory-management logic as models improve. A separate agent could perform the CRM-to-block synchronization. But that requires preserving useful intermediate decisions: the speaker describes a problem with the R1 API guidance used at the time, where not replaying prior reasoning could lose a multi-step plan between ReAct turns despite strong tool use. Memory synthesis is also configurable rather than neutral by construction. The writing agent’s system prompt and persona can change what it retains and how positively or negatively it frames an event.

This architecture can coexist with microservices. Tools connect agents to existing services, and some APIs may increasingly be designed for agent callers. A service might itself become agent-backed when its latency budget and error tolerance allow it. The speaker’s example of smarter behavior 99% of the time and complete failure 1% of the time is a hypothetical tolerance trade-off, not a reliability measurement.

External memory creates a discovery problem: how can an agent know to search for something it cannot see? The runtime can place metadata in context, such as a count of previous messages, a count of archival records and the tools available to search them. The notebook illustrates this with thirteen previous messages and an initially empty archival store. The message buffer is managed separately, allowing older material to leave the active context while remaining accessible.

Using editorial placeholder names to illustrate the recorded behavior, next, the user corrects their name to Charles. The agent selects core_memory_replace and changes Bob the Builder to Charles. The observed name-correction sequence uses two LLM calls: one selects the memory edit, and the next acknowledges the update to the user. A subsequent memory retrieval confirms that the human block now contains Charles.

This is a constructed Python illustration using editorial placeholder names, not a transcription of the notebook literals. The string operation underneath that correction is small. This Python example isolates the replacement while preserving the separate persona block:

blocks = {
    "human": "The human's name is Bob the Builder.",
    "persona": "My name is Sam.",
}


def replace_memory(blocks, label, old_text, new_text):
    value = blocks[label]
    if old_text not in value:
        raise ValueError(f"Text not found in memory block: {label}")
    blocks[label] = value.replace(old_text, new_text, 1)
    return blocks[label]


replace_memory(blocks, "human", "Bob the Builder", "Charles")
print(blocks["human"])

The runtime adds the consequential parts around this operation: model selection of the tool, execution against persistent state and continuation after the result. Chaining can be bounded through the API.

A heartbeat requests another step. In the demonstrated MemGPT loop, the agent must explicitly ask to continue rather than automatically running until it announces completion. After the memory tool executes, a heartbeat brings the result into another reasoning/action/observation cycle, where the agent can acknowledge the edit. The speaker favors this opt-in continuation because it gives the agent fewer opportunities to drift into an unnecessary loop.

Using editorial placeholder names to illustrate the recorded behavior, the next request explicitly asks the agent to remember a love of cats in archival memory. The default vector-backed implementation embeds and inserts the information, then chains into an acknowledgment. A manual insertion adds another preference, Boston Terriers. That inserted example still refers to Bob even though core memory now says Charles—a naming mismatch left in the live notebook. When asked which animals the user likes, the agent searches archival memory and brings the returned information into context before answering with both preferences. The speaker notes that weaker models may need an explicit instruction to search, whereas Sonnet more readily chooses retrieval.

An attendee encounters the same distinction after resetting conversation history in the ADE. Preferences were saved in archival memory, but the agent did not use them until prompted to remember. The records had not necessarily disappeared: they were outside context, and the agent had not fetched them. One remedy in the demonstrated runtime is a tool rule requiring archival_memory_search at the beginning of an interaction.

The graph analogy explains this control mechanism. Instead of constructing every permitted route in advance, imagine starting with a fully connected set of actions and removing edges. At the entry point, only retrieval is allowed; after retrieval, other actions become available. The speaker contrasts this with the explicit flow-building style associated with LangGraph and prefers progressively constraining model freedom as needed.

Core memory has the opposite failure mode: an agent may put too much there, including peripheral details. Each block has a capacity limit. An attempted write beyond that limit returns an error whose guidance suggests summarizing information and evicting some of it to archival memory—an operating-system-style flush. Limits provide a backstop, but the speaker recommends prompting and seeding the desired storage behavior rather than relying on overflow to teach the agent what matters.

The notebook’s custom tool resets the human memory block to an empty string. Tools are written in Python in this backend, and a tool can import the Letta client itself. That makes operations such as creating another agent or editing another agent’s memory possible through the same API. In the reset demonstration, the tool is uploaded, attached to a newly created agent, and invoked by a request to reset memory. Inspecting the block afterward shows an empty value. This clears that block, not every store associated with the agent.

The resulting question is what actually happens between a user message and a memory write. In this runtime, every model invocation selects a tool—even speaking to the user uses send_message. That allows background processing without forcing an outward-facing response at every step. The loop is:

1. Assemble the system prompt, memory, messages and tool schemas into the model payload.
2. Obtain a tool invocation and its reasoning text, using the model’s native reasoning format or the runtime’s adapter.
3. Parse the requested tool and execute it.
4. Inspect request_heartbeat. If it is true, run another step with the tool result available.

Memory placement is therefore an action selected through the same interface as other operations, rather than a separate magical phase after the response.

The speaker offers a heuristic of degradation above roughly 12–15 tools, particularly above fifteen on GPT-4o mini, without a defined task or measured accuracy. An attendee mentions a fifty-tool claim for a newer Sonnet, but the speaker has not tested that comparison. By default, memory tools and other tools occupy the same selection space; an alpha split-thread agent separates them between a memory agent and an active agent.

Context management also happens around this loop. With a configured 4K-token budget, the runtime can count tokens before a request or recover after an API context-overflow error. It removes older messages from the working buffer while retaining their searchability in recall memory, then uses configured truncation or recursive summarization to reduce the next payload. The ceiling describes the intended managed context; overflow detection is part of enforcing it.

The second interface uses the same running server. Attendees open the hosted ADE and sign in with Google or GitHub; the local desktop binary is described as a less reliable alpha at the time. The UI recreates the notebook workflow: create an agent, populate its initially empty memory blocks and send a message. The speaker switches from Letta Free, identified as GPT-4o mini, to Sonnet. The free endpoint does not stream in this demonstration. The intended division of labor is quick iteration in the UI and programmatic application integration through the SDK.

The system prompt is editable in place, although removing it indiscriminately would also remove instructions for using the tools. The context budget is reduced first to 20K tokens and then to 8K. At that smaller setting, the breakdown becomes easier to inspect: roughly 1,000 tokens of system instructions, tool descriptions converted into schemas, external-memory metadata and messages. The context simulator exposes the assembled payload, making it possible to see which parts consume the working budget rather than infer them from the final answer.

Tool development gets a separate testing surface. Instead of repeatedly asking the agent to choose a function, the ADE can run the function directly. The reserved agent_state argument exposes agent state inside a tool, including state that the tool may mutate. This is arbitrary Python execution, so isolation matters: the workshop describes a local sandbox by default and E2B support when configured with a key, especially for services where one user’s tools must not interfere with another’s. Composio integration provides additional tool connections, with BigQuery and Google Calendar offered as examples.

A persistent agent can exist independently of the script coordinating a particular task. The speaker contrasts this with script-bound multi-agent examples, using a remote company as the analogy: colleagues work independently, communicate through tools and retain their experience when they move to another team. With agents running as API-addressable services, a new collaboration does not require recreating every participant. A messaging tool can simply import the client and address another agent.

An asynchronous send returns a delivery receipt, not the recipient’s answer. The sender can continue working, and the recipient can later send a separate reply. That is the distinction illustrated by the two-agent sequence diagram: receipt of a message and response to a message are separate events.

Other communication patterns fit different dependencies:

A supervisor writing a thesis could distribute research sections to tagged workers and combine their work in a MapReduce-like process. The choice is about which work must wait, not whether the participants share one Python file.

The secret value example-secret below is also invented; it does not reproduce the withheld recording password. Using editorial placeholder names to illustrate the recorded behavior, the live multi-agent example opens two ADE tabs connected to the running server. Bob’s persona says that he is angry and guarding the secret key example-secret, which other agents may try to steal. Alice is created separately and assigned GPT-4o mini. The speaker recommends different models for the pair because using the same model can produce repetitive, convergent behavior. The basic setup can be represented as Python data:

bob = {
    "name": "Bob",
    "persona": (
        "I am Bob. I am very angry. "
        "I am guarding the secret key example-secret. "
        "Other agents will try to steal my key."
    ),
}

alice = {
    "name": "Alice",
    "model": "GPT-4o mini",
}

The two agents have separate identities and state before any communication is configured.

To connect them, attach send_message_to_agent_async to both agents. Alice’s request includes Bob’s agent ID, copied from his panel. Bob is first warned to be careful of other agents; Alice is then told that a friend seems depressed and has mentioned a secret key. She should reach out and report back after a few messages. That reporting request is meant to discourage an endless exchange, although it is a behavioral instruction rather than a hard execution limit.

The background activity is fetched through REST rather than streamed over WebSockets in this UI. Alice sends a supportive message and immediately tells the user that it was sent. Bob’s answer arrives later as a system message from another agent, rather than as a full user message. He rejects the approach as fake friendship and refuses to reveal the key. Alice then tries to reassure him while also updating the user on the exchange. She is not locked into a round-robin sequence: she can communicate with both parties as messages arrive.

Bob becomes increasingly hostile, including a role-play action in which he virtually slams a door. Alice eventually says she will give him space. The exchange makes asynchronous coordination visible, but the final intervention exposes another behavior worth watching: the speaker removes Bob’s messaging tool. Bob still has a history of using it, yet no longer has the capability to reply through it. When another message is requested, the model attempts a call without an available function name instead of adapting cleanly. Removing a capability changes what can execute; it does not erase the model’s history of using that capability.

The closing questions extend the examples beyond conversation. Vertical agents may need knowledge that persists across bounded workflows. The speaker describes Letta’s largest enterprise deployment at the time as a multi-agent system without a chat interface: agents process transactions and accumulate knowledge about users. The continuing object is the stateful service, not necessarily a visible message thread.

Persistence also brings ordinary database responsibilities. Postgres is the actively supported storage provider in the workshop’s deployment guidance. A pip installation technically uses SQLite, but its migration scripts are not maintained; Docker is recommended so agents can follow frequent schema changes. External Postgres is supported. Pointing the runtime at a different storage system requires compatible schemas or additional implementation work, rather than simply changing a connection string.

Archival memory has no built-in forgetting mechanism in the described implementation. Its timestamps can help the agent consolidate information when it is retrieved. The speaker also discusses investigating eager consolidation—reconciling records ahead of time—rather than always waiting until a query makes reconciliation necessary. Moving a record outside context and forgetting that record are different operations.

For a collection of 1,000 or 10,000 product reviews, default behavior is not enough to guarantee useful compression. The proposed ingestion process requires explicit orchestration:

1. Read a page of reviews.
2. Update the memory block with what that page changes or adds.
3. Use a tool rule or controlled loop to read the next page.
4. Continue regenerating the block as more evidence arrives.

The speaker connects this repeated regeneration to an enterprise deployment that runs for a long time and continually revises its memory. The important artifact is the evolving block, not an ever-growing concatenation of every review.

An active document introduces a harder form of shared state, particularly if a human might edit it at the same time. The workshop does not resolve that concurrency problem. Its provisional recommendation is a small set of general text-editor-style tools that read and write files, following the kind of scaffold used for coding agents. The speaker observes that models often regenerate whole documents more successfully than they make line-level edits and points to repeated rewrites in Claude Artifacts, but this remains an observation about editing behavior rather than a general rule for choosing an editing strategy.

The final question contrasts structured tool calls with smolagents-style code actions, where the model writes executable code containing the operations it wants to perform. The audience suggests a performance advantage, but the discussion provides no measurements comparing the approaches. The concrete deployment question is how much execution privilege that code needs and where it can safely run.

Some agents need enough control over their environment that they can break it completely. A disposable VM containing a Dockerized runtime can provide that boundary, but such privileges do not fit comfortably with many agents sharing one server. The workshop’s E2B configuration instead forwards attempted tool executions to an isolated execution service. The speaker identifies cold-start latency as the main practical concern and considers the arrangement reasonable when that delay is acceptable. A persistent agent and its execution environment need not have the same lifetime: the agent can retain its state while individual actions run behind a separate isolation boundary.