Architecting Agent Memory: Principles, Patterns, and Best Practices

How do you turn a stateless AI application into one that remembers its users? Repeating instructions in every prompt can establish behavior for a single interaction, but continuity requires persistence. Richmond Alake opens with three connected changes: stateless applications become stateful, repeated prompt setup gives way to persistent information, and isolated responses become relationships across interactions. The intended result is an agent that is believable, capable, and reliable.

The progression begins with LLM chatbots such as ChatGPT, then retrieval-augmented generation, or RAG, which supplies domain-specific knowledge for more relevant responses. In Alake’s account, scaling compute and data helped bring reasoning and tool use into the picture, leading to agents and agentic systems. He treats agency as a spectrum, using levels of self-driving capability—and his encounter with Waymo—as an analogy. At one end is an LLM running in a loop; farther along are systems with tools and greater freedom to act under minimal prompting.

His working definition combines perception of an environment, cognition through an LLM, action through tools, and short- or long-term memory. Memory supplies continuity for reflection, interaction, proactive behavior, and responses to changing circumstances. Alake, a MongoDB AI/ML developer advocate, develops that architecture through MongoDB examples throughout the talk.

Short-term and long-term describe how long information remains useful, but they do not describe everything the application needs to remember. Alake also names conversational memory, entity memory, knowledge, data stores, caches, and working memory. These categories distinguish information by its role in the system. His motivation comes from human recall: if AI aims to reproduce aspects of human intelligence, the ability to retain and recover information is an architectural concern. That is a useful analogy, rather than a complete definition of intelligence.

The human comparison adds semantic, episodic, and procedural memory alongside short-term, long-term, and working memory. Alake asks whether anyone can do a backflip to make procedural memory concrete: knowing a routine or skill differs from remembering a fact or an event. He associates that skill with the cerebellum in a simplified biological account. The engineering implication is to consider distinct representations for different kinds of retained information, rather than treating every memory as another chat message.

Agent memory is persistent state that can inform a later execution step. An agent accumulates information, retains it, and brings it back when deciding what to do next. Storage alone is therefore insufficient: the application needs a memory-management system that organizes what enters the model’s context window. A large context window provides room for relevant memories; it is not a reason to insert the entire database. Selection and structure determine whether retained information helps produce a relevant response.

Alake outlines a lifecycle of generation, storage, retrieval, integration, update, and deletion. He then argues for exploring forgetting mechanisms: memory management should consider which experiences remain salient, not only whether records exist. The talk offers this as a design direction, without supplying a forgetting algorithm. The human-memory analogy does not remove an application’s need to delete data when required. Within this lifecycle, retrieval receives the most attention because it connects accumulated information to the next decision.

RAG already provides a familiar version of that connection, and retrieval need not mean vector search alone. Different information needs call for different search methods. MongoDB is Alake’s proposed provider for those capabilities, rather than a prerequisite for RAG. Agentic RAG changes who controls access: expose retrieval as a tool, and the agent can choose when to request information instead of relying entirely on a fixed retrieval step before every response.

Alake positions MongoDB as a memory provider: a storage and retrieval foundation on which developers can build memory behavior. His open-source companion project, Memorizz, pronounced “Memoriz” in the talk, collects patterns for the memory types he introduces. Persona is the first example. After criticizing ChatGPT’s recent personality changes, he identifies the underlying goal as worthwhile: agents should have characteristics that make interactions believable and relationships more continuous. Persona memory represents those characteristics in database records.

Toolbox memory addresses a different constraint: the model does not need every available tool definition in every request. Alake attributes a rough guideline of 10–21 tool schemas to OpenAI. Treat that as his historical account, not an API limit; the current function-calling guide gives a soft suggestion of fewer than 20 initially available functions and recommends evaluating tool counts. The architectural pattern is to keep the full toolbox outside the context window and retrieve a relevant subset before calling the LLM.

The toolbox procedure is straightforward:

1. Store tool definitions as documents, including names, descriptions, and JSON parameter schemas.
2. Search those records for tools relevant to the current request.
3. Supply the selected definitions to the LLM for its next decision.

The selected slide also includes embedding vectors in the toolbox schema. Retrieving a definition makes a tool available to the model; it does not itself execute the tool.

This pattern depends on two complementary database properties: documents can represent different memory structures, and retrieval can match the information need. Alake lists graph, vector, text, and geospatial capabilities in MongoDB. Tool discovery is one use of that flexibility; the same storage foundation can hold persona records and other kinds of memory without requiring them all to share one shape.

Conversational memory stores the back-and-forth exchanges familiar from ChatGPT or Claude. Alake’s records include timestamps and a conversation ID, then add recall recency and an associated conversation ID. Those extra fields are intended as memory signals: metadata that could help a management system reason about when information was recalled and how conversations relate. He connects them to his experimental work on forgetting in Memorizz. The fields show what the system can record; they do not establish a completed forgetting policy.

Workflow memory supplies the clearest feedback example. An agent executes step one, then step two, then fails at step three. That failure is experience worth retaining. Store it, retrieve it during the next execution, and use it to inform the LLM that it should avoid repeating the failed step or explore another path. The adaptation occurs through retrieved context, with no model-weight update described. A retained failure can influence another attempt; it does not guarantee that the next attempt succeeds.

A compact Python representation makes the boundary between recorded experience and pending action explicit. Here, workflow-1 is a teaching identifier for the three-step workflow. The next prompt includes its failed step, but no retry has occurred:

import json

workflow_memory = [
    {
        "workflow_id": "workflow-1",
        "steps": [
            {"step": 1, "outcome": "success"},
            {"step": 2, "outcome": "success"},
            {"step": 3, "outcome": "failure"},
        ],
    }
]


def build_retry_prompt(workflow_id, memories):
    prior_attempts = [
        memory for memory in memories
        if memory["workflow_id"] == workflow_id
    ]
    return (
        f"Plan another attempt at {workflow_id}.\n"
        "Use the recorded outcomes to consider an alternative "
        "to the failed step. No retry has executed yet.\n"
        f"Prior attempts:\n{json.dumps(prior_attempts, indent=2)}"
    )


next_prompt = build_retry_prompt("workflow-1", workflow_memory)

In a persisted implementation, the retrieval supplies the relevant workflow records from the database. The essential operation is the same: prior outcomes become input to the next decision.

The database example on screen is an expanded document in memorizz.workflow_memory, with a tool execution step, arguments, a result, timestamps, and an outcome. Its visible outcome is success, while the spoken example concerns failure. Both belong in execution history: workflow memory records what happened so that later decisions can use that experience.

Alake then names episodic memory and long-term memory, followed by an agent registry that stores agent information, including its tools and persona. Entity memory is another category, though it receives no detailed implementation walkthrough. These examples are patterns to study: Alake explicitly describes Memorizz as experimental and educational.

The distinction between infrastructure and behavior matters when choosing an architecture. Alake cites MemGPT, Mem0, and Zep as tools focused on memory management, while positioning MongoDB as a provider for custom solutions.

His practical point is that applications do not all need the same memory policy. A common data foundation can support different choices about how memory informs an agent.

That retrieval emphasis leads to MongoDB’s acquisition of Voyage AI. Alake describes its text and multimodal embedding models and rerankers as ways to improve retrieval and reduce hallucinations. He calls the models market-leading, but supplies no benchmark task, model version, or measured hallucination reduction. These are vendor claims about the intended benefits, not demonstrated results from the workflow example.

The roadmap then moves more retrieval work toward the database. Alake forecasts bringing Voyage AI embedding models and rerankers into MongoDB Atlas within a few months, with the ambition of relieving developers of chunking and retrieval-strategy work. This is a prospective integration in the recording, not a deployment walkthrough. It supports the broader product promise he closes this portion with: helping developers build AI features quickly and securely while the surrounding technology changes.

The closing returns to biology through David Hubel and Torsten Wiesel, whose work on visual information processing earned them a share of the 1981 Nobel Prize in Physiology or Medicine. Alake describes their visual-cortex research in terms of hierarchical representations: edges, contours, and more abstract shapes. He uses that account to explain how neuroscience inspired ideas associated with convolutional neural networks and applications such as face and object detection. The analogy is architectural: studying a biological system can suggest useful computational structures without requiring software to reproduce the whole brain.

For agent memory, Alake proposes a similar exchange between disciplines. He recounts a recent meeting with Tengyu Ma, whom he introduces as MongoDB’s chief AI scientist and Voyage AI’s founder, alongside neuroscientists and Charles Packer, a coauthor of MemGPT associated with Letta. One participant, Kenneth, is described as a longtime researcher of the brain and memory. The meeting supplies the talk’s final direction: bring people who study memory together with people who implement agent systems. Nature offers ideas for what to retain, how to recall it, and how experience might guide future action; developers still have to turn those ideas into explicit mechanisms.