AI Engineer World's Fair 2025
Architecting Agent Memory: Principles, Patterns, and Best Practices
Read the talk
Architecting Agent Memory: From Persistent Data to Useful Experience
Agent memory requires more than saving chat history: it needs retrieval, context selection, and records that let earlier interactions shape later actions.
From a talk by Richmond Alake
Before you start: Familiarity with LLM context windows, retrieval-augmented generation, and tool calling will help you follow the architecture and examples.
From stateless responses to persistent relationships
How do you turn a stateless AI application into one that remembers its users? Repeating instructions in every prompt can establish behavior for a single interaction, but continuity requires persistence. Richmond Alake opens with three connected changes: stateless applications become stateful, repeated prompt setup gives way to persistent information, and isolated responses become relationships across interactions. The intended result is an agent that is believable, capable, and reliable.
The progression begins with LLM chatbots such as ChatGPT, then retrieval-augmented generation, or RAG, which supplies domain-specific knowledge for more relevant responses. In Alake’s account, scaling compute and data helped bring reasoning and tool use into the picture, leading to agents and agentic systems. He treats agency as a spectrum, using levels of self-driving capability—and his encounter with Waymo—as an analogy. At one end is an LLM running in a loop; farther along are systems with tools and greater freedom to act under minimal prompting.
His working definition combines perception of an environment, cognition through an LLM, action through tools, and short- or long-term memory. Memory supplies continuity for reflection, interaction, proactive behavior, and responses to changing circumstances. Alake, a MongoDB AI/ML developer advocate, develops that architecture through MongoDB examples throughout the talk.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retention time is only one dimension of memory
Short-term and long-term describe how long information remains useful, but they do not describe everything the application needs to remember. Alake also names conversational memory, entity memory, knowledge, data stores, caches, and working memory. These categories distinguish information by its role in the system. His motivation comes from human recall: if AI aims to reproduce aspects of human intelligence, the ability to retain and recover information is an architectural concern. That is a useful analogy, rather than a complete definition of intelligence.
The human comparison adds semantic, episodic, and procedural memory alongside short-term, long-term, and working memory. Alake asks whether anyone can do a backflip to make procedural memory concrete: knowing a routine or skill differs from remembering a fact or an event. He associates that skill with the cerebellum in a simplified biological account. The engineering implication is to consider distinct representations for different kinds of retained information, rather than treating every memory as another chat message.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Stored information becomes memory when it informs execution
Agent memory is persistent state that can inform a later execution step. An agent accumulates information, retains it, and brings it back when deciding what to do next. Storage alone is therefore insufficient: the application needs a memory-management system that organizes what enters the model’s context window. A large context window provides room for relevant memories; it is not a reason to insert the entire database. Selection and structure determine whether retained information helps produce a relevant response.
Alake outlines a lifecycle of generation, storage, retrieval, integration, update, and deletion. He then argues for exploring forgetting mechanisms: memory management should consider which experiences remain salient, not only whether records exist. The talk offers this as a design direction, without supplying a forgetting algorithm. The human-memory analogy does not remove an application’s need to delete data when required. Within this lifecycle, retrieval receives the most attention because it connects accumulated information to the next decision.
RAG already provides a familiar version of that connection, and retrieval need not mean vector search alone. Different information needs call for different search methods. MongoDB is Alake’s proposed provider for those capabilities, rather than a prerequisite for RAG. Agentic RAG changes who controls access: expose retrieval as a tool, and the agent can choose when to request information instead of relying entirely on a fixed retrieval step before every response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Persist persona, retrieve the tools needed now
Alake positions MongoDB as a memory provider: a storage and retrieval foundation on which developers can build memory behavior. His open-source companion project, Memorizz, pronounced “Memoriz” in the talk, collects patterns for the memory types he introduces. Persona is the first example. After criticizing ChatGPT’s recent personality changes, he identifies the underlying goal as worthwhile: agents should have characteristics that make interactions believable and relationships more continuous. Persona memory represents those characteristics in database records.
Toolbox memory addresses a different constraint: the model does not need every available tool definition in every request. Alake attributes a rough guideline of 10–21 tool schemas to OpenAI. Treat that as his historical account, not an API limit; the current function-calling guide gives a soft suggestion of fewer than 20 initially available functions and recommends evaluating tool counts. The architectural pattern is to keep the full toolbox outside the context window and retrieve a relevant subset before calling the LLM.
The toolbox procedure is straightforward:
- Store tool definitions as documents, including names, descriptions, and JSON parameter schemas.
- Search those records for tools relevant to the current request.
- Supply the selected definitions to the LLM for its next decision.
The selected slide also includes embedding vectors in the toolbox schema. Retrieving a definition makes a tool available to the model; it does not itself execute the tool.
This pattern depends on two complementary database properties: documents can represent different memory structures, and retrieval can match the information need. Alake lists graph, vector, text, and geospatial capabilities in MongoDB. Tool discovery is one use of that flexibility; the same storage foundation can hold persona records and other kinds of memory without requiring them all to share one shape.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Conversation history needs retrieval signals
Conversational memory stores the back-and-forth exchanges familiar from ChatGPT or Claude. Alake’s records include timestamps and a conversation ID, then add recall recency and an associated conversation ID. Those extra fields are intended as memory signals: metadata that could help a management system reason about when information was recalled and how conversations relate. He connects them to his experimental work on forgetting in Memorizz. The fields show what the system can record; they do not establish a completed forgetting policy.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use a failed execution as context for the next attempt
Workflow memory supplies the clearest feedback example. An agent executes step one, then step two, then fails at step three. That failure is experience worth retaining. Store it, retrieve it during the next execution, and use it to inform the LLM that it should avoid repeating the failed step or explore another path. The adaptation occurs through retrieved context, with no model-weight update described. A retained failure can influence another attempt; it does not guarantee that the next attempt succeeds.
A compact Python representation makes the boundary between recorded experience and pending action explicit. Here, workflow-1 is a teaching identifier for the three-step workflow. The next prompt includes its failed step, but no retry has occurred:
python
import json
workflow_memory = [
{
"workflow_id": "workflow-1",
"steps": [
{"step": 1, "outcome": "success"},
{"step": 2, "outcome": "success"},
{"step": 3, "outcome": "failure"},
],
}
]
def build_retry_prompt(workflow_id, memories):
prior_attempts = [
memory for memory in memories
if memory["workflow_id"] == workflow_id
]
return (
f"Plan another attempt at {workflow_id}.\n"
"Use the recorded outcomes to consider an alternative "
"to the failed step. No retry has executed yet.\n"
f"Prior attempts:\n{json.dumps(prior_attempts, indent=2)}"
)
next_prompt = build_retry_prompt("workflow-1", workflow_memory)
In a persisted implementation, the retrieval supplies the relevant workflow records from the database. The essential operation is the same: prior outcomes become input to the next decision.
The database example on screen is an expanded document in memorizz.workflow_memory, with a tool execution step, arguments, a result, timestamps, and an outcome. Its visible outcome is success, while the spoken example concerns failure. Both belong in execution history: workflow memory records what happened so that later decisions can use that experience.
Alake then names episodic memory and long-term memory, followed by an agent registry that stores agent information, including its tools and persona. Entity memory is another category, though it receives no detailed implementation walkthrough. These examples are patterns to study: Alake explicitly describes Memorizz as experimental and educational.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate the memory provider from memory management
The distinction between infrastructure and behavior matters when choosing an architecture. Alake cites MemGPT, Mem0, and Zep as tools focused on memory management, while positioning MongoDB as a provider for custom solutions.
| Layer | Responsibility |
|---|---|
| Memory provider | Store records and expose retrieval capabilities |
| Memory management | Decide what to retain, retrieve, and integrate |
His practical point is that applications do not all need the same memory policy. A common data foundation can support different choices about how memory informs an agent.
That retrieval emphasis leads to MongoDB’s acquisition of Voyage AI. Alake describes its text and multimodal embedding models and rerankers as ways to improve retrieval and reduce hallucinations. He calls the models market-leading, but supplies no benchmark task, model version, or measured hallucination reduction. These are vendor claims about the intended benefits, not demonstrated results from the workflow example.
The roadmap then moves more retrieval work toward the database. Alake forecasts bringing Voyage AI embedding models and rerankers into MongoDB Atlas within a few months, with the ambition of relieving developers of chunking and retrieval-strategy work. This is a prospective integration in the recording, not a deployment walkthrough. It supports the broader product promise he closes this portion with: helping developers build AI features quickly and securely while the surrounding technology changes.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Borrow architectural ideas from the study of memory
The closing returns to biology through David Hubel and Torsten Wiesel, whose work on visual information processing earned them a share of the 1981 Nobel Prize in Physiology or Medicine. Alake describes their visual-cortex research in terms of hierarchical representations: edges, contours, and more abstract shapes. He uses that account to explain how neuroscience inspired ideas associated with convolutional neural networks and applications such as face and object detection. The analogy is architectural: studying a biological system can suggest useful computational structures without requiring software to reproduce the whole brain.
For agent memory, Alake proposes a similar exchange between disciplines. He recounts a recent meeting with Tengyu Ma, whom he introduces as MongoDB’s chief AI scientist and Voyage AI’s founder, alongside neuroscientists and Charles Packer, a coauthor of MemGPT associated with Letta. One participant, Kenneth, is described as a longtime researcher of the brain and memory. The meeting supplies the talk’s final direction: bring people who study memory together with people who implement agent systems. Nature offers ideas for what to retain, how to recall it, and how experience might guide future action; developers still have to turn those ideas into explicit mechanisms.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Richmond Alake’s experimental Python framework for persistent agent memory, with installation instructions and examples.
The original MemGPT paper explains how an agent manages information across memory tiers beyond its immediate context window.
A memory layer for personalized AI assistants and agents, with an open-source implementation and setup examples.
The February 2025 announcement outlines MongoDB’s acquisition of Voyage AI and plans to integrate embedding and reranking models.
Further reading
Explains Zep’s use of Graphiti to combine conversations and structured business information while retaining temporal relationships.
Updates since the talk
Current guidance for defining tool schemas, controlling initially available tools and handling function calls.
Documents automatic embedding generation for indexed text and queries using Voyage AI models. The inspected documentation labels this feature Preview.
Read the complete timestamped transcript
- 0:00
[upbeat music] In the next ten to fifteen minutes, here's, uh, I guess, my promise to you.
- 0:20
I'm going to give you some information that will be high level. There will be some practical component to it, but
- 0:29
this information I'll give you within the next six month will be very relevant, and it will put you in the best position to build the best AI applications, to build the best agents that are believable, capable, and reliable.
- 0:42
Nothing on this screen. I know. [laughs] We, we, we gon' get there. [laughs]
- 0:48
You know what? Just for you. Thank you. There we go. You're welcome. So we're gonna be talking about memory. Um,
- 0:56
we're gonna be talking about the stateless applications that we're bui- building today, and how we can make them stateful. We're gonna be talking about the prompt engineering that we're doing today, and how we can reduce that by focusing on persistence.
- 1:11
We're gonna be turning the responses in our AI application and making our agents build relationship with our customers, and all of it is gonna be centered around memory.
- 1:24
So I'm gonna do a very quick evolution of what we've been seeing for the past two to three years.
- 1:33
We started off with chatbots, LLM-powered chatbots. They were great. ChatGPT came out November 2022, and yeah, exploded. Then we went into RAG. We gave these chatbots more domain-specific relevant knowledge, and they gave us more personalized responses.
- 1:50
Then we begin to scale the compute, the data we're giving to the LLMs, and they gave us emerging capabilities, right? Reasoning, uh, tool use. Now we're in the world of AI agents and agentic systems,
- 2:04
and the big debate is what is an agent, right? What is an AI agent? I don't like to go into that debate because that's like asking what is consciousness.
- 2:14
Um, it's a spectrum. The agenticity, and that's a word now, agenticity- [laughs]
- 2:21
... of, uh, of an agent is it is a spectrum, so there are different levels.
- 2:28
I came here and I saw Waymo, and to me it was pure sorcery. We don't have that in the UK, and there are different levels of, um, self-driving, so you can look at the agentic spectrum in that respect.
- 2:38
We have a minimal agent, where it's an LLM running a loop. Great. Then you have, uh, level four is autonomous agent. A bunch of agents that have access to tools.
- 2:48
You ... They can do whatever they want. They're not prompted in any way or a minimal way. But this is how I see things, as a spectrum. So what is an AI agent?
- 2:58
It's a computational entity with awareness of its environment through perception, cognitive abilities through an LLM, and also can take action through tool use. But the most important bit is there is some form of memory, short term or long term.
- 3:15
Memory is important. It's important because we're trying to make our agents reflective, interactive, proactive, and reactive, and autonomous, and every, most of this, if not all, can be solved with memory.
- 3:30
Um, I work at MongoDB, and we're gonna make ... we're gonna connect the dots. Don't worry. So this is all nice and good. This is what, what you look at if you, um, double-click into one AI agent is...
- 3:41
But the most important bit to me is... I- I'll go to slide. People are taking pictures, sorry.
- 3:48
All right, let's go. The most important bit is memory, and when we talk about memory, the, the easy way you can think about it is short-term, long-term, but there are als- there are other distinct forms, right?
- 3:56
Um, conversational, entity memory, knowledge, data store, cache, working memory. We're gonna be talking about all of that today. So these are the high level concepts.
- 4:05
But let me go a little bit meta.
- 4:09
Why we're all here, um, today in this conference is because of AI, right? We're all architects of intelligence. The whole point of AI is to build some form of computational entity that surpasses hu- human intelligence or mimics it.
- 4:23
Then AGI, we're focused on making that intelligence surpass humans in all tasks we can think of. And if you think about the most intelligent humans you know,
- 4:36
what determines their intelligence is their ability to recall. It's their memory. So if we, if AI or AGI is meant to mimic human intelligence, it's a no-brainer, no pun intended, that we need memory within the agents that we're building today.
- 4:50
Does anyone disagree? Good. I would have kicked you out. [laughs]
- 4:56
Um, okay, let's go. So humans, you, in your brain right now, you have these, you have this. This is not what it looks like, but it's close enough. You have different forms of memory, and that's what makes you intelligent.
- 5:06
That's what makes you retain some of the information I'm gonna be giving you today. There is short-term, long-term, working memory, semantic, episodic, procedural memory. Um, in your brain right now, there is something called the cerebellum.
- 5:19
I always get the word wrong, but that's where you store most of the routines and skills you can do. Can anyone here do a backflip?
- 5:27
Really? [laughs] Wow. You should see my excitement. Um,
- 5:32
your, the information or the knowledge of that backflip is actually stored in that part of your brain. So I heard it's 90% confidence, by the way. [laughs]
- 5:41
That is actually... [laughs] It is, right? [laughs] I'm not gonna do one, but. But it's stored in that part of your brain. Now, you can actually mimic this in agents, and we're g- I'm gonna show you how.
- 5:53
But now we're talking about agent memory. Agent memory is the mechanisms that we are implementing to actually make sure that states persist in our AI application-
- 6:08
Our agents are able to accumulate information, turn data into memory, and have it inform the next exe- execution step. But the goal is to make them more reliable, believable, and capable.
- 6:22
Those are the key things. And the core topic that we are going to be working on as AI memory engineers is on memory management. We're gonna be building memory management systems.
- 6:37
And memory management is a systematic process of organizing all the information that you're putting into the context window. Yes, we have, like, large context window, but that's not for you to stuff all your data in.
- 6:49
That's for you to pull in the relevant memory and structure them in a way that is effective, that allows for the response, um, to be relevant.
- 6:58
So these are the core components of memory management: generation, storage, retrieval, integration, update, and deletion. There's a lie here, because you, you don't delete memories. Humans don't delete their memories, except if it's traumatic one and you wanna forget.
- 7:14
But we really should be looking at implementing forgetting mechanisms within the memory management systems that we're building. You don't want to delete memories, and there are different research papers that are looking at how to implement some form of forgetting within agents.
- 7:28
But the most important bit is retrieval, and we get into the MongoDB part.
- 7:35
This moving around, um, this is RAG. It's very simple, right? Because we've been doing it as AI engineers. Um, MongoDB is that one database that is core to RAG pipelines, because it gives you all the retrieval mechanisms.
- 7:51
RAG is not just vector. Vector search is not all you need. You need other type of search, and we have that with MongoDB, anything you can think of. You're gonna be hearing a lot about MongoDB in this, um, in this conference today.
- 8:04
But this is what RAG is, and you level up, you go into the world of agentic RAG, right? You give the retrieval capability to the agent as a tool, and now it can choose when to call on information.
- 8:19
There's a lot going on. I'll, I'll send this somehow to you guys, or you can come to me and I'll, um, LinkedIn it to you. Add me on LinkedIn and just ask for the slides and I'll send it to you.
- 8:31
Richmond Alake on LinkedIn. Um, this is memory.
- 8:37
MongoDB is the memory provider for agentic systems, and when you understand that we provide the developer, the AI memory engineer, the AI engineer, all the features that they need to turn data into memory to make the agents believable, capable and reliable, you begin to understand the importance of having a technology partner like MongoDB on your AI
- 9:01
stack. So these are... This is the same, um, image, but just a bit more focused on all the different memories. I'm gonna skip through this slide 'cause I go into a bit of detail.
- 9:13
Um, I'm also gonna give you a library. I'm working on a open source library. I'm ashamed of the name. I was trying to be cool when I came up with it.
- 9:21
It's called Memoriz. [laughs] Um, you can type that on Google, you'll find it, but it has all the design patterns of all of this memory that I'm showing you, all these memory types and that I will show you as well.
- 9:34
But there are different forms of memory in AI agents and how we make them work. So let's start with persona. Is, is anyone here from OpenAI?
- 9:43
Leave. I'm joking. [laughs] Um, well, a couple, a couple months ago, right? So they, they gave ChatGPT a bit of personality, right? Um, and they didn't do a good job. [laughs]
- 9:56
But they are going in the right direction, which is we are trying to make our systems more believable, right? We're trying to make them more human. We're trying to make them create relationship with the consumer, with the users of our systems.
- 10:09
Persona memory helps with that, and you can model that in MongoDB, right? This is Memoriz. You... If you spin up the library, it helps you, um, spin up all of this, um, different type of memory types.
- 10:23
So this is persona. Um, I have a little demo if we have time. Um,
- 10:29
but this is persona memory. This is what it would look like in MongoDB. Then there's toolbox. Um, the guidance from OpenAI is you should only put, uh, the schema of maybe 10 to 21 tools in the context window.
- 10:45
But when you use your database as a toolbox where you're storing the adjacent schema of your tools in MongoDB, you can scale. Because just before you hit the LLM, you can just get the relevant tool use in any form of search.
- 11:01
So that's toolbox. That's mem- that's a toolbox memory, and that's what it would look like, right? You store all... This is how you model it in, uh, MongoDB. You store all the information of your adjacent schema.
- 11:13
Now you're beginning to understand that MongoDB gives you that flexible data model. The document data model is very flexible. It can adapt to wherever data, wherever model you want your data to take, wherever structure, and you have all of the retrieval capabilities, graph, vector, text, geospatial query, in one database.
- 11:33
Conversation memory is a bit obvious, right? Back and forth conversation with, uh, ChatGPT, with Claude. You can store that in your database as well, in MongoDB, as conversational memory, and this is what that would look like.
- 11:45
Timestamp, timestamp, and you have a conversation ID. And you can see something there called recall recency and associate conversation ID, and that's my attempt at implementing some memory signals.
- 11:58
Um, but... And that's goes into the forgetting mechanism that I'm trying to implement in my very famous library Memoriz. Um, I'm gonna go through the next slides a bit quicker because I wanna get to the end of this.
- 12:12
Workflow memory is very important. You build your agentic system, they execute a certain step. Step one, step two, step three, it fails. But one thing you could do is the failure is experience.
- 12:22
It's a learning experience. You can store that in your database. I see you nodding. You're like, "Yeah." Um, you can store that in your database, and you can then pull that in in the next execution to inform the LLM to not take this step or explore other paths.
- 12:36
You can store that in MongoDB as well. You can model that because what you have with MongoDB is that memory provider for your agentic system, and that's what... This is what that looks like when you model it.
- 12:46
An example of it, anyway. So we have episodic memory, we have long-term memory, we have an agent registry. You can store the information of your agent as well. Um, and this is how I do it.
- 12:57
Uh, you can see the agent has tools, persona, all the good stuff. There's entity memory as well. So there's different forms of memory. And the memory re- the Memoris library is very experimental and educational, but it, it encapsulates some of the memory and implementation and design patterns that I'm thinking of on an everyday basis, that we're thinking
- 13:18
of in MongoDB. So MongoDB, you probably get the point now, the memory provider for agentic systems. There are tools out there that focus on memory management. Um, MemGPT, Memzero, Zap, they're great tools.
- 13:33
But after speaking to some of you folks and some of our partners and customers here, there is not, there is, there is not one way to solve memory, and you need a memory provider to build your custom solution to make sure the memory management systems that you're able to implement are effective.
- 13:53
So we really understand the importance of managing data or managing memory, and that's why earlier this year, we acquired Voyage AI. Now they create the best,
- 14:06
no offense, over there, embedding models in the market today. Voyage AI embedding models are... We have, uh, text, multi-modal, we have re-rankers, and this allows you to really solve the problem or at least reduce AI hallucination within your RAG and agentic systems.
- 14:25
And what we're doing, and what we're focused on, the mission for MongoDB, is to make the developer more productive by taking away the considerations and all the concerns around managing different data and all the process of chunking retrieval strategies.
- 14:41
We, we put that into the database. We are redefining the database. And that's why in a few months, we're gonna be pulling in Voyage AI, the embedding models and the re-rankers into MongoDB Atlas, and you will not have to be writing chunking strategies for your, um, for your data.
- 15:00
I see a lot of people nodding, yeah. That's good. So MongoDB is a, i- is a household name, to be honest. We've, um... I watched MongoDB IPO back when I, back when I was in university.
- 15:11
I bought the stocks when I was in university, um, free, just free. I only had about £100. Um, I was broke. But
- 15:21
we are very focused, and we take it very seriously making sure that you guys can build the best AI products, AI features very quickly in a secure way. So MongoDB is built for the change that we are gonna experience now, tomorrow, in the next couple years.
- 15:37
I wanna end with this. You know who these two guys are?
- 15:41
Damn. Okay. This is Hubel and Wiesel. They won a Nobel Prize, um, in the late '90s, but they did some research on the visual cortex of cats. Um, they experimented with cats.
- 15:53
They, that... This probably wouldn't fly now, but back in the '50s and '60s, things were a bit more relaxed. But they found out that the visual cortex of the brains between cats and humans actually worked by learning different hierarchies of representation, so edges, contours, and abstract shapes.
- 16:11
Now, people that are in deep learning will know that this is how convolu- convolution neural network works. And the research that these guy- these guys did inspired and informed convolutional neural networks.
- 16:24
That's face detection, object detection. It's, it all comes from neuroscience. So we are architects of intelligence, but there is a better architect of intelligence, it's nature. Nature's created our brains.
- 16:37
It's the most effective form of intelligence and, well, some humans that I meet, but it's the most effective form of intelligence that we have today, and we can look inwards to build this agentic system.
- 16:47
So last week, Saturday, myself and Tengyu, he's the chief AI scientist at MongoDB, also the founder of Voyage AI, we sat with these three guys in the middle, are neuroscientists.
- 16:59
Kenneth has been exploring human brain and memory for over twenty years. And, and over here is Charles Parker. He's the creator of MemGPT or Letta. And we are having this conversation.
- 17:11
And once again, we're mirroring how we're bringing neuroscientists and application developers together to solve and push us on the path of AGI. So that's my talk done. Check out Memoris, and you can come talk to me about memory.
- 17:27
Add me on LinkedIn if you want this presentation. Thank you for your time. [upbeat music]