Architecting Agent Memory: Principles, Patterns, and Best Practices

Talk transcript

1. 0:00

   [upbeat music] In the next ten to fifteen minutes, here's, uh, I guess, my promise to you.

2. 0:20

   I'm going to give you some information that will be high level. There will be some practical component to it, but

3. 0:29

   this information I'll give you within the next six month will be very relevant, and it will put you in the best position to build the best AI applications, to build the best agents that are believable, capable, and reliable.

4. 0:42

   Nothing on this screen. I know. [laughs] We, we, we gon' get there. [laughs]

5. 0:48

   You know what? Just for you. Thank you. There we go. You're welcome. So we're gonna be talking about memory. Um,

6. 0:56

   we're gonna be talking about the stateless applications that we're bui- building today, and how we can make them stateful. We're gonna be talking about the prompt engineering that we're doing today, and how we can reduce that by focusing on persistence.

7. 1:11

   We're gonna be turning the responses in our AI application and making our agents build relationship with our customers, and all of it is gonna be centered around memory.

8. 1:24

   So I'm gonna do a very quick evolution of what we've been seeing for the past two to three years.

9. 1:33

   We started off with chatbots, LLM-powered chatbots. They were great. ChatGPT came out November 2022, and yeah, exploded. Then we went into RAG. We gave these chatbots more domain-specific relevant knowledge, and they gave us more personalized responses.

10. 1:50

    Then we begin to scale the compute, the data we're giving to the LLMs, and they gave us emerging capabilities, right? Reasoning, uh, tool use. Now we're in the world of AI agents and agentic systems,

11. 2:04

    and the big debate is what is an agent, right? What is an AI agent? I don't like to go into that debate because that's like asking what is consciousness.

12. 2:14

    Um, it's a spectrum. The agenticity, and that's a word now, agenticity- [laughs]

13. 2:21

    ... of, uh, of an agent is it is a spectrum, so there are different levels.

14. 2:28

    I came here and I saw Waymo, and to me it was pure sorcery. We don't have that in the UK, and there are different levels of, um, self-driving, so you can look at the agentic spectrum in that respect.

15. 2:38

    We have a minimal agent, where it's an LLM running a loop. Great. Then you have, uh, level four is autonomous agent. A bunch of agents that have access to tools.

16. 2:48

    You ... They can do whatever they want. They're not prompted in any way or a minimal way. But this is how I see things, as a spectrum. So what is an AI agent?

17. 2:58

    It's a computational entity with awareness of its environment through perception, cognitive abilities through an LLM, and also can take action through tool use. But the most important bit is there is some form of memory, short term or long term.

18. 3:15

    Memory is important. It's important because we're trying to make our agents reflective, interactive, proactive, and reactive, and autonomous, and every, most of this, if not all, can be solved with memory.

19. 3:30

    Um, I work at MongoDB, and we're gonna make ... we're gonna connect the dots. Don't worry. So this is all nice and good. This is what, what you look at if you, um, double-click into one AI agent is...

20. 3:41

    But the most important bit to me is... I- I'll go to slide. People are taking pictures, sorry.

21. 3:48

    All right, let's go. The most important bit is memory, and when we talk about memory, the, the easy way you can think about it is short-term, long-term, but there are als- there are other distinct forms, right?

22. 3:56

    Um, conversational, entity memory, knowledge, data store, cache, working memory. We're gonna be talking about all of that today. So these are the high level concepts.

23. 4:05

    But let me go a little bit meta.

24. 4:09

    Why we're all here, um, today in this conference is because of AI, right? We're all architects of intelligence. The whole point of AI is to build some form of computational entity that surpasses hu- human intelligence or mimics it.

25. 4:23

    Then AGI, we're focused on making that intelligence surpass humans in all tasks we can think of. And if you think about the most intelligent humans you know,

26. 4:36

    what determines their intelligence is their ability to recall. It's their memory. So if we, if AI or AGI is meant to mimic human intelligence, it's a no-brainer, no pun intended, that we need memory within the agents that we're building today.

27. 4:50

    Does anyone disagree? Good. I would have kicked you out. [laughs]

28. 4:56

    Um, okay, let's go. So humans, you, in your brain right now, you have these, you have this. This is not what it looks like, but it's close enough. You have different forms of memory, and that's what makes you intelligent.

29. 5:06

    That's what makes you retain some of the information I'm gonna be giving you today. There is short-term, long-term, working memory, semantic, episodic, procedural memory. Um, in your brain right now, there is something called the cerebellum.

30. 5:19

    I always get the word wrong, but that's where you store most of the routines and skills you can do. Can anyone here do a backflip?

31. 5:27

    Really? [laughs] Wow. You should see my excitement. Um,

32. 5:32

    your, the information or the knowledge of that backflip is actually stored in that part of your brain. So I heard it's 90% confidence, by the way. [laughs]

33. 5:41

    That is actually... [laughs] It is, right? [laughs] I'm not gonna do one, but. But it's stored in that part of your brain. Now, you can actually mimic this in agents, and we're g- I'm gonna show you how.

34. 5:53

    But now we're talking about agent memory. Agent memory is the mechanisms that we are implementing to actually make sure that states persist in our AI application-

35. 6:08

    Our agents are able to accumulate information, turn data into memory, and have it inform the next exe- execution step. But the goal is to make them more reliable, believable, and capable.

36. 6:22

    Those are the key things. And the core topic that we are going to be working on as AI memory engineers is on memory management. We're gonna be building memory management systems.

37. 6:37

    And memory management is a systematic process of organizing all the information that you're putting into the context window. Yes, we have, like, large context window, but that's not for you to stuff all your data in.

38. 6:49

    That's for you to pull in the relevant memory and structure them in a way that is effective, that allows for the response, um, to be relevant.

39. 6:58

    So these are the core components of memory management: generation, storage, retrieval, integration, update, and deletion. There's a lie here, because you, you don't delete memories. Humans don't delete their memories, except if it's traumatic one and you wanna forget.

40. 7:14

    But we really should be looking at implementing forgetting mechanisms within the memory management systems that we're building. You don't want to delete memories, and there are different research papers that are looking at how to implement some form of forgetting within agents.

41. 7:28

    But the most important bit is retrieval, and we get into the MongoDB part.

42. 7:35

    This moving around, um, this is RAG. It's very simple, right? Because we've been doing it as AI engineers. Um, MongoDB is that one database that is core to RAG pipelines, because it gives you all the retrieval mechanisms.

43. 7:51

    RAG is not just vector. Vector search is not all you need. You need other type of search, and we have that with MongoDB, anything you can think of. You're gonna be hearing a lot about MongoDB in this, um, in this conference today.

44. 8:04

    But this is what RAG is, and you level up, you go into the world of agentic RAG, right? You give the retrieval capability to the agent as a tool, and now it can choose when to call on information.

45. 8:19

    There's a lot going on. I'll, I'll send this somehow to you guys, or you can come to me and I'll, um, LinkedIn it to you. Add me on LinkedIn and just ask for the slides and I'll send it to you.

46. 8:31

    Richmond Alake on LinkedIn. Um, this is memory.

47. 8:37

    MongoDB is the memory provider for agentic systems, and when you understand that we provide the developer, the AI memory engineer, the AI engineer, all the features that they need to turn data into memory to make the agents believable, capable and reliable, you begin to understand the importance of having a technology partner like MongoDB on your AI

48. 9:01

    stack. So these are... This is the same, um, image, but just a bit more focused on all the different memories. I'm gonna skip through this slide 'cause I go into a bit of detail.

49. 9:13

    Um, I'm also gonna give you a library. I'm working on a open source library. I'm ashamed of the name. I was trying to be cool when I came up with it.

50. 9:21

    It's called Memoriz. [laughs] Um, you can type that on Google, you'll find it, but it has all the design patterns of all of this memory that I'm showing you, all these memory types and that I will show you as well.

51. 9:34

    But there are different forms of memory in AI agents and how we make them work. So let's start with persona. Is, is anyone here from OpenAI?

52. 9:43

    Leave. I'm joking. [laughs] Um, well, a couple, a couple months ago, right? So they, they gave ChatGPT a bit of personality, right? Um, and they didn't do a good job. [laughs]

53. 9:56

    But they are going in the right direction, which is we are trying to make our systems more believable, right? We're trying to make them more human. We're trying to make them create relationship with the consumer, with the users of our systems.

54. 10:09

    Persona memory helps with that, and you can model that in MongoDB, right? This is Memoriz. You... If you spin up the library, it helps you, um, spin up all of this, um, different type of memory types.

55. 10:23

    So this is persona. Um, I have a little demo if we have time. Um,

56. 10:29

    but this is persona memory. This is what it would look like in MongoDB. Then there's toolbox. Um, the guidance from OpenAI is you should only put, uh, the schema of maybe 10 to 21 tools in the context window.

57. 10:45

    But when you use your database as a toolbox where you're storing the adjacent schema of your tools in MongoDB, you can scale. Because just before you hit the LLM, you can just get the relevant tool use in any form of search.

58. 11:01

    So that's toolbox. That's mem- that's a toolbox memory, and that's what it would look like, right? You store all... This is how you model it in, uh, MongoDB. You store all the information of your adjacent schema.

59. 11:13

    Now you're beginning to understand that MongoDB gives you that flexible data model. The document data model is very flexible. It can adapt to wherever data, wherever model you want your data to take, wherever structure, and you have all of the retrieval capabilities, graph, vector, text, geospatial query, in one database.

60. 11:33

    Conversation memory is a bit obvious, right? Back and forth conversation with, uh, ChatGPT, with Claude. You can store that in your database as well, in MongoDB, as conversational memory, and this is what that would look like.

61. 11:45

    Timestamp, timestamp, and you have a conversation ID. And you can see something there called recall recency and associate conversation ID, and that's my attempt at implementing some memory signals.

62. 11:58

    Um, but... And that's goes into the forgetting mechanism that I'm trying to implement in my very famous library Memoriz. Um, I'm gonna go through the next slides a bit quicker because I wanna get to the end of this.

63. 12:12

    Workflow memory is very important. You build your agentic system, they execute a certain step. Step one, step two, step three, it fails. But one thing you could do is the failure is experience.

64. 12:22

    It's a learning experience. You can store that in your database. I see you nodding. You're like, "Yeah." Um, you can store that in your database, and you can then pull that in in the next execution to inform the LLM to not take this step or explore other paths.

65. 12:36

    You can store that in MongoDB as well. You can model that because what you have with MongoDB is that memory provider for your agentic system, and that's what... This is what that looks like when you model it.

66. 12:46

    An example of it, anyway. So we have episodic memory, we have long-term memory, we have an agent registry. You can store the information of your agent as well. Um, and this is how I do it.

67. 12:57

    Uh, you can see the agent has tools, persona, all the good stuff. There's entity memory as well. So there's different forms of memory. And the memory re- the Memoris library is very experimental and educational, but it, it encapsulates some of the memory and implementation and design patterns that I'm thinking of on an everyday basis, that we're thinking

68. 13:18

    of in MongoDB. So MongoDB, you probably get the point now, the memory provider for agentic systems. There are tools out there that focus on memory management. Um, MemGPT, Memzero, Zap, they're great tools.

69. 13:33

    But after speaking to some of you folks and some of our partners and customers here, there is not, there is, there is not one way to solve memory, and you need a memory provider to build your custom solution to make sure the memory management systems that you're able to implement are effective.

70. 13:53

    So we really understand the importance of managing data or managing memory, and that's why earlier this year, we acquired Voyage AI. Now they create the best,

71. 14:06

    no offense, over there, embedding models in the market today. Voyage AI embedding models are... We have, uh, text, multi-modal, we have re-rankers, and this allows you to really solve the problem or at least reduce AI hallucination within your RAG and agentic systems.

72. 14:25

    And what we're doing, and what we're focused on, the mission for MongoDB, is to make the developer more productive by taking away the considerations and all the concerns around managing different data and all the process of chunking retrieval strategies.

73. 14:41

    We, we put that into the database. We are redefining the database. And that's why in a few months, we're gonna be pulling in Voyage AI, the embedding models and the re-rankers into MongoDB Atlas, and you will not have to be writing chunking strategies for your, um, for your data.

74. 15:00

    I see a lot of people nodding, yeah. That's good. So MongoDB is a, i- is a household name, to be honest. We've, um... I watched MongoDB IPO back when I, back when I was in university.

75. 15:11

    I bought the stocks when I was in university, um, free, just free. I only had about £100. Um, I was broke. But

76. 15:21

    we are very focused, and we take it very seriously making sure that you guys can build the best AI products, AI features very quickly in a secure way. So MongoDB is built for the change that we are gonna experience now, tomorrow, in the next couple years.

77. 15:37

    I wanna end with this. You know who these two guys are?

78. 15:41

    Damn. Okay. This is Hubel and Wiesel. They won a Nobel Prize, um, in the late '90s, but they did some research on the visual cortex of cats. Um, they experimented with cats.

79. 15:53

    They, that... This probably wouldn't fly now, but back in the '50s and '60s, things were a bit more relaxed. But they found out that the visual cortex of the brains between cats and humans actually worked by learning different hierarchies of representation, so edges, contours, and abstract shapes.

80. 16:11

    Now, people that are in deep learning will know that this is how convolu- convolution neural network works. And the research that these guy- these guys did inspired and informed convolutional neural networks.

81. 16:24

    That's face detection, object detection. It's, it all comes from neuroscience. So we are architects of intelligence, but there is a better architect of intelligence, it's nature. Nature's created our brains.

82. 16:37

    It's the most effective form of intelligence and, well, some humans that I meet, but it's the most effective form of intelligence that we have today, and we can look inwards to build this agentic system.

83. 16:47

    So last week, Saturday, myself and Tengyu, he's the chief AI scientist at MongoDB, also the founder of Voyage AI, we sat with these three guys in the middle, are neuroscientists.

84. 16:59

    Kenneth has been exploring human brain and memory for over twenty years. And, and over here is Charles Parker. He's the creator of MemGPT or Letta. And we are having this conversation.

85. 17:11

    And once again, we're mirroring how we're bringing neuroscientists and application developers together to solve and push us on the path of AGI. So that's my talk done. Check out Memoris, and you can come talk to me about memory.

86. 17:27

    Add me on LinkedIn if you want this presentation. Thank you for your time. [upbeat music]