Stop Using RAG as Memory

Talk transcript

1. 0:00

   [upbeat music] I'm here today to tell you that there's w- no one-size-fits-all memory. [coughs]

2. 0:20

   Um, and why you need to model your memory after your business domain. So if you saw me a little bit earlier and I was talking about Graphiti, Zep's open source temporal graph framework,

3. 0:36

   um, you might have seen me just speak to how you can build custom entities and edges in the Graphiti graph for your particular business domain. So business objects from your business domain. [coughs]

4. 0:51

   What I'm gonna demo today is actually how Zep implements that and how e- easy it is to use from Python, TypeScript or Go.

5. 1:00

   And what we've done here is we've solved a fundamental problem plaguing memory,

6. 1:05

   and we're enabling developers to build out memory that is far more cogent and capable for many different use cases. So I'm gonna just show you a quick example of

7. 1:22

   where things go really wrong. So many of you might have used ChatGPT before. It generates facts about you in memory, and you might have noticed that it really struggles with relevance.

8. 1:34

   Sometimes it just pulls out all sorts of arbitrary facts about you. And unfortunately, when you store arbitrary facts and retrieve them as memory, you get inaccurate responses or hallucinations.

9. 1:47

   And the same problem happens when you're building your own agents. So here we go. We have an ex-example media assistant, and it should remember things about jazz music, NPR podcasts, The Daily, et cetera.

10. 2:01

    All the things that I like to listen to. But unfortunately, because I'm in conversation with the agent, or it's picking up my voice when I'm, you know, it's a voice agent, um, it's learning all sorts of irrelevant things.

11. 2:13

    Like, I wake up at seven AM, my dog's name is Melody, et cetera. And the point here is that irrelevant facts pollute memory. They're not specific to the media player business domain.

12. 2:28

    And so the technical reality here is as well that many frameworks take this really simplistic approach, approach to generating facts. If you're using a framework that has memory capabilities, agent framework, it's generating facts and throwing it into a vector database.

13. 2:45

    And unfortunately, the facts dumped into the vector database or Redis mean that when you're recalling that memory, it's difficult to differentiate what should be returned. We're gonna return what is semantically similar.

14. 2:58

    And here we have, um, a bunch of facts that are semantically similar to my request for my favorite tunes. Um, we have some good things, and unfortunately, Melody is there as well because Melody is a dog named Melody, and that might be something to do with tunes.

15. 3:16

    Um, and so bl- bunch of irrelevant stuff.

16. 3:23

    So basically, semantic similarity is not business relevance,

17. 3:29

    and this is not un-unexpected. I was speaking a little bit earlier about how vectors and are just basically projections into an embedding space. There's no causal or relational, uh, relations between them.

18. 3:44

    And so we need a solution. We need domain-aware memory, not better semantic search.

19. 3:52

    So with that, I am going to unfortunately be showing you a video because the wifi has been absolutely terrible. Um- [laughs]

20. 4:04

    And let me bring up the video. [clears throat] Okay. So

21. 4:13

    I built a little application here, and it is a finance coach. And I've told it I wanna buy a house. [clears throat]

22. 4:23

    And it's asking me, well, how much do I earn a year? It's asking me about what student loan debt I might have. And we'll see that on the right-hand side, what is stored in Zep's memory are some very explicit c-

23. 4:44

    business objects. We have financial goals, debts, income sources, et cetera. These are defined by the developer, and they're defined in a way which is really simple to understand.

24. 4:59

    We can use Pydantic or Zod or Go structs, and we can apply business rules. So let's go take a look at some of the code here. We have a TypeScript financial goal schema using Zep's underlying SDK.

25. 5:15

    We can define these entity types. We can give a description to the entity type. Uh, we can even define fields, the business rules for those fields, so the values that they take on.

26. 5:27

    And then we can bu-build tools for our agent to retrieve a financial snapshot, which runs multiple Zep searches at the same time concurrently and filters by specific node types.

27. 5:43

    And when we start our Zep application, what we're gonna do is we're gonna register these particular goals, uh, sorry, objects, with, uh, Zep, so it knows to build this ontology in the graph.

28. 5:58

    So let's do a quick little addition here.

29. 6:03

    I'm gonna say that I have five thousand dollar a month rent.

30. 6:08

    I think it's rent. And in a few seconds, we see that Zep's already parsed that new message and has captured that five thousand dollars. And we can go look at the chart, the graph.

31. 6:21

    This is the, the Zep front end. And we can see the knowledge graph for this user has got a debt account entity. It's got fields on it, um, that we've defined as a developer.

32. 6:35

    And so again, we can really get really tight about what we retrieve from Zep by filtering. Okay, so we're at time. So just very quickly, we wrote a paper about how this, all of this works.

33. 6:47

    You can get to it, uh, by that link below, and appreciate your time today.

34. 6:54

    You can look me up afterwards. [upbeat music]