Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications

Talk transcript

1. 0:00

   [upbeat music] So thank you everyone. We have a, a very full room today.

2. 0:19

   Um, so we'll get started with the repo here in a second. There is a QR code here, and then in a couple other slides that all link to the same place.

3. 0:26

   We'll be using a, uh, Python notebook. Um, and we'll be doing GraphRAG with some real data using [REDACTED:username]. So it'll be myself, um, I'm Zach, and then we have, uh, Andreas or ABK in the back who will be helping.

4. 0:40

   And then we have Michael Hunger, um, somewhere else in the room who can help, um, answer questions.

5. 0:48

   And Tomasz.

6. 0:49

   Um, and-

7. 0:49

   Tomasz

8. 0:49

   ... and Tomasz as well. Hello, Tomasz, um, back there in the beige who can help answer questions as well as we go through. Um, so, um, this is an interactive secti- session.

9. 1:00

   We have only two hours, so we're asking to save the big questions to the end. But basically, we'll be hopping back and forth between the slides, um, and then between hands-on work inside of a notebook.

10. 1:10

    Um, so if you do get stuck with anything, raise your hand. Um, we'll come over and try to help you through, and try to stick to our schedule as well with around 10 to 15 minutes, um, per each section, and there's about five sections for us to walk through.

11. 1:23

    So before we start, there's two things we want to do if you haven't done them already. The first thing is we wanna create a blank [REDACTED:username] Graph Data Science Sandbox.

12. 1:35

    This is gonna be at sandbox.[REDACTED:username].com. So basically what you're gonna do, if I go ahead and click this link, um, you might have to create an account really quickly on this page.

13. 1:46

    Um, and then you're gonna click For Data Science here, and then it's gonna be this, if you can see, um, this blank graph data science thing here. So you're gonna click on that blank sandbox graph data science, um, and then you're...

14. 2:01

    that'll create an instance for you. Um, and it should come up with a username and password that you're gonna use inside of the notebook. So this will be the database that we'll use throughout the course.

15. 2:12

    Um, the next thing that you're going to do is go to the notebook. Um, so the notebook is going to be in this link. Um, and also, by the way, this QR code.

16. 2:21

    So I'll go back to that in just a second if you missed it. Um, but before that, I just wanna show you this GitHub repository, um, and it's GenAI Workshop is the notebook.

17. 2:31

    So what you're gonna wanna do is you're gonna wanna go to this notebook, and then you're gonna wanna click Open in Colab. Um, this will require a Google account.

18. 2:40

    If you don't have a Google account, um, you can use VS Code or you can run it locally. That's fine. Um, but then when you go in, if you are using Google Colab, um, the Internet's a little bit slow, so it might take a while, um, for it to load here.

19. 2:55

    Um, hopefully once everyone's get settled, it will go a little bit faster. Um, but basically what will happen is you just wanna go to the top menu, um, and you're basically gonna go, like, File, um, and you're gonna Save a Copy in Drive, um, just so that you have the ability to edit it.

20. 3:13

    Um, so you go File, and then you say Save a Copy in Drive, and then it will create another copy for you.

21. 3:21

    So I'm gonna go back to the slide, and I'll, um, give everyone, um, a little while to get that all set up. Um, actually, too, while we're waiting on that-

22. 3:33

    15 people can still sit in the front if you don't want to stand in the back. So feel free.

23. 3:37

    Yes. Um, it might be a good idea for you to start running the notebook cells, so, um, especially the first cell, the pip install, um, just 'cause that might take a little while.

24. 3:50

    Um, if... When you just start, that might... It's probably the longest running cell in the whole notebook. It's just loading all the requirements for today. Um, so to say again, create your sandbox, uh, blank sandbox at, um, blank sandbox, uh, GDS, graph data science, and then you're going to open that notebook, create a copy of it or

25. 4:13

    have it locally, and then start running that, uh, pip install, uh, cell right here.

26. 4:21

    Um, so what happens... I, I did the blank sandbox instead of the blank, uh,

27. 4:28

    data science sandbox. I did the blank sandbox instead of the blank data science sandbox.

28. 4:37

    That one will maybe work, but what I'd recommend doing is just terminating that instance. So if you go to the, um... If you go to the screen, once it's there, just, uh, press Terminate.

29. 4:49

    There's, like, a garbage can icon.

30. 4:51

    Oh.

31. 4:51

    And then go and create a graph, uh, data science sandbox.

32. 4:55

    Okay. Thank you.

33. 4:55

    Yeah, blank. It's blank sandbox-graph datascience is the one we want.

34. 5:01

    Thank you.

35. 5:02

    So the main difference is actually that the graph data science sandbox has a little bit more of memory. So as you're doing some more involved operations, it's just faster to complete.

36. 5:11

    So the, the-

37. 5:13

    Yes

38. 5:13

    ... regular blank one should still work, but it will take longer. So the graph data science one will just be faster. So it runs with a little bit more memory.

39. 5:20

    Yes. It, it says blank, so it's-

40. 5:23

    Yeah, I know. It's... We need to work on that. It's a little bit tricky.

41. 5:28

    It's

42. 5:28

    All righty. So, um, I'll leave this here. Is there... Can I get a show of hands of how many people have the notebook set up?

43. 5:39

    Okay. So we're pretty good. Um, is there, is there anyone who still needs to stay on this PR code?

44. 5:49

    No.

45. 5:50

    No? Okay. We'll go ahead and move on Um, so I'm gonna do, like, my general spiel that I normally do. I think for a lot of us, this is gonna be review.

46. 5:59

    Um, we're gonna be talking a lot about retrieval-augmented generation today. Um, and for those who are, you know, unfamiliar, right, retrieval-augmented generation is just when you have this step in the middle when you're communicating with a large language model, where instead of sort of like in a ChatGPT-like way, where you're just sending a, a question and getting

47. 6:18

    a response back, um, you're going to have that, uh, LLM application go and pull relevant data through some query, um, that it can then use to reduce hallucinations, provide more domain-specific context, um, enable better traceability, et cetera.

48. 6:34

    And when we talk about graph-powered retrieval-augmented generation, and we'll use the word GraphRAG kind of throughout this course, um, is we're talking about the combination of three things that we're gonna look at today.

49. 6:46

    So that's gonna be vector search, which I'm sure a lot of you are already fairly, fairly familiar with; knowledge graph traversals, which is gonna be traversing over structured data in addition to the vector search; and then we're gonna do some cool things with graph data science.

50. 7:00

    So we're actually gonna create a new type of embedding, a graph embedding, and we're gonna use that, that vector to do some very interesting things for recommendations with structured data.

51. 7:11

    Um, and today, so, um, everyone does chatbots, which chatbots are wonderful. I wanted to do something a little bit different today. Um, so we're actually gonna do an example of using a large language model to construct an email, um, sort of like a recommendation, recommendation type of email.

52. 7:29

    Um, and this is going to be for clothing. So imagine, um, someone's looking for something like a halter neck top, right? How do you choose what to buy? Well, you need a list of things that are available in your online store.

53. 7:42

    You need to match that specifically for that specific customer, right? Which different customers might have different preferences. Um, and then you might wanna have other recommendations to pair with that item, like maybe a pair of pants, um, or, or a hat or other accessories and things of that nature.

54. 7:59

    Um, and so what we're gonna do today is sort of build an AI fashion assistant, so to speak, to help scale this. Um, so we're gonna use real-world data from a Kaggle competition from H&M, um, and we're gonna do some search and retrieval.

55. 8:14

    We're gonna add some context for personalization, a small recommendation engine, and we're gonna wrap that into sort of an LLM chain inside of LangChain to show you how to generate that response.

56. 8:24

    Um, and the idea behind this, right, is again, we're combining sort of this knowledge graph, which provides this context and this enrichment with the large language model, giving that reasoning and that creativity and sort of the-- We're giving some liberty to the large language model to kind of take what's in the knowledge graph based on the customer's

57. 8:42

    purchase behavior and other textual data and vectors, um, and, and kind of say, "Use your creativity to put something together for this customer," basically. Um, and while we're doing sort of a fashion recommendation thing here, this same process could work for different types of support agents, um, or, or a lot of different types of internal search and

58. 9:01

    customer experience use cases. Um, and this is what we're gonna build today. Um-

59. 9:08

    Will you be sharing the deck? What's that? Will you be sharing the deck? Deck is in the GitHub repo. I need to-- It's not the most-- It's not exactly the one that I have here, but I'll, I'll update it tonight.

60. 9:18

    The PDF? PDF. The workshop slides. PD-- A p- Workshop slides. The, the PDF of the workshop slides is in the repo. Yep. Yep, yep.

61. 9:26

    Um, so this is what we're gonna build today. It's a little Gradio app, um, where basically we're gonna enter in some customer information, their ID, a time of year, um, and some of the things they're interested in, and we're gonna generate an email that provides, uh, various recommendations to that customer.

62. 9:43

    Um, so imagine you're trying to create targeted marketing content or whatever it may be. This is sort of an example of how to do that. Um, and these are the specific technologies we'll be using.

63. 9:53

    Um, so we're gonna use OpenAI for the language model. We could, of course, substitute that out, um, with Vertex AI or Gemini or, or whatever we wanted to there.

64. 10:02

    Um, we have LangChain that we're using to orchestrate everything, so that's gonna be creating the vector stores and also the large language model chain. Um, we're using Colab, obviously, and then Gradio just to kind of show you an example of a very simple UI that you can put on top of that, um, just for demonstration purposes.

65. 10:21

    Um, and this is gonna be how we're splitting the course up. Um, so basically, there's this journey that we're gonna follow. We're gonna start with building our graph. That's gonna be built inside of that sandbox instance that you just made.

66. 10:34

    Um, after that, we're going to be showing you how to do some very simple vector search. Then we're going to, um, explore how to integrate knowledge graph traversal patterns into that to better personalize the retrieval data, so not just based on, um, sort of text similarity, but also on, um, similarity based on co-purchasing behavior.

67. 10:56

    Um, and then when we get to recommendations, we're actually gonna use an unsupervised machine learning model to generate graph embeddings. Um, and we're going to use, um, k-nearest neighbor there to locate other recommendations that we can make inside of the graph.

68. 11:09

    And then we're gonna put that all together inside of, um, an LLM chain to show you how you can use that to create a better, more curated response with the language model.

69. 11:20

    All righty, so that was a lot. Um, I'll go ahead and get started with graph building.

70. 11:27

    Um, again, the way this will work is I'll spend a little bit of time in slides just talking about what we're gonna do. Then we're gonna hop over to the notebook, and we-- we're gonna run through those cells.

71. 11:38

    You, you might already be running through those cells, um, 'cause I know a lot of us, we, um-- It's gonna be pretty easy to just sort of click through.

72. 11:46

    Um, there's not actual-- a lot of hands-on coding you need to do, um, but we're gonna walk through that anyway. So- Um, this is the data model that we're gonna use.

73. 11:56

    A reminder, right, that we're working inside of a graph database, um, which means that everything that we're working with is natively represented as nodes and relationships inside of a graph.

74. 12:06

    So we have these entities. Um, we have customers and articles and then products. So basically, what this graph is telling us is, um, a sample of customers that exists, um, different purchases that they made of articles of clothing.

75. 12:20

    So think of these like the halter tops, the jeans, the boots, you know, all the accessories. A variant of, um... So an article is a variant of a product in the sense that you can have a certain style of jeans, but they can be, you know, have different color patterns, different sizes, et cetera.

76. 12:37

    So the product is sort of like the product, and then the article is like the variant of that product. Um, we also have, um, each department that the article is sold in.

77. 12:48

    Um, we're not gonna use that department node very much, though, um, in this specific workshop.

78. 12:55

    Um, and of course, all of these things can have, um, properties associated with them. So we have, for example, the customer ID to uniquely identify a customer. For articles, we have product, uh, we have color and different, uh, design patterns.

79. 13:09

    And then for the product, we have sort of the description of the product, the garment group that is in, the type of the product, and the name of the product, et cetera.

80. 13:19

    Um, so again, um, I think we've, we've all done this step hopefully by now. Uh, so I will go ahead and, um,

81. 13:29

    enter into our notebook. So hopefully everyone can see this. Um, this was the first cell that we wanted to run before just 'cause it takes a while. If you haven't run the, uh, first cell for pip install, please go ahead and do that now.

82. 13:45

    Once you've run that, um, go ahead and just kinda keep going down here, run the, um, import. Um, this just gets us all the libraries we need. And now here's the part where we're going to have to, um, inject some things here.

83. 13:59

    So when you made that sandbox instance-- I'll actually go to my-- I have to log out and go to a different account here.

84. 14:10

    Um, you should get connection details. So if I go and I just click this and I go to connection details. So please don't use mine so that we're not fighting on the same database. [laughing]

85. 14:27

    Um, but copy, copy the Bolt URL here, um, and you're gonna go back into your, uh, workshop code, and you're gonna put that here for your [REDACTED:username] URI. It's gonna be that Bolt connection.

86. 14:39

    The password as well is gonna be, um, is gonna be this one right here, the password. And then you're gonna keep the username as [REDACTED:username], so you don't have to change either of these two.

87. 14:52

    We're gonna keep the LLM as GPT-4o. Um, and then for the OpenAI key, um, if you have your own, please, please do use that. If you don't, um, if you click on this link, uh, it'll take you to a workshop key that we'll keep live for a little bit that you can use, um, to run through the

88. 15:11

    workshop. And so basically you're just gonna copy it in, kind of like I've done here, um, and you're gonna give that a run. And then this next cell is just if you wanted to do it the proper way and have a separate environments file, um, which you can skip, or you can do that if you want, um,

89. 15:27

    uh, but that, uh, is not necessary. So once you've run that, we get down to the knowledge graph building section, and we've already talked about the graph we're gonna import.

90. 15:37

    And basically what you're going to do is you're gonna connect to [REDACTED:username]. Um, so this is our graph data science client. We have multiple different drivers and, um, different wrappers around those drivers to connect to the database.

91. 15:51

    I'm using graph data science for this just to simplify things. It's, um, basically you're just gonna hand it the URI, the username, and the password, um, and that'll get you connected.

92. 16:00

    And there's some warnings there that we're gonna get rid of. Um, and then if you can run this cell successfully right after it, this GDS debug, um, that means you're connected and you're good to go.

93. 16:11

    So you're connected to a blank database. Once that is run, um, there's one cell here that I have for pretty much loading ninety percent of the data. So you're gonna go ahead and run this.

94. 16:25

    Um, what this is doing is I've gone ahead and there's another notebook inside of this repository that's prepared these CSVs. So there's, uh, five CSVs here. Um, there's department, product, article, uh, customer, and transaction.

95. 16:41

    Um, and so most of these are self-explanatory. The department is the department node. The product is the product node. Article is the article node, et cetera. Customer the, uh, the customer node.

96. 16:50

    And then transactions are all the purchase transactions between customers and articles. Um, so the-- If you were to dig into this clo-- code, and because we only have two hours, I mean, we can spend all our time talking about this, but I decided, uh, not to so we can, we can get to more of the vector stuff.

97. 17:05

    Um, this is gonna create uniqueness constraints, which are gonna be useful for querying later, um, in [REDACTED:username], so it's just, uh, a general best practice. Um, and then if I go, there's basically two functions here, one for loading the nodes and the next for the relationships.

98. 17:21

    Um, if you were doing this sort of in a non-automated setting, um, this is kind of what the Cypher looks like. So just like, um...

99. 17:34

    So just like in SQL how you, or for RDBMS, you have SQL for graphs, um, and for [REDACTED:username] specifically, you have Cypher. So what this code is doing that I have above is basically for loading the different nodes.

100. 17:47

     It's generating this Cypher. So you see it says, "Use this Cypher query, unwind merge set." So That's the actual code behind the scenes, right? The Cypher code that's being run to load this data.

101. 17:58

     Um, so if you are interested in how to load more data into [REDACTED:username], you can take a look at that. Um, and we have other GraphAcademy resources as well if, if you wanted to dig into that piece, um, in more detail around just how to model data and how to load data into a graph.

102. 18:15

     Um, and then we'll stop there, and I think we're at fifteen minutes now, so we can probably do, um, I'd say five to ten minutes, and we can wait for people to catch up.

103. 18:32

     As a quick show of hands, how many people have made it through the loading?

104. 18:37

     Awesome. Okay. That's really good. Um, and then Michael and Tomasz- [background chatter]

105. 19:02

     Oh, so do you have-- Yeah, if you go to this tab here, it should-- So instead of this, um...

106. 19:15

     Yeah. So it should-- Let's see. Actually, so do you-- You've got the credentials as well. Yeah. Okay. So instead of, uh, single sign-on, there's username, password. Just add the password over there.

107. 19:32

     It says we're leaving in- So this is like the built-in Cypher query kind of thing. As soon as it connects.

108. 19:47

     So one thing it's gonna do is if you click on this database icon here,

109. 19:54

     it should end up looking for... It looks like it might be blank still, or it's just slow to load. Okay. So end up updating. So for every node that has a label, it'll show the labels that exist, the relationship types, obviously the properties.

110. 20:11

     If you click on any of those, actually, if you go to the top, click on anything, it'll basically run a query. This is a Cypher match query, so it'll match for records that have a label article, and it'll return twenty-five nodes.

111. 20:24

     Yeah. Yeah. Cool. [background chatter]

112. 22:49

     Do you

113. 23:11

     have Josh Michael in here?

114. 23:20

     Josh? [background chatter]

115. 24:20

     Does anybody else need some help that I can tell Tomasz to help with? [laughs]

116. 24:39

     Everyone's good? So there, there might still be some people having network connectivity issues. We've seen a couple different variations on that. We'll try to help figure those out and get that sorted.

117. 24:51

     Um, I think we can maybe- Okay. Yep. Awesome. Slides have been- Um. We just-

118. 25:01

     Can you try to unplug and re-plug it?

119. 25:10

     Oh, it's crowd sh... It's the stupid- Okay.

120. 25:17

     What?

121. 25:17

     By the way, I have to say, you're all pretty good for such a big room, everyone being on top of things. I'm really impressed, so well done, everyone.

122. 25:24

     It's the software they installed on the... Yes. Fun, uh, security software on my laptops preventing me from connecting USB devices. So hopefully it's there. If it disappears again, let me know.

123. 25:38

     Um, one thing that I wanted to point out, someone had a good question about, uh, what we were loading. Um, so there is, uh, there's a couple things that we do kind of at the end of this block.

124. 25:48

     One of them is I'm gonna create a fake sort of dummy URL, um, 'cause it wasn't included in the data, so we'll use that to help. The LLM will basically provide a link to the product, which won't be a real link, but you can imagine substituting a real link into the database, so we're just creating a new

125. 26:02

     URL property for that. And then this thing here is creating-- It's basically taking other properties that were in the database, so when we looked at the product, um, and we'll see what these fields actually look like in a second.

126. 26:14

     But, um, there's a product name, a type, a group, its description, all this stuff, and this statement here is just creating a new text property to kind of squish all of that together in one place.

127. 26:25

     And we're gonna use that, um, for our upcoming vector search. So we're gonna index that. So that's why we're creating this one text property. Um, so just wanted to mention that really quickly before we move on to our text section.

128. 26:42

     Um, so now we're gonna, we're gonna start talking about vector search, and we're gonna do some initial, uh, search and retrieval. So when we think about what embeddings are, and I'm assuming a lot of people in this room are already, uh, quite familiar, right?

129. 26:57

     But an embedding is basically a type of data compression that takes some messy data like text, images, or audio, and it puts it into a compact format that's machine learning friendly, right?

130. 27:09

     Um, and so these are gonna be most often numeric vectors or arrays with hundreds or thousands of elements. Um, and they preserve information such that similar items have proportionately similar embedding vectors.

131. 27:21

     And this similarity is gonna be measured mathematically using something like cosine or Euclidean or ex- et cetera. Um, and another thing that I wanna mention is we will, and maybe I'll save it a little bit for the end, but just like you can, um, embed audio, text, and videos, uh, you can also embed graph structures.

132. 27:38

     Um, and so when you have co-purchase relationships in a graph or you're trying to do entity linking in a graph or anything like that, that can become very valuable for then using downstream, um, in vector search.

133. 27:50

     Um.

134. 27:51

     Are you gonna come to the model statements then?

135. 27:54

     Yes. Yep, we will walk through that. Um, so similarity, um, for something like a text embedding is referring to semantic similarity, so the meaning behind a text sequence. For a graph embedding, right, similarity can mean similar in position or structure inside of a graph, and that can have semantic meaning too.

136. 28:13

     Um, and right now what we'll be using vector search for is just on text embeddings, and later when we get to recommendations is when we're gonna use graph embeddings and specifically node embeddings, um, that we're gonna use there inside a vector search.

137. 28:28

     Um, so talking a little bit about just what's inside of [REDACTED:username]. So [REDACTED:username] has multiple indices, um, and you can use these in combination, um, with each other. So we have things like range indices, which is just a general, um, kind of index that can do, um, you know, range comparisons and equality comparisons on numbers and dates

138. 28:47

     and things like that. Um, point indices, which are geospatial or I guess point space indices, right? Um, text indices and full text indices. So we have a full text index, um, that's backed by Lucene, for example, that is tokenization and analyzers.

139. 29:03

     Um, and today what we're gonna be focusing on primarily, um, are vector indices. And our vector index uses, um, HNSW, hierarchical navigable small world. Yes. There we go. For approximate nearest neighbor search.

140. 29:19

     Um, and so a lot of this might be review for the folks in this room, um, right? But the idea behind this is that if we have some product, some text about a product, um, and let's say a, a search prompt on the right here, um, for a halter neck top, we can start to identify things that

141. 29:36

     might be more similar to that, um, using that embedding vector, right? And sort of the, the semantic similarity inside of that space.

142. 29:46

     Um, and right, and that can-- that semantic similarity can be measured by either in Euclidean distance, right, or cosine distance. Um, and like I said before, we're using HNSW, um, to perform approximate nearest neighbor search, or we will be here in just a second

143. 30:03

     Um, so we're gonna start, um... You know what, let me just go ahead and jump into the notebooks. I think it'll be easier, um, to do live. So, uh-

144. 30:13

     May I ask a question before you-

145. 30:14

     Yep

146. 30:15

     ... For those who are not familiar with [REDACTED:username], how to visualize a node from... We, we have created something, but how to-

147. 30:24

     Yes. So, um, right when we're done with the vector indexing, um, I will go into Browser, which is one of our tools for visualization, um, and we can take a look at that.

148. 30:34

     But basically, um, if you go into, uh, this page here, I can go to Open, um, and then it, it will take me basically into the Browser tool. This is gonna take a while because of the internet lag, um, and we can visualize in there.

149. 30:49

     Um, if you're, if you're willing to bear with me, um, in creating a vector index first, 'cause I wanna just go over that very quickly, um, and then we can go into the visualization portion.

150. 31:00

     Um, but yes, we will very soon be, uh, visualizing the data. Um, what I wanted to do first, though, was just kind of get it, um, get our index set up and just show you how to do some searching.

151. 31:14

     Um, so basically what we're gonna do with our data, we're specifically gonna match the product node. So if you remember, we have that, um, product node which is a variant of an article.

152. 31:26

     Um, and the way that this is going to work,

153. 31:30

     um, you can call embedding models externally too, uh, but what we're gonna do here is, uh, basically match the product where it has a detailed description. So basically, some of these products don't have complete information, so we're just gonna take the ones that have complete information.

154. 31:44

     Um, and then all this thing is doing is it's using this code, GenAI vector encode batch, um, to, on the server side, create embeddings and then set that embedding property on a property called text embedding.

155. 32:00

     Um, we're using OpenAI for this. There's a few other models that we can use here, um, or a few other API providers rather. Um, by default, this is using text embedding, um, ADA or A-D-A.

156. 32:13

     Um, but you could also use Vertex AI here, um, as well as Azure OpenAI or, um, [paper rustles] AWS Bedrock. And then you can also just generate your own embeddet- embeddings externally, um, and adjust them for any other type of model.

157. 32:29

     So if you go ahead and give this a run, it will start creating your embeddings.

158. 32:34

     Um, and then the cell after that is gonna create your vector index. So this isn't creating an, an index. All this is doing is basically populating in batch a text embedding property.

159. 32:46

     Um, and then after you create that, this cell down here, if you run that, is going to create your, um, your vector index. We're gonna set it ahead of time to use cosine as a similarity function.

160. 33:00

     Um, and then you're just, this next call is just to wait for it to come online since once you set it, it just takes a little bit of while for it to index all the data and become available for querying.

161. 33:13

     Yes.

162. 33:14

     Can you, can you just clarify the difference between generating the embedding, which is the semantic representation [coughs] of the text, right, versus creating the vector index, like what's the-

163. 33:24

     Right. So when you're, when you're doing the, uh, embedding generation, all you're doing is you're taking this text field that I talked about earlier, you're sending it to OpenAI to say, "Hey, embed this."

164. 33:38

     You take that back, and then you set that as a property. So all this cell is doing is basically making these, uh, vector array properties inside of the node.

165. 33:48

     And then the vector index actually sets the index to make that searchable.

166. 34:01

     Um, once you've, once you've gotten that done, we can do a couple examples of search. Um, and again, this is, this is sort of just using Cypher here, and you can do this externally as well.

167. 34:13

     Um, but in this cell I'm taking a search prompt that is very simple, just denim jeans, um, and then I'm passing it to this function, which is running Cypher.

168. 34:22

     All this is doing actually is just encoding. So basically, it's just creating the query vector, right? So if I were to just encode denim jeans, this is the query vector that that creates.

169. 34:35

     Um, and then if you take that query and you go to the next, uh, cell here, what this will do is it will do that same encoding. So this is the first part of the query that we just ran, which just creates a query vector.

170. 34:48

     Um, and then inside of Cypher, I say call database index vector query nodes, um, and I say hit this index that we just created, give me the top 10 with that query vector, and yield the node as, um, product and the score, um, and then give me some fields like the product text and the product code, et

171. 35:06

     cetera. And so when you run that, it should hopefully give you some results that look something like this, and this is returning different products. So you see all of these are, uh, denim jeans or denim products, um, or they should at least be jeans and/or denim, right?

172. 35:21

     I think all jeans are technically denim. Maybe not. Um, but you see here we have the product descriptions, um, and the product text that it brought back, so you can see all the descriptions, and that's basic vector search inside of [REDACTED:username].

173. 35:40

     Should we let you know if we have a problem with something?

174. 35:42

     Um, yeah.

175. 35:44

     Sure.

176. 35:47

     So unknown property warning.

177. 35:51

     Oops.

178. 35:53

     We're still seeing it for one specific page.

179. 35:59

     Hmm. Oops.

180. 36:03

     I, I have a question.

181. 36:04

     Yes.

182. 36:05

     Do you, do you build the HNSW on every, uh-

183. 36:10

     Do we build it on every node? Well, we build the index, and then every node has a property, so it's built on top of all those properties, right?

184. 36:20

     One HNSW for the entire graph.

185. 36:22

     Um, maybe I don't understand. It's one index.

186. 36:25

     Well, HNSW is its own graph, right?

187. 36:28

     Yes.

188. 36:29

     It's a different algorithm.

189. 36:29

     Yeah, this-- So our graph representation is not the HNSW graph representation.

190. 36:34

     Right. So I'm just trying to understand the scale. You put an HNSW, and now we're talking graph to graph. HNSW its own graph for every node in your graph or is it, or is it one HNSW for the entire [door shutting] graph?

191. 36:49

     Wouldn't it be one HNSW for every index?

192. 36:53

     Yeah. It seems.

193. 36:54

     Yeah.

194. 36:54

     I'm not familiar with what-- Are you talking about a Lucene index?

195. 36:58

     We are using a Lucene index on the back end. Yeah.

196. 37:00

     Okay, okay.

197. 37:01

     Yeah, this is a Lucene implementation.

198. 37:03

     Yeah. Okay.

199. 37:05

     Other questions?

200. 37:09

     Yes.

201. 37:11

     If I already have embeddings, would it be easier to just, like, load them?

202. 37:15

     Yeah, if you do, I have, um-

203. 37:18

     If they match the size of the index.

204. 37:21

     Yes. Well, yes. If you, if you already have embedding... So I commented this out. Um, I haven't run this block for a while. But basically, you've-- this is just showing you how you can create, like, you know, use OpenAI embeddings here.

205. 37:36

     Um, I'm using LangChain in particular. But this is a workflow where you have embeddings external, right, and then you can import them into [REDACTED:username]. Um.

206. 37:45

     And, uh, thank you. Same question for a vector database. You have a-- another system where I have vector database. It would be easy to call the vector database to get an answer?

207. 37:55

     Or does the vector database need to be instanced, like, in the application?

208. 38:01

     Um, so there's multiple different ways to do it. Um, if you have your vector index inside of [REDACTED:username], that can make certain things more efficient. Um, but there's certain situations for scale where using something like LangChain or another orchestration framework, for example, um, you, you can query your vector data inside of your chosen vector database and then

209. 38:21

     do your graph traversal stuff in [REDACTED:username], right? [door shutting]

210. 38:27

     Yes.

211. 38:29

     Um, why do I need a graph database if I'm using vector search?

212. 38:34

     We're gonna get to that in a second.

213. 38:36

     What was his question? Do we know?

214. 38:39

     It's why do you need a graph database if you're, um, if you're already using vector search? And so what we're gonna explore today is there's a couple different things.

215. 38:47

     One of them is using structured data or data that's been structured inside the graph to better sort of personalize or make the responses more relevant for a large language model, um, for search.

216. 38:57

     And then we're also gonna do some things with graph embeddings, which is a different type of embedding that we can create, um, inside of this database.

217. 39:04

     I have a follow-up. Is there a possibility to do the-

218. 39:10

     Okay. Let's go fast

219. 39:12

     ... getting the cosine similarity or the distance where you pre-compute the distance and the distance vector, not doing it like right then so that you can have the same queries and things really fast.

220. 39:23

     Yes, we do that here. There's an example of that. We'll do KNN, and we'll show you how to draw relationships for that. Yes.

221. 39:30

     I was gonna ask just a question or just to follow up that. Since it's structured data, what would be the advantage of using [REDACTED:username] over something else?

222. 39:39

     Um, well, so it's a good question, and we should, we should definitely have a longer conversation afterward. But the advanta-- There's a few different advantages. Um, we have, um, index-free adjacency, and we have very performant, like, multi-hop patterns.

223. 39:52

     So you'll see some of the patterns in here are very complicated, and it's easier to do inside of a graph. First, for just writing the code to do it, but then also the performance on it will be better.

224. 40:02

     Um, and so there's-- and there's a very flexible schema, so it's very robust in terms of being able to change and pivot things around very quickly. Yep.

225. 40:11

     I have one more. With the HNSW, do you scale up quantization?

226. 40:15

     Not yet. Um, not yet. All righty. So I'm gonna, I'm gonna move on a little bit here. Um, this next part of the vector search portion, I'm gonna just integrate LangChain into the mix.

227. 40:35

     Um, so basically, the reason I'm gonna start using LangChain here is because when we get to the LLM portion, it'll just be a natural fit where we'll create a chain for this.

228. 40:45

     Um, but all we're doing here is we're gonna create a vector store using [REDACTED:username] vector. So this is part of the, um, LangChain vector stores. Um, again, instantiate, um, an OpenAI embedding model.

229. 40:59

     We're just gonna use the default with fifteen thirty-six as the default size. And then you see we can create a LangChain vector store from index, um, with that embedding model and our same credentials and the index name that we created earlier.

230. 41:14

     Um, and when we do that, basically what this will allow us to do, uh, is start running from our search prompt, which is denim jeans. We're gonna be able to run the results through, um, this vector store here.

231. 41:25

     So this is what we're gonna be using throughout the rest of the notebook. Um, instead of just using our Cypher, our query language directly, um, we're gonna be using LangChain, which gets this done under the hood for us.

232. 41:37

     Um, so that's that, and you can see it brings back the different documents with the product descriptions and the representative URL and all the metadata. Um, and if you want it to be in a pretty data frame so it's easier to see, um, you can run that to kind of get the-- to get the view we had

233. 41:54

     before, and this should match what we were doing earlier.

234. 41:59

     Um, and of course, you can try yourself with your own prompt. Um, so if you were to type in

235. 42:06

     something else here, you can see it'll bring back, right, a result that's relevant to that. So I typed in oversized sweaters, and it will bring back, um, different oversized sweaters.

236. 42:24

     All righty. So, um, how many people were able to get through the vector search section, if you just raise your hands? Okay. Good. Very fast class. Okay. Anyone having trouble, struggling, need help?

237. 42:41

     One person over there. All righty.

238. 43:17

     Can I, can I ask one more question?

239. 43:18

     Yep.

240. 43:19

     So, like, if you're using Lucene, how do you determine how many Lucene indices do you need?

241. 43:27

     Do you pre-compute that on your side because the user has to figure that out? Because each HNSW can only take up around 50 gigs.

242. 43:35

     So right now you... I might not fully understand your question. Um, when you cr- you basically instantiate an index for one or a set of properties that can be on a node or it can be on a relationship, right?

243. 43:48

     So if I make one index to cover, you know, the property of text, that's one index. I can create another index to cover, like, a product description or another name or something, right, if I wanted to.

244. 43:58

     Um, but that's-- there's nothing that would, you know... So it would be like it's one index at a time, right? Does that, is that getting towards you or is that totally off?

245. 44:06

     Let's say we have, like, billions of documents. I'm gonna vectorize the hell out of them. I'm gonna create an index. So you create... You di- you said you're using Lucene, right?

246. 44:14

     Right.

247. 44:15

     So Lucene is HNSW compatible, so each Lucene index takes 15 gigs of data. What do you do on your s- I'm just curious, what do you do on your side so it doesn't puke?

248. 44:25

     Because after 15 gigs, it's, there's diminishing returns.

249. 44:27

     Right.

250. 44:27

     Are you, like, automatically auto-sharding it? Like, what are you doing to make sure-

251. 44:31

     There are, there are ways to, uh, federate the database. Like, you could, you could create different shards based off of metadata values. Um, but we're... That's stuff that we're, like, currently building on and experimenting with.

252. 44:46

     Um, and for very, very large, like, if you, if you truly had, like, tens of billions of documents, um, you can also integrate with your vector store of choice, right?

253. 44:55

     And then just basically you substitute out the vector retrieval call, um, with- for that vector database, and then you can use [REDACTED:username] for doing a lot of the graph traversals.

254. 45:04

     But it's one of those things where we're continually improving. So, um, we're coming out with new features to handle that sort of stuff.

255. 45:18

     All righty. So let's start with semantic search with graph patterns. Um, so this is going to be, um, basically building on top of vector search where we start to work with structured data.

256. 45:31

     So when we think about [REDACTED:username] and semantic search, there's sort of multiple layers. Uh, so the first part is vector similarity search, which we just went over, which is just finding relevant documents.

257. 45:42

     Um, after that, there's graph pattern matching, which is basically saying, "Hey, can I find entities associated to these documents and the sort of patterns in that connected data?" Um, and then we have the more data science and machine learning part on top of it, where we can improve search relevance and do other things by running graph algorithms

258. 46:01

     like embeddings, right? Um, so when we think about semantic search patterns with graph, um, there's connections that we have, um, based on purchase behavior. So basically given that you have one person who purchases relatively the same things as another person in this graph, you can start to make inferences about what they might want to purchase next, right?

259. 46:21

     So this is sort of like if you've ever dealt with collaborative filtering or sort of those types of patterns, this is a very common recommendation pattern that we can do with structured data.

260. 46:31

     Um, and when we go back to the graph, we'll start exploring the dataset now in a little bit more detail with visualizations. So

261. 46:41

     what I want you to do, and I don't have these calls saved, but I'll go ahead and copy them over. Um,

262. 46:51

     basically you're gonna go back to your browsing window, um, and you're gonna open your, uh, blank instance. So I'm gonna click Open.

263. 47:04

     And hopefully that'll work, and if not, I might actually start, uh,

264. 47:09

     hotspotting myself. Yeah, that part failed for at least some of us. Yeah.

265. 47:25

     It took me, like, a minute to refresh.

266. 47:28

     Yeah. Me.

267. 47:47

     All

268. 48:13

     righty. Hotspot. Yeah, I'm hotspotting. All right. All right. Thanks. Turn on the Wi-Fi. I'm on the-- I think I'm on the Wi-Fi.

269. 48:23

     All righty. So basically what I did is I just entered into something called Browser from our sandbox. So again, you just go here, or I guess I'm, I'm back to this window now.

270. 48:34

     You go here, and you press Open, and then you can either do the SSO sign-on, or you can use your username and password. And so once I'm here, I'm gonna copy.

271. 48:42

     If you go back to the notebook, you can copy these calls. This is just Cypher that you can put in to help visualize, uh, the data that we've ingested.

272. 48:51

     So this will give me my schema, right? Um, you won't have this relationship yet, but what you should have is these four nodes, um, which,

273. 49:00

     let me get rid of this window, um, will give you an idea of your data model. That should reflect what we have in that picture inside of the notebook.

274. 49:07

     We have our articles, our products, our departments, um, and our customers. [coughs]

275. 49:13

     Excuse me. Um, and then if we go back to our notebook, I'm gonna pull a sample of that data just so you can see the graph. So this Cypher statement here, um, is just saying, "Hey, get me all the products, match all of the articles that are variants of those products, um, and then all the customers that

276. 49:33

     purchased those articles." And I'm just gonna return all of that, and I'm gonna limit it to results.

277. 49:39

     Um, and so if I do that, what's gonna happen, and again, the Wi-Fi is, is kind of hurting me right now,

278. 49:46

     um, I'll get this pretty graph. So this is a sample of the data that we loaded. So customers are in orange, articles are in blue, and products are in red.

279. 49:55

     So what you'll start to see, right, is we have these customers. Some of them make multiple purchases. So here's a customer making multiple purchases.

280. 50:05

     Make that a little bit bigger so you can see. When I go to purchase, it should have a transaction date, a price, all that sort of stuff. Here's my customer with the customer ID and some metadata about the customer, and then here's my article, and my article is a variant of a product.

281. 50:21

     So what we've done so far, um, is if you go down here, we've created, right, this text field for every product. Um, that's what the text description looks like.

282. 50:30

     And then we've created this text embedding. This is our OpenAI embedding that we have here. And if we zoom out, the graph structure that I was talking about before in slides, we start to see that.

283. 50:41

     So you start to see that you have these customers that are co-purchasing things, right, at different times. And so we can start leveraging that graph structure, and we can use that graph structure to start informing our search and our LLM how to return better responses.

284. 50:55

     So for example, um, what I'm gonna do is I'm gonna feed it a specific customer ID. So all this does is it basically caches this parameter for this specific customer ID so I can, um, use that in later query calls.

285. 51:09

     So I'm gonna go ahead and put that in.

286. 51:14

     And then I am going to, uh, take this query here, and basically what this query is just gonna do is tell me the history of this specific customer, their purchase history.

287. 51:26

     So I'm gonna go back here, and I'm gonna say, "Hey, get me the customer with that ID, and then get me all the things that they purchased, and then all of the product categories that those were in, basically, and just return some metadata about that."

288. 51:39

     So I'm gonna go ahead and give that a run, and it will tell me this person purchased the specific product names right here, um, the product type that they belong to, et cetera, um, and the description of those products.

289. 51:52

     Um, and then the cool part about graph, um, is I can basically start taking, uh, I can pull the latest purchase history, and then I can start making recommendations off of that latest purchase history.

290. 52:03

     So there's lots of more complicated and fancy ways to do this, um, but the basic idea here, right, is that if I go with, um, this, I can pull the latest purchases.

291. 52:13

     So similar to what we did before, just say, "Hey, take this customer, get everything they purchased, get sort of the max date as the latest purchases," and then I'm gonna do this long query here.

292. 52:24

     And basically what this long match statement is saying is say, "Hey, take this customer, take their latest purchases," right? "Look at other customers that purchased the same thing, and then look at what they purchased," right?

293. 52:39

     That's what it's saying. So it's like, right, if ABK, if I buy the same things as ABK does, I can say, right, which, okay, thumbs up, right? Okay, and then I can basically say, "Well, um, for me, look at what ABK has purchased and a few other people that have similar purchasing behaviors, and is there anything there

294. 52:56

     that I haven't bought yet?" That's basically what this query is doing. So when you run this query, um, what you'll see is you'll start to get these different sweaters and everything, and this will change for every customer.

295. 53:07

     And so the idea here is that this structured data that can feed us this, this, these, you see here how they have these common purchase scores. This is basically scoring the amount of times that you had, um, that you had common purchases between, uh, customers and then, like, sort of the next thing that they buy.

296. 53:24

     So for example, um, eight other, there's been eight other common purchases between people that have bought this and between something that this particular customer has bought. And so these top, um...

297. 53:37

     Go ahead.

298. 53:38

     Uh, you can imagine this as your peer group, right? So the people that are like you, that, that buy the same stuff or are interested in the same stuff, so that's your peer group.

299. 53:45

     And you just look what, what else has your peer group bought that you haven't bought yet, right? So that's kind of what is behind that-

300. 53:51

     Yes, exactly

301. 53:51

     ... as collaborative filtering.

302. 53:52

     And these are the top ranked things in the peer group. So this specific sweater, this T-shirt.

303. 53:58

     Can you just say slowly again what, what the, what the common purchase score means in plain language?

304. 54:04

     Yep.

305. 54:05

     Thanks.

306. 54:05

     Um, and let me look at it again just to make sure I have it right here. Um, so we're basically counting the number of paths between the number of co-purchase paths.

307. 54:18

     So basically, this customer, when we go out and look at what other people have bought in that peer group, basically, how many times has that item been bought in that peer group?

308. 54:27

     Does that make sense? Yes.

309. 54:31

     It says high score means lots of people who bought it.

310. 54:35

     Yes, exactly.

311. 54:36

     Thank you.

312. 54:36

     Yep. Is that clear for everyone?

313. 54:41

     How does it-- Sorry, how does it know it's a peer group?

314. 54:44

     Peer group being that it's sort of the local graph structure. So what I'm doing, right, is I'm basically, um...

315. 54:53

     If I look at, if I look at this graph-

316. 54:56

     Yeah

317. 54:57

     ... I'm basically going, like, I'm starting at a specific customer, then I'm looking at the article, and then I'm looking at everyone that bought this, and then I'm going out one more hop, right, to see everything they bought.

318. 55:08

     And then I'm gonna count up how many times those purchases appear, and the more purchases there are, the more likely it is, right, that that individual customer, um, is likely to like that product just based on the common purchase behaviors in the graph.

319. 55:21

     Does that make sense?

320. 55:23

     Yeah. Uh, so in this case then, the customer that you were just pointing at, um,

321. 55:30

     the-- Well, yeah, before you scrolled away.

322. 55:32

     Oh, sorry.

323. 55:33

     I'm just-- I wanna make sure I understand the graph. So you picked that one random customer, and you said you go one hop away from...

324. 55:42

     Uh, so yeah, so assuming that person, then the, that blue product, um, I guess, so that's the one they bought, and then the one to your left, I suppose, uh, that would...

325. 55:54

     Or I guess right, whatever. The next, the, the closest blue dot, that's another product.

326. 55:59

     Yep.

327. 55:59

     Because there are links between, uh... Oh my goodness.

328. 56:03

     Yes. There's lots of products, right? So it's basically going, bought this, this person bought that, and then for all of... See, and there's a lot of interconnected relationships too.

329. 56:13

     And then it, and then, yes, for all of that, and then go out one more purchase relationship. Um, so you're basically going out and then you're saying, "Okay, how many times do these other blue nodes show up?"

330. 56:24

     Right?

331. 56:25

     Okay.

332. 56:25

     Um, and the more that they show up inside of that graph, the more shared purchasing behavior you have.

333. 56:31

     Yeah, yeah.

334. 56:31

     Yeah. So it's basically just counting that, um-

335. 56:34

     And you could write the query to go out further if you wanted to.

336. 56:37

     You could.

337. 56:37

     You could build on this.

338. 56:38

     Yeah.

339. 56:38

     And w- and part of the machine learning piece, it does that, um, automatically, so, but yes.

340. 56:46

     Can I ask a question about time window? So [clears throat] for example, here you are considering, like, you know, the person, no matter if it's one month or one year, that's one transaction.

341. 56:57

     Versus if I'm considering window, right? One month win-- versus one year versus one way, is there a way, like, you know, to-

342. 57:05

     Yeah

343. 57:05

     ... make this even?

344. 57:06

     Yeah. So you notice, um... Let me get out of this view.

345. 57:10

     Um, if I go up to, uh, the query that I just made here, and maybe I went through this a little bit too fast. Um, but you should see,

346. 57:20

     um, in here I'm saying I'm just going on max date. But you could filter the date to say, like, uh, tdat, which is the purchase date variable. You could say, "Do that for only the last week or only the last two weeks," or whatever filter was relevant for that.

347. 57:37

     Okay.

348. 57:39

     Um, and so that's basically the idea that we're gonna use, um, to provide additional, um... We're actually gonna do a graph post-filtering pattern with vector search. Um, so what we're going to do is if you, if you come back to the notebook, um, and you go down and run.

349. 57:55

     So this is-- here's the query, right? So basically what we're doing is we're gonna instantiate another vector store. Um, except here what we're going to do is we're gonna say, we're gonna put a retrieval query at the end.

350. 58:09

     So what this is gonna do is it's gonna do vector search like before, but then after it does the vector search, it's gonna try to match for that customer.

351. 58:16

     It's going to create that, uh, sort of peer group, that local graph, and then it's going to score, well, how many times does that product show up? So this match is maybe almost backwards, making it probably a little bit hard to read, right?

352. 58:30

     But it's the same query that we had before where we take the customer ID, and then we go sort of one or, I guess, two hops. But really we're saying, "Okay, get me that peer group where we go to the article, we look at all the purchases, um, the common purchases between customers, and then we compute that

353. 58:47

     same score." And there is some multiplication here that I do basically to combine the vector search score with the, um, what we're calling the purchase or the co-purchase score here.

354. 58:57

     Um, but basically what this does is it now allows me to s- pass a customer ID into the search result. Um, and then when I, when I do similarity search with that, um, it will start sorting things by, um, basically this, this purchase score.

355. 59:15

     So now instead of getting... 'Cause denim jeans is, like, this really generic thing, right? So different customers, and we'll see when we run the app later, if you plug in different, uh, customer IDs, this result is gonna be different.

356. 59:28

     It's personalized now to this customer. So basically with vector search, we're now using this post-filtering pattern in the graph. Um, and we can also do a pre-filtering pattern, but we're using a filtering pattern, right, to now rescore these things.

357. 59:41

     So now, okay, we're just returning, for example, these different denim items which are much more relevant for this specific customer.

358. 59:53

     Yes.

359. 59:53

     Can you explain the difference between search score and purchase score?

360. 59:57

     Yes. Um, let me see what I did here. So search score is basically just the vector score that came back from the cosine similarity. So when you're doing similarity between two vectors, right, for the text description, that's gonna be your search score.

361. 1:00:12

     Um, purchase score is going to be what we were showing before, which is showing the popularity inside of the peer group. Um, and then I'm just multiplying them together to create a combined score here, but there's different ways that you can do that.

362. 1:00:24

     So are you going to use this product as, as a way of ranking the, the final results?

363. 1:00:30

     Yes. So if you look down here, um, these results are now ranked differently, and it's actually different from the query we ran before. So we ran the same search prompt with denim jeans.

364. 1:00:39

     Um, but here you can see we get, like, Rachel HW denim. We get Jade HW denim skinny and all this sort of stuff. If you look at what we ran before- Um, if I were to scroll up to the top here,

365. 1:00:54

     um, this one, um, you, you'll see that these are different, and actually this is the wrong one. But this is the right one here. Um, you'll see that these are actually different, um, these are different...

366. 1:01:06

     It's a different ordering. There's some similarities here, like this is number three where it was number one below. Um, but the idea is that you're sort of re-ranking this, right?

367. 1:01:15

     Now that you're personalizing it to this specific customer. And this gets really powerful as you get more and more, um, structured co-purchasing data, right? Yes.

368. 1:01:25

     Why are there a bunch of, uh, purchase score zeros in the result?

369. 1:01:29

     Because there's going to be if you-- It's an optional match, so if you notice

370. 1:01:35

     the, the sort of literal reason is that, um, this is an optional match. Um, but the other reason too is that there could be a situation where s- they're searching for something that isn't bought in their peer group yet.

371. 1:01:49

     Um, so this is just organized so that even if you didn't catch anything in the graph structure, you would still have a way to return results based on the vector scoring.

372. 1:01:57

     Um-

373. 1:01:58

     So it could start from the graph.

374. 1:01:59

     Yeah.

375. 1:01:59

     So if a customer doesn't want anything, it could return at least some results, like, that are based on the vector search only. So what's another question?

376. 1:02:09

     So the basic idea is like whether, for example, if I search denim-

377. 1:02:13

     Yeah

378. 1:02:13

     ... so it brings up all the, uh, denim-related products from the vector search and, uh, the number of people who bought in my peer group using the graph. Right?

379. 1:02:24

     Yes. And it will, and it will try to see if there are-- if anyone else has bought anything of that nature, right? And if they have, then it will, it will help reorder the ranking.

380. 1:02:33

     Yeah.

381. 1:02:34

     Yeah. Um, and you could do it the other way around too, where basically, and we have another app, and I might take it to our, um, booth, where you could do the graph traversal first, right?

382. 1:02:44

     So, like, if you, if you had a scale problem that was, you know, very large, one of the, one of the ways we can scale is through, um, basically, hey, I would do the graph search first, um, and then after that, I can do the, the vector search.

383. 1:02:59

     So, um, whatever you are getting in this, uh, purchase score from your graph database, uh, how is that different from, like, batch, uh, recommendation for products that you may have?

384. 1:03:09

     Let's say you have a separate system for feeding batch recommendation for you. That-- Let's say gives you top 10, uh, recommended products for you. Is that, is that easy to just use those recommendations instead of traversing this graph to get the purchase score?

385. 1:03:24

     Well, you should repeat the questions, otherwise people don't know what you're asking.

386. 1:03:27

     Oh, people can't hear. Um, so the que- the question was, um, why can't you use, like, an external batch recommendation? Um, so you potentially could. What you're getting here is basically as this data is updated in real time in a transactional store, right?

387. 1:03:44

     You're going to be able to re-query it. Um, so you're getting everything kind of in real time, right? So if you're doing RAG and you want answers to be, you know, adjusted for what people had just purchased, right?

388. 1:03:55

     This would be a way to accomplish that inside of a graph database. Um, and then there's a lot of flexibility. You can change the way this query is... You know, you can shorten the time period.

389. 1:04:04

     You could, um, create a different type of traversal to go more than one hop out, et cetera. And-

390. 1:04:10

     There's so many reason we had a question.

391. 1:04:13

     Okay.

392. 1:04:13

     Go ahead.

393. 1:04:14

     Okay. Sorry, I'm hang- hiding in the corner over here.

394. 1:04:19

     It's okay.

395. 1:04:19

     Um, I was wondering if you could speak more about combining the scores. It seems like, um, just multiplying them together is a good way to get started, but is there a better theory there?

396. 1:04:28

     Or it seems like something that have pitfalls, you know, to just multiply the two scores together. You don't know, are they normalized? What's the distribution between them? Like, how are you going to achieve-

397. 1:04:38

     Yeah

398. 1:04:38

     ... the results you want?

399. 1:04:39

     Yeah, that's true. Um, yeah, so again, this is just an example to get you started. Um, I suppose, like, you could maintain a rank ordering where you say, "Hey, rank by, um, like the search score first, and then rank by the vector score," which would be kind of independent of scale.

400. 1:04:58

     Um, but yeah, there's, there's a few different ways you can do it. This was just a quick way for me to put it together, and I could probably do it better.

401. 1:05:06

     Um, but yes. And, you know, with these systems, it's always good to have labeled data and sort of figure out how you can fine-tune things. So, like, you know, in a production system, you would have recommendations and whether or not those recommendations were successful, and then you can try different methodologies and see which ones get the best

402. 1:05:23

     historic score too.

403. 1:05:25

     For normalization, Cato, if you-- Just one second. One second. One altogether. Just make sure.

404. 1:05:31

     Thank you.

405. 1:05:32

     Ask him a question.

406. 1:05:32

     Yes, please.

407. 1:05:33

     Uh, maybe a follow-up, right? If I thought-- If I had this problem in front the first time, I would probably gone filtering. So filtering on the graph relations and then filtering on a fixed threshold on the semantic matching.

408. 1:05:51

     Can you speak to how you would think about doing it my way [chuckles] uh, versus computing scores like this and multiplying them?

409. 1:06:00

     So you're saying, um, filtering. So basically, you would do the graph pattern first.

410. 1:06:05

     Yeah. I would restrict all the graph to only the sub-graph that matches these patterns.

411. 1:06:11

     Yes.

412. 1:06:12

     And then there filter to only the articles that ma- that have a similarity score above threshold.

413. 1:06:19

     Yep.

414. 1:06:20

     Um, return those.

415. 1:06:21

     Yep. You could, you could do it that way. Um, and in fact, we have a demo of exactly, I think, something very close to what you're describing. Um, the issue is for this course, like if there's a chance that whatever you put inside of the graph, inside of that local graph could have a cold start problem, right?

416. 1:06:39

     Where it might not very well match the vector search. Um, so it is a way of doing it. It just might end up that you're-- you could be over-restricting, right, to a local area of the graph that might not have, in this case, good denim jeans, for example.

417. 1:06:54

     Um, but you could do it that way. Uh, you could also do a filtering where, like you said, for example, give me, um- I guess you could do vector search filtering first, um, and then filter down to the graph, and then if there was nothing in the graph, default to the vector search, right?

418. 1:07:09

     Results and the scores there. So you could do it that way too.

419. 1:07:12

     Any intuition onto, you know, what you expect to work best in your training steps for multiplying into one score?

420. 1:07:20

     Um, I think multiplying into one score is a good way to-- it's just a way for me to get started. Um, and some of it will depend on the scale of your graph, right?

421. 1:07:29

     So if you-- the more, um, sort of purchase transactions you have, the more well connected the graph is gonna be, and the better you're gonna be able to, you know, sort of match things almost like with a pre-filtering pattern where you just restrict to that area of the graph first.

422. 1:07:44

     When it's a little bit more sparse, then, um, you're going to probably not have that same reliability, and you're gonna need to either do an optional match or some post-filtering pattern where, um, you make sure not to, not to over-filter things.

423. 1:07:56

     I have a question.

424. 1:07:59

     Yep.

425. 1:07:59

     Okay. Um, just two real quick. Uh, I, I wasn't gonna ask you about scoring, but since there's been a couple. Um, normalization, a typical way to do it is re-reciprocal rank fusion.

426. 1:08:09

     Is that available here?

427. 1:08:12

     Um, not, not that specific one, no. We have some normalization. Um, we have-- it's the graph data science library. There are some normalization functions, but they're not going to normalize things at query time in the way you're thinking.

428. 1:08:26

     Okay. Yeah.

429. 1:08:27

     Yeah.

430. 1:08:27

     That's the most popular way. Okay. And then, uh, the other one is just, um... And if, if I'm getting ahead of myself, just tell me and I'll be quiet.

431. 1:08:34

     Yeah.

432. 1:08:34

     But everything we've done so far, it seems like creating the peer group, you can do that with a graph database, right? And, and infer relationships. But what part of this is actually the knowledge graph part of it, which is infer unknown relationships based on unsupervised understandings?

433. 1:08:52

     Have we gotten to that yet?

434. 1:08:53

     Yeah. So we'll get to some of that in a little bit.

435. 1:08:55

     Okay.

436. 1:08:55

     Um, yes. So I-- we'll, we'll get through, we'll get through to that.

437. 1:08:59

     By the way, there are also some empty chairs here if someone wants to sit down, uh, such.

438. 1:09:07

     Thanks.

439. 1:09:07

     Any other questions? Otherwise, we probably need to progress. By the way, we also have a really big booth with really comfy chairs over there. Uh, so you can always come the next two days and ask Zach, and Andreas, and Tomasz, and me many more questions.

440. 1:09:24

     Right. So... And we have swag at its store-- booth as well, so.

441. 1:09:33

     Awesome. Thank you, Michael. All righty. Um, so now we're gonna start, um, on mantric semantic search for knowledge graph inference. So this will get a little bit into what you were talking about.

442. 1:09:46

     Um, there's a lot more, so I didn't-- Like, we have a whole knowledge graph builder where we build relationships from unstructured data that we're not doing in this course, but we do have it at our booth.

443. 1:09:56

     Um, so basically, this is where we're gonna start using the graph data science portion of [REDACTED:username].

444. 1:10:08

     So, um, we have this thing called graph data science, um, inside of our product. Uh, and basically, what it does is it offers various graph algorithms to you and this special workspace and data representation.

445. 1:10:22

     So you can effectively create a projection of part of your graph, um, in sort of this sparse matrix representation, and that allows you to run a lot of algorithms very efficiently.

446. 1:10:32

     So things like pathfinding, centrality, community detection, um, different machine learning algorithms, and of course, embeddings and KNN similarity, which is what we're gonna talk about here, um, with a lot of focus on this graph embedding piece.

447. 1:10:47

     Um, this is a, this is a customer who created a very nice, uh, sort of periodic table of all of our algorithms for reference. Um, but there's a lot of different areas, right?

448. 1:10:57

     So there's a lot of things that we aren't going to get to talk about here. So things like community detection, pathfinding, centrality, um, topological link prediction, and, um, directed, directed acyclical graph algorithms.

449. 1:11:11

     Um, but we are gonna talk about similarity and embeddings.

450. 1:11:15

     So basically, a node embedding, um, works just like any other embedding, except it's for a graph where the things you're embedding are nodes in a graph. So this is a picture of a very small graph.

451. 1:11:28

     I think it's Zach's Karate Club is the name of the graph, and it's basically the relationships between, uh, different people in a karate club. Um, but the idea is that I can, uh, take this graph, and I can turn it into a set of vectors.

452. 1:11:41

     And in this case, it's a 2D vector, which is a 2D representation. We're gonna create longer vectors for the notebook. Um, but the idea here, right, is that the yellow nodes that are similar in the graph in the sense that they're really close in the graph, um, are gonna be really close in the embedding space, and the

453. 1:11:58

     orange nodes, and the green nodes, and the blue nodes, et cetera. So that's the idea of what we're doing. We're basically embedding what we call homophily, which is basically this fancy term for, like, the locality inside of the graph structure.

454. 1:12:11

     And once we do that embedding, what we're gonna be able to do is start inferring relationships. So someone mentioned, um, being able to do sort of inference beforehand, so calculations are available at query time.

455. 1:12:23

     Um, and we're gonna use K-nearest neighbor for this. So basically, this is, this is happening outside of an index. Uh, what we're gonna be doing is creating graph embeddings, um, specifically on the articles.

456. 1:12:36

     Um, and then we're gonna be tr-- inferring these K-nearest neighbors between them based on, um, KNN, which is this semi-supervised or unsupervised algorithm. Um, and you can do this for a lot of different things.

457. 1:12:47

     We are going to use co-purchase relationships for this and kind of scale that process for multiple hops. Um, but you could do this for other forms of link prediction.

458. 1:12:55

     Um, for example, things like entity resolution, or if you're trying to resolve, you know, maybe you have people and locations in the same graph and you're trying to resolve whether or not a person is in a specific location.

459. 1:13:06

     Um, you can use these sorts of models to start, um, making those knowledge graph Um, analytics and, and drawing those relationships.

460. 1:13:16

     Um, so let's dive into the notebook to see how that works.

461. 1:13:20

     Question.

462. 1:13:20

     Yes.

463. 1:13:21

     You mentioned you're embedding the nodes. Are we essentially throwing away the edges? Uh, we're not embedding the edges?

464. 1:13:28

     Um, so we're doing a node embedding, which is specifically focused on nodes. We could do different types of knowledge graph embeddings, which embed, like, the edge and, like, right, it embeds the, what is it?

465. 1:13:41

     The, the head and the edge to the tail, right? So there are those types of embeddings, um, and there's different types of subgraph embeddings. Uh, for today, we are-- the relationships are used to understand where nodes fit inside of that vector space, right?

466. 1:13:55

     Um, in terms of clustering of, in this case, what's going to be co-purchase behavior. So we're not throwing them away. They're gonna be used, um, but we-- and we'll use the-- Yeah, you'll see as we go along.

467. 1:14:06

     We'll continue to use the relationships.

468. 1:14:10

     Okay.

469. 1:14:10

     Um, so basically what we're gonna do here, um, is we're going to create a customers also like relationship. So basically, um, this cell, so it's gonna be the first cell after you get to augmented semantic search with knowledge graph and ML, and if you run that cell, it's going to run our GDS algorithms.

470. 1:14:31

     So, um, I have some utilities to kinda help clear-- clean things up. Um, and then basically, we're gonna create something called a graph projection, and we don't have a whole ton of time to go into detail on these in a two-hour course, but the basic idea, right, is that I'm going to look for these, um, common co-purchase

471. 1:14:51

     relationships between articles and customers, um, and then I'm going to project that as a co-purchase relationship to an in-memory graph. So I'm creating basically this graph that contains articles and then a weighted co-purchase relationship between them.

472. 1:15:08

     After that, I'm going to run our, uh, node embedding model. So we have supervised models, um, that we can use, but in this case, I'm gonna use something called FastRP, um, which stands for fast random projection.

473. 1:15:22

     And basically, what this is is a graph embedding, um, that is going to use, um, a lot of sort of this matrix multiplication, um, with sort of this sparse random matrix.

474. 1:15:33

     So the math behind it basically allows it to calculate an embedding really quickly without necessarily needing a neural network. Um, I'm gonna make these embeddings a hundred and twenty dimensions.

475. 1:15:44

     I'm gonna specify certain things like an iteration weight that will tell me how many hops to go out when looking for, um, other similar nodes.

476. 1:15:52

     Um, and then once I, once I create that embedding, um, I'm going to write GDS kNN write. So basically, what this is gonna do is it's going to find nodes with similar, um, embeddings, um, and it's gonna write a customer also likes relationship between those nodes.

477. 1:16:08

     Um, and then there's some sampling procedures that I can use inside of kNN to make it run faster, which I'm doing here. Um, and then once I have that, I can write my, both my embeddings back to the graph, um, and in kNN, I just wrote the relationships back.

478. 1:16:24

     So now we have these customer also like relationships between articles inside of the graph.

479. 1:16:30

     Um, and I get some stats when that's run telling me about whether or not it converged, the number of new relationships written, which is gonna be generally a lot because it's doing this quadratic, uh, comparison, um, and each relationship is scored.

480. 1:16:44

     Um, I had a similarity cutoff at, I think, seventy-five, um, and this was using cosine similarity. Um, so, um, it would be negative one to one, I think, uh, that it was using.

481. 1:16:57

     Um, and we can visualize node embeddings as well, and that's interesting to do in the sense that it will tell me, uh, clusters of purchase behavior. So these, uh, cells that I have here, I've, I've commented out because they take a while to run.

482. 1:17:11

     Um, but basically, this should still be able to be rendered here, this visual. And basically, what this visual will show you is you see we get these clusters, and basically what I did is I took those one hundred and twenty-eight dimensional embeddings, and I used, uh, t-SNE, which is just, um, a way to condense that down to

483. 1:17:29

     a smaller space, and I condensed it down to a 2D space. Um, and basically what you'll see is you'll get these clusters of commonly purchased things. And so this is sort of doing the multi-hop thing that you were referring to earlier, where now it's starting to look at, okay, well, these different clusters, right, these clusters represent items

484. 1:17:49

     that are often co-purchased together.

485. 1:17:52

     Baskets, right?

486. 1:17:52

     Baskets, yeah. Um, and yes, baskets in a sort of sense. Right now, it's just co-purchasing, but you could organize into orders inside of the graph, and you could do co-purchasing baskets as well.

487. 1:18:04

     Um, and so with that information, we can start to sort of scale how we power just plain recommendation. Um, so I can create a new, uh, query here where basically, instead of going to the vector database, I can just say, "Okay, well match the customers, get me everything they purchase as the articles, um, get me what else

488. 1:18:23

     customers also like to buy, and get me the product behind that." So it's similar to what we did before, um, except it's scaling it to sort of a, to a multi-hop pattern, and you can substitute other machine learning models in here if you wanted to too.

489. 1:18:37

     So if you had, um, for example, a supervised model that you wanted to use for recommendation, um, that could be substituted in here to make these inferences, and then you all of a sudden have these customer also purchase relationships.

490. 1:18:49

     Um, you could also create a vector index for that graph embedding, um, which we're not doing here. We've sort of done the kNN beforehand, right? And we've written the relationships to the graph, um, and that allows me to do this customer also liked relationship that I can draw in the query, um, and that gives me a set

491. 1:19:05

     of recommendations. Again, not necessarily based off of a search pattern, um, but just based off of what the customer had purchased before. Um, and to your point earlier, we could create time, date, and day cutoffs and things like that.

492. 1:19:20

     Um, so this is basically just a way to kind of scale things with the machine learning model and leverage graph embeddings. So that was it for the, uh, the graph embedding and sort of the graph machine learning section.

493. 1:19:33

     And again, um, you can use a lot of different types of machine learning models here to basically infer these relationships, um, some of which exist, like, inside of our platform through graph data science and others that you can bring in externally.

494. 1:19:47

     Um, we're at four forty-eight, so we have, um, what, forty-five minutes left, a little under, and we haven't taken a break yet. Um, wondering if we should power through.

495. 1:20:00

     Yeah.

496. 1:20:01

     How do people feel? Should we keep going?

497. 1:20:03

     Yeah. Yeah, keep powering through.

498. 1:20:05

     Okay. Yes.

499. 1:20:07

     Uh, I have a question again with the graph embeddings.

500. 1:20:10

     Yep.

501. 1:20:11

     Do you always have to recalculate it? Is there like any limitation when new customers are coming in?

502. 1:20:16

     So right now you do, yes, have to recalculate the graph embedding. Um, ideally, right, if you... Like, over time, if customers have been in there for a while, that graph embedding, like, will-- it'll slow down how much it changes.

503. 1:20:31

     Um, we are working on things for the future that might make that easier, um, in terms of getting things in real time. Um, but right now for calculating graph embeddings, it is something where you have to recalculate it for the graph.

504. 1:20:43

     Thankfully, though, um, with something like Fast RP, that'll work. I mean, we've-- I've seen it work on graphs with billions of nodes and relationships, and it can calculate in like half an hour.

505. 1:20:54

     Um, but yes, right now you do have to recalculate it.

506. 1:21:04

     Um, you could use other methods too, like averaging the neighborhood as well for sort of like intermediate embeddings as well. So, like, um, if you didn't wanna recalculate it right away, you could take like the average of the neighborhood and use that as like a stand-in embedding.

507. 1:21:24

     All righty.

508. 1:21:25

     Is there an upper limit, like how large-

509. 1:21:28

     What's that?

510. 1:21:31

     Is there an upper limit in graph embeddings?

511. 1:21:31

     Is-

512. 1:21:31

     How many-- Is there an upper limit? Like now you have hundred twenty-eight, which is not a lot.

513. 1:21:39

     Yes. Um.

514. 1:21:41

     Do you want me to repeat?

515. 1:21:43

     Oh, yeah. Go for it. I think, um,

516. 1:21:48

     you, you had a question about dimensionality. Sorry.

517. 1:21:51

     Yes, about the dimension. Is there an upper limit to the dimension? I think you had now hundred twenty-eight, which is quite small.

518. 1:21:58

     There's, there's-- Yeah, there's no upper limit to dimension. Um, right now, inside of this graph, we only have... Let me see how many customers we have. Um. [keyboard clicking]

519. 1:22:17

     So this graph only has a thousand customers, right? There's been other examples, um, and demos that we have where it's like millions or tens of millions of customers, right?

520. 1:22:26

     So the more transactions you have, the more customers and articles you have, the higher that dimension needs to be, right? For demo purposes right now, when we're working off of sandbox instances, I kept it pretty small.

521. 1:22:37

     Um, but you can expand this to two fifty-six or keep going up in orders of two. Um, I found that these generally don't need to be as big as, like, the text embeddings that you get from OpenAI because it's a smaller space, right?

522. 1:22:50

     When you-- Even if you have billions of nodes in your graph, it's still a smaller space than the entire human language, right? So you-- So they tend to be on the-- tend to be smaller, but y- they can go up, and there's no upper limit that we, that we inherently have inside of the product, no.

523. 1:23:06

     And what is the biggest use case you have at the moment deployed?

524. 1:23:10

     For embeddings?

525. 1:23:11

     Uh, graph embeddings, yes.

526. 1:23:13

     Um, I'd say, um, entity resolution, um, different forms of classification, like for fraud detection, um, and then also for, um, recommendation and things like customer segmentation, where you wanna, like, create clusters, um, based on, you know, behavior or other data inside of the graph.

527. 1:23:32

     Okay.

528. 1:23:44

     Um, is it possible to semantically retrieve the information about the graph structure itself? So let's say that you have ten different types of pants, but you want to ask a question, how many types of pants do you have?

529. 1:23:59

     Which is-- Like, it's not in the data itself specifically. It's kinda metadata about it.

530. 1:24:07

     Right. Yes. So, um, very good question. So there's different types of retrieval patterns that are better at those sorts of queries. So we have something called text to Cypher.

531. 1:24:16

     Um, we have it. It's sort of-- It's a pattern, right, where you can actually get the LLM to generate like the, a Cypher query to query the data, and those tend to work well for aggregation.

532. 1:24:28

     So, like, if you were building out a more mature system, you can have, um, like a semantic layer with different tools. And so if you knew that you had more of like an aggregation question about types of things, um, then you can send it to that tool, which can do like a ty-- a text to Cypher or

533. 1:24:43

     another, um, sort of, um, a Cypher snippet that was like pre-generated or template that was pre-generated. Does that make sense?

534. 1:24:51

     Yeah. Uh, have you tried using that with like function calling so you don't have to like specifically know in advance that you'd like to use it, but rather like-

535. 1:24:59

     Yes

536. 1:24:59

     ... get the LLM the ability to do that?

537. 1:25:00

     That is something that Michael, if he's still here, has done some work on for function calling.

538. 1:25:09

     Yeah. So, uh, yes, in general, um, you can make many of these graph operations available as, as functions, uh, to an LLM, and then basically, uh-

539. 1:25:22

     Depends on the selectivity of how you describe the, the function, uh, and what you pass in, uh, to make this available, right? So you can do, uh, this kind of, uh, embedding work, but also other graph queries can be made available as functions, uh, to an LLM, and then it just selects them.

540. 1:25:38

     So for instance, get a neighborhood of a node, or find the shortest path between two nodes, or find similar, uh, nodes to this nodes using, uh, graph embeddings or whatever.

541. 1:25:47

     And then you have these kind of some generic functions that you offer to the LLM, and then you have-- could have also specific functions that are basically built by an subject, subject matter expert that, uh, for instance, for legal domain or for an f-fintech domain, they have s-specific graph queries that are really specific to that domain, and

542. 1:26:07

     then add them as functions as well. So you can have, you know, five, 10 functions available, and then the LLM picks the ones that are necessary to answer the questions and such.

543. 1:26:16

     But it should work with all the LLM models that support function callings and such.

544. 1:26:26

     Yes. Thanks.

545. 1:26:30

     Well, if you have more questions, you can always come to talk to us at tomorrow-

546. 1:26:34

     Yes

547. 1:26:34

     ... tomorrow, uh, on Thursday.

548. 1:26:37

     Um, so I was just trying to gain a little more intuition about the idea of graph embeddings. I'm pretty new to graphs in general. It seems to me like the whole motivation for using embeddings that's really taken off in the last four years or so, and it's beca-because you've got transformers, you've got unsupervised pre-training where you can

549. 1:26:55

     leverage all of, you know, the text on the internet or all of the images on the internet and whatnot, and train these embedding models that are just general purpose foundation models.

550. 1:27:05

     And this is very different from that because you're just completely training your embeddings, um, or if it even is training from scratch. I'm wondering if there's like a deeper intuition to

551. 1:27:17

     why you'd be using embeddings for this instead of-- It almost seems like there's a lot of more like traditional machine learning techniques that you could be using. Um, I, I just don't know what the state of the art is, and I'm curious what the intuition is for using embeddings for this.

552. 1:27:31

     Yeah, I mean, so embeddings here basically is helping you, um, kind of scale these other graph traversals, right? To go-- 'Cause when we do that embedding, what we're basically doing is we're taking a graph traversal that's similar to the one we went over and going multiple hops, right?

553. 1:27:48

     So you're sort of taking the structure of the graph and you're scaling your ability to search for related things inside of the graph without having to do a Cypher query every single time.

554. 1:27:58

     Um, so it's basically-- And it, it can help you also infer information that you didn't know before. So like in this example, right, we're, we're making these relationships on customer also liked.

555. 1:28:09

     Um, we could also do things like say, "Hey," like if we had like an identity graph of people and different identifiers and, and transactions, we could start resolving people in the same household, for example, using embeddings.

556. 1:28:21

     Um, so even though it is kind of local to your graph, um, you're-- the idea behind embeddings is that you're able to infer relationships between things inside of your graph that, you know, would be very hard to do without some sort of machine learning or just traditional methods, if that makes sense.

557. 1:28:39

     A little bit?

558. 1:28:40

     Some sense. [laughs]

559. 1:28:41

     Yeah. Yeah, because you're, you're basically using, um, depending on the, depending on the graph embedding you're using, um, you're basically taking the structure of the relationships and that topology, and you're using it as features, right, to power what something should look like in a representative space.

560. 1:28:59

     Um, and then you can then use those to basically predict where links should exist inside of the graph.

561. 1:29:05

     Yeah. Okay, that makes sense.

562. 1:29:06

     Yep.

563. 1:29:07

     Thanks.

564. 1:29:07

     Hopefully. I don't know if it really did, but [laughs]

565. 1:29:10

     It's-

566. 1:29:11

     Any more questions?

567. 1:29:13

     I mean, it's that you're, uh, you're providing this sort of intermediate representation where you can do more general purpose things with it.

568. 1:29:23

     Yes.

569. 1:29:24

     Is that good paraphrasing?

570. 1:29:25

     Yes, I think so.

571. 1:29:26

     Okay.

572. 1:29:27

     Yeah.

573. 1:29:27

     Understood something. [laughs]

574. 1:29:30

     All righty. Awesome. So this is gonna be the final step for tonight, and then, uh, we'll see, maybe we'll even be able to get out a little bit early and get to, uh, the reception.

575. 1:29:41

     So we're now gonna take what we have, and we're gonna start using a large language model. Um, so we're basically gonna use the retrievers that we were just working on, and we're gonna take them, um, into this prompt, and we're gonna have a language model generate an email for us.

576. 1:29:56

     Um, so this might be a little bit hard to see, but it's also in the notebook, um, for when we go there. Um, this is the prompt that we're gonna use.

577. 1:30:03

     We're gonna instruct an LLM. We're gonna give it a name. We're gonna say it works for a certain company, and we're gonna tell it to write an engaging email to a customer, um, to promote and summarize products relevant to them, given, one, the current season and time of year, and two, recent searches and interests, which is gonna

578. 1:30:19

     be our search prompt. Um, and so all of sort of the red things are where we're gonna inject things inside of the prompt. And then, then the RAG part comes in.

579. 1:30:28

     So we're gonna tell it, "Hey, please only use, um, the below relevant products." So there's gonna be what we're calling search products and recommendation products, which search products is gonna be based on a vector search combined with a graph traversal, so that will give us something that matches the customer interest.

580. 1:30:44

     The recommendation products are gonna leverage our graph embeddings to give us additional recommendations that we can add to that. So there's gonna be sort of two lists that it gets.

581. 1:30:54

     Um, and there's some other instruction there to tell it to only pick, um, a certain number from those things and kinda be creative with that and make sure it aligns to the current season and time of year.

582. 1:31:04

     Um, and this is kind of what it looks like. I'll just go to the end of this thing. So, um, if you were to think about the chain that we're gonna put together, um, we're gonna come in with Basically four things, that search prompt, the customer ID, the customer name, and the time of year.

583. 1:31:20

     The search prompt is gonna go with the customer ID for the personalized search, which is gonna be that, um, sort of peer group pattern. The customer ID is gonna go directly to get recommendations ba-based off of graph embeddings.

584. 1:31:34

     And then these four things here, the, the products that were recommended here and the customer name and the time of year are gonna go into the prompt to then generate an answer.

585. 1:31:43

     Um, so we're doing basically two retrievers here. Um, one for recommendations, um, and then another for basically search results. You can think of it that way.

586. 1:31:54

     All righty. So let's go and dive into the notebook. So,

587. 1:32:00

     um, we're gonna use LangChain for this. Um, we're gonna instantiate the LLM. That's gonna be based off of GPT-4 Omni, which we have here.

588. 1:32:12

     Uh, and then we are gonna recreate, um, two retrievers. So the first retriever here is gonna be for the personalized search. This is going to use the, uh, text embeddings, and then it's gonna do this sort of post-filtering pattern with that, um, combined score where we do this optional match to look at the peer group to kind

589. 1:32:31

     of filter those results. Um, we're gonna create another function here based off of that. I'll zoom into this a little bit more. Um, where basically we are going to take the, um...

590. 1:32:43

     This function is just so we can pass a customer ID parameter in. Um, so this is the KG personalized search function, which is gonna use this retrieval pattern. And then we have a KG recommendations function, which is gonna use that customer also liked, uh, relationship, um, that's going to just basically go off of customer history.

591. 1:33:03

     Um, and then both of these will sort of join. It'll do some formatting for the documents, um, and return them, um, for injection into the prompt.

592. 1:33:12

     Um, here's the prompt itself. So this is just the prompt we looked at in the slide before. Um, it mentions, you know, customer name, time of year, and then it has the relevant products where you put in the, um, the search products, um, and then other recommendations generated from the graph embedding.

593. 1:33:31

     Um, and then when you create the chain, so you can see here on this chain, um, we have four inputs, right? So the search products, which again comes from the personalized search pattern, the recommendation products, which comes from the recommendation pattern.

594. 1:33:45

     So there's these two retrievers that are supplying each of these respectively, the customer name and the time of year and the, uh, customer interests as well. So I guess all five of these things, right, get passed and injected into that prompt, go to the LLM.

595. 1:34:00

     This is all, uh, LangChain expression language. This is kind of a LangChain thing where the-- we have this pipe operator that's just basically taking this dictionary of things and then passing it to be injected into the prompt, calls the LLM, in this case, GPT-4o, and then just gets outputted.

596. 1:34:17

     And so if we, uh, look at what it looks like when we invoke that chain, again, using our search prompt, which is denim jeans, our customer ID, and then I gave it a customer name, um, and a time of year.

597. 1:34:29

     And the customer name is probably something that could be stored in a database too. It says that this data was anonymized, so we don't have that, um, we don't have that in the, in the database itself.

598. 1:34:40

     Um, but anyway, what this will do, you see it'll give me, um, its responses. It'll make a subject header for the email. "Hi, Alex. I hope this email finds you well and enjoy the start of the summer.

599. 1:34:50

     As the days get warmer, it's a perfect time to refresh your wardrobe." Um, and then it will give you the denim jeans. So these are the items that came from the search that was personalized to this user.

600. 1:35:01

     Um, and you see we get, um, a set of five, uh, jean products here. Summer essentials to pair with your denim. And then this is where it uses the graph embeddings to provide those other recommendations down here.

601. 1:35:14

     So it's basically, right, using a combination of this vector search to pull things back, personalizing it so that it becomes, um, specific to just this user, and then adding additional recommendations to that as well.

602. 1:35:27

     And of course, you can change the prompt to be just the recommendations portion if you wanted to, or just the search portion or, you know, whatever, whatever was appropriate, um, for the application at hand.

603. 1:35:38

     Um, and, um, if we just looked at... So basically this cell, if I run it, all this next cell will do is show you the, um, the prompt, the full prompt that was set to the language model.

604. 1:35:51

     So if we look at this, um, this is basically showing you, if I were to scroll all the way to the top, it provides quite a few products. Um, you can see here this is the data that it retrieved and formatted from the database.

605. 1:36:02

     For the relevant products, all the different garments that it selected. Um, and then if I were to go down more, it would eventually bring me down to the recommendation products, which I think it already did here.

606. 1:36:14

     So if I go up further, um, yes, relevant products, um, and then it should give me about ten of those, and then other recommendations from the graph embeddings that go along with that going down here.

607. 1:36:28

     And then it basically chose from those products to generate that, uh, given response.

608. 1:36:36

     So-

609. 1:36:36

     So we are, we are limiting the number of products, right, because of the context length so that-

610. 1:36:43

     Yes. Yeah. If you look at the way those queries were written, um, in the, in the cells before, I think the first one, um, for the search responses, I limited them to ten.

611. 1:36:53

     And then for the recommendations, I'm actually passing a hundred, so I'm passing quite a few. Um, but yeah, eventually, if you provide enough of them, you will hit a context window issue.

612. 1:37:04

     Um, and then, so basically, once we have that, we can run a short demo app. So in this case, we're using Gradio. Um, this is just some examples, um, preset examples for-- that we can use in the application.

613. 1:37:18

     Um, and this is running the application So I'll go ahead and give it a run, and if you have it in your notebook, it should just give you a link that you can go to.

614. 1:37:27

     And basically, let me make this bigger so it's seeable. So here's just a UI of what we had before, right? So this is the example that we ran earlier.

615. 1:37:37

     So if I say submit, um, for this specific user, um, it should pull back hopefully in the next 20 seconds or so 'cause the internet's slow. Normally, it's around 10 to 15 seconds.

616. 1:37:49

     Um, yep, and this will pull back the email response. And the interesting thing about this is if I took... You see this customer ID is, like, [REDACTED:account_number]. If I select a different customer ID but the same prompt, so say it's this person,

617. 1:38:06

     so I click that, and then, yes, let's change to this other customer. This answer should, um, be different, and if it isn't different, it's because they have a very similar, um, peer group and a very similar, um, piece in the graph.

618. 1:38:21

     But if you see HW denim and skinny jeans and then this, you'll see they get a bunch of different recommendations and search patterns. So the idea here, right, is even though these two, um, individuals have the same interest, based on their purchase behavior in the graph, we're able to give them very different recommendations because they exist in

619. 1:38:40

     very different parts of the graph and have different interests based on their purchase history. And you can play around with this. Like, if I... You'll see it'll give me some summer things here.

620. 1:38:50

     If I change this to, like, February, right? This is nothing to do with graph. This is purely an, an LLM thing at this point, and I submit it. Um, it should...

621. 1:39:00

     The email should change to winter, and then it should give you, um, instead of summer must-haves, it should give you, um, something about winter, and it should be able to select, um, different products down here that should, um, hopefully if the LLM [laughs] is doing its job correctly, um, go for more winter-related items like sweaters that you see

622. 1:39:19

     here. Um, so that's the, that's the full app, um, that kind of leverages all the things that we just went over.

623. 1:39:29

     Cool.

624. 1:39:29

     All right.

625. 1:39:29

     Thank you so much, everyone. This was really great.

626. 1:39:31

     Thank you so much. [outro music]