Anchoring Enterprise GenAI with Knowledge Graphs

Talk transcript

1. 0:00

   [upbeat music] Hey, so it's so great to be back in New York City.

2. 0:20

   Actually, I grew up nearby here, and pleased to be co-speaking with Jonathan

3. 0:24

   Thank you, Stephen. Good to be here

4. 0:27

   And, you know, we're here to kind of talk about leadership, talk about how you can actually do a bunch of the things you've been hearing in practice. We're gonna talk about strategy, we're gonna talk about technology.

5. 0:38

   But let's start with analysts. Who, who here trusts Gartner when Gartner says something? They're predicting the AI wave, they're predict... Okay, nobody does. Nobody, no hands went up in the room, for the record.

6. 0:50

   But when they are predicting failure and catastrophes, I, I, I try hard to trust that, right? So last year, they predicted 30% of generative AI projects are gonna be abandoned by the end of 2025. [laughs]

7. 1:05

   Now, anybody in the room, uh, this is a really, real honest check, has anyone been on a failing GenAI project? Okay, now, now brave souls. Amazing. Give th- give those guys a round.

8. 1:18

   That took a lot of courage. [clapping] Now, to make them feel better, [clears throat] who, who hasn't yet got to production on their GenAI app?

9. 1:29

   Okay, so the rest of the hands went up, right? So this, this is the challenge. [laughs]

10. 1:34

    So we all wanna be successful with GenAI. We all want to do amazing things. We're getting asked to do amazing things, but we, we need to have the right way of approaching this in our organizations with leadership to sell this internally, to, to build it on technologies which they can understand.

11. 1:50

    And the, the vision, it's, it's hard to get a vision that's technically achievable when the guy at the head of the table is, is this, is this guy. He's, he's the executive who's heard about GenAI.

12. 2:02

    His kids are using it for their school courses, and he's like, "Oh, yeah, yeah, just it solves all the problems. Insert here success. I want it in production in two months."

13. 2:13

    Now, um, I think the, the great thing about having, having Jonathan as my co-presenter is that he, he's actually done this in a big life sciences company, and he's had to navigate all of this, um, leadership challenges, organizational challenges, silos, to build a system which actually is something we can take to production.

14. 2:35

    So tell us a little bit more about that, Jonathan.

15. 2:37

    Thanks, Stephen. Now, as I've been introduced, Jonathan Lowe, you may know me as Jonathan when we're out in the hallway, or when I give you a bit more information about my experience launching GenAI, these capabilities in business, you may think of me as Debbie Downer. [laughs]

16. 2:56

    AI is so exciting until the singularity. Wah, wah. [laughs]

17. 3:04

    So that's how I actually approached the problem I'm about to explain to you,

18. 3:11

    but it, it actually worked. So the business case was technology transfer, which means in biopharma, scaling up from lab bench, think beakers and human scale drug development, to industrial scale, making a million doses a day.

19. 3:29

    And to get from that lab bench level to multiple factories around the world making lots and lots of product very quickly takes years because the industrial people that build the factories and build the equipment need to sift through hundreds of thousands of documents and notes and test outcomes that were created at the science level.

20. 3:53

    Another challenge with doing that is, I'll go to a statistic. In 2019, a study said that the average tenure of manufacturing workers, tenure being how many years they had spent in their companies, was about twenty.

21. 4:11

    Twenty years of average tenure. What do you think the average tenure is in manufacturing companies today?

22. 4:20

    The study said three years. So we've gone from twenty down to three, and all that expertise has, has or will soon be retiring because the boomers are growing old.

23. 4:37

    So we really need generative AI. We need a machine to take a lot of the intelligence that's captured in documents or even in tacit people's heads and get it to the new people showing up to do this technology transfer.

24. 4:52

    So we take all these millions of documents, and we've loaded them into a graph.

25. 5:00

    Now, we haven't necessarily loaded the document into the graph. We've loaded the chunks into the graph. And one of the things that we really le- liked using the graph to accomplish was we structured the chunks, the document, the block, the paragraph, the line, because we wanted to understand when we searched for those chunks with similarity

26. 5:25

    search, which ones really returned the results that people wanted the most. We wanted to really refine how we stored and managed the chunks. So at this, at this point, it was, it was a totally new space, and because we were able to structure in the graph that level of chunking, we were able to eventually learn and get

27. 5:43

    better and better at how we chunked the documents in the first place.

28. 5:48

    Yeah, so I, I think what's really amazing for me about this is, um, we, we were talking about business challenges and like, like projects failing, and in the study that Gartner did, the, the biggest failure mode was not having a business use case which would actually solve real problems and then be monetizable.

29. 6:06

    And, um, this is not only, like, a great business use case, but it's also something which potentially is saving lives because you're, you're getting life-saving drugs to folks faster, you're able to accomplish this quicker.

30. 6:17

    But the problem is always the humans in the middle, right? So the teams you work with

31. 6:27

    probably have a little bit of GenAI not invented here syndrome.

32. 6:31

    Mm.

33. 6:32

    Where you come along with this great solution, like I'm gonna use GraphRAG, I'm gonna load all these documents into the, my, my, you know, my, my big, um, store, and they're like, "No, no, no, I-- we, we've heard-- seen this research paper, we watched this talk.

34. 6:44

    There's some other platform we wanna use. There's some other framework." Um, or maybe it's, maybe it's too expensive. I mean, compared to classic computing and cloud computing, GenAI architectures have the potential to be much more expensive if they're not well-architected and, in general, are going to re- increase the cost of the organization.

35. 7:03

    So how do you convince people to go from a system which is, is working, but not working well enough, to a much more expensive system, which is R&D, investment, redevelopment, to go towards a GenAI architecture?

36. 7:15

    So what are, what are some of the challenges you hit internally, and how did you address that at Pfizer?

37. 7:22

    Great. So for this one, it's more of an entrepreneurial use case within a big organization. I wonder how many of you have worked in organizations with fifty thousand or more people.

38. 7:34

    A lot of hands going up. Uh, my current organization has over a hundred thousand people. I've also worked at IBM, Deloitte, big organizations. And

39. 7:47

    if you are like me in these organizations, you'll be that little red guy going like this with the, with the light bulb over his head saying, "I have an idea that might help the company.

40. 8:00

    And I have a team of X number of data scientists and developers and SREs, and we can bring that value, that capability to the company."

41. 8:13

    If you're like me, if you're that red guy,

42. 8:16

    who's the first group of people on this slide that you're most interested in connecting with?

43. 8:24

    Well-

44. 8:26

    CEO. You go for the top.

45. 8:30

    You go for the top. Y- you're better than I am. I, I, I joined this whole profession because I love building applications that delight the people that use them.

46. 8:39

    So my instinct has always been to go to the bottom first and say to those users, "Hey, do you really want this tool?" And what are those users gonna tell you?

47. 8:50

    They'll like your tool if...

48. 8:54

    It's good.

49. 8:55

    That's right. What makes it good? It takes away boring stuff that they don't wanna do. But it can't just take away boring stuff, it also has to give them accurate results.

50. 9:07

    It also has to work in a performant way. They can't push the button, go get coffee, and come back. And I feel like that's the easy part, right? More and more these days, you can build accurate, fast applications quickly.

51. 9:22

    So where's the real challenge? So somebody said you go to the top first. What's the likelihood in a company of fifty thousand to a hundred thousand people that you're gonna meet the CEO if you're the guy with the idea at the level four of the hierarchy?

52. 9:39

    The likelihood is pretty small. Did anyone here ever see the movie Dirty Dancing?

53. 9:44

    Dirty Dancing? [laughs] Maybe? Do you remember the part in Dirty Dancing when Baby, the lead, leading woman in the, in the movie, meets Johnny, the amazing dancer, for the first time?

54. 9:55

    And she, she's, uh, she's unable to speak. She's so flustered, and finally she blurts out, "I carried a watermelon." And then off he goes, and she goes, "I carried a watermelon?"

55. 10:05

    Two weeks ago, I stepped into the elevator on the [REDACTED:location] of, of the [REDACTED:location] of, of my company, and there was my CEO in the elevator, and I felt like Baby in Dirty Dancing.

56. 10:17

    I couldn't think of what to say. I locked up. Fortunately, he's a good guy. He broke the ice. "I'm just back from vacation, rolling up my sleeve. Can't wait to get to work.

57. 10:27

    What are you up to?" And then, thank God, ding, we got to his floor, the doors opened- [laughs] ... and, and out he went. [laughs] And as he went out the door, I blurted out, not, "I carried a watermelon," but, "I'm working with LLMs." [laughing]

58. 10:43

    Off he went. So when you're trying to p- promote your work within a big company like this, it would help to know what that executive is trying to accomplish.

59. 10:51

    And the way that he gets to that point is he talks to consultants who say, "Let us tell you how to be a leader in your industry and not fall behind the competition."

60. 11:02

    So an example of something that an executive at that level might do is create a purpose blueprint or something like that name, and the number one message has to be a few words and convey something that the whole company can follow.

61. 11:16

    So an example of that might be change a billion lives a year. In life sciences, a big aspiration. Now why do you have to care about that? In the elevator, maybe you'll reference it.

62. 11:28

    "I'm changing a billion lives a year with the most amazing AI search engine." Bing, and off he goes. But that, that message that he gives trickles down to the next level, the chief digital officer, the chief scientific officer, the chief supply officer.

63. 11:42

    What do you think they're gonna say? They're gonna try to take his message and turn it into their specific flavor. So the digital officer will say, "I want to lead the industry in AI."

64. 11:53

    And the scientific officer will say, "I wanna take on the world's biggest diseases." And the supply officer will say, "I want to accelerate supply." Still very high level, and you probably won't meet these people either.

65. 12:07

    Who will you meet, though? You'll meet their level twos and their level threes, and what are they gonna say?

66. 12:16

    At this point, they don't really say taglines. Instead, they say, "I want cost savings, I want cost avoidance, I want earlier realized revenue, or I want more balanced head count."

67. 12:35

    So when you're talking to these people, your slides have to have numbers and times, and your promises about how your tool or your capability or report or whatever is gonna ma- meet those times and those numbers.

68. 12:51

    Now, you may not get to meet them either. If your big company has a, a form of, um, a role called the client partner, where your digital people talk to the client partner and the client partner talks to the business, then that's the other person you have to convince.

69. 13:10

    And the problem with this is that the client partners tend to stay within their particular departments. There might be a client partner who works exclusively in R&D or one who works exclusively in supply.

70. 13:20

    What would they say? Sometimes they don't say the same thing. One of them might say, "R&D already has five or six or ten search engines. Why build another?"

71. 13:32

    Or they might say, "Search engine [clears throat] is a great idea. Why don't you incorporate that capability into every tool in the supply organization?" So either your scope goes to nothing or it goes to everything, and you need to be able to negotiate and navigate that.

72. 13:49

    Are you done? If you can satisfy all those people and cross through all those gauntlets, well, no. Because as you're starting to build, the vendor comes to you and says, "Why build in-house when you can buy our tools?"

73. 14:05

    And they've been talking to the chief digital officer about build versus buy and which one is more economically realistic and appropriate. Well, maybe you get through that, and then you're done, right?

74. 14:17

    Who else would possibly stand in the way of your incredible AI search tool?

75. 14:24

    Friendly fire is the answer. Your own colleagues, either a level above or at the same level, may say, "Dude, I was here first. AI search is my turf." Or they might just say, "Hey, that client partner over in supply is right.

76. 14:40

    Can you please integrate with the stuff I've built?" So I guess my message is, we've heard a lot of talks about failure and challenge and Gartner not liking this.

77. 14:51

    It's an incredible time to be in this, in this amazing industry and this amazing change in both, for me, life sciences and more generally for the information technology industry.

78. 15:04

    And I love that we're hearing all this concern about failure because it just means we're at the beginning of a really exciting time. But as representatives of that, my advice to you is know your audience, personalize for all of them, and get your human wetware chatbot speaking the right language at the right level.

79. 15:25

    No, that's, that's amazing. Um, we've chatted about a bunch of these challenges, right? So we've chatted about getting a good business use case that can actually provide value to the organization, how to navigate, like, like peoples and, and different failure modes, um, within the organization where the organization has a huge quantity of people who can be your

80. 15:46

    allies or can work against you, depending upon how you work with them. But it's also a technology problem. You have to have the right technology to solve your use case.

81. 15:56

    Now, one of the, the biggest challenges I think a lot of us who have been building RAG and enterprise applications has been the LLMs themselves fighting us with, with hallucinations.

82. 16:06

    This is getting better with newer models. Um, it's getting easier to feed the right sort of information in with vector databases. But I think that [clears throat] you've chosen a rather unique approach using graph databases.

83. 16:19

    Why did, why did you choose to use a graph database for your implementation at Pfizer?

84. 16:25

    Well, um, there are a lot of things that graphs are good at, things like genealogic sequences of recipes or social networks or hierarchies or time series, and all of those applications were prevalent opportunities within Pfizer.

85. 16:44

    So that was the, that was the original impetus for using a graph. But I also discovered that the more data we consolidated in the graph, the faster my data scientists and engineers and developers and SREs were able to understand the data landscape.

86. 17:02

    What used to take three months to consolidate, understand, clean up took three weeks or less for, for a new project. So I know the reason a lot of people take on graph is because traversal becomes so much easier in terms of data search and, uh, and, and performance gets better.

87. 17:19

    But I found that team performance also took a really big boost from using that tech.

88. 17:24

    Cool. And for folks who aren't familiar with, with knowledge graphs and LLMs or, or GraphRAG, um, this isn't a new idea, although I, I would put you guys on the early adopter, where you're actually in production now with something that uses this.

89. 17:38

    But, uh, Microsoft kind of wrote the seminal paper on GraphRAG and used it basically taking existing documents, LLMs, to chunk it into a graph and then showed superior results coming out of it.

90. 17:51

    It-- On the spectrum of technologies using LLMs directly, you can get good results, but it lacks that context. It lacks that enterprise knowledge. Using a vector database or, or baseline RAG, you, you can get better results where now it's actually pulling in organizational knowledge, but the answers tend to be a little bit generic.

91. 18:11

    There's a lot of hallucinations. Um, GraphRAG kind of pulls us to the end of the spectrum where now you're, you're getting answers From that, that knowledge graph you built, you can evolve over time and much more precise answers, which actually get to the heart of, of real problems in, in life sciences, in manufacturing, in business-critical industries where

92. 18:34

    you can't afford to be wrong

93. 18:36

    And also where in, in industries that are complicated, if there are a lot of connections that might not appear in a relational database because no one bothered to make the joins permanent, whereas in a graph, those joins are there to begin with.

94. 18:49

    So if you search for one thing, suddenly the neighborhood of related stuff becomes available to you to share with an LLM for better contextual knowledge

95. 18:58

    Yeah. And, you know, I think just if, if folks are implementing this or folks are thinking about how to, how to think about architectures for GraphRAG, um, this is a really simple way of thinking about it.

96. 19:08

    So basically what you're doing is you're taking your GenAI application and you're doing both a vector and a knowledge graph representation of the data. So you're both asking the vector for the answer, you're getting relationally close nodes from the graph database where you're getting additional context and passing that into the LLM, and then this gives you more

97. 19:32

    contextually relevant results coming out of your, your expert system. So I think this is a great way to use a knowledge graph, either that you built up over time or that you have the LLM construct to kind of get those superior results where you can do better governance, you can put controls and properties on the graph nodes

98. 19:50

    to control who has access to the information.

99. 19:54

    You can get better explainability now because when you're getting an answer from the LLM, you're no longer looking at statistical probabilities in the vector space, you're actually looking at graphs and nodes and edges, which we can reason about, and we can start to understand the relationship between, understand like what, what things are related to manufacturing, which things

100. 20:15

     are re- unrelated to that. They're just, you know, general terms

101. 20:19

     Mm-hmm

102. 20:20

     And for the right application, maybe we're saving lives, getting drugs to people more quickly, and using GenAI for a good cause. So thanks so much for joining us for our presentation, AI Engineering Summit, and, um, appreciate everybody.

103. 20:38

     Thank you. [clapping] [outro music]