Your Moat Is Your Data Model

Talk transcript

1. 0:00

   [upbeat music] Yes, my talk today is about the title, Your Data Model is Your Moat.

2. 0:16

   We have a enterprise-wide platform that we had just rolled out here this past month, and so I'll go into details on this. I'll give you some, hopefully some practical lessons here and why we made decisions we made for this, uh, how, how you could picture your processes within a similar type framework.

3. 0:35

   So first, just a, a quick introduction. So this gets into the, the title here of the talk, and the, the framing of, you know, what I hope you take from this.

4. 0:43

   But with AI moving very fast at the frontier, what, what's defensible? You know, you can move, you can build things very quickly with Clo- Claude Code, um, but once you push things to production, there's constraints.

5. 0:59

   You know, you find how much of your, your, uh, deployed stack do you wanna actually own? You know, there's monitoring, there's upkeep, there's, uh, you know, people building dependencies off your stack that you have to be prepared to handle.

6. 1:11

   How much, uh, appetite do users have for decentralized access? This, this gets into... Uh, I'll show you what, what we built, but, you know, what, what's, what's the access point for users?

7. 1:21

   You know, is it another chat app? Is it Claude? Is it ChatGPT? Uh, is it something else? What's your product differentiation from, from those different SaaS products? And so our team then, you know, with this context in mind, you know, thought through here, you know, what, what's our skill set here?

8. 1:39

   What's our competitive advantage in this environment? And

9. 1:43

   this is what I, you know, really hope that you, you take from this talk and, you know, picture yourself in this. But our moat here was our understanding of our internal processes, the tacit knowledge that you need to, to run successful AI.

10. 1:58

    And, and this is true, I think, no matter how good AI gets, how good models get, you know, new releases that different companies put out when, when, uh, Mythos comes out or when there's a new app from Claude.

11. 2:08

    Yeah, I'm not, I'm not worried because the part that we've built is the defensible part that, that's, that's durable.

12. 2:16

    So these are the... I'll, I'll, I'll tell you what this means here in, in, in more detail, but these are the, the processes, tacit knowledge that we've modeled into what we call the Strategic Intelligence Platform, or SIP.

13. 2:26

    And it rolled out here this past month in production for enterprise use across, uh, f- the Gates Foundation, so about four thousand people.

14. 2:36

    So first, uh, I know this is an engineering talk, but the, the, the scope of this talk gets into data modeling, internal operations processes, and so I wanna give very quick background here over what the Gates Foundation does because then this is what we're, we're modeling.

15. 2:52

    So as you're probably familiar, the, the Gates Foundation has a, has a very wide scope and it's a very ambitious work that we've been doing for the past twenty-five plus years.

16. 3:02

    And there's all kinds of, you know, broad initiatives that we're doing, you know, whether it's for, uh, child mortality, whether it's for nutrition, agriculture, uh, education. And these are kind of broadly the, the different buckets that these different initiatives fit, fit into.

17. 3:19

    Creating market incentives, spurring innovation, collaboration between public and private sectors, and then the fourth one here kinda gets into the, the lens that we're building here. You know, high-quality data, trying to derive data-driven insights from the, from the actual investments, the, the grants that we've put out.

18. 3:39

    And over twenty-five years, there's a ton of structure, there's a ton of data that's developed, and trying to extract those insights at scale i,i- is difficult, and that's what we're trying to solve.

19. 3:52

    So this slide here is a snapshot of the, of some of the different, uh, of the work that went out in twenty twenty-three within the foundation. Th- this gives you an idea.

20. 4:02

    I just put this here to, to show some of the structured, the, the structure that we have that we're, we're working across. So you have over two thousand grants in one year.

21. 4:11

    So ma- many of these are five million plus. Uh, many cou- hundred-plus countries that, uh, that, that are targeted with these grants. Uh, alumni, so there's four thousand different employees of the foundation.

22. 4:24

    Um, you know, many different strategies within the foundation. The US, within the US, across al- almost all the states. Grantees. The total annual disbursement, over seven billion dollars.

23. 4:37

    And so this gives you some idea of structure that we're, we're working with. And, uh, this one, just finally here, when I show the data model, this will make more sense, but we have different divisions.

24. 4:46

    That, that funding goes out through different divisions. And so this breaks down some of those divisions so you can see different priorities, and it'll, it'll make more sense in a second here.

25. 4:56

    But global development, global health, uh, gender equality, USP are just a sample of the different divisions.

26. 5:05

    Okay, so the, the fun stuff here now, I hope. The, uh, Strategic Intelligence Platform, so

27. 5:12

    i- in a, in a, in a nutshell here, structuring operational data for agentic retrieval. So we're building a knowledge graph with the idea of the agent consumer.

28. 5:24

    And here is an end-to-end look of what this looks like. So we, we have different systems of record, structured, unstructured. These have been siloed traditionally.

29. 5:36

    The... So part of our team here, the work has been to create what's essentially a da- uh, data lakehouse, putting everything under one roof. This is our internal enterprise-wide data.

30. 5:47

    It's also different, different programmatic data that are, uh, uh, outputs of different investments. Once it's there, it's easy for us to consume, so we have a data curation layer that does different processing to it.

31. 5:59

    And then finally, SIP here at the end with, uh, agentic chat, agentic workflow as the-

32. 6:04

    As the, the U- the UX, how users are consuming our platform. And so it's a cross-system semantic graph layer that agents can reason, reason across.

33. 6:14

    Okay, uh, so some of this I'll try to speed through here just for the sake of time, but th- this one is critical.

34. 6:20

    When you're dealing with systems of record with lots of complexity, engagement is critical. This is something that we've, we've found here repeatedly. We have to engage data owners to understand, you know, this tacit knowledge we're trying to, to model.

35. 6:33

    What's the full meaning of different fields, the structure of the dataset? How do we join things together? How do we, uh, understand limitations, systematics of the data, safeguards, security trimmings, uh, reporting conventions?

36. 6:45

    You know, it's not enough just to answer a question a certain way. You have to answer it the way that it's been answered in the past. And so this is the-- comes back to the moat here.

37. 6:55

    This is the procedural understanding, tacit knowledge that AI needs, and it's, you know, it's the part that we, that we own, that's, you know, that's ours. That, um, and that's what we're, we're modeling here.

38. 7:09

    Okay. So going back here just very quickly for this one, this is the, a snapshot here of different data curation considerations that we're-- that go into this pipeline. So you have, for different datasets, whether it's structured, unstructured, there's different preprocessing, filtering, deduplication.

39. 7:24

    There's an order to different documents. There can be, uh, inconsistencies across documents. Those need to be, uh, handled up front. There's extraction, so structured field extraction, semantic chunking for unstructured documents.

40. 7:36

    If you have figures, you need to convert this into text in some way so you can do retrieval across this. Uh, various forms of tagging that th- these can form connections in your graph.

41. 7:46

    Structured metadata that you create during this pipeline, and that becomes different properties in your graph. And then the third bucket here, governance, and this is a, an important one that, uh, I think AI makes more acute.

42. 7:59

    Things that were, that were accessible previously, they're much more accessible now with, with AI, and so you have to consider this. Your, your risk sphere is, is larger. So things like PII need to be masked.

43. 8:12

    You need to reconsider different, uh, sensitive data, classifying this, um, making sure that, that there's the right entitlements for each user who's accessing your system.

44. 8:23

    Okay, so that's the overview here. The, this, the, the data model itself. Now, this is the part I'll walk through here.

45. 8:30

    There's a, a nice animation here, but hopefully the takeaway is you can picture your own, your own organization's story within what I show here. I'll get somewhat technical, but it's only to hope- hopefully to give you an idea of how we, how we solved our problem, and then you can hopefully, uh, model this to yours as well.

46. 8:48

    Graph is a very flexible, um, practical representation of a physical model.

47. 8:54

    Okay, so I'll zoom through a few of these here, but the-- just the, the entry point here, we have over eighty different strategy teams. These teams have annual reviews that happen.

48. 9:04

    This is how the budgeting for each year is derived. And then so we model this here in the graph. The, the, the meetings are where unstructured documents, uh, enter into this system from, but then there's-- they have a structured connection to your other systems of record.

49. 9:19

    Um, what I show here is a conceptual data model, so it's flat. So you're not seeing instantiation. You have the actual graph. There's many different nodes. Cardinality, is it one-to-one, one-to-N?

50. 9:29

    So the actual graph, it's, you know, even more complicated. But

51. 9:34

    for the data model itself, let's-- let me, let me show you the first different hi-- So we have multiple hierarchies that exist within what we've modeled. The-- There's different types of hierarchies you can have.

52. 9:43

    In this case, this is a, hopefully you can see all this very well, but it's a, it's an, it's a, um, additive DAG. So there's a-- all five levels here of this hierarchy from the top to the bottom matter.

53. 9:59

    So you have to consider everything together. And so then there's different roll-up patterns you can do to work across this, this sort of pattern. In our case, we have a in path shortcut here that connects the funding path.

54. 10:12

    Uh, funds to BAU is where we have the, um, the budget for each of these different funding teams that, that's stored.

55. 10:24

    So we have funding. What's the, the internal funding teams have portfolios. These portfolios then go towards different investments. Multiple funding teams fund an ind- individual investment, so it's a end-to-end relationship there.

56. 10:38

    The investments are the thing that are our product. It's our, it's our, our business. But internally, we have funds that then prioritize different, different types of investments, and that's what's shown here.

57. 10:50

    And so you can take this down to the transaction level, or you can have different, uh, annual-based aggregations that you map here as well.

58. 10:59

    And then from investment, there's a lot of interesting things you can do. You can map to all the different organizations, and you can have different types of organizations, and there's actually a lot here that is still kind of a green space that we wanna fill in.

59. 11:11

    We have all these different observables that people have produced in the investments that we wanna model here. So published reports, products, you know, all this stuff is structured and connects to the entire, uh, organizational picture.

60. 11:27

    So I mentioned that there's different hierarchies. This is the second type of hierarchy. At this hierarchy, each level matters in and of itself, and so it's not a, a DAG necessarily, and so you can actually do things like pre-computing the, the-- some of these, these different shortcuts.

61. 11:44

    So the hierarchy, it goes from the top to the bottom, contains, connects it. This is showing the investment management side of the, of the organization. And there's concepts of direct team management, so one team at, like, team level two manages the investment.

62. 12:01

    But then there's also a concept of indirect management, so that the, uh, children below team level two still should be attributed to the team level two. And so there's different things you can, different games you can play with these sort of roll-ups to pre-compute.

63. 12:15

    I don't know if you can see this, but roll-up Manages Inv is a, is a, a derived edge that we, that we create after we create the Contains and Manages, Manages Inv edge.

64. 12:28

    So I've shown two different, two different lenses for one investment. There's the funding lens, the management lens, and you can model both of these here then within, within the graph.

65. 12:38

    A third hierarchy here is people. You have organizations, you have org charts, and you have people who are owners, you have people who are attendees in meetings, you have people who are, uh, directors.

66. 12:48

    Th- there's all kinds of different roles they have. You can model these here. You can have their, their, uh, you know, who they report to, what their, uh, team structure is.

67. 12:57

    And th- these all are structured data that connects across systems. Traditionally, they existed in a, just a HR source system, but they're relevant for the context of the, the full story.

68. 13:11

    And then that leads to this connectedness. So we have different source systems that were siloed. We-- To understand the entire picture, for the agent to understand correctly across the structure, you need to find these common ba- these common, uh, these common, uh,

69. 13:29

    entities that you stitch together. And so that's what's shown here. These are different source systems, but they're related quantity, entities that exist there. And now the agent can traverse here and understand this pretty complicated organ- organizational structure.

70. 13:49

    One last part here that I haven't shown yet is the, the document part. So... And this is still, there's, there's a lot more we can do to, to this part.

71. 13:56

    We've just been, uh, ingesting one different document source so far. But this is where you combine unstructured and structured, and this gets into part of the magic here that you can model with Neo, Neo4j.

72. 14:08

    But we have meetings that have documents, documents that have different semantic sections that you can, or chunks that you can, uh, that you can model here. You can put full text indexes across these to, to aid in the different, uh, search and retrieval approaches for the agent.

73. 14:26

    There could also just be a pure graph retrieval that, that the agent does. And then all these things then connect back to your, your, your main organizational structure.

74. 14:38

    So then as a whole, th- this is what the data model looks like. So I've been zooming in here. Now you can see the, the full interconnectedness of this.

75. 14:45

    Four different systems, one graph, uh, one semantic layer that's exposed through an MCP then to the, to the agents.

76. 14:56

    And so this is the-- So if you think of the agent's perspective, this is the, the structure that it can dynamically discover and reason across at query time. And for the developer, it's also a very cool thing because it exposes, you know, what you don't know about your, your, the thing you're modeling.

77. 15:12

    You know, you very, very soon you find out that there's a gap in your understanding or there's some data set that you're not, you know, fully including. And so this, this process in and of its, in and of itself is very valuable.

78. 15:26

    Okay. Let me give you a sense here what we do with this now. So this is the-- I showed you the platform, the graph, but then how does this relate to AI?

79. 15:34

    So we've, we've connected this through MCP, and I, I, you know, I discussed earlier what the, what's durable, what's defensible. To us, what was not defensible was the, was the, the chat interface, was the UI, and even in some cases, the, the general chat cases, the, um, you know, the, the agent interaction.

80. 15:54

    And so we, u- users themselves are in Claude already or ChatGPT, and so we serve the platform where they are. And so it's served here now through MCP. Here's an example, just kind of a innocuous, uh, question here.

81. 16:09

    But Neo4j has some off-the-shelf, uh, MCP servers here. We, we've actually modified these quite a bit here. We forked it, and then there's various updates to the schema, uh, things to, to pass state back to the, to, to our system.

82. 16:25

    You know, the, the conversation, uh, IDs, the, the message, uh, numbers, stuff like this we, we, we've, we've modified in these MCP tools.

83. 16:35

    But so that's a general chat experience. That's one entry point. The other part that we're building right now too, that's very exciting, is more constrained workflow experiences. And so these can also be offered through things like CoWork or, uh, Claude Chat.

84. 16:48

    And you can do things like, um, you can have your, you can have MCP apps be the, the, you know, the standard entryway that users access. So, you know, different UIs that are, uh, ported into your, your, your, your chat experience.

85. 17:02

    And you can have different sandbox-based agents that then run the, the workflow. And so these are, these are active things that we're working on. It helps to constrain the experience compared to chat, but it, at the same time, it pulls from that same knowledge graph-based, uh, backend platform.

86. 17:23

    Okay. I've got a couple of minutes. I'll, kind of speed through this. But the, the, the way evals then relate to data modeling is that a- as you're doing evals, you find, you find gaps, you find ambiguities in your data model.

87. 17:36

    You find ways in which users are asking questions that, uh, that are ambiguous or it's, um, you know, not, it's not returning things that conform with the reporting standards.

88. 17:47

    So what, what we've done here then is we've worked with data owners. We've d- we've, uh, built targeted, uh, eval questions that, that they, that match their reporting standards.

89. 17:59

    We've separated these into different complexity tiers. One challenge is that the, the structured data is constantly changing, so we have to have the graph query itself that we, that we create for each of these different questions.

90. 18:11

    And then at runtime for the evals, we, we pull from the live graph, and then we compare that to what the agent is delivering for that question.

91. 18:20

    And so that's what's shown here then. And there's a feedback loop that you can do for this. So as you're running, uh, an eval pipeline, an eval structure pipeline, you have an LLM as a judge.

92. 18:28

    We've modeled things like pass@1, uh, stability. So if you ask the same question multiple times, you get the same answer back. You can use LLM-as-a-judge to, to, to measure this.

93. 18:39

    And then there's a feedback loop here that you, you can update then your, your data model, you can update your, uh, your domain rules, your, uh, schema descriptions to help, to help, uh, fill those, tho- those gaps that you, that you find.

94. 18:58

    Then after you do this, th- this is, uh, this is just some, some, some eval reporting here that we, uh, we show the pass@1 and the, the stability for our system.

95. 19:08

    So we- we've gotten this very, ver- very strong. The, the questions that we, that we end up do missing, it tends to be things that are ambiguous in some way.

96. 19:15

    And so it's not wrong, it's just that it's things that

97. 19:20

    might be right, but not what the user intended. So that, that's kinda the constant struggle that we, that we, that we're working around.

98. 19:27

    30 seconds here. What's ahead for SIP? So we're continuing, continuing to fill out our existing, uh, data from system, systems of record, so things that fit into our current data model.

99. 19:36

    We wanna expand the primary graph to additional enterprise-wide datasets. There's a lot of, there's a lot of demand for a federated graph experience. So we have a, a main enterprise system, but we have specific teams that have their own data that they wanna link to this, and so we're working on how to do this.

100. 19:53

     Uh, different agentic experiences, like I mentioned as well.

101. 19:58

     And that's it. Uh, so yeah, please, if you wanna ask questions, if there's things that you wanna talk about, I'll be out back, or you can add me on LinkedIn here and, you know, keep the conversation going.

102. 20:08

     Thank you. [audience applauding] [upbeat music]