Context Graphs for Explainable, Decision-Aware AI Agents

Talk transcript

1. 0:00

   [upbeat music] Hello.

2. 0:15

   Good evening, developers.

3. 0:18

   Good evening.

4. 0:18

   How is everyone doing?

5. 0:20

   Good.

6. 0:20

   Welcome to AI Engineer, and welcome to the Neo4j, uh, Context Graph session number two. Excited to be here together with my peer, uh, ABK, to cha- share with you more about context graphs, how they can help AI a- agents to become, uh, more decision-aware.

7. 0:38

   So, um, one of the big challenges, uh, that we are trying to solve these days in the AI era is basically, um, using knowledge graphs to unlock AI, where we want to use graphs to fill the gap of knowledge, give, give AI agents the right tools, and en- enrich them with the right, uh, con- um, content,

8. 1:03

   uh, to do. To, to, uh, to, to solve certain, uh, challenges. So we all know that AI agents are really good in, in language, reasoning, and creativity, but we want to miss, uh, or fill the, the, the missing puzzle piece of, of knowledge and, um, um, for, for the agents.

9. 1:30

   So... And big, um, topic we have been-- we also saw before at Steve's talk is, is memory. So, uh, to, um... We talk about different types of memory, so short-term, long-term, and reasoning memory.

10. 1:45

    So in a short-term memory, we, um, we try to, to understand and capture the, um, the, the conversations that a user had with, with a AI agent. And in a long-term, uh, memory, uh, we, we capture more, um, contextual, uh, knowledge, in this case, um, things around or- organizations, uh, people,

11. 2:10

    and things, basically. So, uh, that really give us more the generalized overview of, um, um, of the, of the context. And of course, we need here reasoning, helping basically the AI to, to, to, to make the right decisions to, um, achieve a certain task.

12. 2:30

    We saw also that, uh, at the, um...

13. 2:36

    The, the-- For, for the background of this, we need, uh, we, we need a graph. So, um, a graph basically is made out of nodes, uh, and relationships to basically help, um, help you understand the deep and complex, uh, relations and connections between different type of data.

14. 2:55

    So the big question is now why context graphs? So we also have been talking also lately a lot about context engineering, and now basically the big ask again, why, uh, context graphs?

15. 3:08

    So basically, we are still within the sphere of context engineering when we talk about context graph, but context graph is really the shift of having, um, AI agents that already are very good in, in providing knowledge, um, to, uh, to their users, having the right tools and content and context.

16. 3:28

    But with context graphs, we would really like to, um, provide additionally to the knowledge, um, the right rules and policies, um, to the agents to help basically them become more capable to, to, to, to driv- uh, to drive, um, uh, decisions.

17. 3:48

    So not only, only knowledge, but also, um, decisions. So that's why we're moving not only, okay, how an agent or what an agent can do, but also now we want to really capture within a context graph the missing why.

18. 4:02

    So why an agent needs to do something, and this is something we capture with data that focuses on policies and rules.

19. 4:14

    So, uh, when we talk about Neo4j, um, we say also that, uh, graphs are everywhere. So this is a basic, uh, replica of an organization. Um, each organization is basically also reflected here from finance, uh, or department of an organization, from finance to product to suppliers, and so on and so forth.

20. 4:35

    We saw also a good example by Steve before on, um, on financial services. So, um, if someone is ba-- is eligible to, to get a certain amount of money here, so yes or no.

21. 4:47

    So we, um, part of this chapter, we are really going deeper with our customers, uh, and, and verticals to understand their needs and basically how we can help them solve the why question and help them drive better, better agents that can, can be more decision, um, decision-aware.

22. 5:07

    Um, this is also, uh, how a memory graph, uh, would look like. So if we want to, to, um, an agent to also drive the right decisions, we need to also, um, give them memory capabilities.

23. 5:23

    So this is a quick overview how the memory graph would look like. In green, we see the short-term memory, so we are capturing again the, the, the conversations, the, the state history.

24. 5:34

    And then in long-term memory is more like, uh, the generalized overview of, of organizations, people. Um, and then most importantly, also the, the reasoning. So why an AI agent should, should, um, um, do a certain task based on, on, on predefined policies and rules.

25. 5:56

    So, and behind, behind this story is always, uh, our, let's say, the foundation, if we would like to build agentic GraphRAG applications. Um, we always have the, uh- The AI agent, uh, where the user sends, uh, first a u- a query.

26. 6:15

    Um, we-- the agent looks for this knowledge, uh, or for this, um, topic if it's available in its, uh, knowledge source. If not, we, uh, jump to the graph database.

27. 6:26

    We have a certain, um, asset of tools like Text to Cypher that translates text, human, human text to our, uh, query language, and then we tra- uh, we, yeah, uh, traverse the, the graph on the right content and hopefully coming back to the user with more, uh, reliable and qualitative content.

28. 6:48

    So, uh, Abhik, you have been building, um, a decision framework in the last couple of days. Let's have a look on that.

29. 6:55

    Sure. Thank you, Zaid.

30. 6:56

    Thank you, Abhik.

31. 6:57

    So we've been hearing a lot about, like, what does the data model look like for how do you store memories? And we're talking about context graphs, and it's all about decision-making.

32. 7:06

    Why do we really care about decision-making? And maybe you noticed the awesome animation. How many people have a claw? Claw owners, some of you. Some of us have claws these days, right?

33. 7:16

    At least you've heard about claws. What's awesome about OpenClaw and all the dif- different claws is they really exaggerate the need for good decision-making, where decision-making is basically when you run into a p-point in the workflow or the life of the agent where it has to do something that it doesn't have in its setup.

34. 7:34

    You haven't given it instructions about what to do in a certain circumstance. Autonomous agents run into this all the time. If you just let them off and go and do stuff for free, give them a credit card, give them access to your Amazon account and say, "Hey, keep my stock f-for completely-- my, my fridge stocked with, like,

35. 7:50

    Red Bull at all times." Okay. If they notice that you need some Red Bull, they're gonna go order some Red Bull. They may not know, like, "Oh yeah, I should order Red Bull unless the rent is coming up, and I don't have enough money for rent."

36. 8:02

    You may not have anticipated that. You can go and fix that with some prompt engineering and keep improving the instructions for the agent. But really there's a meta problem, which is: how do you make good decisions?

37. 8:12

    I'm gonna talk through, given if you have some memory, if you have a graph around, you've got an agent running, what does an agentic workflow look like for actually making good decisions?

38. 8:23

    Because the crab wants help. Help the poor crab.

39. 8:28

    Okay. And actually, it's even worse, of course, if you've gone down this path at all. You start with maybe one agent that's running. That becomes multiple agents that have specialized tasks.

40. 8:36

    They're all off doing things. They all have to collaborate together. They all have to be self-aware of each other and themselves. The problem gets worse and worse at scale.

41. 8:45

    So you need an, a thoughtful framework for how to make decisions. If you have friends who are in MBAs that, that went through some business school, perhaps here in the Judge Business School in, in the UK, they probably have entire courses on how to make good decisions.

42. 8:59

    I'm gonna talk through a framework that you can give to an agent. You can implement yourself and your choice of framework, whether it's LangGraph or if you just wanna write some skills, you can give that a try as well.

43. 9:08

    But this overall workflow is just how you make decisions generally as a human. And as we know for much of the engineering we do these days, transferring our implicit understanding about how to do things in an explicit way to the agents is what helps agents do their job really well.

44. 9:24

    So given a context graph, given some memory, here's a workflow for actually making good decisions. Okay. Starts at the top with just framing the problem, basically. It's the local context that matters.

45. 9:35

    And for the local context, there's how did you get here in the first place? That's the, the causality of the second one, actually. I would probably lead with that.

46. 9:44

    You went through some reasoning chain, you went through some actions, and suddenly you're at a point where, like, you have some amount of uncertainty, and rather than just making a choice about what to do next, you're like, "Hang on, let me go into this sub-process, which is decision-making, and figure out what to do."

47. 9:57

    You've got an objective in mind, something you wanna get out of that process. And the example we had before for finance was, like, actually instigated by somebody who wants to increase their loan or something like that.

48. 10:06

    Fine. Could be that, could be ordering Red Bull, whatever the decision might be. An objective, the causality that led to having that objective be a thing you wanna resolve, and then there's an environment within which that's actually operating, that's purchasing, that's actually perhaps medical decision-making, guidance in that realm.

49. 10:24

    So it goes from, "Hey, can I order Red Bull?" to, "There's a life at stake." Those are very different environments, but those environments matter for, like, what kinds of decisions you end up making.

50. 10:34

    That's the kind of thing that we understand because we experience these things. You have to tell the agent these things so that they understand it as well.

51. 10:42

    Once you've got the framing, that feeds into a larger context, which is the global context around what did you do before in this situation? That's always a good idea.

52. 10:50

    If you can stay consistent, that's really good. But then also there are global rules, both the hard and soft rules of a business. Maybe they're formally described in a business process language or something like that, or they're just informal in some kind of guidance in Slack channels, in Google Docs, whatever.

53. 11:05

    Hard and soft rules come to la- place here for having alignment over the decisions you're making. And it's important to keep both of those things in balance. What did you do before?

54. 11:13

    Could be the right thing to do again. But also, what are the global rules that are applying now that might've been different before? So there might be a reason to overrule what prior decisions were.

55. 11:22

    These are all part of the framework. You've got the inputs. You've got a framework, uh, for the decision-making that is global, and now you've actually gotta go through and do the analysis.

56. 11:32

    And this is, of course, just the classic risk value analysis. And you're gonna figure out for the risk side, that's probably where an agent should spend most of their time.

57. 11:40

    If it's a serious thing, then you spend even more time. And by serious, I would mean things like this first point is called a reference class validation, where a reference class is, uh, rather than assuming what is good and what matters, you have to try to decide for the players involved, what is the most important thing for

58. 11:57

    them? And so what is your reference point for, like, what's important to them? The example I love here is, like, if you have, uh, in, in medical care, this is a very dangerous area to be doing-- letting OpenClaw loose, right?

59. 12:10

    If you're su- prescribing drugs for somebody- And like 99% of the time, you might prescribe drug X for symptom Y, and that's the right thing to do 99% of the time.

60. 12:20

    But for the 1% of the time, if you're in the small 1% of the population, giving that same drug might be fatal. So before deciding what you should do, it's very important to know if you're part of the 99% or the 1%.

61. 12:32

    Statistical, you know, behavior does not really help you there. That's an extreme example, but there's flavors of that in many decision-making circumstances that right now are kind of implicit for us.

62. 12:42

    A lot of our practice as AI engineers is being explicit about the implicit knowledge that we carry with us. So keep that in mind, actually classifying what matters, what doesn't matter, what is the actual risk involved.

63. 12:53

    Is the decision reversible? It-- Can you take it back and be like, "Oh, sorry, that was obv- obviously a bad idea. Let's just back it off and, and do it again"?

64. 13:00

    That changes the dynamics of how you assess things. And then also, what's the cost of being wrong generally? Again, like, is a life at stake, or is it the, "Okay, I'm gonna not have Red Bull in my refrigerator"?

65. 13:12

    Not great, but not the end of the world, so it's okay if that's a bad decision. Then also, of course, on the value side, it isn't obvious what you're maximizing.

66. 13:22

    Are you trying to maximize your saving your budget, so actually don't spend all your money on Red Bull, save some aside because you're saving for a vacation or for buying a house or whatever it might be?

67. 13:31

    What is it-- Are you maximizing some value, or are you trying to minimize some cost? That has to be part of it as well, and it ha- has to be explicit.

68. 13:38

    Without that, an agent is gonna take its general knowledge of conditions and be like, "Okay, well, most of the time this is the right thing to do," but the particulars matter.

69. 13:47

    The particulars always really are important. And then the final thing is that the output of this stage is actually not making a decision. I like to think in terms of multi-agent systems.

70. 13:55

    For me, I love compartmentalized, highly focused agents working together. This agent's job, only thing would be to, like, actually come up with a proposal of some alternatives and to give some pro/cons for those alternatives, not to actually make the choice.

71. 14:09

    The choice then would be handed off to the next agent,

72. 14:13

    which would be actually deciding whether or not it can act. Does it have the authority? And if it does, it can go ahead and, like, rank the, the s- the options that it has available to it and take those actions and have the impact actually be felt.

73. 14:25

    Or it can decide, "Actually, I don't have certainty here," or, "I don't have authority here, so let me escalate this to either another agent or another actor that could be a human that does have the authority, that can make the decision."

74. 14:37

    So this is incredibly important here. Either act or don't act. Let somebody else make the call. And the oversight here is basically the sub-process that would be kicked off for, like, a human in the loop basically to step in or an agent with higher privileges.

75. 14:51

    The final part of this is really an important self-learning part. Once you're done, once the decision has been made, either a decision has been made or there was not enough information or certainty to make the decision.

76. 15:02

    You might defer actually doing anything. That's a possible state as well. Or you make the decision, and if you had, record the entire reasoning process, what you considered, what you didn't consider.

77. 15:12

    This is part of the tracing, the accountability that we talked about in all of the other moments around context graphs. That all gets saved into the graph, along with the decision itself and the actions that were taken.

78. 15:23

    That lets future agents do better at their job because they have this now as precedent that they can refer to.

79. 15:29

    Okay, that was my speed run. This is the full diagram of the workflow. You can implement this again if you want to in LangGraph, in 80K, whatever you've got on hand.

80. 15:38

    Write up some skills doing this. That works well. I will say it's very hard to generalize each of these things. This is a general framework. The particulars of every step end up being very domain specific, so bit of, bit of a caveat there.

81. 15:50

    Um, we're obviously working on, like, lis- lots of different examples in different domains for this stuff. So if you've got a domain that we haven't really talked about yet, I'd love to talk about it so we can add it to our kind of catalog of things.

82. 16:00

    Okay. Very quickly, that was me. Call to action is the same as Steve had as well before. If you wanna learn more about this stuff, we have free online courses at GraphAcademy.

83. 16:10

    You can learn more about context graphs, learn more about graphs. Come talk to us at the booths. Come talk to us after this talk. Uh, we love to talk about this all day long.

84. 16:19

    Thank you. [audience applauding] [upbeat music]