Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes Leone, AWS

Talk transcript

1. 0:00

   Hi. Today, we are going to talk about how to stop AI agents' hallucinations with five techniques beyond the prompt. Each one is a code change, not a prompt change.

2. 0:16

   Let's see. Every time your AI agent responds, you are paying for the words going in and the words coming out. In your bill, you will see those calling tokens.

3. 0:30

   Yeah. And the more tokens you send in, the more you pay. And if what you send is not quite right, too much or missing something important, your agent start to hallucinate. [chuckles]

4. 0:47

   There are five techniques to help reduce tokens waste, improve accuracy, and catch failure before users see them. And each one is a code change, not a prompt change at all.

5. 1:05

   So let's see it. First, we have semantic tool selection. You filter which tool go into context on every call. The model only see what is needed for that specific query.

6. 1:23

   Second, we have GraphRAG for persist queries like aggregation, counts, multi-hop reasoning, and you replace the text retrieval with a structured graph query.

7. 1:38

   The model gets a compute, uh, verifiable answer, not a sample, as RAG do.

8. 1:47

   Three, multi-agent validation. A second agent can check every response before it reaches the users. And four, neuro-symbolic guardians. You rule live in Python, not in the prompt, and the model cannot skip them.

9. 2:09

   So five, runtime guardians, because if you don't want to block, you can steer, and you don't c-- you don't need to block everything. And when a rule fires, the agent self-correct and complete the task.

10. 2:26

    No hard stop, no user retries. So for each technique, I will show you the agent without it and then with it so you can compare. And all the demos are...

11. 2:40

    I'm using a travel agent that I built using Strands Agent. And Strands Agent is a open source agent framework that we maintain on AWS.

12. 2:56

    And I am Elizabeth Fuentes Leone. I am a Developer Advocate for AWS. I'm focused on agentics application. And here in this QR code, you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment.

13. 3:21

    So let's get into... Let's get into it.

14. 3:32

    So semantic tool selection. Our travel agent has 29 tools: flights, hotels, payments, weather, cancellations. All, all are dummies tools, are not like a travel agent for real.

15. 3:50

    But every time that user sends a message, all the 29 tools description go into the context windows.

16. 4:00

    The model reads all of them before deciding what to do. And if your agent has memory, the rows to every conversation adds more context that gets sent with every single message.

17. 4:20

    And you pay for every single one of those tokens, whether the model ends up using that tool or not. To understand where those tokens come from, you need to see what a tools actually looks like to the model.

18. 4:39

    In Strands, you write a function with the tool decorator, that's the tool, and a name, a description, and a docstrings typed parameters. Then Strands take that and generate a schema with name, description, parameters, and the schema is what goes into the context windows on every

19. 5:04

    call. Each tool schema is about seventeen or two hundred tokens, depending on how many parameters it has. If our travel agent has twenty-nine tools, that adds up to somewhere around three thousand tokens per call just for the tool

20. 5:29

    description. Before your message, before the response, every single call. By creating a tool database, we can filter the tools that the agent may needs before the agent is invoked.

21. 5:48

    With this filter, the models sees only three most relevant tools.

22. 5:55

    Tokens usage drops from thousands to fewer than three hundred. Let me show you in the code Here in my QID. This is the... Let me clear all the output.

23. 6:12

    So first, we need to install the requirements. Here is the requirement file. This is a Jupyter Notebook because it's more simple to show everything, but it is one application that you can run if it is more comfortable for you.

24. 6:27

    So here are the requirements. I have the Strands agent, and because I'm using OpenAI as a model invocation, I'm using the API from OpenAI, and I need a Strands agent for an OpenAI.

25. 6:41

    You can use Open-- a Strands agent with, oh, my God, almost all the model provider and with a Strand- with Amazon Bedrock, of course, because we as AWS, we are the maintaining of this framework.

26. 6:55

    And to use Amazon Bedrock, you don't need to add, uh, the model provider. And because I need to embedding, I need to create embeddings for my vector tools database, I'm using the sentence transformer.

27. 7:12

    This is a super simple model that runs locally, so it's free. So you want... You can run this locally in your computer without expense another model embedding. And, eh, you can use Strands with Ollama too.

28. 7:28

    So if you have a local model, you can run everything in this, in your computer for free without spend any tokens. And I use infise as my vector store.

29. 7:42

    It's local, super simple. And well, right now I'm gonna use Neo4j. This is for other, eh, demo that I'm going to show you in a moment. And because I'm using some environments, I'm using Python of.

30. 7:56

    And let's see. So here, I'm going to run this. Let me put it bigger. I know that you're having some problems. Put bigger. Little mer, little less.

31. 8:07

    And here, this is better. So I install my requirement. I already did that. And I'm using OpenAI, so I need my a-my API, my API key, API key. And here, I invoke my Strands.

32. 8:23

    I need the agent because I'm going to build an agent. I'm using OpenAI, and I have a bunch of tool, dummy tools here. I le- I'm going to show you in a moment.

33. 8:33

    And I'm going to use this because I have some, eh, functions that I'm using to, uh, create, build a index for my vector store, and I need a search tools.

34. 8:46

    When I use the vector store, I put my query there. I search for the, I search for the tool that I'm going to use using vector, um, um,

35. 8:55

    search for vectors, and I swap the tools. I'm going to show you that in a moment. So let me show you all my dummy tools. Where is the engage?

36. 9:05

    Register. Here. Here I have all my tools. Uh, this is for swap, search, build index, and where are my tools? Choo, choo, choo, choo. Here. So here are all the dummy tools I create for this.

37. 9:21

    You know, it's super... This is a demo, please. This is not something that you can use to go to production. No, please. And okay. So what is this? This one.

38. 9:32

    So I run this, and I build my semantic index. I already did that by, uh, yeah. So I have twenty-nine tools here, twenty-nine dummy tools, and I'm going to test this with a lot of different queries, and this is the ground truth.

39. 9:49

    Eh, I have a ground truth because I, I know what is the best, um, the best tool to answer the question so we can know if our agent is okay or not okay.

40. 10:00

    So let's run this. We have, eh, ninety queries and twenty-nine tools. So this is some helper functions. I don't care. Yeah, I care, but I'm not going to explain you that.

41. 10:13

    So this is my agent, my traditional agent, where I'm putting all my twenty-nine tools inside the agent. I have my model, some helper function. This is my agent. I only...

42. 10:26

    This is the way I create a agent using a Strands agent. I put all my tools, and my system prompt is, "You are a travel assistant. Use the correct tool to answer question."

43. 10:36

    Oh, my God. Super. [chuckles] And the model. Let's run this. And this is, uh, whatever. This is some, eh, function to count the tokens because in-inside the Strands agent you can, you can count the tokens.

44. 10:51

    You, you can know how many tokens are using for in and out the, um, the agent. This is some helper function. And yeah,

45. 11:00

    this is every time that I put a, uh, a prompt inside this agent, I'm only sending the prompt and the agent respond. Then I, I use a new agent because this is a, uh, a for, so I'm not having a conversation with the agent.

46. 11:15

    I only said, uh, a question and I receive an answer. So for each question is spent around two thousand tokens. So in the end, not always it give me the right answer here.

47. 11:31

    So yes, I have, um, a not so good accuracy and the average is one thousand tokens. Now, let's use my new agent with the semantic approach.

48. 11:44

    The thing, the first thing that I do is for every single query, for every single, eh, prompt, I send the query to my search tools and I'm going to receive the top Q, the top K three, uh, most relevant, um, with more probability to response my query because this is a semantic search inside my vector store.

49. 12:09

    And- Then I use that response, I'm going to select that name, and I'm going to put that tools inside my agent. So I only going to use the three tools that this semantic search retrieved me for my agent, and I send that only tools.

50. 12:28

    And this is my function helper, and that's it.

51. 12:32

    So I am going to send the, all the queries again the same. And for the first question, I have 4,000 tokens to read blah, blah, blah. And yeah, we have a huge difference because I don't send them all the 29 to- tools in all the queries.

52. 12:51

    So yeah, pam-pam. And okay, semantic memory. Yeah. Okay, now come on, finish. So well, let's, let's, let's go to the next one. So this agent is only sending a question, and I receive a response.

53. 13:08

    I don't having a conversation with this agent. What happen if I start to have a conversations, if the agent remember me? So we need to, we need to send only the tools that this agent is going to use because if I give the, all the tools I'm using in the conversation history, uh, then I...

54. 13:28

    It's going to be in a moment that I'm going to have the 29 tools inside my agent. So we are not resolving, like we are not having a, a, um, we are not resolving our problems in the agent when we're having a conversation with the agent.

55. 13:44

    So we need to put the tools, and then we need to remove the tools, and we can do that with the swap_tools. And let's run this. And what is doing swap_tools?

56. 13:58

    I will run this. No. Let me, let me show you first the swap_tools

57. 14:04

    function here in the register. I delete this because with Strands,

58. 14:11

    in each invocation, you have complete control of the status because this is a agentic loop, and you can use this agentic loop

59. 14:23

    how or... Because this is an agentic loop, and you can do almost whatever you want. You can put and remove everything inside the loop with some few lines of code.

60. 14:40

    So let's see. Here we have the agent. We have the tool register through the tool registers. It's something inside the agent status. So we can clear the tools, and that's it.

61. 14:55

    So in the next invocation, we can clear the all tools, and we can add the new tools.

62. 15:03

    Here, where is my... I have a lot of things here.

63. 15:09

    So I run this. It's going to take a moment.

64. 15:28

    And yes, we can see that in each invocation, my amount of token is increasing. Why I have more? Because I'm sending the chat history as well. So I send the tools that I need, only the tools that I need, and the chat history.

65. 15:46

    So that's why I can see that, uh, my amount of token is bigger and bigger.

66. 15:59

    And that's it. The accuracy is better. And yeah, we have more tokens.

67. 16:08

    So yes, probably we don't have the best accuracy here because this is a super demo with super dummy tools, and some queries are ambiguous on purpose because search for something, book for something, check something, and the demo has a generic tools, dummy tools with some similars name.

68. 16:33

    And when all the 29 tools are visible, the model sometimes pick the wrong one. And with the filtering, those generic tools, uh, only appears when the query actually match them.

69. 16:48

    So you can run everything, and you can test this. So this is only the run only local. But what happen when you want to put this demo in production?

70. 17:01

    You can, of course, you can build a bigger vector store. I don't know. You can use Postgres. Yeah, I think it's too much. But we on AWS, we have Amazon Bedrock Agent Core.

71. 17:14

    So Amazon Bedrock Agent Core is a service dedicated only to agents in production. And inside of Agent Core, we have Agent Core Gateway. So Agent Core Gateway, Gateway allows you to build this index, uh, alone.

72. 17:36

    So you only have to say, "Hey, this is my tools," and Agent Core Gateway is going to build everything for you. And it can have the, the vector search inside.

73. 17:48

    So the routing layer is inside the Agent Core, and it handles the tool selection automatically. So you register your tools once, and it find the right one for each request.

74. 18:01

    It's the same principle but without infrastructure to manage. Now let's see the next one.

75. 18:11

    GraphRAG. You know, RAG is Retrieval-Argument Generation. Is how agents access your own data. You take the user question, search your documents for the most similar content using vector search, and pass what you find to the model.

76. 18:31

    The model answer from that, it's worked well from open question, "Find me something about this topic," but there is a category of question where that breaks down. What is the average rating across all hotels in Paris?

77. 18:48

    How many hotels have a pool? Vector search always returns something, even when nothing is truly relevant, and the agent only sees the top N chunks of your old data at a time.

78. 19:06

    It cannot aggregate, count, or traverse relationship across all the full dataset, so it estimates, and it present that estimate as a real fact, you know? A real answer.

79. 19:22

    So here we have RAG. You build the vector store, and it retrieve three chunks from three hundred documents, and the model guesses.

80. 19:34

    If we are using GraphRAG, you can run a query across all the data and returns a compute results. Graph address this differently, because instead of retrieving text chunk, you can build a knowledge graph from the documents, nodes, relationship, structured data.

81. 20:00

    For the demo, I'm going to use Neo4j locally, and the model is going to write a Cypher query to search it. Cypher query is Neo4j query language. It's similar to SQL.

82. 20:16

    So the graph run that query across all the data. The model gets back a compute verified results, not a sample. And before running this demo, I'm... we need to install the dependence.

83. 20:32

    Let me show you. Let, let's go to the code.

84. 20:38

    So graph. This is the notebook that I want to sh-share with you, so we need to install the requirements again. What we have here, let me see.

85. 20:48

    This is the first one, yeah. Requirements. I have... I'm going to use OpenAI again. Let me just close this. I'm going to use Neo4j, and we need Neo4j for GraphRAG.

86. 21:01

    And for the, uh, agent that is using normal RAG, we are going to build a super simple, uh, vector store in Faiss, and we are going to use again the sentence transformer.

87. 21:15

    And let's see. I already have all. I invoke my OpenAI.

88. 21:21

    This is my tools. I, I... because I want to create some tools here, I'm going to use OpenAI, and I'm using graph database. And this is my... I have my Neo4j locally.

89. 21:34

    And let's run this. And this is something to check Neo4j. So I'm building my, uh, vector store. This is my Faiss, and this is my Neo4j that I have here locally.

90. 21:47

    I don't know if I can show you. I think not. And I have a tool, a normal tool created with Decorator. This is to search inside the, my vector store.

91. 21:58

    You see that super... I, I have the query. I co- I create, I dec- I create the, the embedding for my query. Then I search inside the vector store, and I have the query for the knowledge graph.

92. 22:10

    So here the model need to understand that to search in the query, in the graph knowledge base, it need to build a Cypher query. So I put that in the context because we already know how to build, uh, tools, right?

93. 22:26

    I put that in the context. I have my driver to send the data to, uh, to connect with the vector store and to send the Cypher query that the model is going to create for me, and this is to read the results, and that's it.

94. 22:41

    That's the only tool that I need to search in my knowledge graph. Okay. This is my model OpenAI. This is my RAG, my RAG agent, and this is my graph agent.

95. 22:52

    I have two different agent to compare the results. Let's see what, uh, what happen. Yeah. I'm ready. So let's do... Let's run the first test, aggregation.

96. 23:10

    Here, what is the question? What is the average guest rating across all hotels in Paris? So meanwhile this agent is, is... Oh, it's already done. So what I have here,

97. 23:23

    I have the average guest rating across the hotels listed in Paris is, um, it calculates. You know, when something, when I'm using RAG, it go to the, to the vector store.

98. 23:36

    It receive the N, uh, possible answers for this question. Because this is an aggregation, it's going to use the data that it receive to build a mathematical, uh, operation.

99. 23:50

    This is the thing that I have here. So everything that we see here is the, is the model reasoning. So when I run the graph agent... Oh, let me...

100. 24:01

     The agent is only going to give me the answer, you know, only these tokens here. Because the Cypher query, it can give me the mathematical operation itself. It can do that.

101. 24:13

     So we don't need the agents, the LLMs, the model, uh, do that for me because the Cypher query already give me the right question. So it is forty-seven and the average.

102. 24:27

     So here in this, um, first, we are lucky Because probably they are, yeah, only two hotels. But what happened if this vector store have more than two or three hotels in the, in the vector store.

103. 24:44

     So we are not going to give a real answer. It's going to c-- the LLM is going to calculate with the only three answer that it received. So it's going to build this operation with only three hotels.

104. 24:59

     But if the vector store have more than three, we are going to have some problems. Something that, um, and I, um, yeah. So let's run the next one, the precise counting.

105. 25:13

     Something similar to how many hotel have a swimming pools as a amenity? How many again? So it give me the, the traditional, "It's appear that the search did not returns any specific information about hotels in Paris."

106. 25:30

     Okay, what happened with the other one? "There are currently no hotel that offer swimming pools." No, why? There is no hotels. So the other one is like, "Mm, would you like to ask about hotel with other specific amenity or information?"

107. 25:46

     It's like, mm, maybe it is or maybe I don't know. Yeah. It's not so accurate, okay? So multi-hop reasoning.

108. 25:58

     Oh, what is my question? What are the room types and price for the highest-rated hotels? What are the rooms types and price? These two question. So it search the fact I had to have only one.

109. 26:14

     "The higher-rated hotels in any company Paris," blah, blah, blah. I, I don't speak French. With the guest rating is that one. Whatever. I currently don't have... Okay, blah, blah, blah.

110. 26:26

     A lot of data there. So what happened with the other one? Receiving notification for... Oh, I have error. [chuckles]

111. 26:35

     Uh, here in the second one I received the, the question. "The higher-rated hotel is Harmony," blah, blah. "They offer following types. But unfortunately, these room are not available." So I receiving a que- a answer.

112. 26:49

     I don't receive a lot of blah, blah, blah there. I only receive the answer what I need. And for the next one, out-of-domain detection.

113. 26:59

     What is the question? Tell me about hotels in Antarctica. A spoiler: there is no hotel in Antarctica. There is zero hotels in Antarctica. And let's see. "It's appear that the search did not return specific information about hotel."

114. 27:14

     Okay, because there no. Ah. "As such, currently do not have detail," blah, blah, blah.

115. 27:20

     If you are looking for particular type experience or specific inquiry about visiting Antarctica, no, please let me know and I can assist you for it. Okay, a lot of tokens that is pending there in that answer for LLM.

116. 27:34

     What happened with the other one? "There are currently no hotel listed in Antarctica." Of course, because it create the cyber query, the cyber query, and it receive zero. So it no, it give me a honest answer.

117. 27:49

     Here's some summary that, uh, Claude create for me for this in Jupyter Notebook. But here is something that I want to show you before go to the other one.

118. 28:00

     So here, something that I love for Neo4j, because why I'm using Neo4j? Because in the library,

119. 28:09

     the Neo4j give me, uh, it can build, it use a LLM, I'm using the OpenAI as well, to build a knowledge graph. So how I build my knowledge graph, I only have a bunch of data, a bunch of TXT, only text, and I send that data to the Neo4j library that I miss it.

120. 28:36

     Here, I send that data to this, and Neo4j using this all, this, um, library just right here, it can understand all my data, and it can build the graph for me.

121. 28:51

     So I don't need to create that using the simple Knowledge Graph Pipeline inside the Knowledge Graph library. So, uh, that's why I'm using Neo4j. It's amazing and it's super simple to use, so I invite you to use that.

122. 29:08

     So let's go to the next one. So multi-agent validation.

123. 29:17

     Sometimes a agent fails and nobody find out. It calls a tool, the tool returns an error, and the agent does not surface that error. It generate a confident success response instead.

124. 29:33

     The user think it work, you think it work, it did not. The agent acts and validate its own output in the same loop. There's no separation, no second opinion, so when something goes wrong, it rationalize and tells you, "It's okay, it work."

125. 29:55

     Here is what happens. Inside a single agent, when it fail, it calls the tools, gets an error, rela- re-rationalize it, and returns a success response. The user never sees the error.

126. 30:14

     You can address this by adding a validation layer. You can have three agents in sequence. One acts, one checks, and other approve or rejects.

127. 30:30

     Strands Agent has a built-in class for this called Swamp. It manage the handoff between agents automatically. You just define the role of each agent in a system prompt.

128. 30:48

     And let me show you that. So this demo only needs a Strands Agent with OpenAI integration. Let's see the requirements here. We only need a Strands Agent. We don't need anything more.

129. 31:04

     Let's go to the notebook here. And the key important thing here is the... I already run this. Let me see here, is the swarm. The swarm is what lets you to connect multiple agents together without create like a for or while manually to put the all these agent

130. 31:29

     together. It can build the chain and manage the handoff between them automatically. So here, we are going to create three agents. Let me go that because we have the normal and we have some ground true data.

131. 31:47

     So here is to create a, a single agent. So we have... We are going to test three different scenarios to valid a booking. They sa-- they we expect true.

132. 31:56

     To unavailable hotels, we expect false. And no existing hotels and missing booking, we expect false too. This is how we create a single agent. We need a prompt, some tools.

133. 32:08

     And we are using the tools that we have here

134. 32:12

     and here. We know that this is the data, so the agent can, can give us the answer that we are looking for. And let's see. Let me go to the other ones.

135. 32:23

     Uh, wait. Here. So how we build the swarm.

136. 32:28

     I already run this. I want to show you the swarm. So we are going to build three different agents: the executor, the validator, and the critic.

137. 32:37

     The-- We have a system prompt. You are an executor agent for a hotel booking system. Use the provided tools to fulfill requests accurately and blah, blah, blah. The validator is you are a validator agent.

138. 32:52

     Review what the executor did and output exactly on/off.

139. 32:58

     And we have the critic that is going to say yeah, approve or not yet. Let's run this. And oh, what is here? Oh, model. Didn't define the model. Sorry.

140. 33:10

     Sorry, please forgive me the life because I didn't run this one.

141. 33:17

     Let's run, uh, because this is the, the normal and the model is there, right? What? Build path, yes. Run that.

142. 33:29

     What's happened with you? Let me go this. Let me copy and paste this one. I don't know why this give me problem.

143. 33:39

     Multi-agent swarm. Is it... I'm the same name. Come on. Yeah, you can see this is life. I don't going to edit this. [chuckles]

144. 33:48

     So yes. Okay. We have... Mm. Thank you. And the swarm. How we, how we create the swarm? We have swarm, and the swarm function, we put all the agents together, and the entry point is the executor.

145. 34:05

     So all everything is going to start in the executor, and then it's going to hand off

146. 34:10

     and six time because you can handle that too. So get the swarm final response. Let's do a, a for and let's send all the response. And book around hotels.

147. 34:22

     I have booked around hotel for Alice tonight. Booking blah, blah, blah. Hand off. Valid. Okay. Let's go. Verdict, approve. So the critics is approving the this. This is the other one.

148. 34:38

     And we can run some comparison here if you want.

149. 34:44

     So here we can see that the swarm handles the flow between them, between all the, uh, three different agent. It was what happens when the single agent tries to confirm something that does not exist in the system.

150. 35:00

     And now the same request through the swarm, the executor gets the error, the validator catches, the critic reject this, and the user never see a fabricated response. So you can...

151. 35:14

     We can test here with a single agent that it's, uh, substituted on no entry and return success.

152. 35:23

     And the swarm, it have a executor that got the error, validate, say, "Hey, come on, man," hallucination, and critic, they reject. So the user sees a clear failure, the fabricate confirmation, so missing the user here.

153. 35:41

     And you can run this to compare the two different agents. So here we can see that the single agent, it have everything like, uh, yeah, it's okay. It's correct.

154. 35:54

     But at the multi-agent swarm, you can see that the last one is saying, "Mm, you know, missing issue here. Uh, something is happen." Now, let's go to the next one.

155. 36:06

     Okay. Neuro-symbolic guardians. You have a rule for your agent. Let's, uh, say maximum ten guesses per reservation. You write it in the system prompt. You even write it in the tool description.

156. 36:22

     And the agent still calls the tool with fixed ten. Not because it is ignoring you. Because prompts probably are suggestion, not constraints. The model process them as a text No as a logic it has to execute.

157. 36:42

     It's probabilistic. Only code execute logics. A rule in the prompt, the model reads it as a suggestion. A rule in the code, the model cannot skip it. Neuro-symbolic guardians rules put the rules in the code.

158. 37:02

     Strands Agent has a feature called hooks. The f- the function that Strands calls automatically at specific moments in the agent loop. It is this case right before a tool executes.

159. 37:18

     So you have a hook, you write the rule, check the parameters, and if they fails, you cancel the call. Let me show you this in the code.

160. 37:31

     Here we have the Jupyter Notebook. We have the requirements here. Um, we have the same requirements that the previous demos. We only need... I will start everything again. We only need Strands and OpenAI.

161. 37:45

     We don't need anything else. And, um, my API.

162. 37:53

     And here is the thing important. In this demo are these three hooks provided in the Strands base class, so you s- that allows you to create the hook. So we have hook provider, hook register, and before_to_call event.

163. 38:10

     And the tool regi- hook register is what Strands pass you to register your callback, and before_to_call event is the event that fires every time the model is about to execute a tool.

164. 38:27

     That last one is what makes it possible they intercept the call before it runs. So you have the before_to_call event and of course we have after_to_call event that we are not going to use that here.

165. 38:42

     And let's see. The rule in the prompt, remember, the model reads as a text, and it may follow or not follow that. So let's see this. I'm going to run this.

166. 38:55

     So here we have some simulated state that we are going to use in this agent. And let me... This is the symbolic rule. So booking rule. If, if rule...

167. 39:08

     Booking rule as a... This is something that

168. 39:11

     we create. Let me show you the rules here. So we have the rules. Rule one, validate dates. Check is month before checkout equals validate date checkout. This is something that it check.

169. 39:27

     This is, uh, something that is going to invoke when the rule is, is, uh, is most... We have to use it. And,

170. 39:39

     uh, we have another rule for max guests. Maximum tech guests per booking. So if I want to use, I don't know, if I want to booking, uh, uh, I want to do a booking for 11, it's going to block me.

171. 39:56

     It's going to reject the booking. And we have some confirmation rule, payment before confirm. You know you can't confirm if you don't have the payment. Cancellation rule. Cancellation within...

172. 40:10

     cannot cancel within 48 hours of check-in. So this is something that I've created here, you know, to this demo. So I have booking rules and I have confirmation rule.

173. 40:21

     I don't use in here the cancellation rule. So create validation hook. We create the validation hooks here, right, like this. Neuro-symbolic rule. And we use the hook provider. So we have a bunch of code here that we add the booking rules, the confirmation rules in the state.

174. 40:45

     And you can check this by yourself later, but we want to see the demo running so that we define the clean tools that we are going to use the- for the booking hotels and the presents payment.

175. 40:58

     This is the, the normal tools, you know. And we need to add the tools to the hooks. I think I add that.

176. 41:07

     Yeah. Tool name, book hotel. So this thing is going to get triggered when the book hotel is using the...

177. 41:19

     Yeah, the book hotel. Okay? So let's create the agents for the comparison. So we are going as the other demos. We have three different scenarios. Confirm booking without payment.

178. 41:33

     Payment must be verifiable for the confirmation. This is the rule that it must trigger this. We have booking hotel exceeding guest limit, and we have valid booking for five guests.

179. 41:46

     So yeah, we have the three scenarios, and we have the normal agent, the normal agent, the baseline agent, and we have the agent with the neuro- the neuro-symbolic guardians.

180. 41:58

     So we have the hook, the neuro-symbolic hook.

181. 42:03

     Uh, this is the same here. Yeah. And

182. 42:07

     we add this. If you can see, the normal agent is only three lines, the tools, the model. And this is the line that give us the difference between these two agents.

183. 42:19

     So yeah, really run that. Now let's go to

184. 42:25

     confirm a booking without payment. So I'm running this, my agents. The first one is obvious. Come on, man. I... Yeah, because I... Yeah, it confirm the booking without payment because it doesn't have any rules and the prompt is super basic.

185. 42:41

     And the other one is Let's open this, and is booking blocked. Payment must be verified before the confirmation. Thank you. So it's okay. Now let's run the other one.

186. 42:57

     Test second scenario. What is the question? Booking hotel excess guest limited. So the question is... Ah, yeah, I have more than, so

187. 43:12

     here, uh, the hotel have successful booking for... Yeah, because the prompt is basic, and I don't put in any rule there. And for the other one, it seems that the Grand Hotel has maximum capacity of 10 guests per booking.

188. 43:25

     Additional booking must be made at least one day in advance. I don't remember the day. Probably this is something that I give them. And to validate booking is both agent execute all rule pass because it, yeah, it's only a validation for booking.

189. 43:42

     So book hotel for blah, blah, blah, all rule passes.

190. 43:47

     It seems that the booking still needs to make... Ah, I don't... I didn't remember the question. So booking is made provide... Ah, yeah. Okay.

191. 43:57

     Okay, I get it. And okay. Now run all that in this question. Confirm without payment, booking safe max, and what happened here? This is all the scenarios that we just run, so this is, uh, something to compare the results.

192. 44:12

     So valid booking kinsa ka, uh, five guesses. Allowance, correct. Wrong,

193. 44:19

     50. Blocked, wrong. Correct, confirm. So the... We have here the comparison between the two agents.

194. 44:34

     So what we are, what we have here is same model, same tools, same prompt, and the different the outcome because the rules are in Python knowing the prompt. This pattern of enforcing rules in code before the tools run is also what Amazon Agent Core Policies service that we

195. 44:58

     have does at infrastructure level. So the same concept, but managed for you in production, and you only have to create the rules. So the hooks are all or nothing.

196. 45:12

     They block everything or approve. But sometimes you want the agent to adjust and keep going, not stop and leave the user waiting. That is what I will show you in the next.

197. 45:32

     Runtime guardians, runtime steer, steer don't block. This is the next one. Hooks blocks unconditionally. The agent stop, and the user has to retry.

198. 45:49

     For a hard constraint, that is exactly what you want. But sometimes the rule is softer. Maybe room fits for four guests, but a group of six could go book two different rooms.

199. 46:08

     Or a flight is full, but there is availability on the next one. You do not... You don't want to block everything. Probably you want the agent to find a option and complete the task.

200. 46:23

     That is steering. Here we have the hook that fires and the task fail. And with Agent Control steer the models and the task complete. The other difference here is operational because with the hooks it's as changing a rule means changing code and redeploy the all harness,

201. 46:48

     all the agent. And with the Agent Control, we use the name of the open source library that we are going to use here. Rules are registered on a local server via API.

202. 47:02

     You update them without touching the agent code because the agent picks them up immediately. Let me show you that here in the code.

203. 47:14

     Yeah, this is the notebook. In this demo, we only need a extra packet, the

204. 47:23

     Agent Control SDK. So the Agent Control is the one that helps us to create the steering. And here we are using the setup control. This is a, a little application, a little app that I create with the local server and the steering rules.

205. 47:46

     Here we have the local server. We have the control. There are the steering rules. First, we have the,

206. 47:54

     the steer max guests. Steer, you know, guide. So guide agent to reduce guest count when exceeding maximum of 10,

207. 48:06

     and it's going to steer. And we have some control that deny. For example, deny no payment. Block, blocking confirmation without PR payment.

208. 48:20

     And here, well, this is to, um, to create the, um, the service, the server. And let's go to the, uh, notebook. So I already... Well, let me go here.

209. 48:36

     Yeah. So I need the environment. So this is the agent. Uh, we're going to have error here. Wait, wait, wait. I don't want to use Bedrock. Why? Why? No, let me comment this one because this is going to give me a error.

210. 48:57

     And let's run this. Yeah, so we have the hooks as we did in the demo. So we have book any company in Lisbon, and we have some prompts. You are a hotel booking assistant.

211. 49:10

     When booking first, describe what you will book and blah, blah. And this is the prompt for my agent. Where is my agent? Agent control. This is for the... Well, this is some, some helper.

212. 49:23

     And this is my hook that you already know because we create this. So we have the system prompt, and we have the hooks. So let's test this agent with the, uh, book any company Lisbon for 50 guests.

213. 49:38

     And if you remember, this only can book for, uh, uh, less than 10 guests. And of course, it's blocking. Now, let's go to the agent control new agent.

214. 49:54

     Here we have two key importance or two key imports that are important for the agent control SDK. So we have the agent control plugin that captures agent events and sends them to the agent control server.

215. 50:11

     And we have the agent control steering handler that listen for a steering decision for the server and delivers them back to the model.

216. 50:21

     Together here, they are what connects Strands to the agent control steering logic. So

217. 50:31

     let's go this with this. So the steering agent

218. 50:39

     is going to... Uh, we book a room for any company Lisbon, uh, 50 guests. Let's see what it's doing. Yeah, I have successful booking for a stay any company Lisbon for 50 guests for May.

219. 50:53

     So it's the reservation have been split into two rooms. So it took that it itself, you know, one room for one and other room for five, and that's it.

220. 51:07

     So use hook for hard constraints, agent control for softer rule. Now, we have five technique all running locally, but how do you take this to production without maintaining service, without building infrastructure?

221. 51:29

     Let me show you how. Everything that I just built here in the previous demos runs locally. For the production version, Amazon Bedro- Bedrock... Amazon Bedrock Agent Core give you the runtime, a gateway, a short-term and a long-term memory, a CloudWatch observability

222. 51:54

     built-in, no servers to manage. Here the architecture. The Strands Agent runs inside the runtime, and you know, inside the runtime you can put every framework that you want, not only, uh, run, uh, Strands.

223. 52:10

     The gateway routes tools calls to the Lambda function automatically as a tools. And the steering rules from the previous demo live in the DynamoDB. So you change them there, and they are live on the next call.

224. 52:29

     And you don't need to deploy anything. And you want to use Neo4j? Of course, you can use Neo4j AuraDB. That is a external graph database. We also has a free tier.

225. 52:44

     And the code is in the repo here.

226. 52:51

     It's in the repo. And you will need AWS credential if you're using Amazon Bedro- Bedrock Agent Core. And in the resource link, there is some credits. I hope that you can find it because I always try to h- to give away some credit for AWS, so you can deploy everything for free.

227. 53:15

     And, uh, if you are new here in AWS, I have a repository that you can use to deploys everything of this architecture, uh, using a notebook too. But if you are familiar with CDK, Cloud Developer Kit, you can do it that as well to deploy everything at once.

228. 53:36

     Uh, both option are in the repository, of course, and you can go deeper in Agent Core with all the documentation that I left there.

229. 53:45

     So please don't stop at the demo and try going to production with Amazon Bedrock Agent Core.

230. 53:52

     So let me bring it all back. Tokens waste on every request. You can fix it with semantic tool selection.

231. 54:04

     And confident answer, it's never computed. Possible... It's possible because you are asking how many. So you can use GraphRAG and query the data. Don't sample it. Sometimes we have fabricated success confirmation.

232. 54:22

     You can use multi-agent validation and second pass corrected. And rules the models are query skipped. You can use, uh, neuro-symbolic guardians to enforce in the code.

233. 54:39

     Don't trust in the prompt. And for the hard blocks that stop the user, you can use runtime steering, self-correct, and finish.

234. 54:51

     Each demo that I just showed you is in the, in the repository as a notebook and an application as well. Start with the demo one, then go to the demo five, and if you want, deploy it into productions.

235. 55:07

     So have you tried any of this in your own agents?

236. 55:12

     Thank you to join me in this session, and happy building.