Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes Leone, AWS

Every agent response has two token costs: the context sent to the model and the answer it generates. Irrelevant context increases the bill; missing or poorly selected context can also leave the agent answering without what it needs. Reliability starts before the model produces its first word.

Five interventions change what happens around that response: select relevant tools, compute answers through graph queries, validate execution independently, enforce business rules before tools run, and steer recoverable requests toward a valid outcome. These intervene at different points in the agent loop, from context assembly to execution and recovery.

Elizabeth Fuentes Leone, an AWS Developer Advocate, demonstrates each intervention with a travel assistant built using Strands Agents, the AWS-maintained open-source agent framework. The companion repository provides the examples behind the walkthrough.

The travel assistant has 29 dummy tools covering flights, hotels, payments, weather and cancellations. In the baseline, every message carries all their descriptions into the model’s context, whether the request needs them or not. Conversation history adds another growing source of input tokens.

A tool is more than a Python function from the model’s perspective. Strands takes the function’s name, description, docstring and typed parameters and generates a tool schema. That schema is what the model reads when choosing an operation. Fuentes Leone estimates roughly 3,000 tool-description tokens per call for the full registry, falling below 300 when only three relevant tools are supplied. These are illustrative schema-overhead estimates, separate from total usage across an agent run.

The semantic-selection demo uses a Jupyter notebook, with an application version also available. Strands calls an OpenAI model through its provider integration and an API key; Bedrock is another supported path. Sentence Transformers creates embeddings locally, and Faiss stores the tool vectors. Ollama offers a local model option without hosted inference-token charges. The registry and its index-building, search and swapping helpers are demonstration code; the travel tools themselves simulate operations.

The spoken evaluation uses 90 queries against 29 tools, with an expected tool assigned to each query. The baseline exposes the full registry and gives the agent a simple instruction to use the correct tool. Each question gets a fresh agent, so this comparison excludes accumulated chat history. Strands usage accounting tracks input and output tokens; the live narration does not establish a consistent total-token savings figure or a precise accuracy percentage.

The filtered version moves tool discovery ahead of agent invocation:

1. Embed the incoming question.
2. Search the tool index for the top three matches.
3. Resolve the returned names to callable tools.
4. Construct the agent with those tools and send it the question.

For a normalized Faiss index, the selection boundary can be expressed directly in Python:

from collections.abc import Callable, Mapping, Sequence

import faiss
from sentence_transformers import SentenceTransformer


def select_tools(
    query: str,
    encoder: SentenceTransformer,
    index: faiss.Index,
    tool_names: Sequence[str],
    registry: Mapping[str, Callable],
    k: int = 3,
) -> list[Callable]:
    if index.ntotal == 0:
        return []
    query_vector = encoder.encode(
        [query], normalize_embeddings=True
    ).astype("float32")
    _, positions = index.search(query_vector, min(k, index.ntotal))
    return [
        registry[tool_names[int(position)]]
        for position in positions[0]
        if position >= 0
    ]

The tool descriptions used to build the index must use the same encoder and normalization. The demonstration reruns the same questions with the smaller tool set, so the changed variable is the available tools rather than a rewritten task prompt.

A fresh agent per question avoids a second problem: tool accumulation. In a conversation, continually adding the latest search results can eventually put the entire registry back into context. The demo’s swap_tools helper replaces the active set instead. It clears the registry’s tool collections and installs the newly selected tools before the next invocation, while preserving the conversation messages. It is a locally defined helper around Strands state, not a built-in method to assume exists on every agent.

This keeps tool exposure bounded, but it does not make the whole context constant in size. The conversational run still sends growing chat history, and its token usage rises. Fuentes Leone reports improved tool-selection accuracy while explaining why the test remains difficult: some questions deliberately blur searching, booking and checking, and several generic dummy tools have similar names. Filtering removes some of those competitors unless the query actually matches them.

For deployment, the same discovery step can use a larger self-managed vector store, such as Postgres, or Amazon Bedrock AgentCore Gateway semantic search. Gateway provides managed indexing and discovery for registered tools. The current documented flow requires enabling semantic search and invoking x_amz_bedrock_agentcore_search; it returns relevant tool definitions for subsequent use. Discovery and execution remain separate operations, even when the indexing infrastructure is managed.

Vector retrieval is useful for an open question such as finding information about a topic. It is a poor basis for answering “What is the average guest rating across all hotels in Paris?” unless the retrieved context contains every relevant record. A nearest-neighbor search that returns three chunks from 300 documents has selected a small subset, not established a complete population for counting or averaging. Without a relevance threshold, it can also return nearest matches when none actually answer the question.

The hotel GraphRAG demonstration replaces that sampling step with a structured query. Documents become nodes, relationships and properties in local Neo4j. The model generates Cypher, and the database executes it over the matching records. The database performs the aggregate; the model explains the returned result. Correctness still depends on the graph representing the source accurately and the query expressing the intended question.

The comparison uses two agents with different tools. One tool embeds the question using Sentence Transformers and searches Faiss. The other describes the graph-query interface to the OpenAI model, accepts generated Cypher, submits it through a Neo4j driver and returns the records. This separates language interpretation from the database operation.

The first question asks for the average guest rating across all Paris hotels. The graph returns an average rating of 4.7 for the demonstration’s Paris hotels. The notebook shows the retrieval-based answer calculating the same average from ratings of 4.9 and 4.5, while the graph answer presents the computed result directly. In this small case, retrieval happens to include the needed hotels; once the matching population exceeds the retrieved subset, the same calculation no longer answers the all-hotels question.

The next tests exercise different consequences of that distinction:

• Precise counting: Asked how many hotels have swimming pools, the retrieval agent says it did not find specific information. The graph agent reports no matching hotels. The difference is between missing evidence in retrieved text and a computed empty match in this database.
• Relationship traversal: Asked for room types and prices at the highest-rated hotel, the graph path must first identify the hotel and then follow its room information. The live run includes an error or notification before returning a hotel answer and reporting unavailable rooms; it does not establish a clean, complete price result.
• Out-of-domain requests: Asked about Antarctica, retrieval produces a longer uncertainty response. The graph query finds no matching records, allowing the answer to say there are no hotels listed there in the demonstration database.

The shorter graph responses illustrate the benefit of returning computed facts rather than asking the model to perform arithmetic or fill in missing relationships. No measured output-token comparison accompanies these examples.

The graph itself is built from plain text files with LLM-assisted extraction. Fuentes Leone passes the documents through Neo4j’s Simple Knowledge Graph Pipeline, using OpenAI to help extract graph structure instead of manually constructing every node and relationship. That makes graph construction easier, while leaving the extracted structure as another part of the system whose fidelity matters.

A tool can return an error while the agent tells the user that everything succeeded. The failure is especially hard to detect when the same agent both performs the operation and judges its own response: a missing booking can become a confident confirmation. An independent validation pass checks whether the reported outcome follows from the execution evidence.

The demonstration divides the work among three roles, with Strands Swarm handling handoffs rather than a manually written orchestration loop.

The roles are supplied through system prompts. The test data includes a valid booking and failure cases involving unavailable hotels, nonexistent hotels and missing bookings. Only Strands with its OpenAI integration is needed for this example.

The executor is the swarm’s entry point, and the demonstration configures a six-handoff limit. Alice’s Grand Hotel booking passes from execution to validation and then approval. For a missing or nonexistent record, the executor encounters the error, the validator catches it, and the critic rejects the result. The user receives a clear failure instead of the fabricated success produced by the single-agent comparison. This is a check on the response after attempted execution; it does not replace enforcement before an operation with side effects.

A maximum of ten guests per reservation is an executable condition. Writing it in a system prompt or tool description only gives the model text to interpret. A pre-tool validation hook checks the actual proposed arguments and cancels the invocation when they violate the condition. This is the neuro-symbolic pattern in the demonstration: language-based planning combined with deterministic checks at the execution boundary.

Strands hooks provide the interception point. A HookProvider registers callbacks through HookRegistry; BeforeToolCallEvent fires before the proposed tool executes. AfterToolCallEvent is also available but is not used here. The callback examines the tool name, arguments and relevant state before deciding whether execution may proceed.

The rules operate on simulated booking state: check-in must precede checkout, a booking may contain at most ten guests, and confirmation requires verified payment. A cancellation rule forbidding cancellation within 48 hours of check-in is defined but not used in this run. The core booking and confirmation predicates can remain separate from the framework callback:

from datetime import date


def booking_errors(
    check_in: date, check_out: date, guests: int
) -> list[str]:
    errors = []
    if check_in >= check_out:
        errors.append("Check-in must be before checkout.")
    if guests > 10:
        errors.append("Maximum 10 guests per booking.")
    return errors


def confirmation_errors(payment_verified: bool) -> list[str]:
    if not payment_verified:
        return ["Payment must be verified before confirmation."]
    return []

The hook routes a proposed book_hotel call to the booking checks and a confirmation call to the payment-state check. An eleven-guest proposal fails before the booking tool runs. Keeping these predicates outside the model makes the enforcement decision inspectable and repeatable.

The comparison tests confirmation without payment, an over-limit booking and a valid five-guest booking. The added hook configuration is the intended difference between the baseline and guarded agents. The baseline’s basic prompt permits confirmation without payment; the guarded response blocks it. The over-limit case similarly produces a guarded response citing the ten-guest maximum.

The final notebook table exposes an additional failure that the narration’s description of the valid case does not capture. In the displayed three-case run, baseline accuracy is 1/3 and guarded accuracy is 2/3; the guard blocks both invalid operations but also rejects the valid booking. Enforcement must be evaluated for valid requests as well as invalid ones. The code can reliably apply a rule while the overall configuration still prevents work that should be allowed.

Fuentes Leone connects this pre-tool enforcement pattern to AgentCore Policy at the infrastructure level. The next problem is what happens after a rejection: binary approval or cancellation is appropriate for some constraints, but other requests have a valid alternative that the agent could find without making the user start again.

A room that accommodates four cannot hold six guests, but two rooms might satisfy the request. A full flight might have a suitable alternative departure. In the preceding hook configuration, rejection stops the attempted operation. Steering adds a recovery path: send the reason and corrective guidance back to the model, then let it propose an admissible alternative.

The Agent Control demo also moves rule management out of the agent’s Python source. Its SDK connects to a local control server, where rules are registered through an API. Updating those rules does not require editing and redeploying the agent. Two controls demonstrate the distinction: steer requests above ten guests toward an acceptable arrangement, but deny confirmation without prior payment.

The concrete request is to book AnyCompany Lisbon for 15 guests. The hook-only baseline blocks it because a single booking exceeds the limit. The steering integration introduces two components: AgentControlPlugin captures agent events and sends them to the control server; AgentControlSteeringHandler receives steering decisions and delivers them back to the model. Together they close the feedback loop between a rejected proposal and the agent’s next action.

For that same 15-guest request, the local demo reports a successful reservation split into two rooms: ten guests in one and five in the other. The original over-limit proposal is not itself a successful booking; the reported success follows the adjusted arrangement. Hard constraints still govern each operation, while steering gives the agent a way to preserve the user’s overall goal.

The production architecture moves the local agent into Amazon Bedrock AgentCore Runtime. Runtime can host Strands or other agent frameworks, with short-term and long-term memory and CloudWatch observability supporting operation beyond a notebook.

Gateway routes tool calls to Lambda functions. Steering rules live in DynamoDB, so the proposed architecture can pick up rule changes on the next call without redeploying the agent. Neo4j AuraDB supplies the external graph database option; Fuentes Leone mentions its free tier. The deployed policy and its recovery behavior still need to be checked independently: the current production example describes reducing a 15-guest request to ten, whereas the local demonstration preserves all 15 guests through the two-room split.

AgentCore deployment requires AWS credentials. Fuentes Leone points to possible promotional credits and describes notebook and AWS Cloud Development Kit deployment paths. The inspected repository documents CDK infrastructure deployment and a notebook for local testing, so use its current CDK instructions for provisioning rather than assuming the notebook deploys the infrastructure. Credit availability is separate from the architecture’s operating costs.

The five techniques answer different diagnostic questions. If every request carries irrelevant tool schemas, filter before invocation. If an answer requires a count or aggregate that was never computed, query the dataset. If execution errors become success messages, separate execution from validation. If a business rule must hold, enforce it before the tool runs. If rejection leaves a feasible task unfinished, return guidance that lets the agent try a valid alternative.

The learning path is incremental: work through the five local demonstrations, available as notebooks and applications, then carry the controls into deployment. Each step gives a distinct behavior to inspect—what entered context, what the database computed, what execution actually returned, which rule fired, and whether recovery preserved the request. Those observations make the final question practical: which of these failures can you already reproduce in your own agent?