Your Agent Is Wasting Tokens and You Don't Know It

Your agent sends a large system prompt on its first call. On the next call, the instructions are unchanged: why pay to process them from scratch again? This is the opening example in Erik Hanchett’s five controls for agent token costs. Hanchett introduces himself as a senior developer advocate at AWS and demonstrates the controls using Strands Agents, an SDK that supports different model providers.

The demonstration places cache_prompt="default" on BedrockModel and supplies the repeated instructions through the agent’s system_prompt. The following Python preserves that configuration pattern:

from strands import Agent
from strands.models import BedrockModel

SYSTEM_PROMPT = """
You are a support assistant. Use the supplied product documentation
when answering questions. Ask for missing details before diagnosing
an issue, and distinguish documented behavior from your suggestions.
"""

agent = Agent(
    model=BedrockModel(cache_prompt="default"),
    system_prompt=SYSTEM_PROMPT,
)

The short prompt makes the wiring visible; actual cache eligibility depends on the model’s minimum prefix length. Hanchett describes subsequent calls as sending a much smaller system prompt. More precisely, Bedrock prompt caching reuses computation for an eligible, unchanged prefix and discounts cache reads; it does not necessarily reduce the prompt transmitted. Cache expiration and possible cache-write premiums affect the savings. The demonstrated cache_prompt setting belongs to the Bedrock provider and is now deprecated in favor of cache_config.

System instructions are only the first candidate. Repeated tool definitions and stable message content can also be cached where the provider supports it. Cache the content that stays the same across calls so repeated work can benefit from reuse.

A demanding task may justify a newer frontier model. A simpler task may not need that expense. Hanchett’s comparison uses Claude Haiku for cheaper, simpler work and Claude Sonnet for more difficult work, without specifying model versions or a price ratio.

The useful distinction is the capability the task requires. An agent does not need to send every operation to its most expensive model.

Routing can begin with an ordinary if statement. This function accepts two already configured Strands agents and chooses between them using a difficulty decision supplied by the application:

from strands import Agent


def route_task(
    task: str,
    difficult: bool,
    cheap_agent: Agent,
    capable_agent: Agent,
):
    if difficult:
        return capable_agent(task)
    return cheap_agent(task)

Alternatively, a separate inexpensive model can classify the task and select the destination. In either case, the routing decision becomes an explicit part of the agent’s workflow, allowing different use cases to reach different models.

A tool can return far more information than the model needs for its next decision. Hanchett introduces a manual Strands approach: store the large result locally or in cloud storage, then supply a summary. He also mentions supporting APIs without naming them.

The cost appears repeatedly when that large result remains in the context carried through the agent loop. The model can receive the same bulky output again on each subsequent call, even after the useful facts have already been extracted.

1. Persist the full tool result outside the model context.
2. Summarize the information needed for the next decision.
3. Carry that compact representation into subsequent calls instead of repeatedly including the full result.

Offloading separates retaining the data from repeatedly paying to include it in context. Keeping a copy in storage alone does not achieve the goal if the full result still travels through every model call.

Summarizing tool output reduces the size of repeated calls. The next control limits how often those calls happen. Hanchett reports encountering agents that invoke the same tool over and over. He warns that an uncapped tool loop might run ten or twenty times, or continue indefinitely. These are failure examples, not measured workload results.

Set an explicit maximum iteration count so the agent cannot continue without a bound. Hanchett supplies no recommended limit; the appropriate budget depends on the work the agent needs to complete.

Before deployment, inspect each tool with observability tooling:

• Call duration: How long does each execution take?
• Repetition: How many times does the agent call the tool or cycle through the loop?

Inspecting both reveals whether the expense comes from slow individual calls, repeated calls, or both. Those observations give you a basis for judging tool efficiency and changing the workflow.

A multi-turn conversation creates another source of repeated input. Each exchange grows the history, and subsequent model calls can resend everything that came before. Hanchett describes whole-history resending as consuming hundreds or thousands of tokens as conversations grow. The issue is cumulative: an old exchange can keep contributing to the input long after it stops helping the current task.

A SlidingWindowConversationManager bounds the recent history retained for the model. Hanchett uses a configurable ten-message window as his example. Set that value explicitly to express the example in Python:

from strands import Agent
from strands.agent.conversation_manager import (
    SlidingWindowConversationManager,
)

agent = Agent(
    conversation_manager=SlidingWindowConversationManager(
        window_size=10,
    ),
)

Ten messages is the talk’s example, not the current SDK default: the current Python reference defaults to 40 and accounts for valid tool-use/result relationships during reduction.

The trade-off appears when the window drops the beginning of the conversation. Earlier requirements or decisions may disappear along with less useful exchanges. Hanchett’s remedy is to summarize the older history and put that smaller representation into the context when the window boundary is reached. The retained context then combines recent detail with a compact account of what came before. A sliding window alone does not supply that summary; summarization is an additional step.

The closing recap brings the controls back to the agent as a whole: reuse stable prompt content, choose models by difficulty, and keep bulky tool data out of repeated context. Hanchett then returns to measurement—inspect tool durations and loop counts, and iterate from what they show. For longer conversations, keep checking what history gets carried forward as new messages arrive. The practical work is to examine the repeated model call: which content still belongs there, which model should handle it, and whether another iteration is justified.