# Your Agent Is Wasting Tokens and You Don't Know It

Erik Hanchett

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/uiP88SpCi1Q-your-agent-is-wasting-tokens-you-dont) · [Video](https://www.youtube.com/watch?v=uiP88SpCi1Q)

## Your Agent Is Wasting Tokens and You Don’t Know It

Repeated prompts, oversized tool results, unchecked loops, and growing histories all add to an agent’s bill. Five controls target those costs at different points in execution.

You should be familiar with system prompts, model context windows, and the way an agent alternates between model calls and tool execution.

### Cache repeated prompts

[0:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=0s)

Your agent sends a large system prompt on its first call. On the next call, the instructions are unchanged: why pay to process them from scratch again? This is the opening example in the speaker’s five controls for agent token costs. the speaker introduces himself as a senior developer advocate at AWS and demonstrates the controls using [Strands Agents](https://strandsagents.com/docs/user-guide/concepts/model-providers/amazon-bedrock/), an SDK that supports different model providers.

The demonstration places `cache_prompt="default"` on `BedrockModel` and supplies the repeated instructions through the agent’s `system_prompt`. The following Python preserves that configuration pattern:

```python
from strands import Agent
from strands.models import BedrockModel

SYSTEM_PROMPT = """
You are a support assistant. Use the supplied product documentation
when answering questions. Ask for missing details before diagnosing
an issue, and distinguish documented behavior from your suggestions.
"""

agent = Agent(
    model=BedrockModel(cache_prompt="default"),
    system_prompt=SYSTEM_PROMPT,
)
```

The short prompt makes the wiring visible; actual cache eligibility depends on the model’s minimum prefix length. the speaker describes subsequent calls as sending a much smaller system prompt. More precisely, [Bedrock prompt caching](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html) reuses computation for an eligible, unchanged prefix and discounts cache reads; it does not necessarily reduce the prompt transmitted. Cache expiration and possible cache-write premiums affect the savings. The demonstrated `cache_prompt` setting belongs to the Bedrock provider and is now deprecated in favor of `cache_config`.

![Slide titled “Cache the System Prompt” shows Agent and BedrockModel code with cache_prompt="default" and system_prompt=BIG_SYSTEM_PROMPT, alongside a presenter inset.](https://ai.engineer/api/talk-essay-media/512ace096db7298882be1d6da7d48231baaa75950df6cd7bb47cedae06d65b8f)

Cache the system prompt: the code example sets cache_prompt="default".

System instructions are only the first candidate. Repeated tool definitions and stable message content can also be cached where the provider supports it. **Cache the content that stays the same across calls** so repeated work can benefit from reuse.

### Route tasks by difficulty

[1:01](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=61s)

A demanding task may justify a newer frontier model. A simpler task may not need that expense. the speaker’s comparison uses **Claude Haiku** for cheaper, simpler work and **Claude Sonnet** for more difficult work, without specifying model versions or a price ratio.

| Task | Model choice in the example |
|---|---|
| Simpler work | Claude Haiku |
| More difficult work | Claude Sonnet |

The useful distinction is the capability the task requires. An agent does not need to send every operation to its most expensive model.

Routing can begin with an ordinary `if` statement. This function accepts two already configured Strands agents and chooses between them using a difficulty decision supplied by the application:

```python
from strands import Agent


def route_task(
    task: str,
    difficult: bool,
    cheap_agent: Agent,
    capable_agent: Agent,
):
    if difficult:
        return capable_agent(task)
    return cheap_agent(task)
```

Alternatively, a separate inexpensive model can classify the task and select the destination. In either case, the routing decision becomes an explicit part of the agent’s workflow, allowing different use cases to reach different models.

### Offload large tool results

[1:51](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=111s)

A tool can return far more information than the model needs for its next decision. the speaker introduces a manual Strands approach: store the large result locally or in cloud storage, then supply a summary. He also mentions supporting APIs without naming them.

The cost appears repeatedly when that large result remains in the context carried through the agent loop. The model can receive the same bulky output again on each subsequent call, even after the useful facts have already been extracted.

1. Persist the full tool result outside the model context.
2. Summarize the information needed for the next decision.
3. Carry that compact representation into subsequent calls instead of repeatedly including the full result.

**Offloading separates retaining the data from repeatedly paying to include it in context.** Keeping a copy in storage alone does not achieve the goal if the full result still travels through every model call.

### Cap loops and inspect tool behavior

[2:41](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=161s)

Summarizing tool output reduces the size of repeated calls. The next control limits how often those calls happen. the speaker reports encountering agents that invoke the same tool over and over. He warns that an uncapped tool loop might run ten or twenty times, or continue indefinitely. These are failure examples, not measured workload results.

Set an explicit maximum iteration count so the agent cannot continue without a bound. the speaker supplies no recommended limit; the appropriate budget depends on the work the agent needs to complete.

Before deployment, inspect each tool with observability tooling:

- **Call duration:** How long does each execution take?
- **Repetition:** How many times does the agent call the tool or cycle through the loop?

Inspecting both reveals whether the expense comes from slow individual calls, repeated calls, or both. Those observations give you a basis for judging tool efficiency and changing the workflow.

### Trim history while preserving a summary

[3:37](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=217s)

A multi-turn conversation creates another source of repeated input. Each exchange grows the history, and subsequent model calls can resend everything that came before. the speaker describes whole-history resending as consuming hundreds or thousands of tokens as conversations grow. The issue is cumulative: an old exchange can keep contributing to the input long after it stops helping the current task.

A [SlidingWindowConversationManager](https://strandsagents.com/docs/api/python/strands.agent.conversation_manager.sliding_window_conversation_manager/) bounds the recent history retained for the model. the speaker uses a configurable ten-message window as his example. Set that value explicitly to express the example in Python:

```python
from strands import Agent
from strands.agent.conversation_manager import (
    SlidingWindowConversationManager,
)

agent = Agent(
    conversation_manager=SlidingWindowConversationManager(
        window_size=10,
    ),
)
```

Ten messages is the talk’s example, not the current SDK default: the current Python reference defaults to 40 and accounts for valid tool-use/result relationships during reduction.

The trade-off appears when the window drops the beginning of the conversation. Earlier requirements or decisions may disappear along with less useful exchanges. the speaker’s remedy is to summarize the older history and put that smaller representation into the context when the window boundary is reached. The retained context then combines recent detail with a compact account of what came before. A sliding window alone does not supply that summary; summarization is an additional step.

### Use observations to guide the next change

[4:51](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=291s)

The closing recap brings the controls back to the agent as a whole: reuse stable prompt content, choose models by difficulty, and keep bulky tool data out of repeated context. the speaker then returns to measurement—inspect tool durations and loop counts, and iterate from what they show. For longer conversations, keep checking what history gets carried forward as new messages arrive. The practical work is to examine the repeated model call: which content still belongs there, which model should handle it, and whether another iteration is justified.

![Recap slide lists Cache the System Prompt, Route by Difficulty, Offload Big Tool Results, Cap Your Tool Loops, and Trim the History, with a presenter inset at lower right.](https://ai.engineer/api/talk-essay-media/ed2a4d39f914f8053c4c0bb867aac911b86c29253991b9e7aef8a225fd1f3a47)

“Same Agent. Smaller Bill.” recaps five fixes for agent token costs.

## Resources

- [Configure prompt caching in Strands](https://strandsagents.com/docs/user-guide/concepts/model-providers/amazon-bedrock/) — Current Bedrock provider configuration, including cache_config and the deprecated cache_prompt option.
- [How Bedrock prompt caching works](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html) — Explains reusable prompt prefixes, cache eligibility, expiration, and cache-read versus cache-write billing.
- [SlidingWindowConversationManager](https://strandsagents.com/docs/api/python/strands.agent.conversation_manager.sliding_window_conversation_manager/) — Python reference for configuring retained conversation messages while preserving valid tool-call relationships.
- [Sliding windows and conversation summaries](https://strandsagents.com/docs/user-guide/concepts/agents/conversation-management/) — Examples of retaining recent messages and summarizing older conversation context.
- [Control tool execution with hooks](https://strandsagents.com/docs/user-guide/concepts/agents/hooks/) — Lifecycle hooks with examples for modifying tool results and enforcing per-tool invocation limits.
- [Offload oversized tool results](https://strandsagents.com/docs/user-guide/concepts/plugins/context-offloader/) — A plugin that moves large tool outputs into storage and keeps previews and retrieval references in context.

## Slides

### Your Agent Is Wasting Tokens

Erik Hanchett introduces five ways to reduce token costs when using and creating AI agents.

[0:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=0s) [Opening slide about cutting AI agent token costs, with an abstract graphic and speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/83ddb3372a75ce06722dc6a4ed10320a4476ccaf8c404bf5abf1366b3514dec6.webp)

Your Agent Is Wasting Tokens

Cutting AI Agent Token Costs

Erik Hanchett
Developer Advocate at AWS

### Costing Too Much

The talk introduces five ways to reduce token costs when using and creating agents.

[0:08](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=8s) [Costing Too Much slide with a bright graphic and speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/4ade60cfa41091d49170857502402e391fb72b7e55b0ec4dc25fb168990a7989.webp)

Costing Too Much

3X

### Cache System Prompt

The speaker proposes caching the system prompt to reduce token costs: the first agent call sends the full prompt, while subsequent calls send a much reduced prompt.

[0:18](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=18s) [Cache System Prompt slide with a network illustration and speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/2cc1e3c05bf58f3d5ef4964b5b07dbe90f7b41186f4a3cca5ba0663747454626.webp)

Cache System Prompt

### Cache the System Prompt

The pseudocode caches the static system-prompt prefix. The speaker says the first call sends the full prompt, while subsequent calls send much less.

[0:23](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=23s) [Code example configuring an agent to cache its system prompt, with speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/5bd14fea1bcdbc5c60603c2c226ff2c1a98b9bc70c3ff2fa5d75ab79fc7d9293.webp)

FIX 01
Cache the System Prompt

agent = Agent(
    model=BedrockModel(
        model_id="claude-sonnet",
        cache_prompt="default",   # cache static prefix
    ),
    system_prompt=BIG_SYSTEM_PROMPT,
)

01 / 05

### Route By Difficulty

Route messages by difficulty: use a cheaper model for simple tasks and consider a frontier model for harder ones.

[1:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=60s) [Route By Difficulty slide with a network illustration and speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/9c314ab9c81fcdb11c1afedd055dc77f9334c84b67111835103de5874845efe8.webp)

Route By Difficulty

### Route by Difficulty

The example routes simple tasks to Claude Haiku and harder tasks to Claude Sonnet. A cheap model could also decide which model to use.

[1:08](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=68s) [Code example choosing a cheaper model for simple tasks and another model for harder tasks, with speaker video.](https://assets.ai.engineer/sites/aiecode2025/sha256/b84fabbfd22fab6f049d1b3a2231e7a84747ce134e3709eea009da2f2f04f85b.webp)

FIX 02
Route by Difficulty

def pick_model(task):
    if task.is_simple:
        return "claude-haiku"     # cheap
    return "claude-sonnet"        # expensive reasoning

### Offload Big Tool Results

Store a large tool result and return a summary with a reference, so the full result does not consume context on every agent loop.

[1:56](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=116s) [Code stores a fetched report and returns a summary with a reference.](https://assets.ai.engineer/sites/aiecode2025/sha256/db701853f14b077907e07f209c409df4578be746891be0b7efb69c6aa1eb4279.webp)

FIX 03
Offload Big Tool Results

@tool
def fetch_report(id: str) → str:
    data = api.get(id)            # 10k tokens
    key = store.put(data)         # offload
    return summarize(data, key)   # summary + ref

### Cap Your Tool Loops

An agent can repeatedly call a tool, potentially entering an infinite loop and consuming tokens. Cap tool loops to limit that repetition.

[2:46](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=166s) [Cap Your Tool Loops title beside a circuit illustration.](https://assets.ai.engineer/sites/aiecode2025/sha256/618fd1aa22faaaadf75e5726d881a9f541a6c3507ee1cac451db3eef99faa644.webp)

Cap Your Tool Loops

### Cap Your Tool Loops

Set max_iterations=8 to cap repeated tool calls. Before deployment, inspect each tool’s runtime and loop count to assess efficiency.

[2:50](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=170s) [Code sets an agent's maximum iterations to eight to limit tool loops.](https://assets.ai.engineer/sites/aiecode2025/sha256/846ac325650c9f82063ce8572add73ff2ef81eecdf1b6a72728b07b4046fafd4.webp)

FIX 04
Cap Your Tool Loops

agent = Agent(
    tools=[...],
    max_iterations=8,     # stop runaway loops
)

### Trim the History

A sliding window sends only the last ten messages to the model, reducing repeated history. Earlier messages fall out of the window; the speaker suggests summarizing them to retain context.

[3:41](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=221s) [Code configures a sliding window conversation manager with a window size of ten.](https://assets.ai.engineer/sites/aiecode2025/sha256/d9c5c9f727bfaf0f0117bd24b2ebf77ed0771753621f09bf54efa19986decdf0.webp)

FIX 05
Trim the History

from strands.agent.conversation_manager import (
    SlidingWindowConversationManager,
)

agent = Agent(
    conversation_manager=SlidingWindowConversationManager(
        window_size=10,
    ),
)

### Same Agent. Smaller Bill.

Cache static prompts, route simple calls to cheaper models, summarize large tool results, cap loops, and trim history to reduce repeated context in agent calls.

[4:49](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=289s) [Recap listing five ways to reduce agent costs: cache the system prompt, route by difficulty, offload large tool results, cap tool loops, and trim conversation history.](https://assets.ai.engineer/sites/aiecode2025/sha256/4c4926d3ede8e6adfbc8dc11983e9bdfb928bf1c4317cc018053d4a3111bc3c3.webp)

RECAP // FIVE FIXES
Same Agent. Smaller Bill.

01 Cache the System Prompt — stop resending the static prompt every turn
02 Route by Difficulty — cheap model for simple calls
03 Offload Big Tool Results — keep 10k-token blobs out of context
04 Cap Your Tool Loops — max_iterations, no runaways
05 Trim the History — window or summarize old turns

### Thanks!

Erik Hanchett closes with his blog and social media details for readers who want to go deeper.

[5:36](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=336s) [Closing thank-you slide with presenter contact information and a network graphic.](https://assets.ai.engineer/sites/aiecode2025/sha256/419b4341e7944428a1bca6360cb965f485a2c35e6bea0914b6466e0e61747edc.webp)

Thanks!

Go Deeper
Erik Hanchett

Social Media
ErikCH

Site
programwitherik.com

## Full transcript

[0:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=0s) Hey everyone. My name is [REDACTED:username]. I am a senior developer advocate at AWS, and I'm gonna talk to you about how you can save on token costs. Now I'm gonna show you five ways that you can reduce your token costs while using and creating agents.

[0:18](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=18s) So the first way you can do that is to cache your system prompt. Let me show you some code. Now I'm using AWS's Strands Agents. This works with all different providers.

[0:33](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=33s) This is a little bit of pseudocode, but the idea is that you can add cache_prompt=default, and what that'll do is on the first call of your agent, it will send the full system prompt over, and then on every subsequent call, it will have a much reduced system prompt being sent over.

[0:54](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=54s) So it'll be cached. You can also cache the tool prompts and messages as well.

[1:01](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=61s) This may sound obvious, but you want to look into routing your different messages based on the difficulty. So here's a code example. Let's imagine that you have a task that's very difficult.

[1:15](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=75s) You may wanna use one of the newer frontier models. However, if it's something simpler, you want to use a cheaper model. In this case, maybe we use Claude Haiku for a cheap-- something cheap, and then use Claude Sonnet for something a little bit more difficult.

[1:30](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=90s) And then you can use an if statement. You can even have another model that's very cheap decide which model to use. So you can play around with this, but I highly recommend don't use the most expensive model for everything you're doing.

[1:43](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=103s) You want to use multiple different models based on the use case, and then try to route to it inside your agent.

[1:51](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=111s) Another good tip is to offload the tool result. Let me show you some code on here. Once again, I'm using Strands Agents. This is a, a manual way to do it.

[2:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=120s) There is some additional APIs that the Strands Agents offers to do this. If you have a large tool result that's coming back, you can store it locally or in the cloud and then use some kind of summarization that saves on tokens.

[2:18](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=138s) So that way when it's being called over and over again, the tool result isn't added into the context every time, every time the tool loops or every time the agent loops.

[2:30](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=150s) So if you can find any way that where you have this tool result that you don't necessarily send it on every single call back to the large language model, that will save a lot of tokens for you.

[2:41](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=161s) And like I said, there's a few APIs to do this, but e-essentially you can do the summarization technique. You can also cap your tool loops. So when you're dealing with the agent loop and it decides to do a tool call, I've had this happen often where it calls the tool over and over and over again.

[3:00](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=180s) And if you don't cap that tool call, then it might run ten, twenty times. It might get into an infinite loop, which would be very bad for your token usage.

[3:10](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=190s) So always set a max iterations of how many times it will loop. A, a good thing you can do before you deploy your agent is to run some observability tools and take a look at the tool call use for every single tool, and then see how long each one of them is running and how many times they're

[3:30](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=210s) looping. And that way you can get an idea of how efficient the tool call is.

[3:37](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=217s) Last but not least, we can trim the history.

[3:42](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=222s) So if we're using a, a multi-turn agent and we are talking back and forth, you will find at times that the conversation history will get very large. On every single call, that whole conversation history will be sent back to the large language model, and this can eat through hundreds if not thousands of tokens.

[4:03](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=243s) In Strands Agents, we have something called Sliding Window Conversation Manager, which, which this does is it looks back at the last ten messages and only sends those back, and you can set this to whatever you want.

[4:16](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=256s) And that way you're not sending these huge message histories back to the agent every single time a new message comes in. The downfall of this, or the trade-off of this I should say, is that you will lose the message history from the beginning.

[4:33](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=273s) The way you wanna deal with that is you can use, uh, some sort of summarization of the history and then put that into the context window. So rather than sending all of it, you may send a small amount once you hit this sliding window.

[4:46](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=286s) Uh, this will save you a lot of tokens.

[4:51](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=291s) So in conclusion, we have five things. Cache the system prompt, and if you can, maybe the tool prompts and messages. Route by difficulty. Don't use the same expensive model for everything you're doing, for every single task.

[5:05](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=305s) Offload these big tool results. So if you have a large tool result, you can summarize it and not have it being sent in the agent loop and overloading your context.

[5:16](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=316s) You can cap those tool loops. Make sure you use observability tools to see how long tool calls are taking and how many loops they're, they're doing and, and then try to iterate over that.

[5:27](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=327s) And then of course trim the history if you're using a multi-turn conversation with your agent, so that way the whole conversation isn't being sent over and over again for very long conversations.

[5:37](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=337s) Thank you for listening to my very quick almost lightning talk today. If you'd like to go deeper, find me on LinkedIn at [REDACTED:username]. I also blog at programwitherik.com, and then you can find me on social media at [REDACTED:username].

[5:52](https://www.youtube.com/watch?v=uiP88SpCi1Q&t=352s) Love to talk to you guys more. Thanks.
