Without Caching, Your AI Agent Costs 10x More - George Cameron, Artificial Analysis

Give an agent access to thousands of files and ask it to produce a report. What should that report cost? On the AA Briefcase Agentic Knowledge Work Benchmark, which includes assignments such as creating presentations and marketing videos, Artificial Analysis found that an Opus model call averaged about 13 cents. The agent took an average of 165 steps to finish an assignment. At those rounded figures, the model bill came to roughly $21 per task.

George Cameron, co-founder of Artificial Analysis, uses that result to explain why the output token price tells an engineer so little about the price of a report. At each step, the model processes input, reasons and produces an answer. Input volume was the largest contributor to the cost of a call in this workload. The number of calls then multiplied that expense. A short final answer can sit at the end of a long and expensive sequence of decisions.

An agent carries message history from one step into the next. Material the model has already processed can therefore appear in its input many times over the course of an assignment. In Cameron’s example, an input token that hit the prompt cache received roughly a tenfold discount. Two runs with the same number of input tokens can produce very different bills depending on what fraction of those tokens qualify for the discount.

Cameron repriced the Briefcase example with a zero percent cache hit rate. The estimated cost rose from about $21 to more than $200 per task. He gives a concrete way an engineer could end up in that position: running inference with vLLM and leaving prompt caching disabled. The $200 figure is a zero-hit calculation for this particular workload. How much caching saves elsewhere depends on how much input repeats, how often it hits and what the inference provider charges for it.

To explain a bill, record the input token count and the number of cached input tokens on each call. Price the hits at the cached rate and the remaining input at the ordinary rate. Include the charges for reasoning and answer tokens, then sum across every call in the task. Keeping the count of turns alongside that total lets you distinguish a costly input history from an agent that simply took more steps.

Turning up reasoning effort changes more than the number of reasoning tokens generated on the current call. It can also change how many turns the agent takes. In the setup Cameron describes, reasoning carried through the message history contributes input tokens on subsequent turns. Part of the expense of one decision can therefore show up several calls later, under a different line item.

On Briefcase, Cameron reported that Opus 5.5 cost about $1 per task at low reasoning effort and about $21 at maximum effort. That spread makes the effort setting worth testing even after a team has settled on a model. The figures alone leave the quality question open. To decide what the extra $20 bought, compare the completed work at both settings and keep the turn counts, input volumes and cache hits from each run.

Cameron starts model selection with a plot of intelligence against cost per task. The Pareto frontier consists of the choices for which improving one measure requires giving something up on the other. With a fixed budget, look for the highest-scoring model you can afford. With a known capability requirement, work in the other direction and look for the cheapest model that clears it.

Which question to ask depends on the assignment. Consider calculating last month’s subscription spending. In Cameron’s example, given the right tools to access the records, Luna and Astra would both return $80. Once a model can reliably retrieve the records and calculate the correct total, additional capability has little value for that question. Cameron calls this a limited intelligence task.

Now ask the agent to analyze what drives costs across the business and identify ways to reduce them. There is room for a better analysis even after the agent has the arithmetic right. For this unlimited intelligence task, in Cameron’s terminology, the team has to decide how much a better result is worth. The same financial records can justify two different model choices depending on what the engineer asks the agent to do with them.

A public benchmark can narrow the field. The Artificial Analysis Intelligence Index combines ten evaluations spanning capabilities including scientific reasoning, knowledge work, coding and terminal use. Individual evaluations can suggest candidates the aggregate ranking would obscure. Cameron points to Gemini’s performance navigating complex spreadsheets on the AA Analyst Agent finance and accounting benchmark. For an agent that spends its time in spreadsheets, that result earns a model a closer look.

For a prototype, Cameron suggests trying the shortlisted models in the agent itself. For a production rollout, he recommends building a benchmark from the team’s own work and comparing a shortlist of perhaps five models. Score the deliverable the user cares about and measure the full cost of producing it. The result is a trade-off curve specific to the agent, its tools and its tasks.

That comparison also has a shelf life. Artificial Analysis tracked the cheapest way to reach a fixed score on its index. When models first reached a score of 30, the measured cost was a little over a dollar. Roughly a year later, it was about ten cents. That historical decline gives teams a reason to keep their workload benchmark around. When a cheaper candidate arrives, they can test whether it clears the quality threshold they already established.