Tokens Are the Cheap Part — Steve Jamison, Jellyfish
Read the talk
Tokens Are the Cheap Part
Steve Jamison uses Jellyfish’s engineering data to explain how a much larger AI bill can coexist with a lower allocated cost per merged pull request—and why throughput, quality and work size belong beside that calculation.
From a talk by Steve Jamison
At a glance
Ideas worth remembering
Combined salary-plus-AI cost per merged PR can fall while the AI bill rises, because additional output spreads the much larger salary expense across more merged changes.
The heaviest-spending fifth reported 67% throughput growth and a 32% decline in cost per PR, versus around 40% and 25% for lower-spending peers. These comparisons do not isolate AI’s causal contribution.
Reverts showed no worrying increase, but did not demonstrate quality improvement. Rising lines per PR also complicate the result: more merged changes can bring more review work.
Before adding token limits or investing in cost optimization, compare the likely effect with throughput enablement. Track total cost per merged PR, spending, throughput and quality together each month.
A 60-fold bill increase meets 43% more merged PRs
AI spending rose 60-fold over a year, while merged pull requests per engineer rose 43%. Steve Jamison, AI research and engineering leader at Jellyfish, opens with those two observations from the platform’s engineering workflow and AI telemetry data. They make an awkward budget conversation: the bill grew much faster than the output count, yet neither percentage tells you whether the additional spending was worthwhile.
The timing looks suggestive. AI spending increased in early 2026, around the point when throughput began a steady rise. Jamison interprets the pattern as evidence that agentic coding is helping developers produce more. These are observational comparisons, however: the reported association does not isolate how much of the throughput change AI caused.
That leaves two practical risks. A cap might stop waste, or it might prevent useful work. A higher allowance might unlock more output, or simply enlarge the invoice. Comparing bills with another company does little to resolve the choice because the bill alone says nothing about what either organization produced. The missing input is the largest engineering expense: developer salary.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put salary and AI spending in the same calculation
Jamison calls the combined expense the “intelligence costs” of a PR: what the organization pays its engineers plus what it pays for AI. The accounting question is how much of that combined expense falls on each merged change. For a consistent reporting period, the idea is:
Adding AI raises the numerator. If merged output grows sufficiently, the larger denominator can still reduce the ratio.
The relative sizes matter. In the August 2026 data presented, AI accounted for about 2.5% of this combined cost, with salary accounting for the other 97.5%. A dramatic increase in a small expense can therefore be outweighed by a more modest improvement in how much work the much larger salary expense supports. That is the specific sense in which “tokens are the cheap part.”
Across the year, the reported combined cost per merged PR fell 26%, including the 60-fold increase in AI costs. Eighty-three percent of the companies studied showed a decline. The denominator counts merged PRs rather than created PRs: review time is included in the framing, and the changes have cleared some level of quality gates. More completed changes spread salary across more units of output. This is an allocation result; it does not by itself mean payroll fell or the organization realized equivalent cash savings.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Heavier users had larger throughput gains
The population average hides a wide range of usage. Jellyfish divided companies into quintiles—five groups—by token usage and compared their results. The lightest group used about 60 million tokens per developer per month, with AI making up roughly 0.3% of combined PR cost. The heaviest fifth used about 2.5 billion tokens per developer per month, with AI accounting for 8.5%. Even in that heavy group, salary remained most of the expense.
Did the heavier bill accompany enough additional output to improve the combined ratio? The reported year-over-year comparisons put the two outcomes together:
| Company group | Growth in merged PRs per developer | Decline in combined cost per merged PR |
|---|---|---|
| Lower four spending quintiles | Around 40% | About 25% |
| Heaviest spending quintile | 67% | 32% |
The heavy group’s larger throughput increase accompanied a larger reduction in allocated unit cost. The arithmetic explains how that can happen: spreading salary across more merged PRs had a larger effect than the additional AI expense.
Jamison reads this as a reason to keep investing in adoption. The comparison supports a narrower decision rule: evaluate a larger AI bill alongside the output and combined unit cost it accompanies. It does not establish that moving a particular company into the heaviest-spending group will reproduce that group’s results.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A flattening curve can still leave useful returns
How quickly does the benefit shrink as usage rises? Jamison describes a chart with tokens used on the horizontal axis and cost per PR on the vertical axis. The curve drops steeply at first, then flattens: relatively small initial usage accompanies large unit-cost improvements, while further usage accompanies smaller improvements. The top 10% of company-months by token usage used about eight times the tokens of the median and had a 25% lower cost per PR. The curve makes the distinction visible: diminishing returns can still be positive returns.
There can eventually be a point where additional token expense outweighs the output benefit. Jamison’s assessment is that most companies in the observed range had not reached it. That supports watching the combined ratio as usage grows, rather than assuming either that every extra token pays or that a flattening curve means spending should stop.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Compare developers with their own lower-usage weeks
The next comparison changes the unit of analysis from companies to developer-weeks. Each developer’s weeks with heavy agent skill use are compared with that same developer’s weeks without skills. This holds the person constant rather than comparing two organizations with different teams and practices.
During the heavy skill-use weeks, token spending was about twice as high, merged PRs were 27% higher, and combined salary-plus-token cost per PR was about 18% lower. The same accounting pattern appears at this smaller scale: a higher AI expense accompanies enough additional output to lower allocated unit cost. Comparing a developer with themselves removes one source of difference, but weeks can still contain different work; this remains an observational comparison.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Check for mistakes—and for inflated output counts
A lower cost per PR is less useful if the organization ships more mistakes, or if developers merely divide the same work into more PRs. Jamison examines those objections separately because they challenge different parts of the result: quality challenges the usefulness of the output, while work size challenges what the output count represents.
Quality: look at reverts. A revert is a merged PR that undoes an earlier PR. The heaviest-spending fifth already had the lowest revert rate before most of the token growth. Revert rates then declined slightly across every group, but the changes were within the noise; Jamison explicitly declines to call this evidence of improvement. The useful finding is that this measure did not show a worrying increase in severe mistakes as throughput rose. Reverts remain a limited proxy, rather than a complete assessment of software quality.
Work size: look at lines per PR. Lines of code per PR increased rather than decreased, which runs against the explanation that throughput rose simply because changes became smaller. But more lines can mean several things. Jamison’s recognizable examples include verbose inline comments and files full of largely irrelevant tests. Generating more code can enlarge a change without adding equivalent value.
Cheaper creation can also change how developers package work. An edge case that once needed another ticket may now feel like “one, maybe two prompts.” A developer adds it to the current PR or combines related elements, leaving fewer PRs to manage but more material inside each review. More capable AI can also encourage more ambitious changes. Those are plausible explanations offered in the talk, rather than an isolated account of what drove the line-count increase. They expose a real tension: coding throughput can rise while review becomes more demanding.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Judge spending controls against the cost of delivery
The closing budget question returns to the opening numbers. A 60-fold spending increase and 43% more merged PRs describe inputs and output, but their percentages alone cannot settle the tradeoff. Adding salary and dividing by merged throughput supplies a common unit for comparison. In the reported population, that unit cost fell 26%.
This framing changes how to compare two kinds of investment:
- Reduce AI expense. Model routers, post-training a model and token limits are possible ways to control the AI portion of the numerator. Runaway jobs still deserve attention.
- Increase useful throughput. AI enablement may help engineers merge more work, spreading the larger salary expense across more output.
The talk does not measure those interventions against one another. It supplies the question to ask before choosing: which effort is more likely to improve combined cost per merged PR?
The practical recommendation is to track the measures together each month: the AI bill, merged PR throughput, and combined human-plus-AI cost per merged PR. Put reverts and other quality measures beside them as a counterweight. A falling unit cost is useful only if the work remains worth delivering; PR counts and line counts alone cannot establish customer value or account for all later maintenance.
Jamison’s recommendation for the companies represented is to keep investing while those efficiencies continue. The memorable line is “The bill tells you what you spent.” Combined cost per PR adds what the invoice leaves out: how much merged output the engineering expense supported. Bring that ratio to budget season, then keep asking whether throughput, delivery cost and quality are moving together.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:02
Hi, I'm Steve Jamison, AI research and engineering leader at Jellyfish. I'm here to show you why tokens are the cheap part. The research that I'll present is drawn from the Jellyfish platform, the industry's largest set of engineering workflow data joined to AI telemetry data. It's no secret that AI costs are climbing. They're up sixty X from this time last year. The use cases are wide and varied. The models and harnesses are getting much better. The
- 0:32
implications are exciting. But discussions around budget and spend are not as much fun. These conversations involve your peers and senior leadership and the board, not always the easiest audiences, especially when you don't have the right data, and it's real money these days. Output is up too. Merged PRs per engineer are up forty-three percent over the same one-year horizon. This is phenomenal. We've not seen productivity inputs like this,
- 1:02
especially on this timescale. There's plenty of research and anecdotal data to know that AI and agentic coding are driving these throughput gains. Heck, even our charts show the correlation. Notice where the inflection points are. AI spend increased in early twenty twenty-six, when throughput gains start a steady increase of their own.
- 1:25
There's a tension with these two numbers, though. How do you reconcile it? Spend is up sixty X, throughput is also up forty-three percent. Is that good? Bad? Forty-three percent is a big number, but so is sixty X. How do you know if you're spending enough? Is there more productivity gain locked behind a checkbook? Should we increase spend fifty percent, two X more? Or how do you know
- 1:55
if you spent too much? I'm sure many of you have felt the tension here. The CFO or board are asking for the receipts of the spend. Can you show them? Will merged PRs are up forty-three percent satisfy them? Also, what happens if you cap use and spend? Are you stopping from setting tokens on fire, or are you stunting further efficiency gains? You can ask peers at other companies what they're paying. For many, it's still a moving target,
- 2:25
though. But how do you equate that to your org? Surely, just comparing their bill to yours isn't adequate. We've heard the reports of co-- We've heard the reports of companies spending one hundred K, five hundred K, even millions on AI. Setting then resetting their AI budgets as reality sets in. Ultimately, these two numbers can't tell the story, and even together, they miss the biggest line on the cost side.
- 2:58
These two numbers leave out the biggest cost of all, developer salary. AI is making developers more productive on a scale we haven't seen before. It isn't just another tool, it's intelligence. This intelligence dramatically changes how much each dev can produce. That's why we need to think about its cost differently. Not off to the side as overhead, but as part of what it takes to produce each PR. We're gonna call this intelligence costs of a
- 3:27
PR.
- 3:32
What you pay your engineers plus what you pay for AI, put them together in one cost bucket. AI is about two and a half percent of the cost to produce another unit of merged code. Salary is the other ninety-seven and a half percent. This is in line with other reported figures, and this data is from August twenty twenty-six. Here, tokens are the cheap part. But that's
- 4:02
current relative costs, not dollars. Let's look at what happened to the cost of a merged PR over the last year.
- 4:16
The dollar cost of a merged PR fell twenty-six percent over the last year. Not PRs created, but PRs merged. Review time is factored in, and some level of quality gates has been cleared. What-- That decline in cost per merged PR includes a sixty X increase in AI costs over the same timeframe. The change is nearly universal too. Eighty-three percent of the companies we studied showed a drop in cost per PR. What does this tell us?
- 4:46
The additional throughput gains are being spread... are, are spreading salary thinner across PRs, and this is happening faster than the AI costs are increasing.
- 5:00
What's the range in token spend? We ranked companies by token usage, then grouped them into quintiles or fifths. The range here is quite large. The lightest group, about zero point three percent of the cost of a PR is, is AI. This represents about sixty million tokens per dev per month. The top twenty percent of AI spend per dev, we're gonna
- 5:30
call the heaviest spend group. There, eight and a half percent of the PR cost is AI. This represents about two point five billion tokens per dev per month. About two orders of magnitude more than the lightest group. We'll use these groups on the next slides to see how token spend influences other metrics.
- 5:54
First, PR throughput measured as merged PRs per dev, split by the same AI spend quintiles on the previous slide. The- The bottom four-fifths of spenders saw PR throughput gains of around forty percent, give or take. But the heavy spend group improved more at a sixty-seven percent increase. More AI spend led to more throughput gains.
- 6:24
Now let's look at the cost per PR. Positive results are seen again across the board. Cost per PR is dropping for all AI usage groups. However, again, the heaviest fifth, so the, the companies that spent the most, improved the most. They saw a cost per PR decrease of thirty-two percent over the last year. The other four groups saw about a twenty-five
- 6:54
percent decrease in cost per PR. Investing and spending more on AI fueled more of an improvement in cost per PR compared to peers who leaned in less. So we saw that the pop-- the same, same as we saw in the population average. Cost per PR changes are driven largely by PR throughput, not by AI spend. The heavy spenders experience PR throughput gains of
- 7:23
sixty-seven percent, compared to about forty-ish percent for their peers. Spreading the same salary across more PRs in the month has more of an effect than the rising AI costs, because tokens are the cheap part.
- 7:42
This is not to say that orgs should light tokens on fire and spend with abandon. Here we're looking at tokens used on the horizontal axis and cost per PR on the vertical axis. There's a steep drop at the beginning, where relative small AI spend results in large gains initially. As token usage increases, the cost per PR improvement decreases. This is diminishing returns. I wanna stress, though, diminishing
- 8:11
returns is not the same as no returns. We're seeing that the top ten percent are spending eight x the tokens compared to the median and achieving a twenty-seven-- twenty-five percent lower cost per PR. There will be a time when tokens aren't the cheap part, but by and large, for most companies out in the wild, we're not close to that threshold.
- 8:42
More tokens leads to more PRs, which leads to a lower cost of delivery.
- 8:55
So far we've talked about companies. Next, I'd like to take a tangent and talk about a study we did with developers. Here we looked at developer weeks with heavy token use compared to the same developer weeks with lower token use.
- 9:16
Here we saw the same phenomenon, just at the individual level. Compare each developer's heavy agent skill weeks with their own weeks without skills. They spend about twice as much on tokens, merge twenty-seven percent more PRs, and reach PR costs... and each PR costs about eighteen percent less once salary and tokens are counted together.
- 9:49
There are two fair curiosities I get when I talk about this work. First, does faster work result in worse work? And second, are the more PRs, the throughput gains that we're seeing, are they being driven by smaller chunks of work?
- 10:12
First curiosity, is cheaper, faster work, worse work? If AI makes PRs faster and cheaper, are we just shipping more mistakes? One proxy for this is reve- reverts, merged PRs that undo an earlier PR. The heaviest fifth started lowest before most of the token growth, and revert rates dropped slightly for every group, though these deltas are just slightly below the noise in the data. It's
- 10:42
not evidence of improvement here. However, more importantly, we're not seeing any worrying signs of rampant or severe quality issues with higher token spends and faster PR throughput.
- 11:02
The second curiosity, is throughput driven by smaller PRs? And this is demonstrably no, not smaller PRs. In fact, AI is driving an increase in line lev-- lines of code generated per PR. This is no surprise to the devs in the room. Raise your hand if you've rolled your eyes at verbose, unnecessary inline AI comments or a file full of mostly
- 11:32
irrelevant tests. Gen AI, Gen AI likes to generate code. Also, the bar to create is low for devs, and there's a pool to do more per PR instead of spinning up another ticket. It's just one, maybe two prompts, and I'll handle this edge case, or I can combine this and that element. Fewer PRs to wrangle. This doesn't mesh well with the PR review stress folks are feeling these days, though. There's...
- 12:02
Also, AI is just becoming more capable, so we're getting more ambitious. It is beyond this talk to dig into what's actually driving those level of... Lines of code changes. However, the increase in PR throughput is not coming from smaller PRs. This is intuitive for most, but it's still good to verify.
- 12:27
Back to the opening question. Is sixty X more spend worth forty-three percent more output? You can't tell in isolation. You should track both. They are important numbers, but they're supporting metrics. AI spend per PR is the intelligence that you're buying as a resource to help make your devs go faster. PRs per engineer is what you're getting with that spend. However, looking
- 12:57
at those hides the trade-off.
- 13:04
A-a-- But put salary into the equation and take the ratio of cost to throughput, and there you get your answer. Cost per PR is down twenty-six percent over that same time period. It's clear AI is driving cost efficiencies despite the increasing bills.
- 13:27
And the AI sp- the AI spend needed to get those efficiencies is small, relatively speaking. AI is still a sliver of what a P-PR costs, two and a half percent on average, under ten percent for the heavy spender group. Tokens are the cheap part. In the range that we see, more spend kept paying. Heavier token users saw sixty-seven percent PR throughput gain compared to
- 13:57
forty percent for their lower spend peers. This drove cost per PR down thirty-two percent compared to about twenty-five percent for the lower spend peers. Looking at the PR cost to token ratio curve, the top ten percent of company months by token usage resulted in ten or eight X the tokens, but twenty-five percent lower cost per PR compared to the median.
- 14:28
AI is making developers more productive on a scale we haven't seen before. It isn't just another tool, it is intelligence, which changes how much each dev can produce. That's why we need to think about its cost differently. Not off to the side as overhead, but part of what it takes to produce each PR. You need to add the AI spend and the human spend, the human salary spend together
- 14:59
to calculate your real costs per PR. This isn't to say spend frivolously. Sure, you should watch for runaway jobs. You should track your AI spend. I'm not saying don't look at how much AI is going-- how much AI spend is going in each PR. I'm saying don't let that be your driving metric. A new framing might steer how and when you think about other investments, like integrating
- 15:28
model routers, post-training your own model, implementing token limits, et cetera. Will you get more efficiencies out of trying to control AI costs or by continuing to invest in things that increase throughput, like more AI enablement? Either way, track the total intelligence costs, human plus artificial, per PR every month next to that AI bill and next to the PR throughput.
- 15:59
How are you trending? Is throughput continuing to improve? What about those cost efficiencies, the cost to deliver? Watch reverts and quality metrics bes-beside all of this as a counterweight. We don't need AI slop mucking up the gears. So heading into budget season, bring cost per PR to the table. Not just your token bill, not just the throughput bump. The bill tells you what you spent. Cost p-per
- 16:29
PR tells you whether it worked. And right now, for most companies, it says keep investing. Tokens Are the Cheap Part to get efficiency gains.
- 16:44
Well done. You made it to the end. Here's how you can stay up to date with our latest research, which again draws from the largest engineering workflow and AI telemetry data set in the industry. You can follow me on LinkedIn, my colleagues or Jellyfish. Subscribe to our Substack, or you can check out our Jellyfish blog on the website. If you want help tracking your org's engineering workflow and AI effectiveness, including
- 17:13
cost per merged PR, go to our website and book a demo or reach out to me and I'll get you connected to the right folks. Again, I'm Steve Jamison, AI research and engineering leader at Jellyfish. Thanks for watching. Cheers.