Stanford & Michigan Ran the Numbers on Agent Costs

The bill nobody had actually measured

Until April 2026, nobody had published a rigorous, large-scale count of what an agentic coding run actually costs, in tokens, broken down by type, across multiple frontier models, on a reproducible benchmark. Practitioners had intuitions. Tool vendors had hand-wavy marketing claims. But a controlled experiment, eight models, 500 real GitHub issues, four independent runs per task, every token logged? That had not been done. Then Longju Bai, Zhemin Huang, Xingyao Wang, and collaborators from the University of Michigan, Stanford's Digital Economy Lab, All Hands AI, Google DeepMind, Microsoft AI, and MIT did exactly that. They published the results in arXiv:2604.22750, and I read it carefully, because this is precisely the territory I work in. The numbers are striking. Not because they're surprising — I've watched agents burn tokens up close for years — but because they're now on the record, with methodology you can audit.

What 500 SWE-bench tasks actually cost

The researchers ran the OpenHands agent framework on SWE-bench Verified: 500 real GitHub issues, each requiring the agent to read a live repository, reason about the problem, call tools, and submit a patch. Eight frontier models: Claude Sonnet-3.7, Sonnet-4, Sonnet-4.5, GPT-5, GPT-5.2, Qwen3-Coder-480B, Kimi-K2, and Gemini-3-Pro Preview. Four independent runs per task, per model. The headline number from the paper: 4.17 million tokens per task on average, costing an average of $1.857 per task. For context, they also measured the same kinds of coding problems in two other modes — single-turn code reasoning (think: one-shot problem solving, no tools) and multi-turn code chat. The comparison is almost uncomfortable to look at: That gap — 3,500× more tokens than a single-turn reasoning task — is not a rounding error or a quirk of one model. It is the structural cost of the agentic loop: every round of action appends to the conversation, and the full conversation history is re-sent to the model on the next round. Context accumulates. The model does not forget. You pay for every token it has ever seen, on every subsequent call.

Input tokens are the real bill

One number in the paper deserves a second read: the average input-to-output ratio in agentic coding is 153.85. For every token the model generates, it processes 154 tokens of input. In code chat that ratio is about 1.33. In agentic coding it is two orders of magnitude higher. This is the context-window compounding problem in a single number. The agent reads files, runs terminal commands, gets back verbose output, then re-reads all of that — plus everything it has done before — on the next turn. Output (the model actually thinking and writing code) is cheap. The accumulated, re-ingested, copy-forwarded input is where the money goes. The paper confirms this is true even when prompt caching is active. In their phase-level analysis of Claude Sonnet-4.5, cache-read tokens were the single largest cost category in every phase of the task — Setup, Explore, Fix, Validate, Closeout — because the sheer volume of context being reused is large enough that even at the discounted cache-read rate, it dominates over the much more expensive but much smaller output token count. You can have prompt caching enabled and still have a large bill, because the cheapest token type is still the largest by a factor of thousands.

More tokens, worse results

Here is the finding I keep coming back to, because it is genuinely counterintuitive until you understand the mechanism. The paper shows that runs on the same task can vary by up to 30× in total token consumption across four independent attempts. You might solve the same GitHub issue in 800,000 tokens one run and 24,000,000 the next. Same task, same model, same environment. When the researchers ranked the four runs for each task by cost — MinCost, LowerCost, UpperCost, MaxCost — and measured accuracy at each level, they found that accuracy peaks at intermediate cost and then saturates or declines at the highest cost. The MaxCost run is not the best run. It is often worse than the LowerCost run. Why? Expensive runs are characterised by more repeated file views and repeated edits: the agent revisiting the same files it has already read, making modifications it has already made, accumulating context without making proportional progress. The AI coding agent token cost data I track independently converges on the same explanation — context bloat is not productive reasoning, it is often the agent thrashing. Scaling token usage does not scale accuracy. There is no dollar amount at which the agent reliably closes the remaining gap on a hard problem. The researchers confirmed this against human-rated difficulty labels: 6.7% of tasks rated "under 15 minutes" consumed more total tokens than the average task rated "over 1 hour". And 11.1% of the hard tasks consumed fewer tokens than the average easy task. Human-perceived difficulty and model computational effort are weakly correlated at best (Kendall τb = 0.32).

Eight models, wildly different token appetites

Not all models are equally expensive. The paper's per-model comparison is the clearest data I have seen on this. On the same 500 tasks, Kimi-K2 and Claude Sonnet-4.5 consumed on average over 1.5 million more tokens than GPT-5. This gap persists on the shared-success subset — tasks every model solved correctly — meaning it is not driven by some models attempting harder problems. It is a behavioral property of the model: how much context it accumulates before it acts, how often it revisits files, how much output it generates per round. The paper also separates what happens on tasks every model fails. On those, GPT-5 and GPT-5.2 spend only modestly more tokens than on the tasks they succeed at (under 500K extra). Kimi-K2 spends roughly 2 million more tokens on the tasks it cannot solve. That is the cost of not having a reliable stopping signal: models continue exploring, retrying, and re-reading context when stuck, accumulating expense without progress. Some models do this much more than others. The practical implication for anyone running agentic coding at scale: model selection on cost grounds alone could cut the bill by 30–40% without changing accuracy, because the cheaper models achieve comparable accuracy at substantially lower token counts.

What no model can tell you in advance

The third contribution in the paper is also the most sobering. They tested whether agents can predict, before starting a task, how many tokens they will spend. The answer: barely. Across all models, correlation between predicted and actual total token usage peaked at r = 0.39 — weak-to-moderate. Output tokens were somewhat more predictable than input tokens, which makes sense: what the model will write is somewhat plannable; how much context it will accumulate through tool calls and file reads is not. All models also systematically underestimated their actual token usage. The models do not know how expensive they will be. This matters for the LLM API token pricing problem more than most people realize. You cannot set a per-task budget by asking the model what it will cost. The model will tell you a number that is too low, and it will be meaningfully wrong.

What this means for your costs (ESTIMATE)

The paper's findings converge on one actionable lever: input token volume, not output, is what drives agentic coding cost. The input-to-output ratio of 153.85 means that a 40% reduction in input tokens cuts the total bill by roughly 39% — 40% × (153.85 / (153.85 + 1)) ≈ 39.7%. The output contribution is almost noise. The mechanisms behind excess input tokens are well-understood — they are exactly what the token reduction guide covers:

• Context growing unboundedly through the conversation, re-sending already-cached material
• Verbose tool output (full stack traces, raw git log, entire test suite output) injected wholesale into the next prompt
• Full-directory reads at session start when semantic retrieval would find the relevant files in a fraction of the tokens
• Repeated file reads when the content has not changed

If you take the paper's average of $1.857 per task and a conservative 40% input-token reduction: These are rough estimates extrapolated from the paper's average figures. Real costs vary substantially by model, task complexity, and how many of the above patterns your agent triggers. The token cost calculator lets you run these numbers for your own volume and model choice. The reduction percentages are grounded in what input-token optimization actually does: the 40–50% range is what controlled benchmarks cite for semantic retrieval replacing directory dumps, output filtering on tool calls, and cache-stable prefix structuring. It does not change what the model generates. It changes how much context the model is forced to re-read on each turn.

The study that every agent builder should read

What makes Bai et al. worth reading is not just the numbers. It is that they measured the mechanism, not just the output. They traced individual trajectories round by round. They separated input from output, cached from non-cached, phase by phase. They showed what high-cost runs actually do differently (more repeated file access), not just that they cost more. The finding I will keep citing: accuracy peaks at intermediate cost and plateaus. The MaxCost run is not the best run. There is no version of "just give it more tokens" that reliably closes the capability gap. What we can do — what tooling like Tokenade targets — is strip the wasted context out so the tokens that reach the model are the ones that matter. Free up to approximately 10 million tokens saved per month. Start free — no card required.