What does the context window have to do with my token bill?
The context window is the single biggest driver of what you pay, because your AI coding agent re-sends almost the whole thing on every turn and bills you for it each time. People assume the cost comes from the model "thinking" or from the length of its replies. It doesn't. The expensive part is the input — the pile of code, history, tool definitions, and command output the model has to re-read before it can write a single line back to you. I have a PhD in AI and I build token-reduction tooling for a living, so I look at a lot of session logs. The pattern is always the same: the output is a rounding error, and the input is the whole bill. An agent that writes you a tidy 40-line patch may have read 80,000 tokens of context to get there — and if it took six turns, it read most of those 80,000 tokens six times. That is the mechanism. Everything else in this article is just consequences of it. The window is also a hard ceiling, not only a meter. When it fills up the agent starts dropping or summarising earlier turns, which is where the "wait, why did it forget what we just did?" moments come from. So the window controls two things at once: how much you pay, and how coherent the agent stays. Both get worse as it fills.Why does the agent re-send the whole window every turn?
It re-sends the window because the model is stateless — it has no memory between API calls, so the client has to ship the entire conversation back on every single turn. There is no server-side session quietly holding your transcript. Each request to the model is independent, and to continue a conversation the agent reconstructs the full state and sends it again. For a one-shot chat that's fine. For an agent it's brutal, because an agent runs in a loop: read a file, run a test, read the output, edit, run the test again. Every one of those steps is a fresh API call carrying the accumulated context so far. The transcript grows monotonically — turn 1 carries 5k tokens, turn 10 might carry 60k — and you pay the input price on the full payload each time. This is exactly why agentic coding is so much more token-hungry than a single completion. The cost isn't linear in how much work you ask for; it's closer to quadratic in how long the session runs, because each new turn re-pays for every turn before it. The honest mental model is "input tokens × turns," and the window is the multiplier you can actually control.Doesn't prompt caching make this free?
Prompt caching makes the re-sent context much cheaper, but it does not make it free, and it does not save you from a bloated window. On Anthropic's models a cache read costs roughly 10% of the normal input price (Anthropic pricing, 2026). That's a real, large discount on the stable prefix of your context — the system prompt, your project rules, files that haven't changed. Use it. It's one of the best levers you have, and I cover it in prompt caching to cut input cost. But caching has limits that a fat window walks straight into. A cache hit only applies to a prefix that is byte-for-byte identical to a previous request; the moment something near the top of the context changes, everything after it has to be re-read at full price. Agents reorder tool results, inject fresh command output, and edit files mid-session — all of which can invalidate the cached prefix. And cache entries expire (Anthropic's default window is a few minutes), so a slow human in the loop loses the discount entirely. So caching reduces the unit price of re-reading context. It does nothing about the deeper problem: the context should not have been that big in the first place. A cached 80k-token window is cheaper than an uncached one, but it's still more expensive — and less accurate — than a well-shaped 15k-token window. Caching is a discount on waste, not a substitute for not generating the waste.What actually fills the window — and what does it cost?
Four sources fill an agent's window, and they don't contribute equally. Here's how a typical mid-session payload breaks down, with the part that matters most: whether it gets re-sent on every turn.| Source | Typical share | Re-sent each turn? |
|---|---|---|
| Conversation history | Grows unbounded | Yes |
| File contents the agent read | Large, spiky | Yes, until trimmed |
| Tool / MCP definitions | 5k–25k tokens | Yes |
| Command + test output | Spiky, often huge | Yes |
npm install or one verbose test run can dump tens of thousands of tokens of progress bars and stack traces into the transcript, and now you re-send that noise on every subsequent turn too.
Put real prices on it. On Claude Opus 4.8 input is $5 per million tokens; on Sonnet 5 it's $2; GPT-5.5 input is $5 (Anthropic pricing; OpenAI pricing, 2026). A 60k-token window that rides through 15 turns is 900k input tokens — about $2.70 on Sonnet, $4.50 on Opus, for one feature, most of it spent re-reading the same stuff. For the full breakdown of where coding-agent spend goes, see AI coding agent token costs and LLM API token pricing.
How do I keep the window small without losing context?
You keep it small by changing what enters the window in the first place, not by cleaning it up after the fact. There are four levers, roughly in order of payoff:- Retrieve, don't dump. Instead of letting the agent read whole files "to be safe," give it semantic code search so it pulls the three functions that matter and leaves the other 580 lines on disk. This is the single biggest win because file-reading is the spikiest source.
- Filter command output. Strip progress bars, repeated warnings, and the 200 passing-test lines before they hit the context. Output filtering on a noisy test run routinely cuts a 30k-token blob to under 2k with zero loss of signal.
- Load tools lazily. Don't ship every MCP schema on every turn. Expose tool definitions only when the agent is actually about to use them, so the per-turn floor stays low.
- Compress and skeletonise. When the agent does need a big file for orientation, give it a skeleton — signatures and structure, not full bodies — and let it ask for the full implementation only if it needs it.
What goes wrong when you ignore the window
The failure mode is gradual, which is why people miss it: the agent doesn't break, it just gets quietly more expensive and slightly dumber over a session. Here are the anti-patterns I see most. Letting the session run forever. The longer a single conversation lives, the bigger the window and the more each turn costs. A two-hour session can cost 5–10x what the same work split into fresh, scoped sessions would. When a task is done, start a new conversation; don't drag the whole history into the next one. Eager MCP loading. Connecting every server you might one day want means paying for all those tool schemas on every turn, forever. Load lazily. See the best MCP servers for Claude Code for which ones are actually worth the per-turn cost. Treating context bloat as a "the middle of the window gets ignored" model problem. It's real — Liu et al. showed retrieval accuracy drops for facts buried in the middle of a long context ("Lost in the Middle", 2023, arxiv.org/abs/2307.03172) — but the fix isn't a bigger window. It's a smaller, denser one. A stuffed window costs more and hides the signal the model needs. Trimming the noise raises both your margin and your accuracy at the same time. Trusting the meter you can't see. Most people have no idea their window is 70k tokens until the bill arrives. Measure it. If your tooling shows per-turn input size, you'll start to feel which actions are expensive, and that intuition is worth more than any single optimisation.Frequently asked questions
What is a context window, in one sentence?
A context window is the maximum amount of text — measured in tokens, not words or characters — that a model can consider at once, covering everything sent to it plus everything it writes back. The context window glossary entry has the fuller definition; this article is about what it costs you.How big is Claude's context window?
Claude models have shipped with large windows, and the frontier tier now reaches 1M tokens. But the published maximum is a ceiling, not a target: you are billed on what you actually put in the window each turn, not on the size of the window you were allowed. A 1M-token window used at 40k tokens costs exactly what 40k tokens cost. That distinction matters more than the number itself, which is why we don't quote per-model figures that go stale within a release cycle — check the vendor's current model page, then measure your own usage.Do bigger context windows cost more?
Not by existing — only by being filled. Vendors don't charge for headroom. What a bigger window changes is behaviour: it removes the forcing function that used to keep sessions tidy, so people read more files "to be safe" and pay for all of them on every subsequent turn. Some vendors also apply premium pricing above a token threshold on long-context requests, which turns a lazily-filled window into a rate change as well as a volume increase. Check your provider's pricing page for a long-context tier before assuming linear cost.Is a bigger context window better for accuracy?
Not above the relevance threshold. Models attend least to material buried in the middle of a long window — the "lost in the middle" effect — so padding a window with marginally-related files tends to lower answer quality while raising the bill. Relevance beats volume, which is the whole argument for retrieval over dumping.How do I see how much of my window is actually used?
In Claude Code,/context shows what is occupying the window right now, which is the reading that tells you what to compact. Do it at the start of a session before typing anything: whatever is already there — system prompt, CLAUDE.md, MCP tool schemas — is your standing overhead on every turn. Measuring agent token usage covers the cross-agent version.
Does the window reset between sessions?
Yes — a new session starts with an empty transcript, which is why starting fresh per task is the cheapest context management there is. Compacting shrinks an existing window by summarising earlier turns; it makes future turns cheaper but does not refund what you already spent.See also:
- LLM cost: what drives it and how to cut it — the whole bill, not just the window.
- Context engineering for AI coding agents
- How to reduce Claude Code token usage
- Glossary: context window
Ranked #1 on the Token-Harness Optimizer Leaderboard.
Tokenade ranks #1 in the Token-Harness Optimizer Leaderboard — an end-to-end benchmark of agent token optimizers measured on real coding sessions. Set it up once, it works on every prompt. Works with Claude Code, Cursor, Codex, Copilot & more.
