Prompt Caching: How to Cut Your Input Bill

Prompt caching cuts your input bill by letting the model re-read a stable prefix at roughly 10% of the normal input price instead of paying full freight every call. That is the whole trick: you mark the unchanging front of your prompt — system instructions, tool schemas, the file you keep referencing — and on the next request the provider serves it from cache instead of re-tokenizing it from scratch. If your agent loops over the same context twenty times in a session, that prefix is the single biggest line item, and caching is the cheapest lever you have. I spend most of my time building tooling that shaves tokens off AI coding agents, and prompt caching is the one optimization I'd reach for first because it costs you nothing in quality. You aren't dropping context, summarizing, or throwing away history. You are paying less for bytes the model has already seen. The catch — and there's always a catch — is that caching only works if you stop reshuffling your prompt. Most people break their own cache without noticing. Part of: Reduce AI Coding Agent Token Usage

What is prompt caching and why does it cut cost?

Prompt caching stores a tokenized prefix of your prompt on the provider's side so repeat requests pay a fraction of the input price for that prefix. With Anthropic's API, a cache read costs about 10% of the normal input token price; the trade-off is that the initial cache write costs a bit more than a plain input token (roughly 1.25x for the 5-minute tier). So the first call is slightly more expensive and every subsequent call that reuses the prefix is dramatically cheaper. (See Anthropic's prompt caching docs.) Put numbers on it. Claude Opus 4.8 lists input at $5 per million tokens (MTok) and output at $25. A cached read on that input runs around $0.50/MTok — a tenth. If your coding agent carries a 30,000-token system-plus-context prefix and makes 15 tool-call round trips in a session, uncached you pay for ~450,000 input tokens of prefix alone. Cached, you pay full price once and ~10% on the other 14 reads. That's the difference between $2.25 and roughly $0.78 on the prefix for a single session — before you've written a line of useful output. Multiply by every session, every day. The same math holds on the cheaper models. Sonnet 4.6 is $3/$15 per MTok, Haiku 4.5 is $1/$5, and GPT-5.5 is $5/$30. Caching scales the input side down proportionally on each. It does nothing for output tokens — generation is always billed in full — which is exactly why I treat caching as an input optimization and pair it with other tactics for the output side. If the vocabulary here is fuzzy, the token and prompt-caching glossary entries are short and worth two minutes.

How do you structure a prompt so the cache actually hits?

Put everything stable at the front and everything volatile at the back, because caching matches on an exact prefix. The cache is a longest-common-prefix lookup: the provider compares your new request against what it stored and reuses tokens up to the first byte that differs. One changed character near the top invalidates everything after it. So the rule is brutally simple — order your prompt from most-stable to least-stable. A coding-agent prompt that caches well looks like this, top to bottom:

System prompt and role instructions — these almost never change within a session.
Tool / function schemas — your MCP tool definitions, fixed for the session.
Pinned context — the files, docs, or specs the agent keeps referring to.
Conversation history — append-only, so older turns stay in the cached prefix.
The current user turn — the only thing that genuinely changes each call.

You then place the cache breakpoint after the last stable block. Anthropic lets you set up to four cache_control breakpoints; the practical move is to mark the end of your tool definitions and the end of your pinned context. Everything before the breakpoint gets cached; the fresh user message after it is the only part billed at full input price. The mistakes that quietly kill your hit rate:

Injecting a timestamp or random ID into the system prompt. "Current time: 2026-06-15T14:32:09Z" at the top means a fresh cache miss every single call. If you need the time, put it in the last user message, not the prefix.
Reordering tool definitions. If your framework serializes tools from a dict with non-deterministic ordering, your prefix changes shape between calls. Sort them.
Editing history mid-conversation. Summarizing or pruning earlier turns rewrites the prefix and burns the cache. Append; don't rewrite.

There's a deeper principle here that I keep coming back to in context engineering: a prompt is not a bag of strings, it's a layout. Treat the layout as a contract with the cache.

Does prompt caching expire, and when does it stop helping?

Yes — caches have a time-to-live, and that TTL is where most of the disappointment lives. Anthropic's default cache lives about 5 minutes from the last access, with a 1-hour tier available at a higher write cost. OpenAI's prompt caching, by contrast, is automatic for prompts over 1,024 tokens and there's no write surcharge, but you have far less control over what gets cached and the discount differs by model (see OpenAI's prompt caching guide). Different providers, different rules; read the one you're actually billing against. The 5-minute window matters more than it sounds. If a developer fires a burst of agent calls, walks away to think, and comes back 8 minutes later, the cache has evaporated and the next call pays the full write price again. Within an active coding session you'll mostly stay warm; across the gaps between sessions you won't. This is why caching is a session-level win, not a magic global discount. Caching also stops helping in three honest cases:

One-shot calls. If you never reuse the prefix, you paid the 1.25x write premium for nothing. Single-turn prompts shouldn't set a breakpoint.
Tiny prefixes. Below a provider minimum (1,024 tokens for Anthropic on most models), caching isn't even offered. There's nothing to amortize.
A prefix that changes every call. If your "stable" context isn't actually stable, you're writing a new cache entry each time and reading none of them. That's strictly worse than no caching.

And caching never touches output. A verbose agent that emits 4,000 tokens of explanation per turn will still pay $25/MTok on Opus output regardless of how warm your cache is. That blind spot is the reason I don't stop at caching.

How does caching fit with the other ways to cut tokens?

Caching shrinks the cost of the input you keep; the other tactics shrink the amount of input and output in the first place — and you want both. Think of it as two independent dials. Caching answers "how cheaply can I re-read this prefix?" Everything else answers "do I even need these tokens?" In practice I stack four moves on top of caching:

Semantic code search instead of dumping whole files into context. The agent retrieves the three functions it needs, not the 2,000-line module. Smaller pinned context means a smaller (and still cacheable) prefix.
Output filtering on noisy tool calls. A git status or a test runner can spew thousands of tokens of output the model doesn't need; filtering them before they hit context cuts the input that feeds the next turn.
Skeleton compression for large files — give the model the structure (signatures, types, headings) and let it ask for the body only when it needs it.
Lazy MCP loading so you're not pasting the full schema of twelve servers into every prompt. If you run a lot of MCP servers, the right ones for Claude Code plus lazy loading keeps your tool-definition block lean — and a lean tool block is a cheaper thing to cache.

This is exactly what Tokenade automates across Claude Code, Cursor, Codex, Copilot, Windsurf and the rest: it does the semantic retrieval, the output filtering, the skeleton compression and the lazy MCP loading for you, then shows the savings on a dashboard so you can see which lever actually moved the needle. It's source-available under MIT, so you can read exactly what it does to your prompts — and the free tier covers about 10M tokens a month, which is more than enough to see whether the savings are real before anyone pays anything. Pro is $19.90/month excl. tax in the US (€19.90/month TTC in France) with unlimited machines. You set up caching once on the provider side; Tokenade handles the part that changes every call.

A quick before/after worth doing yourself

Run one real session twice and read the bill. Take a typical coding task, run it with your prompt in whatever order your framework defaults to, and note the input-token count and cache-read count from the API response. Then reorder: stable system prompt and tool schemas first, pinned files next, a single cache breakpoint after them, volatile user turn last. Run the same task again. If you were getting near-zero cache hits before — and a lot of agent setups do, because they inject a timestamp or reshuffle tools every call — you'll typically see the cached-read share of your input jump from almost nothing to the majority of prefix tokens, which on Opus means paying $0.50/MTok instead of $5 on the bulk of your input. That's a 90% cut on the cached portion, achieved by changing nothing but the order of your prompt. I've yet to find a cheaper optimization. It is, frankly, a little embarrassing how much money sits on the table here.