Context rot is the gradual degradation in an LLM's output quality as its context window fills up with stale, redundant, or low-signal tokens over the course of a long session. The model doesn't crash; it just quietly gets worse — it forgets instructions you gave twenty turns ago, re-reads files it already saw, contradicts its own earlier decisions, and starts confidently hallucinating because the actual signal is now buried under a pile of noise.The mechanism is straightforward once you stop thinking of the context window as "memory" and start thinking of it as a single flat prompt that gets re-sent on every turn. A coding agent doesn't have a tidy working memory — each turn it ships the entire accumulated transcript back to the model: every file it read, every tool result, every dead end. Attention is finite and roughly competitive across all those tokens, so when 80% of the window is a function body the model glanced at once and a 4,000-line npm install log, the 20% that actually matters has to fight for the model's attention. That's the rot: not a hard limit you hit, but a slow signal-to-noise collapse you slide into.And you pay for it twice. The quality drops and the bill climbs, because every rotten token is re-billed as input on every subsequent turn. I've watched a single agent session balloon to a six-figure-token transcript where maybe a tenth of it was load-bearing — the rest was rot the model dragged around for the whole conversation, at full input price each turn.
Why context rot matters in 2026
It matters because agents are autonomous and long-running now. A 2024 chatbot turn was you, one question, one answer. A 2026 coding agent runs for fifty turns unsupervised, reading dozens of files and shelling out commands whose output it dumps straight into context. Rot compounds linearly with session length, so the longer and more useful the agent should be, the worse it actually gets — exactly backwards from what you want.There's measurable evidence the decay is real, not just vibes. Chroma's Context Rot report found that LLM performance degrades non-uniformly as input length grows, even on tasks well within the advertised window — models with a "1M token context window" do not perform at 1M tokens the way they do at 8K. The window size on the spec sheet is a ceiling, not a promise of constant quality up to that ceiling.The fix is not "buy a bigger window." A bigger window is a bigger bucket to fill with rot; it raises the ceiling without raising the floor. The fix is to control what enters the context in the first place — which is what context engineering is about, and where a tool earns its keep. Tokenade attacks rot at the source: semantic code search so the agent retrieves only the relevant code instead of reading whole directories, skeleton compression so a file enters the window as signatures rather than full bodies, and output filtering so that 4,000-line install log lands as the three lines that mattered. Less garbage in means less rot to drag around — and a savings dashboard so you can actually see the difference. If you're starting from a Claude Code setup, the token-reduction walkthrough covers the concrete moves.
When NOT to worry about it
Short, single-shot tasks. If your agent reads two files, makes an edit, and exits, there's no session long enough to rot. Compression overhead would cost you more attention-tax than it saves.
When the "rot" is actually a recall problem. If the model needs every line of a 200-line file to answer correctly, aggressively summarising it isn't fighting rot — it's amputating signal. Compression is about dropping what the model won't use, not starving it of what it will. Know which one you have before you reach for the knife.
When prompt caching already covers your cost concern but not your quality one. Caching makes re-sent tokens cheap (cache reads run ≈10% of normal input price), so a stable, rarely-changing prefix is nearly free to keep around. But caching makes rot cheaper, not less harmful — a cached pile of noise is still noise competing for attention. Don't let a good cache-hit rate lull you into never pruning.
