Effective Context Engineering for AI Agents

What does effective context engineering for AI agents actually mean?

Effective context engineering means treating the agent's context window as a scarce, expensive resource and deciding — deliberately, per turn — exactly what goes into it, in what shape, and in what order. It is not prompt wordsmithing. It is the discipline of managing the information an agent reads while it works, because in an agentic loop that information is re-read on every single step and you pay for it every time. I have a Ph.D. in AI and I spend most of my week building tooling that cuts what coding agents spend. The single most consistent finding across every agent I've profiled — Claude Code, Cursor, Codex, Copilot, Windsurf — is that the model's answer quality is rarely the bottleneck. The context is. Feed an agent the three functions that matter and it solves the task in one pass. Feed it your whole repository "to be safe" and it gives you a worse answer, slower, for ten times the money. Context engineering is how you stay on the right side of that line. This piece is the practical companion to Context Engineering for AI Coding Agents, which lays out the broader framing. Here I want to be concrete: the mechanism that makes context so expensive, the four techniques that actually move the bill, and the mistakes I see people make every week.

Why is context so expensive in an agentic loop?

Context is expensive because an agent re-reads its entire context on every turn, so anything you put in early is billed again on every step that follows. This is the one mechanical fact that, once you internalise it, reorganises how you think about agent cost. A single-shot chat completion bills the input once. An agentic agent does not work in a single shot. It plans, calls a tool, reads the result, plans again, calls another tool — and on each of those steps the model is handed the full conversation so far: the system prompt, every tool definition, every file it has read, every command's output, and every word it has already said. A 6,000-token file read on turn 2 is still sitting in the prompt on turn 25. You paid for it twenty-four times. The numbers make this vivid. At Claude Opus 4.8 input pricing of $5 per million tokens, a context that has bloated to 150,000 tokens costs $0.75 per turn just to re-read — before the model generates a single new token, and output is far pricier at $25 per million. A twenty-turn session at that size is $15 in re-reads alone. Prompt caching softens this — cache reads run at roughly 10% of the input price — but caching rewards stable prefixes, and an agent that keeps appending fresh reads keeps invalidating the tail of the cache. You can't cache your way out of a context you keep churning. See LLM API token pricing for the current per-model rates. So the goal of context engineering is not "use a smaller model" or "write a cleverer prompt." It is: keep the context small, stable, and relevant, turn after turn.

What are the techniques that actually move the bill?

The techniques that move the bill are retrieval over reading, output filtering, structural compression, and tool-surface pruning — in roughly that order of impact for most coding sessions.

Retrieve, don't read

The biggest lever is to stop reading whole files when you need one function. The default agent behaviour — "read src/auth/session.ts" — pulls 6,000 tokens to answer a question that lived in a 40-line function. The fix is to make the agent locate the relevant symbol first, then read only that body. Semantic code search does this by meaning rather than by exact string, so a query like "where do we validate the refresh token" finds the function even when none of those words appear in it. Done well, this is the difference between a 2,000-token retrieval and a 20,000-token one, and because that read is re-billed every subsequent turn, the saving compounds.

Filter tool output before it lands

The second lever is to never let raw command output flow straight into the context. An agent runs npm test, the suite emits 15,000 tokens of passing checkmarks, timing tables and stack traces, and all of it lands in the transcript. The agent needed maybe 50 tokens: the name of the failing test and the broken assertion. Everything else is noise that now re-bills on every turn. Output filtering — keeping the signal lines and dropping the cargo — is one of the cheapest, highest-return moves available, precisely because tool output is so reliably verbose.

Compress structure, not meaning

The third lever is structural compression: when the agent genuinely needs to understand a large file's shape rather than its full body, give it a skeleton — signatures, types, exported symbols, the class outline — instead of every line. A 500-line module's skeleton might be 400 tokens against 6,000 for the full text, and for "how is this module organised?" the skeleton answers the question completely. This is lossy on purpose: you drop implementation bodies the agent isn't going to touch. See context compression for the general idea.

Prune the tool surface

The fourth lever is the one people forget because it's invisible. Every MCP server you connect advertises its full tool schema on every turn, used or not. Five servers with ten tools each is a fixed tax paid the entire session, re-read on every step like everything else. Load tool definitions lazily — only when a tool is actually about to be used — and that constant overhead largely disappears. My best MCP servers for Claude Code walks through which ones earn their place.

How do you apply this today?

You apply context engineering by changing how you instruct the agent and by putting tooling between the agent and its raw inputs. Concretely:

Phrase tasks as targeted retrievals, not tours. "Find the function that builds the auth header and show me its signature" beats "read the auth module." Ban the open-ended file-tour phrasings — "go through the codebase," "check all the files in X" — from your own habits; they are instructions to bloat the context.
Start fresh per task. Don't run a week of unrelated work in one session. A new task in a clean context is cheaper than the same task dragged behind 80,000 tokens of stale history, because that history re-bills on every turn.
Filter at the source. Pipe noisy commands through a filter so only the relevant lines reach the agent. The full log stays available on disk; the context stays lean.
Reach for skeletons before full reads when the question is about structure or location rather than a specific implementation.
Trim your MCP surface. Disconnect servers you're not using this session, and prefer setups that load tool schemas on demand.
Measure. You cannot engineer what you don't see. Watch where the tokens actually go — usually it's eager reads and unfiltered output — and aim your effort there. The breakdown in AI coding agent token costs is a good map of where the money typically hides.

If you'd rather not hand-roll all of this, that's the gap Tokenade fills: it sits between your agent and its inputs and does the retrieval, output filtering, skeleton compression and lazy MCP loading automatically, with a dashboard showing exactly what each move saved. It works with Claude Code, Cursor, Codex, Copilot and Windsurf; the source is available under MIT, so you can read precisely what it does to your context before you trust it. Free up to ≈10M tokens a month, then Pro at $19.90/month (excl. tax), unlimited machines. The agent-specific playbook lives in How to reduce Claude Code token usage.

What goes wrong (anti-patterns)

The failure modes of context engineering are nearly all variations on one mistake: confusing "more context" with "better context." "Just in case" reading. The agent — or the human prompting it — loads files it might need rather than the files it does need. Every speculative read is re-billed for the rest of the session. More context here means worse, not better: the relevant signal gets diluted by irrelevant code, and answer quality often drops even as cost climbs. Letting logs become context. Treating raw tool output as something the model must see in full. It almost never does. If you find a 12,000-token build log sitting in your transcript, that's twelve thousand tokens re-read on every following turn for the sake of three lines that mattered. Caching as a substitute for discipline. Prompt caching is excellent, but it rewards a stable prefix. An agent that keeps appending fresh reads keeps breaking the cache where it appended. Caching is a multiplier on good context engineering, not a replacement for it. The never-ending session. Running task after unrelated task in one context because starting over feels wasteful. It isn't — the stale history is the waste. The clean restart is the cheaper move almost every time. Optimising the model instead of the input. Reaching for a cheaper model to fix a cost problem that is really a context problem. A bloated context on Haiku 4.5 is still a bloated context; you've just lowered the per-token rate on work that shouldn't exist. Fix the input first. The model choice is a second-order decision. Get the context right and most of the agent-cost problem evaporates — not because you found a discount, but because you stopped paying, over and over, for information the agent never needed.

See also: