Fine-tuning is the process of taking a model that has already been trained on a huge general corpus and continuing to train it on a smaller, task-specific dataset of your own. You're not building intelligence from scratch — you're nudging an existing model's weights toward your particular distribution of inputs and the outputs you want back.Mechanically, you hand the training process a set of input/output pairs ("when the prompt looks like this, respond like that"), and gradient descent adjusts the weights so the model reproduces those targets. Most practical fine-tuning today is parameter-efficient — LoRA and friends — which freezes the base weights and trains a small set of adapter matrices instead. I did a PhD in AI and I'll still tell you the unglamorous truth: 90% of the work is the dataset, not the algorithm. A few hundred clean, consistent examples beat ten thousand noisy ones every time.The payoff that matters for the cost-conscious is this: once a model has internalised your format or domain, you no longer have to re-explain it in every prompt. The instructions move from the context window into the weights, and your per-request prompt gets shorter.
Why fine-tuning matters for token cost in 2026
For coding agents, the dominant spend isn't the clever one-off question — it's the same scaffolding shipped on every single turn. Style guides, output schemas, "always respond in this JSON shape", a wall of conventions in your CLAUDE.md. Each of those lines is re-tokenised and re-billed every request. At Opus 4.8's $5 per million input tokens, a 4,000-token preamble repeated across a day of agentic work adds up faster than people expect.Fine-tuning is one lever against that: bake the recurring instructions into a model and you stop paying for them as input. It's the structural cousin of prompt caching, which attacks the same repetition from the inference side rather than the training side. Both reduce what you re-send; fine-tuning just moves the knowledge permanently.That said, I want to be honest about scope. Fine-tuning teaches behaviour and form — tone, structure, how to map an input to an output. It does not reliably teach facts, and it is a poor tool for "knowing about your codebase". For that you want retrieval, not retraining: semantic code search and RAG pull the relevant chunks in at query time, so the model reasons over current code instead of a stale snapshot frozen into weights months ago. Most of the token savings teams chase actually come from better retrieval and context engineering, not from fine-tuning at all.
When NOT to use it
Your knowledge changes often. A codebase, a docs site, a pricing table — anything that updates weekly is wrong to bake into weights. Retrieval keeps it fresh; fine-tuning freezes it.
You haven't exhausted prompting first. A good system prompt plus a few in-context examples solves a surprising share of "we need fine-tuning" requests, with zero training run and zero new model to host.
You have fewer than ~50 solid examples. Below that the model overfits to quirks in your tiny dataset and gets worse, not better.
You're chasing factual accuracy. Fine-tuning won't stop hallucinations about your data; grounding the model with RAG will.
The recurring overhead is structural, not behavioural. If the bloat is "agent keeps reading whole files and dumping command logs", a custom model won't fix it — output filtering and retrieval will.
How Tokenade fits
Tokenade attacks the same bill from the cheap end, no training run required. It sits in front of your agent — Claude Code, Cursor, Codex, Copilot, Windsurf — and cuts tokens with semantic code search, output filtering, skeleton compression, and lazy MCP loading, with a dashboard so you can see exactly what you saved. For most teams that's where the first 60–90% of savings live, long before fine-tuning is worth the effort. It's source-available (MIT), free up to ~10M tokens a month, and Pro is $19.90/mo (excl. tax) with unlimited machines. Reach for fine-tuning when you've measured a genuinely repetitive behaviour worth distilling — and lean on retrieval and filtering for everything else.