Every few weeks a lab ships a model better than the one you are on, and the same anxiety goes round the team: are we on the right model? It has been the wrong question for about a year. In July 2026 the third-best model in the world became a free download and a 27-billion-parameter model started running on phones. When frontier-adjacent capability is something anyone can have for nothing, having it is not an advantage — your competitors are on the same model, and they will be on the next one the same week you are. Whatever separates you from them, it is not the base model.
This piece makes the strategic case and then gets specific about the mechanics: when to fine-tune rather than prompt or retrieve, how much data you actually need, what LoRA is doing under the hood, how to evaluate the result honestly, and how to deploy it without betting the product on it.
The commoditisation, in three data points
Moonshot released the weights for Kimi K3 — 2.8 trillion parameters, third on the Artificial Analysis index, first in Frontend Code Arena, free to download and two to three times cheaper to serve than the models above it. PrismML released Bonsai 27B, compressing a capable model into 3.9 GB under Apache 2.0 and running it offline on a phone. Alibaba announced a 2.4-trillion-parameter Qwen 3.8 with open weights, having previously kept its largest models API-only.
The mechanism underneath is the efficiency race: the price of any fixed level of capability falls steeply and continuously while the frontier moves on. What changed in 2026 is the interval. The gap between state-of-the-art and free used to run in years. It now runs in months, and at the tier where most production work happens it is close to zero.
Follow that to its conclusion. A capability available to everyone, at no cost, within months of appearing, cannot carry your differentiation. The general model is becoming what a database is — mandatory, and completely uninteresting as a source of advantage. Nobody wins because they have Postgres.
What a general model does not know
A frontier model knows an enormous amount about the world and nothing about your part of it.
It has never read a support ticket from your customers. It does not know that "settlement" in your industry means something specific and different from its ordinary sense, that your three largest accounts have contract terms overriding the standard ones, or that error code 4021 means a hardware fault rather than the configuration problem the wording implies. It does not know which internal documents are current and which are three reorganisations stale. It does not know that when your customers write "it's not working," forty percent of the time they mean one specific thing.
Some of that gap closes with context. Retrieval puts your documents in front of the model, a long system prompt encodes rules, few-shot examples demonstrate format. These should usually be your first move — cheap, fast, reversible, and frequently sufficient.
They also have a ceiling, and the ceiling is structural in four ways.
You pay per request, forever. A 4,000-token system prompt encoding domain rules is billed on every call for the life of the product. At 10 million calls a month that is 40 billion tokens of instructions you re-send because the model cannot remember them.
It competes for attention. Context is not free capacity; it is shared capacity. Rules occupy the same window as the user's question, and models follow the fifteenth item in a list considerably less reliably than the third.
It degrades as it grows. Long-context recall is uneven, and instructions placed in the middle of a large prompt get followed least consistently.
It is not yours. Behaviour prompted into existence changes when the provider updates the model underneath you, and you find out from a regression rather than a changelog.
Fine-tuning moves that knowledge out of the prompt and into the weights. The rules stop being instructions the model has to read each time and become how the model behaves. You stop paying for them per request, they stop competing with the user's input, and they stop depending on a provider's release schedule.
Prompt, retrieve, or tune: the actual decision
The most common expensive mistake is tuning a model to solve something a better prompt would have fixed on Tuesday. The second most common is trying to teach facts with fine-tuning.
Prompt when the behaviour you want can be described. Fast, free, instantly reversible. Exhaust this first.
Retrieve when the model lacks facts — current documents, account specifics, anything that changes. Fine-tuning is a poor and expensive way to install knowledge, and a model tuned on facts will state outdated ones with total confidence long after they change. Facts belong in a retrieval index where you can update them in one place.
Fine-tune when the model lacks behaviour — a house style it cannot be told into, an output schema it violates one time in twenty, a classification boundary specific to your domain, a tone, a reasoning pattern, a set of conventions that would take two thousand tokens to describe and still not stick.
The sequence is prompt, then retrieve, then tune. Tune when a stable, high-volume task has plateaued below where you need it under good prompting, and you have accumulated real examples of the work done correctly. That is a specific situation, and it is also extremely common in any product that has been running six months — most teams are sitting in it without noticing, because the training set is in their logs and nobody has looked.
What specialisation buys
Quality on your distribution. A well-tuned small model routinely beats a much larger general model on the narrow task it was tuned for. This stops being surprising once stated plainly: the general model allocates capacity to poetry, organic chemistry and nineteen languages you do not use, while yours allocates everything to the thing you do. On classification, extraction, structured generation and domain drafting, a tuned 7B–30B model matching or beating a frontier model is an ordinary outcome.
Cost, by roughly an order of magnitude. Worth doing the arithmetic rather than asserting it. Take 10 million requests a month at 1,500 input and 400 output tokens each.
At frontier rates of $3.00 per million input and $15.00 per million output: (1,500 ÷ 10⁶ × $3.00) + (400 ÷ 10⁶ × $15.00) = $0.0105 per request, or $105,000 a month.
At small-model rates of roughly $0.20 input and $0.60 output: (1,500 ÷ 10⁶ × $0.20) + (400 ÷ 10⁶ × $0.60) = $0.00054 per request, or $5,400 a month.
You will not move everything. Route 75% of traffic to the tuned model and keep the hard quarter on the frontier: 0.75 × $5,400 + 0.25 × $105,000 = $30,300, against $105,000. A 71% reduction without touching the quality of the difficult cases, because those never left the frontier model.
Latency and locality. Small models are fast, and they run where large ones cannot — inside your VPC, on a laptop, on a device with no network. For interactive work latency is quality. For regulated work locality is the whole question, covered in data sovereignty.
Consistency. Underrated. A tuned model does the same thing every time in a way a prompted model does not, because the behaviour lives in the weights rather than in an instruction that may be weighted differently on a given pass. Where you have a required schema, a mandatory disclaimer, or a forbidden phrasing, consistency is worth more than raw capability.
The fifth return has no metric: the weights encode work nobody else has done, on data nobody else has, describing a problem nobody else understands as well.
What LoRA is actually doing
Worth understanding, because it explains why tuning is cheap now and why the economics changed.
Full fine-tuning updates every parameter, which means holding the model, its gradients, and optimiser state in memory at once — for a 7B model in mixed precision that is comfortably over 100 GB, and you produce a new full-size model per experiment.
Low-Rank Adaptation starts from an observation about the update rather than the model: the weight change a fine-tune needs, ΔW, has far lower intrinsic rank than the matrix it modifies. So instead of learning a d×k update directly, you learn two thin matrices — B of shape d×r and A of shape r×k — and use their product, with r typically 8 to 64. The base weights are frozen; only A and B receive gradients.
The parameter count falls from d×k to r(d+k). For a 4096×4096 attention projection at rank 16, that is 16.8 million parameters replaced by roughly 131,000 — a 128× reduction on that matrix. Applied across the attention projections of a 7B model at rank 16, a typical adapter lands around 20–40 million trainable parameters, well under 1% of the model, and serialises to tens of megabytes.
Three consequences follow, and they are the practical case for LoRA:
Training runs in minutes to hours on a single GPU rather than days on a cluster, which means you can afford to iterate rather than betting a quarter on one run.
Adapters compose and swap. Because the base is untouched, you can hold several adapters against one loaded base model and select per request — one for support triage, one for contract extraction, one for internal search — without loading three models.
Forgetting is bounded. Full fine-tuning on a narrow dataset degrades general capability, sometimes badly. Freezing the base and constraining the update to a low-rank subspace limits how far the model can drift from what it already knew.
Full fine-tuning still has a place: deep domain shift, an unusual output modality, a genuinely different language distribution. Reach for it when a LoRA run has demonstrably plateaued, not before.
How much data you actually need
The honest ranges, which vary more by task than anyone likes:
Format and schema conformance: a few hundred examples is often enough. You are teaching a shape, and shapes are learned quickly.
Tone, style, house voice: 500 to 2,000. Enough to cover the range of situations, not so many that you are mostly adding duplicates.
Domain classification and extraction: 1,000 to 5,000, weighted toward the boundaries. Fifty examples of an ambiguous edge case teach more than a thousand obvious ones.
Genuine domain shift — vocabulary, reasoning patterns, conventions the base model has never seen: 10,000 and up, and consider full fine-tuning.
Below roughly 200 examples you will overfit and ship something worse than you started with. Quality dominates quantity past a fairly low threshold: 500 examples you have actually read beat 5,000 you have not, because a model trained on your mistakes reproduces your mistakes with excellent fidelity.
If you have the inputs but not the labels — millions of queries and no ground truth — that is the situation distillation exists for. A larger teacher model generates the outputs, converting compute into a training set without an annotation budget, and you tune on top of the result.
Fine-tuning in Swfte
Most teams that should tune do not, because the operational path has historically been miserable: assemble a dataset by hand, rent GPUs, babysit a run, build an evaluation harness from nothing, work out serving, then discover you have no way to tell whether the result is better than what it replaced. Six weeks of infrastructure for something that should take an afternoon.
The pieces are already in the platform. Your traces are being recorded. Your evaluation harness exists. Your routing layer already knows how to send traffic somewhere new.
1. Build the dataset from your traces
In Studio, go to Models → New fine-tune. The default source is your own traffic — the traces Nexus has been capturing across your agents and workflows. Filter to the workload in question by agent, tool, route, or date range, and Studio assembles a candidate set from real requests and real responses.
The most common cause of a failed fine-tune is a training set that does not resemble production, and the surest way to resemble production is to be production. You can also upload JSONL if you have curated data elsewhere, or start from a dataset in the Marketplace.
2. Curate, then read fifty by hand
Studio flags what quietly poisons a run: near-duplicates that cause the model to over-weight one pattern, traces where the user immediately retried (a decent negative-signal proxy), responses that failed schema validation, and length outliers. Accept or reject in bulk.
Then read fifty examples yourself. Half an hour here outperforms any hyperparameter you will touch later, because no tooling can tell you that the "correct" responses in your logs encode a policy you changed in March.
Hold back 10–20% as a test split before training. Studio does this by default and will not let you evaluate on data the run has seen.
3. Pick a base and a method
The base list spans open weights you can host anywhere — Qwen, Llama, DeepSeek and Kimi families among them — with size and licence against each. Bias toward the smallest base that plausibly clears your bar; the entire point of specialisation is that you no longer need the largest one.
Method defaults to LoRA, rank 16, targeting attention projections. That default is right for most tasks. Raise the rank toward 32 or 64 when the behaviour you are teaching is complex and a rank-16 run underfits; lower it when the dataset is small and you are overfitting.
4. Evaluate against the incumbent, on the right shape of work
Studio will not promote a tuned model without an evaluation. Point it at the held-out split and it scores the candidate against the model currently in production on metrics you define: exact match, schema conformance, an LLM-judge rubric, latency, cost per answer.
Two rules that save real pain.
Compare against what you are running now, not an absolute threshold. "87% accuracy" is meaningless without knowing the incumbent scored 91%.
Evaluate on the shape of work you run. If your workload is multi-step and tool-using, single-turn accuracy will not detect the failure that matters. The compounding arithmetic in the quantization numbers shows why: a four-point drop in per-step reliability becomes a thirty-point drop in completion across a twenty-step trajectory. Short benchmarks cannot see it. Trajectory evaluations can.
5. Deploy as a route, not a replacement
Promotion does not swap your model. It registers a candidate in Connect and you send it a slice of live traffic — 5%, then 20%, then more — with the frontier model still handling anything the tuned one is not confident about, and automatic rollback on a metric regression.
This is what makes tuning inside a platform safer than tuning in a notebook. A fine-tune is a hypothesis; routing tests it against reality at whatever pace your risk tolerance allows. Mechanics in intelligent LLM routing. Deployment targets are a managed Connect endpoint, your own VPC, or compressed to the edge through the Embedded SDK.
6. Close the loop
The tuned model produces its own traces, including its failures, and those return to step one. A specialised model is not a project that finishes. It is a system that improves as long as you keep feeding it, and each cycle widens the distance between what you have and what anyone can buy.
What the end state looks like
Not one clever model — a portfolio.
A small tuned model handles the 70–80% of traffic that is routine, at a tenth of the cost and a fraction of the latency, behaving consistently because the behaviour is in the weights. A frontier model handles the hard tail and the high-stakes cases where being wrong costs far more than the tokens. A router chooses per request and swaps in whatever is best and cheapest as the ground moves, which it does roughly every eight weeks.
Under that arrangement a new frontier release is an upgrade to one component rather than a migration. Your accumulated advantage sits in the tuned layer and the dataset behind it, and it survives the churn — you re-tune onto the better base and keep everything learned. The dataset is the durable asset. The adapter is a build artefact.
The bottom line
The model cannot be the moat. It is a download, your competitors have the same one, and next quarter you will both have something better for less. Effort spent winning on base-model selection is spent on a race that resolves to a tie.
The moat is the fit between a model and a problem only you understand well, built from data only you have and failures only you have seen, compounding every month you work on it. Commoditised general intelligence does not erase competitive advantage in AI — it relocates it away from the labs and toward whoever sits closest to the problem.
That relocation is the point. Making capable AI free and universal is how it reaches everyone rather than the few who can afford it, and the moment it is free and universal, the interesting question stops being which model and becomes what do you know that nobody else does. That question has always favoured the specialist.
Start where the data already is: open Studio, look at what your agents have been doing for the last six months, and check whether you have been sitting on a training set without noticing.