"Build our own AI" means at least four different things, and most projects that fail do so because the people funding them and the people building them meant different ones.
It can mean training a model from scratch, which almost nobody should do. It can mean fine-tuning an open-weight model on your data, which many more should. It can mean retrieval, putting your documents in front of a general model at request time. Or it can mean running somebody else's model inside your own perimeter so the data never leaves. Those four have different costs, different timelines and different failure modes, and the word "own" is doing a lot of unhelpful work across all of them.
This is a guide to picking the right one, then building it, with the open source toolchain and the managed path laid out side by side so the trade is visible.
Four things you might mean
Pretrain from scratch. Seven to nine figures, a research team, eighteen months, and a result that will be beaten by a free download before you finish. There are perhaps thirty organisations on earth for whom this is the right call, and if you are reading a blog post to decide, you are not one of them. Skip it.
Fine-tune an open-weight model. Take Qwen, Llama, DeepSeek or Kimi weights and teach them your behaviour. Days to weeks, a few thousand dollars of compute, and the result is a model shaped around your work that you own and can host anywhere. This is what most people should mean.
Retrieval on top of a general model. Index your documents, fetch the relevant ones at request time, hand them to a frontier model. Fastest to stand up, easiest to keep current, and the right first move for anything that is fundamentally a knowledge problem. The architecture is covered in the RAG implementation guide.
Private deployment of an existing model. Run open weights inside your VPC or on your own hardware. Solves data residency without changing model behaviour at all. Often confused with the second option, and frequently the actual requirement hiding behind a request to "build our own".
Most successful projects use three of the four together: retrieval for facts, a fine-tune for behaviour, private deployment for the regulated slice.
Choosing between behaviour and knowledge
The single most useful distinction. Ask what the general model is missing.
If it is missing facts, meaning your documents, your account specifics, your current prices, anything that changes, use retrieval. Fine-tuning is an expensive and unreliable way to install knowledge, and a model tuned on facts will state outdated ones with complete confidence long after they changed. Facts belong in an index you can update in one place.
If it is missing behaviour, meaning a house style it cannot be instructed into, an output schema it violates one time in twenty, a classification boundary specific to your domain, a set of conventions that would take two thousand tokens to describe and still not stick, fine-tune. That behaviour moves out of the prompt and into the weights, which means you stop paying for it on every request and it stops competing for attention with the user's actual question.
The order is prompt, then retrieve, then tune. Exhaust the cheap reversible options first. Tune when a stable, high-volume task has plateaued below where you need it under good prompting, and you have accumulated real examples of the work being done correctly. That situation is far more common than teams realise, because the training set is usually sitting in their logs and nobody has looked at it.
The open source stack, honestly
Here is what you actually assemble if you build this yourself in 2026. Every piece is good software. The problem is never any single piece.
| Layer | Common choice | What it costs you |
|---|---|---|
| Base weights | Qwen 3.6, Kimi K3, DeepSeek V4 | Licence review |
| Training | Axolotl, Unsloth, torchtune | Config archaeology |
| Method | LoRA or QLoRA via PEFT | Hyperparameter guesswork |
| Compute | Rented H100 or B200 hours | Queue time, spot evictions |
| Tracking | Weights and Biases, MLflow | Another service to run |
| Evaluation | lm-eval-harness plus custom | Usually built from nothing |
| Serving | vLLM, SGLang, TGI | Real operational depth |
| Retrieval | pgvector, Qdrant, LanceDB | Index freshness, chunking |
| Routing | Written in-house | Nobody plans for this |
The training itself is the easy part. Unsloth will fine-tune a small model on a single GPU in under an hour, and the tutorials are good. What consumes the quarter is everything around it: assembling a dataset that resembles production, building an evaluation harness that can tell you whether the result is better than what it replaced, standing up serving that does not fall over, and working out how to send it traffic without betting the product on it.
We have watched teams lose two months at the evaluation step alone, because they trained something, could not prove it was better, and did not want to ship it on faith. That is the correct instinct and the wrong position to be in.
The honest recommendation on the open source path: it is the right choice when you have a machine learning engineer who has done this before, when your compliance position requires that every component be yours, or when the specific thing you are doing is unusual enough that a platform's defaults would fight you. Those are real situations. They are just less common than the number of teams attempting this route.
What LoRA is doing, briefly
Worth understanding whichever path you take, because it explains why this got cheap.
Full fine-tuning updates every parameter, which means holding the model, its gradients and optimiser state in memory simultaneously. For a 7B model in mixed precision that is comfortably over 100 GB, and each experiment produces a new full-size model.
Low-Rank Adaptation starts from an observation about the update rather than the model. The weight change a fine-tune needs has far lower intrinsic rank than the matrix it modifies, so instead of learning a full update you learn two thin matrices whose product approximates it, with rank typically 8 to 64. The base weights stay frozen.
For a 4096 by 4096 attention projection at rank 16, that replaces 16.8 million parameters with roughly 131,000. Across a 7B model, an adapter lands around 20 to 40 million trainable parameters and serialises to tens of megabytes. Three things follow: training runs in minutes to hours on one GPU rather than days on a cluster, adapters swap against a single loaded base so you can hold several specialisations at once, and the frozen base bounds how far the model can drift from what it already knew.
How much data you need
The ranges vary more by task than anyone likes.
Format and schema conformance takes a few hundred examples, because you are teaching a shape. Tone and house style takes 500 to 2,000. Domain classification and extraction takes 1,000 to 5,000, weighted heavily toward the boundaries, since fifty examples of an ambiguous edge case teach more than a thousand obvious ones. Genuine domain shift, with vocabulary and reasoning patterns the base model has never encountered, takes 10,000 and up.
Below roughly 200 examples you will overfit and ship something worse than you started with.
Quality dominates quantity past a low threshold. Five hundred examples you have actually read beat five thousand you have not, because a model trained on your mistakes reproduces your mistakes with excellent fidelity. If you have inputs but no labels, that is what distillation is for: a larger teacher generates the outputs, converting compute into a training set without an annotation budget.
The same thing in Swfte
The managed path exists because the pieces most teams are missing are the ones they already have data for and no tooling around.
In Studio, a fine-tune starts from your own traffic rather than an upload. The traces Nexus captures across your agents and workflows are already real requests and real responses, and the most common cause of a failed fine-tune is a training set that does not resemble production. Filter by agent, tool, route or date range and Studio assembles a candidate set from work that already happened.
Curation flags what quietly poisons a run: near-duplicates that over-weight one pattern, traces where the user immediately retried, responses that failed schema validation, length outliers. Accept or reject in bulk, then read fifty by hand yourself. That half hour outperforms any hyperparameter you will touch later, because no tool can tell you the "correct" responses in your logs encode a policy you changed in March.
Base models span the open-weight families with size and licence against each, and the method defaults to LoRA at rank 16 against attention projections, which is right for most tasks. Bias toward the smallest base that plausibly clears your bar.
Studio will not promote a model without an evaluation against the incumbent on a held-out split, which is the step that stalls in-house projects. Compare against what you are running now rather than an absolute threshold, and evaluate on the shape of work you run: if your workload is multi-step and tool-using, single-turn accuracy will not detect the failure that matters, because a four-point drop in per-step reliability becomes a thirty-point drop across a twenty-step trajectory.
Promotion registers a candidate in Connect rather than swapping your model. You send it 5% of live traffic, then 20%, with the frontier model still handling anything the tuned one is unsure about and automatic rollback on a regression. Deployment targets are a managed endpoint, your own VPC, or compressed to the edge through the Embedded SDK.
The five ways these projects die
Having watched a fair number of them, the failure modes repeat with depressing consistency.
Tuning something a better prompt would have fixed. The most common and the most expensive. Before any training run, spend a day trying to instruct the behaviour into existence. If a careful prompt gets you eighty per cent of the way, tuning will get you the rest cheaply. If a careful prompt gets nowhere, understand why before you assume gradients will succeed where language failed.
Trying to teach facts. Covered above but worth repeating, because it is the mistake that produces a model everyone loses confidence in. A tuned model states its outdated training data with total conviction and gives no signal that it is doing so. Retrieval fails loudly. Fine-tuning on facts fails silently, which is far worse.
No held-out split. Evaluating on data the run has seen produces a number that looks excellent and means nothing. Hold back ten to twenty per cent before training, not after, and never let a promotion decision touch it more than once.
Chasing benchmark scores instead of your own metric. A model that improves on MMLU and degrades on your ticket classification is a worse model. Public benchmarks are useful for choosing a base and useless for judging a fine-tune, because your fine-tune was not aimed at them.
Treating it as a project rather than a loop. The tuned model produces its own traces, including its failures, and those are the next dataset. A specialisation that shipped in March and has not been retrained since is drifting away from a business that kept moving. Plan for a retraining cadence before the first run, because retrofitting one is the thing nobody gets budget for.
There is a sixth that is less a mistake than a mismatch. Some teams want a custom model because the phrase sounds like ownership, when what they actually need is private deployment of an existing one. If the requirement is that data never leaves the building, running open weights in your own perimeter solves it completely, in a fortnight, with no training at all. Establishing which of those you are being asked for is worth more than any technical decision that follows.
What it costs either way
A first useful fine-tune on the open source path realistically costs a competent engineer six to ten weeks and somewhere between two and eight thousand dollars of compute across the failed runs nobody budgets for. The compute is not the expensive part.
On the managed path the same result is days, because the dataset, the evaluation harness and the routing already exist. What you give up is control over the internals, and for a team whose whole differentiation is a novel training approach, that is a genuine loss.
Both produce the same durable asset, which is the actual point. A general model is becoming what a database is: mandatory, and useless as a source of advantage, because your competitors have the same one and will have the next one the same week you do. Nobody wins because they have Postgres. What does not commoditise is a model shaped by data only you hold, describing a problem only you understand properly, improving every time it runs.
Related: how to fine-tune an LLM on your own data, RAG architecture, and data sovereignty.