Domain fine-tuning: when to adapt an open-weight model, and how to do it safely

Fine-tune for behaviour, retrieve for facts. If the problem is that the model answers in the wrong shape, uses the wrong register, ignores your house format or wobbles on a narrow classification it sees thousands of times a day, adaptation is the right tool. If the problem is that the model does not know this quarter's policy, a new product or a customer's contract, retrieval is the right tool, because those facts will change next month and a trained model would quietly keep the old version.

Most real systems use both. A model adapted to your domain vocabulary and output format still reads current documents at request time, and still answers only from what the asking person is allowed to see. Training from scratch sits apart from the other three. Choose it only when the thing you need to own is a model whose every training document you can account for and whose licence is yours alone, and accept that this is a research programme, not a project.

One practical test: write down ten failures of the current system. If most are wrong facts, build retrieval. If most are wrong behaviour on correct facts, consider adaptation. If you cannot write down ten, you are not ready to train anything.

Licence comes first, and it is per checkpoint, not per family. Read the licence file that ships with the exact weights you download and record the revision. As a recommendation, start from bases released under permissive licences such as Apache 2.0 or MIT, because derivative weights stay easy to deploy and to sell on. The Llama licence carries an EU restriction on some multimodal models, set out on the deploy-models hub; have counsel read the current text before an EU deployment.

Then size against hardware. A base that only fits your serving hardware at aggressive quantisation is a base you will spend months fighting. Check language coverage for every language your users write in, English included, and test it rather than trusting a model card. Finally, consider EU suitability as a whole: where the weights came from, whether you can host them in region, and whether the maker's terms let you keep everything inside your own infrastructure. Families such as Qwen, Mistral, Gemma and DeepSeek are common starting points; which bases Swfte supports for customer adaptation is <supported base models - founder to fill>.

Consider a compliance operations team in a regulated firm. Every day it receives incoming correspondence that must be sorted into a fixed set of categories and answered with a first draft in the firm's own terminology and structure. A general model, well prompted, gets the categories mostly right but keeps drifting into generic phrasing and occasionally invents a category that does not exist.

The team writes down its failures and finds they are behaviour, not facts: the model knows enough, but does not hold the format or the vocabulary. So it chooses a small open-weight base under a permissive licence, records the revision, and assembles labelled examples from past correspondence its reviewers had already checked, after sanitisation and with its legal team's decision on lawful basis on file. It trains a LoRA adapter, keeps a held-out set the adapter never saw, and runs the domain, safety and multilingual suites against both the adapter and the prompted model in production.

The adapter wins on the domain cases and holds its refusal behaviour, so it is promoted to staging, then to production, with the old configuration kept routable. Policy references still come from retrieval, because they change. The draft still goes to a person before it is sent.

Today Swfte hosts and serves models you have adapted yourself. The Model Vault takes your weights, records a hash manifest and the base model, versions them and promotes them through development, staging and production to a dedicated endpoint, and Connect puts the governed API in front. Swfte does not run fine-tuning jobs for customers today. A small proof-of-concept adapter pipeline has been run internally, at small scale; it is not a service.

Managed adaptation runs, with data selected from the company brain, evaluated before promotion and deployed into the vault, are designed for and on the roadmap. Adaptation methods that would be offered: <supported adaptation methods - founder to fill>. Availability and plans: <custom model availability and plans - founder to fill>.