How to fine-tune an LLM on your own data

Write the task in one sentence with a subject, a verb and a boundary: classify inbound support tickets into five categories; extract invoice totals into a fixed JSON schema; draft a claims summary in the house format. Then write what a correct output looks like, in enough detail that two reviewers would agree.

Before any training, set aside a test set and do not touch it. Pick cases that include the hard and rare ones, not only the typical. A few dozen is enough to start; more makes the result steadier. Score the current prompt on it and record the number and the date. That number is the bar.

Choose the metric to match the output: exact match for labels and fixed formats, field-level accuracy for extraction, a rubric or human review for free text. If you plan to use a model as a judge, read the pitfalls in how to validate your AI first. Decide in advance how much improvement counts as a win and what regressions disqualify the model.

Training data is pairs of an input and the output you want. The common format is chat style: a list of messages ending with the assistant reply. Hugging Face TRL accepts conversational data like this, and also prompt and completion pairs, and applies the model chat template for you. OpenAI's supervised fine-tuning takes JSONL with a messages list in the same shape. One file can serve both routes.

Volume: OpenAI documents a minimum of 10 examples for its service and reports improvements from 50 to 100 examples, and advises starting with about 50 good ones. For open weights, plan on a few hundred to a few thousand reviewed examples as a working range; that is our guidance, not a documented threshold, and the held-out score is the real judge. More low-quality data is worse than less clean data.

Quality rules that matter more than volume. Cover the hard cases. Make labels consistent by settling disagreements in a written guideline. Remove duplicates and near-duplicates, which cause the model to over-weight one pattern. Remove personal data and anything you have no right to use. Record where each example came from, who approved it and when. Our page on data preparation for custom models sets out selection, sanitisation and consent.

Keep three sets apart from the start: train, validation (used to watch for overfitting during training) and the test set from step 1. If one customer, document or conversation yields many examples, split by that unit, or near-copies leak across the boundary.

Read the provider's data terms before uploading. OpenAI states that data sent to its API is not used to train or improve its models unless you opt in, and that abuse-monitoring logs are retained for up to 30 days by default. Those are statements about general API data; confirm how your training files are handled for the specific service and agreement you use, and repeat this check for any other provider.

OpenAI's fine-tuning guide lists several methods. Supervised fine-tuning, where you provide examples of correct responses, is what this guide uses; the others (vision fine-tuning, preference optimisation and reinforcement fine-tuning) exist for different needs. At the time of reading, the page named gpt-4.1-2025-04-14, gpt-4.1-mini-2025-04-14 and gpt-4.1-nano-2025-04-14 as supported for supervised fine-tuning. The tunable list changes, so check the current page before you plan around a model.

The flow is: upload the JSONL file with purpose fine-tune, create a job with the file id and the base model, wait for it to finish, then call the resulting model id like any other model. OpenAI documents the fine-tuned model id as a string beginning ft:. Evaluate it against your test set exactly as you would any other model, and keep its id and the training file id in your model record.

TRL's SFTTrainer is the standard open-source way to run supervised fine-tuning. Pass it a model id and your dataset; add a peft_config to train a small LoRA adapter instead of every weight. The TRL docs suggest a higher learning rate, around 1e-4, when training adapters, and show that conversational datasets get the chat template applied automatically.

Choose a base model whose licence allows your use, because the tuned model inherits it. Read the licence on the exact checkpoint. Start small, in the 0.5B to 4B range, so a run takes minutes to hours. For memory sizing, QLoRA setup, merging the adapter and converting to GGUF, follow how to create your own local model: that guide has the full commands and this one stays at the method level.

Watch validation loss. If it rises while training loss keeps falling, you are overfitting: use fewer epochs, more varied data or a lower rank. Training loss alone proves little. The only result that counts is the held-out comparison in the next step.

Run the tuned model on the frozen test set from step 1, with the same prompts and settings as the baseline, and compare. Read the misses one by one: they tell you whether the data, the task definition or the base model is the limit. Compare on the slices that matter, such as the rare categories and the longest inputs, not only the average.

Then test what you did not train on. Fine-tuning can erode safety behaviour even when the new data is harmless, so run the same refusal, over-refusal and prompt-injection checks on both models and compare. Test every language you serve. Check general ability on a few tasks unrelated to your own. Make these checks gates: a candidate that improves the task score and regresses on safety does not ship. The method for building the suite, calibrating judges and writing the evidence pack is in how to validate your AI, and evaluation and safety for custom models covers the same ground in a governed deployment.

Record the result with the date, the model and dataset versions, the settings and who signed it off. If the improvement is smaller than the margin you set, do not ship it and call it a win. Improve the data, or go back to prompting and retrieval.

Compute is the visible cost and usually the smaller one. Estimate with a formula, using your own numbers. Total cost is roughly the labelling and review hours times their rate, plus training cost, plus the cost of running the model for as long as it stays in service, plus re-testing. Training cost on a hosted API is the number of training tokens times the provider's price per million training tokens, so read the price from their pricing page on the day. On rented GPUs it is GPU hours times the hourly rate.

A worked example with made-up numbers, to show the shape, not to quote: 1,000 examples of 600 tokens each is 600,000 tokens per pass; three passes is 1.8 million training tokens. If a provider charged a hypothetical 10 currency units per million training tokens, training would cost 18 units. If subject experts spent 60 hours reviewing at a hypothetical 50 units an hour, review costs 3,000 units. The review hours dominate the compute, which is the usual pattern.

Then the cost that people forget: running it. A model you host needs serving capacity and monitoring, and a model on a hosted API may be priced differently from the base model. If call volume on the task is low, the people cost of owning a custom model can exceed any saving. Name an owner who will re-test it when the base model, the task or the data changes, and who can retire it.

Put the model behind an interface your application already uses so you can swap it. Many servers expose an OpenAI-compatible API, so the model name and base URL can live in configuration, which is what makes a rollback to the previous model a one-line change. For serving open weights, how to self-host an LLM covers a production server, and a gateway (how to set up an LLM gateway) gives you routing and fallback.

Roll out gradually: send a small share of traffic to the tuned model, compare against the old path on live cases, and keep the old path available. Keep the model record current: base model and revision, licence, dataset and test versions, settings, results and owner.

Set re-test triggers: a new base model version, a change to the task or policy, a drop in the live score, or a fixed review date. Fine-tuned models are not set and forget, because the facts and rules around them keep moving.