EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

adapter
A small set of trained numbers that changes how a fixed model behaves and can be added or removed without retraining it.
base model
The original model a tuning run starts from, left unchanged when only an adapter is trained.
catastrophic forgetting
When training on a narrow task makes a model worse at the general skills it already had.
checkpoint
A saved copy of a model's learned numbers at one point in training, kept so a run can be resumed or compared.
dataset
The collection of examples a model learns from, split into training, validation and test parts.
epoch
One full pass over the training examples; several passes let the model see the data more than once.
hyperparameter
A setting chosen before training, such as how fast the model learns, rather than one learned from the data.
leakage
When information from the test or future data reaches training, making the score look better than it is.
learning rate
How far the model's numbers move on each update; too high overshoots and too low barely moves.
LoRA
A method that leaves the model fixed and learns two small grids of numbers whose product is added to chosen parts.
loss
A number that measures how far the model's predictions are from the answers in the data, and that each update tries to lower.
preference tuning
Training a model to choose the better of two answers, rather than to copy one correct answer.
supervised fine-tuning
Training a model on example inputs and the answers you want it to produce for them.
validation set
Examples held back from training and used to choose settings and the stopping point.

Fine-tuning and LoRA

When to fine-tune (and when not)

Fine-tuning is a behaviour change, not a knowledge upload. It earns its cost when the task is narrow, the data is good and the result can be measured.

Part 1 of 10

Advanced16 min readBack to Courses

What fine-tuning actually changes

Supervised fine-tuning adjusts the model’s weights toward your examples. It is good at teaching a shape: the format of an answer, the tone of a support reply, the boundary between two similar labels, the vocabulary of a domain. It is a poor way to add facts that change, because a fact learned from a snapshot is frozen at the snapshot and may be recited with confidence long after it is wrong. The weights move toward the examples, and the examples define what the model considers normal. It is a change of habit, not a database update.

That distinction decides most projects. If the problem is “the model does not know our current refund policy”, retrieval is the cheaper and more honest fix. If the problem is “the model knows the policy but answers in the wrong structure every time”, fine-tuning is a reasonable tool. The first is a knowledge problem; the second is a behaviour problem. The test is whether the answer lives in a document you can update this afternoon.

Try the cheap levers first

A better prompt, a few worked examples, a JSON schema, a retrieval step, a tool call and a smaller model with a clearer instruction are all reversible and measurable in hours. Each one also tells you something about the task. If a better prompt closes most of the gap, you have learned that the model could already do the task and needed to be asked properly. Each cheap lever is also a test of the task description; a vague task survives none of them.

Fine-tuning becomes justified when the task is narrow, the output distribution is stable, and you have hundreds of examples that show the behaviour you want. “Make it smarter” is not a task. “Turn these messy support notes into a five-field JSON object, every time” is. If you cannot describe the input and output well enough to label a hundred cases, you are not ready to train. A task that needs a paragraph of instructions for every input is not narrow yet.

A supervised fine-tuning job

from openai import OpenAI

client = OpenAI()
job = client.fine_tuning.jobs.create(
    training_file="file-abc123",
    model="gpt-4.1-mini-2025-04-14",
    hyperparameters={"n_epochs": 3},
)
print(job.id)

A decision checklist

Score the proposed project, then flip two answers and see how the recommendation changes. Add a criterion for a measurable target and give it real weight.

Output
Click "Run" to execute your code

The good reasons and the bad ones

The good reasons are consistent format at scale, classification with many subtle classes, distillation from a large model to a small one, domain vocabulary, and latency or cost reduction through shorter prompts. Each has a measurable target. The bad reasons are “more intelligence”, facts that change, a handful of examples, and a deadline that leaves no time to evaluate. A good reason has a target, a baseline and a number that can move. Without those three, the project cannot tell a win from a nicer demo.

A useful test is to write the evaluation before the dataset. If you cannot say how you will know a tuned model is better, you will end up comparing two outputs by eye and choosing the one that sounds more confident. That is not a decision procedure, and it is how a fine-tuning project ships a regression with a nice demo. A deadline is not a reason; it is a constraint on how much evidence you can collect.

Do not fine-tune before you can measure. A tuned model without an eval set is a vibe with a GPU bill, and the bill arrives before the evidence does.

The economics of a narrow model

Fine-tuning has a fixed cost: collecting and labelling data, training runs, evaluation, and the engineering around them. It has a variable benefit: a shorter prompt, a smaller model, fewer retries, or a format that no longer needs post-processing. The break-even volume is the fixed cost divided by the saving per request. The fixed cost is mostly people, and the variable benefit is mostly tokens. A retraining cadence adds to the fixed cost and belongs in the same calculation.

If you serve two hundred requests a month, prompt engineering is almost always cheaper. If you serve two million, a small model that replaces a large one can pay back in weeks. The calculation is not the whole decision, but it is the part that stops a team from spending a quarter on a model that saves eleven dollars. The break-even is a floor, not a forecast: it ignores the cost of being wrong.

When does the saving pay for the work?

Compute the break-even volume for the proposed model, then change the monthly traffic to 500 and to 50,000. Decide which case deserves a pilot.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

What is fine-tuning best at changing?