Fine-tuning and LoRA

Teach a model one behaviour without teaching it everything again

Advanced
10 chapters
168 min read

When fine-tuning helps and when retrieval is the better answer, datasets that do not leak, the masked loss behind supervised fine-tuning, LoRA and adapters, the hyperparameters that matter, catastrophic forgetting, preference tuning, serving adapters, and a capstone training plan with an evaluation gate.

What You'll Learn

Decide whether a task needs fine-tuning or a cheaper lever
Build a chat dataset with group splits and duplicate checks
Explain the masked next-token loss and a training step
Read loss curves and task metrics without confusing them
Attach a LoRA adapter and count what it trains
Choose learning rate, epochs and effective batch deliberately
Detect catastrophic forgetting with a general suite
Turn preference pairs into a DPO objective with a reference model
Serve, route and roll back adapters without cross-tenant leakage
Write a training plan, estimate its budget and apply a release gate

Prerequisites

Comfortable with Python functions, lists and dictionaries
Basic familiarity with a neural network training loop helps
No GPU or API key is needed — the examples model the arithmetic and the decisions

Vocabulary

The terms the chapters assume, in plain words.

adapter: A small set of trained numbers that changes how a fixed model behaves and can be added or removed without retraining it.
base model: The original model a tuning run starts from, left unchanged when only an adapter is trained.
catastrophic forgetting: When training on a narrow task makes a model worse at the general skills it already had.
checkpoint: A saved copy of a model's learned numbers at one point in training, kept so a run can be resumed or compared.
dataset: The collection of examples a model learns from, split into training, validation and test parts.
epoch: One full pass over the training examples; several passes let the model see the data more than once.
hyperparameter: A setting chosen before training, such as how fast the model learns, rather than one learned from the data.
leakage: When information from the test or future data reaches training, making the score look better than it is.
learning rate: How far the model's numbers move on each update; too high overshoots and too low barely moves.
LoRA: A method that leaves the model fixed and learns two small grids of numbers whose product is added to chosen parts.
loss: A number that measures how far the model's predictions are from the answers in the data, and that each update tries to lower.
preference tuning: Training a model to choose the better of two answers, rather than to copy one correct answer.
supervised fine-tuning: Training a model on example inputs and the answers you want it to produce for them.
validation set: Examples held back from training and used to choose settings and the stopping point.

Course Content

01
When to fine-tune (and when not)
Fine-tuning is a behaviour change, not a knowledge upload. It earns its cost when the task is narrow, the data is good and the result can be measured.
Advanced · 16 min read
Not started
02
Datasets and formatting
A fine-tuning dataset is a set of conversations with a consistent shape. Split by group, deduplicate, and decide what the assistant turn is allowed to contain.
Advanced · 17 min read
Not started
03
Supervised fine-tuning
The training loop is next-token prediction with a mask over the prompt. The mask, the batch and the schedule decide what the model actually learns.
Advanced · 17 min read
Not started
04
Loss and evaluation
Loss is the number the optimizer minimizes, not the number the product cares about. Evaluate the task, on held-out cases, with the counts behind every rate.
Advanced · 17 min read
Not started
05
LoRA and adapters
LoRA freezes the base model and learns a low-rank correction. The rank, the target modules and the scaling decide how much capacity the adapter has.
Advanced · 17 min read
Not started
06
Hyperparameters that matter
Learning rate, epochs and effective batch decide most of the result. Rank and alpha tune capacity; everything else is a smaller correction.
Advanced · 16 min read
Not started
07
Catastrophic forgetting
A narrow fine-tune can make a model better at one task and worse at everything else. Measure the general suite, mix replay data, and keep a rollback.
Advanced · 16 min read
Not started
08
Preference tuning
SFT teaches an answer; a preference pair teaches a choice between answers. DPO turns that choice into a loss with a reference model and one leash called beta.
Advanced · 17 min read
Not started
09
Serving adapters
An adapter is a deployment artefact: base digest, tokenizer, template and config. Merge it or swap it, route it from identity, and version every cache key.
Advanced · 17 min read
Not started
10
Capstone: a training plan and its evaluation
A plan you can defend: data, base model, adapter, hyperparameters, compute budget, evaluation gate and rollback. Then the gate decides whether it ships.
Advanced · 18 min read
Not started