EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

adapter
A small set of trained numbers that changes how a fixed model behaves and can be added or removed without retraining it.
base model
The original model a tuning run starts from, left unchanged when only an adapter is trained.
catastrophic forgetting
When training on a narrow task makes a model worse at the general skills it already had.
checkpoint
A saved copy of a model's learned numbers at one point in training, kept so a run can be resumed or compared.
dataset
The collection of examples a model learns from, split into training, validation and test parts.
epoch
One full pass over the training examples; several passes let the model see the data more than once.
hyperparameter
A setting chosen before training, such as how fast the model learns, rather than one learned from the data.
leakage
When information from the test or future data reaches training, making the score look better than it is.
learning rate
How far the model's numbers move on each update; too high overshoots and too low barely moves.
LoRA
A method that leaves the model fixed and learns two small grids of numbers whose product is added to chosen parts.
loss
A number that measures how far the model's predictions are from the answers in the data, and that each update tries to lower.
preference tuning
Training a model to choose the better of two answers, rather than to copy one correct answer.
supervised fine-tuning
Training a model on example inputs and the answers you want it to produce for them.
validation set
Examples held back from training and used to choose settings and the stopping point.

Fine-tuning and LoRA

Datasets and formatting

A fine-tuning dataset is a set of conversations with a consistent shape. Split by group, deduplicate, and decide what the assistant turn is allowed to contain.

Part 2 of 10

Advanced17 min readBack to Courses

An example is a conversation

A supervised example is a sequence of messages: a system prompt, a user turn, and the assistant reply you want the model to produce. The assistant turn is the only part the loss should train on, which is why the format matters before the first step. If the system prompt in the dataset differs from the one in production, the model learns a distribution it will never see again. The assistant turn is the specification; the rest is the condition under which it applies. If the system prompt is empty in the data and long in production, the model learns the wrong opening.

Include the context the production prompt includes. If retrieval injects three documents, use a realistic placeholder or a real retrieved set, not an empty string. Do not include a chain of thought you would not want the model to emit at inference time. The dataset is a specification of behaviour, and every field in it is part of the specification. A placeholder that is shorter than production teaches the model to ignore context it will actually see.

Split by group, not by row

A random row split leaks near-duplicates across train and test. The same customer email, the same document, or the same template with one word changed can appear on both sides, and the evaluation then measures memorisation. Split by customer, document, conversation or template, whichever is the unit a production request belongs to. The split is a property of the dataset, not a line in the training script. A group that appears in both sets is a leak, even if every row is unique.

Keep three sets with three jobs. Training fits the weights, validation chooses the epoch and the hyperparameters, and test is touched once for the final claim. If you tune on test, it becomes validation and you no longer have an unbiased estimate. Stratify the split by the slices you care about, or a rare intent can vanish from the test set. A test set that has been looked at twice is already becoming validation.

Render the chat template once

from datasets import load_dataset

ds = load_dataset("json", data_files="train.jsonl", split="train")
ds = ds.map(
    lambda row: {"text": tokenizer.apply_chat_template(row["messages"], tokenize=False)},
    remove_columns=ds.column_names,
)
print(ds[0]["text"])

Validate before you train

Run the validator over four records. Then add a record whose assistant turn is empty and confirm it is rejected before it reaches the trainer.

Output
Click "Run" to execute your code

Quality beats volume

Five hundred clean examples beat five thousand noisy ones. A wrong assistant answer teaches the model to be wrong in that situation, and no amount of later data fully removes it. Read a random sample of fifty before training; every dataset has a pattern the team did not expect, and finding it now is cheaper than finding it in production. A random sample of fifty is enough to find the systematic mistake, not every mistake. Volume is easy to buy; consistency is not.

Cover the edges on purpose: short and long inputs, refusals, ambiguous cases, the classes that are easy to confuse. Balance the labels enough that the model does not learn to predict the majority class. Remove boilerplate that is identical in every example, because it teaches nothing and consumes context. Label noise is the ceiling on what the model can learn. The edges are where the model earns its keep; the easy middle is already handled by the prompt.

Read fifty random examples before training. If you cannot explain why each assistant turn is the right answer, the dataset is not ready, and a bigger model will only reproduce the confusion.

Format is part of the contract

Use one JSON object per line, UTF-8, no trailing commas, and the same chat template at training and inference. A template mismatch is a silent quality loss: the model sees different special tokens in production and behaves as if the conversation started mid-sentence. Render the template with the tokenizer’s own function rather than rebuilding it by hand. A JSONL file that parses is not the same as a conversation that makes sense. The template is code; review it with the dataset, not after the first bad answer.

Watch the token length. Truncation that cuts the assistant answer teaches incomplete answers; truncation from the oldest turn preserves the reply. Packing several short examples into one sequence saves padding but needs an end-of-sequence boundary between them. Record how many examples were truncated, because a high rate means the model is learning from fragments. The truncation rate belongs next to the loss, because it changes what the loss means. Record the template version with the dataset; a template change is a data change.

Find near-duplicates before splitting

Compare the three pairs by word shingles. Then add a pair that differs by one word and see whether it crosses the threshold.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

Which part of a chat example should the loss train on?