EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

adapter
A small set of trained numbers that changes how a fixed model behaves and can be added or removed without retraining it.
base model
The original model a tuning run starts from, left unchanged when only an adapter is trained.
catastrophic forgetting
When training on a narrow task makes a model worse at the general skills it already had.
checkpoint
A saved copy of a model's learned numbers at one point in training, kept so a run can be resumed or compared.
dataset
The collection of examples a model learns from, split into training, validation and test parts.
epoch
One full pass over the training examples; several passes let the model see the data more than once.
hyperparameter
A setting chosen before training, such as how fast the model learns, rather than one learned from the data.
leakage
When information from the test or future data reaches training, making the score look better than it is.
learning rate
How far the model's numbers move on each update; too high overshoots and too low barely moves.
LoRA
A method that leaves the model fixed and learns two small grids of numbers whose product is added to chosen parts.
loss
A number that measures how far the model's predictions are from the answers in the data, and that each update tries to lower.
preference tuning
Training a model to choose the better of two answers, rather than to copy one correct answer.
supervised fine-tuning
Training a model on example inputs and the answers you want it to produce for them.
validation set
Examples held back from training and used to choose settings and the stopping point.

Fine-tuning and LoRA

Capstone: a training plan and its evaluation

A plan you can defend: data, base model, adapter, hyperparameters, compute budget, evaluation gate and rollback. Then the gate decides whether it ships.

Part 10 of 10

Advanced18 min readBack to Courses

A plan is a set of decisions

A training plan names the goal, the base model, the dataset and its split, the method, the hyperparameters, the compute budget, the evaluation, the release gate and the rollback. Writing it before training forces the vague parts into the open. If the goal cannot be measured or the rollback is “retrain”, the plan is not ready. The plan is also the document you show to the person who approves the GPU spend. It should name the metric that will decide success and the person who owns it. A plan with no rollback is a bet, not a plan.

Keep the plan next to the run, not in a slide. The config, the dataset version and the eval results should be one object that can be reproduced. A plan that lives only in a chat thread is a plan that will be remembered differently by everyone who approved it. A plan that cannot be estimated is usually a plan that has not been thought through. The arithmetic is simple; the discipline is writing it down before the first run.

Estimate before you spend

Tokens seen is examples times average tokens times epochs. Steps is examples divided by the effective batch, times epochs. GPU-hours is steps times step time divided by 3600, plus evaluation and a retry factor. Multiply by the hourly rate and you have a number that can be compared with the value of the improvement. The estimate should be recomputed when the dataset or the base model changes. A cheap run that answers the wrong question is still expensive. A budget that ignores the people who label the data is an underestimate by construction.

The estimate is approximate, and that is fine. It stops the obvious mistakes: a plan that trains for a week to save a tenth of a cent per request, or an evaluation that costs more than the training. Add the cost of the evaluation runs and the retries; the GPU bill is rarely the whole bill. The evaluation is part of the budget, not an afterthought. Cache the base outputs so the comparison does not pay for them twice.

The whole plan in one trainer

from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-1.5B-Instruct",
    train_dataset=train,
    eval_dataset=val,
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"]),
    args=SFTConfig(
        output_dir="out",
        num_train_epochs=2,
        learning_rate=2e-4,
        per_device_train_batch_size=4,
        gradient_accumulation_steps=8,
        max_length=1024,
    ),
)
trainer.train()
trainer.save_model("adapter")

Estimate the run

Print tokens, steps, GPU-hours and cost for the plan. Then double the epochs and add a second evaluation run; decide whether the budget still makes sense.

Output
Click "Run" to execute your code

The evaluation plan

Evaluate the task suite, the general suite and the slices that matter. Run the base model first and cache its outputs, so both models are compared on the same cases with the same prompt and decoding settings. A paired comparison is stronger than two pass rates, and a manual review of twenty samples catches the failures a metric misses. A gate with no owner is a suggestion. The gate should be able to say no, or it is not a gate. The gate should name the suites it covers and the ones it does not.

Write the release threshold before seeing the result. “Task gain at least five points, no general suite down more than three” is a decision rule; “looks better” is not. Name the owner of the gate and the date of the next review, so the decision has a home after the launch. A threshold chosen after the result is a description, not a gate. A sample of failures read by a person is still the best detector of a metric that has drifted.

Write the rollback before the launch. The base model id plus the previous adapter version is a rollback plan; “retrain” is an incident with extra steps.

Ship, iterate, monitor

Start with a small canary and watch the error rate, the cost per success and the feedback. Keep the adapter version in every trace, so a change in behaviour can be joined to a change in weights. The observability course is the other half of this loop: without a trace, the release gate is the last measurement you will have. A canary is a small, reversible exposure to real traffic. The first hour of canary traffic is worth more than a week of offline evaluation.

Retrain on a cadence or when the distribution drifts, not on every prompt edit. Each retrain repeats the same plan: data, split, budget, eval and gate. The model changes; the procedure is what makes the change safe. A cadence without a drift check is a calendar, not a policy. The procedure is the asset; the adapter is disposable.

The gate decides

Apply the release rule to the four suites. Then tighten the general threshold to one point and see whether the same model still ships.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

What belongs in a training plan before the run?