EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

adapter
A small set of trained numbers that changes how a fixed model behaves and can be added or removed without retraining it.
base model
The original model a tuning run starts from, left unchanged when only an adapter is trained.
catastrophic forgetting
When training on a narrow task makes a model worse at the general skills it already had.
checkpoint
A saved copy of a model's learned numbers at one point in training, kept so a run can be resumed or compared.
dataset
The collection of examples a model learns from, split into training, validation and test parts.
epoch
One full pass over the training examples; several passes let the model see the data more than once.
hyperparameter
A setting chosen before training, such as how fast the model learns, rather than one learned from the data.
leakage
When information from the test or future data reaches training, making the score look better than it is.
learning rate
How far the model's numbers move on each update; too high overshoots and too low barely moves.
LoRA
A method that leaves the model fixed and learns two small grids of numbers whose product is added to chosen parts.
loss
A number that measures how far the model's predictions are from the answers in the data, and that each update tries to lower.
preference tuning
Training a model to choose the better of two answers, rather than to copy one correct answer.
supervised fine-tuning
Training a model on example inputs and the answers you want it to produce for them.
validation set
Examples held back from training and used to choose settings and the stopping point.

Fine-tuning and LoRA

LoRA and adapters

LoRA freezes the base model and learns a low-rank correction. The rank, the target modules and the scaling decide how much capacity the adapter has.

Part 5 of 10

Advanced17 min readBack to Courses

Freeze the base, learn a correction

Full fine-tuning updates every weight in the model. LoRA leaves the base weights untouched and adds two small matrices to a projection: A maps the input down to a rank r, and B maps it back up. The adapted output is the original projection plus B times A, scaled by alpha over r. Only A and B receive gradients. The frozen base is also a fixed reference: two adapters can be compared without retraining the base. The correction is low-rank because most behaviour changes do not need a full matrix.

The base model stays exactly as it was, which is why one copy can serve many adapters and why a small dataset is less likely to overwrite general behaviour. The adapter is a correction, not a replacement. That framing also tells you what to version: the adapter alone is meaningless without the base model and tokenizer it was trained against. The saving grows with the size of the matrix, which is why LoRA is most attractive on large projections. At rank 64 the share is still small, which is why the method is practical on a single GPU.

The parameter math

A full weight matrix of shape d by k has d times k parameters. A rank-r LoRA pair has r times d plus r times k, which is r times the sum of the dimensions. At 4096 by 4096 with rank 16, that is about 131,000 parameters against 16.7 million, under one percent of the original. A rank that is too low underfits quietly; the loss looks fine and the behaviour never arrives. A rank change without an alpha change is a different experiment.

Rank is capacity. Too low and the adapter cannot represent the change; too high and it starts to memorize and loses the efficiency that made it attractive. Alpha scales the update, and the alpha-to-rank ratio is the knob most recipes keep near one or two. Dropout on the LoRA path is a small regulariser, not a substitute for clean data. The adapter can be trained on one GPU and served on another, as long as the base matches. A larger base with a small adapter can beat a smaller base with a large one.

Attach an adapter with PEFT

from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, config)
model.print_trainable_parameters()

Count the parameters

Print the full and LoRA parameter counts for two layer shapes and three ranks. Then add a rank of 128 and describe when it stops being a bargain.

Output
Click "Run" to execute your code

Which modules to adapt

The attention projections are the usual start: query and value first, then key and output if the task needs more capacity. Adapting the MLP projections adds parameters and can help a task that needs new associations rather than a new style. More modules is not automatically better; it is more capacity to overfit with. Query and value are a default, not a law; the right targets depend on what the task needs to change.

Start with query and value at rank 8 or 16, train for one or two epochs, and look at the validation curve before adding modules. If the adapter underfits, raise the rank or add targets; if it overfits, lower the rank, add dropout, or get more data. Change one of those at a time, or the result tells you nothing about which one mattered. The target list is part of the experiment record, not a detail.

An adapter is not a model. Version it with the base model id, the tokenizer, the chat template and the training config, or you will not be able to reproduce the behaviour later.

Adapters in production

You can merge the adapter into the base weights for inference, which removes the extra matrix multiplication and gives a normal model file. Or keep it separate and swap adapters per tenant or per task. Merging is simpler to serve; swapping is cheaper when many customers need different behaviours from one base. Merging is a deployment decision, and it should be evaluated like one.

QLoRA quantizes the frozen base to four bits and trains the adapter on top. It makes a larger base fit on a smaller GPU, at the cost of some numerical noise and a more careful merge. Evaluate the model you will actually serve: an adapter that scores well before merging can behave differently after it, and the difference is easy to miss. A separate adapter also lets you roll back one tenant without touching the others.

A LoRA update that starts at zero

Run the forward pass with B at zero, then change B and see the output move. Confirm the base weights never change.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

What does LoRA freeze?