EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

cold start
The problem of recommending for a new user or a new item that has no interactions yet.
collaborative filtering
Recommending items by finding users with similar taste, or items that similar users liked, without reading item content.
explicit feedback
A signal where the user states a preference, such as a rating or a like, rather than one inferred from behaviour.
implicit feedback
A signal inferred from behaviour, such as a click, watch or purchase, that says something happened but not how much it was liked.
matrix factorisation
A method that gives each user and item a short list of learned numbers, so their match is the sum of products.
nDCG
A ranking score that gives more credit to relevant items near the top, then divides by the score of the best possible order.
popularity baseline
A list ordered by how many interactions each item has, used as the simple result every model must beat.
precision at k
The share of the first k items that are relevant, where k is the number of slots shown.
ranking
The second stage that orders a short list of candidates for one user, usually with a slower model.
recall at k
The share of all relevant items that appear in the first k positions.
relevance
How likely one user is to engage with one item; a ranking needs it per user, not as a global score.
retrieval
The first stage that cuts a large catalogue down to a few hundred candidates cheaply.
slot
One position on the screen that can hold an item; the fixed count of them limits how long a list can be.
two-tower retrieval
A design with one network for users and one for items, so the item side is computed before a request.

Recommender systems from the ground up

Matrix factorisation

Matrix factorisation gives every user and item a short vector of latent factors and fits them to the ratings that exist. The empty cells become a dot product.

Part 4 of 10

Intermediate17 min readBack to Courses

A low-rank assumption

The rating matrix is large and mostly empty, but it is not random. Taste has shape. A handful of latent dimensions (hidden axes such as mood, genre, era or intensity) explain much of it. Matrix factorisation assumes the matrix is close to low rank. It learns a short vector for each user and each item, and the dot product of the two predicts the cell.

The number of factors is the capacity dial. Too few, and the model cannot tell distinct tastes apart. Too many, and it memorises single ratings. Values between eight and a few hundred are common. Find the right number on a validation split (a held-out slice used to choose settings), not by feel.

The geometry is simple. Each factor is an axis, and the dot product measures how much a user and an item point the same way. A factor might end up aligned with “quiet versus loud” or with “old versus new”. The model is not asked to name it. Only the inner product matters for the prediction.

Fit only what you observe

The loss sums over observed ratings only. Treating a missing rating as zero is the classic error. It tells the model the user hated everything they never rated. That is false, and it is fatal for the predictions on exactly those cells. The objective is squared error on the known entries, plus a penalty on the factors.

Regularisation is not optional. With millions of parameters and a sparse matrix, an unregularised model can drive training error to zero while it predicts noise everywhere else. The regulariser is what makes the low-rank assumption do any work.

So the loss is a sum of squared errors over the observed cells, plus a penalty on the size of every factor. Squared error is handy rather than sacred. It punishes large mistakes hard, which suits ratings and suits counts less well. The regulariser is the only thing between a flexible model and a lookup table.

SVD with biases in Surprise

from surprise import Dataset, SVD

data = Dataset.load_builtin("ml-100k")
trainset = data.build_full_trainset()
algo = SVD(n_factors=32, n_epochs=20, lr_all=0.005, reg_all=0.02)
algo.fit(trainset)
print(algo.predict("196", "302").est)

Factors by gradient descent

Run the training and watch the RMSE fall, then predict the missing rating for u1 on d. Change the number of factors to one and see how much worse the fit gets.

Output
Click "Run" to execute your code

Biases first

A useful factorisation starts with a baseline: the global mean, plus a user bias, plus an item bias. A bias is a fixed offset for one user or one item. Some users rate everything highly. Some items are liked by everyone. Remove those effects before you fit the interaction factors. The factors then explain taste rather than generosity.

This is also the cheapest model worth shipping. If the bias-only model comes close to the full factorisation on your data, the extra factors buy very little. The simpler model is also easier to explain and to keep stable.

Bias terms also expose data problems. If one item has a huge negative bias, it may be a broken listing rather than a disliked product. If one user has an extreme positive bias, check whether the account is a test fixture. The baseline is a diagnostic as much as a model.

Never fill missing ratings with zero. A zero is a strong negative opinion, and the model will believe it for every item the user has not seen.

SGD and ALS

Stochastic gradient descent (SGD) walks the observed ratings one at a time. It computes the error and nudges the two factors toward it. It is simple, works online, and is easy to regularise. The learning rate and the number of passes matter. A rate that is fine at the start often needs to decay.

Alternating least squares (ALS) fixes one side and solves the other in closed form (an exact formula, not a step). It alternates until the answer stops moving. It parallelises well and is the standard choice for implicit data. There the whole matrix takes part with confidence weights, not only the observed cells.

Convergence is not guaranteed to be fast, and the epoch count (how many passes over the data) is a real setting. Watch the validation error, not the training error. The point where validation turns upward is the point where the factors started memorising. Early stopping is cheaper than guessing the right count in advance. A learning rate that is too high oscillates. One that is too low crawls. Both look like a plateau if you only print every hundred epochs.

The bias-only model

Run the baseline, then compare its RMSE with the factor model from the first example. If the gap is small, the factors are not earning their complexity.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

What does matrix factorisation assume about the rating matrix?