EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

cold start
The problem of recommending for a new user or a new item that has no interactions yet.
collaborative filtering
Recommending items by finding users with similar taste, or items that similar users liked, without reading item content.
explicit feedback
A signal where the user states a preference, such as a rating or a like, rather than one inferred from behaviour.
implicit feedback
A signal inferred from behaviour, such as a click, watch or purchase, that says something happened but not how much it was liked.
matrix factorisation
A method that gives each user and item a short list of learned numbers, so their match is the sum of products.
nDCG
A ranking score that gives more credit to relevant items near the top, then divides by the score of the best possible order.
popularity baseline
A list ordered by how many interactions each item has, used as the simple result every model must beat.
precision at k
The share of the first k items that are relevant, where k is the number of slots shown.
ranking
The second stage that orders a short list of candidates for one user, usually with a slower model.
recall at k
The share of all relevant items that appear in the first k positions.
relevance
How likely one user is to engage with one item; a ranking needs it per user, not as a global score.
retrieval
The first stage that cuts a large catalogue down to a few hundred candidates cheaply.
slot
One position on the screen that can hold an item; the fixed count of them limits how long a list can be.
two-tower retrieval
A design with one network for users and one for items, so the item side is computed before a request.

Recommender systems from the ground up

Implicit feedback

Clicks and plays have no negative examples. Implicit recommenders learn from confidence in what happened, and from negatives they sample themselves.

Part 5 of 10

Intermediate16 min readBack to Courses

What implicit data records

A view, a click, a play, a purchase. There is no one-star rating and no explicit dislike. The absence of an interaction could mean disinterest, ignorance, or that the item was never shown. The matrix is binary or a count. Its zeros are ambiguous in a way that explicit ratings never are.

That ambiguity is the whole problem. An explicit model can treat a missing rating as unknown and skip it. An implicit model has to decide what an unobserved pair means. That decision shapes both the loss and the evaluation.

The signals are not interchangeable. A view is weak. A click is stronger. A purchase is stronger still, and a share is a statement to an audience. Collapse them into one binary and you lose information. Keep them as separate channels and you multiply the matrix. Most systems pick one main signal and use the others as features.

Confidence, not preference

The standard form keeps a binary preference — interacted or not — and adds a confidence that grows with the count. One play is weak evidence. Forty plays is strong. The loss weights each cell by that confidence, so a heavy user does not dominate by having more rows.

Confidence is not the same as preference. A user can play a song forty times because they love it, or because it sits on a playlist they forgot to pause. The model treats repetition as a sign of interest. That is usually right and occasionally very wrong.

The confidence matrix is where the modelling lives. A common choice is one plus alpha times the count, with alpha around forty for many datasets. That number says how much one extra play is worth next to the first. Tune it on held-out interactions rather than copying it from a paper.

ALS for implicit data

from implicit.als import AlternatingLeastSquares

model = AlternatingLeastSquares(factors=64, regularization=0.1, alpha=40)
model.fit(confidence_matrix)
ids, scores = model.recommend(0, confidence_matrix[0], N=10)
print(list(zip(ids, scores.round(3))))

Plays, days and confidence

Run the three rankings and find the item that wins on raw plays but loses on distinct days. Then decide which count you would use to build a confidence matrix.

Output
Click "Run" to execute your code

Where negatives come from

The zeros are unreliable, so implicit models sample negatives from the unobserved set. Some treat all unobserved items as weak negatives. BPR (Bayesian personalised ranking) trains on triples: a user, an item they chose, and an item they did not. A pairwise loss pushes the chosen one above the other.

Sampling is a modelling choice, not a detail. Sample at random over a huge catalogue and most negatives are trivially easy. Sample by popularity and they are hard and more useful, at the cost of bias. The sampler is part of the model, and it belongs in the version string.

There is a cheaper option. Treat every unobserved pair as a weak negative and let the confidence weight do the work. That is what ALS does, and it is why the method scales. The cost is that the model spends most of its effort on pairs that were never shown. That is another reason exposure logging matters.

Treating every unobserved item as a negative teaches the model what was never shown. If the previous ranker hid a category, the new model learns to keep hiding it.

Evaluating implicit models

Leave-one-out is common and optimistic. Hold out the last interaction, rank the whole catalogue, and report hit rate or nDCG. It flatters models on users with many interactions. It says little about the first impression, which is the case the product most wants to improve.

Better: a temporal split (train on the past, test on the future), a fixed number of candidates per user, and a report that separates warm users from cold ones. Also log what was shown. An interaction you never displayed cannot be held against the model. A metric that ignores exposure measures the old policy as much as the new one.

Hit rate at k is the simplest implicit metric: did the held-out item appear in the top k? It is easy to explain and easy to game by recommending the head. Pair it with nDCG or mean reciprocal rank. Always report the candidate set, because ranking ten candidates and ranking a million are different achievements.

A pairwise loss from scratch

Run the trainer and watch the loss fall. Then look at the top three for u1 and note whether an item the user has already seen is still ranked first.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

Why is a missing interaction not a negative example?