EasyCompilers

Vocabulary for this course

The terms the chapters assume, in plain words. You can open this from any chapter.

cold start
The problem of recommending for a new user or a new item that has no interactions yet.
collaborative filtering
Recommending items by finding users with similar taste, or items that similar users liked, without reading item content.
explicit feedback
A signal where the user states a preference, such as a rating or a like, rather than one inferred from behaviour.
implicit feedback
A signal inferred from behaviour, such as a click, watch or purchase, that says something happened but not how much it was liked.
matrix factorisation
A method that gives each user and item a short list of learned numbers, so their match is the sum of products.
nDCG
A ranking score that gives more credit to relevant items near the top, then divides by the score of the best possible order.
popularity baseline
A list ordered by how many interactions each item has, used as the simple result every model must beat.
precision at k
The share of the first k items that are relevant, where k is the number of slots shown.
ranking
The second stage that orders a short list of candidates for one user, usually with a slower model.
recall at k
The share of all relevant items that appear in the first k positions.
relevance
How likely one user is to engage with one item; a ranking needs it per user, not as a global score.
retrieval
The first stage that cuts a large catalogue down to a few hundred candidates cheaply.
slot
One position on the screen that can hold an item; the fixed count of them limits how long a list can be.
two-tower retrieval
A design with one network for users and one for items, so the item side is computed before a request.

Recommender systems from the ground up

Cold start and feedback loops

New users, new items and a new product each break a different assumption. Exploration is how a recommender learns what it was never shown.

Part 7 of 10

Intermediate17 min readBack to Courses

Three cold starts

A new user has no history. A new item has no interactions. A new product has neither. Onboarding questions, content features and a popularity prior solve the first. Item features and a deliberate exploration budget solve the second. Editorial curation and a fast feedback loop solve the third.

Treating them as one problem leads to one bad solution. A content model helps a new item and does nothing for a new user with no stated preferences. An onboarding flow helps the user and does nothing for the item. Name the case before you choose the tool.

The three cases also need different success metrics. A new user needs a list that is safe and easy to react to. A new item needs enough impressions to estimate its quality. A new product needs a reason for anyone to return. Measuring all three with the same click-through rate hides which one is failing.

Explore to learn

A greedy ranker always shows its current best guess. It never collects the data that would correct the guess. Epsilon-greedy shows a random item a small fraction of the time. Thompson sampling draws from a posterior (a belief about each item’s quality) and explores in proportion to how unsure it is. UCB adds an explicit bonus for items that are rarely shown.

Exploration costs today and pays later. Treat it as a budget with an owner, not as a bug in the ranker. A team that cannot say how much traffic is set aside for learning will usually find that the answer is zero.

The exploration policy should match the cost of being wrong. A random item in a feed is cheap. A random item in a checkout recommendation is not. Where a mistake is costly, explore within the ranking of safe candidates rather than across the whole catalogue. Keep the random slice large enough to measure.

Thompson sampling with SciPy

from scipy.stats import beta

clicks = [12, 7, 3]
views = [40, 35, 30]
samples = beta.rvs(a=[c + 1 for c in clicks], b=[v - c + 1 for v, c in zip(views, clicks)])
best = max(range(len(samples)), key=lambda index: samples[index])
print("show item", best)

Epsilon-greedy in a small world

Run the simulation, then set EPSILON to zero and run again. Compare how often the best item is shown and how quickly its estimated rate settles.

Output
Click "Run" to execute your code

The loop closes

The model’s output becomes the next training data. Popular items get more impressions and more clicks, and so more evidence, which makes them more popular. The loop is real even when every single step is correct. It does not show up in a dashboard that only reports click-through rate.

The defences are built in. Log impressions and propensities (how likely each item was to be shown). Hold out a random slice of traffic. Keep a content or popularity fallback in the mix. Measure variety and coverage alongside clicks. None of these needs a better model. They need a decision about what the system is allowed to learn from.

The loop has a timescale. One round of re-training moves the catalogue a little. A year of it can remove whole categories from the training data. Watch how much the impressions bunch up over months, not only the click rate over days. The slow version is the one that is hard to reverse.

A model trained only on its own impressions cannot learn about what it never showed. The catalogue slowly shrinks to the part the model already believes in.

Logging for the counterfactual

To evaluate a policy you did not run, you need to know how likely each shown item was under the policy that did run. That probability is the propensity. Without it, offline evaluation of a new ranker compares two different groups of users and calls it a comparison of two models.

Propensity logging is cheap at serving time. Rebuilding it later is nearly impossible. Log the item, the position, the policy version and the chance of selection. The next evaluation will thank you. The one after that will depend on it.

Inverse propensity scoring is the standard use. Weight each logged interaction by one over its propensity, so rare shows count more. The estimate is unbiased in theory and noisy in practice, especially when some propensities are tiny. Clip the weights and report the effective sample size. Otherwise a handful of rare items will dominate the estimate.

The rich get richer

Run both policies and compare the share of impressions that go to the top three items. Then add a fourth slot and see whether the greedy policy diversifies at all.

Output
Click "Run" to execute your code

Check what you learned

One question at a time. A wrong answer explains itself — take another go.

Part 1 of 5

Which cold-start case is helped most by item features?