The ranking problem

A list, not a label

A recommender answers a question that looks simple. Given a user and a catalogue, what should be shown first? The output is an ordered list. It is not a category, and it is not a score on its own. A model may say an item is relevant with probability 0.8. That number is useful only when you compare it with every other item. Then you cut the list at the number of slots (the positions the screen can show).

The slots are the constraint. A home screen shows twenty tiles, a search page ten, an email ten. Ranking means spending that budget. Every position is a claim that this item deserves more attention than the ones below it. A model can be right about relevance (how likely the user is to engage with the item) and still be wrong about order. That model has failed the interface.

Ranking is also the second half of a two-stage system. Retrieval (the first stage) narrows millions of items to a few hundred candidates at low cost. Ranking then orders those candidates with an expensive model. Teams often confuse the two. A ranking metric measured on the whole catalogue flatters a model that would never see most of that catalogue in production.

Three ingredients

Every recommender is built from three things: users, items and interactions. An interaction can be explicit, such as a rating or a like. It can also be implicit, such as a click, a watch or a purchase. Explicit data says how much someone liked something. Implicit data says only that something happened. That is weaker evidence, and there is far more of it.

Side information — genre, text, price, time of day — is optional for collaborative methods. It is essential for content-based ones. The choice of which of these three to lean on is the first design decision. That choice decides what the system can do when a user or an item is new. A catalogue with no features and no interactions has nothing to rank.

Context is the ingredient teams forget. The same user wants different things on a Monday morning and a Friday night. That user also wants different things on a phone and on a television. A session, a device or an hour of the day can be a feature, a filter or a separate model. Treat context as part of the interaction, not as noise around it. That is often the cheapest personalisation you can buy.

A popularity baseline in pandas

import pandas as pd

events = pd.read_csv("events.csv")
popular = (
    events.groupby("item_id")["user_id"]
    .nunique()
    .sort_values(ascending=False)
)
print(popular.head(10))

Popularity is the baseline

The strongest simple recommender is popularity. Count the interactions per item and show the items with the most. This baseline is cheap and stable. It needs no per-user model, and it is hard to beat on a cold catalogue. Most personalisation projects would do better with a good popularity list than with a fragile model nobody can debug.

That makes popularity the honest baseline. A personal model must beat it on the same split and the same metric. If it cannot, the model has not earned its complexity. Report both numbers, not only the model’s. Keep the baseline in the dashboard after launch, so a regression is easy to see.

Popularity is not one number. You can count unique users, total events, events in the last week, or events weighted by recency. Those four counts give four different lists. The choice should follow the promise on the page. A “trending” shelf means recent and rising. A “most popular” shelf means broad and stable. Compute the version you can defend.

The objective is not the metric

Most teams start by predicting ratings and measuring root mean squared error (RMSE, the average size of the rating error). That is a regression problem, and it is not the problem the product has. A ranking can be excellent while its rating predictions are mediocre. A model can also lower RMSE by being timid, without improving a single list.

What matters is the order at the cutoff (the point where the list stops). The question is whether the item the user wanted is in the top ten. The next chapters make that precise with precision at k, mean average precision and nDCG. For now, keep the distinction clear. Rating prediction estimates a number. Ranking decides an order. Only the order is what the page renders.

Offline metrics are proxies (stand-ins for the real outcome). A proxy can be wrong in ways the model cannot see. A ranker can win on nDCG and lose on the page. It may recommend items that are relevant but already owned. It may put a disturbing item at the top. Keep a human review of a sample, and treat the online experiment as the final judge.