Similarity metrics

Three ways to measure closeness

Cosine compares direction and ignores length: it is the angle between two vectors. Dot product multiplies and sums, so it rewards both alignment and magnitude (how long the vector is). Euclidean distance measures the straight line between the points, so it cares about absolute position as well as direction. Three sensible definitions, three different rankings. Only one of them matches the way your embedding model was trained.

On unit-length vectors (vectors of length one), cosine and dot product rank the same. Euclidean distance ranks in reverse order of dot product. That match is why the choice often seems not to matter, and why it quietly does once the vectors are not normalised. The test is simple: pick two documents with the same direction and very different lengths. Then see whether your ranking puts them in the order you expect.

The practical result is that “similar” is not one idea. Two documents can be similar because they point the same way, because one is longer and louder, or because they sit near each other in absolute space. The metric is the sentence that says which of those you meant. The embedding model is the sentence that says which of them you can even measure.

What normalisation changes

Dividing a vector by its length removes magnitude. After that the dot product is the cosine, and a long document no longer outranks a short one because its embedding is larger. Some models already return unit vectors; others do not, and the difference is invisible until you measure it. A model card that says “use cosine” is telling you the training loss, not promising that the output is normalised.

Normalise once, at write time, and store the result. Normalising on every query is a small cost that hides a larger question: are the stored vectors normalised too? A collection that is half normalised is worse than one that is not normalised at all. The scores are similar enough to look meaningful and different enough to mislead. A migration that changes normalisation is a re-embedding, not a config change.

Cosine, in one line of numpy

import numpy as np

def cosine(a, b):
    a = a / np.linalg.norm(a)
    b = b / np.linalg.norm(b)
    return float(a @ b)

# embeddings must already be unit-length for this to be a cosine
scores = embeddings @ query
best = np.argsort(-scores)[:5]

Choose the metric the model was trained with

Embedding models are trained with a similarity function in their loss. A model trained with cosine is not guaranteed to behave under raw dot product, and a model trained for dot product may use magnitude as a signal. The model card usually says which one. When it does not, the safe default is cosine on normalised vectors. “Safe” here means the failure is a lower score, not an inverted ranking.

Mixing metrics between write and query is the classic failure. If you index with cosine and query with dot, the top result is often right and the fifth is often wrong. That is the worst kind of bug: it passes the demo and fails the audit. The same thing happens when a library changes its default, or when one service normalises and another does not. Write the metric down where the data lives.

Distance, similarity and thresholds

Cosine and dot are similarities: bigger is better. Euclidean is a distance: smaller is better. A threshold means different things in each, and a cutoff copied from one to the other will either admit everything or nothing. Pick the direction once, encode it in the function name, and make the caller unable to pass the wrong one by accident.

Thresholds are also model-specific. A cosine of 0.8 is a strong match for one model and a normal one for another. So calibrate the cutoff on your own labelled pairs rather than borrowing a number from a blog post. Plot the score distribution for known matches and known non-matches; the gap between them is the only threshold that means anything.

A good test set makes this concrete: pairs a human marked as duplicates, pairs that share a topic but are not duplicates, and pairs that are unrelated. Score all three groups and look at the overlap. If duplicates and topic-neighbours share a range, no threshold will separate them. Then the metric or the model needs to change rather than the cutoff.