Keeping the index fresh

An index is a snapshot

The index is a copy of the corpus at the moment it was built. Nothing about a vector store watches the filesystem, so a document edited after the last build is invisible until something re-reads it. The failure is quiet: the answer is fluent, cited and about last month’s policy, and nothing in the response says it is out of date.

Freshness is therefore a budget rather than a wish. Decide how stale an answer may be — minutes for a price list, a day for a handbook — and make the sync schedule meet that number. “We re-index when someone remembers” is not a freshness policy, it is an outage waiting for a question, and the alert should fire on index age rather than on a failed job.

The corpus is not the only thing that changes. A new embedding model, a new chunker or a new metadata field also makes the index old, and those changes are invisible to a document hash. Track the pipeline version beside the content version; without it, a deploy changes how new chunks are built while the old ones stay in the previous format.

Sync by hash, not by clock

A modification time is a weak signal. Copying a file preserves it, restoring a backup changes it without changing the content, and a deployment can touch every file at once. A hash of the text answers the only question that matters: is this document different from the one in the index? It also survives a rename, which a path-based comparison does not.

Compare the hashes and you get four sets: added, changed, deleted and unchanged. The first three need work and the fourth must be skipped, or every sync re-embeds the whole corpus and pays for it again. Deletes are the half everyone forgets, and a deleted document that still answers questions is a compliance problem, not a stale cache. Keep the manifest beside the index, so the next run diffs against what is actually stored.

An incremental upsert

from chromadb import PersistentClient

client = PersistentClient(path="./index")
collection = client.get_or_create_collection("docs")

stored = collection.get(include=["metadatas"])
stale = [
    chunk_id
    for chunk_id, meta in zip(stored["ids"], stored["metadatas"])
    if meta["doc_hash"] != hashes.get(meta["doc_id"])
]
collection.delete(ids=stale)

collection.upsert(ids=chunk_ids, documents=texts, metadatas=metadatas, embeddings=vectors)

Stable ids make an update idempotent

A chunk id should be derived from the document id, the document hash and the position of the chunk inside it. Then re-running a sync replaces the old chunks instead of adding a second copy, and a failed run can simply be run again. Ids from a random generator turn every retry into duplication, and duplication looks like a ranking problem.

The embedding model belongs in the id scheme too, or at least in the collection name. Vectors from two models live in different spaces: a query embedded by the new model and a passage embedded by the old one are not comparable, and the similarity between them is a number without a meaning. Version the collection, not the individual vector.

Build beside, then flip

A rebuild that writes into the live collection is visible while it is half finished: some documents are new, some are missing, and every query in between sees a corpus that never existed. Build the new version in its own collection and point an alias at it only when the build has completed and its count matches the source.

The flip is also the rollback. If the new index is worse, point the alias back at the old collection and the service is where it was a minute ago. Monitor what the alias points at, when it was last synced and how many documents it holds; a stale index should be an alert, not a discovery made by a customer.

Schedule the rebuild before the switch is needed. Re-embedding a large corpus takes hours, and a model migration started on the day the old model is retired is an outage with a project plan. A second collection costs storage; a failed migration costs trust.