Skip to main content

Vector Indexes and Retrieval

An index stores vectors and returns candidates under a similarity rule. Exact search compares every eligible vector; approximate nearest-neighbor indexes trade recall for latency and memory. Metadata filtering, tenant isolation, deletion, versioning, and tie-breaking are part of retrieval—not cleanup after ranking.

Prerequisites​

Understand embeddings, cosine similarity, and labeled relevance judgments.

Learning goals​

  • distinguish exact ranking from approximate candidate search.
  • apply access filters before exposing candidates.
  • compute recall at k and reciprocal rank separately from latency.

Worked example​

The scores are fixed to isolate ranking behavior. d2 is relevant but inaccessible, so the filtered result cannot expose it.

scores = {"d1": 0.91, "d2": 0.89, "d3": 0.82, "d4": 0.40}
audience = {"d1": "employee", "d2": "finance", "d3": "employee", "d4": "employee"}
relevant = {"d2", "d3"}
exact = sorted(scores, key=lambda item: (-scores[item], item))[:2]
filtered = [item for item in exact if audience[item] == "employee"]
ranked = sorted(scores, key=lambda item: (-scores[item], item))
first_relevant_rank = next(index for index, item in enumerate(ranked, start=1) if item in relevant)
recall_at_2 = len(set(exact) & relevant) / len(relevant)

print(f"exact={exact} filtered={filtered}")
print(f"recall_at_2={recall_at_2:.3f} reciprocal_rank={1 / first_relevant_rank:.3f}")

Output:

exact=['d1', 'd2'] filtered=['d1']
recall_at_2=0.500 reciprocal_rank=0.500

In a real system, enforce the caller's permissions inside the retrieval query or before scoring, not only after returning IDs. Evaluate exact and approximate results on the same pinned index snapshot and query set; a latency win does not excuse an unknown recall loss.

Common mistakes​

  • Measuring answer quality without measuring whether relevant evidence was retrieved.
  • Filtering only after unauthorized text has reached a model or log.
  • Forgetting tombstones, re-embedding, and stale replicas during deletion.
  • Comparing index algorithms with different embeddings, hardware, filters, or recall targets.

Practice​

Choose a recall target and latency budget for 100,000 vectors. State which measurements must be repeated after changing the encoder or chunking policy.

Next steps​

Retrieval-Augmented Generation connects retrieval to context assembly and a model boundary.

Primary sources​

Checked 2026-10-16: The Faiss Library, Faiss similarity-search documentation, and scikit-learn nearest neighbors.