Vector Retrieval Metrics — Ground Truth First: A Concise Guide to Disambiguating Evaluation in Embedding-Based Retrieval

In vector retrieval and embedding systems, "Recall@k" is frequently ambiguous — meaning either baseline overlap or truth-based performance. This concise guide exposes the problem and offers a practical fix. One-sentence rule: If a metric name does not state the ground truth, it is operationally ambiguous and not decision-grade. Key elements: Clear ground truth types (GT-H, GT-W, GT-P, GT-L with bias warnings). Canonical names: SetOverlap@k[GT-*] (truth-consistency), BO@k[baseline=*] (baseline-consistency, diagnostic only). Legacy mapping and toy examples showing why baseline metrics can mislead. Visual Venn diagram illustrating divergence between baseline, ground truth, and system rankings. Machine-readable header schema for enforcement. This entry distills the core ideas from the accompanying whitepaper, serving as a quick reference for practitioners and benchmark maintainers. Adopt [GT-*] / [baseline=*] labels in dashboards, papers, and PR reviews to eliminate ambiguity. Related identifier: Primary reference – AQEA Technical Report (DOI: 10.5281/zenodo.18138436) and full whitepaper (DOI of the other record)