PANORAMA: Fast-Track Nearest Neighbors

Retrieval-augmented generation (RAG) has put nearest-neighbor search over learned embeddings on the inference path of language models via purpose-built vector indexes. An index narrows a query result to a set of candidates, each of which must be verified by its $d$-dimensional distance. As embeddings grow wider, this verification comes to dominate query time over index traversal. Most of this verification overhead is wasted: a distance is computed only to decide whether a candidate enters the top-$k$ set; for most candidates the answer is “no”. We introduce Panorama, a verification module that verifies candidates progressively. It rotates vectors into an orthogonal basis fitted to the data, accumulates distances in a piecewise manner, and abandons a candidate as soon as a Cauchy–Schwarz lower bound on its unread remainder pushes its distance past the running top-$k$ threshold. As energy concentrates in a few principal directions, the bound becomes decisive after a small fraction of the coordinates is read. While Panorama’s strict mode returns exactly the top-$k$ of the supplied candidates, one parameter trades recall for throughput. A cost model ties the pruning effect to the spectral decay of the fitted basis. Panorama sits on top of an unmodified index: we realize the bound within inverted-list, product-quantized, graph, reranking, and RaBitQ indexes with layouts that keep the pruning loop vectorized. Across image descriptors and text embeddings, the gain grows with dimension and is largest on the wide embeddings that RAG systems search.