SimSparse: Semantic Search Can Stay Sparse End to End

Scaling semantic retrieval requires balancing corpus storage against the computation needed to access it. Conventional graph indexes retain dense document embeddings for low-latency search, while low-storage approaches often shift part of this cost to query-time document re-encoding. Learned sparse representations offer a different design point: their compact codes support direct scoring without document re-encoding, but compact scoring alone does not determine which documents should be evaluated. We introduce SimSPARSE, a sparse-native graph retrieval system that uses sparse features to organize and navigate the corpus. Shared active features generate candidate connections, active query features select entry points, and sparse inner products guide graph construction, traversal, and ranking. The deployed index retains only sparse codes, bounded-degree graph links, and short routing lists, without dense corpus embeddings or online document re-encoding. Across six benchmarks and two embedding backbones, SimSPARSE achieves competitive retrieval quality with low storage and query latency, while feature-induced topology substantially improves retrieval when sparse representations are held fixed. On the 60-million-document RPJ-Wiki corpus, at matched performance, SimSPARSE achieves single-digit-GiB storage and millisecond-scale latency, approaching the storage efficiency of recomputation-based indexing while retaining the low latency of dense graph search.