acceptodds
Under review as a conference paper at ICLR 2027

SimSparse: Semantic Search Can Stay Sparse End to End

Abstract

Scaling semantic retrieval requires balancing corpus storage against the computation needed to access it. Conventional graph indexes retain dense document embeddings for low-latency search, while low-storage approaches often shift part of this cost to query-time document re-encoding. Learned sparse representations offer a different design point: their compact codes support direct scoring without document re-encoding, but compact scoring alone does not determine which documents should be evaluated. We introduce SimSPARSE, a sparse-native graph retrieval system that uses sparse features to organize and navigate the corpus. Shared active features generate candidate connections, active query features select entry points, and sparse inner products guide graph construction, traversal, and ranking. The deployed index retains only sparse codes, bounded-degree graph links, and short routing lists, without dense corpus embeddings or online document re-encoding. Across six benchmarks and two embedding backbones, SimSPARSE achieves competitive retrieval quality with low storage and query latency, while feature-induced topology substantially improves retrieval when sparse representations are held fixed. On the 60-million-document RPJ-Wiki corpus, at matched performance, SimSPARSE achieves single-digit-GiB storage and millisecond-scale latency, approaching the storage efficiency of recomputation-based indexing while retaining the low latency of dense graph search.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
sparsity, efficiency, RAG, retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kxTnOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SimSparse, a graph-based approximate retrieval index that operates directly on learned latent sparse embeddings. When constructing the graph, for each document, a few strong, rare features (considering activation and IDF) are selected. SimSparse then proposes candidate neighbors from three sources: 1) local windows around the document's position in activation-ordered posting lists; 2) per-feature "landmarks", i.e., the most strongly activated documents; 3) signature buckets of documents sharing the same top-L feature identifiers. Candidates are rescored with an IDF-weighted sparse inner product and pruned to M=16 out-edges. At query time, the query's active features retrieve short per-feature router lists. The best E entry points are selected by sparse score and used to start a standard best-first traversal. Experiments on six BEIR datasets with two backbones report an nDCG@10 at 30-40x lower storage than HNSW/DiskANN. On a 60M-passage RPJ-Wiki corpus, the authors report about 3.5-4 GiB of storage and about 9 ms/query at a matched 90% Recall@3 against a hybrid dense+sparse reference.

Critical strengths
  1. The motivation is clear and relevant.
  2. Experiments are performed on a mid-sized collection of 60M documents.
Critical weaknesses
  1. The paper does not actually consider an entire line of literature developed in the Information Retrieval community, which is the one researching for efficient retrieval over learned sparse representations (LSR) [some examples: 1,2,3,4,5,6,7,8]. Only a few papers in this line are cited. Nothing is used for comparison in the reported experiments. In this line, the authors will find inverted index-based solutions for LSR [1,2,3,5,6,7], along with fast HNSW-based adaptations for LSR (starting from BigANN challenge 2023 [4] up to the last months [8]). I invite the authors to reference the papers and, most importantly, to align and borrow from this line, the evaluation methodology which is built around three main directions: 1) small datasets, MSMARCOv1 (8M docs); 2) out-of-domain evaluation (BEIR), 3) very important, indeed: large datasets (MSMARCOv2, 145M docs). From the referenced papers, the contributions in this line are very strong competitors to SimSparse (reported latencies in the microsecond range for MSMARCOv1 and below 4ms for MSMARCOv2, which is far larger than the 60M used by the authors here). I am recommending testing these solutions in the same experimental evaluation to provide a more comprehensive analysis of the proposed solution.

  2. Some choices in the evaluation methodology are questionable: What is the rationale behind using hybrid reference top-3 neighbors as ground truth? Given k, why not report recall@k over the pre-computed list of k closest documents?

  3. Important experiments are missing: no index build time or peak construction memory are reported for SimSparse and competitors.

  4. Several experimental details are missing:

  • What is the hardware used?
  • What is the HNSW used? Did you compare with SOTA implementations, e.g., kANNolo [9], or other less competitive libraries?
  • How do you tune the hyperparameters for competitors?
  • No parameters reported for the solutions or the quantizations used in the main experiments.
  1. Code is not accessible at the time of the submission, although the authors promised to release it after.

  2. In terms of presentation, the paper is very dense and sometimes difficult to understand.

Overall, although the idea is interesting, the paper is not mature enough to be accepted at ICLR 2027.

[1] Bruch et al. Pairing Clustered Inverted Indexes with κ-NN Graphs for Fast Approximate Retrieval over Learned Sparse Representations. SIGIR 2024. [2] Bruch et al. Investigating the Scalability of Approximate Sparse Retrieval Algorithms to Massive Datasets. ECIR 2025. [3] Bruch et al. Pairing Clustered Inverted Indexes with κ-NN Graphs for Fast Approximate Retrieval over Learned Sparse Representations. CIKM 2024. [4] https://big-ann-benchmarks.com/neurips23.html [5] Mallia et al. Faster Learned Sparse Retrieval with Block-Max Pruning. SIGIR 2024. [6] Carlson et a. Efficient Sparse Retrieval with Lightweight Superblock Pruning. SIGIR 2026. [7] Li et al. SINDI: An Efficient Index for Sparse Vector Approximate Maximum Inner Product Search. ICDE 2026. [8] Martinico et al. Multivector Reranking in the Era of Strong First-Stage Retrievers. ECIR 2026. [9] Martinico et al. Two Indexes, One Foundation: kANNolo and Seismic for Efficient Learned Sparse Retrieval at SISAP 2026.

Questions for authors
  1. How does SimSparse behave against efficient graph-based and inverted-index-based solutions for LSR?
  2. What is the build time or peak construction memory for SimSparse and competitors?
  3. What are the performance of SimSparse over MSMARCOv2?
6 comments

We sincerely thank reviewer kxTn for the constructive feedback and for recognizing the importance of large-scale retrieval evaluation. We appreciate the reviewer’s guidance on efficient learned sparse retrieval methods and acknowledge the need for stronger comparisons. In response, we add specialized inverted-index and graph-based baselines, extend evaluation to the 145M-document MSMARCOv2 corpus, report index construction costs, and clarify implementation details.


W1 & Q1 & Q3: How does SimSPARSE compare with established learned sparse retrieval methods and scale to MSMARCOv2?

W1: The paper does not actually consider an entire line of literature developed in the Information Retrieval community, which is the one researching for efficient retrieval over learned sparse representations (LSR) [some examples: 1,2,3,4,5,6,7,8]. Only a few papers in this line are cited. Nothing is used for comparison in the reported experiments. In this line, the authors will find inverted index-based solutions for LSR [1,2,3,5,6,7], along with fast HNSW-based adaptations for LSR (starting from BigANN challenge 2023 [4] up to the last months [8]). I invite the authors to reference the papers and, most importantly, to align and borrow from this line, the evaluation methodology which is built around three main directions: 1) small datasets, MSMARCOv1 (8M docs); 2) out-of-domain evaluation (BEIR), 3) very important, indeed: large datasets (MSMARCOv2, 145M docs). From the referenced papers, the contributions in this line are very strong competitors to SimSPARSE (reported latencies in the microsecond range for MSMARCOv1 and below 4ms for MSMARCOv2, which is far larger than the 60M used by the authors here). I am recommending testing these solutions in the same experimental evaluation to provide a more comprehensive analysis of the proposed solution.

Q1: How does SimSPARSE behave against efficient graph-based and inverted-index-based solutions for LSR?

Q3: What are the performance of SimSPARSE over MSMARCOv2?

A1: We thank the reviewer for highlighting this important line of work. We agree that comparisons with specialized learned sparse representation (LSR) retrieval systems are essential. To address this concern, we have discussed these works in related work and conducted additional experiments against four representative methods: Seismic, SeismicWave, SINDI, and kANNolo, covering both inverted-index-based and graph-based approaches. For a fair comparison, all methods use exactly the same Contriever-based CSR representations, isolating the effectiveness of their indexing and search strategies. We evaluate on six BEIR benchmarks, including MSMARCOv1, and further extend the evaluation to the 145M-document MSMARCOv2 dataset.

Table I: Comparison of learned sparse retrieval methods on six BEIR benchmarks and MSMARCOv2. Each cell reports nDCG@10 (%) / average query latency (ms).

DatasetSeismicSeismicWaveSimSPARSESINDIkANNolo
HotpotQA28.04 / 0.42628.04 / 0.51528.79 / 0.22528.65 / 0.42926.65 / 0.361
NQ38.96 / 0.48038.94 / 0.50539.26 / 0.30738.93 / 0.54737.52 / 0.422
Climate-FEVER32.13 / 0.25232.13 / 0.26634.00 / 0.23833.74 / 0.43333.13 / 0.153
FEVER44.15 / 0.52444.17 / 0.53744.90 / 0.56544.36 / 0.74243.91 / 0.487
DBpedia23.65 / 0.62123.65 / 0.62724.45 / 0.52323.84 / 0.95024.03 / 0.519
MSMARCO23.12 / 0.96223.11 / 0.98223.08 / 0.90222.81 / 1.77022.40 / 1.069
MSMARCOv23.18 / 13.3743.15 / 15.1933.30 / 13.0853.02 / 15.8963.07 / 8.257
Avg. nDCG@10 (%)27.6027.5928.2527.9127.24

Shown in Table I, SimSPARSE achieves the highest nDCG@10 on 6 of 7 datasets (28.25% on average) and the lowest latency on 3 of 7 datasets. On the 145M-document MSMARCOv2, it achieves the highest nDCG@10 (3.30%) with 13.085 ms latency, demonstrating competitive retrieval quality and efficiency at scale. We have expanded the related work in Section 2 to better cover specialized learned sparse retrieval methods and added these comparisons, including the 145M-document MSMARCOv2 evaluation, to Section 5.1 (Table 2) of the revised paper.

(continue below)

W2: Why use hybrid top-3 neighbors as the retrieval ground truth?

Some choices in the evaluation methodology are questionable: What is the rationale behind using hybrid reference top-3 neighbors as ground truth? Given k, why not report recall@k over the pre-computed list of k closest documents?

A2: We thank the reviewer for this question. RPJ-60M lacks human relevance judgments, so we use pre-computed exact top-3 neighbors as a proxy ground truth. However, using dense or sparse top-3 alone would inherently favor methods based on the corresponding representation. We therefore construct the reference using an equal-weight combination of normalized dense and sparse scores. As shown in Table II, dense and sparse top-3 overlap by only 10.58% on average, while the hybrid reference captures information from both representations.

Table II: Top-3 overlap among exact dense, sparse, and hybrid rankings on RPJ-60M.

Query setDense/SparseDense/HybridSparse/Hybrid
NQ13.17%48.67%32.17%
GPQA8.17%45.50%28.83%
TriviaQA10.33%45.00%31.83%
HotpotQA10.67%44.33%34.50%
Avg.10.58%45.88%31.83%

We have clarified the representation-relative matched-performance criterion in Appendix C.2 and included the dense/sparse/hybrid ranking-overlap analysis in Appendix D.5, with the main findings summarized in Section 5.4 of the revised paper.


W3 & Q2: What are the index construction time and peak memory costs of SimSPARSE and its competitors?

W3: Important experiments are missing: no index build time or peak construction memory are reported for SimSPARSE and competitors.

Q2: What is the build time or peak construction memory for SimSPARSE and competitors?

A3: Thank you for the suggestion. We have conducted additional experiments measuring index construction time and peak memory usage across eight corpora and seven indexing methods. HNSW and DiskANN build their indexes using dense Contriever embeddings, while the remaining methods use sparse Contriever-CSR representations. The results are reported in Table III and Table IV.

Table III: Index construction time (s) across eight corpora for different indexing methods.

CorpusSimSPARSEHNSWDiskANNSeismicSeismicWavekANNoloSINDI
HotpotQA401.101743.322214.74625.69406.61418.45305.62
NQ209.24936.751288.19371.58356.75214.07187.66
ClimateFever451.731895.412572.75587.29671.27590.75404.62
Fever478.302032.793126.83612.35492.11490.42512.89
DBpedia362.681672.181947.06423.43395.86417.66385.13
MSMARCO716.593945.874434.53693.19669.97721.94872.18
RPJ-60M2043.1817589.5022923.382130.441812.052639.422796.29
MSMARCOv23837.7632654.2637820.695892.923668.725234.734783.97

Table IV: Peak index construction memory (GiB) across eight corpora for different indexing methods.

CorpusSimSPARSEhnswdiskANNSeismicSeismicWavekANNoloSINDI
HotpotQA2.0532.5385.772.025.371.931.74
NQ1.2816.8448.461.212.921.250.91
ClimateFever2.1333.6585.392.255.612.121.79
Fever2.1333.7085.432.255.622.121.83
DBpedia1.8928.8780.801.914.811.961.59
MSMARCO3.1854.6381.543.799.112.823.12
RPJ-60M25.94402.95584.1823.4857.6924.3055.70
MSMARCOv253.99848.03977.3452.86141.5052.1559.15

As shown above, SimSPARSE remains competitive with sparse indexing methods. Its peak construction memory is also dramatically lower than that of dense graph indexes. These results demonstrate that SimSPARSE's low deployment storage and query latency do not come at the expense of prohibitive index construction overhead. We have incorporated the complete index construction time and peak memory comparisons into Appendix D.8 (Tables 18 and 19), with the key results summarized in Section 5.4 of the revised paper.

(continue below)

W4: What hardware, baseline implementations, tuning settings, and quantization configurations were used?

Several experimental details are missing: What is the hardware used? What is the HNSW used? Did you compare with SOTA implementations, e.g., kANNolo [9], or other less competitive libraries? How do you tune the hyperparameters for competitors? No parameters reported for the solutions or the quantizations used in the main experiments.

A4: Thank you for pointing out the missing experimental details. We clarify the hardware, baseline implementations, hyperparameter tuning, and numerical precision below.

Hardware. All experiments were conducted on a server with dual Intel Xeon Platinum 8275CL CPUs (48 physical cores, 96 threads) and 1.10 TiB RAM, running Ubuntu 22.04.5.

Baselines and implementations. Our original HNSW baseline uses FAISS IndexHNSWFlat. To strengthen the comparison with state-of-the-art implementations, we additionally evaluated kANNolo, with results reported in A1. We also use the official DiskANN implementation (diskannpy, v0.7.0), Seismic (pyseismic-lsr, v0.5.1), and SINDI/VSAG (v0.18.5) for the corresponding baselines.

Hyperparameter tuning. We follow the standard tuning strategy for each method, varying the main search-time parameters to evaluate the quality–latency trade-off: efSearch for HNSW and kANNolo, nprobe for IVF, search-list size L_search for DiskANN, and candidate/pruning budgets for Seismic and SeismicWave. For SeismicWave, we additionally account for its KNN graph expansion. Construction parameters are fixed as summarized in Table V.

Table V: Baseline implementations and index-construction parameters.

MethodImplementationMain construction parameters
HNSWFAISS IndexHNSWFlatM=16, efConstruction=200
IVFFAISS IndexIVFFlatnlist=4096
DiskANNdiskannpy 0.7.0R=16, L=200
SeismicSeismicIndexRaw 0.5.1n_postings=3500, max_fraction=1.5, centroid_fraction=0.1, min_cluster_size=2, summary_energy=0.4, doc_cut=16
SeismicWaveSeismic native bindingSame Seismic configuration, plus build_knn(32)
kANNoloSparsePlainHNSW 0.15.1m=16, ef_construction=200, graph_type="standard"

Quantization. The main experiments use FP8 quantization for CSR. The alternative precision settings in Section 5.3 are evaluated separately to analyze the storage–accuracy trade-off.

The newly added comparisons with specialized sparse retrieval implementations, including kANNolo, are reported in Section 5.1 of the revised paper. The hardware environment, implementation versions, tuning procedures, and quantization settings are specified above to clarify the experimental setup.


W5: The implementation code was not publicly accessible at submission time.

code is not accessible at the time of the submission, although the authors promised to release it after.

A5: Thank you for highlighting the importance of code availability and reproducibility. We have made the core implementation of SimSPARSE available in an anonymous repository: https://anonymous.4open.science/r/SimSPARSE/.

The repository includes sparse feature-based graph construction, query-conditioned entry-point routing, index building, and top-k graph retrieval. It supports both standard CSR sparse representations and compressed RPJ-style indexes, with CPU and optional CUDA-accelerated construction. The README provides installation instructions, command-line examples, and a Python API for building indexes, loading them, and executing retrieval queries.

We hope this release facilitates inspection of the core indexing and search mechanisms of SimSPARSE.

(continue below)

Reviewer_fnYiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses large-scale semantic retrieval under a storage-computation tradeoff. It frames the design space as: dense graph indexes (HNSW, DiskANN) store full embeddings for fast scoring at high storage cost, while recomputation-based indexes (LEANN) cut storage by re-encoding selected documents at query time, paying query-time compute. The paper proposes a third operating point built on learned sparse representations (CSR / CSRv2): the compact sparse codes serve directly as the stored scoring representation, so neither dense corpus embeddings nor online re-encoding is needed.

Critical strengths
  1. A real new operating point, cleanly argued: one object does storage, scoring, and navigation; Fig. 1(b) positions it well.
  2. The fixed-representation topology experiment (Table 6) is the best evidence: holding sparse codes fixed and varying only the graph, SimSPARSE beats KNN/HNSW/DiskANN on 6/6 (Contriever) and 5/6 (BGE), often by large margins (NQ 32.61 vs 17.46/14.63). Exactly the right ablation.
  3. Mechanistic diagnostics (Table 7): sparse-kNN graphs reach only 73–76% of vertices with 18–22% zero-in-degree, a measurable reason naive sparse-kNN search fails, motivating landmarks + multi-entry routing.
  4. Honest, modest theory: the three propositions are explicitly about the mechanism, not retrieval quality ("Addressability is not exhaustive retrieval"; "routes are hypotheses, not guaranteed edges").
  5. Strong scaling result where it counts (Table 2): ~3.5–4 GiB at ~9 ms/query: LEANN's storage with ~300× lower latency, dense-graph latency at ~50× lower storage.
Critical weaknesses
  1. The strongest, most natural baselines are missing. Seismic and SeismicWave (which already pairs an inverted index with a k-NN graph + one-hop expansion — extremely close in spirit) are discussed at length but never run; SPLADE is discussed but never used as a representation. Every experiment uses CSR only. So the paper shows SimSPARSE beats dense indexes and naive CSR, but not that it is competitive with sparse SOTA — the single most consequential gap.
  2. The 60M headline rests on a proxy ground truth that mixes in sparse signal. With no human qrels, the reference top-3 is an equal-weight sum of normalized dense + sparse similarity (Eqs. 33–35) — and SimSPARSE retrieves by that same sparse similarity, so it may hit 90% Recall@3 more cheaply than dense-only methods. Direction/size of the bias isn't analyzed; global min–max normalization is also outlier-sensitive.
  3. No variance or significance anywhere. Several wins are small (Contriever avg 32.32 vs LEANN 31.15; and on BGE SimSPARSE 45.89 actually trails IVF 46.19 and both dense graphs), on BEIR sets with high per-query variance (DBpedia = 400 queries).
  4. "Competitive quality" understates a trade-off. with BGE, SimSPARSE trails dense graphs on quality while winning ~30–40× on storage; this is a good Pareto result but should be stated as a trade-off, not undifferentiated "competitive quality."
  5. SimSPARSE's own graph is never run through the diagnostics that motivate it. Table 7 shows reachability/zero-in-degree only for sparse-kNN; does SimSPARSE actually fix them? Also the directed graph has bounded out-degree but unbounded in-degree (hubness unexamined).
  6. One representation family, ~15 hyperparameters with sensitivity shown for only 3 (M, W, E); α, β, signature/landmark/router budgets unexamined, and how defaults were chosen (held-out?) isn't stated.
Questions for authors

Refer to weakness stated above

6 comments

We sincerely thank reviewer fnYi for the thoughtful and constructive feedback, and for recognizing SimSPARSE as a meaningful new operating point in the storage–latency trade-off. We particularly appreciate the reviewer's positive assessment of our fixed-representation topology experiments, graph navigation diagnostics, and the careful distinction between the indexing mechanism and retrieval guarantees. We also acknowledge the importance of evaluating against state-of-the-art learned sparse retrieval systems, examining potential biases in the hybrid reference, reporting statistical variability, and more precisely characterizing the retrieval quality–storage trade-off. In response, we provide new comparisons using both CSR and SPLADE representations, additional evaluation and robustness analyses, graph connectivity diagnostics, and a broader hyperparameter sensitivity study.


W1: How does SimSPARSE compare with state-of-the-art sparse retrieval methods such as Seismic and SeismicWave, and does it generalize to representations such as SPLADE?

The strongest, most natural baselines are missing. Seismic and SeismicWave (which already pairs an inverted index with a k-NN graph + one-hop expansion — extremely close in spirit) are discussed at length but never run; SPLADE is discussed but never used as a representation. Every experiment uses CSR only. So the paper shows SimSPARSE beats dense indexes and naive CSR, but not that it is competitive with sparse SOTA — the single most consequential gap.

A1: Thank you for your suggestion. We agree that comparisons with strong sparse retrieval baselines are essential.

To address this concern, we conducted additional experiments comparing SimSPARSE with Seismic [1] and SeismicWave [2] on all six BEIR benchmarks in Table 1 of the original paper, using both Contriever-based CSR and SPLADE representations. Within each representation, all methods use identical document and query embeddings, enabling a direct comparison of their indexing and search effectiveness.

Table I: Comparison using Contriever-based CSR representations. Each cell reports nDCG@10 (%) / average query latency (ms). Bold indicates the best retrieval quality or latency for each dataset.

DatasetSeismicSeismicWaveSimSPARSE
HotpotQA28.04 / 0.42628.04 / 0.51528.79 / 0.225
NQ38.96 / 0.48038.94 / 0.50539.26 / 0.307
Climate-FEVER32.13 / 0.25232.13 / 0.26634.00 / 0.238
FEVER44.15 / 0.52444.17 / 0.53744.90 / 0.565
DBpedia23.65 / 0.62123.65 / 0.62724.45 / 0.523
MSMARCO23.12 / 0.96223.11 / 0.98223.08 / 0.902
Avg. nDCG@10 (%)31.6731.6832.41

Table II: Comparison using SPLADE representations. Each cell reports nDCG@10 (%) / average query latency (ms). Bold indicates the best retrieval quality or latency for each dataset.

DatasetSeismicSeismicWaveSimSPARSE
HotpotQA27.61 / 0.55627.78 / 0.45027.81 / 0.481
NQ45.42 / 0.76345.47 / 0.55645.50 / 0.472
Climate-FEVER30.20 / 0.47130.11 / 0.21632.17 / 0.434
FEVER59.10 / 0.28459.17 / 0.57258.92 / 0.489
DBpedia39.02 / 0.50939.18 / 0.66239.03 / 0.447
MSMARCO32.69 / 0.94133.00 / 1.10732.25 / 0.989
Avg. nDCG@10 (%)39.0139.1139.28

The additional results in Table I and Table II demonstrate that SimSPARSE is highly competitive with state-of-the-art sparse retrieval methods across both representations. With Contriever-based CSR, SimSPARSE achieves the highest nDCG@10 on five of six benchmarks, improving average nDCG@10 to 32.41% while reducing average latency by 15.47% and 19.58% compared with Seismic and SeismicWave, respectively. With SPLADE, SimSPARSE continues to achieve the highest average nDCG@10 (39.28% vs. 39.11% and 39.01%), while maintaining comparable sub-millisecond latency. Importantly, since all methods use identical sparse representations, these results highlight the effectiveness of SimSPARSE's feature-induced graph construction and query-conditioned navigation, demonstrating its advantages beyond CSR and its competitiveness with strong sparse retrieval baselines. We have included these comparisons in Section 5.1 and Appendix D.3, with the generalization results further summarized in Section 5.4 of the revised paper.

(continue below)

W2: Does the hybrid ground truth introduce evaluation bias in favor of SimSPARSE, and how does the score normalization affect the results?

The 60M headline rests on a proxy ground truth that mixes in sparse signal. With no human qrels, the reference top-3 is an equal-weight sum of normalized dense + sparse similarity (Eqs. 33–35) — and SimSPARSE retrieves by that same sparse similarity, so it may hit 90% Recall@3 more cheaply than dense-only methods. Direction/size of the bias isn't analyzed; global min–max normalization is also outlier-sensitive.

A2: We thank the reviewer for raising this suggestion. Since RPJ-60M lacks human relevance judgments, using either dense or sparse top-3 alone would inherently favor the corresponding methods. We therefore adopt an equal-weight hybrid reference. As shown in Table III, dense–sparse overlap is only 10.58%, while dense–hybrid overlap (45.88%) is substantially higher than sparse–hybrid overlap (31.83%), suggesting that the hybrid reference is more aligned with dense retrieval rather than favoring SimSPARSE. We acknowledge that min–max normalization may be sensitive to outliers. Importantly, 90% matched performance refers to 90% of each representation's exhaustive-search hybrid Recall@3, rather than a shared absolute recall target. This accounts for differences in the achievable hybrid agreement between dense and sparse representations.

Table III: Top-3 overlap among exact dense, sparse, and hybrid rankings on RPJ-60M.

Query setDense/SparseDense/HybridSparse/Hybrid
NQ13.17%48.67%32.17%
GPQA8.17%45.50%28.83%
TriviaQA10.33%45.00%31.83%
HotpotQA10.67%44.33%34.50%
Avg.10.58%45.88%31.83%

We have clarified the representation-relative matched-performance criterion in Appendix C.2 and added the hybrid ranking-overlap and recall analyses in Appendix D.5, with the key observations summarized in Section 5.4 of the revised paper.


W3: Are the reported performance improvements statistically significant and robust across queries and datasets?

No variance or significance anywhere. Several wins are small (Contriever avg 32.32 vs LEANN 31.15; and on BGE SimSPARSE 45.89 actually trails IVF 46.19 and both dense graphs), on BEIR sets with high per-query variance (DBpedia = 400 queries).

A3: Thank you for this suggestion. We have conducted five independent runs and report the mean ± standard deviation of nDCG@10 (%) across all six BEIR benchmarks.

Table IV: nDCG@10 (%) across five independent runs on six BEIR benchmarks using Contriever and BGE backbones (mean ± standard deviation).

BackboneDatasetSimSPARSELEANNIVFHNSWDiskANN
ContrieverHotpotQA28.43 ± 0.16118.77 ± 0.04213.88 ± 0.08015.21 ± 0.03314.39 ± 0.117
NQ39.18 ± 0.01336.14 ± 0.04830.92 ± 0.01826.37 ± 0.01626.52 ± 0.061
Climate-FEVER33.62 ± 0.17823.58 ± 0.01916.49 ± 0.0549.92 ± 0.05310.14 ± 0.025
FEVER44.94 ± 0.03359.69 ± 0.10328.73 ± 0.11733.64 ± 0.09534.53 ± 0.044
DBpedia24.40 ± 0.13727.32 ± 0.08630.68 ± 0.06834.06 ± 0.01633.43 ± 0.033
MSMARCO23.13 ± 0.13521.54 ± 0.01620.13 ± 0.04526.15 ± 0.02727.67 ± 0.035
Avg.32.28 ± 0.07631.17 ± 0.05223.47 ± 0.06424.23 ± 0.04024.45 ± 0.053
BGEHotpotQA31.85 ± 0.05419.38 ± 0.02727.20 ± 0.18128.39 ± 0.03928.10 ± 0.002
NQ54.81 ± 0.08051.84 ± 0.09755.17 ± 0.03258.83 ± 0.00957.35 ± 0.023
Climate-FEVER41.32 ± 0.07331.72 ± 0.07230.11 ± 0.09930.17 ± 0.06431.68 ± 0.021
FEVER71.56 ± 0.02173.13 ± 0.08174.54 ± 0.01083.78 ± 0.07782.25 ± 0.175
DBpedia33.47 ± 0.16936.29 ± 0.05644.86 ± 0.05439.42 ± 0.10242.04 ± 0.066
MS MARCO42.39 ± 0.13635.76 ± 0.07245.29 ± 0.00345.46 ± 0.06444.79 ± 0.085
Avg.45.90 ± 0.08941.35 ± 0.06846.20 ± 0.06347.68 ± 0.05947.70 ± 0.062

Table IV shows low run-to-run variance and consistent performance comparisons. Our central claim is competitive retrieval quality with substantially lower storage and latency, rather than superior accuracy. We report five-run statistics in Appendix D.6 and summarize stability in Section 5.4.

(continue below)

W4: Does the claimed competitive retrieval quality adequately reflect the trade-off between retrieval effectiveness and storage efficiency?

"Competitive quality" understates a trade-off. with BGE, SimSPARSE trails dense graphs on quality while winning ~30–40× on storage; this is a good Pareto result but should be stated as a trade-off, not undifferentiated "competitive quality."

A4: Thank you for this helpful suggestion. We agree that the main contribution of SimSPARSE is a favorable storage–latency–quality trade-off rather than uniformly superior retrieval accuracy. As shown in Section 5.1, SimSPARSE achieves substantial storage savings while maintaining competitive retrieval quality, although the strongest dense baselines achieve higher accuracy on some benchmarks. The additional efficiency results in Section 5.4 further characterize this operating point.


W5: Does SimSPARSE actually resolve the graph connectivity issues identified in sparse-kNN graphs, and how does it handle potential in-degree hubness?

SimSPARSE's own graph is never run through the diagnostics that motivate it. Table 7 shows reachability/zero-in-degree only for sparse-kNN; does SimSPARSE actually fix them? Also the directed graph has bounded out-degree but unbounded in-degree (hubness unexamined).

A5: We thank the reviewer for this suggestion. We have additionally evaluated the reachability, zero in-degree ratio, and in-degree distribution of SimSPARSE's own graph across all six benchmarks.

Table V: Graph connectivity diagnostics of SimSPARSE.

DatasetReachableZero in-degree
HotpotQA84.50%13.57%
NQ88.49%7.12%
ClimateFever87.64%11.89%
Fever88.67%8.80%
DBpedia82.17%12.05%
MSMARCO83.49%12.85%

As shown in Table V, SimSPARSE achieves 82–89% reachability across all six datasets, with zero in-degree ratios of only 7–14%. On HotpotQA, under the same diagnostics as Table 7, SimSPARSE improves reachability from 73.12% to 84.50% (+11.38 percentage points) and reduces zero in-degree from 22.28% to 13.57% (−8.71 points) compared with sparse-kNN. These results confirm that our feature-induced construction improves graph connectivity, while query-conditioned multi-entry routing further reduces reliance on a single entry point.

Table VI: In-degree distribution of SimSPARSE's directed graph.

DatasetMedianP90P99P99.9Max
HotpotQA314225188030754
NQ32722711379729
ClimateFever416198165340230
Fever417203158862169
DBpedia315226184139626
MSMARCO316260167024585

We acknowledge that SimSPARSE does not bound in-degree. However, as shown in Table VI, the distribution is highly skewed: median in-degree is only 3–4, while a small number of nodes receive many incoming edges. This is consistent with our design: high-activation landmarks serve as shared semantic representatives, providing nonlocal shortcuts from many documents to promising regions. Importantly, high in-degree does not directly increase traversal cost, since search expands only outgoing edges, which remain bounded by M=16M=16. Thus, such hubs can facilitate navigation without increasing the per-node expansion budget.

We have added the connectivity and in-degree diagnostics of SimSPARSE's own graphs to Appendix D.1, with the main improvements summarized in Section 5.4 of the revised paper.

(continue below)

Reviewer_fQhdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SimSparse is a graph retrieval system built over fixed learned sparse document and query codes. It proposes graph neighbors using local windows in activation-ordered postings, high-activation landmarks, and signatures of strong features. Candidates are scored and pruned to a bounded-degree graph. At query time, short feature conditioned lists propose entry documents, followed by bounded best-first traversal using sparse inner products. The deployed state consists of sparse codes, adjacency, and routing lists; construction postings are discarded, and documents are not re-encoded online. The paper evaluates six BEIR collections with Contriever and BGE backbones, includes component ablations and fixed-representation topology comparisons, and reports approximately 3.49/4.02 GiB indices and 9 ms mean query latency on a 60M-passage corpus at a stated 90% Recall@3 target against a hybrid dense-sparse reference.

Critical strengths
  1. A coherent design linking compact scoring and candidate access. The same sparse codes support feature based graph construction, query-conditioned entry selection, traversal, and ranking. The separation between temporary construction structures and retained deployment state is explicit. This is a concrete way to avoid both dense corpus-vector retention and online document re-encoding, rather than treating sparsity only as a compression format.
  2. Useful scale and controls for the contribution of access structure. The evaluation covers six million-scale BEIR collections with human relevance judgments, two embedding backbones, and a 60M-passage corpus. Figure 4 and Table 6 hold sparse representations fixed across graph constructions, and Table 3 provides cumulative component ablations. These controls give evidence that the proposed access structure contributes beyond the learned representation itself. The appendices also correctly distinguish candidate routes from guarantees about the pruned graph.
Critical weaknesses
  1. The central storage and latency result is not sufficiently auditable. Table 2 reports total SimSparse storage of 3.49/4.02 GiB, but does not instantiate the component level accounting in Equation 19. For scale, at 60M documents, 16 links per document with 32-bit document IDs would require approximately 3.58 GiB for neighbor IDs alone, before codes, offsets, or routing lists. This is not necessarily inconsistent: 16 is a degree bound, and actual degree or encoding may differ. However, actual edge counts, support sizes, numerical formats, compression, and component wise bytes are needed to reconcile the totals. The latency setup also omits hardware, threading, batching/concurrency, cache and memory residency conditions, and baseline execution details. The expansion budget C and optional early stopping setting are unspecified, although the algorithm distinguishes them from queue capacity W. These omissions directly affect the claimed storage-latency advantage.
  2. The matched-performance comparison and closest sparse alternative remain insufficiently validated. On RPJ-60M, the reference top-3 is defined by the sum of separately normalized dense and sparse scores, while the compared systems rank using their own dense or sparse scoring functions. Hybrid Recall@3 therefore combines ranking disagreement with index approximation, and increasing search effort need not improve it monotonically. Table 2 does not give achieved recalls, selected search settings, or exact dense/sparse agreement with the hybrid reference. This is particularly important for CSR-exhaustive, whose exact sparse result has no approximation budget to tune. The hybrid reference is a permissible proxy, but it does not by itself isolate search fidelity. In addition, the study does not evaluate Seismic or SeismicWave on the same sparse codes, despite accurately discussing these directly relevant sparse-access systems. The existing graph comparisons do not establish the claimed operating-point advantage over that alternative access route.
Questions for authors
  1. Please provide the actual corpus and edge counts, support sizes, data formats, and separate bytes for sparse codes, adjacency, offsets, and routing state behind Table 2. What expansion budgets and stopping rules were used, on which hardware and threading/device settings? Please distinguish serialized index size from resident memory and report repeated latency measurements. 2.What hybrid Recall@3 does every Table 2 row achieve in each query domain, including CSR-exhaustive, and at which search settings? Please report agreement among exact dense, exact sparse, and hybrid top-3 results, together with recall-latency curves. Reporting fidelity to each native exact scorer separately would clarify the contribution of the index.
  2. How does SimSparse compare with Seismic or SeismicWave using identical sparse codes and precision at matched quality? A representative collection would already help test the central systems claim. For the existing topology controls, please also specify the shared degree, entry-routing, traversal, and search-work settings.
3 comments

We sincerely thank reviewer fQhd for the thoughtful and constructive feedback, and for recognizing the strengths of our work, particularly the unified use of compact sparse codes for graph construction, query-conditioned routing, traversal, and scoring, as well as the extensive evaluation across multiple retrieval benchmarks and representation backbones. We especially appreciate the reviewer's recognition of our fixed-representation topology comparisons and component ablations, which help isolate the contributions of the proposed access structure. We also acknowledge the importance of making our storage and latency measurements fully auditable, clarifying the matched-performance evaluation protocol, and comparing against specialized sparse retrieval systems. In the following, we address these concerns through detailed storage accounting, explicit experimental configurations, additional evaluation analyses, and new comparisons with strong sparse retrieval baselines.


W1 & Q1: Can the detailed index statistics, memory breakdowns, search configurations, and hardware settings be provided?

W1: The central storage and latency result is not sufficiently auditable. Table 2 reports total SimSPARSE storage of 3.49/4.02 GiB, but does not instantiate the component level accounting in Equation 19. For scale, at 60M documents, 16 links per document with 32-bit document IDs would require approximately 3.58 GiB for neighbor IDs alone, before codes, offsets, or routing lists. This is not necessarily inconsistent: 16 is a degree bound, and actual degree or encoding may differ. However, actual edge counts, support sizes, numerical formats, compression, and component wise bytes are needed to reconcile the totals. The latency setup also omits hardware, threading, batching/concurrency, cache and memory residency conditions, and baseline execution details. The expansion budget C and optional early stopping setting are unspecified, although the algorithm distinguishes them from queue capacity W. These omissions directly affect the claimed storage-latency advantage.

Q1: Please provide the actual corpus and edge counts, support sizes, data formats, and separate bytes for sparse codes, adjacency, offsets, and routing state behind Table 2. What expansion budgets and stopping rules were used, on which hardware and threading/device settings? Please distinguish serialized index size from resident memory and report repeated latency measurements.

A1: We thank the reviewer for highlighting these points. We clarify the actual deployed index sizes, graph statistics, and retrieval settings below.

Storage breakdown and actual graph sizes. Both indexes contain 60450220 chunks, each with 21 nonzero FP8 features in a 65536-dimensional space, with each edge id represented by 26 bits. Importantly, M=16M=16 is an upper bound rather than the actual out-degree. Our deployment applies explicit graph pruning, retaining up to 16 links for hub nodes, resulting in the graph statistics shown in Table I. The component-wise storage breakdown is provided in Table II, including compression metadata and runtime files.

Table I: Actual graph statistics of the deployed SimSPARSE indexes.

Graph statisticsContrieverBGE
Actual directed edges272889668433062432
Average out-degree4.5142877.163951
Hub nodes21817684499
Router entries46604392094828

Table II: Component-wise storage breakdown of the deployed SimSPARSE indexes (GiB).

ComponentContrieverBGE
Sparse codes (IDs, values, normalization)2.6112.666
Graph (including auxiliary structures)0.8611.346
Router (including auxiliary structures)0.0180.008
Other metadata/runtime<0.001<0.001
Total3.4904.020

Latency measurement and execution conditions. Retrieval is CPU-only on a dual Intel Xeon Platinum 8275CL server (48 physical cores, 96 logical threads, approximately 1.1 TiB RAM). The documented query-validation configuration uses batch size 1, one sequential query at a time. Latency is measured including router decoding, candidate deduplication and scoring, and graph traversal. It excludes query encoding, index loading, and offline construction. Index files are accessed via read-only memory mapping with OS-managed paging.

(the latter part of A1 to W1 & Q1 is in the next block)

Baseline implementations. Our comparison uses established implementations rather than substituting construction parameters for retrieval settings: FAISS HNSW (IndexHNSWFlat), official DiskANN (diskannpy), native Seismic/SeismicWave, kANNolo (SparsePlainHNSW), and VSAG/SINDI. HNSW and DiskANN use original 768-dimensional FP32 dense vectors, and the sparse baselines use normalized Contriever-based Top-21 FP8 CSR inputs.

Expansion budget and early stopping settings. Unless otherwise specified, SimSPARSE experiments use E=4E=4, W=128W=128, and C=256C=256, with early stopping disabled. Importantly, WW and CC control distinct aspects of retrieval: WW bounds the number of candidates maintained in the search frontier, whereas CC bounds the number of graph-node expansions. Neither restricts the number of candidates scored during routing or traversal. In particular, router candidates are scored before selecting the top-EE entry points, and neighbors are scored before frontier pruning. Search terminates only when the frontier is exhausted or CC expansions have been performed. The early stopping is implemented but not conducted in the main experiments.

Repeated latency measurements. We conducted five repeated evaluations of SimSPARSE on RPJ-60M across four query domains. Table III reports the mean and standard deviation of per-query latency (ms), demonstrating consistent millisecond-scale performance across repeated evaluations and supporting the latency results in Table 2.

Table III: SimSPARSE retrieval latency across five repeated evaluations on RPJ-60M (mean ± standard deviation, ms/query).

NQGPQATriviaQAHotpotQA
7.97 ± 0.6311.05 ± 0.778.38 ± 0.397.14 ± 0.92

We have incorporated the actual graph statistics, component-wise storage breakdown, and repeated latency measurements into Appendix D.4, with the key findings summarized in Section 5.4 of the revised paper.


W2 & Q2: Does the hybrid Recall@3 evaluation reliably establish matched-performance comparisons, particularly against exact sparse retrieval and Seismic-based alternatives?

W2: The matched-performance comparison and closest sparse alternative remain insufficiently validated. On RPJ-60M, the reference top-3 is defined by the sum of separately normalized dense and sparse scores, while the compared systems rank using their own dense or sparse scoring functions. Hybrid Recall@3 therefore combines ranking disagreement with index approximation, and increasing search effort need not improve it monotonically. Table 2 does not give achieved recalls, selected search settings, or exact dense/sparse agreement with the hybrid reference. This is particularly important for CSR-exhaustive, whose exact sparse result has no approximation budget to tune. The hybrid reference is a permissible proxy, but it does not by itself isolate search fidelity. In addition, the study does not evaluate Seismic or SeismicWave on the same sparse codes, despite accurately discussing these directly relevant sparse-access systems. The existing graph comparisons do not establish the claimed operating-point advantage over that alternative access route.

Q2: What hybrid Recall@3 does every Table 2 row achieve in each query domain, including CSR-exhaustive, and at which search settings? Please report agreement among exact dense, exact sparse, and hybrid top-3 results, together with recall-latency curves. Reporting fidelity to each native exact scorer separately would clarify the contribution of the index.

A2: We thank the reviewer for the valuable suggestions. We clarify the evaluation protocol and provide additional results below.

Hybrid reference and matched-performance criterion. RPJ-60M lacks human relevance judgments, and using either exact dense or sparse top-3 as ground truth would favor the corresponding methods. As shown in Table IV, exact dense and sparse rankings overlap by only 10.58% on average, motivating our use of a shared hybrid reference. We acknowledge that hybrid Recall@3 measures agreement with this proxy rather than pure index fidelity. Importantly, 90% matched performance means achieving 90% of each representation's exhaustive-search hybrid Recall@3, not an absolute Recall@3 of 90%. CSR-exhaustive therefore serves as the exact sparse reference without search-budget tuning.

Table IV: Top-3 overlap among exact dense, sparse, and hybrid rankings.

Query setDense/SparseDense/HybridSparse/Hybrid
NQ13.17%48.67%32.17%
GPQA8.17%45.50%28.83%
TriviaQA10.33%45.00%31.83%
HotpotQA10.67%44.33%34.50%
Avg.10.58%45.88%31.83%

(the latter part of A2 to W2 & Q2 is in the next block)

Achieved Recall@3 and latency sensitivity. Table V reports achieved hybrid Recall@3, including exhaustive baselines, with sparse and dense methods calibrated to their respective exhaustive ceilings. Table VI shows that SimSPARSE latency increases gradually at stricter relative-recall thresholds (90–98%) across four domains.

Table V: Achieved hybrid Recall@3 (%) at selected operating points on RPJ-60M.

MethodNQHotpotQATriviaQAGPQA
CSR-exhaustive32.1734.5031.8328.83
CSR-IVF29.0231.4729.1326.49
Seismic29.1531.4228.8726.11
SeismicWave29.3331.1628.8026.34
SimSPARSE29.2331.3828.9626.27
Dense-exhaustive48.6744.3345.0045.50
HNSW43.8840.3740.7741.14
IVF43.8139.9340.6241.08
LEANN44.1040.5240.9641.33
DiskANN44.0839.9640.8541.01

Table VI: SimSPARSE query latency (ms) at different relative-recall thresholds on RPJ-60M.

ThresholdNQHotpotQATriviaQAGPQA
90%7.977.148.3811.05
92%8.147.358.7711.32
94%8.537.728.9811.54
96%8.768.049.3411.86
98%—8.269.6212.15

Comparison with Seismic and SeismicWave. Using identical Contriever-based CSR representations, Table VII compares all three methods on six BEIR datasets with official relevance judgments. SimSPARSE achieves higher nDCG@10 on five datasets (+0.74 points on average over Seismic), while reducing average latency by 15.47% versus Seismic and 19.58% versus SeismicWave, supporting sparse-native graph access over posting-based alternatives.

Table VII: Comparison with Seismic and SeismicWave on six BEIR datasets (nDCG@10 (%) / latency (ms)).

DatasetSeismicSeismicWaveSimSPARSE
HotpotQA28.04 / 0.42628.04 / 0.51528.79 / 0.225
NQ38.96 / 0.48038.94 / 0.50539.26 / 0.307
Climate-FEVER32.13 / 0.25232.13 / 0.26634.00 / 0.238
FEVER44.15 / 0.52444.17 / 0.53744.90 / 0.565
DBpedia23.65 / 0.62123.65 / 0.62724.45 / 0.523
MSMARCO23.12 / 0.96223.11 / 0.98223.08 / 0.902
Avg.31.67 / 0.54431.68 / 0.57232.41 / 0.460

We have incorporated the specialized sparse retrieval comparisons into Section 5.1, clarified the representation-relative recall criterion in Appendix C.2, and added the ranking-overlap, achieved-recall, and latency-sensitivity analyses in Appendix D.5. The main findings are further summarized in Section 5.4 of the revised paper.


Q3: Does SimSPARSE outperform Seismic or SeismicWave under matched sparse representations and retrieval quality, with controlled graph and search configurations?

How does SimSPARSE compare with Seismic or SeismicWave using identical sparse codes and precision at matched quality? A representative collection would already help test the central systems claim. For the existing topology controls, please also specify the shared degree, entry-routing, traversal, and search-work settings.

A3: Thank you for your suggestion. For the comparison with Seismic and SeismicWave under identical sparse codes and precision, please see A2.

For the existing graph topology comparisons, we clarify that all methods use the same sparse document and query codes, with the following shared settings shown in Table VIII.

Table VIII: Shared settings for the controlled graph topology comparisons.

SettingValue
Graph degreeM=16M = 16
Entry routingLr=128L_r = 128, E=4E = 4
TraversalBest-first search
Search workExpansion budget (C=256C = 256)

The fixed-representation topology comparisons are presented in Section 5.3 and Appendix D.1 of the revised paper. The additional comparisons with Seismic and SeismicWave under identical sparse representations are included in Section 5.1.


We sincerely thank Reviewer fQhd for the valuable feedback. We have provided additional clarifications and new experiments that directly address the points raised, and have incorporated these improvements into the revised submission. If the reviewer finds the revisions helpful, we would greatly appreciate a reconsideration of the rating. We are truly grateful for your time and thoughtful evaluation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Paper Update

We sincerely thank all reviewers for their careful reading, constructive feedback, and valuable suggestions. We have addressed the reviewers' concerns through additional experiments, expanded comparisons, and clarifications in the revised manuscript, extending the main text from 9 to 10 pages. All revisions and additions are highlighted in blue for ease of reference. The main updates are summarized below:

Section 2 (Related Work) [Reviewer kxTn: W1, W6]: Expanded the discussion of efficient learned sparse retrieval, incorporating recent advances in block-max pruning, superblock pruning, Seismic, SeismicWave, and SINDI to better position SimSPARSE relative to existing sparse indexing approaches.

Section 5.1 (Benchmark Performance) [Reviewer fQhd: W2/Q2, Q3; Reviewer fnYi: W1, W4; Reviewer kxTn: W1/Q1/Q3]: (new) Added a comprehensive comparison with four specialized sparse retrieval systems Seismic, SeismicWave, SINDI, and kANNolo, using identical Contriever-based CSR representations (Table 2). Extended the evaluation to the 145M-document MSMARCOv2 corpus, where SimSPARSE achieves the highest nDCG@10 (3.30%) among the evaluated methods.

Section 5.4 (Additional Experimental Analysis) [Reviewers fQhd, fnYi, kxTn]: (new) Added a concise summary of the additional evaluations, organized into three themes: generalization and robustness, index efficiency and construction cost, and retrieval fidelity and graph connectivity.

Appendix C.2 (Evaluations) [Reviewer fQhd: W2/Q2; Reviewer fnYi: W2; Reviewer kxTn: W2]: Clarified that the 90% matched-performance criterion is defined relative to each representation's exhaustive-search hybrid Recall@3, rather than an absolute recall threshold.

Appendix D.1 (Graph Topology and Navigation Diagnostics) [Reviewer fQhd: Q3; Reviewer fnYi: W5]: (new) Added connectivity and in-degree diagnostics for SimSPARSE's own graphs (Tables 9–10), showing improved reachability and reduced zero in-degree compared with sparse-kNN.

Appendix D.3 (Evaluation with SPLADE Representations) [Reviewer fnYi: W1]: (new) Evaluated SimSPARSE against Seismic and SeismicWave using identical SPLADE representations (Table 11), demonstrating generalization beyond CSR.

Appendix D.4 (Index Storage and Latency Statistics) [Reviewer fQhd: W1/Q1]: (new) Provided actual graph statistics, component-wise deployment storage breakdowns, and repeated latency measurements on RPJ-60M (Tables 12–14).

Appendix D.5 (Additional Hybrid Retrieval Analysis) [Reviewer fQhd: W2/Q2; Reviewer fnYi: W2; Reviewer kxTn: W2]: (new) Added exact dense/sparse/hybrid top-3 overlap measurements, achieved hybrid Recall@3 for the evaluated methods, and latency sensitivity under stricter relative-recall thresholds (Tables 15–17).

Appendix D.6 (Performance Variability Across Runs) [Reviewer fnYi: W3]: (new) Reported mean and standard deviation of nDCG@10 across five independent evaluations on six BEIR benchmarks with Contriever and BGE backbones (Table 18).

Appendix D.7 (Additional Hyperparameter Sensitivity) [Reviewer fnYi: W6]: (new) Extended the sensitivity analysis to five additional parameters α\alpha, β\beta, BB, LhL_h, and LrL_r, examining the robustness of the selected configurations (Table 19).

Appendix D.8 (Index Construction Cost) [Reviewer kxTn: W3/Q2]: (new) Added index construction time and peak memory comparisons across eight corpora and seven indexing methods (Tables 20–21), including RPJ-60M and MSMARCOv2.

We hope these revisions and additional results address the reviewers' concerns and provide a clearer, more comprehensive understanding of SimSPARSE's retrieval effectiveness, generalizability, storage efficiency, robustness, and scalability.

AuthorsOpenReview ↗

Rebuttal Summary

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

We sincerely thank the Program Chairs, Senior Area Chairs, and Area Chairs for coordinating the review process, and all Reviewers for their careful evaluations and constructive feedback. We greatly appreciate their recognition of the importance of storage-efficient retrieval, the unified sparse-code-based design of SimSPARSE, and the systematic experimental evaluation.

During the rebuttal and discussion period, we conducted substantial additional experiments and strengthened the manuscript to address the reviewers' concerns. We summarize the key revisions and discussion outcomes below.

Additional Experiments. Following the reviewers' suggestions, we conducted several new experiments and incorporated the results into the revised submission:

  1. Comparisons with specialized sparse retrieval systems, including Seismic, SeismicWave, SINDI, and kANNoIo, under identical sparse representations, covering both Contriever-based CSR and SPLADE. (Raised by Reviewers fnYi, kxTn, fQhd; Section 5.1 and Appendix D.3)
  2. Large-scale retrieval evaluation on the 145M-document MSMARCOv2 corpus, together with index construction time and peak memory comparisons across eight corpora. (Raised by Reviewer kxTn; Section 5.1 and Appendix D.8)
  3. Evaluation of retrieval fidelity through exact dense/sparse/hybrid ranking overlap, achieved Recall@3, and recall-latency sensitivity analyses. (Raised by Reviewers fQhd, fnYi, kxTn; Section 5.4 and Appendix D.5)
  4. Statistical robustness analysis through repeated retrieval-quality and latency evaluations, including mean and standard deviation measurements. (Raised by Reviewers fnYi, fQhd; Appendix D.6)
  5. Graph connectivity diagnostics and additional hyperparameter sensitivity analyses, covering reachability, in-degree distributions, and graph construction and routing parameters. (Raised by Reviewer fnYi; Appendices D.1 and D.7)

Additional Clarifications. We further clarified the experimental methodology, evaluation criteria, and practical contributions:

  1. Detailed component-wise storage accounting, actual graph statistics, hardware configurations, and retrieval settings, now included in Appendix D.4. (Raised by Reviewer fQhd)
  2. Clarification of the hybrid Recall@3 reference and representation-relative matched-performance criterion, now included in Section 5.4 and Appendix C.2. (Raised by Reviewers fQhd, fnYi, kxTn)
  3. Specification of baseline implementations, hyperparameter tuning procedures, quantization settings, and controlled graph topology comparisons, now included in Sections 5.3–5.4. (Raised by Reviewers kxTn, fQhd)
  4. Clarification of SimSPARSE's storage–latency–retrieval-quality trade-off and the scope of its contributions, now highlighted in Section 5.4. (Raised by Reviewer fnYi)
  5. Release of the core SimSPARSE implementation in an anonymous repository to support reproducibility. (Raised by Reviewer kxTn)

Discussion Outcome. We appreciate the constructive feedback and positive developments during the discussion:

  • Reviewer fnYi (3, Weak Accept) explicitly acknowledged that the additional Seismic/SeismicWave comparisons and graph connectivity diagnostics addressed their biggest concern, while maintaining a positive recommendation.
  • Reviewer kxTn (1 → 2, Strong Reject to Weak Reject) raised concerns about specialized sparse retrieval baselines, scalability, and experimental completeness. We addressed these through substantial new experiments and clarifications, and appreciate the reviewer's upward score revision.
  • Reviewer fQhd (2, Weak Reject) raised concerns about storage and latency auditability, evaluation fidelity, and sparse retrieval baselines, which we directly addressed through additional experiments and detailed methodological clarifications.

We believe these additional experiments and clarifications have substantially strengthened the submission and addressed the major concerns raised during review. In particular, the new comparisons with specialized sparse retrieval systems and large-scale evaluations provide stronger evidence for SimSPARSE's favorable storage–latency–retrieval-quality trade-off.

We sincerely thank the Program Chairs, Senior Area Chairs, Area Chairs, and all Reviewers once again for their time, dedication, and valuable feedback. We hope the revised submission will receive favorable consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.