- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Monosemantic Retrieval (MR), which learns per-feature retrieval weights over monosemantic representations from Sparse Autoencoders and scores retrieval over shared sparse features instead of cosine similarity. Across 66 benchmarks, MR matches state-of-the-art dense retrieval with 28× fewer query-time operations and 43% of its index bytes, while outperforming existing lexical and learned-sparse methods. MR also enables inference-time steering, achieving a 20.9% relative improvement over state-of-the-art dense retrievers in a mixed multi-domain corpus.
Critical strengths
- The paper proposes a meaningful research direction by using SAE feature coordinates as retrieval units and further leveraging interpretable features for dynamic control at inference time.
- The paper provides extensive experimental evaluations across 66 benchmarks, covering retrieval tasks in a wide range of domains.
- The paper conducts comprehensive ablation studies to validate the effectiveness of its individual components.
Critical weaknesses
- The claimed 28× reduction in query-time arithmetic may not fully reflect the practical computational efficiency of MR. The cost analysis mainly accounts for the sparse postings scan and the dense scoring of the top-200 candidates, without reporting the complete end-to-end retrieval latency, including the cost of computing dense embeddings on the fly.
- The number of queries in the mixed-index steering simulation appears inconsistent. Section 6.1 and Appendix E state that the index includes nine collections with 100 queries per collection, implying 900 queries in total. However, the inference-time steering experiment uses 978 queries, and Table 10 reports more than 100 queries for several collections.
- The normalization procedure for facet projections is described inconsistently. Section 6.1 states that each document's projection is standardized over the index, whereas Appendix G states that the median and spread are computed from the same positive and negative examples used for facet construction. These two normalization strategies may produce different steering effects.
Questions for authors
- Could the authors provide end-to-end latency and computational cost measurements, including the dense embedding computation required for reranking? Does the reported 28× reduction in per-query arithmetic translate into comparable practical speedups?
- Could the authors clarify how the 978-query stream was constructed and explain why some collections contribute more than 100 queries despite the stated sampling protocol?
- Could the authors clarify whether the facet projections are normalized using statistics from the entire index or from the mined positive and negative examples? Which normalization procedure was used in the reported steering experiments?
4 comments
Thank you for your rebuttal.
However, your responses confirm that all three issues I identified were actual errors and self-contradictions in the original manuscript (especially conflating the 43% and 68% storage settings in the headline claims, while still not providing the complete end-to-end retrieval latency). Essentially, you are patching these flaws during the rebuttal phase rather than resolving misunderstandings.
So I will maintain my score.
Thank you for engaging in the discussion.
We would like to clarify again that we did report both of these numbers in the original submitted manuscript. The settings which give the 43% and 68% storage gains are presented in the Section 5 of the main paper in lines 279-282: "against the trained retriever on 53 shared benchmarks, the sparse index alone reaches 0.445 against 0.473 from 43% of the index bytes, and a dense rerank of its top 200 candidates closes the gap to within 0.005 at 28×to 55× less query-time arithmetic, with the index at 68% of the dense bytes when a 4-bit copy is stored for the rerank." as well as in Appendix L. Our proposal was to state this more explicitly in the paper. Specifically, we will update the claim in the abstract so that the main storage gains also come from the stored-copy setting. The same applies to the other two points in the discussion. In each case, the reported numbers and the implementations are unchanged. The edits we proposed restate what the paper already reports more explicitly, so we respectfully see them as clarifications and not patchwork or actual errors.
End-to-end latency. We reported operation counts because wall-clock latency depends heavily on the hardware, the search kernels and the index library, which are not the focus of this work. Operation counts give a more reliable unit that is comparable across systems.
We report the end-to-end latency including the query encoding, in milliseconds per query:
| Corpus | Docs | CPU: MR-G | CPU: Dense | GPU: MR-G | GPU: Dense |
|---|---|---|---|---|---|
| SciFact | 5k | 23 | 26 | 23.6 | 23.0 |
| FollowIR News21 | 31k | 32 | 38 | 32.3 | 31.6 |
| FiQA | 58k | 29 | 40 | 23.9 | 23.1 |
| Quora (hard negatives) | 177k | 27 | 67 | 26.1 | 25.0 |
| Quora | 523k | 40 | 172 | 29.4 | 27.3 |
On the CPU, MR-G runs on Seismic (a sparse engine), while the dense model runs on FAISS. MR-G is faster at every index size. The latency gap grows with collection size, as we mentioned earlier. On the GPU, the dense search runs on cuBLAS, which is heavily optimized for dense matrix products. Our sparse search there has no comparable kernel, yet it stays within 1 to 3 ms end to end. We will add these measurements to Appendix L with the caveats we noted.
We hope this can address your concerns.
Thank you for the clarification, I have raised my score.
Thank you again for your review. We wanted to follow up on our responses. We clarified that the key results use stored dense vectors for the rerank, and explained the cost calculations. We also explained how the 978-query stream was built, and we confirmed that facet normalization uses statistics from the full index. We will update the paper and appendices to match these descriptions.
We would be happy to discuss anything further. If these responses address your concerns, we would appreciate it if you would consider updating your score.