- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes EigenLI, a training-free method for compressing late-interaction representations. Each document is represented by the leading eigenspace of its token-embedding second-moment matrix. Relevance is then measured by the sum of squared projections of query embeddings onto this subspace. A quadratic feature map gives an exactly equivalent single-vector formulation, EigenLI-SV. Experiments cover several text retrieval models on BEIR and a visual document retriever on ViDoRe-v3, with additional analyses of query compression, centering, and quantization.
Critical strengths
-
The construction is simple and its components fit together well. Document compression, the projection score, and the equivalent single-vector representation follow from the same matrix formulation (§§3–3.1). The method can be applied to existing late-interaction embeddings without retraining. The eigenvalue-weighted formulation for query compression is also a useful consequence of this approach.
-
The experiments reveal meaningful differences between embedding models. The evaluation covers multiple text models and visual retrieval, and the paper reports settings where the method works less well, including GTE-ModernColBERT. The gains over clustering-based compression for ColBERTv2 and AnswerAI provide evidence that document-specific subspaces are worth investigating. Reporting the unfavorable cases makes these observations more informative.
Critical weaknesses
-
The relationship to the closest prior work is not developed sufficiently to establish the contribution. Piwowarski et al., Filtering Documents with Subspaces (ECIR 2010), already construct document subspaces from the leading eigenvectors of a document-specific second-moment matrix and score matches using summed projection quadratic forms, equivalently a trace expression (§§2.1–2.2 of that paper). These are mathematically close to EigenLI’s document representation and score. The brief discussion in §2 does not explain the overlap or identify the new elements precisely. The application to contextual token embeddings, the single-vector implementation, and the empirical findings may provide a contribution, but the current presentation leaves too much of that distinction implicit.
-
The approximation analysis and experiments leave the practical retrieval advantage unresolved. Low-rank document structure alone does not establish preservation of MaxSim scores or rankings. For example, changing the signs of document vectors leaves their second-moment matrix unchanged while potentially changing their MaxSim interactions. The bound in §3.2 and Appendix A is one-sided and concerns a related score, so its implications for ranking quality need to be stated carefully. The efficiency evidence has a separate limitation: comparing an 8,256-dimensional encoding with a padded (512 * 128) representation does not establish actual index savings. Moreover, the largely brute-force evaluation does not show the quality–latency tradeoff of candidate generation followed by reranking. The difficulty reproducing MUVERA’s published effectiveness further complicates that comparison.
Questions for authors
-
Could you provide an explicit mathematical comparison with Piwowarski et al. (2010), identifying which parts of the subspace construction and scoring function are inherited, and which algorithmic or empirical contributions are new?
-
What explains the lower reproduced MUVERA results? Please clarify the candidate-generation and reranking protocols. A comparison using the same reranker, a matched retrieval-quality target, and storage measured from actual non-padding document vectors would be particularly helpful in assessing the practical contribution.
-
In the normalization step of the proof on p.14, dividing a projected vector by a norm at most one does not necessarily increase its inner product with the query when that inner product is negative. The zero-projection case also needs handling. Could this step be replaced by a direct Cauchy–Schwarz argument? This appears to be a repairable proof issue rather than a counterexample to the stated bound.
Dear Reviewers, as the discussion period is ending soon, can you please respond to the Authors. In particular, can you state which of your critical concerns are still outstanding, which ones were solved by the author response, and whether this affects your final score.
Thank you!