- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Query-Conditioned Reconstruction (QCR), a training-free framework for query-adaptive compression in visual document retrieval. To address the limitations of static multi-vector compression, QCR preserves cluster centroids and low-rank residual statistics during offline indexing, and dynamically reconstructs query-conditioned document vectors at retrieval time. The framework further introduces Budgeted MaxSim Coverage to select complementary vectors under a fixed representation budget. Experiments across multiple retrieval backbones and visual document benchmarks demonstrate consistent improvements over static compression baselines, with additional gains in evidence retrieval and downstream long-document question answering.
Critical strengths
1.The paper identifies a meaningful limitation of static compression: document representations are constructed without knowledge of future queries, potentially suppressing query-relevant fine-grained information. Deferring part of the representation construction until retrieval time provides a reasonable direction for addressing this limitation.
2.QCR does not require updating the retrieval encoder or accessing the original full multi-vector representations during retrieval. Its combination of low-rank residual memory, query-aware group activation, and closed-form reconstruction offers a potentially useful approach to query adaptation under constrained storage budgets.
Critical weaknesses
1.Although the paper clearly motivates the limitations of static compression, it does not sufficiently explain why cluster centroids and low-rank residual statistics are adequate for preserving query-relevant information. The residual subspace is selected through SVD based on explained residual energy, which is not necessarily aligned with retrieval relevance. Low-energy directions may still contain critical fine-grained evidence, while high-energy directions may capture less discriminative variations. Consequently, the relationship between residual reconstruction fidelity and retrieval-relevant information preservation remains insufficiently established.
2.QCR constructs a continuous reconstruction region using cluster centroids, a low-rank residual basis, and second-order statistics. However, the reconstructed vectors do not necessarily correspond to actual visual embeddings present in the original document. Moreover, the diagonal residual statistics may overlook correlations between feature directions, potentially allowing combinations that were never observed in the original embeddings. Although the trust radius constrains reconstruction, it does not guarantee semantic fidelity. As a result, query-conditioned reconstruction may artificially increase matching scores for irrelevant pages, particularly hard negatives.
3.The reconstruction procedure primarily seeks to maximize cosine similarity between the query and reconstructed vectors. However, increasing query–document similarity does not necessarily improve ranking discrimination across documents. Both relevant and irrelevant pages may benefit from query-conditioned adaptation, and the paper does not sufficiently analyze whether reconstruction systematically improves positive–negative score margins. While the reported NDCG and Recall improvements support the overall effectiveness of the method, they do not fully explain the underlying mechanism responsible for better ranking.
Questions for authors
1.Why should SVD-based residual energy maximization preserve fine-grained query-relevant information? Is there empirical evidence supporting this assumption?
2.Since reconstructed vectors may not correspond to actual visual features, how does QCR avoid introducing spurious matches, particularly for hard negatives?
3.Maximizing query–reconstruction similarity does not necessarily improve retrieval ranking. Could the authors analyze its impact on positive–negative ranking margins?
3 comments
The additional experiments and analyses have largely addressed my concerns regarding Questions 1 and 3.Regarding Question 2, I acknowledge that exact semantic fidelity is not necessarily required for compressed retrieval representations. Although the potential for spurious matches remains a limitation, the additional analyses have sufficiently alleviated my concerns about its practical impact. I am willing to raise my rating to 3 and encourage the authors to incorporate these additional experiments and analyses into the revised manuscript.
We sincerely thank you for your support of our additional experiments and analyses. As you suggested, we will fully incorporate these additional experiments and analyses into the revised manuscript.
Best
R1: Residual energy versus retrieval relevance.
SVD minimizes residual projection error at a fixed rank, providing a query-agnostic approximation criterion; query-dependent activation subsequently selects useful preserved variation. This does not guarantee preservation of every low-energy discriminative cue. To examine this choice, we compare leading, random, and low-energy subspaces with identical groups and ranks, re-estimating the residual statistics for each variant.
SVD improves NDCG@5 retention and Recall@5 over the random orthogonal, with larger gains over the low-energy SVD subspace. This supports preserving leading residual directions under a fixed rank budget, without implying that every low-energy direction is irrelevant.
R2: Semantic fidelity and spurious matches.
We thank the reviewer for this concern. QCR constructs query-conditioned retrieval vectors from compressed page statistics. Its page-specific residual basis restricts adaptation directions, while group statistics, the trust radius, and the step-size bound constrain the update. These constraints do not guarantee semantic fidelity or rule out spurious matches. To investigate potential spurious matches induced by hard-negative scores, we report Mean inflation score. Scores are normalized by query length, and the positive part of each hard negative's score difference is averaged over a fixed evaluation set.
QCR exhibits higher mean inflation than Static but slightly lower inflation than Sculpting. To better assess the ranking impact of potential spurious matches, we further examine positive--negative margins and pairwise accuracy in R3.
R3: Does similarity maximization improve ranking?
We compare Static, Sculpting, and QCR using fixed positive–hard-negative pairs under matched storage budgets. Scores are normalized by query length, relevance labels determine the preferred page, and ties count as incorrect. QCR achieves larger mean margins and higher pairwise accuracy across the three storage ratios. At 5% storage, it increases the mean margin from 0.0121 to 0.0241 and pairwise accuracy from 71.6% to 80.3% relative to Static. Thus, although reconstruction can increase hard-negative scores, the margin improvements indicate that relevant pages benefit more on average over the evaluated pairs.
QCR consistently achieves larger positive–negative margins and higher pairwise accuracy than both baselines across storage ratios. These results indicate that reconstruction benefits relevant pages more on average and improves ranking discrimination over the evaluated pairs.