- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates why, in "retrieve-and-rerank" systems, it is difficult to achieve significant per-query performance gains through score fusion, even when employing learned fusion weight strategies. Specifically, we conduct a precise contribution analysis regarding the R@1 metric by leveraging the valid range of fusion weights and the "gain-minus-loss" identity. Furthermore, we evaluate various learning strategies across image and text retrieval tasks and propose a novel method called "Winner-Path Duel" (WPD); this approach assigns a score from the second-stage ranker to a candidate result only when that result emerges as the "winner" under a specific fusion weight configuration.
Critical strengths
- Highly efficient and stable two-stage verification (WPD): Going away from global computation, WPD verifies candidates secondarily (only a small proportion of candidates that are more likely to rank first) (only 1.6–2.0 calls per query), it still switches results only when the model is quite confident, avoiding the incorrect rejection of correct results.
- Breaking the bottleneck of static strategies: Unlike the conservative and fixed strategies, WPD well exploits target information, and unlocking 22% (CIRR) and 15% (FashionIQ) of the potential performance headroom on the development set. Superior gains in generalization and scalability: On unseen test data, R@1 generally improves 1.16–2.17 percentage points.
- By using the computational power of bigger models strategically, focusing it precisely where it is needed (evaluating only candidate paths), performance gains increase to 1.85–3.99 percentage points, and up to 36% of the theoretical potential performance is recovered.
Critical weaknesses
- The visual style of the Figures in the paper leans too heavily towards AI generation; many of the tables are misaligned, and there is a noticeable misalignment in Part 2 of Figure 6. This raises doubts about the authenticity of the experiments. Perhaps the authors could adjust the figures using the actual experimental data.
- This paper current experimental comparison concentrates only on the variants from author’s proposal, such as ridge regression, candidate posterior, binding, neural set decoder, WPD, etc. So the core proposition ‘only decoders trained to find the target beat the tuned baseline’ is valid for specific method families and unable to draw a universal conclusion about adaptive fusion. Maybe the author can compare typical realizations of uncertainty gate, query router, generic cascade, pair reranker under the same conditions for more reliable conclusion.
- The evidence on WPD is convincing; but the actual reach of applicability for claims in the paper about its efficiency and robustness is not as wide as explained in the paper. The number of “1.6–2.0 calls per query”, measured by the number of candidate evaluations by the second verifier, is indeed convincing, but it does not readily reduce to end-to-end latency, FLOPs, memory cost, batching performance, cost of VLMs for different image resolution and tokenization sizes. Authors may consider adding an experiment specifically on efficiency.
Questions for authors
The specific issues have been raised in the section Critical Weaknesses.
followup
Dear Reviewer QW35,
Thank you again for your careful review. Since our reply above, we have made several revisions to make the experimental results and comparisons easier to verify. Here is a one-glance map of the changes (all marked with pink bars in the revised PDF):
Figures and tables → every figure was regenerated directly from the saved experiment outputs using a unified plotting pipeline, with an independent script verifying the source values (Appendix Q, pp. 54–55); the reported values remain unchanged.
Comparisons beyond our variants → new Table 1 (p. 10) adds uncertainty gates, query routers, generic cascades, and pair rerankers under the same queries, folds, seeds, and constant (Appendices M and O).
Measured cost → we now report end-to-end latency, FLOPs, and memory across batch sizes, image sizes, and caption lengths (Section 7, p. 9; Appendix P): 29–46% of latency, 26–47% of FLOPs, and 2.7–5.2 GB additional memory.
Regarding your first point, we also strengthened the reproducibility checks behind the figures. All plotted values are read directly from saved experiment outputs, and each figure is associated with its source file and hash; an independent checker additionally verifies the reported values. We would also be happy to provide the corresponding code and data during the discussion period if that would be helpful.
If these changes address your concerns, we would be grateful if you would reconsider your score. If any point remains unclear or insufficiently addressed, we would greatly appreciate your guidance.
Thank you again.
Sincerely, The authors