- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a method called Query-Conditioned Residual Retrieval (QCR2) to address a limitation of existing multimodal retrieval methods, where converting multimodal content into textual descriptions can omit visual or temporal details relevant to specific queries. The method applies orthogonal projection to decompose each multimodal embedding into a component aligned with its textual descriptions and a residual component orthogonal to the description subspace. It then introduces a separate retrieval path based on these residual representations to retrieve potentially relevant information that may not be adequately captured by the textual descriptions.
Critical strengths
1- The proposed method is training-free and clearly defined, which makes it appealing because it can be easily integrated into existing methods.
2- The paper provides an in-depth analysis, a theoretical overview, and comprehensive evaluations that help explain the proposed method and its effectiveness.
Critical weaknesses
1- My main concern is the novelty of the proposed method. QCR2 decomposes multimodal embeddings through orthogonal projection and uses the residual as an additional retrieval channel. However, the conceptual contribution beyond existing projection-based representations and multi-channel retrieval methods remains unclear.
2- QCR2 achieves a relatively small improvement over UniversalRAG. Moreover, Orthogonality to the textual-description subspace does not necessarily imply semantic complementarity or query relevance; residuals may retain irrelevant visual variations rather than useful information omitted by descriptions.
Questions for authors
I have no questions.
5 comments
Any follow-up question?
Dear Reviewer NALs,
Thank you again for your positive evaluation and thoughtful feedback on our work. As the author-reviewer discussion period closes shortly (about 22 hours), we wanted to briefly summarize how our response and magenta-highlighted manuscript revisions address both of your points:
- Conceptual contribution over projection & fusion baselines (W1): We have sharpened our conceptual positioning and contrasted offline item-specific description decomposition against multi-channel fusion, query-adaptive selection, adapted GRAIL, modality-wise centering, and global SVD projection (Official response; Sec. 1, Sec. 2, Sec. 3.3, Sec. 4.2, Sec. 4.4, Table 2, Table 3).
- UniversalRAG comparison & irrelevant visual variation (W2): We have added paired bootstrap confidence intervals and controlled UniversalRAG comparisons, and clarified how residual normalization and zero-mean candidate standardization suppress query-irrelevant visual noise (Official response; Sec. 3.4, Sec. 4.2, Table 2, App. B.4, Table B3, App. C.5, Table C8).
Could you let us know whether these new experiments and clarifications resolve your main concerns, or if there are any remaining questions we can address before the discussion period ends?
Gentle reminder before discussion closes
Dear Reviewer NALs,
This is a gentle reminder that about 6 hours remain in the author-reviewer discussion period. Even a brief note on whether our rebuttal and revised manuscript have addressed your feedback would be greatly appreciated. If your concerns have been resolved, we also kindly ask whether you would consider updating your score accordingly. Thank you again for your time and thoughtful review!
Official response to Reviewer NALs
We sincerely thank the reviewer for the positive evaluation and thoughtful feedback. We are encouraged that you found QCR$^2$ training-free, clearly defined, easy to integrate, and supported by comprehensive theoretical and empirical analyses. Although no explicit questions were listed, we address both of your critical weaknesses directly below and in the revised manuscript (summarized in "Revision summary").
[W1] Conceptual contribution beyond existing projection and multi-channel retrieval methods
While QCR$^2$ intentionally uses lightweight linear projection and dual-index search to remain training-free and indexable at 30.0 ms latency (Table D12), our controlled ablations in Table 2 and Table 3 show that what subspace is projected out, and at which side of retrieval, is critical, rather than projection or multi-channel retrieval alone:
Existing projection-based representations either separate global modality-shared vs. modality-private subspaces across a corpus (41.3 to 41.4) or subtract already-retrieved context from the query across iterative hops (O & Lee, 2026, 41.2). Neither isolates what each individual item's concrete indexing captions omitted. Conditioning the orthogonal subspace on each item's own M=4 descriptions removes 45.1% of redundant caption-explained norm energy per item and accounts for over 70% of the total gain (+1.5 of +2.1 over learned fusion). We have sharpened this conceptual positioning in Sections 1 and 2.
[W2] Improvement over UniversalRAG and geometric orthogonality vs. query-irrelevant visual variations
(1) Improvement over UniversalRAG. In Table 1, UniversalRAG's macro-averaging weights Text, Table, Image, and Video equally, even though QCR$^2$ applies no residual branch to Text or Table items (1[xi∈Dv]=0). Across the six image and video benchmarks where residuals operate, QCR$^2$ outperforms UniversalRAG by +1.08 points (+1.15 on Image, +1.01 on Video; 95% paired bootstrap CI [+0.68, +1.49]), e.g., +1.55 on MRAG-Bench, +1.28 on LVBench, and +1.60 R-L on VideoRAG-Synth, surpassing both UniversalRAG (42.40) and its ground-truth corpus Oracle (42.45) without needing a trained modality-granularity router. Under the strictly matched Table 2 backbone (Qwen3-VL-Embedding-2B), QCR$^2$ also outperforms UniversalRAG (42.2) by +2.7 Recall@5 and +0.70 Avg. ([+0.42, +0.98]).
(2) Why geometric orthogonality helps despite query-irrelevant visual variations. We fully agree that vi⊥ is a geometric residual and can retain query-irrelevant visual variations such as background or viewpoint (as explicitly analyzed in Appendix F and illustrated by our failure case in Figure E4a). QCR$^2$ prevents such irrelevant variations from degrading retrieval through three design mechanisms: