- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose InterEvolve, a training-free loop for multimodal retrieval in which an LVLM judges the current Top-1 and rewrites the query if it looks wrong. A rewrite is accepted only if it passes two checks: RUPA, which asks the LVLM to answer the question directly and requires the answer to become more similar to the new Top-1, and QRV, which limits how much the score distribution over the current Top-10 changes. If either check fails, the search stops. On MMEB-v2 the method improves four embedding models by 0.4 to 2.0 points on average, and beats a naive rewrite loop by 0.61 to 1.69 points on the Image subset.
Critical strengths
- The paper shows a real problem clearly: a naive rewrite loop can do worse than single-pass retrieval, for example 74.14 vs. 74.96 with a weaker LVLM (Table 3). The fix is simple, needs no training, and reuses the existing retriever and LVLM, so it is easy to adopt.
- The method section is precise, and the authors are honest that RUPA measures agreement with the LVLM rather than correctness.
Critical weaknesses
- Some results don't match the algorithm as described. A failed check returns the Top-1 of the last accepted state, so if no rewrite is ever accepted, the output is just the initial Top-1. At δ_RUPA = 1, where almost no rewrite can pass, results should therefore be close to single-pass retrieval (74.96 for Qwen3-VL-2B). Yet in Fig. 3a the curves stay near their best values (about 76.7 for Kimi).
- There are also a few number errors. In Fig. 3b, r_0 = 0.6311 and r_1 = 0.5731, so by Eq. 4 Δr_0 should be −0.058, but it is printed as 0.1311. In Table 4 (Qwen3-VL-8B), Ret goes from 78.97 to 80.62 but is marked +0.56, and GD goes from 92.92 to 93.22 but is marked +1.65.
- It isn't clear where the gain comes from. The only agentic baseline is the naive loop, while RUPA asks a strong LVLM to answer the question directly, which may itself inject useful knowledge. Simple baselines with the same LVLM, such as retrieving with the LVLM's answer or reranking the Top-10, are missing. The cost is also not reported, even though Algorithm 1 can make up to 13 LVLM calls per query. Thresholds were chosen on the same benchmark used for evaluation, with single runs, and the full method is only 0.11 above RUPA alone in Table 2.
- Reproducibility is also a concern: the LVLM prompts and the softmax temperature τ are not given, and there is no Reproducibility Statement, which ICLR strongly encourages.
Questions for authors
see weaknesses