- Rating
- 2
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces EvidenceFlow, a training-free visual replay framework for test-time multimodal reasoning. The design idea of EvidenceFlow is to distinguish between the visual evidence currently required by the reasoning process and the visual support that has actually been realized. The authors thus model multimodal reasoning as a mismatch between current visual demand and history-aware visual support. They define this mismatch as Visual Evidence Debt and uses it to guide visual token replay. Experiments on three MLLMs and five benchmarks show consistent improvements in both reasoning accuracy and visual-compute efficiency.
Critical strengths
-
The problem formulation is clear and quite intuitive. The distinction between current visual demand and realized visual support provides a useful way to characterize why long multimodal reasoning trajectories may gradually lose visual grounding.
-
The empirical results are strong. EvidenceFlow improves over standard thinking and other replay baselines across all three evaluated models and five benchmarks, suggesting that the method is reasonably robust across different model families and tasks. The ablations are also substantial.
Critical weaknesses
-
The Visual Evidence Debt measure is equivalent to the total variation distance between two normalized distributions. Moreover, the approximation guarantee in Proposition 1 follows from the well-established result of Nemhauser et al. (1978). Thus, the novelty seems to lie primarily in the task-specific formulation and integration of components rather than in new theoretical developments.
-
While the paper provides validation for these estimators, it is still less clear whether the same conclusions would hold under alternative definitions of and .
-
The experiments focus mainly on visual QA and reasoning benchmarks, also limited at 8B scale models. It would be useful to understand whether the method generalizes to larger models or broader multimodal settings.
Questions for authors
-
How sensitive is the method to alternative choices for the visual-demand and realized-support estimators?
-
I see that Appendix B.7 provides useful selector comparisons. Could the authors further clarify when clause-level saturation offers the greatest benefit over simpler residual-based selection?
-
Do similar results also hold on substantially larger multimodal models, or for settings such as multi-image reasoning?
Thank you for the detailed explanations and the additional latency experiments. After reading the other reviewers’ comments, particularly those of Reviewer 9mck, I continue to have reservations about the method’s novelty. Specifically, Visual Evidence Debt appears to be mathematically equivalent to total variation distance, while the submodular objective and its approximation guarantee draw on established theoretical results. Although the formulation offers an intuitive perspective on visual replay and the additional experiments strengthen the empirical evidence, the methodological contribution beyond task-specific formulation and integration remains insufficiently clear.
Furthermore, the supplementary code does not appear to provide a complete, executable replay implementation, making it difficult to independently reproduce the reported improvements. This limits my ability to assess the method’s practical utility, while its generalizability to broader settings, such as multi-image reasoning, remains unverified.
I appreciate the authors’ efforts and the additional evidence provided during the rebuttal. However, given these remaining concerns, I will maintain my original score.