- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Reliable Reasoning with Videos (RRV), a training-free inference framework designed to improve the reliability of video question answering. The method combines two mechanisms: within-observation reliability estimation, which assigns uncertainty-based weights to sampled reasoning trajectories, and cross-observation stability verification, which progressively expands the observed video frames and checks answer consistency. Adaptive stopping rules are introduced to reduce unnecessary sampling and terminate inference when consecutive observations produce stable predictions.
Critical strengths
- The paper identifies a relevant limitation of conventional self-consistency methods when reasoning over incomplete video observations.
- The ablation studies, frame-sensitivity experiments, and accuracy–cost comparisons offer insights into how progressive observations and repeated reasoning affect prediction stability.
Critical weaknesses
- The proposed framework primarily combines existing techniques, including uncertainty-weighted self-consistency, adaptive trajectory sampling, progressive frame expansion, and answer-agreement-based stopping. Related ideas have already been explored in Self-Consistency (ICLR 2023), Escape Sky-High Cost (ICLR 2024), and Flexible Frame Selection for Efficient Video Reasoning (CVPR 2025). Although applying these techniques jointly to video reasoning is reasonable, the proposed method remains algorithmically straightforward and does not establish a sufficiently distinctive contribution beyond their combination.
- The theoretical results largely formalize general properties of answer distributions and stopping decisions rather than establishing new guarantees about video reasoning reliability. Lemma 1 states a standard margin-stability condition, while Proposition 1 guarantees preservation of the full-budget voting leader rather than answer correctness. Proposition 2 bounds answer reversal using a distribution-shift term between observations, but this term is not empirically estimated or meaningfully constrained and can dominate the bound. Consequently, the theoretical guarantees do not establish that cross-observation agreement implies sufficient visual evidence or a correct answer. The mathematical analysis appears considerably stronger in presentation than in its actual explanatory contribution.
- The entropy-based trajectory weights show relatively weak discrimination between correct and incorrect predictions, with within-group AUROC values of only 58.5–60.8% in the reported VideoTemp-o3/MMVU analysis.
- RRV uses substantially more inference computation than standard self-consistency, with approximately 6.76–8.18 times the output tokens on MMVU while achieving relatively modest accuracy gains. Although adaptive stopping reduces computation relative to the full RRV procedure, the experiments do not establish superiority over simpler approaches at matched inference budgets.
- The method assumes that agreement between two consecutive expanded observations provides a useful indicator of prediction reliability. However, agreement does not necessarily imply correctness, and the paper itself provides examples of consistently incorrect predictions.
- Although RRV generally outperforms the reported baselines, many improvements are relatively small, particularly on MLVU and several individual benchmark–backbone combinations.
Questions for authors
- How do Lemma 1 and Propositions 1–2 provide meaningful guarantees about answer correctness or sufficient visual evidence, rather than simply characterizing voting stability?
- Could the authors compare RRV against self-consistency, confidence-weighted voting, and progressive frame expansion using identical numbers of generated trajectories, visual tokens, and inference budgets?
- Could the authors clarify which component constitutes a genuinely new mechanism rather than an adaptation of established inference-time techniques?