- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether the performance improvements achieved by methods for mitigating hallucinations in large video models truly stem from enhanced temporal awareness, or whether they can be obtained through other information sources and evaluation settings. The authors analyze the impact of different visual occlusion benchmarks and answer scoring rules on evaluation results, and design a DPO control experiment that alters only the training video input to analyze the source of the performance gains. The experiments show that different evaluation settings significantly affect performance results.
Critical strengths
The research subjects and starting point of this study are well-chosen.
Critical weaknesses
-
The writing in this paper could be significantly improved. It is evident that the current manuscript strives to present the authors’ work, but the current writing is obscure and difficult to understand. It reads more like an initial draft generated directly from code by a LLM, with little or no human review, as the paper contains numerous elements that defy common sense in human writing: for example, the abstract’s phrasing “with matching machine-verified and every arm evaluated on untouched video,” the use of subheadings to introduce every paragraph, and the presentation of content in Section 5. Therefore, I consider this to be a significant weakness.
-
The writing and experimental sections appear to contain many inconsistencies or contradictions. Please see the “Questions” section for details.
Questions for authors
-
In Section 2.5, the authors note that the “whole-set result” may arise naturally from arithmetic because many samples do not require motion at all, and that the truly informative question is what happens with motion-necessary items. However, this MNS conditional analysis was not performed. Therefore, I would like the authors to further explain: in the absence of results for MNS, to what extent should the core interpretation in the Abstract and Conclusion be understood?
-
In Section 2.1, A2 performs a uniform permutation of 32 frames. Based on this, the authors state that A2’s training video carried no motion. However, the paper also explicitly acknowledges that A2 preserves all frame-, object-, scene-, and set-level cues. Although the shuffled frame set lacks temporal order, it may still contain elements such as the range of object displacement and state changes across multiple frames. Therefore, the authors need to clarify the boundaries of the motion-blind definition.