- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose a method to tackle the single-frame bias of large video language models. They do so by re-weighting the GRPO reward function, encouraging the model to achieve better predictions on longer videos compared to shorter reference videos. This is done in a progressive, curriculum-learning manner, where the number of frames of both the longer and shorter videos are gradually increased. In doing so, their proposed method outperforms the base model, as well as the base model post-trained with the standard GRPO objective over the six benchmarks evaluated. Furthermore, the authors perform some ablations and visualizations to gain more insights into the workings of the approach.
Critical strengths
- The problem is important, and the approach is clearly motivated.
- The method is clearly described and makes sense.
- The paper is well-written.
Critical weaknesses
The main weakness of the work is that the contributions of the proposed method can not be isolated, as an experiment comparing it to training the baseline model with curriculum learning only is missing. This is necessary to disentangle the effects of the progressive training from the proposed multi-frame focus and validate the key claims of the paper.
Another important weakness is the fact that the vast majority of the analysis is done with a single model, Qwen2.5, which is quite outdated at this point. The appendix A.1 introduces another small model in InternVL3-2B, but also this is a relatively smaller and older model, and the A.1 table is incomplete as it does not include the baseline model performance. Another issue is that the claim in A.2, "Our method improves the performance from 47.33% to 49.22%, showing that mitigating single-frame bias also benefits long-video understanding," does not specify relative to which baseline this increase in performance is. In general, Appendix A contains a start towards important additions to the paper, such as performance on other models and a wider variety of standard tasks, but is completely unintegrated and unconvincing in its current state.
Questions for authors
- How does the method perform compared to a baseline trained only with curriculum learning? How does it perform to an SFT baseline?
- Does the method work on newer models?
- While the attention visualizations are informative, a quantitative analysis of attention distributions would strengthen the paper.
- Does the method work if the contrast video is always one frame, as the title implies, instead of N/2?
- Why is fps=2 specified if the number of frames sampled is fixed?