- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies video hallucination in video-language models and identifies a limitation of standard outcome-reward GRPO, termed source-blind credit assignment: a correct answer receives the same reinforcement whether it is grounded in the video or can already be produced from the model’s language prior. To address this, the authors propose CF-GRPO, which modifies the GRPO update at three complementary levels. At the pair level, counterfactual pair optimization jointly trains opposite-label prompts sharing the same video so that common answer-shift updates cancel while discriminative updates are preserved. At the sample level, blind-failure advantage shaping estimates how likely the model is to answer correctly without visual input and downweights samples that are already solvable from language priors. At the within-group level, an evidence-consistency reward reallocates credit according to whether the reasoning contains answer-consistent visual evidence. Experiments on Qwen2.5-VL-7B across EventHallusion, VideoHallucer, and VidHalluc show that CF-GRPO improves over standard GRPO and other baselines while largely preserving general video-understanding performance, with additional analyses indicating reduced reliance on answers that remain unchanged when the video is removed.
Critical strengths
-
The paper identifies an important failure mode of outcome-based GRPO and provides a clear way to quantify visual dependence. The distinction between a correct answer that is genuinely grounded in the video and one that can be produced from the language prior is highly relevant for video hallucination. I particularly find the blind-input formulation useful: estimating whether the model can answer correctly without the video provides a direct and interpretable signal for how much a sample actually depends on visual evidence. This makes the source-blind credit assignment problem concrete than purely conceptual.
-
The method is well structured and experimentally well supported. The three components of CF-GRPO target complementary parts of the optimization process—pair-level update direction, sample-level reinforcement magnitude, and within-group credit allocation—and the Evidence-Consistency Reward is especially useful for distinguishing outputs that have the same final correctness but differ in whether their reasoning is consistent with the visual evidence. The paper also provides strong empirical validation through matched GRPO comparisons, ablations, answer-distribution analysis, zero-video dependence analysis, and general video-understanding benchmarks, making the overall engineering contribution convincing.
Critical weaknesses
-
It remains unclear how much of the improvement is specific to GRPO or to the counterfactual training recipe itself. CF-GRPO is compared carefully against standard GRPO, but the paper does not include a matched non-RL baseline trained with the same counterfactual pairs, hard negatives, evidence supervision, and blind-failure signal. In particular, the blind-failure weight can also be interpreted as an adaptive curriculum or sample-weighting mechanism, and the counterfactual pairs themselves provide strong discriminative supervision. The current SFT baseline is only the initialization from which the RL variants are trained, not a counterfactual/curriculum SFT baseline using the same data. This makes it difficult to determine whether the gains arise from GRPO-specific credit assignment or from the underlying data construction and curriculum. A matched comparison against counterfactual SFT or weighted/curriculum SFT would substantially strengthen the methodological claim.
-
The conceptual novelty is somewhat limited, with the contribution appearing primarily as a strong integration of existing ideas into GRPO. The observation that multimodal models can produce correct answers from language priors without relying on visual input is already well established, and counterfactual training has also been explored extensively for hallucination mitigation. The paper’s novelty therefore lies more in organizing these ideas into a unified GRPO framework—targeting update direction, sample-level reinforcement magnitude, and within-group credit allocation—than in introducing a fundamentally new learning principle. I view this as a well-executed engineering and methodological contribution with strong experiments, but the lack of matched non-GRPO baselines makes the incremental conceptual novelty more consequential to my overall assessment.
Questions for authors
*Could the authors compare CF-GRPO against other curriculum-learning such as VideoHallu? VideoHallu provides a carefully constructed expert dataset and benchmark, together with a curriculum-based training strategy for improving video understanding and reducing hallucination. Since several key ingredients of CF-GRPO—such as counterfactual samples and emphasizing examples that require stronger visual grounding—could also be incorporated into an curriculum-learning framework. A comparison using the same initialization, training data, and comparable training budget would be particularly informative. If CF-GRPO can consistently outperform such a strong curriculum-learning SFT/RL baseline, it would provide much stronger evidence for the specific advantage of Counter Factual setups, and I am more than happy to raise my score accordingly.
3 comments
Thank you for your response. Videohallu’s Curriculum RL is a classic and good baseline to compare with . The biggest issue in the paper is too few existing good methods to compare with. Visual hallucinations especially in video is a hard problem. Please add this baseline into your main paper or next version. It will make the paper better. I’ll raise my score, good luck.
Thank you for your positive feedback and constructive suggestions. We agree that comparisons with strong existing methods are important for evaluating progress on video hallucination. In the revised manuscript, we will include the VideoHallu-inspired curriculum GRPO baseline, together with RFT and weighted RFT, in the main experimental section, and clarify its connection to VideoHallu and implementation details. We appreciate your suggestions, which have helped strengthen the paper.
We thank the reviewer for the detailed and constructive comments and for acknowledging that our work identifies an important failure mode of outcome-reward GRPO and that the method is well structured and experimentally well supported. We respond to each point below and include the requested matched baselines.
W1. Can matched comparisons against counterfactual SFT, weighted SFT and a curriculum-learning RL baseline such as VideoHallu be added, with the same initialization and training data?
Yes. As shown in Table R1, CF-GRPO outperforms matched RFT, weighted RFT, and curriculum GRPO on the overall metric of all three benchmarks. All baselines use the same SFT initialization and CF-GRPO training prompts. The baselines are implemented as follows.
CF-GRPO exceeds the strongest of these baselines by 1.95, 1.57, and 0.72 percentage points on EventHallusion, VideoHallucer, and VidHalluc, respectively, supporting its benefit beyond the shared data and the tested weighting and curriculum strategies.
Table R1: Comparison with non-RL and curriculum baselines trained on the same data as CF-GRPO.
W2. The work makes a solid methodological contribution, supported by strong experiments. Could you further clarify its conceptual novelty beyond integrating existing ideas into GRPO?
Thank you for the positive assessment of our methodological contribution and experiments. CF-GRPO’s conceptual contribution is a unified theoretical framework that explains both the failure of outcome-only credit assignment and the complementary roles of our interventions.