acceptodds
Withdrawn from ICLR 2027

Reinforcing Video Reasoning via Fusion-Aware Chain-of-Thought

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has recently been extended to video reasoning, where perceptual grounding must be integrated with reasoning across space and time. However, Group Relative Policy Optimization propagates a single outcome-based advantage across all tokens, assigning similar credit to decision-promoting and decision-undirected information within Chain-of-Thought. Our analysis reveals a fundamental contrast: whereas image–text RLVR concentrates probability mass and progressively reduces trajectory entropy, video sustains correct trajectories across high probability–entropy bands. This observation suggests that video reasoning benefits from preserving uncertainty during temporal evidence accumulation and reducing it during reasoning. We therefore propose erception–easoning usion-ware olicy ptimization , which derives a perception–reasoning fusion gap from probability–entropy statistics over perceptual-prefix and reasoning-suffix segments. PR-FAPO maps this signal to token-wise discount coefficients, enabling fusion-aware credit assignment that promotes informative early perception and selectively fuses articulated visual information into subsequent reasoning. Experiments on reasoning-centric and temporal-grounding benchmarks validate the effectiveness of PR-FAPO, demonstrating performance improvements together with improved training stability.

Primary area:
optimization
Keywords:
reasoning, discounting, reinforcement learning, video language model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qgweOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Perception–Reasoning Fusion-Aware Policy Optimization (PR-FAPO) to improve reinforcement learning for video reasoning. The authors observe that successful video reasoning trajectories can maintain relatively high entropy, while VideoAuto-R1 tends to exhibit entropy collapse and shortened Chain-of-Thought (CoT) trajectories during training. Based on these observations, they argue that video reasoning requires different optimization signals for early perception and later reasoning. PR-FAPO constructs a perception–reasoning fusion gap from probability and entropy statistics over perceptual prefixes and reasoning suffixes. This signal is incorporated into multi-scale segment-level discounting and advantage estimation to produce token-wise weights for GRPO. Experiments on video QA and temporal grounding benchmarks show improvements on several reasoning-oriented tasks, with mixed results on perception-oriented tasks.

Critical strengths
  1. The paper studies how probability and entropy change during reinforcement learning for video reasoning. It also compares different training methods and modalities.

  2. PR-FAPO improves GRPO by using segment-level information, prefix and suffix statistics, adaptive discounting, and different window sizes, without needing extra visual labels.

  3. The paper tests the method on several video QA and temporal grounding benchmarks using Qwen2.5-VL and Qwen3-VL. It also includes ablation studies to show the effects of different components.

Critical weaknesses
  1. The main motivation is not well supported. The authors observe that successful video reasoning often has higher entropy and use this to support their method. However, higher entropy does not always mean better perception or reasoning. When visual evidence is clear, low entropy can also be correct. In addition, the paper mainly shows a correlation between entropy and performance, but does not prove that higher entropy during perception leads to better reasoning.

  2. The proposed fusion gap may not reflect how well the model uses visual evidence. The fusion gap is calculated from probability and entropy statistics, but these values cannot directly show whether the model correctly understands the video. For example, a model using correct visual evidence and a model reasoning from incorrect visual information may have similar statistics. Therefore, it is unclear whether the method really improves the connection between perception and reasoning, or simply changes the advantage weights based on probability and entropy.

  3. The performance improvements are limited and inconsistent. On Qwen3-VL-8B, PR-FAPO improves Video-MMMU from 65.0 to 67.6, but the gains on MMVU (71.1 to 72.2) and MVP (43.0 to 43.7) are small. Performance also drops on MVBench and LVBench. For temporal grounding, Charades-STA improves only from 63.7 to 64.0, while ActivityNet remains unchanged at 51.9. Considering the complexity of the method, these results do not provide strong evidence for its effectiveness.

Questions for authors

see weakness

Reviewer_dXayOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies credit assignment in video RLVR. The authors argue that GRPO spreads one trajectory-level advantage evenly over all tokens. It therefore cannot tell apart tokens that acquire visual information from tokens that reason over it. The method, PR-FAPO, builds on the answer→think→answer framework of VideoAuto-R1. It splits the CoT into a <perceive> prefix and a <think> suffix. At several window scales (τ = 4^k), it aggregates prefix and suffix probability and entropy statistics into a "perception–reasoning fusion gap" V_p. A TD/GAE-style recursion turns V_p into a token-level multiplier f_{i,t}, which scales the GRPO advantage.

Critical strengths
  1. The method needs no extra annotation and leaves the reward unchanged. It only modifies the advantage. It is plug-in in principle and could transfer to other GRPO-style algorithms.

  2. The authors report gains over VideoAuto-R1 on reasoning benchmarks (Video-MMMU, MMVU, MVP, SVBench). Temporal grounding is roughly on par. The authors also claim more "structured" training dynamics and better stability.

  3. The ablations are fairly complete. Tables 8 and 9 separate prefix/suffix anchors, token-level vs. segment-level shaping, discounting, and multi-scale windows. The trends are consistent, so each design choice appears to matter.

Critical weaknesses
  1. The causal attribution of the core observation is weak: the "video vs. image-text" contrast is heavily confounded. The narrative starts from the claim that video RLVR has different entropy dynamics from image-text RLVR. The evidence is a comparison between Video-R1 (T-GRPO, length reward, think→answer) and VideoAuto-R1 (answer→think→answer, consistency reward scaling). These two systems differ in both algorithm and reward design. VideoAuto-R1's collapse to low entropy and short CoT has a more natural explanation in its reward structure. The direct answer comes first, and the reviewed answer only needs to be no worse than it, so long CoT is less necessary. This need not have anything to do with the video modality. Without a video vs. image comparison under the same algorithm and reward, the claim that "video sustains correct trajectories across high probability–entropy bands" cannot be attributed to video.

  2. I may miss some deductions, but I doubt that the theory is disconnected from the method, and the derivation has serious problems.

  • The decomposition in Eq. 4 is formal. The policy only applies a softmax over y_t. The "perceptual latent z" is never separately modeled or sampled. So ln π(y,z|s) = ln π(z|s) + ln π(y|s,z) has no operational meaning in this model. Eq. 8 then treats z as a discrete variable sharing the same logit vector a ∈ R^|V| as y. The set O is described as a "temporal-centric reference set", but the paper never explains how it is constructed.
  • Prop. 3.2 is an upper/lower bound on the gradient norm. It does not imply the position-dependent conclusion "retain entropy early, reduce it late", nor does it yield the specific form of V_p. From Eq. 11–12 to Eq. 17–20 there is a clear heuristic jump, which the paper presents as a "derivation".
  • The use of Lemma 3.1 is hand-waving. The chain "action-uniform logit changes → small covariance → high-entropy latent states persist" has no quantitative support.
  1. The gains are overstated, and the story does not fully match the main claim. The parenthesized deltas in the table are relative to the base model, not to VideoAuto-R1. This invites readers to treat +6.6 as a gain over VideoAuto-R1, which is misleading. The only real highlight is Video-MMMU. On temporal grounding (the RL rows of Tables 2 and 7), the difference from VideoAuto-R1 is about ±0.3 mIoU. ActivityNet at 8B is identical (51.9). The abstract's "experiments on reasoning-centric and temporal-grounding benchmarks validate the effectiveness" is too strong. Besides, Table 5 (image and text math) is mixed. Geo3K is +6.7, but DynaMath is −2.9, WeMath −2.5, MathVista −1.8, and AIME24 −2.9. Three of the eight image benchmarks drop noticeably, which conflicts with the text's "consistently outperforms across most image reasoning benchmarks".
Questions for authors

To be frank, this year’s four-point scoring system makes the rebuttal phase quite difficult. If a more granular range of scores were available during the rebuttal, I would be happy to engage in discussions with the authors and potentially adjust my score upon seeing new experimental results. Given the current system, however, I am compelled to set my expectations higher.

  1. Please train on video data and on image data under the same algorithm and reward (for example, both with Video-R1's reward, or both with VideoAuto-R1's reward), and show the entropy–probability dynamics. Can the authors rule out that the retained entropy comes from T-GRPO, the length reward, or the KL coefficient?

  2. Does the "high-probability, high-entropy band" in Fig. 1 persist after bucketing by response length? What is the correlation between trajectory-level probability and length?

  3. Does VideoAuto-R1's entropy collapse come from its "reviewed answer ≥ direct answer" reward structure? If that reward is replaced, does the collapse still occur?

  4. There seems to be no evidence that the model actually "fuses" visual information. Please run perturbation experiments: replace, truncate, or shuffle the <perceive> content, and compare the answer sensitivity of PR-FAPO and VideoAuto-R1. Can you evaluate the agreement between <perceive> content and annotated evidence on NExT-GQA?

    1. Why are the ablated variants in Tables 8 and 9 worse than both the base model and VideoAuto-R1? Please add a VideoAuto-R1 row at the same number of steps, and state whether the ablated runs converged stably.
  5. Please add the "VideoAuto-R1 + same <perceive>/<think> prompt" control to isolate the effect of the prompt structure.

Reviewer_g7SWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes PR-FAPO for video reasoning. Its motivation is that GRPO applies a shared outcome-derived advantage across a response, providing insufficient discrimination between useful visual observations, irrelevant content, and subsequent reasoning. The authors analyze probability–entropy dynamics and argue that successful video reasoning benefits from preserving uncertainty during perceptual evidence accumulation while reducing it during reasoning. PR-FAPO structures responses into perception and reasoning segments, constructs a “fusion gap” from prefix–suffix probability and entropy statistics. Evaluated on several video QA benchmarks, Video-PR-FAPO shows promising performance.

Critical strengths
  • The model shows promising performance improvements against existing video MLLMs over seven video QA benchmarks, especially for reasoning-centric tasks.
  • The paper offers a rigorous analysis regarding video GRPO and identify a broad high-probability, high-entropy regime in video RLVR.
Critical weaknesses
  • The main experiments use Qwen3-VL-8B, whereas most competing methods reported in Table 1 use Qwen2.5-VL backbones. These comparisons therefore conflate improvements from the proposed method with differences in backbone capabilities, making it difficult to attribute the gains to PR-FAPO itself. Although the matched-backbone comparison with VideoAuto-R1 is informative, comparisons with additional methods under the same backbone, training data, and computational budget are needed.
  • The whole paper is overly dense and lacks clarity. Particularly, the main paper is poorly presented and not self-contained, since many important material including related work and key ablation study are relegated to the appendix. This organization substantially reduces readability.
  • Probability and entropy statistics measure model confidence, but do not directly establish that reasoning uses correct visual evidence. The component ablations show that particular design choices matter within PR-FAPO, yet do not establish that its gains specifically arise from perception–reasoning fusion. A clearly documented comparison using the same explicit perception/reasoning prompt with ordinary GRPO, together with a simpler entropy- or position-based weighting baseline, is needed. Appendix C’s token-masking analysis only partially addresses this. This distinction matters because temporal-grounding gains over VideoAuto-R1 are limited.
Questions for authors
  • Do the VideoAuto-R1 comparisons use the identical <perceive>/<think> prompt?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.