- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FrameGRPO varies frame-sampling density across rollouts for the same question. Two of eight rollouts use the full view; the others use uniformly subsampled views. Correct sparse-view answers receive extra credit only when both full-view answers fail. All answers are then optimized under the full-view input. Experiments on two backbones report generally improved performance over GRPO without an additional inference-time module.
Critical strengths
- Low integration cost. The method reuses one decoded frame sequence and requires no architecture changes. Table 9 reports lower peak GPU memory and training time comparable to GRPO on Qwen2.5-VL, with some overhead on Qwen2-VL. No extra module is needed at inference.
- Controlled comparisons and repeated runs. The evaluation includes same-data GRPO baselines, two backbones, and three seeds on Qwen2.5-VL (Tables 1 and 5). Repeated improvements on several benchmarks strengthen the empirical case.
- Evidence across data budgets. Table 6 compares GRPO and FrameGRPO with 2K, 4K, and 8K training examples. Improvements on several tasks at each scale suggest that the benefit is not confined to one training-set size, although it is not universal across benchmarks.
Critical weaknesses
- Cross-view optimization needs justification. Answers are sampled under a sparse view, but both terms of the likelihood ratio in Eq. 5 condition on the full view. This is not an importance ratio relative to the actual sampling distribution. Table 3 supports the choice empirically, but the implications of applying sparse-view successes and failures to the full-view policy remain unclear.
- Reward diversity does not necessarily imply useful exploration. A lower all-correct rate can arise because sparse views lose necessary evidence. The gating ablations are informative, but aggregate curves do not quantify how often sparse views reveal additional successes rather than merely introduce failures. This distinction matters to the proposed mechanism.
Questions for authors
- How should Eq. 5 be interpreted under the change in visual conditioning? How large is the old-policy log-likelihood difference between sampled and full views, especially at low sampling ratios?
- Can existing logs report gate frequency and correctness by sampling ratio, including groups where only sparse-view answers succeed versus only full-view answers succeed?
- What frame counts, sampling rules, pixel budgets, and decoding settings are used at evaluation? Are they matched between GRPO and FrameGRPO, and how do they relate to the 32-frame training view?
5 comments
Response to Reviewer fhZW (Part 2/2)
W2 & Q2: Useful exploration and outcomes by sampling ratio
We agree that increased reward diversity alone does not establish useful exploration. Noise perturbation degrades visual appearance, while frame shuffling disrupts temporal order; both can make reasoning more difficult. In contrast, changing the sampling ratio affects reasoning non-monotonically: fewer frames can remove distractions, while more frames can recover missing evidence.
Beyond the lower all-correct rate, the all-incorrect group rate decreases from 11.8% with GRPO to 5.6% with FrameGRPO, consistent with Figure 1. Thus, greater reward diversity coincides with fewer all-incorrect groups. By exposing complementary temporal evidence, frame sampling helps uncover successful answers in cases where rollouts sharing a fixed view all fail, alleviating the loss of learning signals from uniformly incorrect groups.
We further analyzed accuracy by sampling ratio and the complementary success patterns. Accuracy is measured over individual rollouts within each ratio bin:
| Sampling ratio | Accuracy |
|---|---|
| (75%, 100%] | 77.1% |
| (50%, 75%] | 75.8% |
| (25%, 50%] | 73.9% |
| (0%, 25%] | 69.9% |
The following frequencies are measured over all FrameGRPO rollout groups:
| Group-level event | Frequency |
|---|---|
| Gate active: all full-view reference rollouts fail | 15.5% |
| Only sparse-view answers succeed | 10.0% |
| Only full-view answers succeed | 0.9% |
“Only sparse-view answers succeed” means that at least one sparse-view answer is correct while all full-view reference answers are incorrect; “only full-view answers succeed” denotes the converse.
Although accuracy decreases on average at lower sampling ratios, 10.0% of groups contain successful sparse-view trajectories absent from the full-view reference outcomes. This directly demonstrates complementary discoveries beyond merely introducing additional failures. TEC selectively reinforces these verified successes, turning the additional exploration into useful learning signals.
Q3: Evaluation settings and their relationship to training
GRPO and FrameGRPO use identical evaluation settings for each backbone across all seven benchmarks, including frame sampling, pixel budgets, prompts, and decoding parameters. We follow Video-R1 [1] and use the decoding configuration from the official Qwen2.5-VL demo.
| Setting | Evaluation configuration |
|---|---|
| Frame count | Target 2 FPS; 4–64 frames, subject to available source frames, rounded down to an even count. |
| Sampling | Uniformly spaced indices across the entire video, rounded to the nearest integer; temporal order preserved. |
| Pixel budget | 100,352–200,704 pixels per frame; aspect ratio approximately preserved, with dimensions aligned to multiples of 28. |
| Decoding | Temperature 0.01; top-p 0.001; maximum 1,024 generated tokens; one response per example. |
| Context limit | 16,384 tokens. |
For longer videos, the 64-frame cap lowers the effective sampling rate, while the selected frames still span the entire video.
The 32-frame setting refers to the full-ratio view during training. At evaluation, both methods use the same protocol above, with up to 64 frames. Each example uses one video view and one generated response. Multi-Frame Rollouts and TEC operate only during training, so FrameGRPO receives the same evaluation input and decoding budget as GRPO, with no additional inference overhead.
Reference
[1] Feng et al. Video-R1: Reinforcing Video Reasoning in MLLMs. NeurIPS 2025.
Dear Reviewer fhZW,
With the rebuttal period ending in less than 24 hours, we would appreciate your feedback on our response. We have added experiments and analyses on cross-view likelihood gaps by sampling ratio, gate frequency, and complementary sparse-view successes, and clarified the off-policy interpretation of Eq. 5 and the matched evaluation settings.
We hope these additional results and clarifications address your concerns. Please let us know if any questions remain; we would be happy to discuss them further. Thank you for your time and consideration.
Best regards,
The Authors
Response to Reviewer fhZW (Part 1/2)
We sincerely thank the reviewer for recognizing FrameGRPO’s low integration cost, absence of additional inference modules, controlled comparisons and repeated improvements, and effectiveness across different training-data budgets. We appreciate the constructive questions and provide additional measurements and clarifications below.
W1 & Q1: Cross-view optimization and likelihood differences
Thank you for highlighting this distinction. Eq. 5 is an off-policy trajectory-transfer surrogate: the behavior policy generates trajectories under sampled temporal views, while the target policy learns from them under the full view. The likelihood ratio measures policy change relative to the old full-view policy, rather than correcting for the sampling probability under each generating view. Group-relative advantages determine whether a sampled trajectory is reinforced or suppressed under the full context, while TEC selectively adds credit to successful sparse-view trajectories. Recent studies establish an off-policy interpretation of group-relative REINFORCE [1] and demonstrate the effectiveness of off-policy learning in language reasoning [2] and image reasoning [3].
To quantify the difference between the generating and optimization views, we evaluate the same trajectory under both views using the same old policy: Di=logπold(oi∣Vρi,qi)−logπold(oi∣V1,qi). Here, the first term uses the sampled view that generated the trajectory, and the second uses the full view. A positive gap indicates that the trajectory is more likely under its generating view. The mean gaps by sampling ratio are:
We also compare frame sampling with noise perturbation following NoisyRollout [3], and frame shuffling, as used in Video-R1 [4]:
Frame sampling yields the lowest average gap, supporting its suitability for transferring exploratory trajectories back to the original view. These measurements complement Table 5’s empirical evidence for full-view optimization.
References
[1] Yao et al. Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends. ICLR 2026.
[2] Yan et al. Learning to Reason under Off-Policy Guidance. NeurIPS 2025.
[3] Liu et al. NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation. NeurIPS 2025.
[4] Feng et al. Video-R1: Reinforcing Video Reasoning in MLLMs. NeurIPS 2025.