acceptodds
Under review as a conference paper at ICLR 2027

Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning

Abstract

Video large language models (Video LLMs) can achieve strong video-QA accuracy without reliably tracking spatiotemporal dynamics. A model may answer a motion question from static cues, for example, and give the same prediction even after the underlying motion is reversed. Correctness-based reinforcement learning does not directly address this problem because it rewards the final answer without requiring the policy to respond to task-relevant video dynamics. We propose Counterfactual Relational Policy Optimization (CRPO), which explicitly trains Video LLMs to respond to controlled changes in the visual input. For each training example, CRPO constructs a counterfactual video, such as a horizontally flipped or temporally reversed version of the original, and jointly optimizes rollouts from both videos under a shared policy. Factual supervision anchors what the model should answer, while counterfactual supervision constrains when that answer should change: predictions should change when an intervention alters task-relevant dynamics and remain stable when the queried property is preserved. This coupling provides a direct behavioral learning signal without requiring ground-truth labels for transformed videos or annotated spatiotemporal reasoning traces, while discouraging indiscriminate answer changes. To evaluate this property, we introduce DyBench, a paired counterfactual benchmark with 3,014 videos and a strict pair-accuracy metric. Across paired spatiotemporal evaluations and standard video benchmarks, CRPO improves sensitivity to motion and temporal changes while improving performance on general video understanding. The gains also extend to segment reordering, a transformation never used during training, suggesting that CRPO learns sensitivity to video dynamics beyond the training interventions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video LLM, Reinforcement Learning, MLLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_g6enOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates spatiotemporal sensitivity as a counterfactual relational learning problem and proposes Counterfactual Relational Policy Optimization (CRPO), a dual-branch reinforcement learning framework. CRPO uses data augmentation to construct relational supervision, training models to answer both original video questions and auxiliary relation questions. This aims to improve sensitivity to spatiotemporal changes while preserving general video understanding. The paper also introduces DyBench, a paired diagnostic benchmark, and evaluates CRPO under controlled and out-of-distribution settings.

Critical strengths
  1. The paper addresses Video LLMs’ limited sensitivity to temporal order and motion dynamics by framing spatiotemporal sensitivity as a counterfactual relational learning problem.

  2. It proposes Counterfactual Relational Policy Optimization (CRPO), a dual-branch RL framework that uses data augmentation to jointly train models on original video questions and auxiliary relation questions, improving spatiotemporal sensitivity while preserving general video understanding.

  3. It introduces DyBench, a paired diagnostic benchmark. Controlled and out-of-distribution evaluations show that auxiliary tasks involving reversible dynamics, motion direction, and event order consistently improve spatiotemporal sensitivity in both Qwen3-VL-4B and Qwen3-VL-8B.

Critical weaknesses
  1. For dynamic questions, the counterfactual branch reward gives full credit to any response that differs from the original factual label. Rewarding any deviation does not encourage the policy to model the specific counterfactual physics or semantics—it simply rewards departing from the factual label. The authors argue that factual supervision grounds the policy, but might rewarding arbitrary alternative predictions risk encouraging unprincipled hallucinations or random drift under visual perturbations?

  2. Could you clarify the train–test mismatch: the model is trained with a fifth option ("None of the above") but evaluated on standard benchmarks without it?

  3. The paper claims that prior perturbation-based RL methods such as T-GRPO and ArrowRL suffer from "reward cancellation" under group-wise advantage normalization. In ArrowRL, however, the reverse reward is rollout-dependent rather than constant across the group, so its auxiliary signal does not cancel out under normalization.

  4. Although performance on spatiotemporal benchmarks improves, the model performs on par with or worse than T-GRPO on general tasks such as VideoMME and MVBench. This raises the biggest concern: auxiliary tasks should help, or at least not hurt, performance on the main task.

Questions for authors

I would appreciate it if the authors could kindly address the concerns raised in the Critical Weaknesses section.

8 comments
AuthorsOpenReview ↗

Author Responses (1/4)

Arbitrary answer changes, hallucinations, and counterfactual correctness (Weakness 1)

Thank you for this thoughtful concern about counterfactual semantics and possible arbitrary answer changes. We agree that the dynamic behavioral reward alone does not specify the exact counterfactual physics or semantics. We appreciate your emphasis on validating actual correctness. CRPO combines this relational constraint with factual supervision through a shared policy and invariance supervision on static examples. Rather than infer correctness from the behavioral reward, we now measure it directly through a manual counterfactual-answer audit.

At 5%, 50%, and 95% of the total training steps, we randomly sampled 64 counterfactual video QA examples from one global batch across all devices. We manually annotated answers by inspecting the transformed videos and their questions and choices, annotating “None of the above” (the Null Option defined in the paper) as correct when none of the original answer choices was correct for the transformed video. We scored all eight counterfactual rollouts per example, giving 512 rollouts per stage. These labels are used only for analysis and never enter training.

Training stageDynamic reward precision (%)Null precision (%)Counterfactual accuracy (%)Dynamic samples (%)
Early (5%)71.766.239.378.1
Middle (50%)84.779.756.484.4
Late (95%)89.989.863.779.7

The metrics have the following meanings:

  • Dynamic reward precision: the proportion of rewarded dynamic answers that match the manually annotated counterfactual label.
  • Null precision: the proportion of “None of the above” selections that are correct.
  • Counterfactual accuracy: accuracy over all audited counterfactual rollouts, including dynamic answers that fail to change.
  • Dynamic samples: the percentage of sampled examples routed as dynamic.

All rollout-based metrics pool the corresponding counts across the sampled groups.

Dynamic reward precision rises from 71.7% to 89.9%, showing that approximately nine out of ten rewarded dynamic answers are exactly correct in the late-stage audit. Null precision reaches 89.8%, directly confirming that the added option is used appropriately in most audited selections.

Counterfactual accuracy increases from 39.3% to 63.7%, a gain of 24.4 percentage points. This includes all audited rollouts, including dynamic answers that fail to change. Together, the results show higher exact counterfactual correctness alongside increasingly reliable rewarded behavior.

Exact counterfactual answers are not provided by the existing training data, and obtaining them at scale would require additional annotation. CRPO derives a simple change-or-invariance signal from the transformation and the existing factual labels. The audit and paired benchmark results show that this simple relational signal, coupled with factual grounding, supports accurate counterfactual predictions without any additional counterfactual answer annotations for RL training. The manual labels are used only to assess the trained policy.

Manuscript changes: We added “Reliability of the Counterfactual Relation Reward” in Section 5.3, with the audit protocol, metric definitions, analysis, and results in Table 4. These details are consolidated in the main text. Appendix D.1 distinguishes Null usage frequency from the correctness of individual selections.

AuthorsOpenReview ↗

Author Responses (2/4)

Training-time Null Option and evaluation choices (Weakness 2)

Thank you for raising the training–evaluation choice mismatch. We appreciate the opportunity to clarify why the option is introduced and to test whether it affects evaluation. The added Null Option provides a valid answer when a training transformation makes the original candidate set incomplete. For example, suppose a ball moves left and the choices are “left,” “up,” “down,” and “stationary.” A horizontal flip makes it move right, which none of those choices expresses. “None of the above” makes the transformed question answerable without assigning an incorrect original option.

During training, we transform videos while retaining their original questions and choices, so an additional option is needed to cover answers that the transformations move outside that choice set. It is included in the shared choices of both branches; the factual ground-truth label remains unchanged, and the added option is always incorrect on the factual branch.

Adding an extra “None of the above” option at benchmark evaluation is unnecessary, because the provided choices already include the annotated correct answer. This also holds for DyBench: each side of a pair has a verified correct answer among its provided choices. The added option therefore does not fill any gap in the candidate set. We retain each benchmark's original choices and evaluation protocol, including any none-like choices already present.

To directly address your concern, we also tested whether providing the additional option changes performance. We appended “None of the above” to every DyBench question and evaluated the same Qwen3-VL-4B base and CRPO checkpoints:

ModelAnswer choicesAcc (%)O-Acc (%)P-Acc (%)
BaseOriginal65.176.245.4
BaseOriginal + Null65.076.245.2
CRPOOriginal70.379.554.8
CRPOOriginal + Null70.179.354.5

All score changes are at most 0.3 percentage points. CRPO's gains over Base are 5.2/3.3/9.4 points in Acc/O-Acc/P-Acc with the original choices and 5.1/3.1/9.3 with the added option. Thus, the additional option has a negligible effect in this evaluation, and CRPO's improvement remains nearly unchanged. The option addresses incomplete candidate sets created by training transformations; the observed DyBench gains do not depend on its omission at evaluation.

Manuscript changes: We clarified the training/default-evaluation option protocol and why an additional Null Option is unnecessary for benchmark evaluation in Appendix D.1. The additional-Null evaluation is reported in Appendix D.1, Table 12.

AuthorsOpenReview ↗

Author Responses (3/4)

ArrowRL and reward cancellation (Weakness 3)

We are grateful for this careful correction. You are right that ArrowRL's reverse reward is rollout-dependent and does not generally cancel under group-wise normalization. We apologize for the overly broad attribution in the original manuscript. Your observation has helped us correct the analysis and clarify the distinction between the methods.

ArrowRL Section 3.2 defines the reverse reward as

\qquad r_i=r_i^{\mathrm{fid}}+\alpha r_i^{\mathrm{rev}},$$ where the current policy generates the reverse-conditioned response $\tilde{o}$, and the reverse term is disabled when that response is sufficiently similar to the factual target. Because the reverse reward depends on each factual rollout, it can change relative rewards within the group. For a simple MCQ example, let the factual target be A, the reverse-conditioned prediction be B, and three factual rollouts answer A, B, and C. With the reverse term active, their rewards are $1 + \alpha$, $0$, and $\alpha$. The two incorrect answers receive different rewards, so the result is not a common rescaling of binary correctness and the auxiliary signal survives normalization. The cancellation derivation applies to the narrower case in which the shaped reward is a common positive rescaling of correctness, $R_i = (1 + C)c_i$, ignoring the format term and a negligible numerical stabilizer. T-GRPO's correctness-gated temporal bonus has this structure. ArrowRL can also exhibit cancellation in particular groups: if every factual answer differs from B, its reverse reward is constant and is removed by group centering. If the group contains only A and B, the total reward becomes $(1 + \alpha)c_i$, so the common factor cancels. **These are conditional cases, not a general property of ArrowRL.** We also removed the interpretation that ArrowRL's low, flat raw auxiliary-reward curve demonstrates normalization-induced cancellation. For our MCQ ArrowRL baseline, Similarity is exact answer matching, following ArrowRL's MCQ definition. The reference answer is generated by the current policy from the reversed video. No ground-truth answer for the reversed video or LLM similarity judge is needed. This implementation still produces rollout-dependent reverse rewards. The distinction we retain is about **which responses are used for training and how their rewards are normalized**. ArrowRL uses the reverse-video response to compute rewards for original-video responses. CRPO trains on responses to both the original and transformed videos. **CRPO uses a shared standard deviation for both branches.** Normalizing each branch separately would cancel its multiplicative CRR factor. The shared denominator preserves the branches' relative reward scales, allowing CRR to influence their relative contributions to optimization. We have revised Section 3.4 to explain both mechanisms without attributing generic reward cancellation to ArrowRL. **Manuscript changes:** We specified the common-factor condition in **Section 1**, narrowed the transition in **Section 3.3**, and revised **Section 3.4** to explain both the cancellation condition and its limited applicability to ArrowRL. **Appendix D.3** now gives ArrowRL's reward formula and MCQ examples with and without cancellation. We clarified our MCQ implementation in the ArrowRL description in **Appendix B.3** and removed the normalization-based interpretation of the raw-reward curves in **Section 5.3**. The experiments and reported baseline scores are unchanged.
Reviewer_hTZLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CRPO to improve the spatiotemporal sensitivity of video LLMs. It jointly trains on original videos and transformed videos with horizontal flip or temporal reversal. The original branch uses answer correctness and the transformed branch rewards expected answer changes or invariance. This paper also introduces DyBench, a dataset contains 1507 video pairs. Experiments on Qwen3-VL show improvements on pair accuracy.

Critical strengths
  1. The pair accuracy evaluation matches the motivation well. On the 4B model, DyBench pair accuracy reaches 54.8%, which is the best among all methods. Gains also appear on EventSequence segment reordering, which is an OOD transformation.

  2. The controls are useful. Doubling GRPO rollouts or data does not recover the same gain. The paper also provides component ablations, a manual router check, routing-noise experiments and tests with disrupted visual inputs.

Critical weaknesses
  1. A changed answer is not necessarily a correct counterfactual answer. For dynamic questions, any answer different from the original label can receive the behavioral reward, including a wrong option or Null Option. Shared training and static examples help, but do not remove this ambiguity.
Questions for authors
  1. CLEVRER is a source of TVBench. Does the filtered training set include CLEVRER videos, and how are source videos and near-duplicates checked against the evaluation sets?
5 comments
AuthorsOpenReview ↗

Author Responses (1/2)

Does rewarding an answer change encourage correct counterfactual answers? (Weakness 1)

Thank you for this important observation. We agree that the behavioral reward does not uniquely identify the correct counterfactual answer, and we appreciate your request for direct evidence of correctness. It provides a relational constraint, while factual supervision and static examples constrain the shared policy. To directly assess the correctness of the resulting predictions, we added a manual audit of counterfactual training examples.

At 5%, 50%, and 95% of the total training steps, we randomly sampled 64 counterfactual video QA examples from one global batch across all devices. We inspected each transformed video and its question and choices to annotate the correct answer. When none of the original answer choices was correct for the transformed video, we annotated “None of the above” (the Null Option defined in the paper) as the correct answer. All eight counterfactual rollouts per example were scored, giving 512 rollouts per stage. These labels are used only for analysis and never enter training.

Training stageDynamic reward precision (%)Null precision (%)Counterfactual accuracy (%)Dynamic samples (%)
Early (5%)71.766.239.378.1
Middle (50%)84.779.756.484.4
Late (95%)89.989.863.779.7

The metrics have the following meanings:

  • Dynamic reward precision: the proportion of rewarded dynamic answers that match the manually annotated counterfactual label.
  • Null precision: the proportion of “None of the above” selections that are correct.
  • Counterfactual accuracy: accuracy over all audited counterfactual rollouts, including dynamic answers that fail to change.
  • Dynamic samples: the percentage of sampled examples routed as dynamic.

All rollout-based metrics pool the corresponding counts across the sampled groups.

Dynamic reward precision rises from 71.7% to 89.9%, showing that approximately nine out of ten rewarded dynamic answers are exactly correct in the late-stage audit. Null precision reaches 89.8%, directly confirming that the added option is used appropriately in most audited selections.

Counterfactual accuracy increases from 39.3% to 63.7%, a gain of 24.4 percentage points. This includes all audited rollouts, including dynamic answers that fail to change. Together, the results show higher exact counterfactual correctness alongside increasingly reliable rewarded behavior.

Exact counterfactual answers are not provided by the existing training data, and obtaining them at scale would require additional annotation. CRPO derives a simple change-or-invariance signal from the transformation and the existing factual labels. The audit and paired benchmark results show that this simple relational signal, coupled with factual grounding, supports accurate counterfactual predictions without any additional counterfactual answer annotations for RL training. The manual labels are used only to assess the trained policy.

Manuscript changes: We added “Reliability of the Counterfactual Relation Reward” in Section 5.3, with the audit protocol, metric definitions, analysis, and results in Table 4. These details are consolidated in the main text. Appendix D.1 distinguishes Null usage frequency from the correctness of individual selections.

AuthorsOpenReview ↗

Author Responses (2/2)

CLEVRER training videos and source-video overlap (Question 1)

Thank you for carefully checking the training and evaluation sources. We appreciate the opportunity to clarify a point that was not sufficiently explicit in the original manuscript. The filtered TVBench training subset retains 172 CLEVRER videos. We checked all 172 source-video IDs against the CLEVRER IDs in the DyBench evaluation set used for the reported experiments and found zero overlapping IDs.

CLEVRER filenames follow video_xxxxx, where the numeric suffix identifies the generated source video. We extract this source ID and map flipped or reversed variants back to the same original ID before comparison. This comparison operates at the source-video level and includes all transformed variants of the same underlying video. The retained CLEVRER training videos and the CLEVRER evaluation videos are distinct source videos.

Our training corpus uses the filtered training dataset released by VideoAuto-R1 (CVPR 2026), which excludes benchmark test samples from its training data. We inherit that filtering and additionally perform the DyBench-specific source-ID comparison above. CLEVRER is therefore a shared underlying dataset, while the individual training and evaluation source videos are disjoint.

Manuscript changes: We added the retained CLEVRER count, source-ID extraction and variant mapping procedure, and the zero-overlap result in Appendix A.1, “Source-video separation.”

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer hTZL, as the discussion period draws to a close, we kindly follow up on our responses, particularly the counterfactual reward audit and source-video overlap verification. We hope these additions address your concerns and would welcome any further questions or suggestions. Thank you again for your time and constructive feedback.

Reviewer_c7eCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CRPO trains Video LLMs to respond to controlled video changes: answers should change when an intervention alters task-relevant dynamics and stay stable when the queried property is preserved. A factual branch uses ground-truth answers, a counterfactual branch (horizontal flip or temporal reversal) uses relational rewards, and both directly update a shared policy. The paper also introduces DyBench, a paired benchmark with a strict pair-accuracy metric, and evaluates on five benchmarks with controls and ablations.

Critical strengths
  1. The training objective directly matches the problem, and the paired results are clear. CRPO beats T-GRPO on DyBench P-Acc by 5.0 points on 4B and 3.6 on 8B, and on TimeBlind I-Acc by 2.7 and 3.5 (Table 1). TimeBlind is an existing benchmark, so the evidence does not rest only on the authors' DyBench.

  2. The controls are strong. Doubling rollouts or data reaches only 50.2 and 48.4 P-Acc versus 54.8 for CRPO (Table 2), so the gain is not simply extra sampling. Removing the counterfactual branch drops P-Acc to 48.8. Event Sequence, built from a transformation not used in training, also improves (Table 5).

Critical weaknesses
  1. A changed answer is not a correct answer. On dynamic questions, the behavioral reward b_j = 1[a_j != y] pays for any different answer, including a wrong option or the Null Option. A Null Option rate near 25% rules out always choosing Null, but does not show that each choice is appropriate. There is no audit with exact counterfactual labels, which limits how far the relational reward can be trusted.

  2. The gains are uneven and the claims should be narrower. T-GRPO is slightly ahead on Moving Direction P-Acc (21.8 vs 21.0 on 4B, 20.0 vs 19.8 on 8B). On 4B, CRPO is below T-GRPO on VideoMME (63.0 vs 63.4) and MVBench (68.6 vs 70.3). No multi-seed results are reported, so the stability of the smaller differences is unknown.

Questions for authors
  1. On a small audit set with exact counterfactual labels, what are the rates of wrong answer changes and of appropriate Null Option use?
  2. Can you report results over multiple random seeds?
  3. Horizontal flips can also change single-frame spatial cues on some direction questions. Can you narrow the claim that paired evaluation makes static shortcuts ineffective, and rely on the measured shortcut-isolation results instead?
4 comments
AuthorsOpenReview ↗

Author Responses (1/3)

Exact counterfactual correctness and appropriate Null use (Weakness 1; Question 1)

Thank you for suggesting this audit and for highlighting the distinction between an answer change and a correct counterfactual answer. We agree with your concern, and your suggestion has helped us directly assess reward reliability and appropriate Null use. The behavioral reward specifies the expected relation to the factual answer rather than an exact counterfactual label. We therefore added a manual audit that directly measures both answer correctness and the correctness of rewarded dynamic changes.

At 5%, 50%, and 95% of the total training steps, we randomly sampled 64 counterfactual video QA examples from one global batch across all devices. We manually annotated each answer by inspecting the transformed video and its question and choices. When none of the original answer choices was correct for the transformed video, we annotated “None of the above” (the Null Option defined in the paper) as the correct answer. We evaluated all eight counterfactual rollouts per example, giving 512 rollouts per stage. These annotations are used only for analysis and never enter training.

Training stageDynamic reward precision (%)Null precision (%)Counterfactual accuracy (%)Dynamic samples (%)
Early (5%)71.766.239.378.1
Middle (50%)84.779.756.484.4
Late (95%)89.989.863.779.7

The metrics have the following meanings:

  • Dynamic reward precision: the proportion of rewarded dynamic answers that match the manually annotated counterfactual label.
  • Null precision: the proportion of “None of the above” selections that are correct.
  • Counterfactual accuracy: accuracy over all audited counterfactual rollouts, including dynamic answers that fail to change.
  • Dynamic samples: the percentage of sampled examples routed as dynamic.

All rollout-based metrics pool the corresponding counts across the sampled groups.

Dynamic reward precision rises from 71.7% to 89.9%, showing that approximately nine out of ten rewarded dynamic answers are exactly correct in the late-stage audit. Null precision reaches 89.8%, directly confirming that the added option is used appropriately in most audited selections. To answer your question directly, wrong rewarded dynamic changes decrease from 28.3% to 15.3% to 10.1% across the three stages.

Counterfactual accuracy increases from 39.3% to 63.7%, a gain of 24.4 percentage points. This includes all audited rollouts, including dynamic answers that fail to change. Together, the results show higher exact counterfactual correctness alongside increasingly reliable rewarded behavior.

Exact counterfactual answers are not provided by the existing training data, and obtaining them at scale would require additional annotation. CRPO derives a simple change-or-invariance signal from the transformation and the existing factual labels. The audit and paired benchmark results show that this simple relational signal, coupled with factual grounding, supports accurate counterfactual predictions without any additional counterfactual answer annotations for RL training. The manual labels are used only to assess the trained policy.

Manuscript changes: We added “Reliability of the Counterfactual Relation Reward” in Section 5.3, with the audit protocol, metric definitions, analysis, and results in Table 4. These details are consolidated in the main text. Appendix D.1 distinguishes Null usage frequency from the correctness of individual selections.

AuthorsOpenReview ↗

Author Responses (2/3)

Training variability and performance across tasks and model sizes (Weakness 2; Question 2)

Thank you for requesting multiple training seeds and for identifying the task-level differences that deserve closer examination. Following your suggestion, we added two training runs with different seeds for each 4B method, giving three independent runs per method including the original, and two additional benchmarks at both 4B and 8B. These assess the stability of the paired gains and broaden the evidence for general video understanding.

Independent training seeds. For each of T-GRPO and CRPO on Qwen3-VL-4B, we added two training runs with different seeds to the originally reported run, giving three independent runs per method. Data, budget, and hyperparameters are fixed within each method. Evaluation uses fixed, uniformly sampled frames and greedy decoding, so changing only inference seeds would not assess training variability. Given rebuttal time and compute constraints, we focus on these two methods at 4B and evaluate DyBench, VideoMME, and MVBench. Entries are mean ± sample standard deviation in percentage points; Base is the fixed pretrained checkpoint.

Dataset / taskMetricBaseT-GRPOCRPO
DyBench overallAcc65.168.2 ± 0.470.4 ± 0.1
DyBench overallP-Acc45.449.9 ± 0.354.7 ± 0.1
Reversible DynamicsAcc78.982.1 ± 0.386.4 ± 0.4
Reversible DynamicsP-Acc64.368.5 ± 0.677.2 ± 0.4
Moving DirectionAcc40.143.2 ± 1.342.8 ± 0.9
Moving DirectionP-Acc18.021.3 ± 0.521.4 ± 0.5
Event SequenceAcc75.478.4 ± 0.480.8 ± 0.4
Event SequenceP-Acc51.157.7 ± 0.662.5 ± 0.5
VideoMMEAcc62.263.1 ± 0.363.4 ± 0.4
MVBenchAcc67.570.0 ± 0.269.2 ± 0.6

CRPO exceeds T-GRPO on overall DyBench P-Acc in every run, with a 4.8-point mean advantage. It improves both Acc and P-Acc on Reversible Dynamics and Event Sequence in every run. The small mean differences on Moving Direction P-Acc and VideoMME are within the observed variability; T-GRPO retains higher mean Moving Direction Acc and MVBench accuracy. Relative to Base, CRPO improves VideoMME and MVBench by 1.2 and 1.7 points, respectively.

Two additional benchmarks at both model sizes. We added AoT-VQA from AoTBench, testing temporal direction understanding, and MMVU MCQ, testing general, expert-level multidisciplinary video understanding. We evaluate Base and all four RL methods under the shared protocol. All entries are accuracy (%):

BackboneDatasetBaseGRPOT-GRPOArrowRLCRPO
4BAoT-VQA60.9461.8965.5662.6167.06
4BMMVU MCQ64.4865.1265.7665.6067.04
8BAoT-VQA63.5065.2865.6766.2868.72
8BMMVU MCQ68.0068.6468.9668.9669.28

CRPO ranks first in all four settings. At 4B, it exceeds T-GRPO by 1.50 points on AoT-VQA and 1.28 on MMVU. At 8B, it exceeds the strongest competing RL method by 2.44 and 0.32 points, respectively. MMVU provides direct evidence of benefits to general video understanding at both scales.

The 8B results on the five primary benchmarks also lead the RL comparison:

BenchmarkMetric (%)GRPOT-GRPOArrowRLCRPO
DyBenchP-Acc52.454.552.258.1
TimeBlindI-Acc30.532.529.336.0
TempCompassAcc75.575.474.977.4
VideoMMEAcc64.565.065.165.6
MVBenchAcc69.169.569.769.7

8B CRPO exceeds T-GRPO on all five benchmarks, including VideoMME and MVBench. It leads every compared RL method on four benchmarks and ties ArrowRL on MVBench. These results support improved spatiotemporal sensitivity and general video understanding while retaining a task-dependent interpretation of the 4B comparisons. We plan to extend independent training-seed experiments to the remaining methods, the 8B backbone, and all benchmarks.

Manuscript changes: Independent training-seed results and task-dependent comparisons are in Appendix F, Table 13. The additional benchmarks are introduced in Section 5.1, with results and analysis in Section 5.2, Table 2. The 8B primary-benchmark results are in Table 1 and discussed in Section 5.2.

AuthorsOpenReview ↗

Author Responses (3/3)

Single-frame spatial shortcuts and Narrow the claim (Question 3)

Thank you for pointing out this important distinction. We agree that horizontal flipping can change single-frame spatial cues. Your suggestion helped us correct the scope of our claims. We have narrowed the claim in the manuscript and now rely on the measured shortcut-isolation results to support temporal reliance. The specific changes are:

  • Section 1: We deleted “making fixed-answer shortcuts ineffective.” The sentence now reads: “Its strict pair accuracy requires both sides of a pair to be answered correctly.” Section 4 also retains this joint-correctness definition without the redundant unchanged-answer claim.
  • Appendix A.1: We removed the assertion that a model using a single keyframe or language prior cannot answer both videos correctly. The remaining guarantee is limited to pairs with different correct answers: predicting the same semantic answer for both cannot satisfy the pair. This does not imply that every single-frame shortcut is excluded.
  • Appendix A.4: We explicitly acknowledge that horizontal flipping changes single-frame spatial cues. We replaced the broad conclusion that DyBench necessarily requires temporal information with: “These reductions show substantial reliance on ordered video input under the evaluated settings.” This conclusion follows from the disrupted-input measurements in Table 6.

Our Moving Direction construction also reduces the specific shortcut of inferring displacement from object orientation. As illustrated in Figure 3, the GOT-derived questions jointly ask about displacement direction and forward/backward motion, with choices covering all four combinations. For example, a car facing right can move forward to the right or reverse to the left. Heading alone cannot determine the joint answer. This design applies to 300 GOT-derived pairs, accounting for 60% of Moving Direction. The 200 CLEVRER-derived pairs instead use four choices drawn from diagonal directions and, where included, “The object is stationary.” Stationary-target pairs test answer invariance. Appendix A.2 now describes these two source-specific constructions separately.

The existing shortcut-isolation results provide the empirical evidence: Qwen3-VL-8B + CRPO achieves 58.1% DyBench P-Acc with the full video, versus 7.4% with a single frame, 13.2% with shuffled frames, and 1.1% with text only. These reductions support substantial reliance on ordered video input under the evaluated settings.

Manuscript changes: We removed the broad or redundant shortcut statements in Sections 1 and 4 and specified the scope of pair accuracy in Appendix A.1. We clarified the purpose of the joint motion choices in Section 4 and rewrote Appendix A.2 to distinguish the GOT and CLEVRER option designs and specify the random selection of horizontal flipping or temporal reversal. Appendix A.4 now acknowledges that horizontal flips alter single-frame cues and grounds the temporal-reliance claim in the measured shortcut-isolation results (Table 6).

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer c7eC, as the discussion period draws to a close, we kindly follow up on our responses, including the counterfactual audit, independent training-seed results, and narrowed shortcut claims. We would appreciate any further feedback and would be happy to address any remaining questions. Thank you again for your time and thoughtful review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

Dear Reviewers,

We sincerely thank all three reviewers (c7eC, hTZL, g6en) for their careful evaluation and constructive feedback. We appreciate your recognition of the alignment between our objective and paired evaluation (c7eC, hTZL), the counterfactual relational formulation and dual-branch training framework (g6en), the value of the compute controls, ablations, and diagnostic analyses (c7eC, hTZL), and the improvements under controlled and out-of-distribution evaluations (c7eC, hTZL, g6en).

We have carefully addressed your comments in the individual responses and incorporated the following additions and revisions:

  • Reliability of the counterfactual relation reward: A manual audit at three training stages evaluates rewarded answer changes, counterfactual answer accuracy, and appropriate “None of the above” use. The annotations are used only for analysis, and CRPO training requires no additional counterfactual answer annotations. Section 5.3, Table 4. Raised by all three reviewers (c7eC, hTZL, g6en).

  • Additional temporal and general video benchmarks: We added AoT-VQA from AoTBench for temporal direction understanding and MMVU MCQ for general, expert-level multidisciplinary video understanding, evaluating Base and all compared RL methods at both 4B and 8B. Sections 5.1 and 5.2, Table 2. These experiments address the performance concerns raised by c7eC and g6en.

  • Independent training seeds: We added two training runs with different seeds for each of 4B T-GRPO and CRPO, giving three independent runs per method including the original. We report means and standard deviations on DyBench, VideoMME, and MVBench. Appendix F, Table 13. Requested by c7eC, and also relevant to the general-performance concern of g6en.

  • Training and evaluation choices: We clarified why training transformations require an additional “None of the above” option while benchmark questions do not, and added a DyBench evaluation with the extra option. Appendix D.1, Table 12. Raised by g6en.

  • Reward cancellation and ArrowRL: We corrected the scope of the cancellation argument, clarified ArrowRL's rollout-dependent MCQ reward, and explained how CRPO's shared standard deviation preserves the relative reward scales of its two branches. Sections 3.4 and 5.3, Appendices B.3 and D.3. Raised by g6en.

  • Narrower shortcut claims: We removed the broad claim that paired evaluation makes static shortcuts ineffective. The remaining guarantee applies to unchanged predictions on pairs with different correct answers. We clarified the joint motion choices and grounded temporal reliance in the measured shortcut-isolation results under the evaluated settings. Sections 1 and 4, Appendices A.1, A.2, and A.4, Table 6. Requested by c7eC.

  • CLEVRER source-video separation: We documented the retained training-video ID check against DyBench and clarified the inherited training-data filtering protocol. Appendix A.1. Raised by hTZL.

The revised manuscript distinguishes improvements over the base models from comparisons with T-GRPO and clarifies the benchmark-dependent 4B results. The additional evaluations strengthen the evidence for both spatiotemporal sensitivity and general video understanding.

We have incorporated these changes in the revised manuscript, with additions and modifications highlighted in blue. Added tables have shifted the numbering relative to the original submission, and all references above follow the revised manuscript. If you have any further questions or suggestions, we would be happy to discuss them and address any remaining concerns during the rebuttal period.

Thank you again for your time and consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.