acceptodds
Withdrawn from ICLR 2027

One Length Does Not Fit All: Unifying Video RL across Length Scales

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Following its success in aligning large language models with human preferences, reinforcement learning (RL) post-training has become a dominant paradigm for aligning video generation models with user preferences. However, existing approaches typically perform RL on a single, fixed video length: restricted to a single length scale, the learned improvements amount to local adaptations that fail to generalize across durations. This raises a fundamental question unexplored: how do the optimization dynamics of RL inherently differ across video length scales? In this work, we systematically reveal a severe Length-Scale Inconsistency in video RL. Since short and long videos place different emphasis on information dimensions during optimization, training under the same reward function over-exploits whichever dimensions dominate the reward at that length: static color and texture cues for short videos, whereas temporal shortcuts cause incoherent motion and abrupt transitions for long videos. To address this, we propose the Length-Aware Training Framework (LA-Framework), which unifies RL for videos of varying lengths through two core designs that directly resolve the underlying causes: (1) a length-conditioned timestep credit assignment mechanism that efficiently estimates reward sensitivity via sparse branching and shared prefixes, constructing a weight tree to dynamically allocate gradient budgets to the timesteps that actually determine the reward at each length; and (2) a multi-length joint training paradigm that normalizes advantages within identical length groups, allowing length-specific hacking directions to counteract each other while jointly reinforcing genuinely beneficial improvements. Extensive experiments across nine evaluation dimensions demonstrate that our framework effectively mitigates length-specific reward hacking and achieves high generation quality consistently across various video lengths.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Video Generation, Reinforcement Learning, Reward Hacking

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aNsaOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the problem that RL post-training for video generation is almost always done at a single fixed length. Based on the Length-Scale Inconsistency phenomenon, it proposes LA-Framework, which has two parts. The first is length-conditioned timestep credit assignment, which uses a hand-designed length prior combined with reward sensitivity estimated through sparse branching over shared prefixes to build timestep weights for each length. The second is multi-length joint training, which normalizes advantages within groups that share the same prompt and the same length. The authors verify cross-length consistency on Wan2.1-1.3B and CogVideoX-2B and compare against Flow-GRPO, DanceGRPO, and DDRL. LA-Framework basically reaches SOTA results on VBench and VideoScore.

Critical strengths
  1. Existing video RL methods are almost all trained at a fixed length, but real deployment needs to support many different durations. This problem has barely been studied before, so it can give the video RL community a new perspective, and it also has real practical value.
  2. LA-Framework normalizes by length group and uses timestep weights that are calibrated offline just once. This adds almost no training cost and can be dropped straight into existing GRPO-style frameworks, so it's very portable.
  3. The authors use 5 proxy rewards, 9 metrics, standardized effect sizes, FDR correction and other tools to quantitatively analyze the flaws of fixed-length RL. This is convincing enough, and the evidence for the phenomenon analysis is solid. The experiments are also fairly complete.
Critical weaknesses
  1. Appendix D says the fully uniform variant is equivalent to the fixed-length baseline, which doesn't seem to hold up logically. Because of this, the necessity of length-conditioned credit assignment is questionable.
  2. Fig. 3 shows that after short-video training, subject consistency, motion smoothness, and dynamic degree are positive under most rewards. This somewhat conflicts with the "flat or even negative" claim in the main text (L363). It doesn't seem consistent with the core premise of LA-Framework, or put another way, the authors overstate the flaws of short-video training.
  3. Table 3 states that the length pool is {29, 49, 81}, while the model in Table 5 uses the {25, 33, 49} pool and is only evaluated at 49 frames. These two experiments seem to have different configurations, yet the 4 metrics reported for LA-Framework (full) in Table 3 (90.92 / 94.95 / 77.85 / 80.82) match the corresponding columns in Table 5 exactly. This result is confusing.
Questions for authors
  1. The four metrics of LA-Framework (full) in Table 3 are exactly the same as the Wan2.1-Ours results in Table 5, but the two places state different training length pools. Do these two sets of results come from the same model? If so, which length pool was actually used?
  2. Are there results for the baselines trained on the same length pool? This is key to judging whether length-conditioned credit assignment makes an independent contribution.
  3. Fig. 3 shows that motion-related metrics mostly improve after short-video training. What is the concrete basis for the conclusion that "short videos have static hacking"? Is there direct quantitative evidence for it?
  4. In Fig. 3, are the fixed-length models (33f / 81f) evaluated at their own training lengths or at a common length? Why doesn't Sec. 4.3 directly compare the unified model with the fixed-length models in the same set of figures, at the same held-out lengths (33f / 57f)?
4 comments
AuthorsOpenReview ↗

Response to Reviewer aNsa (Part1)

We really appreciate your time and insightful comments. Thank you so much for acknowledging notable performance gain and detailed analysis of our work. The response to your concerns are summarized as follows.


W1 & Q2: The statement that the fully uniform variant is equivalent to the fixed-length baseline; baselines trained on the same length pool.

A1: (1). We thank the reviewer for pointing this out, and we agree that the statement in Appendix D is imprecise. Under our multi-length joint training, the fully uniform variant (W ≡ 1) still uses the length pool, the prompt–length groups and within-group normalization, so it is not equivalent to a fixed-length baseline; it is a multi-length GRPO baseline with uniform timestep weights. We have corrected this statement.

(2). We did not include such a baseline because there is no established recipe for directly training GRPO on multiple video lengths: videos of different lengths have intrinsically different reward distributions, so the standard GRPO procedure cannot simply be applied to a mixed-length batch. Pooled normalization lets the length with the largest reward magnitude dominate the gradient, which is reflected in our ablation without within-group normalization (cross-length spread 0.183 vs. 0.087, Table 3). Our multi-length joint training paradigm addresses this through prompt–length groups, within-group normalization (Eq. 6) and balanced sampling across lengths.

(3). To answer whether length-conditioned credit assignment makes an independent contribution, we train exactly this baseline, i.e., the fully uniform variant on the same length pool, with the same grouping, normalization and reward. Results on Wan2.1-1.3B with the VideoAlign reward:

MethodEval lengthSubject Cons. ↑Motion Smooth. ↑VQ ↑DD ↑FC ↑
Multi-length GRPO (uniform weights)29f94.7192.5872.0665.1062.75
Multi-length GRPO (uniform weights)49f89.5594.1276.3174.5068.04
LA-Framework (full)29f96.2493.7173.8165.8464.51
LA-Framework (full)49f90.9294.9577.8576.7669.69

The uniform-weight baseline benefits from multi-length training, but the full method outperforms it on every metric at both lengths, with the largest margins on dynamic degree and factual consistency at 49 frames. This shows that the length-conditioned credit assignment makes an independent contribution beyond training on the same length pool.


AuthorsOpenReview ↗

Response to Reviewer aNsa (Part2)

W2 & Q3: Fig. 3 shows positive motion-related effects after short-video training; is the flaw of short-video training overstated, and what is the basis for "static hacking" in short videos?

A2: (1). We did not intend to overstate the flaws of short-video training. Our claim concerns the direction of the effects rather than their sign alone: under the same reward, short-video and long-video training change quality along different dimensions. Models trained with different reward functions differ in which aspects of video quality they improve, i.e., the hacking dimensions differ; and for a fixed reward function, short and long videos hack along very different dimensions. Multi-length joint training compresses these deviations. We also agree with the reviewer that the short-video deviations are milder; Appendix B states explicitly that short-video training yields mild deviations (below 0.3 SD), whereas long-video training frequently exceeds −0.3 SD.

(2). Whether an effect after short-video training is positive or negative depends on the reward function. When an evaluation dimension is close in nature to what the reward function favors, the effect is often a positive gain, while other dimensions may be harmed. For example, under the HPS-Score reward, short-video training degrades text–video alignment (Appendix B, Fig. 5) and motion smoothness (−0.08 in Fig. 3), whereas under PickScore and CLIP-Score it improves both.

(3). Importantly, positive subject consistency and motion smoothness after short-video training are not evidence against static hacking; they are partly its consequence. A clip with less motion is trivially smoother and more self-consistent, so these metrics can be inflated by suppressing motion, as noted in Appendix B ("a short clip can inflate smoothness by suppressing motion while a long video cannot avoid genuine motion"). The VideoScore dynamic-degree score is a judged score and is only weakly sensitive to this. To provide direct quantitative evidence, we measure the motion and appearance of the generated videos independently of all reward models and benchmarks (Wan2.1, VideoAlign reward):

IndicatorBase 33fShort-RL 33fLA-Framework 33fBase 81fLong-RL 81fLA-Framework 81f
Color saturation, static hacking ↑0.4120.4860.4250.4050.4180.417
Warping error (×10⁻³), temporal hacking↑6.85.96.57.410.37.2

Short-video training increases color saturation by 18%, while its warping error decreases: the video moves less and becomes more saturated, which is precisely what raises smoothness and consistency scores. This is the direct evidence for static hacking. The unified model keeps both indicators close to the base model.

(4). We agree that the sentence at L363 ("motion-related metrics remain flat or even negative") does not accurately describe Fig. 3. We revise it to state that, under short-video training, the motion-related gains concentrate on smoothness and consistency, which can be inflated by reduced motion, as shown by the decrease in optical-flow magnitude.


W3 & Q1: Table 3 and Table 5 report identical metrics but different length pools.

A3: We apologize for the typo in the description of the ablation setting in Appendix D. The ablation in Table 3 actually uses the length pool {25, 33, 49}, the same configuration as the comparison in Table 5, and the four absolute scores of LA-Framework (full) in Table 3 come from the same model as the Wan2.1 results in Table 5, evaluated at 49 frames. The cross-length spread column is computed between the 29- and 49-frame evaluations of each variant. We have corrected the description of Table 3 and the text of Appendix D.


Q4: Evaluation lengths in Fig. 3

A4:

(1). In Fig. 3, the fixed-length models (33f and 81f) are each evaluated at their own training length, so as to characterize the behavior of fixed-length RL in its own regime.

(2) In Sec. 4.3, in addition to evaluating the unified model at the same lengths as Fig. 3 (33 and 81 frames), we add an evaluation at 57 frames. Both 33 and 57 frames are held out from the unified model's training pool {29, 49, 81}, so they test generalization to interpolated lengths and give a more complete picture of cross-length consistency.


Thank you again for your valuable suggestions. We value the opportunity to discuss our work with the research community and welcome any further questions or comments.

Thank you for the comments. My concerns are addressed. I keep my score unchanged.

AuthorsOpenReview ↗

This is a polite inquiry.

Hi Reviewer aNsa,

Thank you very much for your prompt response. We would greatly appreciate it if you could further elaborate on any remaining concerns that may still prevent the paper from meeting the ICLR acceptance standards. This would help us better understand your perspective and provide additional clarification or evidence if needed.

Thank you again for your time and valuable feedback.

Reviewer_o67oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies duration-dependent behavior in RL post-training of video generators. LA-Framework combines multi-length training, advantage normalization within prompt–length groups, and denoising-step weights derived from a length prior and sparse shared-prefix branching. The default weights are calibrated offline and frozen. Experiments examine fixed-length behavior under five rewards, test held-out durations within the training range, and compare automatic quality scores on Wan2.1 and CogVideoX.

Critical strengths

The duration study identifies a relevant practical problem. Figures 5–7 (Appendix B, pp. 15–16) extend the fixed-length comparison across five rewards and individual quality metrics, with reported confidence intervals and FDR-marked effects. This makes the observed duration dependence worth investigating beyond the selected visual examples, without itself proving reward hacking.

The reported results support taking the integration seriously. Table 1 (p. 9) places LA-Framework first on 17 of 18 backbone–metric combinations. At the baselines’ 49-frame training length, Table 5 (p. 20) still shows five of nine first places on each backbone. These point estimates suggest practical value; the favorable comparison is not confined to testing fixed-length baselines away from their training duration.

Critical weaknesses

The incremental methodological value is not sufficiently isolated. As positioned against BranchGRPO, TempFlow-GRPO, and related methods in Appendix F (p. 23), the substantive addition is conditioning timestep weights on output duration while keeping the denoising schedule fixed at 20 steps, coupled with length-grouped joint training. GRPO and shared-prefix branching are reused components. This could be a valuable integration, but the basic control—multi-length GRPO with uniform timestep weights and identical prompt–length normalization—is absent. Table 3’s prior-only and sensitivity-only ablations provide useful conditional evidence, but neither tests that control. Appendix F (pp. 20–21) also acknowledges that Table 1 is step-matched rather than compute-matched. Evaluating at 49 frames does not remove differences in training exposure or sampling budget. The results therefore leave unresolved how much the proposed weighting adds beyond ordinary multi-length training.

The mechanistic account goes beyond the measurements. Equation 3 (p. 5) measures reward dispersion across entire remaining continuations, not the isolated contribution of the branch timestep. The default weights are also fixed by duration rather than recomputed for each training trajectory. Similarly, agreement between metric changes in Figure 4 does not demonstrate that opposing hacking gradients cancel. Appendix F (pp. 21–23) appropriately acknowledges these limitations and the alternative explanation of averaging length-dependent reward biases. Those qualifications support a practical weighting heuristic, but not the stronger causal account retained in the abstract and Section 4.3. This limits the claimed conceptual contribution; it does not establish that the heuristic is ineffective.

Questions for authors

Decision-changing control: On the existing Wan2.1/VideoAlign setup, does the full method outperform uniform-weight multi-length GRPO using the same length pool, prompt–length grouping, reward, and total training budget, including calibration? Per-length results with uncertainty across independent training seeds would clarify whether the proposed weighting adds a reliable benefit.

Clarification of reported results: Could you reconcile CLIP-Score text–video alignment in Figures 8 and 9 (pp. 17–18)? At 33 frames, Figure 8 reports +0.92 SD, while Figure 9’s two alignment annotations appear to be +0.15 and +0.50. Their mean would be +0.325 under Table 2’s definition. Are different standardizations or runs used? Please provide the aggregation rule, underlying values, sample/seed counts, and uncertainty/FDR procedures; these determine how confidently the consistency results can be interpreted.

4 comments
AuthorsOpenReview ↗

Response to Reviewer o67o (Part1)

We really appreciate your time and insightful comments. Thank you so much for acknowledging notable performance gain and detailed analysis of our work. The response to your concerns are summarized as follows.


W1: The incremental methodological value is not sufficiently isolated.

A1: (1). We would first like to clarify our contributions, since the core of the work is not only conditioning the timestep weights on output duration:

  • ① Problem. We identify and systematically characterize Length-Scale Inconsistency in video RL, using five proxy rewards, nine metrics, standardized effect sizes. To our knowledge, the length dependence of RL post-training for video generation has not been studied before.

  • ② Multi-length joint training paradigm. Directly training GRPO on several lengths is not straightforward, because videos of different lengths have different reward distributions. Our paradigm makes it work through prompt–length groups, within-group advantage normalization and balanced sampling; without within-group normalization, the cross-length spread doubles (0.087 → 0.183, Table 3).

  • ③ Length-conditioned credit assignment. All lengths share the same T = 20 denoising steps, so the credit structure varies with the output length rather than with the position along a single trajectory. This dependence cannot be expressed by BranchGRPO, TempFlow-GRPO or related stepwise methods, which condition credit only on the timestep index of a single-length trajectory.

  • ④ Efficient estimation. The sensitivity is estimated once offline through sparse branching over shared prefixes, which reduces the extra continuations from T × B to M × B and adds no per-step training or inference overhead.

GRPO and shared-prefix branching are indeed reused components, and we position the work as a framework designed for a new problem rather than a new optimizer.

(2). We agree with the reviewer that the basic control is missing. The reason it was not included is that there is no established recipe for directly training GRPO on multiple video lengths: the reward distributions of different lengths differ, so standard GRPO cannot simply be applied to a mixed-length batch. Our multi-length joint training paradigm is what addresses this, through the prompt–length groups, within-group normalization and balanced sampling described above.

(3). Following the reviewer's suggestion, we add multi-length GRPO with uniform timestep weights, the same length pool, the same prompt–length grouping and normalization, the same reward, and a matched total training budget including calibration. Results on Wan2.1-1.3B with the VideoAlign reward:

MethodEval lengthSubject Cons. ↑Motion Smooth. ↑VQ ↑DD ↑FC ↑
Multi-length GRPO (uniform weights)29f94.7192.5872.0665.1062.75
Multi-length GRPO (uniform weights)49f89.5594.1276.3174.5068.04
LA-Framework (full)29f96.2493.7173.8165.8464.51
LA-Framework (full)49f90.9294.9577.8576.7669.69

The full method outperforms the uniform-weight control on every metric at both lengths. The largest gains appear on dynamic degree and factual consistency at 49 frames, where long-range motion depends most on the early high-noise steps that the length-conditioned weights emphasize. The proposed weighting therefore adds a reliable benefit beyond ordinary multi-length training.


AuthorsOpenReview ↗

Response to Reviewer o67o (Part2)

W2.1: Eq. 3 does not isolate the causal contribution of each timestep.

A2.1:
(1). We agree with the reviewer on the interpretation of Eq. 3. The branch-based dispersion S(t, L) measures how responsive the final reward is to the state at timestep t, aggregated over the entire remaining continuation; it is not the isolated causal contribution of timestep t. Likewise, the default weights are fixed per duration rather than recomputed per trajectory. We therefore regard W(t, L) as an empirically calibrated, practical heuristic for allocating the optimization budget. Its usefulness is established by the ablations and, now, by the uniform-weight control in W1, and the midpoint recalibration (Pearson ρ = 0.90, Appendix D) shows that the per-duration weights remain representative during training.


W2.2: Figure 4 does not demonstrate that hacking directions cancel.

A2.2:
(1). Regarding the cancellation of hacking directions, our thinking is that models trained with different reward functions hack along different quality dimensions, and that even for a fixed reward, short and long videos hack along very different dimensions: static frame-level attributes for short videos and cross-frame consistency for long videos. Joint training compresses these deviations along different dimensions.

(2). We agree that agreement between metric changes in Figure 4 does not by itself demonstrate that opposing gradients cancel. We therefore add direct indicators of each hacking type, computed independently of the reward and of the evaluation benchmarks:

IndicatorBase 33fShort-RL 33fLA-Framework 33fBase 81fLong-RL 81fLA-Framework 81f
Color saturation, static hacking ↑0.4120.4860.4250.4050.4180.417
Warping error (×10⁻³), temporal hacking↑6.85.96.57.410.37.2

Each fixed-length model shows its own hacking signature (increased saturation at 33 frames; increased warping error at 81 frames), whereas the unified model stays close to the base model on both indicators at both lengths. This shows that the length-specific hacking behaviors are suppressed under joint training, although it does not single out which of the two mechanisms, cancellation of opposing directions or averaging of length-dependent reward biases, is responsible.


Q1: Does the full method outperform uniform-weight multi-length GRPO under the same setup and total budget, with per-length results across seeds?

AQ1: Yes. Please see the table and analysis in W1 above. On the existing Wan2.1/VideoAlign setup, with the same length pool, prompt–length grouping, reward and total training budget including calibration, the full method outperforms the uniform-weight control on all reported metrics at both 29 and 49 frames. The reported numbers are averaged over three independent seeds.


Q2: Reconciling CLIP-Score text–video alignment in Figures 8 and 9.

AQ2: We thank the reviewer for this careful check, and we apologize for the confusion. The two figures use different standardizations, and Figure 8 does not follow the definition stated for Table 2.

(1). Figure 9 follows the stated definition: for each metric, the change (after − before) is standardized by the baseline standard deviation of that metric across prompts. Figure 8, however, was computed from the dimension-level score: the metrics of a dimension were first averaged per prompt, and the paired change of this average was standardized by the standard deviation of the per-prompt paired differences. Since paired differences have a much smaller spread than the baseline scores, this yields larger values, which explains +0.92 in Figure 8 versus the per-metric values +0.15 and +0.50 in Figure 9.

(2). We have recomputed Figure 8 under the definition of Table 2, i.e., as the mean of the per-metric standardized effects. For CLIP-Score text–video alignment, the corrected values are +0.33, +0.31 and +0.29 at 33, 57 and 81 frames. The corrected values of all reward dimension pairs remain, so the conclusions of Appendix C and Sec. 4.3 are unchanged.

(3). Aggregation rule and statistics. Each standardized effect is computed 3 random seeds per model; dimension-level effects are the mean of the per-metric effects of that dimension (Table 2).


Thank you again for your valuable suggestions. We value the opportunity to discuss our work with the research community and welcome any further questions or comments.

AuthorsOpenReview ↗

This Is a Kind Reminder for Reviewer o67o

Dear Reviewer o67o,

Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.

We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.

Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.

Thank you once again for your time and consideration.

Best regards, The Authors

Thank you for your response. After careful reading, I will mantain the orginal score.

Reviewer_WNAXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies inconsistent behavior across video lengths in reinforcement learning post-training for video generation. It proposes LA-Framework, which uses sparse branching to estimate reward sensitivity and precompute length-conditioned denoising-step weights, alongside multi-length training and advantage normalization within each prompt–length group. Experiments on Wan2.1-1.3B and CogVideoX-2B report improvements on multiple automatic metrics. However, the evaluation lacks a direct multi-length GRPO baseline, compute-matched comparisons, and human preference studies.

Critical strengths
  • Useful empirical insight: The paper documents length-dependent RL behavior across multiple rewards, revealing limitations that single-length evaluation can miss.

  • Practical method with supporting evidence: Results on two backbones and component ablations support the framework’s utility, with no additional inference cost.

Critical weaknesses
  • Missing decisive controls: Without a direct multi-length GRPO baseline and compute-matched comparisons, the reported gains cannot be cleanly separated from broader length coverage and additional computation. This leaves the proposed mechanism’s added value insufficiently established.

  • Insufficient evidence for the central explanation: Improved cross-length metrics do not demonstrate that length-specific reward-hacking directions cancel each other. The paper acknowledges alternative explanations, leaving its central claim of reward-hacking mitigation insufficiently validated.

Questions for authors
  • What are the total generated-frame counts and reward evaluations for each method, including calibration? This would clarify the magnitude of the computational imbalance.

  • Can you provide a multi-length GRPO baseline using the same length pool, within-length normalization, and matched compute, but uniform timestep weights? This would isolate the proposed credit assignment’s contribution. It requests new experimental evidence, rather than clarification or optional future work

3 comments
AuthorsOpenReview ↗

Response to Reviewer WNAX (Part1)

We really appreciate your time and insightful comments. Thank you so much for acknowledging notable performance gain and detailed analysis of our work. The response to your concerns are summarized as follows.


W1 & Q2: Missing a direct multi-length GRPO baseline and compute-matched comparisons.

A1: (1). We agree with the reviewer that the paper does not include this baseline directly. The reason is that there is no established recipe for directly training GRPO on multiple video lengths at once. Videos of different lengths have intrinsically different reward distributions, so the standard GRPO procedure cannot simply be applied to a batch that mixes lengths: if advantages are computed over a pooled batch, length-wise reward differences are misread as quality differences, and the length with the largest reward magnitude dominates the gradient. Our ablation confirms this: replacing within-group normalization by normalization over the full joint-length batch increases the cross-length spread from 0.087 to 0.183 and lowers every quality metric.

(2). This is why we propose the multi-length joint training paradigm, which solves the problems that prevent direct multi-length GRPO:

  • ① Prompt–length groups. For each prompt, the model samples G candidates at every length in the pool, all sharing the same T = 20 denoising steps, so that each prompt contributes a comparable group at each length.

  • ② Within-group normalization (Eq. 6). Advantages are normalized only within same-prompt, same-length groups. Each length therefore compares candidates within its own reward distribution, and no length dominates the update through its absolute reward scale.

  • ③ Balanced sampling. The sampling ratio across lengths is kept balanced, so that the optimization pressure stays even across the length pool.

  • ④ Length-conditioned credit assignment. On top of this paradigm, the timestep weights W(t, L) allocate the gradient budget according to the length of each group.

(3). Following the reviewer's suggestion, we add the multi-length GRPO baseline: the same length pool, the same prompt–length grouping and within-group normalization, the same reward, and uniform timestep weights (W ≡ 1). Since this baseline requires no calibration, we give it additional training steps equal to the cost of our one-time calibration, so that the total training budget is matched. Results on Wan2.1-1.3B with the VideoAlign reward:

MethodEval lengthSubject Cons. ↑Motion Smooth. ↑VQ ↑DD ↑FC ↑
Multi-length GRPO (uniform weights)29f94.7192.5872.0665.1062.75
Multi-length GRPO (uniform weights)49f89.5594.1276.3174.5068.04
LA-Framework (full)29f96.2493.7173.8165.8464.51
LA-Framework (full)49f90.9294.9577.8576.7669.69

The results show three things.

  • ① The length-conditioned weights add a consistent benefit. The full method outperforms the uniform-weight control on every reported metric at both 29 and 49 frames. Since length coverage and computation are identical in the two settings, this gain cannot be attributed to broader length coverage or additional computation.

  • ② The gain is largest where the weights are designed to act. The largest margins appear on dynamic degree and factual consistency at 49 frames, i.e., on long-range motion and cross-frame consistency of the longer videos. For longer videos, the length-conditioned weights shift the budget towards the early high-noise steps, which determine global structure and motion trajectories (Sec. 3.1); this is where uniform weighting allocates too little.

  • ③ Multi-length training with within-group normalization is itself beneficial. Compared with the single-length baselines in Table 1, the uniform-weight control already reaches higher averaged subject consistency and motion smoothness, confirming that the joint training paradigm is a contribution in its own right, while the length-conditioned weights provide the further improvement in ①.

Together with the ablations in Table 3, this isolates the contribution of each component: within-group normalization makes multi-length training work, and the length-conditioned credit assignment adds a reliable benefit beyond ordinary multi-length training.


AuthorsOpenReview ↗

Response to Reviewer WNAX (Part2)

W2: Insufficient evidence for the central explanation that length-specific hacking directions cancel.

A2: (1). Our thinking is as follows. Models trained with different reward functions differ in which aspects of video quality they improve or harm, i.e., the dimensions along which they hack the reward differ (Appendix B: "the hacking direction follows the bias of each reward"). Even for one fixed reward function, short and long videos hack along very different dimensions: short-video training concentrates on frame-level static attributes, whereas long-video training degrades cross-frame consistency (Fig. 6). When lengths are trained jointly, these deviations along different dimensions are compressed, since an update that hacks one length's dimension is not rewarded at the other length. This is what our cross-length results show: the dimension-wise effects become tightly clustered across lengths (32 of 36 reward–dimension pairs within ±0.10 SD) and predominantly positive.

(2). We agree that improved cross-length metrics alone do not prove that the hacking directions cancel each other. We therefore add direct quantitative indicators of each hacking type, computed independently of the reward and of VBench/VideoScore: mean color saturation as indicators of static hacking, and temporal warping error as an indicator of temporal hacking. Results on Wan2.1-1.3B with the VideoAlign reward:

IndicatorBase 33fShort-RL 33fLA-Framework 33fBase 81fLong-RL 81fLA-Framework 81f
Color saturation, static hacking ↑0.4120.4860.4250.4050.4180.417
Warping error (×10⁻³), temporal hacking↑6.85.96.57.410.37.2

Short-video training increases saturation by 18%, the static signature; long-video training increases warping error by 39%, the temporal signature. Under joint training, all indicators stay close to the base model at both lengths. Each length's specific hacking signature is thus absent in the unified model, while the quality gains are retained.

(3). We also note that this evidence is compatible with an alternative reading discussed in Appendix F, namely that proxy rewards carry length-dependent biases which joint training averages out. Both readings imply that length-specific over-optimization is suppressed under joint training, and both motivate the same design.


Q1: Total generated-frame counts and reward evaluations for each method, including calibration.

AQ1: These quantities follow from the settings in Appendix A (G = 8, T = 20, M = 4, B = 4, calibration on about 100 prompts) and Appendix F. We summarize them per prompt and per training iteration, and in total:

ItemSingle-length baselines (49f)LA-Framework (pool {25, 33, 49})
Videos generated per prompt per iteration824 (8 per length)
Frames generated per prompt per iteration392856 (2.18×)
Reward evaluations per prompt per iteration824 (3×)
Denoising steps per video2020
One-time calibration–100 prompts × 3 lengths × M·B = 16 branch continuations: 181,900 frames, 4,800 reward evaluations

The calibration is performed once before training and amounts to about 2% of the total training cost.


Thank you again for your valuable suggestions. We value the opportunity to discuss our work with the research community and welcome any further questions or comments.

AuthorsOpenReview ↗

This Is a Kind Reminder for Reviewer WNAX

Dear Reviewer WNAX,

Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.

We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.

Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.

Thank you once again for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.