acceptodds
Under review as a conference paper at ICLR 2027

GraftOOD: Grafting Out-of-Distribution Exemplars into Advantage-Weighted Rectification for Video Reward Optimization

Abstract

Video reward optimization can increase proxy rewards while reducing video dynamics, a mismatch we identify as motion collapse, including under a video-level motion-quality reward. We propose GraftOOD, which replaces part of the online rollouts in selected prompt groups with out-of-distribution video exemplars. A shared velocity-regression objective combines group-relative reward rectification with sequence-level exemplar supervision, requiring neither exemplar generation trajectories nor an additional supervised-loss coefficient. We characterize its pointwise optimal field as a source-posterior-weighted combination of the two source-specific target fields. We construct GraftSet-Video, containing 9,588 format-matched exemplars for 2,397 prompts, to supply external supervision. On Wan2.1-1.3B, GraftOOD raises Dynamic Degree from 4.86 to 44.41 relative to reward-only FlowAWR, while HPSv3 decreases from 8.10 to 7.32 and VisionReward remains close. At the main setting, GraftOOD leads nine of ten metrics among three exemplar-augmented methods. Ablations of exemplar frequency and within-group count reveal dynamics–reward trade-offs rather than uniform gains from increasing exemplar usage. These results support external exemplar supervision as a complement to scalar reward feedback, without equating higher measured dynamics with coherent, visually plausible motion in the generated videos.

Primary area:
generative models
Keywords:
Reinforcement Learning, Video Generative Models, Flow Matching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_sxz4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies motion collapse during video reward optimization and proposes GraftOOD, which combines reward-rectified online samples with external video examples in a shared velocity-regression objective. Specifically,online rollouts are trained with reward-rectified velocity targets, while external exemplars are re-noised and assigned standard conditional flow-matching targets, allowing both sources to be optimized through a shared regression objective without access to the exemplars’ original generation trajectories.

Critical strengths

1.The paper is motivated by a practically important failure mode. As a user of video benchmarks and reward models, I find the observation that proxy-reward optimization can reduce video dynamics both credible and relevant. The paper further demonstrates this behavior with well constructed experiments.

2.The proposed formulation offers a clean and practically convenient way to incorporate terminal-only exemplars into reward-guided velocity regression without requiring their original generation trajectories.

Critical weaknesses
  1. The claim that GraftOOD avoids an additional SFT-loss coefficient is somewhat overstated. The relative strength of exemplar supervision is instead controlled by the empirically selected ρ\rho and noodn_{\mathrm{ood}}, making their selection analogous to tuning an explicit loss coefficient rather than eliminating the balancing problem. Moreover, changing noodn_{\mathrm{ood}} simultaneously changes both exemplar exposure and the number of online rollouts used for advantage estimation, making their individual effects difficult to isolate.

  2. The practical cost of constructing GraftSet-Video is not reported. The dataset relies on 9,588 videos collected from or generated by multiple models, yet the paper does not provide the generation time and computational or API cost. Since acquiring these examples is a central component of the method, this omission makes its practical feasibility and reproducibility difficult to assess.

  3. The motivation for the unified framework, and its advantages over simply combining FlowAWR with exemplar-based SFT, are not sufficiently clear or empirically isolated (see Questions).

Questions for authors
  1. Online rollouts provide full generation trajectories, yet GraftOOD discards this information and trains from re-noised terminal samples. Could the authors clarify the advantages and limitations of velocity regression relative to trajectory-based optimization?

  2. What benefit does the unified regression objective provide over a naïve combination of FlowAWR and a separately weighted exemplar SFT loss?

9 comments
AuthorsOpenReview ↗

Our Response to Reviewer sxz4 (1/2) [W1, W3, Q2]: Exemplar Strength, Rollout Controls, and the Unified Objective

We thank the reviewer for the careful and constructive review. We respond to each point below.

W1: Avoiding an SFT-loss coefficient is overstated: ρ\rho and noodn_{\rm ood} play that role, and noodn_{\rm ood} also changes the online rollout count.

A1: We agree that "without an additional supervised-loss coefficient" overstates the point: ρ\rho and noodn_{\rm ood} do set exemplar strength. The formulation uses unit-weight residuals in the same velocity space; it makes supervision strength a data-composition choice, but does not eliminate balancing.

Changing noodn_{\rm ood} also changes the number of online rollouts and the statistics used for reward normalization. We separate these factors at ρ=0.167\rho=0.167, 160 steps:

SettingDDVRVQMQTAHPSv3
FlowAWR4.865.110.060.47-0.658.10
+ GraftSet prompts (8 rollouts, no exemplars)10.215.02-0.210.35-0.687.12
+ only 4 rollouts in these groups (no exemplars)6.275.080.040.41-0.627.23
+ 4 exemplars (= GraftOOD)44.415.08-0.25-0.07-0.457.32

Same evaluation set and protocol as the main table.

Introducing GraftSet prompts alone gives DD 10.21; reducing those groups to four rollouts gives 6.27; adding four exemplars at that fixed online count gives 44.41.

The existing sweep shows that the benefit is not restricted to one ratio. Across all nine positive settings, DD is 30.58–48.68 (FlowAWR: 4.86) and HPSv3 is 6.54–7.74 (base: 4.70). At ρ=0.167\rho=0.167, nood=1n_{\rm ood}=1, exemplars constitute about 2% of training samples, yet DD reaches 30.58 and VR 5.52 (FlowAWR: 5.11). The ratios select different points on the dynamics–reward trade-off.

W3 & Q2: What benefit does the unified objective provide over FlowAWR plus a separately weighted exemplar SFT loss?

A2: GraftOOD applies advantage-weighted regression to a group containing online rollouts and terminal-only exemplars. Its decomposition into FlowAWR and SFT terms follows from this formulation. For exemplars, an uncentered unit coefficient cancels the unavailable reference field and yields a computable flow-matching target; exemplars are excluded from online reward normalization; and their relative weight is the group fraction. These choices define how external videos enter reward-relative learning.

We compare against FlowAWR + λ\lambda·SFT with the same GraftSet prompts and exemplars: four exemplars are appended to eight online rollouts in selected groups, with λ=1\lambda=1 and all other settings matched, including prompt mixing, shared timestep, and online-only reward normalization. At nood=4n_{\rm ood}=4, G=8G=8, this matches GraftOOD's relative online/exemplar loss weight; the comparison is append versus replace, including the resulting rollout allocation. It generates more videos per round (384 vs. 352), yet ends below GraftOOD on HPSv3, VR, and DD (6.96/4.89/36.58, versus 7.32/5.08/44.41). The full comparison, including SFT only (3 epochs) and SFT → FlowAWR (160 further RL steps):

MethodDDVRVQMQTAHPSv3
Wan2.1-1.3B48.611.26-0.59-0.34-0.664.70
FlowAWR4.865.110.060.47-0.658.10
SFT only50.211.48-0.42-0.23-0.624.89
SFT → FlowAWR7.175.020.110.51-0.537.82
FlowAWR + λ\lambda·SFT36.584.89-0.14-0.15-0.606.96
GraftOOD44.415.08-0.25-0.07-0.457.32
AuthorsOpenReview ↗

Our Response to Reviewer sxz4 (2/2) [Q1, W2]: Terminal Regression, Trajectory-Based Optimization, and Dataset Cost

Q1: What are the advantages and limitations of velocity regression from re-noised terminal samples relative to trajectory-based optimization?

A3: The reviewer is correct that GraftOOD trains from re-noised terminal samples rather than the sampled denoising transitions, for both online videos and exemplars. This is inherited from FlowAWR. For online samples, terminal rewards define group-relative advantages, which rectify the reference velocity at the re-noised state. Retaining this reward-guided target does not mean retaining the original trajectory in the training objective.

  • Why this interface. Both sources provide terminal video latents. Re-noising them gives inputs in the same velocity space, allowing online reward rectification and exemplar supervision to enter one regression objective. External generators need not expose denoising trajectories, transition likelihoods, or their velocity fields.
  • What it trades off. Trajectory-based optimization assigns terminal reward through likelihood terms along sampled transitions. Terminal regression does not use that transition-level information. Its practical advantage coefficients also approximate state-conditional quantities using finite-group reward normalization. Our pointwise-optimum result characterizes the regression target; it does not establish equivalence to a full trajectory policy gradient or guarantee better video quality.
  • Empirical comparison. FlashOOD provides the trajectory-based alternative: it combines Flash-GRPO's single-SDE-transition update with exemplar SFT at the matched noise level. With the same exemplar pool and main allocation, GraftOOD attains DD/VR/HPSv3 of 44.41/5.08/7.32, versus 58.26/1.65/5.31 for FlashOOD. Thus regression supports a useful dynamics–reward balance in this comparison; we do not claim that it universally dominates trajectory-based optimization.

W2: The practical cost of constructing GraftSet-Video is not reported.

A4:

  • Collection and selection. We retain videos from public video-reward-model resources, keeping only sources whose mean HPSv3 is above 7, then supplement prompts with fewer than four exemplars. All nine sources in the final corpus are generators substantially stronger than the base policy (source HPSv3 means 8.656–10.155). The final corpus contains 9,588 videos for 2,397 prompts. Supplementary generation fills the missing slots; the reported API spending is for this stage, not for regenerating every video or reproducing the source datasets.
  • Acquisition cost. HPSv3 scoring for selection took under one day on 8 RTX PRO 6000 GPUs. Supplementary generation cost under 2,000 USD in API fees and took about three days in parallel. These figures cover compute and API fees, not manual effort.
  • Training cost. The main Wan2.1-1.3B RL comparisons use 160 optimizer steps. Normalized to 24 RTX PRO 6000 GPUs (96 GB), including evaluation, FlowAWR takes about 29 h (≈700 GPU·h); GraftOOD uses ≈700/660/650 GPU·h at ρ=0.167/0.33/0.5\rho=0.167/0.33/0.5. These are approximate measured costs, with wall-clock variation from reward-service waiting. In the main setting, each round contains 48 groups: eight groups replace four rollouts with exemplars, reducing generation from 384 to 352 videos. Exemplars are not scored by the reward model during training.
  • Release. The de-anonymized version will link the prompts and videos, subject to source-dataset licenses and generation-service terms. Where video redistribution is not permitted, we will provide prompts, source indices, and preprocessing scripts.
  • Exemplar statistics. On the same GraftSet prompts, exemplar DD is 80.14, versus 53.61 for the base Wan2.1-1.3B (0–100 scale); per-source DD ranges from 58.96 to 95.02. The GraftSet mean HPSv3 is 9.735; downstream TA is −0.45 for GraftOOD, versus −0.65 for FlowAWR and −0.66 for the base model.

In summary, we agree that ρ\rho and noodn_{\rm ood} set exemplar strength. The new controls isolate the rollout-count confound, and at the same relative loss weight the append variant ends below GraftOOD on HPSv3, VR, and DD. Supplementary generation cost under 2,000 USD in API fees. We hope these results address your concerns and are happy to clarify any remaining points.

AuthorsOpenReview ↗

Our Kind Reminder to Reviewer sxz4: Summary of Our Response (Exemplar Strength, Unified Objective, Terminal Regression, Cost)

Dear Reviewer sxz4,

Thank you again for your careful review. As the discussion period is nearing its end, we briefly summarize our response to your questions. The General Response outlines the starting point of GraftOOD and collects the additional results.

  • Exemplar strength and rollout count (W1; Response (1/2), A1). ρ\rho and noodn_{\rm ood} set exemplar strength through data composition. To separate exposure from the online rollout count at ρ=0.167\rho=0.167, GraftSet prompts alone give DD 10.21, reducing those groups to four rollouts gives 6.27, and adding four exemplars at that fixed online count gives 44.41. Across all nine positive settings, DD stays within 30.58–48.68 (FlowAWR: 4.86).
  • Unified objective versus FlowAWR + SFT (W3 & Q2; Response (1/2), A2). The exemplar target, the exclusion of exemplars from reward normalization, and their group-fraction weight follow from one dual-source group definition, so GraftOOD regresses toward the in-distribution reward-rectified target and the out-of-distribution exemplar field within one group. An append control (FlowAWR + λ\lambda·SFT) instead keeps all eight rollouts and adds the same exemplars as a separate SFT loss at GraftOOD's relative weight. Although it generates more videos per round (384 vs. 352), it ends below GraftOOD on HPSv3, VR, and DD (6.96/4.89/36.58 vs. 7.32/5.08/44.41).
  • Terminal regression versus trajectory-based optimization (Q1; Response (2/2), A3). Training from re-noised terminal samples, inherited from FlowAWR, lets online videos and exemplars without denoising trajectories enter one regression objective; it does not use transition-level likelihoods, and we do not claim equivalence to a full trajectory policy gradient. Against FlashOOD, the trajectory-based alternative, GraftOOD attains DD/VR/HPSv3 of 44.41/5.08/7.32 versus 58.26/1.65/5.31, a better dynamics–reward balance in this comparison.
  • Construction and training cost (W2; Response (2/2), A4). HPSv3 scoring for selection took under one day on 8 RTX PRO 6000 GPUs, and supplementary generation cost under 2,000 USD in API fees over about three days in parallel. Training takes about 700 GPU·h for FlowAWR and 650–700 GPU·h for GraftOOD.

We hope that our response resolves your concerns, and we would be grateful if you could take it into account when reconsidering your final recommendation.

Reviewer_ncVoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies "motion collapse" in video reward optimization, where the proxy reward rises while measured dynamics drop. GraftOOD mixes online rollouts with external video exemplars in some prompt groups. Online samples keep the FlowAWR reward-guided regression target, and exemplars provide a standard flow-matching/SFT target; both are trained with one velocity-regression loss. The authors analyze the optimal field of this mixed objective and build GraftSet-Video (2,397 prompts, 9,588 videos). On Wan2.1-1.3B, GraftOOD greatly increases Dynamic Degree over FlowAWR, at some cost in HPSv3 and with VisionReward roughly unchanged. The paper also compares three exemplar-augmented objectives and studies the trade-off between dynamics and reward as exemplar usage varies.

Critical strengths
  1. The problem is practically relevant. Even when optimizing a video-level Motion Quality reward, the reward can rise while dynamics fall. This suggests that, in the tested settings, proxy rewards alone do not adequately constrain the desired temporal behavior.

  2. The method is simple and practical. External videos only need to be re-noised to provide supervision; their generation trajectories are not required. The paper implements versions for three training objectives and ablates the exemplar prompt ratio and the number of exemplars per group, showing the trade-off between supervision strength and reward optimization.

Critical weaknesses
  1. The novelty appears somewhat limited, as the method is essentially a combination of SFT and RL, and the gains rely on external supervision. In Equation 16, the exemplar coefficient is 1 and the reference field cancels, leaving a standard SFT target. For groups with exemplars, the loss can thus be written as (1−δ)·L_FlowAWR + δ·L_SFT with δ = n_ood/G, and Proposition 1 mainly characterizes the minimizer of a mixed squared loss as a conditional mean. The results show that external supervision can regularize reward optimization, but it is unclear what the in-group mixing adds over other standard training strategies.

    The ratio ablations still show a trade-off: using more exemplars raises Dynamic Degree but lowers HPSv3. Avoiding an extra loss coefficient therefore does not remove the need to choose a mixing ratio. In addition, ρ also changes the prompt distribution, and the per-group exemplar count also changes the number of online rollouts and the reward normalization; these factors are not yet separated. The method also depends on additionally selected and generated videos, but there is no analysis of exemplar quality, sources, or construction cost, which makes it hard to judge how much the results depend on this external supervision.

  2. The practical value of recovered dynamics needs more direct validation. Compared with FlowAWR, Dynamic Degree rises substantially, but Motion Quality and Video Quality both drop. These scores and HPSv3 are still above the base model, so this is not simply a quality loss; equally, higher Dynamic Degree does not show that motion is more coherent, plausible, or prompt-consistent. The authors acknowledge the metric's limits, but a few sampled frames are not enough to tell whether this trade-off matches human preference; full-video evaluation is needed.

  3. Experiments are limited to a single small generator. All experiments use Wan2.1-1.3B, and apart from the Motion Quality motivation experiment, the main comparisons are limited to HPSv3 optimization. Larger models may behave differently in both motion collapse and the effect of exemplar supervision, so it is unclear whether the conclusions carry over. It would strengthen the paper to validate the method on at least one larger or more recent backbone, such as Wan2.2 TI2V-5B or T2V-A14B or MiniMax-H3.

Questions for authors
  1. What does in-group mixing add over standard RL + SFT? Could you compare it, with the same prompts, exemplars, and comparable budget, against strategies such as exemplar-only SFT and SFT followed by RL?

  2. Could you add a FlowAWR control that keeps the same prompt mixing ratio but uses no external videos? Please also control for, or explain, differences in the number of online rollouts, reward normalization, and sampling budget, so that the effect of the data distribution can be separated from that of exemplar supervision.

  3. Could you run a blind evaluation on full videos, covering motion sufficiency, temporal coherence, physical plausibility, and deformation, to check whether the metric trade-off relative to FlowAWR matches human preference?

  4. How sensitive are the results to exemplar quality and source? Since exemplars were initially selected with HPSv3, which has limited sensitivity to motion, could you describe the motion quality of the final exemplars and report the cost of selection and supplementary generation? Please also describe deduplication and redistribution rights.

  5. Could you report the total training steps, EMA decay rate, and the actual sampling and compute budget of each method?

7 comments
AuthorsOpenReview ↗

Our Response to Reviewer ncVo (1/3) [W1, Q1, Q2]: Dual-Source Formulation, SFT Baselines, and Mixing Controls

W1 & Q1: The method is essentially SFT + RL. What does in-group mixing add over standard RL + SFT (SFT only, SFT → RL)?

A1: The reviewer's decomposition (1−δ)LFlowAWR+δLSFT(1-\delta)\mathcal{L}_{\rm FlowAWR}+\delta\mathcal{L}_{\rm SFT} (δ=nood/G\delta=n_{\rm ood}/G) holds for the sample-averaged loss, but it follows from the formulation rather than being its starting point. GraftOOD applies FlowAWR's advantage-weighted regression to a group whose members come from two sources: online rollouts and terminal-only exemplars. For exemplars from closed-source generators, trajectories and the velocity field voodv^{\rm ood} are unavailable, so rectification toward that reference cannot be computed directly. Within this group definition: (i) treating exemplars as uniformly top-reward gives an uncentered unit coefficient that cancels voodv^{\rm ood}; centering it would give 00, removing the exemplar's flow-matching target and leaving the unavailable reference field; (ii) exemplars are excluded from reward normalization, so they do not shift the online baseline; (iii) their weight is the group fraction δ\delta, the prior of the source posterior in Proposition 1. The exemplar term reduces to a flow-matching target because of (i). Proposition 1 then shows what this objective learns: the optimal field interpolates the reward-rectified field and voodv^{\rm ood} with the state-dependent source posterior rather than a constant δ\delta, so exemplars shape the target where their noised density dominates. The contribution is the dual-source group formulation and its resulting targets and weights.

For Q1, we compare standard combinations using the same GraftSet-Video prompts and exemplars:

MethodDDVRVQMQTAHPSv3
Wan2.1-1.3B48.611.26-0.59-0.34-0.664.70
FlowAWR4.865.110.060.47-0.658.10
SFT only50.211.48-0.42-0.23-0.624.89
SFT → FlowAWR7.175.020.110.51-0.537.82
FlowAWR + λ\lambda·SFT36.584.89-0.14-0.15-0.606.96
GraftOOD44.415.08-0.25-0.07-0.457.32

Same evaluation set and protocol as Table 1. SFT only: 3 epochs (metrics plateau, then decline). SFT → FlowAWR: 160 further RL steps. FlowAWR + λ\lambda·SFT: 160 steps, λ=1\lambda=1 (as in FlashOOD), four exemplars appended to eight rollouts per GraftSet group; other settings (prompt mixing, shared timestep, online-only normalization) match GraftOOD.

  • SFT only improves over the base model, but its VR/HPSv3 (1.48/4.89) gains on the evaluation set are limited.
  • SFT → FlowAWR: despite a larger total budget, DD falls from 50.21 to 7.17, close to FlowAWR's 4.86: SFT pretraining does not protect dynamics during RL.
  • FlowAWR + λ\lambda·SFT generates more videos per round (384 vs. 352), yet ends below GraftOOD on HPSv3, VR, and DD (6.96/4.89/36.58).

At nood=4n_{\rm ood}=4, G=8G=8, λ=1\lambda=1 matches GraftOOD's relative online/exemplar loss weight. This compares appending exemplars with replacing group members, including the resulting rollout allocation. The results support maintaining exemplar supervision through the dual-source group during RL.

W1 & Q2: A mixing ratio must still be chosen; prompt distribution, rollout count, and reward normalization are not separated.

A2: We agree that "without an additional supervised-loss coefficient" overstates the point: ρ\rho and noodn_{\rm ood} do set exemplar strength. The formulation uses unit-weight residuals in the same velocity space; it makes supervision strength a data-composition choice, but does not eliminate balancing.

Across all nine positive settings (ρ∈0.167,0.33,0.5\rho\in\\{0.167,0.33,0.5\\}, nood∈1,2,4n_{\rm ood}\in\\{1,2,4\\}), DD stays within 30.58–48.68 (FlowAWR: 4.86) and HPSv3 within 6.54–7.74 (base: 4.70). At ρ=0.167\rho=0.167, nood=1n_{\rm ood}=1, about 2% of samples are exemplars, yet DD reaches 30.58 and VR 5.52 (FlowAWR: 5.11).

We separate prompt distribution, rollout count, and exemplar supervision at ρ=0.167\rho=0.167, for 160 steps:

SettingDDVRVQMQTAHPSv3
FlowAWR4.865.110.060.47-0.658.10
+ GraftSet prompts (8 rollouts, no exemplars)10.215.02-0.210.35-0.687.12
+ only 4 rollouts in these groups (no exemplars)6.275.080.040.41-0.627.23
+ 4 exemplars (= GraftOOD)44.415.08-0.25-0.07-0.457.32

Same evaluation set and protocol as Table 1.

  • Prompts: changing prompts raises DD from 4.86 to 10.21, far below GraftOOD's 44.41.
  • Rollouts: reducing online count lowers DD from 10.21 to 6.27, so fewer rollouts alone do not recover dynamics.
  • Exemplars: the last comparison holds four online rollouts fixed and adds four exemplars, changing DD from 6.27 to 44.41.

Original-prompt groups retain eight rollouts throughout. The sampling budget is given in A6.

AuthorsOpenReview ↗

Our Response to Reviewer ncVo (2/3) [W1, Q4, W2, Q3]: Exemplar Quality, Dataset Construction, and Blind Video Evaluation

W1 & Q4: How sensitive are the results to exemplar quality and source? Please report motion quality, cost, deduplication, and redistribution rights.

A3:

  • Motion in exemplars. On the same GraftSet prompts, exemplar mean DD is 80.14, compared with 53.61 for the base Wan2.1-1.3B (0–100 scale). The video-count-weighted GraftSet mean HPSv3 is 9.735, an independent aesthetic/visual-quality reward in addition to DD. Our visual inspection also finds better exemplar visual quality. By manual categorization of each prompt's primary motion-bearing subject, the prompts cover human activities (49.19%), animals (21.44%), vehicles (16.44%), and object motion (12.93%).
  • Sources and quality. Each prompt has four videos from four different generators; the largest source (Wan2.2) contributes 22.6%. Per-source DD ranges from 58.96 to 95.02; source HPSv3 means range from 8.656 to 10.155. The count ablation also varies exemplar selection: nood=1n_{\rm ood}=1 and 22 use the top-1 and top-2 by HPSv3. At every tested ρ\rho, top-1 attains the highest HPSv3 and VR, whereas all four attain the highest DD. Source-specific retraining costs about 700 GPU·h per run, so we report per-source statistics and the existing count ablation rather than additional source-specific runs.
  • Prompt separation. GraftSet contains 2,397 distinct prompts. They come from reward-model resources, focus on motion-rich scenes, and are disjoint from the 19,700 policy-training prompts and the VidProM evaluation prompts. This establishes prompt-list separation, not absence of shared semantic categories. No deduplication step was needed: the corpus is consolidated from the preference pairs used to train video reward models, and supplementary API generation only adds videos for existing prompts.
  • Cost. Selection relies on HPSv3 scoring, which took under one day on 8 RTX PRO 6000 GPUs. Supplementary generation cost under 2,000 USD in API fees and took about three days in parallel; these figures cover filling missing exemplar slots, not regenerating the corpus.
  • Release. The de-anonymized version will link the prompts and videos, subject to source-dataset licenses and generation-service terms. Where video redistribution is not permitted, we will provide prompts, source indices, and preprocessing scripts.

W2 & Q3: Higher Dynamic Degree does not show better motion; a blind full-video evaluation is needed.

A4: We agree that DD alone cannot establish motion quality, and we add a blind full-video study.

Eight raters (lab members who are not authors) compare full videos for 50 randomly sampled VidProM evaluation prompts, blind to method, with randomized left–right order and ties allowed. Each pair receives 8 ratings (400 judgments per comparison). Entries are counts of GraftOOD win/tie/loss:

Dimensionvs. FlowAWRvs. FlashOOD
Motion sufficiency343/29/28263/63/74
Temporal coherence159/38/203308/48/44
Physical plausibility212/31/157294/33/73
Subject preservation/deformation154/78/168328/29/43
Overall quality203/77/120368/17/15

GraftOOD is preferred in overall quality over both baselines: 50.8% wins versus 30.0% losses against FlowAWR, and 92.0% versus 3.8% against FlashOOD. Against FlowAWR it wins on motion sufficiency and physical plausibility, while FlowAWR is preferred on temporal coherence (203 vs. 159) and slightly on subject preservation (168 vs. 154). This matches FlowAWR's nearly static outputs (DD 4.86): a video with little motion trivially avoids flicker and deformation, yet raters still prefer GraftOOD overall. Against FlashOOD it wins on all five dimensions.

  • Subject preservation. GraftOOD's SC is 98.20, versus 94.58 for the base model and 94.68 for FlashOOD; their DD values are 44.41, 48.61, and 58.26. FlowAWR's SC is slightly higher (98.48), but its videos are nearly static (DD 4.86). SC provides complementary evidence, not a direct deformation test: it is measured on VBench prompts, whereas DD is measured on VidProM.
  • MQ and VQ. Optimizing MQ itself reduces DD in our Appendix experiment, so higher MQ does not by itself establish better motion; nearly static videos avoid motion artifacts, which favors FlowAWR on MQ/VQ and SC. GraftOOD remains above the base model on MQ (−0.07 vs. −0.34) and VQ (−0.25 vs. −0.59), and improves TA over both FlowAWR (−0.45 vs. −0.65) and the base model (−0.45 vs. −0.66).
AuthorsOpenReview ↗

Our Response to Reviewer ncVo (3/3) [W3, Q5]: Validation on 14B, Reward Choice, and Training Budget

W3: Experiments are limited to a single small generator and to HPSv3 optimization.

A5:

  • Larger backbone. We use Wan2.1-1.3B for the main study, as in Flash-GRPO, because of cost: in our earlier HPSv3 runs on Wan2.1-T2V-14B, one optimizer step took about 19.7 GPU·h, so a 160-step run needs about 3,150 GPU·h, versus ≈700 GPU·h on 1.3B. We therefore validate on Wan2.1-T2V-14B, the larger model of the same family: the pipeline carries over with all hyperparameters unchanged, the only change being the distributed-training strategy (DDP → FSDP). We ran FlowAWR and GraftOOD on 14B for 160 steps with the main allocation (ρ=0.167\rho=0.167, nood=4n_{\rm ood}=4), evaluated on the same evaluation set:
Method (14B)DDVRVQMQTAHPSv3
Wan2.1-T2V-14B52.433.62-0.40-0.25-0.455.72
FlowAWR9.175.961.171.24-0.749.81
GraftOOD48.295.780.961.02-0.528.83

Motion collapse recurs on 14B: FlowAWR raises HPSv3 from 5.72 to 9.81 while DD falls from 52.43 to 9.17. GraftOOD keeps DD at 48.29, close to the base model's 52.43, while retaining most of the reward gain (HPSv3 8.83; VR 5.78 vs. 5.96) and a better TA than FlowAWR (−0.52 vs. −0.74).

  • Reward choice. Following Flash-GRPO, we considered HPSv3 and VideoAlign MQ. Optimizing MQ reduces DD (Appendix) and later lowers VisionReward (Figure 2(c), Motion Quality curve), so collapse is not specific to HPSv3. We use HPSv3 as the main reward, as in Flash-GRPO's main comparison.

Q5: Please report total training steps, EMA decay, and the sampling and compute budget of each method.

A6: The main Wan2.1-1.3B RL comparisons use 160 optimizer steps; the SFT controls have the budgets specified in A1. Each round has 48 prompt groups of size 8 and two updates (80 rounds total). FlowAWR generates 384 videos per round; GraftOOD generates 384−48ρnood384-48\rho n_{\rm ood}, or 352 in the main setting (eight GraftSet groups). The EMA decay is ηi=0\eta_i=0 for i<75i<75 and min⁡(0.0075(i−75),0.999)\min(0.0075(i-75),0.999) otherwise, where ii is the optimizer step, following DiffusionNFT.

Normalized to 24 RTX PRO 6000 GPUs (96 GB), including evaluation, FlowAWR takes about 29 h (≈700 GPU·h). GraftOOD uses ≈700/660/650 GPU·h at ρ=0.167/0.33/0.5\rho=0.167/0.33/0.5. Costs are approximate; reward-service waiting causes wall-clock variation. The other baselines have comparable cost. All methods use 20-step training rollouts. DiffusionNFT and NFTOOD share FlowAWR's sampling and differ in the loss; Flash-GRPO and FlashOOD take a single SDE step within the rollout and train only on that transition.

Thank you for the detailed and constructive review. We hope these results address your concerns and are happy to clarify any remaining points.

Reviewer_wWyNOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates the "motion collapse" problem in video generation models during reward optimization: the agent reward continues to rise, but the video dynamics decreases, which can still occur even when introducing video-level motion-quality rewards. The authors propose GraftOOD, which incorporates the sequence level supervision of external video exemplars into group-relative velocity regression by replacing part of the online rollout with out-of-distribution video examples. Thus, it complements dynamic properties that cannot be adequately constrained by scalar rewards. GraftOOD does not introduce additional supervised-loss coefficient. Instead, it combines the online reward-rectified target with the exemplar reference target in a unified velocity-regression objective.

In terms of experiments, the authors constructed GraftSet-Video, which contains 2,397 prompt and 9,588 video exemplars, and evaluated it on Wan2.1-1.3B. The main results show that compared with reward-only FlowAWR, GraftOOD improves the Dynamic Degree from 4.86 to 44.41.

Critical strengths
  1. The motivation for the problem is solid and supported by empirical evidence. In the reinforcement learning for video generation, the rewards often increase, but the motion quality decreases. This paper uses three curves (HPSv3, Dynamic Degree, VisionReward) to clearly show the motion collapse phenomenon in reward-driven video RL. It shows that even adding RAFT dynamic rewards cannot prevent dynamic decay.

  2. The approach is reasonably motivated: complementing scalar rewards with an external sequence-level exemplar Instead of continuing to rely on more sophisticated reward engineering, the authors propose to introduce an external video exemplar as a reference for desired sequence-level behavior. This idea naturally responds to the problem that agent reward cannot fully constrain temporal behavior.

  3. The unified regression formulation has technical simplicity. GraftOOD unifies the online reward-relative target and OOD exemplar supervision into the velocity-regression objective, It also avoids the extra supervision, which makes the method more elegant than a weighted combination of simple SFT + RL.

Critical weaknesses
  1. The construction of GraftSet-Video and the explanation of data attributes are not sufficient. At present, the scene category distribution, motion intensity distribution, dynamic sample filtering criteria, and content overlap risk at the semantic level of the dataset are insufficiently explained. It is thus unclear whether this dataset truly provides high-quality, coherent motion supervision beyond reference videos of high visual quality.

  2. The improvement of dynamic degree is not enough to prove the real improvement of motion quality and lacks the support of user study. It is difficult to demonstrate true improvement in motion naturalness, temporal consistency, and subject stability with dynamic degree alone. The paper lacks a user study to verify whether the dynamic promotion conforms to human perception and whether it introduces subject collapse, motion artifacts, or temporal inconsistency.

  3. Dynamic degree improvement is accompanied by the decrease of other quality/reward metrics, and there is a dynamics-quality trade-off, but the reason analysis is insufficient.

Questions for authors
  1. It is suggested to introduce the construction details of GraftSet-Video, including scene/content category distribution, motion intensity distribution, dynamic sample selection criteria, and whether there is semantic or content level overlap with the evaluation set. The paper mentions that this dataset is partly selected based on HPSv3, which has limited motion sensitivity, so it needs to be stated whether these exemplars really provide coherent motion supervision.

  2. Since Dynamic Degree is based on RAFT optical flow, which may be affected by subject deformation, background drift or incoherent motion, it is suggested to supplement the user study to evaluate motion naturalness, temporal consistency, subject preservation, artifact severity, and overall video quality.

  3. The results in Table 1 show that although GraftOOD significantly improves dynamic degree, there is a decrease in HPSv3/VisionReward or other quality metrics. It is recommended that the authors further analyze whether this decrease comes from OOD data distribution, reduced number of online rollouts, or an inherent conflict between dynamic degree and visual quality.

4.It is recommended to add controlled experiments such as SFT-only on GraftSet-Video for baseline (wan2.1) or FlowAWR + standard SFT loss, to distinguish between the contributions of the GraftSet-Video data itself and the GraftOOD training objective.

5 comments
AuthorsOpenReview ↗

Our Response to Reviewer wWyN (1/2) [W1, Q1, W2, Q2]: GraftSet-Video Construction and Blind Motion Evaluation

W1 & Q1: GraftSet-Video construction: content and motion distribution, selection criteria, semantic overlap, and motion quality of exemplars.

A1: GraftSet-Video contains 9,588 videos for 2,397 distinct prompts, with four videos from four different generators per prompt. The videos match the policy format: 81 frames, 832×480, and 16 fps.

  • Selection. We first retain videos from public video-reward-model resources, keeping only sources whose mean HPSv3 is above 7, then fill remaining slots using external generators, including Seedance and Kling. All nine sources are generators substantially stronger than the base policy (means below). Supplementary videos fill slots rather than replace retained videos through pooled reranking. HPSv3 provides offline visual-quality selection, not a motion-coherence test. There is no separate dynamic filter: motion content comes from the motion-centric prompts, and the video statistics below characterize the resulting supervision.
  • Content distribution. We manually assign each prompt to one mutually exclusive category by its primary motion-bearing subject: human activities: 1,179 (49.19%), animals: 514 (21.44%), vehicles: 394 (16.44%), and object motion: 310 (12.93%). The corpus contains 1,247 Chinese and 1,150 English prompts; human activities are the largest category in both languages (47.15% and 51.39%, respectively).
  • Motion distribution. On the same GraftSet prompts, exemplar mean DD is 80.14, versus 53.61 for the base Wan2.1-1.3B (0–100 scale). The video-count-weighted GraftSet mean HPSv3 is 9.735, an independent aesthetic/visual-quality reward alongside DD. Our visual inspection also finds better visual quality in exemplars than in same-prompt base outputs. For prompt alignment, downstream TA is −0.45 for GraftOOD, compared with −0.65 for FlowAWR and −0.66 for the base model.
  • Sources and quality. Per-source means (DD / HPSv3) are Seedance 1.5: 95.02 / 10.155; Veo 3.1: 85.28 / 10.014; Kling 3: 85.03 / 8.656; Vidu Q2: 84.52 / 9.885; Seedance 2: 81.89 / 9.366; Wan2.2: 77.17 / 9.977; Hailuo: 76.39 / 9.487; Kling 2.5: 68.31 / 9.918; Sora 2: 58.96 / 9.534. DD spans 58.96–95.02. The largest source, Wan2.2, contributes 2,164 videos (22.6%).
  • Overlap. GraftSet prompts come from reward-model resources and are disjoint from the 19,700 policy-training prompts and the VidProM evaluation prompts. This establishes prompt-list separation, not absence of shared semantic categories. OOD is defined relative to the policy-training setting; the source resources may overlap reward-model training data. A prompt-only control (A3) trains on these prompts without exemplars and raises DD only to 10.21 (GraftOOD: 44.41), so exposure to these prompts alone does not account for the gain.

W2 & Q2: Dynamic Degree alone cannot prove better motion quality; a user study is needed.

A2: We agree that DD alone cannot distinguish natural motion from deformation, background drift, or incoherent movement. We add a blind pairwise study:

  • Sample and raters. Eight raters (lab members who are not authors) evaluate full videos for 50 prompts randomly sampled from the VidProM evaluation set.
  • Comparisons. GraftOOD versus FlowAWR and GraftOOD versus FlashOOD, using the same prompt within each pair. Method names are hidden and left–right order is randomized. Raters see the prompt and can choose either video or a tie for each dimension.
  • Criteria. Motion sufficiency asks whether the action required by the prompt is adequately expressed; temporal coherence assesses continuity and flicker/drift; physical plausibility assesses naturalness; subject preservation/deformation assesses identity, shape, and deformation artifacts; overall quality is the holistic preference.
  • Aggregation. Each pair receives 8 ratings, giving 400 valid judgments per comparison. Entries are counts of GraftOOD win/tie/loss.
Dimensionvs. FlowAWRvs. FlashOOD
Motion sufficiency343/29/28263/63/74
Temporal coherence159/38/203308/48/44
Physical plausibility212/31/157294/33/73
Subject preservation/deformation154/78/168328/29/43
Overall quality203/77/120368/17/15

Raters prefer GraftOOD in overall quality over both baselines: 50.8% wins versus 30.0% losses against FlowAWR, and 92.0% versus 3.8% against FlashOOD. Against FlowAWR, GraftOOD wins on motion sufficiency and physical plausibility, while FlowAWR is preferred on temporal coherence (203 vs. 159) and slightly on subject preservation (168 vs. 154). This matches FlowAWR's nearly static outputs (DD 4.86): a video with little motion trivially avoids flicker and deformation, yet raters still prefer GraftOOD overall. Against FlashOOD, GraftOOD wins on all five dimensions. The supplementary material already contains video comparisons with NFTOOD and FlashOOD.

AuthorsOpenReview ↗

Our Response to Reviewer wWyN (2/2) [W3, Q3, Q4]: Dynamics–Quality Trade-off, Controlled Ablations, and SFT Baselines

W3 & Q3: Does the dynamics–quality trade-off come from the OOD data distribution, fewer online rollouts, or an inherent conflict?

A3: The main comparison shows a trade-off with the optimized reward, while VisionReward remains close: DD changes from 4.86 to 44.41, HPSv3 from 8.10 to 7.32, and VR from 5.11 to 5.08. We separate the proposed explanations at ρ=0.167\rho=0.167, 160 steps:

SettingDDVRVQMQTAHPSv3
FlowAWR4.865.110.060.47-0.658.10
+ GraftSet prompts (8 rollouts, no exemplars)10.215.02-0.210.35-0.687.12
+ only 4 rollouts in these groups (no exemplars)6.275.080.040.41-0.627.23
+ 4 exemplars (= GraftOOD)44.415.08-0.25-0.07-0.457.32

Same evaluation set and protocol as the main table.

  • Prompt distribution. Keeping eight rollouts but introducing the same GraftSet prompt ratio gives DD 10.21 and HPSv3 7.12. The HPSv3 decrease is already present without exemplars, while DD remains far below GraftOOD's 44.41.
  • Online rollout count. Reducing only those groups to four rollouts, still without exemplars, lowers DD from 10.21 to 6.27, so reducing rollouts and changing reward-normalization statistics alone do not recover dynamics.
  • Exemplar supervision. At a fixed online count of four, exemplars raise DD from 6.27 to 44.41 and HPSv3 from 7.23 to 7.32, with VR unchanged at 5.08. VQ and MQ fall from 0.04/0.41 to −0.25/−0.07, so the remaining trade-off concerns those quality metrics rather than a uniform loss across rewards.

Exemplar strength also changes which behavior is emphasized. At every tested ρ\rho, the top-1 exemplar by HPSv3 gives the highest HPSv3 and VR, whereas all four give the highest DD. At ρ=0.167\rho=0.167, the top-1 setting reaches DD 30.58 and VR 5.52, exceeding FlowAWR's VR 5.11. Thus, greater dynamics does not necessarily require lower VR. The observed trade-off depends on exemplar composition and the reward being optimized; it does not establish an inherent conflict between motion and visual quality.

The MQ experiment provides further evidence: optimizing MQ itself reduces DD, so FlowAWR's higher MQ does not by itself establish better motion; nearly static videos avoid motion artifacts, which favors FlowAWR on MQ/VQ and SC. GraftOOD remains above the base model on MQ (−0.07 vs. −0.34) and VQ (−0.25 vs. −0.59), and improves TA over FlowAWR (−0.45 vs. −0.65). Its SC is 98.20, versus 94.58 for the base model and 94.68 for FlashOOD; FlowAWR's 98.48 comes with nearly static videos (DD 4.86). SC is complementary evidence on VBench prompts, not a direct deformation test on the VidProM videos; A2 evaluates motion and artifacts together on the same videos.

Q4: Please add controls such as SFT only and FlowAWR + standard SFT loss, to separate the data from the training objective.

A4: Using the same GraftSet prompts and exemplars:

MethodDDVRVQMQTAHPSv3
Wan2.1-1.3B48.611.26-0.59-0.34-0.664.70
FlowAWR4.865.110.060.47-0.658.10
SFT only50.211.48-0.42-0.23-0.624.89
SFT → FlowAWR7.175.020.110.51-0.537.82
FlowAWR + λ\lambda·SFT36.584.89-0.14-0.15-0.606.96
GraftOOD44.415.08-0.25-0.07-0.457.32

Same evaluation set and protocol as the main table. SFT only runs for 3 epochs, after which metrics plateau and decline; SFT → FlowAWR continues from it for 160 RL steps. FlowAWR + λ\lambda·SFT (λ=1\lambda=1, 160 steps) appends four exemplars to eight rollouts in each GraftSet group, sharing GraftOOD's prompt mixing, timestep, and online-only reward normalization.

SFT only yields limited evaluation gains, and subsequent FlowAWR reduces DD from 50.21 to 7.17. The append variant matches GraftOOD's relative online/exemplar loss weight and generates more videos per round (384 vs. 352), yet ends below GraftOOD on HPSv3, VR, and DD. These results distinguish the contribution of GraftSet from its allocation during RL.

What the unified formulation emphasizes is how terminal-only exemplars enter reward-relative learning: their flow-matching target, their exclusion from reward normalization, and their weight all follow from the dual-source group definition.

Thank you for the supportive and constructive review. We hope the dataset statistics, the blind study, and the controlled comparisons address the points you raised, and we are happy to clarify further.

AuthorsOpenReview ↗

Our Kind Reminder to Reviewer wWyN: Summary of Our Response (GraftSet-Video, User Study, Trade-off, SFT Controls)

Dear Reviewer wWyN,

Thank you again for your supportive and constructive review. As the discussion period is nearing its end, we briefly summarize our response to your questions. The General Response outlines the starting point of GraftOOD and collects the additional results.

  • GraftSet-Video construction (W1 & Q1; Response (1/2), A1). The corpus has 9,588 videos for 2,397 prompts, with four videos from four different generators per prompt. We report the selection criteria, content categories (human activities, animals, vehicles, and object motion), per-source statistics, and prompt-list separation from training and evaluation. On the same prompts, exemplar DD is 80.14, versus 53.61 for the base model, and training on these prompts without exemplars raises DD only to 10.21.
  • Blind user study (W2 & Q2; Response (1/2), A2). Eight raters (lab members who are not authors) compared full videos on motion sufficiency, temporal coherence, physical plausibility, subject preservation/deformation, and overall quality, with 400 judgments per comparison. GraftOOD is preferred in overall quality over FlowAWR (50.8% wins vs. 30.0% losses) and FlashOOD (92.0% vs. 3.8%). FlowAWR's nearly static outputs are preferred on temporal coherence and slightly on subject preservation, while GraftOOD wins all five dimensions against FlashOOD.
  • Source of the trade-off (W3 & Q3; Response (2/2), A3). Introducing GraftSet prompts without exemplars already lowers HPSv3 from 8.10 to 7.12. At a fixed online count, adding exemplars raises DD from 6.27 to 44.41 while HPSv3 rises slightly and VR is unchanged. With only the top-1 exemplar at ρ=0.167\rho=0.167, DD reaches 30.58 and VR 5.52, above FlowAWR's 5.11, so greater dynamics does not necessarily require lower VR.
  • SFT controls (Q4; Response (2/2), A4). With the same prompts and exemplars, SFT only yields limited gains, and subsequent FlowAWR reduces DD from 50.21 to 7.17. An append control (FlowAWR + λ\lambda·SFT) keeps all eight rollouts and adds the exemplars as a separate SFT loss at GraftOOD's relative weight; it ends below GraftOOD on DD, VR, and HPSv3.

We hope that our response resolves your concerns, and we would be grateful if you could take it into account when reconsidering your final recommendation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Our General Response: Starting Point, Significance, and Additional Results

We thank all reviewers. Reviewers found motion collapse a practically relevant, credible, and empirically supported failure mode (ncVo, wWyN, sxz4); the method simple and practical (ncVo, sxz4); and the unified regression formulation clean (wWyN, sxz4).

Starting point

Video rewards evaluate only the final video. When the proxy reward does not adequately constrain temporal behavior, optimizing it can raise the reward while reducing dynamics. This motion collapse appears when optimizing HPSv3 or VideoAlign Motion Quality, and recurs on Wan2.1-T2V-14B; adding a RAFT motion reward mitigates but does not prevent it. Rather than further reward engineering, we keep reward optimization and add external videos as sequence-level references for the behavior the reward under-constrains. This raises three obstacles: exemplars from closed-source generators are terminal-only, with no denoising trajectories, transition likelihoods, or velocity field, so trajectory-based policy gradients cannot use them; they should not shift the online reward baseline; and their influence must be set relative to online samples.

GraftOOD applies advantage-weighted velocity regression to a prompt group containing two sources, online rollouts and re-noised exemplars. Within this group definition: (i) treating exemplars as a uniformly top-reward reference gives an uncentered unit coefficient, which cancels the unavailable reference field voodv^{\rm ood} and leaves a computable flow-matching target; (ii) exemplars are excluded from reward normalization, so online advantages are computed from online rewards only; (iii) their weight is the group fraction δ=nood/G\delta=n_{\rm ood}/G, set through data composition (ρ\rho, noodn_{\rm ood}).

All three components follow from one group definition rather than separate additions. On Wan2.1-1.3B, GraftOOD raises Dynamic Degree (DD) from 4.86 to 44.41 relative to reward-only optimization (FlowAWR), with VisionReward (VR) close (5.11 vs. 5.08).

Significance

  • Beyond combining SFT and RL (ncVo, wWyN, sxz4). The sample-averaged loss decomposes into a reward-weighted regression term and a flow-matching term, but the pointwise optimum v†=(1−ws)vold⋆+wsvoodv^\dagger=(1-w_s)v_{\rm old}^\star+w_s v^{\rm ood} shows what the group learns: wsw_s is the state-dependent source posterior with prior δ\delta, so exemplars shape the target where their noised density dominates. Empirically, SFT before RL does not protect dynamics during RL. GraftOOD regresses toward the in-distribution reward-rectified target and the out-of-distribution exemplar field within one group; an append control keeps the online group intact and adds the same exemplars as a separate SFT loss at the same relative weight. Despite generating more videos, append ends below GraftOOD on DD, VR, and HPSv3.
  • Terminal-only supervision (sxz4). FlashOOD adds exemplar SFT beside Flash-GRPO's single-SDE-transition update. With the same exemplar pool and allocation, GraftOOD reaches DD/VR/HPSv3 of 44.41/5.08/7.32, versus 58.26/1.65/5.31 for FlashOOD. This indicates a better dynamics–reward balance here, not universal dominance.
  • Allocation (ncVo, sxz4). Across all nine positive settings of ρ\rho and noodn_{\rm ood}, DD stays within 30.58–48.68 (FlowAWR: 4.86). The two controls select points on the dynamics–reward trade-off.

Additional results

ExperimentReviewersKey result
SFT only, SFT → FlowAWR, append control (FlowAWR + λ\lambda·SFT)ncVo, wWyN, sxz4SFT → FlowAWR: DD 50.21 → 7.17; append: DD/VR/HPSv3 36.58/4.89/6.96 vs. GraftOOD 44.41/5.08/7.32
Prompt and rollout-count controls (ρ=0.167\rho=0.167)ncVo, wWyN, sxz4DD 10.21 with GraftSet prompts only; 6.27 with four rollouts; 44.41 after adding four exemplars at that online count
Blind full-video study (8 raters)ncVo, wWyNOverall quality: 50.8% wins vs. 30.0% losses against FlowAWR; 92.0% vs. 3.8% against FlashOOD
Wan2.1-T2V-14BncVoFlowAWR: DD 52.43 → 9.17; GraftOOD: DD 48.29, HPSv3 8.83 (FlowAWR 9.81)
GraftSet-Video statisticsncVo, wWyN, sxz4Exemplar DD 80.14 vs. 53.61 for the base model on the same prompts

All Wan2.1-1.3B RL comparisons use the same training prompts, 20-step training rollouts, group size 8, 160 optimizer steps, and the evaluation set and protocol of Table 1. Exemplar-augmented methods share the GraftSet-Video pool and the main allocation (ρ=0.167\rho=0.167, nood=4n_{\rm ood}=4); the append control also matches prompt mixing, timestep sharing, and reward normalization. We do not equate DD with motion quality: the blind study reports all five dimensions, including temporal coherence, where FlowAWR's nearly static outputs are preferred, yet raters prefer GraftOOD overall.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.