acceptodds
Under review as a conference paper at ICLR 2027

DynPrompt-Driven: Unleashing Motion Dynamics in Autoregressive Video Generation

Abstract

Autoregressive video generation has emerged as a promising paradigm for efficient streaming and video synthesis, and has witnessed rapid progress in recent years. However, existing autoregressive video diffusion models are prone to motion dynamics degradation during generation: in certain scenarios, videos become increasingly static as generation proceeds, and later segments tend to exhibit weaker motion than early ones. In this work, we ask a central question: how can we explicitly unleash and preserve motion dynamics in autoregressive video generation, thereby enhancing overall video dynamism? Through empirical analysis, we identify a previously under-explored phenomenon: the temporal variation of text cross-attention gradually decreases during autoregressive rollout, as reflected by a progressive reduction in the KL divergence between consecutive text cross-attention maps. Motivated by this observation, we propose DynPrompt-Driven, a dynamic-prompt-driven framework for autoregressive video diffusion. Specifically, we introduce Temporal Prompt Recalibration (TPR), which dynamically recalibrates static prompt embeddings using the accumulated visual KV cache, enabling different textual tokens to be adaptively activated as time evolves. Furthermore, we design Motion-Guided Gradient Alignment (MGGA), which leverages high-motion query videos to measure the dynamic degree of generated videos, steering distribution matching distillation (DMD) toward high-dynamism directions during optimization. Extensive experiments on video generation benchmarks demonstrate that our method substantially improves dynamic degree over strong autoregressive video generation baselines, while introducing only negligible additional parameters and computational overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autoregressive Video Generation, Video Diffusion Models, Motion Dynamics

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_i49ROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the degradation of motion dynamics during autoregressive video generation and proposes DynPrompt-Driven with two components. Temporal Prompt Recalibration (TPR) uses the existing visual KV cache to predict token-wise scaling weights for the text embeddings at each generation step. Motion-Guided Gradient Alignment (MGGA) computes gradient similarities between generated videos and a small library of high-motion reference videos using the fake score model, and uses these similarities to reweight per-sample DMD losses.

The method is implemented in a Wan2.1-1.3B self-forcing-style distillation pipeline. Experiments include short-video generation, 60-second adaptations of Rolling Forcing and Reward Forcing, component ablations, prompt-removal diagnostics, and a human study.

Critical strengths
  1. Substantial motion improvements supported by controlled component ablations.
    In the short-video ablation, dynamic degree increases from 66.38 to 73.61 with TPR, 71.28 with MGGA, and 83.88 with both. Appendix A states that the configuration without either module receives matched additional fine-tuning, addressing the possibility that the gains merely come from extra training. The 60-second component ablation similarly supports contributions from both modules.

  2. Evidence beyond a single short-video configuration.
    The method improves 60-second dynamic degree from 27.00 to 44.35 on Rolling Forcing and from 62.71 to 73.57 on Reward Forcing. The prompt-removal intervention and human study provide complementary evidence. The paper reports individual quality dimensions and acknowledges modest trade-offs rather than showing only an aggregate score.

Critical weaknesses
  1. The core method is not specified sufficiently for reliable reproduction.
    TPR maps each text-to-cache attention row to a scalar using a learned linear projection, but the cache dimension can vary and the handling of this variation is not explained. The cache layers and heads used are also unspecified. For MGGA, Equation 6 describes sigmoid-based input masking, whereas Appendix C describes a splatted, min–max-normalized motion map without clearly relating the two. The projected-gradient dimensionality and aggregation across Fastfood blocks are also insufficiently specified. These details determine the implemented algorithm rather than being peripheral settings.

  2. The central interpretation extends beyond the strongest available evidence.
    Dynamic degree measures increased motion but does not establish sustained execution of the actions requested by the prompt, particularly in the 60-second experiments. The paper acknowledges this limitation, but some conclusions still suggest improved prompt-motion alignment. In addition, a smaller absolute prompt-removal effect late in a rollout may partly reflect the lower baseline motion level, rather than uniquely identifying reduced prompt influence as the cause of degradation. MGGA’s additional analysis concerns prompt motion semantics, not the motion quality or usefulness of the generated samples. The results support motion enhancement, but the stronger mechanism and alignment interpretations remain less established.

Questions for authors

These questions primarily request clarification of existing implementation and results rather than new large-scale experiments.

  1. TPR implementation:
    Which cache layers and heads are used, and how is an attention row with a potentially varying number of cache tokens mapped to a scalar by a learned linear projection? Please specify the cache-size handling and the hardware/settings used to measure inference overhead.

  2. MGGA implementation:
    How do the motion masks in Equation 6 and Appendix C relate to the actual implementation? Please specify the fake-model loss, parameter subset, gradient-projection dimension, aggregation across blocks, and whether the resulting sample weights are detached during the generator update.

  3. Evaluation protocol:
    Please provide the prompt and seed counts for the 60-second comparisons and clarify the matched generation and adaptation settings. For the prompt-removal diagnostic, what are the sample count, baseline motion levels, and variability of the reported effects?

  4. Interpretation of the alignment signal:
    Do existing training records show an association between MGGA alignment and measured motion in the generated videos, beyond the prompt-label association reported in Appendix C? If not, please scope the interpretation as an empirical weighting proxy and distinguish motion enhancement from prompt-specified action adherence.

10 comments
AuthorsOpenReview ↗

1 Insufficient details on the core TPR implementation and inference overhead

Thank you for requesting these implementation details. TPR adjusts text conditioning based on the visual content already generated. The key distinction is between the complete history cache and the fixed window actually read at each update. Taking the Rolling adaptation as an example, we use the V cache from Transformer block 9 (index 8), retain all 12 attention heads, and concatenate the 128-dimensional features from each head into 1,536 dimensions. Each time, we take the most recent 3 latent frames from the end of the valid history, corresponding to 4,680 visual tokens. TPR itself does not introduce an additional multi-head attention module.

The text passes through a two-layer projection of 4096→4096→4096, and the visual features through a two-layer projection of 1536→4096→4096. After RMSNorm, they are used to compute B×L×4680 text–visual correspondences. Each row associated with a text token always contains 4,680 elements, which are mapped to a scalar by a shared Linear(4680,1), followed by a gating function to reweight the text. Thus, although the total cache length can vary, the input dimension of the linear layer remains fixed, and the same parameters are shared across tokens and generation steps.

The features read here come from the original model's V cache and are then projected into the key features used for TPR matching; they are distinct from the original model's K cache. We will clarify this correspondence and the fixed-window extraction procedure before Eq. (4). Each update uses the original text embedding as its reference, and the visual cache is detached first. The first chunk uses the original text, and the update is skipped when the valid history is shorter than a complete chunk.

Regarding inference overhead, MGGA computes sample weights only during training. During inference, TPR uses the existing cache without adding generator denoising calls. The speed measurements in Table 1 of the paper use a single NVIDIA H100. We follow the official default single-video inference settings of Self Forcing, with a generation batch size of 1. The current inference code uses BF16, outputs at a resolution of 832×480, and decodes 21 latent frames into 81 frames, saved at 16 FPS for a duration of approximately 5 seconds. Each chunk contains 3 latent frames and uses 4 denoising steps, followed by an update to the clean-history cache after chunk generation.

AuthorsOpenReview ↗

2 Unclear implementation of MGGA masks, gradients, and projections

Thank you for pointing out the insufficient clarity between Eq. (6) and the appendix. The specific definition of σ in Eq. (6), and its relationship to the motion-map construction in Appendix C, were not adequately explained. The actual implementation uses the motion map constructed from AllTracker trajectories in Appendix C: we first compute the L2 magnitude of displacement between adjacent time points, then splat at the trajectory coordinates in the destination frame, using weighted aggregation based on the product of visibility and confidence from the adjacent frames. The motion magnitude of the first frame is set to zero. The motion map is min–max normalized over all spatiotemporal positions within each video, with 10⁻⁶ added to the denominator, and then aligned to the latent grid through temporal linear interpolation and spatial bilinear interpolation. The actual implementation does not use sigmoid thresholding. We will make the main text and appendix consistent and clarify this construction.

The motion mask weights the prediction loss at each position, while the model input retains the complete video. Let z denote the clean latent, z_t the noisy latent, ε the sampled noise, and c the prompt condition. Under the current flow parameterization, the loss used to compute the alignment gradients is

Lattr=mean⁡ ⁣[M⊙(fϕ(zt,c,t)−(ϵ−z))2].L_{\mathrm{attr}}=\operatorname{mean}\!\left[M\odot\bigl(f_\phi(z_t,c,t)-(\epsilon-z)\bigr)^2\right].

Here, mean averages over all frames, channels, and spatial positions; it does not divide by the sum of the mask weights. This loss corresponds to L_fake used to extract gradients in the original Eq. (7), and is distinct from L_DMD used to update the generator. The reference and generated videos use the same fake model, share the attribution timestep and noise, and retain their respective prompts. The motion map is detached during this process. The original main text described multiplying the mask by the video input first, which did not accurately describe the actual computation. We will revise Eqs. (6)–(7) to the motion-weighted prediction loss above. The reference gradient is used only to compute alignment weights and does not directly update the fake model.

The gradient comparison uses the weights and bias of the fake model's final linear output head. During computation, the output-head features are retained and detached, and the gradients for this parameter subset are computed from the attribution loss's error signal with respect to the output, without retaining a full backward graph for the entire network. We will specify this subset in the appendix to avoid describing the computation as taking gradients with respect to all parameters individually.

The final projected vector has 512 dimensions. The current implementation partitions the gradient into 512-dimensional blocks, zero-padding the final block when needed. After applying a Fastfood transform to each block, the results of all blocks are summed into the same 512-dimensional vector, followed by frame-length normalization and L2 normalization. The 8 sets of random tables are reused cyclically according to the block index, and the projection random seed is fixed at 2026. Thus, “8 sets of tables” means neither dividing the gradient into only 8 blocks nor concatenating 8 projected vectors. We will add the definition of cross-block summation and clarify the meaning of T in the appendix accordingly.

Sample weights are computed from the cosine similarity between the projected gradients of generated and reference videos. In the current configuration, each GPU generates one training sample. After gathering the scores across GPUs, we first compute B×softmax(s/0.2), where B is the total number of samples being weighted. We then clip the weights at 0.5 and 2.0 and renormalize them so that their mean is approximately 1. The temperature of 0.2 here corresponds to an inverse temperature of 5; we will also correct the statement in the main text that the default inverse temperature is 1. The final weights after renormalization are not necessarily still within the clipping interval. The weights are detached before being multiplied by the corresponding per-sample DMD losses. Therefore, the generator does not backpropagate through the similarity to the reference gradient or the weight-computation path, and no second-order gradients along this path are required.

AuthorsOpenReview ↗

3 Insufficient details on the long-video protocol and prompt-removal statistics(1/2)

Thank you for reminding us to distinguish dynamics, text responsiveness, and action rationality. Below, we separately report the long-video evaluation, prompt-removal diagnostic, teacher-forcing comparison, and segment-wise motion rationality results, specifying which question each piece of evidence addresses.

For reproducibility, the generation configuration in the Rolling Forcing adaptation code uses BF16, 832×480 resolution, 16 FPS, and 3 latent frames per chunk. The denoising timestep list is [1000, 800, 600, 400, 200], with timestep shift=5. The rolling window contains 5 chunks, the clean-history KV cache has a capacity of 24 latent frames per layer, and TPR reads the most recent 3 frames from the end of the valid history. To generate a 60-second video, the code first generates 243 latent frames in complete chunks, decodes them into 969 frames, and then saves only the first 960 frames to ensure consistent output duration.

The training configuration for this adaptation uses AdamW, with a learning rate of 1.5×10⁻⁶ for the generator and TPR and 4×10⁻⁷ for the fake score model. Both use β=(0, 0.999), with a per-GPU batch size of 1. The fake score model is updated every iteration, while the generator and TPR are updated once every 5 iterations. The real score model, text encoder, VAE, and AllTracker remain frozen.

Within each backbone, the baseline and the model with our method start from the same pretrained checkpoint and receive additional training under the same budget: 600 outer iterations and a global batch size of 8. Apart from adding TPR and MGGA, both use the same training prompts, optimizers, and learning-rate settings. Here, 600 refers to outer training iterations, not 600 generator updates. The comparison uses a baseline with matched-budget continued training.

The 60-second comparison in Table 2 follows the long-video prompt settings of Reward Forcing, using the first 128 prompts from MovieGen. Rolling Forcing, Reward Forcing, and their respective configurations with our method use the same prompt list. Each prompt uses 5 inference seeds to generate 5 videos of 60 seconds, yielding 640 videos per method. Within the same backbone, the baseline and the model with our method use identical seeds and initial noise, with matched generation duration, resolution, frame rate, denoising steps, and cache settings. Evaluation follows the VBench-Long protocol in Appendix F: each video is divided into non-overlapping 2-second segments, and the six metrics are reported using the evaluator's aggregation procedure. Drift is computed as the sample standard deviation of the MUSIQ scores over the 30 segments of each video, then averaged across videos. Rolling Forcing's DD increases from 27.00 to 44.35, while MS changes from 98.94 to 98.76. These results show improved dynamics accompanied by a small change in smoothness; by themselves, they do not establish that all motion follows the actions specified in the prompt.

Regarding text influence, the prompt-removal experiment in Figure 1 removes the prompt at 5 seconds and 45 seconds, respectively, and compares the change in average motion intensity over the following 5 seconds. The early/late decreases for Rolling Forcing are 1.7/0.2, respectively, and become 1.9/0.8 with our method.

This diagnostic uses the first 128 MovieGen prompts, with 5 different inference seeds per prompt, yielding 640 paired samples for each method at each removal time. Each pair branches from the same history into continuations with the text retained or removed, sharing the subsequent sampling noise. The interventions at 5 seconds and 45 seconds each start from a history generated normally with the text retained. Motion intensity follows the definition used in Figure 1. The table below reports both the motion level with the text retained and the removal effect.

ModelRemoval timeMean with text retainedMean after text removalMean decrease (original paper value)
Rolling Forcing5 seconds4.302.601.70
Rolling Forcing45 seconds1.601.400.20
With our method5 seconds4.802.901.90
With our method45 seconds2.902.100.80

As you pointed out, a lower initial motion level may limit the absolute decrease at later stages, and this explanation needs to be considered. We therefore interpret the existing prompt-removal results as indicating that a more pronounced text response is retained at later generation stages, and do not treat them alone as the sole proof of the cause of dynamics decay.

Reviewer_jczqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DynPrompt-Driven to address the progressive motion decay problem in autoregressive video generation. The method consists of two modules: Temporal Prompt Recalibration (TPR), which dynamically adjusts text embeddings based on the visual KV cache, and Motion-Guided Gradient Alignment (MGGA), which reweights DMD training samples using gradient directions derived from high-motion reference videos. Experimental results demonstrate significant improvements in motion dynamics for both short-video and 60-second long-video generation.

Critical strengths
  1. The research problem is practically important. The paper addresses motion decay in autoregressive long-video generation and provides targeted motivation through Prompt Removal experiments and Cross-Attention analysis.
  2. The method offers some methodological novelty. TPR dynamically adjusts text conditioning based on historical visual information, while MGGA leverages gradient directions to enhance motion during training. The two modules operate at different stages of the generation framework.
  3. The experimental improvements are substantial. The method achieves a Dynamic Degree of 83.88 for short-video generation. For 60-second videos, it improves the Dynamic Degree of Rolling Forcing from 27.00 to 44.35 and that of Reward Forcing from 62.71 to 73.57, with relatively low inference overhead.
Critical weaknesses
  1. The core computation of TPR contains an unexplained dimensionality issue. According to Equation (4), the length of the Attention Map increases as the KV cache grows. However, the paper employs a learnable linear projection to map this variable-length representation to a scalar without explaining how variable-length inputs are handled. This leaves the mathematical formulation and implementation of a core component insufficiently specified.
  2. The gradient alignment in MGGA does not necessarily capture motion-specific information. Parameter gradients of the Fake Model may encode appearance, scene characteristics, and semantic information rather than motion alone. The 83.7% result reported in the appendix only establishes a correlation between gradient similarity and motion-related textual semantics, without directly demonstrating its relationship with actual video motion intensity. Therefore, the central assumption underlying MGGA remains insufficiently validated.
  3. Improvements in motion metrics do not necessarily indicate better motion generation quality. VBench Dynamic Degree primarily measures the amount of motion and cannot determine whether the generated motion follows the actions and speeds specified in the prompt. Table 4 also shows that Subject Consistency, Background Consistency, and Temporal Flickering scores are all worse than those of Self Forcing, suggesting that increased motion may come at the expense of visual consistency. Furthermore, the 60-second experiments lack systematic time-segmented evaluations of action fidelity, making it difficult to establish whether the method genuinely addresses motion decay in later stages of generation.
Questions for authors

See Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

1 Variable-length inputs and linear projection dimensions in TPR are unspecified

Thank you for pointing out that the input dimensions were not sufficiently specified in the original manuscript. Before computing text–visual correspondence, TPR extracts a fixed-length visual representation from the KV cache, so the input dimension of the subsequent linear layer is fixed. Taking the Rolling adaptation as a concrete example, we use the valid V cache of the 9th Transformer block (index 8), extracting the most recent 3 latent frames from its end. Each latent frame corresponds to 1,560 tokens, giving a fixed total length of 4,680; the features from all 12 attention heads, each with 128 dimensions, are concatenated into 1,536 dimensions. The cache itself can vary in length, but the block passed to TPR has a fixed length. The update is skipped when the valid history is shorter than a complete block, and the first chunk uses the original text.

Let the text have shape B×L×4096 and the selected visual block have shape B×4680×1536. The text and visual features each pass through two projection layers and RMSNorm to produce a text–visual correspondence matrix of shape B×L×4680. Each row, corresponding to a text token, always contains 4,680 elements. A shared Linear layer maps this row to a scalar, which is then passed through a gating function to obtain the reweighting coefficient for that token. Thus, the input length of the linear layer does not grow as video generation proceeds, and separate parameters are not needed for different generation durations.

At each generation step, reweighting is based on the original text embedding rather than being multiplied cumulatively across steps. The visual cache is detached first, and the same projection and gating parameters are shared across tokens and generation steps. What is read here is the original model's V cache, which is then projected into the key features used for TPR text–visual matching; it is not the original model's K cache. The original manuscript refers to the cache-side input in Equation (4) as key vectors. We will clarify that these are the features obtained by projecting the V window described above and add the fixed-window extraction step: the total length of the history cache can vary, while the number of visual tokens participating in matching remains fixed at 4,680. The same linear projection can therefore be used at different stages of generation.

AuthorsOpenReview ↗

2 Whether MGGA gradient alignment actually corresponds to video motion(1/2)

Thank you for raising this question. In MGGA, the motion mask and gradient alignment serve different roles. The motion mask is constructed from the displacement that actually occurs in the video and is used to weight the fake model's prediction loss at each position, after which the parameter gradient of this loss is computed. This increases the relative contribution of motion regions to the gradient comparison while retaining the complete video input. Motion information is provided by changes in trajectories within the video itself. The reference videos depict basic motions against relatively simple backgrounds, which also helps the comparison focus on motion regions.

Gradient alignment further asks whether the adjustments needed by the current model to reduce prediction errors in the motion regions of the generated and reference videos are similar. We feed the noisy latent representations of the two complete videos into the same fake model, using the same timestep and noise but their respective prompts. Each prediction loss is weighted by its own motion mask, and the resulting gradient directions are computed and compared. Closer directions indicate more consistent local learning directions in the current model; we use this to adjust the relative weight of the generated sample in the DMD loss. The reference video provides a direction for comparison, while the generator update is still determined by the weighted DMD loss.

Thus, optical-flow magnitude describes how much a video moves, whereas gradient alignment compares whether the model's learning directions for these motions are consistent. The motion map is normalized within each video, and the gradients are also normalized by direction. Neither a larger motion magnitude nor a larger gradient magnitude therefore directly corresponds to a higher sample weight. We use the training comparisons below to examine whether this choice is useful, and diagnostics on actual videos to examine its relationship with motion performance.

First, we keep TPR, the initial model, the training budget, and the evaluation settings identical, and compare uniform weighting, direct weighting by optical-flow magnitude, MGGA without the motion mask, and full MGGA. The version without the mask retains the same Veo 3 reference library, gradient alignment, normalization, and sample-weight computation. It only sets the loss-weighting masks for both the reference and generated videos to 1, so that prediction errors at all positions contribute equally to the gradient computation; the model inputs remain unchanged. This comparison directly tests whether emphasizing motion regions remains helpful when gradient alignment is already present.

Weighting method (TPR enabled in all settings)DD ↑MS ↑MR ↑
Uniform weighting73.6198.3339.08
Direct weighting by optical-flow magnitude75.0098.4538.51
MGGA (without the motion mask, Veo 3 reference library)77.5098.3639.08
MGGA (full method, Veo 3 reference library)83.8898.6439.66

Compared with the version without the motion mask, full MGGA increases DD from 77.50 to 83.88, MS from 98.36 to 98.64, and MR from 39.08 to 39.66. In this comparison, both the reference videos and gradient alignment are retained; the difference is whether the prediction loss is weighted by a motion mask. It therefore supports the role of motion-region weighting in the gradient comparison. Compared with uniform weighting, full MGGA increases DD from 73.61 to 83.88, MS from 98.33 to 98.64, and MR from 39.08 to 39.66, further examining the effect of the complete weighting method on the final generated results. The direct optical-flow weighting control compares weighting based solely on motion magnitude with weighting that incorporates gradient direction information.

Second, regarding your question about the 83.7% in the appendix, this statistic examines the association between the ranking of video-gradient similarity to the reference gradient and motion-related prompt labels. To further address your concern, we additionally conducted a human evaluation to further demonstrate the effectiveness of the method: we used 100 prompts and generated 5 videos per prompt, fixing settings such as the rule for selecting the reference query. For each video, we paired its gradient's similarity to the reference gradient with its optical-flow motion magnitude, and asked 5 blinded evaluators to rate action fidelity on a scale of 1–5. After mean-centering separately within each prompt, the Pearson correlation coefficient between similarity to the reference gradient and optical-flow motion magnitude is 0.85, and the correlation with action fidelity is 0.94. This result indicates that, under the same prompt, videos with higher similarity to the reference gradient generally better satisfy the action requirements in the prompt, balancing motion enhancement and action fidelity.

AuthorsOpenReview ↗

2 Whether MGGA gradient alignment actually corresponds to video motion(2/2)

The independent ablation in the paper also shows that, without TPR, MGGA increases short-video DD from 66.38 to 71.28 and 60-second DD from 27.00 to 34.21. Training ablations can establish whether this weighting is effective, the control without the mask examines the contribution of motion-region weighting, and the evaluations above demonstrate the relationship with actual video performance. These results show that MGGA can serve as a training signal that favors motion.

Reviewer_q2pYOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses motion dynamics degradation over long rollouts in autoregressive video generation, where videos become increasingly static as generation proceeds. Through paired prompt-removal experiments, KL divergence between consecutive text cross-attention maps, and the declining cumulative fraction of dynamic segments, the authors identify "prompt suppression," a weakening of late-stage text contributions. They then propose DynPrompt-Driven: Temporal Prompt Recalibration (TPR) uses attention between text tokens and the accumulated visual KV cache to recalibrate the static prompt embedding token by token, trained jointly with the causal DiT during DMD; Motion-Guided Gradient Alignment uses high-dynamic query videos and derives per-sample importance weights from the cosine similarity between query and local gradients, motion-aware reweighting of the DMD loss. The method is validated on short and 60-second videos using autoregressive backbones such as Rolling Forcing and Reward Forcing.

Critical strengths

The empirical gains are significant and consistent across settings: short-video Dynamic Degree reaches 83.88, 12.6 points above the next best baseline, while on 60-second videos Rolling Forcing and Reward Forcing improve by 17.35 and 10.86 points respectively, with reduced drift. The module ablations and the static/fixed/dynamic prompt-strategy ablations cleanly isolate the contributions of TPR and MGGA, and the 20-participant user study corroborates the automatic metrics. The engineering cost is low: TPR adds only 47M parameters with negligible impact on throughput, and MGGA introduces no trainable parameters, making the method practical to deploy.

Critical weaknesses
  1. The causal diagnosis is only correlational: prompt removal and decreasing KL divergence cannot rule out competing explanations — Self Forcing++ attributes motion loss to autoregressive error accumulation, and no discriminative experiment (e.g., ground-truth history at late stages) is provided.

  2. MGGA's query library relies on 100 videos synthesized by the closed Veo 3 model, raising reproducibility and distribution-bias concerns.

  3. Dynamic Degree judges whether a video is dynamic solely from optical-flow magnitude, measuring motion amount rather than naturalness, which can be gamed by artificially amplifying motion; imaging quality also degrades slightly on long videos.

Questions for authors
  1. How do you rule out autoregressive error accumulation as the primary cause? Can you provide a late-stage teacher-forcing discriminative experiment?

  2. How do you verify that the Dynamic Degree gains do not stem from unnatural motion amplification? Please provide optical-flow distributions and motion-naturalness evidence.

  3. How can the Veo 3 query library be reproduced, and is an open-source alternative available?

4 comments
AuthorsOpenReview ↗

1 Autoregressive Error Accumulation Cannot Be Ruled Out as an Explanation

Thank you for this suggestion. We consider the weakening influence of text conditioning to be one possible cause of declining dynamics in long videos, and we also consider that it may act jointly with autoregressive error accumulation. Our question is whether the model's response to text at late stages of generation still differs after error accumulation in the generated history is reduced. The prompt-removal experiment and attention analysis in the paper support this interpretation from different perspectives. As shown in Figure 1, as motion in the video gradually diminishes, removing the prompt from Rolling Forcing at 5 seconds reduces motion intensity over the subsequent 5 seconds by 1.7; removing it at 45 seconds reduces motion intensity by only 0.2, a reduction of approximately 88% in the effect. This indicates that continuing to provide text conditioning has a substantially smaller influence on motion at late stages of generation. Meanwhile, the KL divergence between adjacent text cross-attention maps gradually decreases, indicating that the attention distributions become increasingly similar as generation proceeds. The prompt-removal experiment directly examines the influence of text on motion, while the attention analysis provides supporting evidence; both observations are consistent with the video gradually becoming static. With our method, the effects of prompt removal at early and late stages are 1.9 and 0.8, respectively. The late-stage effect is 4 times that of the baseline, which is also consistent with the stronger preservation of motion. This indicates a strong correlation between the influence of text conditioning and long-video dynamics.

Following your suggestion, we added a late-stage teacher-forcing comparison. The two settings differ only in the preceding conditioning used to generate the next video chunk: standard autoregressive generation uses preceding video chunks generated by the model itself, whereas teacher forcing uses the corresponding conditioning frames from a ground-truth video chunk by chunk; the next chunk being evaluated is still generated by the model. Model weights, prompts, and other generation settings remain unchanged. In both settings, generation proceeds to 45 seconds before the next video chunk is generated. Within each setting, we retain or remove the prompt under the same preceding conditioning and sampling noise, and compare the motion intensity of this video chunk. This tests whether the influence of text on subsequent motion changes when the preceding conditioning frames do not contain errors introduced by repeated model generation, and whether our method remains effective.

Under standard autoregressive generation, the decrease in motion intensity after prompt removal is 0.34 for Rolling Forcing and 0.82 with our method; under teacher forcing, the corresponding values are 0.49 and 0.87. With teacher forcing, the effect of removing text from the baseline increases from 0.34 to 0.49. This indicates that error accumulation and dynamics decay are indeed correlated, and that improving the historical conditioning can restore part of the response to text. Meanwhile, under the same ground-truth conditioning frames, our method still increases this value from 0.49 to 0.87, indicating that improving text guidance remains effective under teacher forcing.

We therefore consider that error accumulation and a weakening influence of text may jointly cause the decline in dynamics. Our work focuses on enabling the model to continue effectively using action information in the prompt at late stages of generation, which is complementary to work on reducing autoregressive error accumulation. The independent TPR ablation also supports this design: short-video DD increases from 66.38 to 73.61, and DD for 60-second videos increases from 27.00 to 36.44, indicating that modifying text conditioning alone can improve motion.

AuthorsOpenReview ↗

2 Reproducibility and Distribution Bias of the Closed-Source Veo 3 Query Library

Thank you for your concern about reproducibility. If the paper is accepted, we will release the complete set of 100 high-motion reference videos actually used in our experiments, together with their prompts, motion categories, and preprocessing configurations. Other researchers can directly use the same reference library to reproduce the experiments without regenerating these videos.

The reference library provides gradient references for the motion-weighted prediction loss in MGGA; the reference videos are not supervision targets that the generator directly fits. Veo 3 is used only to construct this library offline, and the MGGA computation does not depend on Veo 3 itself. We added a comparison using the open-source model Wan2.1-T2V-14B to construct the reference library, keeping the 10 motion categories, 10 videos per category, and other training conditions unchanged. The reference-library comparison is based on Self Forcing: Wan is used only to generate the offline reference videos, while the generative model being trained and evaluated remains unchanged. We consistently report DD, Motion Smoothness (MS), and Motion Rationality (MR): DD reflects the degree of dynamics, MS examines the smoothness of motion between frames, and MR evaluates whether actions and their outcomes are reasonable. Together, these metrics compare how the amount and quality of motion change when the source of the reference library is replaced.

Self Forcing ConfigurationReference LibraryDD ↑MS ↑MR ↑
Original baselineNot used66.3898.2438.51
TPR + uniform weightsNot used73.6198.3339.08
TPR + MGGA, full methodVeo 383.8898.6439.66
TPR + MGGA, alternative reference libraryWan2.1-T2V-14B78.3398.1539.08

The DD and MS values for the original baseline and the full method, as well as the DD value for TPR + uniform weights, come from the paper's 5-second video experiments; MS for uniform weights, the Wan reference-library results, and MR are from the additional evaluation in this response. The alternative reference-library configuration uses the same training budget as the default configuration, and comparisons are made under the same prompts and sampling settings for each metric. Different metrics use their respective evaluation settings.

Compared with TPR + uniform weights, using the Wan reference library increases DD from 73.61 to 78.33, leaves MR unchanged, and changes MS from 98.33 to 98.15. This indicates that Veo 3 is not necessary for improving motion, while the choice of reference library affects the magnitude of the gains; the current Veo 3 reference library performs better on these three metrics.

Regarding distribution bias, we understand your concern to be whether the reference library favors the visual style of a particular generative model or covers only a few types of motion, thereby affecting how MGGA works on other videos. When constructing the library, we sought to present clear basic motions and reduce background distractions. The purpose is to make the reference gradients more strongly reflect prediction errors in moving regions. Our reference library covers 10 typical motion categories, with 10 videos in each, preventing a single action from dominating the reference signal. At the same time, the construction procedure itself cannot replace validation, so we use the source-replacement comparison above to directly examine whether the improvements depend on videos generated by Veo 3.

These 10 motion categories organize a category-balanced reference set; they are not a fixed action classification space of the algorithm. MGGA compares the gradients of the motion-weighted losses for generated and reference videos, without first classifying a generated video into one of these categories. The current number of categories is therefore a library-construction choice. Adding new motion references does not require changing the network or the weighting rule, and the adequacy of the reference content is examined by expanding the library's size and category coverage.

We also expanded the reference library to 1000 videos and broadened and refined its coverage of motion types. Under the same evaluation settings, DD decreases by 0.7, MS decreases by 0.27, and MR increases by 0.45. The expanded library produces relatively small changes in motion rationality, degree of dynamics, and smoothness. Considering these metrics together with the ease of sharing the reference library, we therefore use the current smaller, category-balanced, and easily shareable library of 100 videos. The source-replacement comparison examines dependence on Veo 3, while the size and category-expansion comparison examines the effect of reference-library coverage; releasing the reference videos actually used in the experiments will allow other researchers to directly reproduce the same setting.

AuthorsOpenReview ↗

3 Dynamics Metrics May Be Inflated by Unnatural Motion

Thank you for raising this issue. We aim to increase motion that is consistent with the scene and action requirements, so the evaluation needs to answer three questions together: whether the video contains more motion, whether that motion is smooth, and whether the actions and their outcomes are reasonable. In addition to DD, we added Motion Rationality from VBench-2.0 to examine whether actions are actually performed and produce outcomes consistent with common sense, and we also report Motion Smoothness.

ModelBaseline Motion Rationality ↑With Our Method ↑Change
Self Forcing38.5139.66+1.15
Rolling Forcing36.2136.78+0.57

With our method, motion rationality scores increase slightly for both frameworks. This provides additional evidence regarding motion rationality alongside the increase in dynamics. We also report Motion Smoothness to examine, from the perspective of inter-frame motion continuity, whether increased motion is accompanied by less smooth changes:

ModelVideo DurationBaseline Motion Smoothness ↑With Our Method ↑
Self Forcing5 seconds98.2498.64
Rolling Forcing5 seconds98.5798.51
Rolling Forcing60 seconds98.9498.76

In the paper's 5-second experiments, Self Forcing's DD increases from 66.38 to 83.88, while MS increases from 98.24 to 98.64; increased motion is not accompanied by a decline in smoothness. In Rolling Forcing's 60-second experiments, DD increases from 27.00 to 44.35, while MS changes from 98.94 to 98.76, a decrease of 0.18 points. We therefore describe the results according to the actual performance of each framework: Self Forcing improves both the degree of dynamics and smoothness, while Rolling Forcing's long videos exhibit a small change in smoothness alongside substantial dynamics gains. The added MR evaluation provides further evidence regarding action rationality.

Regarding the optical-flow distributions you requested, we further examined whether the increase in motion is accompanied by a small number of particularly large motions. The two methods use the same test prompts and sampling settings. Videos are standardized to 832×480, and the full-frame average optical-flow magnitude is computed for each pair of adjacent sampled frames. The median optical-flow magnitudes for Rolling Forcing and our method are 1.67 and 2.72, respectively, reflecting the typical level of motion; the 95th percentiles are 7.81 and 8.13, respectively, characterizing larger motion magnitudes. All values are in pixels per sampled frame. In these results, the median has a larger relative increase, indicating that the typical level of motion improves while the increase in larger motion magnitudes is relatively limited. We examine the full distributions for abnormally large motions and use them together with the Motion Smoothness and Motion Rationality results above as complementary assessments of motion magnitude, motion smoothness, and action rationality, respectively.

Regarding the changes in imaging quality you noted, in Table 2 of the paper, Rolling Forcing's Imaging Quality changes from 71.34 to 70.98, while DD increases from 27.00 to 44.35; Reward Forcing's Imaging Quality changes from 69.88 to 69.71, while DD increases from 62.71 to 73.57. We report these changes separately from the motion metrics so that both the dynamics gains and the changes in image quality can be directly compared.

Dear Reviewer q2pY,

We would like to follow up on our responses to your three main concerns. We have provided a late-stage teacher-forcing comparison, an experiment using an open-source model to construct the reference library, and optical-flow statistics alongside additional motion-quality evaluations.

Reviewer i49R has reviewed our responses and updated their rating.

Could you please review our responses before the discussion period closes and let us know whether these additions address your concerns, and whether any specific questions remain? We are available to further clarify the experimental settings, results, and their interpretation.

Thank you for your thoughtful questions and the time you have devoted to reviewing our work.

Comments on the paper

1 comment

Dear reviewers,

As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. If you have not yet responded, please indicate whether your concerns have been adequately addressed. Many thanks!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.