acceptodds
Under review as a conference paper at ICLR 2027

Self-Reflecting Video Diffusion with Boosted Motion Dynamics

Abstract

Modern video diffusion models have demonstrated strong performance, enabling the generation of high-fidelity and temporally coherent videos. A key component behind this success is temporal attention, which enables tokens to exchange information across frames to maintain consistency over time. However, we argue that this very mechanism also contributes to limited motion dynamics, a persistent failure mode where generated videos remain overly static. Specifically, we find that weak motion is closely related to tokens over-attending to other frames, a pattern that emerges as early as the initial denoising steps. Building on this observation, we propose Self-Reflecting Optimization (SRO), a training-free framework that optimizes the initial noise to encourage tokens to attend to their own frame, while keeping it close to the original sample and consistent with the Gaussian prior. Since SRO modifies neither the model nor the conditioning signals, it applies to both text-to-video (T2V) and image-to-video (I2V) generation, in stark contrast to existing approaches tied to specific conditioning types. Extensive experiments on both T2V and I2V backbones demonstrate that SRO substantially boosts motion dynamics with only marginal degradation in text-video alignment. Our project page, including additional demo videos, is available at https://sro-psi.vercel.app/.

Primary area:
generative models
Keywords:
generative model, video diffusion model, motion dynamics

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_F59NOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Self-Reflecting Optimization (SRO), a motion-enhancement method applicable to both text-to-video and image-to-video models. The paper argues that although temporal attention in video diffusion models helps preserve subject identity, appearance, and scene consistency through cross-frame information exchange, excessive cross-frame dependence can suppress temporal variation and lead to overly static videos. Before the actual sampling process, SRO optimizes the initial Gaussian noise to increase the proportion of within-frame attention in an early Transformer layer. The method is evaluated on several models, including Wan2.1-1.3B, Wan2.1-14B, CogVideoX-5B, and Wan2.1-I2V, using T2V-CompBench, Gemini-generated motion-intensive prompts, and VBench-I2V. The results show that SRO substantially improves motion-related metrics such as RAFT Flow, DEVIL, and DINO-Segm while introducing only modest additional inference time. However, some metrics related to text alignment, subject consistency, and temporal stability decline.

Critical strengths
  1. The paper is clearly motivated, and the core observation, optimization objective, and constraint design are well aligned.

  2. The proposed method demonstrates good generality and practical applicability. SRO does not require retraining the video generation model. Instead, it directly optimizes the initial noise before the actual sampling process, allowing the same framework to be applied to both text-to-video and image-to-video generation.

  3. The improvements in motion dynamics are substantial, and the experiments include multiple forms of validation.

Critical weaknesses
  1. The existing motion metrics cannot adequately distinguish meaningful motion from unstable or distorted inter-frame variation, while several quality-related metrics consistently deteriorate. RAFT Flow, DINO-Segm, lower DINO similarity, and lower SSIM all reward greater inter-frame variation. However, such variation may result from correct subject motion, but it may also be caused by identity drift, structural deformation, camera shake, flickering, or physically implausible changes.
  2. The practical efficiency of the per-instance optimization procedure remains unclear. Images and scenes with different content may require different initial-noise perturbations. It is therefore unclear whether SRO must be executed separately for every input condition to identify an appropriate perturbation. Such per-instance optimization could affect the method’s practical runtime and scalability.
Questions for authors
  1. Could the authors repeat the correlation analysis between the initial self-reflecting score and the motion dynamics of the final video across a larger and more diverse collection of prompts and motion categories?

  2. Could the authors include compute-matched comparisons with best-of-NN initial-noise selection, random low-frequency perturbations, constrained noise optimization without the self-reflecting loss, and alternative initial-noise optimization objectives?

3 comments
AuthorsOpenReview ↗

Response to Reviewer F59N (1)

We thank Reviewer F59N for the constructive comments. In response, we directly measured artifacts such as flicker, identity drift, and camera shake to verify that the increased motion is meaningful, clarified the cost of per-instance optimization, repeated the score-motion correlation analysis on new prompts covering diverse motion categories, and added compute-matched comparisons with alternative strategies, including Best-of-NN selection and alternative optimization objectives. We respond to each point below.


W1. The existing motion metrics cannot adequately distinguish meaningful motion from unstable or distorted inter-frame variation, while several quality-related metrics consistently deteriorate. RAFT Flow, DINO-Segm, lower DINO similarity, and lower SSIM all reward greater inter-frame variation. However, such variation may result from correct subject motion, but it may also be caused by identity drift, structural deformation, camera shake, flickering, or physically implausible changes.

To address your concern, we directly measured each failure mode raised by the reviewer, and found that SRO's additional motion is not primarily explained by such artifacts: despite producing substantially stronger motion than MotionCFG, SRO exhibits fewer artifacts and remains close to the standard sampler.

On Wan2.1-1.3B, we measured flicker and deformation using flow-compensated warping error, identity drift using DINOv2 similarity to the first frame (separately for the CLIPSeg-masked subject and the entire frame), camera shake using temporal jitter in the optical-flow field, and structural integrity using a blind Qwen2.5-VL judge.

MethodWarping error ↓Subject identity ↑Global identity ↑VLM integrity ↑Camera Shake ↓
Standard0.01660.8470.8494.030.918
MotionCFG0.01930.8010.7903.841.185
SRO0.01740.8330.8863.970.922

Compared to MotionCFG, SRO achieves lower warping error, better identity preservation, and higher structural integrity, while staying close to the standard sampler on all four measures. If the additional motion were mainly due to flicker or deformation, these artifact-specific metrics would degrade substantially, which is not the case.

Regarding the decrease in quality metrics, we show that it largely reflects an inherent motion-quality trade-off also present in standard generation and can be mitigated by the trust-region radius (see our response to Reviewer sB6c, W2 & W3); moreover, human evaluators preferred SRO in visual quality over both baselines (Table 9 in our revised version).


W2. The practical efficiency of the per-instance optimization procedure remains unclear. Images and scenes with different content may require different initial-noise perturbations. It is therefore unclear whether SRO must be executed separately for every input condition to identify an appropriate perturbation. Such per-instance optimization could affect the method’s practical runtime and scalability.

SRO is indeed run once per sample, as are other initial-noise optimization methods (e.g., InitNO [1], FreeInit [2], and CNO [3]). However, we argue that this does not limit its practicality, for two reasons: the optimization is lightweight, and it requires no per-input tuning.

First, SRO incurs only a modest overhead, since forward and backward passes are restricted to an early block (ℓ∗=2\ell^* = 2) without backpropagating through the denoising trajectory. As shown in Table 6 in our revised version, the added time is small relative to the full sampling time on all backbones, and is lower than that of FlowMo and FreeInit, which require several times the baseline runtime, as well as MotionCFG with its larger LLM. Second, we use the same hyperparameters (ε\varepsilon and KK) for all inputs without any prompt or image-specific tuning, and this single setting consistently improves motion across all evaluated backbones and motion categories.


References

[1] InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization, CVPR 2024.

[2] FreeInit: Bridging Initialization Gap in Video Diffusion Models, ECCV 2024.

[3] Diverse Text-to-Image Generation via Contrastive Noise Optimization, ICLR 2026.

AuthorsOpenReview ↗

Response to Reviewer F59N (2)

Q1. Could the authors repeat the correlation analysis between the initial self-reflecting score and the motion dynamics of the final video across a larger and more diverse collection of prompts and motion categories?

We find that the score-motion relationship in Figure 3 generalizes across various motion types. We repeated the analysis on a new set of prompts covering three distinct motion categories (foreground-only motion, fast background motion, and complex multi-object motion), and in all categories, seeds with a higher initial score consistently produce larger motion in the final video, including subject motion after compensating for camera movement.

To verify this, we generated videos with the standard sampler on Wan2.1-1.3B. Since the initial score S(ℓ∗)\mathcal{S}^{(\ell^{\ast})} at t=1t=1 depends only on the initial noise, we first scored a large pool of random seeds and selected 100 seeds per prompt whose scores evenly span the observed range, without looking at the generated motion. We split the optical flow into subject and background regions using CLIPSeg masks, and additionally report camera-compensated subject motion, obtained by subtracting the global camera motion from the subject flow. We then compute the Spearman correlation between S(ℓ∗)\mathcal{S}^{(\ell^{\ast})} and the motion of the final video.

Motion regimeρ(S,Subject flow)\rho(\mathcal{S}, \text{Subject flow})ρ(S,Background flow)\rho(\mathcal{S}, \text{Background flow})ρ(S,DEVIL)\rho(\mathcal{S}, \text{DEVIL})ρ(S,Camera-compensated subject motion)\rho(\mathcal{S}, \text{Camera-compensated subject motion})
Foreground only+0.56+0.41+0.64+0.54
Fast background+0.91+0.85+0.87+0.78
Multi-object+0.74+0.89+0.87+0.71

All correlations are positive and significant across categories, and remain strong after camera compensation, indicating that the score reflects genuine subject movement rather than camera motion. Moreover, in the foreground-only category, the score correlates more strongly with subject motion than with background motion, consistent with the motion specified by the prompt.


Q2. Could the authors include compute-matched comparisons with best-of-(N) initial-noise selection, random low-frequency perturbations, constrained noise optimization without the self-reflecting loss, and alternative initial-noise optimization objectives?

We conducted all the suggested comparisons, which further validate that SRO's gains stem from its self-reflecting objective, rather than from perturbing the noise or from additional computation.

On Wan2.1-1.3B, we compare SRO with the four alternatives suggested by the reviewer. All optimization-based variants use the same constraint operator (κ=0.25\kappa=0.25, ε=0.003\varepsilon=0.003), iteration budget KK, learning rate, and sampler as SRO. Reversed loss minimizes S\mathcal{S} instead of maximizing it. Feature-difference loss replaces S\mathcal{S} with an attention-free motion proxy, maximizing the normalized difference between adjacent-frame features at the input of block ℓ∗\ell^*. Best-of-NN selects the initial noise with the highest S\mathcal{S} among candidates evaluated within the same runtime budget as SRO.

MethodS(ℓ∗)S^{(\ell^*)}FlowDEVILCLIPVLM qualityTime
Standard0.29512.470.23127.103.63—
Random low-frequency0.2981.930.16926.543.560 s
Reversed loss0.2200.230.11426.573.978.6 s
Feature-difference loss0.33713.830.31426.863.197.4 s
Best-of-N by S (N=70)0.37829.350.41026.843.138.0 s
SRO0.37046.890.46626.242.838.0 s

Random low-frequency perturbations do not increase motion, and reversing the objective suppresses it, showing that the gains come from the direction of the objective rather than the perturbation itself. The attention-free feature-difference loss yields only marginal improvements.

Notably, Best-of-NN by S\mathcal{S} also clearly improves motion, which confirms that S\mathcal{S} is a meaningful motion indicator. Under the same budget, Best-of-NN better preserves quality, while SRO achieves substantially stronger motion at a quality cost consistent with the motion-quality trade-off (see W1 and our response to Reviewer sB6c, W2 & W3). We will include these results in the revision.

Thank you for your response.

Reviewer_sB6cOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes SRO to tackle the "temporal smoothing" effect in video diffusion model. It offers a novel, training-free way to enhance video motion. The proposed method is simple and is applicable to both T2V and I2V generation models. The empirical gains shown in the quantitative analysis on motion-related metrics seems substantial.

Critical strengths

The paper proposes a novel, training-free way to enhance video motion. It addresses an important issue in video diffusion models. The proposed method is simple and is applicable to both T2V and I2V generation models. The proposed method was evaluated on several backbone models with varying model size, and under both T2V and I2V settings. The empirical gains shown in the quantitative analysis on motion-related metrics seems substantial.

Critical weaknesses
  1. The paper used Wan2.1, Wan2.1-14B, and CogVideoX-5B as backbones. Why was HunyuanVideo or Open-Sora, etc not considered? Can SRO be generalized to different types of diffusion models or is it only suitable for certain type(s)? If the method can not be generalized, the claim should be narrowed accordinlgy.
  2. The paper is grounded on a 'key observation: temporal attention, while essential for maintaining temporal coherence, can suppress motion when tokens over-attend to other frames' and suggests increasing self-attention within each frame may produce more meaningful motion. Discussions on the trade-offs and risks that the proposed methodology may introduce should be included.
  3. The paper claims "SRO improves motion dynamics while largely preserving visual quality and semantic fidelity, suggesting that temporal self-reflection can serve as a unified signal for controlling video dynamics across various conditioning types." However, the quantitative results shown (Table 1~4) indicate that SRO lowers several quality measurement scores.
  4. I would suggest to extend the baseline comparison study to include some recently proposed training-free video diffusion enhancement solutions. Currently, the paper mainly focuses on comparison study with FlowMo and MotionCFG. There are quite a few published works on training-free based video diffusion model improvement between 2024 ~ 2026. Even though it may not specifically tackle the 'temporal smoothing' issue, it would be a valuable to add a couple more as baseline.
  5. The user study is a valuable addition to the paper, However, Section A.6 currently lacks sufficient detail, making it difficult to fully interpret the results. I would strongly encourage the authors to expand this section with a more comprehensive description and justification of the study design, procedure, and findings.
Questions for authors

Please see above comments.

5 comments
AuthorsOpenReview ↗

Response to Reviewer sB6c (1)

We thank Reviewer sB6c for the constructive comments. In response, we extended the evaluation to two additional T2V architectures, included further training-free baselines, analyzed the motion-quality trade-off in more detail, and expanded the user study description. We also agree that the quality cost and the risks of excessive intervention should be stated more explicitly, and we have revised the manuscript accordingly. We respond to each concern below.


W1. The paper used Wan2.1, Wan2.1-14B, and CogVideoX-5B as backbones. Why was HunyuanVideo or Open-Sora, etc not considered? Can SRO be generalized to different types of diffusion models or is it only suitable for certain type(s)? If the method can not be generalized, the claim should be narrowed accordinlgy.

We find that SRO generalizes beyond Wan and CogVideoX: on HunyuanVideo-1.5 and Open-Sora 1.3, it improves all motion metrics over both the standard sampler and MotionCFG. For this experiment, we used the same motion-intensive Gemini prompts as in Table 2.

ModelMethodRAFT Flow ↑DEVIL ↑DINO-Segm ↑DINO ↓SSIM ↓CLIPScore ↑PickScore ↑X-CLIP ↑
HunyuanVideoStandard19.930.2790.4150.9410.78326.7521.1122.37
MotionCFG18.890.2870.4040.9350.87126.4720.9722.36
SRO (Ours)33.010.3380.4690.9240.70026.5221.0422.32
Open-SoraStandard2.020.1550.2690.9730.85725.9921.1720.72
MotionCFG4.830.1590.2290.9730.94723.9820.1820.06
SRO (Ours)5.480.1970.3940.9550.65225.6120.9920.56

On both backbones, SRO achieves the best results on all five motion metrics. Text alignment and quality remain comparable to the standard sampler on both HunyuanVideo-1.5 and Open-Sora 1.3.

Together with the original backbones, SRO is now validated on five T2V models spanning different scales and attention designs, including full spatio-temporal attention (Wan), joint text-video attention (CogVideoX, HunyuanVideo), and factorized spatial-temporal attention (Open-Sora), as it only requires attention among tokens of different frames. We will add these results to the revision.

AuthorsOpenReview ↗

Response to Reviewer sB6c (2)

W2. The paper is grounded on a 'key observation: temporal attention, while essential for maintaining temporal coherence, can suppress motion when tokens over-attend to other frames' and suggests increasing self-attention within each frame may produce more meaningful motion. Discussions on the trade-offs and risks that the proposed methodology may introduce should be included.

W3. The paper claims "SRO improves motion dynamics while largely preserving visual quality and semantic fidelity, suggesting that temporal self-reflection can serve as a unified signal for controlling video dynamics across various conditioning types." However, the quantitative results shown (Table 1~4) indicate that SRO lowers several quality measurement scores.

We agree that SRO lowers several quality metrics and that our phrase "largely preserving visual quality" understates this trade-off. We will revise it to "while yielding some compromise in frame-level visual quality." At the same time, we argue that this degradation is not intrinsic to SRO, but is explained by two factors: (i) an inherent trade-off between motion and quality metrics, which also appears in standard generation, and (ii) the choice of optimization strength, which can be tuned to mitigate it. We verify each factor below.

(1) Motion-quality trade-off in standard generation. Even without SRO, stronger motion accompanies lower quality scores. We generated multiple seeds per prompt with the standard sampler on Wan2.1-1.3B, divided the seeds of each prompt into five quantiles by RAFT Flow, and averaged each quantile across prompts.

QuantileRAFT FlowDEVILMUSIQAestheticPickScore
Q16.220.19257.764.98221.28
Q212.340.24155.114.90121.31
Q317.580.26752.754.91121.21
Q427.200.31351.004.82121.09
Q547.840.37848.404.73920.98

MUSIQ decreases monotonically with motion, and Aesthetic and PickScore show similar overall trends. Since the quantiles are formed within each prompt, this trend is not explained by differences in scene content across prompts. Hence, part of the degradation observed with SRO naturally accompanies the increased motion itself.

(2) Controlling the trade-off via ε\varepsilon. The remaining degradation can be mitigated by reducing the trust-region radius ε\varepsilon, which governs the strength of the noise optimization. With a smaller ε\varepsilon (0.0015 instead of 0.002), we obtain the following I2V results:

MethodDynamic Degree ↑\uparrowI2V Subject ↑\uparrowMUSIQ ↑\uparrowAesthetic ↑\uparrow
Standard11.40.95670.695.37
ALG28.60.94569.485.32
SRO (smaller ε\varepsilon)34.30.94669.915.34

With this setting, SRO still achieves a higher Dynamic Degree than ALG while slightly outperforming it in subject fidelity and quality.

Risks of excessive intervention. The main risk of SRO is over-suppressing cross-frame attention, which is essential for temporal coherence. If the optimization is too strong, consistency across frames can break: removing the trust region causes generation to collapse (Table 5 in our revised version), and an overly large ε\varepsilon reduces subject fidelity. For instance, I2V subject consistency drops from 0.946 at ε=0.0015\varepsilon=0.0015 to 0.931 at ε=0.002\varepsilon=0.002. The trust region is therefore a necessary safeguard rather than an optional component, and ε\varepsilon should be chosen to balance motion against quality. We will add this discussion and the analyses above to the revision.

AuthorsOpenReview ↗

Response to Reviewer sB6c (3)

W4. I would suggest to extend the baseline comparison study to include some recently proposed training-free video diffusion enhancement solutions. Currently, the paper mainly focuses on comparison study with FlowMo and MotionCFG. There are quite a few published works on training-free based video diffusion model improvement between 2024 ~ 2026. Even though it may not specifically tackle the 'temporal smoothing' issue, it would be a valuable to add a couple more as baseline.

We find that SRO achieves the strongest motion among a broader set of training-free methods. In addition to FlowMo and MotionCFG, we evaluated three recent training-free methods covering different intervention types: Enhance-A-Video (attention-based), FreeInit (initial-noise-based), and VideoGuide (guidance-based). All methods are evaluated on Wan2.1-1.3B with the same Gemini prompts and seeds.

MethodRAFT Flow ↑DEVIL ↑DINO-Segm ↑DINO ↓SSIM ↓CLIPScore ↑PickScore ↑X-CLIP ↑
Standard12.470.2310.2480.9310.90727.1020.7721.32
FlowMo4.310.1730.1870.9470.94627.1021.0021.57
MotionCFG25.270.2760.3190.9230.91126.6820.6922.35
Enhance-A-Video14.720.2390.2640.9330.91027.1720.6021.24
FreeInit6.610.1800.2040.9420.89926.8420.9022.24
VideoGuide3.080.1220.1330.9700.98325.8920.4121.66
SRO (Ours)46.890.4660.5480.8490.78626.2420.4122.50

SRO achieves the best results on all five motion metrics by a large margin. Notably, FreeInit, which also operates on the initial noise, reduces motion compared to the standard sampler, as it targets temporal consistency rather than dynamics. SRO shows a modest drop in frame-level metrics (CLIPScore and PickScore), consistent with the trade-off discussed in W2 & W3, while achieving the highest video-level text alignment (X-CLIP). We will add these baselines to the revision.


W5. The user study is a valuable addition to the paper, However, Section A.6 currently lacks sufficient detail, making it difficult to fully interpret the results. I would strongly encourage the authors to expand this section with a more comprehensive description and justification of the study design, procedure, and findings.

We agree that the user study should be described in sufficient detail to interpret and reproduce it, and we will expand Section A.6 with the design, procedure, and findings summarized below.

Design. 30 participants compared video pairs, each consisting of SRO and a baseline (the standard sampler or MotionCFG) generated from the same prompt and the same initial noise (before optimization for SRO). Each comparison consists of 10 pairs, with prompts drawn from T2V-Compbench and Gemini same as Tables 1 and 2 and videos generated with Wan2.1-1.3B, Wan2.1-14B and CogVideoX. We adopt a pairwise forced-choice protocol, as it is more reliable than absolute ratings for subtle differences in motion and quality.

Procedure. The two videos were displayed side by side with randomized left-right placement, and the order of pairs was randomized for each participant. For each pair, participants answered three questions: (1) "Select the more dynamic video," (2) "Select the video with better visual quality," and (3) "Select the video that better aligns with the text."

Findings. SRO was preferred over both baselines for all three criteria (Table 9 in our revised version). We will additionally report per-question inter-rater agreement and statistical significance (e.g., a binomial test) in the revised appendix.

Reviewer_QHQoOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work targets the over-static artifact in video diffusion models. The authors attribute weak motion dynamics to excessive cross-frame attention emerging in early denoising steps, and propose Self-Reflecting Optimization (SRO), a training-free scheme that optimizes initial latent noise. SRO promotes within-frame attention while respecting the Gaussian prior and original sampling trajectory. It requires no model modification and supports both T2V and I2V generation. Evaluations across multiple backbones demonstrate improved motion dynamics with minor text-video alignment degradation. Ablations validate core design choices, and the method introduces only modest inference overhead.

Critical strengths
  1. SRO is conditioning-agnostic and training-free. Unlike many motion-boosting techniques tailored exclusively for T2V or I2V, SRO operates solely on initial noise without modifying model weights or condition signals. This plug-and-play nature makes it broadly applicable to existing video diffusion backbones.
  2. The connection between over-attending across frames and suppressed motion provides a clean explanation for static generation failure, and low computational overhead makes the approach practically appealing.
Critical weaknesses
  1. The evidence connecting the self-reflecting score to motion dynamics is insufficiently comprehensive (Figure 3). Current results only cover limited motion types. Without testing diverse motion regimes, including foreground-only motion, fast background motion, and complex multi-object motion, it remains unclear whether the self-reflecting score reliably correlates with motion strength across all scenarios.
  2. The video quality drop shown in Table 3 lacks discussion. While the method successfully boosts motion dynamics, this quality trade-off is not adequately discussed. Without understanding the root cause (e.g., over-optimization of attention signals, temporal flicker, or distortion of the Gaussian prior), readers cannot judge whether this is an inherent limitation or a tunable side effect.
Questions for authors
  1. Could you extend the analysis in Figure 3 to foreground-only motion, rapid background motion and complex multi-object motion, to verify whether the self-reflecting score generalizes reliably?
  2. Table 3 and the first case in Fig.6 shows inferior video quality for SRO compared to other baselines. Is this quality degradation intrinsic to your optimization objective, or can it be mitigated via better regularization or hyperparameter tuning?
  3. Would SRO degrade outputs when the base video diffusion model is already capable of generating videos with naturally large motion?
4 comments
AuthorsOpenReview ↗

Response to Reviewer QHQo (1)

We thank Reviewer QHQo for the constructive comments. In response, we extended the score-motion analysis to diverse motion regimes, analyzed the source of the quality degradation and how to mitigate it, and evaluated SRO on already-dynamic seeds. We respond to each point below.


W1. The evidence connecting the self-reflecting score to motion dynamics is insufficiently comprehensive (Figure 3). Current results only cover limited motion types. Without testing diverse motion regimes, including foreground-only motion, fast background motion, and complex multi-object motion, it remains unclear whether the self-reflecting score reliably correlates with motion strength across all scenarios.

Q1. Could you extend the analysis in Figure 3 to foreground-only motion, rapid background motion and complex multi-object motion, to verify whether the self-reflecting score generalizes reliably?

We find that the score-motion relationship in Figure 3 generalizes across various motion types. Specifically, in all three regimes suggested by the reviewer, seeds with a higher initial score consistently produce larger motion in the final video, including subject motion after compensating for camera movement.

To verify this, we constructed prompts for foreground-only motion, fast background motion, and complex multi-object motion, and generated videos with the standard sampler on Wan2.1-1.3B. Since the initial score S(ℓ∗)\mathcal{S}^{(\ell^{\ast})} at t=1t=1 depends only on the initial noise, we first scored a large pool of random seeds and selected 100 seeds per prompt whose scores evenly span the observed range, without looking at the generated motion. We split the optical flow into subject and background regions using CLIPSeg masks, and additionally report camera-compensated subject motion, obtained by subtracting the global camera motion from the subject flow. We then compute the Spearman correlation between S(ℓ∗)\mathcal{S}^{(\ell^{\ast})} and the motion of the final video.

Motion regimeρ(S,Subject flow)\rho(\mathcal{S}, \text{Subject flow})ρ(S,Background flow)\rho(\mathcal{S}, \text{Background flow})ρ(S,DEVIL)\rho(\mathcal{S}, \text{DEVIL})ρ(S,Camera-compensated subject motion)\rho(\mathcal{S}, \text{Camera-compensated subject motion})
Foreground only+0.56+0.41+0.64+0.54
Fast background+0.91+0.85+0.87+0.78
Multi-object+0.74+0.89+0.87+0.71

All correlations are positive and significant across regimes, and remain strong after camera compensation, indicating that the score reflects genuine subject movement rather than camera motion. Moreover, in the foreground-only regime, the score correlates more strongly with subject motion than with background motion, consistent with the motion specified by the prompt.

AuthorsOpenReview ↗

Response to Reviewer QHQo (2)

W2. The video quality drop shown in Table 3 lacks discussion. While the method successfully boosts motion dynamics, this quality trade-off is not adequately discussed. Without understanding the root cause (e.g., over-optimization of attention signals, temporal flicker, or distortion of the Gaussian prior), readers cannot judge whether this is an inherent limitation or a tunable side effect.

Q2. Table 3 and the first case in Fig.6 shows inferior video quality for SRO compared to other baselines. Is this quality degradation intrinsic to your optimization objective, or can it be mitigated via better regularization or hyperparameter tuning?

Thank you for the insightful comment. We argue that the degradation is not intrinsic to SRO, but is explained by two factors: (i) an inherent trade-off between motion and quality metrics, which also appears in standard generation, and (ii) the choice of optimization strength, which can be tuned to mitigate it. We verify each factor below.

(1) Motion-quality trade-off in standard generation. Even without SRO, stronger motion accompanies lower quality scores. We generated multiple seeds per prompt with the standard sampler on Wan2.1-1.3B, divided the seeds of each prompt into five quantiles by RAFT Flow, and averaged each quantile across prompts.

QuantileRAFT FlowDEVILMUSIQAestheticPickScore
Q16.220.19257.764.98221.28
Q212.340.24155.114.90121.31
Q317.580.26752.754.91121.21
Q427.200.31351.004.82121.09
Q547.840.37848.404.73920.98

Observe that MUSIQ decreases monotonically with motion, and Aesthetic and PickScore show similar overall trends. Since the quantiles are formed within each prompt, this trend is not explained by differences in scene content across prompts. Hence, part of the degradation observed with SRO naturally accompanies the increased motion itself.

(2) Controlling the trade-off via ε\varepsilon. The remaining degradation can be mitigated by adjusting the trust-region radius ε\varepsilon, which governs the strength of the noise optimization. A larger ε\varepsilon yields stronger motion at some cost in frame-level quality and subject fidelity, whereas a smaller ε\varepsilon (0.0015 instead of 0.002) gives the following I2V results:

MethodDynamic Degree ↑I2V Subject ↑MUSIQ ↑Aesthetic ↑
Standard11.40.95670.695.37
ALG28.60.94569.485.32
SRO (smaller )34.30.94669.915.34

With this setting, SRO still achieves a higher Dynamic Degree than ALG while slightly outperforming it in subject fidelity and quality.


Q3. Would SRO degrade outputs when the base video diffusion model is already capable of generating videos with naturally large motion?

We find that SRO does not degrade already-dynamic videos: on such seeds, it only moderately changes the motion and preserves the perceived quality, while it yields large gains on initially static seeds. To examine this, we generated multiple seeds per prompt with Wan2.1-1.3B, grouped them by the optical flow of their standard generations, and applied SRO with the same hyperparameters to both groups. VLM quality denotes a quality score rated by Qwen2.5-VL .

Seed groupMethodFlowDEVILMUSIQVLM quality
High-motionStandard47.50.37748.53.19
SRO50.20.49444.23.18
Low-motionStandard6.50.19157.63.91
SRO29.90.40048.53.37

For high-motion seeds, SRO only slightly increases Flow while improving DEVIL, and keeps the VLM quality nearly unchanged. The decrease in MUSIQ is consistent with the motion-quality trade-off of frame-level metrics observed in our response to W2 (see details therein). For low-motion seeds, SRO substantially increases both Flow and DEVIL, showing that its gains are concentrated on initially static generations.

Thanks for the additional experiments and the careful revision, which have addressed some of my concerns. I will maintain my original score.

AuthorsOpenReview ↗

Response to Reviewer QHQo

Thank you for the response and for acknowledging our additional experiments. We are glad that our revision addressed your concerns, and appreciate that you maintain a positive assessment of our work.

If anything remains unclear, please let us know, and we will address it during the remaining discussion period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their constructive feedback. We are encouraged that the reviewers found SRO to be training-free and conditioning-agnostic, applicable to both T2V and I2V (QHQo, sB6c, F59N), clearly motivated with a clean explanation of static generation (QHQo, F59N), and effective with substantial motion gains at low overhead (QHQo, sB6c, F59N).

Below we summarize how we addressed the main concerns; detailed results are provided in the individual responses, and all changes in the revised PDF are highlighted in blue.

  • Generality of the score-motion relationship (QHQo, F59N). We repeated the analysis on new prompts covering foreground-only, fast-background, and multi-object motion. In every category, the initial score is strongly correlated with the final motion, including subject motion after compensating for camera movement.

  • Generalization to other architectures (sB6c). We applied SRO to HunyuanVideo-1.5 and Open-Sora 1.3, covering full spatio-temporal, joint text-video, and factorized spatial-temporal attention. SRO improves all motion metrics on both.

  • Comparison with more training-free methods (sB6c). We added Enhance-A-Video, FreeInit, and VideoGuide, over which SRO achieves substantially stronger motion.

  • Quality degradation (QHQo, sB6c). We show that the quality drop partly reflects a motion-quality trade-off already present in standard generation and can be mitigated via the trust-region radius ε\varepsilon, the key hyperparameter of SRO. We also discuss the risks of excessive optimization and revised our claim of "largely preserving visual quality" accordingly.

  • Meaningful motion vs. artifacts (F59N). We directly measured flicker, identity drift, camera shake, and structural integrity, and found that SRO introduces fewer artifacts than MotionCFG, the current state-of-the-art, despite producing stronger motion.

  • Source of the gains (F59N). Compute-matched comparisons with random low-frequency perturbations, reversed-loss optimization, an attention-free objective, and Best-of-NN selection by the score confirm that the gains stem from the self-reflecting objective.

  • Cost and user-study details (F59N, sB6c). We clarified the per-instance optimization cost and expanded the user-study protocol in Section A.6, with the aggregate preference rates in the text corrected to match Table 9.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.