- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies why videos generated by text-to-video diffusion models often show less realistic temporal dynamics than real videos. The authors characterize the discrepancy through two statistics: motion magnitude, which measures how much change occurs, and temporal pacing, which describes when the change occurs. Comparing caption-matched real and generated videos from SSv2 and Panda-70M, they observe that LTX-Video, Wan2.1, and CogVideoX all produce less motion than real videos, while the required correction differs across models and samples. Based on these observations, they propose Adaptive Temporal Coordinate Calibration, a training-free method that reparameterizes the temporal RoPE coordinates during denoising. At a chosen denoising step, the method predicts the final temporal statistics from intermediate hidden states, retrieves similar samples from calibration banks of real and generated videos, and adjusts a temporal scale and a pacing warp to move the statistics toward those of real videos. On held-out SSv2 and Panda-70M videos, the method reduces M-Gap by 17.2% and Prog- by 19.0% on average across the three models, with less than 0.7% inference overhead. It is also evaluated on ChronoMagic-Bench and Ref4D-VideoBench.
Critical strengths
S1. Well-motivated problem with a simple, training-free method
Generated videos often show less motion than real videos. The paper makes this measurable through caption-matched comparisons, which show under-motion in all three models, with generated-to-real ratios of 0.32, 0.55, and 0.33 (Fig. 2). It then defines temporal realism through two statistics, motion magnitude and pacing, and calibrates them by reparameterizing the temporal RoPE coordinates. The method needs no training, preserves the frame order, adds less than 0.7% inference overhead, and is applied to three different backbones.
Why it is consequential: The method can be applied directly to existing video models, which makes it practical. If the concerns in W1 and W2 are addressed, it would be a useful tool for improving temporal dynamics at inference time.
S2. Systematic analysis of the readability and controllability of temporal statistics
The paper shows that motion magnitude and pacing can be predicted from intermediate hidden states, and measures how the temporal coordinates change these statistics (App. C.4). These analyses are used to choose when and how to intervene, so each part of the method is connected to an observed problem. The appendix also covers a wide range of analyses, including alternative target constructions, local versus global targets, and hyperparameter sensitivity, and the limitations section openly discusses cases where global corrections are competitive.
Why it is consequential: This analysis gives insight into how temporal dynamics are represented in video diffusion transformers, which may be useful independently of the proposed calibration.
Critical weaknesses
W1. Unclear whether the increase in motion statistics reflects more realistic dynamics
The main claim is that calibration makes temporal dynamics more realistic. However, the calibration targets and the evaluation may not distinguish realistic motion from other temporal changes.
(a) Motion and pacing measures. The magnitude target is computed from pixel differences between frames rather than from motion itself. Pixel differences also increase with flickering, lighting changes, or local texture changes, so increasing toward the real value does not necessarily increase realistic motion. For pacing, summarizes the timing of motion with a single number. As a result, a realistic video whose motion is concentrated late, such as a throw at the end, and a failure case that stays static and then changes abruptly can have similar values.
(b) Evaluation of temporal quality. The supplementary videos suggest that the added motion is not always realistic. In several Wan2.1 examples, the calibrated videos show more flickering or local texture changes rather than more realistic motion. However, the paper does not report standard temporal quality metrics or a user study, and most of the reported metrics measure the same properties that the method calibrates.
Why it is consequential: If part of the increase in comes from flickering or local artifacts, or if the pacing correction changes videos whose timing is already appropriate, the improvements in Tables 1 and 2 may overstate the main claim. (See Q1–Q3.)
W2. Insufficient experimental evidence
(a) Baselines. The CogVideoX baseline videos in the supplementary material are nearly blank, which raises the question of whether the baseline was run properly. In addition, the method is compared with the base models alone, without any existing training-free method for improving motion.
(b) Mixed results. The improvements do not hold consistently. For every model, at least one metric worsens after calibration, especially on the external benchmarks (Tables 1 and 2).
(c) Benefit of per-sample calibration. The main contribution is adapting the correction to each video. However, compared with applying the same correction to every video, the adaptive version is better on ChronoMagic (0.576 vs. 0.808), similar on the held-out set (1.087 vs. 1.080), and worse on Ref4D (1.634 vs. 1.473) (Tables 13 and 18).
(d) Reliability of the targets and per-model design. The per-sample targets are built from caption-matched pairs of real and generated videos, and from neighbors retrieved for the current video. However, the gap within a pair may not always reflect a limitation of the generator. For example, real videos for "Pushing something from left to right" can show either a quick or a slow push, so a generated video with a slow push may appear to under-move simply because its paired real video is quick. It is also unclear whether the retrieved neighbors are relevant to the current video. In addition, several components are set separately for each model, and the offline cost of building the banks is not reported.
Why it is consequential: These comparisons are the main evidence for the method. With a possibly faulty baseline, no comparison with existing methods, and mixed results, it is unclear whether the calibration reliably improves over the base models, and whether the per-sample components and their cost are justified. (See Q4–Q9.)
Questions for authors
I find the problem setting valuable. The paper treats the magnitude and pacing of motion as concrete problems, and each part of the method is designed to address them, which I find meaningful. This is an important problem for video generation. The analysis is systematic, and the effort to connect each observed problem to a specific solution is clear throughout the paper. However, I have concerns about the experiments and the evaluation, and from the current results it is unclear whether the calibration works as intended. My questions below focus on these points, and the answers could affect my assessment.
[R] = additional result requested, [C] = clarification.
1. Realistic dynamics (W1)
-
Q1 [R/C] (a). The paper notes that captures "related but distinct" variation from optical-flow magnitude (App. C.2), and shows that the improvement holds when evaluated with residual flow. Why was a pixel-based measure chosen as the calibration target rather than a flow-based one, and would a flow-based target change the results?
-
Q2 [R/C] (a). A video in which something is thrown at the end and a video that stays static and then changes abruptly at the end can both have their motion concentrated late, and thus similar values. The former may be realistic, whereas the latter is the kind of failure that the introduction describes (L36). How does the method distinguish the two? Could the authors show such examples before and after calibration?
-
Q3 [R] (b). In the Wan2.1 examples, the added motion is not always realistic or related to the prompt.
- For "Putting trivet underneath candle holder," the calibrated video flickers more than the baseline even while the scene is static. Later, a hand appears abruptly and lights the candle, which is already lit.
- For the snowy-mountain prompt, the main change is larger and more frequent snowflakes.
- For the explosion prompt, the main change is a bubbling texture on the fireball.
Apart from per-frame Imaging Quality, the reported metrics measure motion magnitude and pacing, and two of them (-Gap and -) use the same statistics as the calibration. Could the authors report standard temporal quality metrics, such as VBench Subject Consistency, Background Consistency, Temporal Flickering, and Motion Smoothness, along with a text-alignment metric? A user study would also help.
2. Experimental evidence (W2)
-
Q4 [C] (a). App. A.3 states that the official inference configuration is used, but all four CogVideoX baseline videos are nearly blank. In my experience with the official CogVideoX-5B setting, such outputs are uncommon. What are the exact resolution, number of frames, and guidance scale? The SSv2 prompts are also short (e.g., "Putting a black pen on a surface"), while CogVideoX recommends prompt extension. Was prompt extension applied, and how often does the baseline produce nearly blank outputs?
-
Q5 [R] (W2a). The experiments compare the calibrated models with the base models alone, without any existing method for improving motion or temporal dynamics. How does the method compare with existing training-free approaches, such as FlowMo (cited in Sec. 2.2)? FlowMo has public code and is evaluated on Wan2.1-1.3B and CogVideoX-5B, two of the backbones used in this paper, so such a comparison seems feasible.
-
Q6 [C] (b). Several metrics worsen after calibration. Ref4D Prog- increases for LTX-Video (0.0210 → 0.0278) and Wan2.1 (0.0261 → 0.0341). For CogVideoX, Ref4D Motion decreases (69.85 → 68.83), and Flow- Gap increases on both Ref4D (0.3569 → 0.4332) and the held-out set (0.5845 → 0.5945). Imaging Quality decreases for Wan2.1 (0.6919 → 0.6650 in Table 1). Could the authors discuss these cases?
-
Q7 [C] (W2d). Both the analysis in Sec. 3 and the calibration banks rely on caption-matched pairs, where each real video is compared with a video generated from its caption. However, the paper does not describe how the captions were obtained. In the examples in Table 19, the SSv2 prompts are short action labels (e.g., "Putting trivet underneath candle holder") that do not specify the speed or extent of motion, and the Panda-70M prompts mainly describe the appearance of the scene. Could the authors describe how the captions were obtained, and discuss whether they are detailed enough to determine the expected motion? Since the paper notes "substantial sample-level variation" in these pairs (App. C.1), it would also be helpful to show how much varies across seeds of the same prompt.
-
Q8 [R] (W2d). The signature averages hidden-state changes over spatial positions and channels, so it may carry little information about the content. For example, if the generator produces nearly static videos for both "Throwing something" and "Pushing something from left to right" in SSv2, the two may have similar signatures and retrieve each other as neighbors. However, the corresponding real videos differ in motion, so the two would require different corrections. Could the authors show the retrieved neighbors for such example queries, and whether these videos actually receive different corrections (e.g., different values of and )?
-
Q9 [C] (W2d). The intervention steps (Table 6), the bank size and neighbor count (Table 9), and the ranges of and (App. B.4) are set per model. How were they chosen, and how sensitive are the results to them? In addition, the paper reports less than 0.7% inference overhead (Table 5), but applying the method to a new model also requires generating videos for two banks and running response sweeps. What is this offline cost for each model? Since a single global correction performs comparably on some benchmarks (W2(c)), this cost is important for judging the practical benefit of the method.
Suggestions (not affecting the assessment)
- Fig. 5 uses symbols such as , , , , and that are not defined in the text.
- The distance used for Prog- (e.g., cosine or L2 between DINOv2 features) is not specified.
9 comments
Response to XC5w (2/4): Corrected CogVideoX Evaluation and FlowMo Comparison (Q4–Q5)
Q4 [C] — CogVideoX generation settings and nearly blank outputs
We have corrected the CogVideoX generation pipeline and regenerated both Baseline and Ours using the recommended inference configuration. The revised results confirm improvements in motion magnitude, temporal pacing, and benchmark-specific motion scores.
Our investigation identified deviations from the recommended pipeline. The original experiments used 720 × 480 resolution, 49 frames (8 FPS), guidance scale 6.0, 50 inference steps, and bfloat16 precision, but dynamic CFG was disabled and the original short dataset captions were used without prompt extension. The original pipeline also used the CogVideoX DDIM scheduler.
We corrected these settings by enabling dynamic CFG, adopting extended prompts, and using the DPM trailing scheduler. Both Baseline and Ours were regenerated under the same corrected configuration.
The updated results show:
Held-out evaluation:
- M-Gap: 0.925 → 0.813 (−12.1%)
- c1-W1: 0.0297 → 0.0206 (−30.6%)
- Dynamic Degree: 0.830 → 0.880
Ref4D:
- Motion: 67.46 → 67.81
- Prog-W1: 0.0817 → 0.0750
ChronoMagic:
- MT: 3.12 → 3.24
- Flow-M Gap: 1.010 → 0.715 (−29.2%)
These results replace the earlier CogVideoX evaluation and provide evidence of calibration benefits under the corrected generation pipeline.
Some metrics remain model- and benchmark-dependent, including held-out Flow-M Gap (0.225 → 0.293). We report these results transparently in the revised manuscript.
Revision: Tables 1–2 and Sec. 5.1.
Q5 [R] — Direct comparison with FlowMo
Our new FlowMo comparison demonstrates that Ours achieves stronger held-out motion-magnitude alignment and better semantic progression on ChronoMagic, while also showing complementary benefits when combined with FlowMo.
We evaluated Baseline, FlowMo, Ours, and FlowMo + Ours on Wan2.1-T2V-1.3B using matched prompts and seeds.
On the held-out set, Ours outperforms FlowMo on:
- M-Gap: 0.810 vs. 1.017 (20.4% lower)
- Flow-M Gap: 1.251 vs. 1.366 (8.4% lower)
- Dynamic Degree: 0.700 vs. 0.560
Combining FlowMo with Ours further reduces M-Gap to 0.801 and c1-W1 to 0.0227.
On ChronoMagic, Ours achieves better semantic progression (Prog-W1: 0.0587 vs. 0.0697), while FlowMo produces a lower Flow-M Gap (0.515 vs. 0.553).
On Ref4D, the combination achieves the lowest Flow-M Gap (1.550), demonstrating that the two approaches can be complementary.
These comparisons highlight the distinction between the methods: FlowMo targets motion coherence through inference-time guidance, whereas Ours directly calibrates motion magnitude and pacing toward real-video statistics.
Importantly, our method provides measurable improvements over an existing training-free alternative without additional denoising steps, strengthening the evidence for its practical contribution.
Revision: Appendix F.4, Table 26.
Response to XC5w (3/4): Understanding Metric Discrepancies and Caption Variability (Q6–Q7)
Q6 [C] — Why do some metrics worsen after calibration?
Our additional component-wise and real-video analyses identify concrete sources of disagreement between temporal calibration objectives and independent evaluation metrics. These findings clarify the behavior of the method beyond simply reporting improvements in targeted statistics.
(1) Ref4D semantic progression.
Although Ours improves motion alignment on Ref4D, Prog-W1 increases for LTX-Video (0.0210 → 0.0278) and Wan2.1 (0.0261 → 0.0341).
To investigate this discrepancy, we separately evaluate magnitude and pacing calibration on LTX-Video.
- Baseline Prog-W1: 0.0212
- Stage I only: 0.0191
- Stage II only: 0.0500
The Stage-I improvement demonstrates that motion-magnitude calibration can also benefit semantic progression. In contrast, the Stage-II result identifies pacing calibration as a major source of the degradation in this setting.
This distinction is informative because our c1 statistic captures pixel-level temporal allocation, whereas Prog-W1 measures progression using DINOv2 semantic features. The discrepancy is consistent with a domain-dependent mismatch between pixel-based pacing targets and semantic progression.
(2) Wan2.1 Imaging Quality.
We additionally measure Imaging Quality on real videos at matched resolutions to contextualize the observed decrease.
Across SSv2, Panda-70M, and Ref4D, real-video scores are 58.12, 64.58, and 69.88, compared with baseline generation scores of 66.45, 71.99, and 72.88.
Ours obtains intermediate scores of 63.22, 71.12, and 72.24, respectively.
Thus, the decrease in Imaging Quality moves the measured score toward the corresponding real-video distribution. However, because Imaging Quality is not a direct perceptual-realism measure, this observation does not by itself establish improved visual quality.
(3) CogVideoX under corrected inference settings.
Following the configuration correction detailed in Q4, Ref4D Motion improves from 67.46 to 67.81, and Prog-W1 decreases from 0.0817 to 0.0750, while Flow-M Gap increases slightly from 0.466 to 0.504. On ChronoMagic, MT improves from 3.12 to 3.24, accompanied by increases in Dynamic Degree (0.54 → 0.58) and Imaging Quality (0.6118 → 0.6147), although Prog-W1 increases from 0.0154 to 0.0213.
Together, these results demonstrate improvements in motion dynamics and temporal progression under the corrected CogVideoX configuration, while highlighting that different temporal metrics capture complementary properties and may respond differently to calibration. We have incorporated this analysis into the revised discussion.
Revision: Appendix C.5, Table 13; updated Tables 1–2.
Q7 [C] — Caption sources and motion variability across seeds
Our three-seed analysis shows that systematic under-motion remains observable despite substantial variation in individual generated trajectories.
We use the original SSv2 and Panda-70M captions without manual speed annotations. These captions describe actions but do not uniquely specify their duration or motion extent.
To quantify the resulting uncertainty, we evaluated 100 held-out prompts with three generation seeds.
The within-prompt standard deviations of log motion magnitude are 0.68, 0.45, and 0.62 for LTX-Video, Wan2.1, and CogVideoX, respectively.
Nevertheless, the generated-to-real motion-magnitude ratios remain consistently below one across all evaluated seeds:
- LTX-Video: 0.27–0.29
- Wan2.1: 0.45–0.46
- CogVideoX: 0.28–0.35
These results demonstrate that the observed under-motion tendency is not an artifact of a particular generation seed.
Our framework therefore uses caption-matched videos to estimate real-generated temporal distributions, rather than assuming that a caption uniquely determines the correct sample-level motion trajectory.
Revision: Appendix F.8, Table 30.
Response to XC5w (1/4): Realistic Dynamics and Independent Validation (Q1–Q3)
We thank the reviewer for the detailed and constructive feedback. We address each question below.
Q1 [R/C] — Pixel-based vs. flow-based calibration targets
Our flow-based target ablation demonstrates that pixel-based calibration achieves nearly identical motion-alignment improvements to residual-flow-based calibration.
We evaluated 100 held-out LTX-Video generations (Stage I only), replacing the pixel-based magnitude target with camera-motion-compensated residual optical flow. Importantly, we re-estimated both the calibration-bank statistics and the control response for the alternative target.
The two approaches exhibit similar control sensitivities (0.533 vs. 0.537) and strongly correlated estimated corrections (r = 0.81).
More importantly, their generated videos achieve nearly identical results:
These results demonstrate that the computationally simpler pixel-based target is sufficient to achieve comparable motion alignment in the tested operating range, without requiring optical-flow estimation for calibration-bank construction.
The near-identical outcomes partly reflect bounded control parameters; we therefore interpret this result as practical equivalence within the evaluated setting, rather than equivalence of the underlying motion measures.
Revision: Appendix F.1, Table 22.
Q2 [R/C] — Realistic late actions vs. abrupt temporal changes
Our additional qualitative analysis demonstrates that temporal calibration can substantially reduce concentrated abrupt changes while preserving continuous late-stage actions.
We added two before-and-after examples: (i) a continuous pouring action occurring late and (ii) a nearly static scene followed by an abrupt hand movement (Fig. 10).
Both examples use the same correction (α = 1.20, γ = 0.15), yet exhibit different temporal responses.
For the continuous action, c1 remains nearly unchanged (−0.25 → −0.24), and the action remains continuous.
For the abrupt-change example, c1 shifts from −0.44 to −0.24, while the maximum single-step share of temporal change decreases substantially from 0.126 to 0.028 (−77.8%).
This shows that temporal-coordinate calibration can redistribute concentrated motion rather than simply increasing overall motion magnitude. We additionally introduce an abruptness diagnostic (Cmax) to characterize temporal discontinuities beyond c1.
While c1 is not a semantic action classifier, the examples demonstrate meaningful corrections to different temporal behaviors.
Revision: Appendix C.7, Fig. 10.
Q3 [R] — Temporal consistency and text alignment
We extended our evaluation beyond motion magnitude to directly examine temporal consistency, visual stability, and text alignment.
We additionally report VBench Subject Consistency, Background Consistency, Temporal Flickering, Motion Smoothness, and ViCLIP-based Overall Consistency across three backbones and three benchmarks.
For example, on Wan2.1 held-out videos:
These results show that the substantial increase in motion activity is accompanied by a small improvement in text alignment, although some temporal-consistency scores decrease modestly.
Thus, the expanded evaluation establishes a more complete picture of temporal calibration: motion alignment can improve without a corresponding loss of text alignment, while visual-stability trade-offs remain measurable and model-dependent.
Revision: Appendix F.7, Table 29.