- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Drift-Gated TTUR, an adaptive training update scheduler for long-horizon streaming video generation with DMD. The proposed method measures changes in the prediction of the fake model using a small fixed probe set. When the measured drift exceed a threshold, the method skips the generator update and execute additional fake model update. The experiments are conducted using LongLive on VBench.
Critical strengths
-
This paper proposes a simple yet effective method to improve the training stablilty of DMD, requiring no changes to the original DMD objective and inference pipeline.
-
The authors compare the proposed method with uniform, random, and skip-only schedules.
Critical weaknesses
-
Adaptive generator-discriminator update scheduling has already been explored in previous work [1]. The proposed Drift-Gated TTUR follows a similar idea, with the main difference being the application on DMD.
-
It is unclear whether the proposed drift measurement is a reliable indicator of critic tracking difficulty. The measured drift reflects changes in both the generator and the fake-score model, rather than directly measuring the critic's estimation error. The paper also uses only two fixed probe samples (m=2), without examining how the number or selection of probe samples affects the results.
-
The results on VBench is promising, however, Figure 4 shows that the drift remains highly variable throughout training, with noticeably larger fluctuations after the initial stage. However, the paper only shows the drift curve for the proposed method. Without comparing the drift dynamics against Fixed TTUR or other scheduling strategies, it is difficult to determine whether the proposed method actually reduces long-term drift or mainly changes the timing of optimizer updates.
-
Besides, all controlled experiments are conducted using LongLive. Although the authors include comparisons with several official checkpoints, they do not evaluate previous works, like Causal Forcing [2], Causal Forcing++ [3] and etc, or demonstrate that the proposed controller can improve other DMD-based video generation frameworks.
[1] Accelerated WGAN update strategy with loss change rate balancing
[2] Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
[3] Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
Questions for authors
-
Adaptive generator-discriminator update strategies have already been studied in previous work, such as [1]. What is the main difference between Drift-Gated TTUR and these methods beyond the choice of scheduling signal?
-
The proposed drift measurement uses only two fixed probe samples (m=2). How sensitive is the method to the number and selection of probe samples? Would using more samples or different probe sets lead to more reliable scheduling decisions?
-
Figure 4 shows that drift remains relatively high and fluctuates throughout training. Does Drift-Gated TTUR actually reduce the overall drift compared with Fixed TTUR? Could the authors provide drift curves for both methods under the same training settings, along with statistics such as the average drift and frequency of high-drift events?
-
Could the authors provide results from multiple independent training runs? The current confidence intervals only account for inference-time variation.
-
The proposed method is only evaluated within the LongLive training framework. Could the authors explain whether the same strategy can be applied to other autoregressive video distillation methods, such as Causal Forcing?
-
In the first example of Figure 5, the results from Drift-Gated TTUR appear somewhat oversaturated compared with other methods. Could the authors provide more qualitative examples, preferably randomly selected, to show whether this behavior is consistent across different prompts?
[1] Accelerated WGAN update strategy with loss change rate balancing
4 comments
W4/Q5. Applicability beyond LongLive
We thank the reviewer for raising this question. The motivation behind our controller is not specific to LongLive, but arises from the generator–fake-score coupling inherent in DMD. As the generator distribution evolves, the online fake-score model must continually adapt, making the required critic update budget potentially time-varying.
Our method operates at the optimizer-scheduling level and is largely orthogonal to the student architecture, rollout strategy, and preceding initialization stages. It only requires monitoring the online fake-score model through fixed-paired probes and adjusting generator/critic updates, without modifying the DMD objective or inference pipeline.
In particular, Causal Forcing differs from Self Forcing primarily in its initialization pipeline: it introduces an additional teacher-forcing stage to train an autoregressive teacher, followed by causal ODE initialization. The subsequent DMD training procedure follows Self Forcing, making our controller conceptually compatible with Causal Forcing and potentially complementary to its improved initialization, without requiring changes to that pipeline.
To further examine generality, we conducted an additional short-horizon T2V experiment (81 frames, 16 fps, without long-horizon rollout). Drift-Gated TTUR improves VBench Total from 79.64 to 80.90 over Fixed 5:1 and increases Dynamic Degree from 0.153 to 0.597 (Appendix C).
These results show that the scheduling benefit extends beyond long-horizon rollout within the LongLive-based training pipeline. Direct evaluation in Causal Forcing or Causal Forcing++ remains future work.
To further examine whether our improvements depend on long-horizon autoregressive rollout, we conducted an additional short-horizon T2V experiment (81 frames, 16 fps, without long-horizon rollout). Drift-Gated TTUR improves VBench Total from 79.64 to 80.90 over Fixed 5:1 and increases Dynamic Degree from 0.153 to 0.597 (Appendix C).
These results demonstrate that the scheduling benefit extends beyond long-horizon rollout within the LongLive-based DMD pipeline, supporting the broader applicability of the underlying mechanism. Direct evaluation on Causal Forcing and other DMD frameworks remains future work. We have clarified this distinction in Section 3.2.
Q6. Color saturation and qualitative coverage
We added six prompt groups (24 videos) in Appendix H (Figures 9–10), spanning objects, landmarks, and outdoor scenes. All four methods use matched temporal positions and original aspect ratios, without color correction. The examples include both neutral and vivid scenes, broadening direct inspection of color behavior across prompts. Section 5.6 now references these comparisons.
We hope these additions address the concerns and encourage reconsideration of the rating.
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns.
However, I still have some concerns about the reliability and novelty of the proposed scheduling strategy.
First, the new probe ablation shows noticeable sensitivity to the probe configuration. In particular, increasing the probe size from m=2 to m=4 reduces VBench Total from 80.93 to 79.34, despite recalibrating the threshold. This is a relatively large drop compared with the reported average improvement over Uniform Scheduling. Could the authors clarify how the probe configuration affects the frequency of recovery steps and the resulting generator/critic update counts?
Second, although the authors have clarified the difference from previous adaptive GAN scheduling methods, it is still unclear whether fake-score drift provides a meaningful advantage over simpler adaptive signals, such as critic loss changes or gradient statistics.
Overall, I appreciate the improvements in the rebuttal, particularly the additional training runs and drift comparisons. However, the sensitivity to probe configuration and the limited evidence for the distinct advantage of the proposed scheduling signal remain important concerns. I therefore plan to maintain my current rating.
1. Probe-bank sensitivity. We thank the reviewer for encouraging us to look more closely at the controller behavior. Motivated by this comment, we conducted an additional controlled experiment to better understand the interaction between probe-bank size and threshold calibration.
In the original m=4 run, calibration yielded a higher threshold than m=2 (0.1105 vs. 0.0927), accompanied by fewer recovery events (146 vs. 162), more generator updates (254 vs. 238), and fewer recovery critic updates (730 vs. 810). This helped us identify that applying the same calibration rule across probe configurations does not necessarily lead to comparable effective update schedules.
We therefore repeated the m=4 experiment with a threshold approximately matching the default m=2 setting, while keeping all other settings unchanged. VBench Total improved from 79.34 to 80.72, recovering approximately 87% of the original performance drop and narrowing the gap to m=2 (80.93) to just 0.21 points.
These findings suggest that drift-conditioned scheduling is not limited to the original m=2 probe configuration, provided that the threshold is appropriately calibrated. The substantial recovery in performance also suggests that the original degradation was largely associated with changes in the effective update schedule, rather than probe-bank size alone. We appreciate the reviewer bringing attention to this interaction, as it highlights an important consideration when applying the controller across different probe configurations.
We have included this additional control in Section 5.4 and Appendix E.2. Among the tested settings, m=2 remains our preferred choice due to its lower computational overhead, as m=4 increases training time by approximately 23% without a clear improvement in generation quality.
2. Choice of scheduling signal. We thank the reviewer for raising this point. Indeed, our initial investigation began with simpler signals derived from the critic's prediction loss, before we developed the fake-score drift probe. However, our preliminary experiments suggested that these measurements did not reliably reflect errors in the practical DMD generator update.
Specifically, in controlled frozen-generator experiments, the critic's prediction error on held-out samples showed weak, near-zero, or negative correlations with the DMD-gradient deviation from a well-trained reference. In contrast, temporal fake-score drift exhibited strong Spearman correlations of 0.968–0.980 across seeds under the 14B real-score model.
These observations were an important motivation for developing the fixed-paired drift probe. Rather than relying on loss values that can vary with sample difficulty or diffusion timesteps, we monitor changes in on-policy fake-score predictions under controlled sampling conditions, aiming to capture the temporal nonstationarity of the generator–critic system.
We agree that direct end-to-end comparisons with alternative loss- or gradient-based adaptive controllers would further strengthen this evidence, and appreciate the reviewer highlighting this direction.
Overall, we sincerely appreciate the reviewer's constructive feedback, which has helped us improve this work.
W1/Q1: Distinction from prior adaptive GAN scheduling
We thank the reviewer for pointing out Ouyang et al. (2021). We agree that adapting generator–discriminator update frequencies is not new in itself, and have added an explicit citation and discussion in Related Work.
Our contribution differs in the optimization setting, monitoring signal, and intervention mechanism. Ouyang et al. adapt WGAN training using relative changes in generator and discriminator losses. In contrast, DMD relies on an online fake-score model that must track the evolving generator distribution, and its estimation error directly perturbs the practical generator update direction.
Motivated by this structure, we introduce a fixed-paired probe that measures temporal changes in the on-policy fake-score field while holding the probe variables fixed. The controller uses elevated drift to selectively skip generator updates and allocate critic recovery steps, without modifying the DMD objectives.
We therefore position our contribution not as the first adaptive two-timescale schedule, but as a DMD-specific drift-conditioned controller, supported by a local tracking analysis and controlled scheduling experiments.
W2/Q2: Interpretation and sensitivity of the drift probe
We agree that the measured drift is not a direct estimator of critic tracking error. As clarified in Section 3.2 and Appendix B.3 (Eq. (36)), the statistic captures temporal changes arising from both generator-induced input shifts and critic parameter updates. We therefore use it only as a one-sided intervention signal: elevated drift triggers recovery, whereas low drift is not interpreted as evidence of accurate critic tracking. To examine probe sensitivity, we conducted an additional ablation (Exp. 084) varying probe-bank composition and size, with κ = 3 and M = 40 and separate threshold calibration for each configuration.
Replacing the probe bank leads to a relatively modest reduction in Total (−0.35), while increasing the bank size to four produces a larger decrease (−1.59) and approximately 23% additional training overhead. These results indicate that scheduling is not dependent exclusively on the original probe samples, but is sensitive to probe-bank configuration. Increasing the number of probe samples does not necessarily improve scheduling performance under the tested settings. The ablation is reported in Appendix E.2, Table E.1.
W3/Q3: Drift dynamics relative to Fixed TTUR
We thank the reviewer for this suggestion. We added a comparison of Drift-Gated TTUR and Fixed 5:1 using the same fixed-paired probe, evaluated every five outer iterations. Fixed TTUR records drift passively, with the gate disabled. We analyze three independent training seeds (42, 43, and 45), with 359 post-calibration probe evaluations per arm and seed.
Across seeds, the ratio of mean drift under Fixed TTUR to that under Drift-Gated TTUR is approximately 3.1×. Mean high-drift event frequency decreases from approximately 99% to 44%. The temporal curves show increasingly elevated drift under Fixed TTUR and substantially lower drift under adaptive scheduling at later stages. We have expanded Figure 4 to include both schedules and added the remaining curves and three-seed statistics in Appendix F.
W4/Q4: Robustness across independent training runs
We added two independent training replications (three runs total). Uniform matches the adaptive run's generator/critic update counts separately for each run. Evaluation uses 946 VBench prompts and seven inference seeds (Appendix G).
Our method outperforms both controls in all three runs, averaging +0.82 ± 0.61 over Uniform and +0.77 ± 0.23 over Fixed (mean ± SD across training runs).