- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces RaMP, a rollout-aware mode-preconditioning framework designed to reduce oversaturation and bad-mode collapse in few-step autoregressive video distillation. The authors examine how imperfect student initialization and inaccurate fake-score estimation can amplify undesirable modes during SF-DMD. To counter this, RaMP gradually shifts from teacher-forced to self-forcing consistency distillation and leverages student rollout states to improve initialization before standard SF-DMD. Experiments on Wan2.1-T2V-1.3B show gains in VBench scores and color fidelity, and controlled Gaussian-mixture experiments together with ablation studies further support the proposed mechanism.
Critical strengths
-
Insightful analysis of bad-mode collapse. The paper offers a insightful examination of two possible causes of bad-mode collapse in SF-DMD, namely imperfect student initialization and inaccurate fake-score estimation, and supports these explanations with both theoretical derivations and controlled experiments.
-
Comprehensive experiments. The experimental evaluation is fairly comprehensive, covering quantitative and qualitative comparisons, controlled Gaussian-mixture experiments, and ablation studies, all of which support the effectiveness of the proposed method.
-
Clear presentation. The paper is well written and clearly organized. The motivation, theoretical analysis, and proposed method are easy to follow, and the figures and visualizations are helpful.
Critical weaknesses
-
Insufficient training cost comparison. Although the authors highlight the use of only 8 GPUs and a comparable number of training steps, the actual training time and computational cost are not clearly reported. Since different training stages may have substantially different per-step costs, a more detailed comparison of GPU-hours would strengthen the efficiency claims.
-
Limited generalization across models and settings. The experiments are primarily conducted on Wan2.1-T2V-1.3B. Evaluations on larger models or longer video generation would help demonstrate the generalizability of the proposed method.
Questions for authors
-
How sensitive is RaMP to the choice of backbone and video length? Is there any evidence that the improvements extend to larger models or longer autoregressive rollouts?
-
Beyond the Gaussian-mixture experiments, is there any direct evidence from real video models showing that oversaturation stems from bad-mode amplification, particularly when it is driven by inaccurate fake-score estimation?
3 comments
Thank you to the authors for the response. These answers have addressed my questions. Since I have already provided positive feedback, I will keep my score unchanged.
Thank you for keeping your positive score!
Thank you for the positive assessment of the mechanism analysis and for requesting clearer evidence on cost, longer rollouts, and fake-score estimation. We address these separately so that the scope of the evidence is explicit.
Thank you for the suggestion. We report the per-100-step cost and total GPU-hours for both training stages below. All timings are measured on NVIDIA H200 GPUs, using 8 GPUs for RaMP and the official 64-GPU recipe for Causal Forcing++ [1].
Although RaMP's mixed TF-CD/SF-CD initialization has a higher per-step cost (30.0 vs. 18.5 min / 100 steps) and more steps (5,500 vs. 3,000), its 8-GPU training reduces total compute from 837.3 to 250.7 GPU-hours (3.3× reduction).
Following all compared baselines (CausVid[2], Self-Forcing[3], Causal Forcing[4], Causal Forcing++, and DistillAlign[5]), we use Wan2.1-T2V-1.3B as the student backbone because official baseline code and checkpoints are only available on this model, ensuring a fair comparison.
To assess generalization to longer horizons, we additionally evaluate our 81-frame-trained models on 161-frame autoregressive rollouts, approximately twice the training length. As shown in the table below, on the VBench-Long[6] benchmark, RaMP consistently outperforms Causal Forcing++ across Total, Quality, and Semantic scores, while reducing HSV saturation and improving CFM-7B. These results support RaMP’s generalizability to longer videos, with consistent improvements in generation quality and color fidelity.
Thank you for raising this question.
We conduct a real-video ablation using 1, 5, or 10 fake-score updates per student update, while keeping the total number of student updates fixed. We use this ratio as a practical proxy for fake-score estimation accuracy: more updates allow the fake-score model to better track the evolving student distribution. As shown in the table below, increasing the ratio from 1 to 5 and 10 raises VBench Total from 0.7674 to 0.7998 and 0.8040, and Quality from 0.7984 to 0.8333 and 0.8474. These results show that fake-score training materially affects real-video distillation, consistent with our hypothesis that insufficient fake-score estimation can impair optimization.
Should you have any additional questions, please do not hesitate to follow up. We greatly look forward to your thoughtful comments.
Reference
[1] Zhao M, Zhu H, Zheng K, et al. Causal forcing++: Scalable few-step autoregressive diffusion distillation for real-time interactive video generation[J]. arXiv preprint arXiv:2605.15141, 2026.
[2] Yin T, Zhang Q, Zhang R, et al. From slow bidirectional to fast autoregressive video diffusion models[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 22963-22974.
[3] Huang X, Li Z, He G, et al. Self forcing: Bridging the train-test gap in autoregressive video diffusion[J]. Advances in Neural Information Processing Systems, 2026, 38: 167283-167308.
[4] Zhu H, Zhao M, He G, et al. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation[J]. arXiv preprint arXiv:2602.02214, 2026.
[5] Li J, Zou K, Zhou C, et al. Distillalign: Coordinating mode covering and mode seeking in autoregressive video distillation[J]. arXiv preprint arXiv:2607.26811, 2026.
[6] Huang Z, Zhang F, Xu X, et al. Vbench++: Comprehensive and versatile benchmark suite for video generative models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.