RaMP: Rollout-Aware Mode Preconditioning for Video Distillation

Few-step autoregressive video generators enable efficient inference, but prolonged self-forcing distribution matching distillation (SF-DMD) can lead to oversaturated colors and abnormal contrast. We attribute this failure to SF-DMD favoring narrow, undesirable regions that are easier for an imperfect student to fit than broader high-quality regions, with fake-score errors further amplifying this bias. These observations motivate reducing undesirable mode before SF-DMD. We therefore propose RaMP, a rollout-aware mode-preconditioning framework that gradually shifts consistency distillation from teacher-forced states to states visited by the student's own autoregressive rollouts before switching to pure SF-DMD. Autoregressive rollout can move states out of the basin of narrow undesirable modes and into transition regions where the broader high-quality mode dominates the teacher posterior. Consistency supervision can then correct these rollout-induced deviations, while the resulting improved prefixes provide better contexts for subsequent autoregressive blocks, progressively reducing undesirable visitation before SF-DMD. Compared with existing autoregressive distillation methods, RaMP achieves the best overall VBench score and lowest HSV saturation, while maintaining strong CFM-7B performance, effectively reducing oversaturation without compromising generation quality.