Drift Forcing: Sample-Based Drift Fields as Initialization for Autoregressive Video Diffusion Distillation

Autoregressive video diffusion distillation makes a bidirectional teacher causal, initializes a few-step causal student and refines it with distribution matching distillation (DMD), in which the bidirectional teacher scores the student's own rollouts. The released initializations imitate the causal teacher one trajectory at a time under teacher forcing: the student is trained on ground-truth prefixes, then handed to a stage that conditions it on its own samples. We ask whether the initialization can instead be trained on those samples directly, with no teacher trajectories, no score targets and no trained critic. does this with a drift field in the feature space of a frozen causal teacher: the student's own rollouts are attracted toward positive clips, one bidirectional-teacher clip per training prompt, and repelled from one another, in the self-rollout regime that DMD and inference share. The main recipe runs a coarse field over clip-level features and then a finer field over per-chunk features, for about 400 H800-hours on the public Causal-rCM synthetic training set. On Wan2.1-1.3B, with the DMD trainer, data and budget held fixed, the student that DMD makes from the drift initialization reaches the highest VBench Total among all products compared (84.63), above DMD from the released consistency-distilled initializations and from the causal diffusion model with no initialization stage. The lead comes from VBench's dynamic-degree dimension, the share of clips that move. Code is available at https://anonymous.4open.science/r/DF0E51.