Horizon Distribution Matching for Autoregressive Video Distillation

Autoregressive video diffusion offers a scalable route to streaming generation, but maintaining video quality under few-step distillation remains challenging. On-policy learning reduces the discrepancy between training and inference histories, yet leaves open which temporal distributions should be matched along the generated trajectory. For example, full-video matching constrains the sequence distribution, whereas isolated window matching may omit dependencies on preceding history. To study this choice, we introduce *HorizonDMD*, a generalized formulation of on-policy distribution matching over history–horizon views. Each view is specified by its conditioning history and horizon extent, encompassing zero-history global views and local views conditioned on student-generated histories. Full-video matching is a special case. This formulation compares the distributions of noisy horizons conditioned on noisy histories, allowing supervision to be allocated across history depths and temporal extents. Under consistent student and reference marginals, we show that this objective decomposes into a coverage-weighted sum of autoregressive conditional divergences. Consequently, positive coverage preserves the zero-divergence target of full-sequence matching, including identification of the clean sequence law under nondegenerate Gaussian corruption. We further show that prefix matching and conditional horizon matching yield the same gradient when the clean-history distribution is held fixed, providing a basis for horizon-restricted distillation. We also show that prefix and restricted full-video score discrepancies need not agree, so the choice of temporal view can change the distributional guidance.