- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces HorizonDMD, a framework for on-policy distribution matching in autoregressive video distillation that generalizes full-video matching to a family of history–horizon views. Each view specifies a conditioning history and a horizon over which the generator receives distribution-matching supervision. The paper provides a conditional coverage theorem showing when the idealized objective identifies the full sequence distribution, establishes gradient equivalence between prefix and conditional horizon matching under a fixed clean-history distribution, and characterizes why prefix scores generally differ from restricted full-video scores. Experiments on VBench show competitive results relative to Self Forcing DMD, with a modest improvement in total and quality scores accompanied by a decrease in semantic performance.
Critical strengths
The paper offers a clear and useful formulation of the temporal scope of distribution matching in autoregressive video distillation. Rather than treating full-video matching and local supervision as unrelated design choices, it unifies them through a distribution over history–horizon views. The conditional coverage theorem provides a principled explanation of when such views preserve sequence-level identification, while the counterexample concerning unconditioned window marginals clarifies why complete-history conditioning matters.
The analysis of prefix and full-video scores is also valuable. The paper shows that restricting the coordinates of a full-video score does not generally recover the score of the corresponding prefix distribution. This establishes a meaningful distinction between two supervision strategies and helps explain why the temporal scope of scoring can affect the generator update. The authors also explicitly acknowledge that their theoretical results do not guarantee convergence or equivalence for the practical training procedure.
Critical weaknesses
The empirical evaluation is limited, and the improvements over Self Forcing DMD are modest. The best configuration improves the total VBench score only slightly, while its semantic score decreases. The current results therefore do not establish a clear overall advantage for HorizonDMD.
The contribution of the view-selection strategy remains unclear. The evaluated configurations differ in training duration and other settings, and the results at a common checkpoint do not show a clear advantage from changing the full-video-view probability. It is therefore difficult to determine how much the observed gains come from the proposed supervision strategy itself.
The theoretical analysis concerns an ideal objective, whereas the practical method uses learned score estimates, normalization, and gradient truncation. Although the authors acknowledge this gap, the experiments do not show whether the proposed theoretical properties translate into a meaningful practical benefit.
Questions for authors
-
What is the main practical benefit of using HorizonDMD over standard full-video DMD, given the small improvements in the current evaluation?
-
How should we interpret the theoretical results in light of the approximations used in the actual training procedure?
2 comments
Q2. How should the theoretical results be interpreted given the approximations in training?
The theory characterizes the ideal matching target and distinguishes alternative update constructions. It does not establish that the implemented algorithm follows the exact population gradient or converges to that target. Its practical role is to specify what should be held fixed, what differs between views, and which comparisons can test those differences.
Theorem 1: target preservation and block coverage. Under finite KL and consistent marginals of the same two sequence laws, complete-history conditionals with positive weighted coverage identify the full noisy sequence law at zero risk. Under Gaussian corruption with a nonzero signal coefficient, this also identifies the clean law. This result concerns the target, not the optimization trajectory.
Appendix C.2 uses this decomposition to construct D: every single-block target retains its full generated history, and its sampling probability matches C's nominal per-block loss weight. Consequently, C and D share the same ideal weighted conditional-KL objective, with equal full gradients at the same student parameters wherever differentiable. Their observed difference therefore tests how matching is organized in the practical updates, rather than a difference in the ideal weighted objective. This design does not imply that D lacks cross-block dependencies.
Proposition 1: the conditional interpretation of a horizon update. The clean-history distribution is frozen before corruption. Under this construction, the history-marginal KL is constant, so prefix-KL and conditional-horizon derivatives agree. The score-based gradient formula additionally requires exact scores and the stated differentiability conditions. While the conditional objective uses noisy histories, the generator still receives clean history. This result does not equate the frozen-history derivative with the total on-policy gradient, which also changes the history distribution.
Proposition 2: scoring scope changes the signal. For each sequence law, a prefix score averages the corresponding full-video score coordinates over that law's conditional suffix distribution. Merely masking a full-video score generally gives a different signal. This motivates the B-C comparison, which holds horizon supervision fixed and compares full-video versus prefix scoring (including their corresponding auxiliary fitting). The proposition predicts non-equivalence, not which choice performs better; Table 1 provides the empirical comparison.
What remains exact, and what is approximate?
The detached surrogate exactly implements the specified normalized prediction-difference update through the retained computation graph (Appendix B.6). Identifying that update with an ideal KL gradient would require additional conditions not guaranteed in training: learned or guided predictions need not be exact posterior means, normalization changes sample weighting, and the retained Jacobian can differ from the full conditional-generation Jacobian. Restricting the direct loss to the continuation does not detach every gradient route through the generated history (Appendix C.1). Teacher predictions on shorter inputs also need not satisfy marginal consistency across input lengths (Section 3.5).
We therefore connect theory and practice through experiments on the distinctions identified by the analysis. B-C tests scoring scope, C-D tests update organization at matched ideal coverage, and the probability study tests allocation within a family preserving the ideal zero-divergence target. These experiments do not certify that the practical approximations satisfy the theorem's assumptions.
Appendix D.3 (Tables 8-9) further examines the dependence on the retained gradient path. Keeping the horizon-length distribution fixed, the allocation variant mixes uniform history sampling with horizons placed at the beginning or end of the video:
| Gradient configuration | Original allocation | Endpoint-mixed allocation | Change in Total |
|---|---|---|---|
| Original gradient path | 84.6618 | 84.1324 | -0.5294 |
| Modified cache derivatives | 84.2382 | 84.4958 | +0.2576 |
All four variants are evaluated after 800 training iterations. The observed effect of allocation reverses with the derivative configuration. This illustrates why allocation must be evaluated together with the retained gradient path. These are earlier paired experiments: the base is H1 reevaluated under that protocol, not C in Table 1. They do not isolate the cause of the main C-D gain.
We hope the matched experiments and this distinction between target characterization and practical optimization address your concerns about the empirical value and interpretation of the theory.
Thank you for recognizing the value of the history-horizon formulation and scoring-scope analysis. We added matched ablations to examine whether these distinctions matter in practice. This reply addresses Q1 and your concern about view selection; our second reply addresses Q2.
The main practical benefit is higher aggregate VBench performance, primarily through increased motion activity, under the same autoregressive inference procedure. In the new matched comparison, PrefixHorizon (C) improves over FullVideo (A) by 0.38 Total and 0.55 Quality points, with a 0.32-point Semantic decrease. We agree that this does not establish a uniform improvement across all aspects of video quality.
Table 1 compares four settings that share the same teacher, backbone, initialization, training seed, optimizer settings, rollout policy, and an effective batch size of 64. Each receives 50 auxiliary warmup updates followed by 800 training iterations: 160 generator and 850 auxiliary updates in total. All four settings are evaluated at this common endpoint using four-step EMA inference and the same 4,730 prompt/sample pairs with paired initial noise. Table 2 remains a separate reference comparison with Self-Forcing.
Here, history denotes all preceding generated blocks, and continuation denotes the selected consecutive blocks that follow. Both score models receive the noised scoring input; auxiliary regression covers those same blocks. The direct generator loss column specifies direct loss support.
B and C use the same view distribution (full-video probability 1/3), horizon loss, length weighting, normalization rule, and rollout policy. Moving the score inputs and auxiliary regression from the full video to the visible prefix raises Total by 0.4070 points. D retains complete history and matches C's nominal per-block loss weights; C exceeds D by 0.60 Total points. These comparisons examine scoring scope and horizon organization under the same optimizer-update budget.
The trade-off remains important: C's Dynamic degree is 64.44 versus A's 49.72, but its other six quality dimensions are lower than those of A and B (Table 4). Dynamic degree measures activity, not correct execution of a requested action. Section 4.3 reports these limitations. Regarding training cost, budgets are matched by update counts rather than GPU-hours: observed optimizer-update costs for A/B/C were 150.60/147.42/138.00 GPU-hours, including warmup. C used 8.4% less than A in these runs; this is not a controlled throughput benchmark, and D lacks a complete timing trace (Appendix C.1).
Contribution of view selection and checkpoint choice
Your observation about the common 700-iteration checkpoint is correct: the settings with 1/3 and 3/8 full-video probabilities achieve nearly identical Total scores. We now report the progression at common checkpoints within the earlier single-view probability study (Table 5):
These recipes retain uniform history sampling for each horizon length and the same length-weighting rule. The gap changes over training: the 1/3 and 3/8 settings are nearly tied at 700 iterations but diverge at 800. This documents sensitivity to allocation; it does not establish a universally optimal probability or explain the main gains solely through probability tuning. These earlier runs are separate from the matched Table 1 experiments.
For the main ablations, Table 3 and Figure 6 show that A leads in Total after 500 and 600 iterations, while C leads after 800 and exceeds D at all three checkpoints. Each setting reflects a single training run. The curves describe observed training progress, and the evaluation samples are not independent training repetitions. We therefore report the aggregate benefit and its trade-offs without claiming statistical significance or a consistent advantage at every checkpoint.