acceptodds
Under review as a conference paper at ICLR 2027

Horizon Distribution Matching for Autoregressive Video Distillation

17 views

Abstract

Autoregressive video diffusion offers a scalable route to streaming generation, but maintaining video quality under few-step distillation remains challenging. On-policy learning reduces the discrepancy between training and inference histories, yet leaves open which temporal distributions should be matched along the generated trajectory. For example, full-video matching constrains the sequence distribution, whereas isolated window matching may omit dependencies on preceding history. To study this choice, we introduce *HorizonDMD*, a generalized formulation of on-policy distribution matching over history–horizon views. Each view is specified by its conditioning history and horizon extent, encompassing zero-history global views and local views conditioned on student-generated histories. Full-video matching is a special case. This formulation compares the distributions of noisy horizons conditioned on noisy histories, allowing supervision to be allocated across history depths and temporal extents. Under consistent student and reference marginals, we show that this objective decomposes into a coverage-weighted sum of autoregressive conditional divergences. Consequently, positive coverage preserves the zero-divergence target of full-sequence matching, including identification of the clean sequence law under nondegenerate Gaussian corruption. We further show that prefix matching and conditional horizon matching yield the same gradient when the clean-history distribution is held fixed, providing a basis for horizon-restricted distillation. We also show that prefix and restricted full-video score discrepancies need not agree, so the choice of temporal view can change the distributional guidance.

Primary area:
generative models
Keywords:
Distribution Matching, Autoregressive Video Generation, Diffusion Models, Distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zy3aOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces HorizonDMD, a framework for on-policy distribution matching in autoregressive video distillation that generalizes full-video matching to a family of history–horizon views. Each view specifies a conditioning history and a horizon over which the generator receives distribution-matching supervision. The paper provides a conditional coverage theorem showing when the idealized objective identifies the full sequence distribution, establishes gradient equivalence between prefix and conditional horizon matching under a fixed clean-history distribution, and characterizes why prefix scores generally differ from restricted full-video scores. Experiments on VBench show competitive results relative to Self Forcing DMD, with a modest improvement in total and quality scores accompanied by a decrease in semantic performance.

Critical strengths

The paper offers a clear and useful formulation of the temporal scope of distribution matching in autoregressive video distillation. Rather than treating full-video matching and local supervision as unrelated design choices, it unifies them through a distribution over history–horizon views. The conditional coverage theorem provides a principled explanation of when such views preserve sequence-level identification, while the counterexample concerning unconditioned window marginals clarifies why complete-history conditioning matters.

The analysis of prefix and full-video scores is also valuable. The paper shows that restricting the coordinates of a full-video score does not generally recover the score of the corresponding prefix distribution. This establishes a meaningful distinction between two supervision strategies and helps explain why the temporal scope of scoring can affect the generator update. The authors also explicitly acknowledge that their theoretical results do not guarantee convergence or equivalence for the practical training procedure.

Critical weaknesses

The empirical evaluation is limited, and the improvements over Self Forcing DMD are modest. The best configuration improves the total VBench score only slightly, while its semantic score decreases. The current results therefore do not establish a clear overall advantage for HorizonDMD.

The contribution of the view-selection strategy remains unclear. The evaluated configurations differ in training duration and other settings, and the results at a common checkpoint do not show a clear advantage from changing the full-video-view probability. It is therefore difficult to determine how much the observed gains come from the proposed supervision strategy itself.

The theoretical analysis concerns an ideal objective, whereas the practical method uses learned score estimates, normalization, and gradient truncation. Although the authors acknowledge this gap, the experiments do not show whether the proposed theoretical properties translate into a meaningful practical benefit.

Questions for authors
  1. What is the main practical benefit of using HorizonDMD over standard full-video DMD, given the small improvements in the current evaluation?

  2. How should we interpret the theoretical results in light of the approximations used in the actual training procedure?

2 comments

Thank you for recognizing the value of the history-horizon formulation and scoring-scope analysis. We added matched ablations to examine whether these distinctions matter in practice. This reply addresses Q1 and your concern about view selection; our second reply addresses Q2.

Q1. What is the main practical benefit over full-video DMD?

The main practical benefit is higher aggregate VBench performance, primarily through increased motion activity, under the same autoregressive inference procedure. In the new matched comparison, PrefixHorizon (C) improves over FullVideo (A) by 0.38 Total and 0.55 Quality points, with a 0.32-point Semantic decrease. We agree that this does not establish a uniform improvement across all aspects of video quality.

Table 1 compares four settings that share the same teacher, backbone, initialization, training seed, optimizer settings, rollout policy, and an effective batch size of 64. Each receives 50 auxiliary warmup updates followed by 800 training iterations: 160 generator and 850 auxiliary updates in total. All four settings are evaluated at this common endpoint using four-step EMA inference and the same 4,730 prompt/sample pairs with paired initial noise. Table 2 remains a separate reference comparison with Self-Forcing.

Here, history denotes all preceding generated blocks, and continuation denotes the selected consecutive blocks that follow. Both score models receive the noised scoring input; auxiliary regression covers those same blocks. The direct generator loss column specifies direct loss support.

Ablation settingScoring inputDirect generator lossTotalQualitySemanticDynamic
A: FullVideoFull videoFull video84.160384.636482.255749.72
B: FullScoreHorizonFull videoContinuation84.128684.724281.746453.33
C: PrefixHorizon (ours)History + continuationContinuation84.535685.186581.932164.44
D: SingleBlockHistory + one blockOne block83.936584.384482.144746.39

B and C use the same view distribution (full-video probability 1/3), horizon loss, length weighting, normalization rule, and rollout policy. Moving the score inputs and auxiliary regression from the full video to the visible prefix raises Total by 0.4070 points. D retains complete history and matches C's nominal per-block loss weights; C exceeds D by 0.60 Total points. These comparisons examine scoring scope and horizon organization under the same optimizer-update budget.

The trade-off remains important: C's Dynamic degree is 64.44 versus A's 49.72, but its other six quality dimensions are lower than those of A and B (Table 4). Dynamic degree measures activity, not correct execution of a requested action. Section 4.3 reports these limitations. Regarding training cost, budgets are matched by update counts rather than GPU-hours: observed optimizer-update costs for A/B/C were 150.60/147.42/138.00 GPU-hours, including warmup. C used 8.4% less than A in these runs; this is not a controlled throughput benchmark, and D lacks a complete timing trace (Appendix C.1).

Contribution of view selection and checkpoint choice

Your observation about the common 700-iteration checkpoint is correct: the settings with 1/3 and 3/8 full-video probabilities achieve nearly identical Total scores. We now report the progression at common checkpoints within the earlier single-view probability study (Table 5):

Full-video probabilityTotal after 600 iterationsAfter 700After 800
1/483.803184.191884.1950
1/384.419584.463484.6441
3/884.335884.463184.2262

These recipes retain uniform history sampling for each horizon length and the same length-weighting rule. The gap changes over training: the 1/3 and 3/8 settings are nearly tied at 700 iterations but diverge at 800. This documents sensitivity to allocation; it does not establish a universally optimal probability or explain the main gains solely through probability tuning. These earlier runs are separate from the matched Table 1 experiments.

For the main ablations, Table 3 and Figure 6 show that A leads in Total after 500 and 600 iterations, while C leads after 800 and exceeds D at all three checkpoints. Each setting reflects a single training run. The curves describe observed training progress, and the evaluation samples are not independent training repetitions. We therefore report the aggregate benefit and its trade-offs without claiming statistical significance or a consistent advantage at every checkpoint.

Q2. How should the theoretical results be interpreted given the approximations in training?

The theory characterizes the ideal matching target and distinguishes alternative update constructions. It does not establish that the implemented algorithm follows the exact population gradient or converges to that target. Its practical role is to specify what should be held fixed, what differs between views, and which comparisons can test those differences.

Theorem 1: target preservation and block coverage. Under finite KL and consistent marginals of the same two sequence laws, complete-history conditionals with positive weighted coverage identify the full noisy sequence law at zero risk. Under Gaussian corruption with a nonzero signal coefficient, this also identifies the clean law. This result concerns the target, not the optimization trajectory.

Appendix C.2 uses this decomposition to construct D: every single-block target retains its full generated history, and its sampling probability matches C's nominal per-block loss weight. Consequently, C and D share the same ideal weighted conditional-KL objective, with equal full gradients at the same student parameters wherever differentiable. Their observed difference therefore tests how matching is organized in the practical updates, rather than a difference in the ideal weighted objective. This design does not imply that D lacks cross-block dependencies.

Proposition 1: the conditional interpretation of a horizon update. The clean-history distribution is frozen before corruption. Under this construction, the history-marginal KL is constant, so prefix-KL and conditional-horizon derivatives agree. The score-based gradient formula additionally requires exact scores and the stated differentiability conditions. While the conditional objective uses noisy histories, the generator still receives clean history. This result does not equate the frozen-history derivative with the total on-policy gradient, which also changes the history distribution.

Proposition 2: scoring scope changes the signal. For each sequence law, a prefix score averages the corresponding full-video score coordinates over that law's conditional suffix distribution. Merely masking a full-video score generally gives a different signal. This motivates the B-C comparison, which holds horizon supervision fixed and compares full-video versus prefix scoring (including their corresponding auxiliary fitting). The proposition predicts non-equivalence, not which choice performs better; Table 1 provides the empirical comparison.

What remains exact, and what is approximate?

The detached surrogate exactly implements the specified normalized prediction-difference update through the retained computation graph (Appendix B.6). Identifying that update with an ideal KL gradient would require additional conditions not guaranteed in training: learned or guided predictions need not be exact posterior means, normalization changes sample weighting, and the retained Jacobian can differ from the full conditional-generation Jacobian. Restricting the direct loss to the continuation does not detach every gradient route through the generated history (Appendix C.1). Teacher predictions on shorter inputs also need not satisfy marginal consistency across input lengths (Section 3.5).

We therefore connect theory and practice through experiments on the distinctions identified by the analysis. B-C tests scoring scope, C-D tests update organization at matched ideal coverage, and the probability study tests allocation within a family preserving the ideal zero-divergence target. These experiments do not certify that the practical approximations satisfy the theorem's assumptions.

Appendix D.3 (Tables 8-9) further examines the dependence on the retained gradient path. Keeping the horizon-length distribution fixed, the allocation variant mixes uniform history sampling with horizons placed at the beginning or end of the video:

Gradient configurationOriginal allocationEndpoint-mixed allocationChange in Total
Original gradient path84.661884.1324-0.5294
Modified cache derivatives84.238284.4958+0.2576

All four variants are evaluated after 800 training iterations. The observed effect of allocation reverses with the derivative configuration. This illustrates why allocation must be evaluated together with the retained gradient path. These are earlier paired experiments: the base is H1 reevaluated under that protocol, not C in Table 1. They do not isolate the cause of the main C-D gain.

We hope the matched experiments and this distinction between target characterization and practical optimization address your concerns about the empirical value and interpretation of the theory.

Reviewer_GZuSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HorizonDMD, a variant of DMD for AR video model distillation. The motivation is straightforward. Existing DMD considers the entire generated sequence, but matching only local windows may ignore dependencies on previously generated frames. To address this, the authors propose sampling different history-horizon views instead of always matching the full generated video. The teacher and auxiliary model score the noisy prefix containing both history and horizon, while the DMD loss is applied only to the horizon with the generated history fixed. Experiments compare HorizonDMD with Self-Forcing DMD on VBench.

Critical strengths

The paper provides a unified formulation of full-video and history-conditioned distribution matching. I agree that horizon-based matching can preserve the target of full-sequence matching under certain conditions, which is useful for understanding temporally different supervision strategies in AR video distillation.

Critical weaknesses

W1. First of all, I have some concerns about the evaluation. In table 1, the authors compare SF DMD at step 600 with HorizonDMD at different training steps. More importantly, the SF DMD baseline uses a fixed checkpoint, while the HorizonDMD results are selected based on the highest evaluation score without a separate validation set. The improvement over the current SF DMD baseline is even smaller than the difference between the historical and current evaluations of the same SF DMD checkpoins. Regarding qualitative results, the paper only provides a few sampled frames, without actual playable video results. I think videos are necessary for reviewers to evaluate the proposed method, especially for longer video generation given the motivation of the paper. Currently, I am not convinced that the proposed DMD provides clear strengths over SF DMD.

W2. The proposed method changes both the scoring scope and the region where the DMD loss is applied. However, there is no ablation study that separates these two changes. For example, I would like to see a comparison between full-video scoring with horizon-only loss and prefix scoring with the same horizon-only loss. Also, similar ideas of history-conditioned distribution matching have been explored in previous works such as Context Forcing[1]. I think the paper would be stronger if it clarifies what is technically different from these methods and why the proposed history-horizon formulation is beneficial in practice.

[1] Chen et al, Context Forcing: Consistent Autoregressive Video Generation with Long Context, ICML 2026

Questions for authors

Q1. Regarding W1, why do the authors use different checkpoint selection strategies for SF DMD and HorizonDMD in Table 1? Could the authors provide results under the same training budget and checkpoint selection strategy? And could the authors provide actual video results, including side-by-side comparisons with SF DMD? I would also like to see longer video generation results to check whether HorizonDMD improves temporal consistency over long horizons.

Q2. Regarding W2, could the authors provide an ablation study that separates the effect of scoring scope from the region where the DMD loss is applied? Maybe comparing full-video scoring and prefix scoring with the same horizon-only loss would help clarify this.

Q3. Regarding W2, what is the main difference between HorizonDMD and existing history-conditioned distribution matching methods? I cannot find what practical advantages the proposed formulation provides over these methods.

4 comments

Thank you for the review. We added the ablation you specifically requested: full-video versus prefix scoring with the same horizon loss. All four settings use matched update counts and are evaluated after 800 training iterations. This first post covers W1/Q1 and W2/Q2; our second post addresses Q3 and additional design experiments.

W1 / Q1. Evaluation protocol and checkpoint selection

We first correct a reporting error: the two Self-Forcing (SF) entries in Table 2 are independent training runs evaluated at step 600, not reevaluations of the same checkpoint (Appendix E.3). The original comparison also used asymmetric selection: fixed-step SF checkpoints versus best-observed HorizonDMD checkpoints without separate validation. We retain Table 2 as a reference and use the new Table 1 ablations for matched evaluation.

All settings share initialization, teacher, backbone, training seed, optimizer, rollout policy, and batch size 64. Each receives 50 auxiliary warmup updates followed by 800 training iterations: 160 generator and 850 auxiliary updates in total. Models are evaluated after 800 iterations using four-step EMA inference, the same 4,730 prompt/sample pairs, and paired initial noise.

Here, history refers to all preceding generated blocks, and continuation refers to the selected consecutive blocks that follow. Both score models receive the noised input; auxiliary regression covers the same blocks.

Ablation settingScoring inputDirect generator lossTotalQualitySemanticDynamic
A: FullVideoFull videoFull video84.160384.636482.255749.72
B: FullScoreHorizonFull videoContinuation84.128684.724281.746453.33
C: PrefixHorizon (ours)History + continuationContinuation84.535685.186581.932164.44
D: SingleBlockHistory + one blockOne block83.936584.384482.144746.39

C gains 0.38 Total and 0.55 Quality points over A, with a 0.32-point lower Semantic score. Table 3 and Figure 6 also report checkpoints after 500 and 600 iterations: A leads in Total at these earlier checkpoints, whereas C's Total increases across the three reported checkpoints and is highest after 800 iterations. Each curve reflects a single training run rather than multi-seed averages.

Matching is defined by update counts: observed optimizer-update costs for A/B/C were 150.60/147.42/138.00 GPU-hours including warmup, 8.4% lower for C than for A (Appendix C.1). These are run costs, not a controlled throughput benchmark.

Playable five-second paired videos are available on our project page, complementing the sampled frames in Figures 4 and 7. While our formulation allows general rollout lengths, the controlled evaluation covers the teacher's native five-second horizon; longer-rollout consistency remains to be evaluated (Section 5).

W2 / Q2. Scoring scope under the same horizon loss

B and C implement the comparison you requested. They share the view distribution, continuation loss, length weighting, normalization rule, and rollout policy, differing only in whether both score models (and auxiliary regression) receive the full video (B) or only the history and selected continuation (C).

C gains 0.4070 Total, 0.4623 Quality, and 0.1857 Semantic points over B. Dynamic degree (which measures motion activity rather than action correctness) rises by 11.11 points, while the other six quality scores decrease (Table 4). B remains close to A in Total (84.1286 versus 84.1603), so full-video-scored horizon loss does not reproduce C's gain. Proposition 2 establishes the non-equivalence of prefix and masked full-video scoring, and B–C tests its practical consequence (including the auxiliary fit).

Separating the auxiliary regression region

Appendix D.2 (Tables 6–7) additionally fixes score-model input scope and the generator loss region within each recipe, varying only auxiliary regression support. Both score models process the visible prefix, while generator supervision stays on the continuation. The fixed-endpoint family always scores through the final video block, whereas the random-horizon family also samples earlier endpoints.

Earlier recipe, after 750 iterationsRegress history + continuationRegress continuation onlyTotal gain
Fixed scoring endpoint84.315883.8807+0.4351
Random horizon84.339184.3073+0.0318

Full-prefix regression yields a higher Total score in both pairs, while Dynamic degree increases in the first and decreases in the second. This shows that auxiliary fitting support can affect the result independently of the chosen scoring input scope and generator loss region. These are separate paired recipes rather than the main B–C runs, so their gains do not numerically decompose B–C's 0.4070-point improvement.

W2 / Q3. Added value beyond history-conditioned DMD

HorizonDMD builds on history-conditioned DMD. Context Forcing addresses long-video continuation using a context-capable teacher, a rollout curriculum, and context management. By contrast, our focus is how to distribute matching supervision across generated histories and continuation lengths within a rollout (Sections 2 and 3.3).

The score models receive the jointly noised history and continuation; auxiliary regression covers both, while direct generator supervision covers only the continuation. For example, scoring blocks 1–5 can supervise blocks 3–5 with a two-block history or blocks 4–5 with a three-block history. At a fixed endpoint, changing the history length changes the directly supervised region while leaving the scoring input unchanged; changing the endpoint changes the scoring input itself.

A single-block ablation that retains complete history

The SingleBlock setting D retains all preceding generated blocks and uses the same joint-noise prefix scoring as C, sampling target positions to match C's nominal per-block loss weights, including the first block (Appendix C.2). Thus, D retains cross-block conditional information rather than matching isolated window marginals.

Under Theorem 1's consistent-marginal and finite-KL assumptions, C and D share the same ideal weighted conditional-KL objective at the same student parameters. Positive weighted coverage preserves the full-sequence zero-divergence target; with Gaussian corruption and a nonzero signal coefficient, equality of the noisy laws also identifies the clean laws.

Although the ideal full gradients agree, the implemented DMD updates use learned scores, sample-dependent normalization, and retained rather than full rollout Jacobians. Their scoring contexts, horizon grouping, and auxiliary fitting also differ. C–D therefore tests how matching is implemented at equal ideal per-block coverage.

After 800 training iterations, C scores 0.60 Total points higher than D (84.5356 versus 83.9365), with an 18.06-point higher Dynamic degree and a 0.21-point lower Semantic score. C's Total also exceeds D's at the 500- and 600-iteration checkpoints (Table 3). This supports the practical value of the multi-horizon update organization in our setup. D is an internal conditional-DMD ablation, not a reproduction of Context Forcing or Context-Matched Distillation.

View allocation and the retained gradient path

Appendix D.3 (Tables 8–9) further tests whether the effect of history allocation depends on the gradient path. Keeping the horizon-length distribution fixed, the allocation variant mixes uniform sampling with horizons placed at the beginning or end of the video. We pair this allocation change with a modification to gradients flowing through the attention key–value cache.

Gradient configurationOriginal allocationEndpoint-mixed allocationChange in Total
Original gradient path84.661884.1324−0.5294
Modified cache gradients84.238284.4958+0.2576

All four variants are evaluated after 800 training iterations. The allocation change lowers Total with the original gradient path but raises it with the modified path. This observed interaction supports studying view allocation together with the retained derivatives, without isolating which individual mechanism drives the main C–D gain. Note that these experiments use the earlier paired recipes from Appendix D.3—whose base is a re-evaluation of the H1 checkpoint rather than C in Table 1—and are therefore compared strictly within that family.

Finally, Appendix D.1 (Table 5) notes that varying the full-video probability (1/4, 1/3, 3/8) changes Total scores at 800 iterations despite near ties at 700, showing that view allocation remains an empirical design variable within the coverage-preserving family.

We would appreciate your reassessment in light of the matched evaluation and the scoring-scope ablation you requested.

Thank you for the detailed rebuttal and additional experimental results. Some of my concerns are partially addressed, but others remain. For your response to W1/Q1, I am not able to access the link, so I cannot assess the videos. So, I would like to maintain my original score for now.

Thank you for the follow-up, and we sincerely apologize for the access issue. The initial URL became outdated after we updated our anonymous project subdomain to horizondmd. We have updated the link in the rebuttal and also provide the verified, anonymous, and double-blind-compliant URL directly here (publicly accessible without sign-in, containing the paired five-second comparisons for W1/Q1):

https://horizondmd.netlify.app/

We hope this addresses your concerns. Once you have viewed the videos, we would greatly appreciate your reassessment and would be glad to clarify any remaining questions.

Reviewer_FmhTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

HorizonDMD defines training views using a history length h and a prediction horizon k, combining full-video objectives with local prediction objectives conditioned on history. Under assumptions including consistent sequence marginals and positive conditional coverage, the paper shows that the ideal objective preserves the zero-divergence solutions of full-sequence matching. It also explains why a teacher score for a prefix generally cannot be replaced by simply selecting coordinates from a full-video score. In practice, the auxiliary model denoises the entire selected prefix, while the generator is updated only on the future segment.

Critical strengths

1 A clear formal account of matching over different temporal horizons The unified objective for global and history-conditioned views, together with the observation that truncating a full-video score generally produces a different teacher signal, offers useful guidance for future distillation methods.

2 Transparent reporting of negative results and limitations The paper does not present every metric as improving consistently. Allowing readers to distinguish established findings from those that still need validation.

Critical weaknesses
  1. The experiments do not establish the added value over existing conditional distillation. Table 1 contains only one external method, Self Forcing DMD; H1–H4 are variants of HorizonDMD. Yet Sec. 2 identifies Context Forcing and Context-Matched Distillation as closely related approaches. A comparison with full-video matching cannot determine what the proposed view family adds beyond conditioning distillation on generated history. A controlled conditional-DMD baseline is especially relevant here, since the paper’s main contribution is a generalization of that objective. The theoretical characterization is useful, but its practical significance remains unclear without this comparison. (Sec. 2; Table 1; Appendix C.)

  2. H1 improves the current baseline’s total score by about 0.127 points while reducing its semantic score by about 0.509 points.

Questions for authors
  1. Under matched teacher, backbone, and training compute, what benefit does sampling history–horizon views provide over an existing history-conditioned DMD objective?
  2. What are the separate contributions of prefix-only scoring and restricting the generator loss to the sampled horizon?
5 comments

Thank you for highlighting the need for a conditional-DMD baseline. We have added this baseline along with scoring and supervision ablations (Table 1, Appendix C). This first reply addresses W1/Q1 and Q2; our second reply addresses W2 and presentation.

W1 / Q1. Benefit over history-conditioned DMD under matched teacher, backbone, and training compute

Using the same teacher and backbone under matched optimizer-update counts, PrefixHorizon (C) achieves a 0.60-point higher Total score than the history-conditioned SingleBlock baseline (D), with a 0.21-point lower Semantic score. This provides evidence that organizing supervision across variable-length continuations improves aggregate VBench performance in our setup.

D retains all preceding student-generated blocks for each target block. It uses the same joint-noise prefix scoring as C, with target positions sampled to match C's nominal loss weight for every block, including the first (Appendix C.2). Thus, D preserves cross-block conditional information.

All four configurations share the same teacher, backbone, initialization, training seed, optimizer settings, rollout policy, and an effective batch size of 64. Each configuration undergoes 50 auxiliary warmup updates followed by 800 training iterations (160 generator and 850 auxiliary updates in total). Evaluation uses four-step EMA inference on the same 4,730 prompt/sample pairs with paired initial noise.

Here, "history" refers to all preceding generated blocks, and "continuation" denotes the selected consecutive blocks that follow. Both score models receive the noised scoring input, and auxiliary regression covers those same blocks. The "Direct generator loss" column specifies which blocks receive direct generator supervision.

Ablation settingScoring inputDirect generator lossTotalQualitySemanticDynamic
A: FullVideoFull videoFull video84.160384.636482.255749.72
B: FullScoreHorizonFull videoContinuation84.128684.724281.746453.33
C: PrefixHorizon (ours)History + continuationContinuation84.535685.186581.932164.44
D: SingleBlockHistory + one blockOne block83.936584.384482.144746.39

C also exceeds D in Total score at the 500- and 600-iteration checkpoints (Table 3). Under Theorem 1's consistent-marginal and finite-KL assumptions, C and D induce the same ideal weighted conditional-KL objective evaluated at the same student parameters, and their full gradients agree wherever differentiable. In practice, however, the implemented DMD updates use learned scores, sample-dependent normalization, and retained rather than full rollout Jacobians; their scoring endpoints, horizon grouping, and auxiliary fitting targets also differ. The C-D comparison therefore tests view-based update organization at matched nominal coverage, without isolating the contribution of any single component.

D is an internal history-conditioned DMD baseline using jointly noised histories and targets rather than a reproduction of Context Forcing or Context-Matched Distillation; it specifically tests the value of organizing matching updates beyond single-block conditioning within our training setup (Sections 2 and 3.3).

Note that budgets are matched by optimizer-update counts rather than measured GPU-hours. Observed optimizer-update costs for A/B/C were 150.60/147.42/138.00 GPU-hours including warmup, while a complete timing trace for D was not recorded (Appendix C.1).

Q2. What are the separate contributions of prefix-only scoring and restricting the generator loss to the sampled horizon?

Horizon-restricted loss under full-video scoring (A to B). The Total score changes from 84.1603 to 84.1286 (-0.0317 points), showing that horizon-restricted loss alone does not yield C's observed Total gain.

Prefix scoring with the same horizon loss (B to C). B and C share the view distribution, continuation loss, length weighting, normalization rule, and rollout policy. Changing the score-model inputs and auxiliary regression from the full video to the visible prefix yields gains of 0.4070 Total, 0.4623 Quality, and 0.1857 Semantic points. This comparison includes the coupled change in auxiliary fitting support and thus does not isolate scoring input alone. Proposition 2 explains why the scoring signals generally differ, without establishing the superiority of either.

Auxiliary regression support (Appendix D.2, Tables 6-7). Within each earlier paired recipe, both the score-model input scope and the generator loss region are fixed, and only auxiliary regression support changes. Full-prefix regression raises Total by 0.4351 points when scoring always ends at the final video block, and by 0.0318 points when earlier endpoints are also sampled. Dynamic degree increases in the first pair and decreases in the second. These results examine auxiliary fitting in isolation, though, because they rely on different training recipes, they cannot be used to numerically decompose the main B-C gain.

W2. Size of the gain and semantic performance

We agree with your reading of the original reference comparison: H1's approximately 0.127-point Total gain is accompanied by a 0.509-point Semantic decrease. We retain this result for reference in Table 2, whereas the new Table 1 evaluates our design choices under matched update counts and a common endpoint in separate experiments and evaluation records.

In Table 1, C gains 0.38 Total and 0.55 Quality points over full-video supervision (A), with a 0.32-point Semantic decrease. Relative to the history-conditioned single-block setting (D), C gains 0.60 Total with a 0.21-point Semantic decrease. Against full-video scoring with the same horizon loss (B), both Total and Semantic increase, by 0.41 and 0.19 points, respectively.

The Quality gain is driven by higher Dynamic degree: C scores 64.44 versus 49.72 for A and 53.33 for B, while C's other six quality dimensions are slightly lower than those of A and B (Table 4). Because Dynamic degree measures motion activity rather than whether the requested action is performed correctly, these results show higher aggregate VBench scores together with trade-offs between motion activity and visual consistency.

Semantic changes are also mixed within the matched comparison. Relative to A, C improves Multiple objects (87.058 to 88.308) and Human action (95.800 to 96.800), while Spatial relationship (85.521 to 81.260) and Color (88.613 to 87.399) decline. We report the complete breakdown in Table 4 and discuss the trade-offs in Section 4.3. Note also that these measurements do not establish that increased motion causes the semantic decrease.

Table 3 and Figure 6 further show when the gains emerge: A leads in Total after 500 and 600 training iterations, while C overtakes after 800. Each setting is evaluated from a single training run, and the 4,730 evaluation samples do not represent independent training seeds; we therefore report observed differences without claiming statistical significance.

Presentation

We revised the organization to make the distinctions in Q1 and Q2 easier to follow. Section 3.3 defines the history-horizon views; Section 3.4 and Algorithm 1 specify the practical updates; Section 3.5 states the theoretical assumptions and results. Figure 2 shows view sampling and alternating training, while Figure 3 compares the scoring and supervision regions with full-video DMD.

The experiments now begin with the matched ablations (Section 4.1), followed by the Self-Forcing reference comparison (Section 4.2) and the dimension-level trade-offs and training progression (Section 4.3). Table 1 defines which blocks are scored and fitted by the auxiliary model, and which receive direct generator supervision. The table in our first reply summarizes those support regions in plain language.

We would appreciate your reassessment in light of the added history-conditioned baseline, the scoring-scope ablation you requested, and the revised presentation.

Thank you for the author's response. However, I believe the performance improvements presented in the paper (quantitative and qualitative) are not significant, and the ablatiom regarding the core contributions remains insufficiently clear. Therefore, I maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.