- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the supervision-horizon gap in autoregressive video diffusion: students trained with short-horizon distribution matching receive insufficient supervision for failures emerging during extended rollouts. The authors distinguish between exposing delayed failures through longer self-rollouts and constraining cross-window dependencies through joint trajectory matching. LongForcing retains Causal Forcing’s short-horizon teacher-forcing and causal ODE-distillation stages, while extending the final DMD stage to longer student-generated trajectories supervised by an extended-horizon bidirectional teacher. Experiments cover text-to-video generation and action-conditioned world modeling, supplemented by finite-horizon theoretical analysis and qualitative stress tests.
Critical strengths
-
Relevant and practically important problem. Progressive degradation during autoregressive rollout is a substantial obstacle to streaming video generation and interactive world models. The distinction between rollout exposure and the temporal scope of teacher supervision provides a useful perspective on this problem.
-
Simple integration without additional inference overhead. The method preserves the student architecture and inference procedure, concentrating changes in training. This makes it potentially useful for existing few-step autoregressive generation pipelines.
-
Encouraging evidence beyond short-video benchmarks. The 60-second interactive ablation improves action score from 64.39 to 88.26 and strict accuracy from 42.89 to 85.16. The time-resolved drift analysis and human preference study provide complementary evidence that the longer-supervision configuration improves perceptual rollout quality.
Critical weaknesses
-
Limited methodological novelty relative to closely related work. The central intervention extends standard DMD to longer trajectories using a long-horizon teacher. Context Forcing already studies long-context teacher supervision of student self-rollouts, while Self-Forcing++ supervises segments sampled from long generated trajectories. LongForcing’s distinction of jointly scoring a longer, contiguously noised path is meaningful, but its independent benefit is not established through direct comparisons with these alternatives.
-
The horizon ablation does not isolate supervision length. Section 6.3 states that each text-to-video rollout length uses a separately trained teacher checkpoint. Consequently, changes in supervision horizon are confounded with teacher quality and potentially training resources. The results are also substantially non-monotonic: 30-second Total scores are 81.87, 78.02, 79.18, and 82.83 for increasing horizons. These results establish that the longest tested configuration performs best, but do not establish a reliable temporal scaling relationship. Compute-matched comparisons are also needed.
-
The theoretical analysis is insufficiently connected to the implemented objective. The drift guarantees depend on clean-path KL divergence, whereas training minimizes divergence between noise-perturbed distributions using estimated scores. The main text does not provide a quantitative bridge between these quantities or incorporate score-estimation error. Moreover, Proposition 4.3 assumes teacher recovery from degraded states and sufficiently small conditional student–teacher mismatch, without empirically validating these conditions. The analysis offers a useful conditional interpretation, but limited algorithm-specific guarantees.
-
The claimed advantage over local matching requires more precise qualification. Proposition 4.1 concerns short-window marginal distributions. It does not establish that local conditional matching with sufficiently informative history cannot constrain the joint trajectory distribution. This distinction is particularly relevant to continuation-conditioned methods such as Context Forcing. A controlled comparison is necessary to show that full-path scoring improves upon history-conditioned local supervision.
Questions for authors
-
Can you fix a single long-horizon teacher and compare supervision lengths under matched training budgets? How much of the reported improvement remains after controlling for teacher checkpoint quality?
-
Under identical student initialization and rollout exposure, how does joint long-path DMD compare with short-window scoring and long-history-conditioned continuation matching?
-
What gradient truncation or KV-cache detachment is used during training? How closely does the implemented update approximate the gradient of the stated joint-path objective?
-
What are the long-horizon teacher’s training data, additional training cost, and independently measured capabilities? Can it demonstrably recover from degraded student-generated histories?