acceptodds
Under review as a conference paper at ICLR 2027

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

Abstract

World models, game simulators, and long-take video creation require coherent scene evolution and sustained dynamics over extended durations. Autoregressive (AR) video diffusion provides a natural framework for long-horizon generation, yet extended rollouts often become near-static or lose visual quality. We hypothesize that these failures reflect the limited guidance provided by short-video supervision on how ongoing scene dynamics develops over longer durations. This motivates us to introduce LongTake, a two-stage training pipeline built around Long-Horizon Teacher Forcing (TF) on curated real long videos. Long-Horizon TF trains the AR model to predict later frames conditioned on long ground-truth video prefixes, extending direct supervision beyond the short training horizon. This supervision is designed to help the model sustain dynamics and preserve visual quality during long-horizon generation. Our central finding is that this training stage strengthens direct initialization for distribution matching distillation (DMD) under student self-rollout, without the intermediate few-step distillation stage used in standard pipelines. Under the same five-second DMD training setup, our initialization yields substantially higher dynamic degree than short horizon TF initialization on 30-second rollouts at comparable aesthetic quality, and surpasses the evaluated baselines in both measures. Hybrid DMD further reuses this teacher to extend supervision to later frames of the self-rollout while retaining bidirectional joint supervision over the initial window. On long-horizon self-rollouts, LongTake lies on the Pareto front of dynamic degree and aesthetic quality, and Hybrid DMD attains the highest dynamic degree among evaluated methods at both 30s and 60s.

Primary area:
generative models
Keywords:
Autoregressive, Dynamic, Long video generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NMi7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LongTake uses longer video prefixes during teacher-forcing initialization, followed by 5-second DMD training. It improves the motion–quality tradeoff on 30- and 60-second videos, with optional Hybrid DMD adding supervision for later chunks.

Critical strengths
  1. A useful initialization result. LongTake improves long-video dynamics while keeping the subsequent DMD recipe at 5 seconds. The benefit is already present without Hybrid DMD.

  2. A better motion–quality tradeoff. It produces substantially more motion than SGF with a modest drop in aesthetic quality, and better aesthetic quality than RF†. The supplementary examples help show what these gains look like.

Critical weaknesses
  1. The source of the gain is unclear. Long TF changes the data, adds 3,000 updates, and uses longer prefixes. I'd like to see short-window TF trained on the same data for the same budget; Figure 5 does not separate these factors.

  2. Action quality needs a closer look. The videos show encouraging examples, but Dynamic Degree does not tell us whether the prompted actions continue late in the rollout. A targeted human evaluation would help.

Questions for authors
  1. Was λ=0.2 selected on a separate validation set?
  2. What are the total GPU-hours for each variant, including TF initialization?
3 comments
AuthorsOpenReview ↗

Answer to Reviewer NMi7

We sincerely appreciate the reviewer's interest in our research and acknowledgment of its significant contributions. We have provided detailed responses below.


1. Source of the gain.

  • We appreciate this valuable suggestion. To address the reviewer's concern, we ran the short-window TF on top of Short TF, denoted additional Short TF, that can isolate the source of the gain.

  • Additional Short TF is identical to Long TF in every setting and differs only in training on the first 81 frames of each video without a video prefix (i.e., s = 0 in Algorithm 1).

  • The table below shows that, with the same data and updates, the long video prefixes raise dynamic degree well above that of additional Short TF at comparable subject consistency and aesthetic quality. The added data and updates also help, but additional Short TF still stays below Short TF + CD in dynamic degree.

  • This supports our claim that supervising the later frames conditioned on long video prefixes, rather than the added data and updates alone, brings the gain. We have added these results to the Ablation Study in Section 5.1 and the training details of additional Short TF to Appendix C.1.

Table R1. Initializations followed by the same 5s joint DMD and evaluated on 30s rollouts as in Figure 5.

InitializationSubject Consistency ↑Dynamic Degree ↑Aesthetic Quality ↑
Short TF98.1951.3063.90
Short TF + CD96.0481.7258.82
Additional Short TF97.7473.1263.22
Long TF96.8390.7362.51

2. Prompted actions in later frames.

  • We measured whether later frames still depict the prompted action, which dynamic degree alone does not show, using a VLM judge in place of a human evaluation.

  • The judge uses the same model and settings as our VLM exposure metric. For each 60s rollout, it sees the first and last 10s with the text prompt and judges whether the subject performs the prompted action, without knowing the method or which clip comes first.

  • The table below shows that, in the last 10s, LongTake shows the prompted action about as often as the strongest baselines and as often as in its own first 10s, while keeping the high dynamic degree in Table 1. This suggests that the dynamics LongTake sustains still depict the action described in the prompt.

  • We have added these results and the judge instruction to Appendix D.4.

Table R2. Share of videos (%) in which the VLM judge finds the prompted action in the first and last 10s of 60s rollouts for the 72 dynamic-degree prompts.

Method0–10s ↑50–60s ↑
Self Forcing47.218.1
Causal Forcing44.415.3
LongLive38.937.5
Reward Forcing47.237.5
MemRoPE50.022.2
Rolling Forcing†45.822.2
Self Gradient Forcing29.220.8
Context Forcing37.515.3
LongTake34.736.1

3. Selection of λ = 0.2.

  • Thank you for this question. We clarify that λ was chosen by comparing the settings in Table 2 on the 30s prompts, without a separate validation set.

  • Every λ in Table 2 also keeps LongTake on the Pareto front of dynamic degree and aesthetic quality against the baselines on 30s rollouts, so this conclusion does not depend on the choice of λ.


4. Total GPU-hours.

  • We count every stage of each variant in the table below, including Short TF, which all variants share, and we have added this table to Appendix C.1.

  • Long TF takes the place of CD, so LongTake with either joint or Hybrid DMD costs less in total than Short TF + CD. This is because CD requires two additional networks, a frozen teacher and an EMA target, alongside the trained student, whereas TF trains a single network.

Table R3. B200 GPU-hours of each variant.

VariantShort TFLong TFCDSelf-rollout DMDTotal
Short TF + CD + joint DMD139.8–381.327.9549.0
Long TF + joint DMD139.8338.5–27.9506.2
Long TF + Hybrid DMD139.8338.5–42.9521.2

Thanks for the additional experiments, especially the short-window comparison, which clarifies the benefit of long-horizon training. I maintain my weak accept rating.

We thank the reviewer again for their insightful questions and suggestions. We are glad that our additional experiments helped clarify our contribution, and sincerely appreciate their support of our work.

Reviewer_KmkkOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposed a method to address the dynamics problem in long video generation. The key technique introduces is the Long-Horizon TF model which is combined with regular DMD to form hybrid DMD. Empirical results show that the method is very effective in boosting the dynamics of long videos.

Critical strengths
  1. The paper is well-written and easy to follow. The problem is well motivated.
  2. The proposed method analyzes a key failure of long video generations which is the dynamics problems. The dynamics usually gets lower and lower as the video gets longer. The authors identifies the problem in the teacher model and proposes to introduce long-horizon TF to mitigate the problem.
  3. The empirical results are strong compared to various baseline methods.
  4. The result of showing using better initialization is a good discovery for long video generation.
Critical weaknesses
  1. If long horizon TF is needed, does that mean the model will never be able to extrapolate?
  2. The dynamic degree seems extremely high which could indicate broken generations.
  3. Have the author considered if such problem is caused by position embedding such as RoPE simiar to Infinity-rope [1]

[1] Yesiltepe, Hidir, et al. "Infinity-rope: Action-controllable infinite video generation emerges from autoregressive self-rollout." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

Questions for authors

Please see my questions in weakness, also what's the maximum length the proposed method can generate without breaking down?

6 comments
AuthorsOpenReview ↗

Answer to Reviewer Kmkk (1/2)

We sincerely appreciate the reviewer's interest in our research and acknowledgment of its contributions. Detailed responses are provided below.


1. Extrapolation.

  • We would like to clarify that our argument does not imply that a model trained on short clips cannot extrapolate. Such a model relies on generalization beyond its training length, and inference-time methods such as [1] show that it can generate beyond that length without Long TF.

  • Our claim is narrower. In this paper, we argue that what remains limited is how well the later frames keep their dynamics and visual quality, since generated scenes can still settle into near-static or repetitive states.

  • In methods that retain early sink frames in the conditioning KV-cache [2, 3], the temporal separation between the sink frames and the recent frames grows with the video prefix (i.e., as the generation horizon grows). The later frames are then predicted from KV-cache states that short-clip training does not directly supervise.

  • Long TF provides this missing supervision by building the conditioning KV-cache from real long video prefixes and supervising the later frames that follow.

  • We agree that a carefully designed attention mechanism (including KV-cache management and attention structure) may also narrow this gap without Long TF. The two play different roles and are complementary, since such a mechanism determines how the video prefix is represented and accessed in the KV-cache states, whereas Long TF supervises the later frames predicted from them.

  • Adding Long TF to the training pipeline improves long-horizon dynamics under the cache we use, so we expect it to help such mechanisms as well, and we leave this combination to future work.


2. High dynamic degree.

  • We clarify that we do not rely on dynamic degree alone, since a high dynamic degree can be misleading. For this reason, we report ΔIQ and VLM [4] alongside dynamic degree, and together they suggest that LongTake keeps its imaging quality and exposure stable despite its high dynamic degree.

  • Additionally, for LongTake and all baselines, we evaluate subject consistency, dynamic degree and aesthetic quality in each 10s window of the 60s rollouts (Appendix D.3). LongTake keeps a high dynamic degree in every window, including those beyond the 30s supervision horizon, while its aesthetic quality stays close to or above that of the baselines with low dynamic degree.

  • In contrast, Rolling Forcing, the only baseline with a similarly high dynamic degree, loses far more aesthetic quality in later windows (e.g., drifting into close-ups). The short-horizon baselines Self Forcing and Causal Forcing lose both dynamic degree and aesthetic quality (which is expected since they are trained on 5s clips and keep no sink frames or memory in the KV-cache).

  • A VLM judge further checks whether the prompted action still appears in the last 10s of each 60s rollout for the 72 dynamic-degree prompts (Appendix D.4). LongTake shows it as often as in its first 10s.

  • We argue that the high dynamic degree of LongTake thus reflects sustained motion rather than broken generations. The anonymous project page includes 30s and 60s videos. We have added the per-window and prompted-action results to the revised paper.

AuthorsOpenReview ↗

Answer to Reviewer Kmkk (2/2)

3. RoPE as a possible cause.

  • We thank the reviewer for raising this possibility. In long-horizon generation, positions need to stay within a bounded range, as in [1] (e.g., temporal RoPE indices are re-assigned relative to the current block at every step).

  • Without bounded positions, the model would have to be trained at the generation length itself, so minute- or hour-long generation would require training videos of matching length. Such data could in principle resolve the issue, but collecting it at this scale is not realistic.

  • AR video methods such as [1] and LongTake therefore use a bounded conditioning KV-cache with bounded positions.

  • As in our response above, such a cache reuses the same positions for many different combinations of frames, since the sink frames and the recent frames can lie seconds or minutes apart yet always occupy the same slots. In short-clip training, however, the two always lie close together.

  • We therefore argue that the problem lies in the content of the conditioning KV-cache rather than in the position embedding.


4. Maximum generation length.

  • We tested rollouts of up to 240s, and those that hold up at 60s tend to hold up until the end, with its later frames sustaining the prompted action. Breakdown depends mostly on the prompt and usually appears early in the rollout.

  • We note that over very long rollouts, the same motion tends to repeat (e.g., the camera keeps turning or the subject keeps walking).

  • In this paper, we use a simple conditioning KV-cache with sink retention and FIFO eviction. A more developed KV-cache, used at inference or when building the video prefixes in Long TF, may address this, and we find it an interesting direction for future work.


[1] Yesiltepe et al., Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout.
[2] Yang et al., LongLive: Real-Time Interactive Long Video Generation.
[3] Liu et al., Rolling Forcing: Autoregressive Long Video Diffusion in Real Time.
[4] Cui et al., Self-Forcing++: Towards Minute-Scale High-Quality Video Generation.
AuthorsOpenReview ↗

Gentle Reminder

We were wondering whether our response has sufficiently addressed the reviewer's questions as the discussion period nears its end. If so, we would greatly appreciate it if the reviewer could consider updating the score to reflect this.

If the reviewer has any additional comments or questions, please let us know, and we will do our utmost to address them before the deadline. Thank you for your time and consideration.

Reviewer_rAtdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

LongTake introduces a two-stage training pipeline that uses Long-Horizon Teacher Forcing on curated real long videos to directly initialize self-rollout DMD, eliminating a separate few-step initialization stage. Its Hybrid DMD combines bidirectional joint supervision of the initial window with autoregressive conditional supervision of later chunks, achieving the highest dynamic degree among evaluated methods on both 30- and 60-second rollouts while maintaining competitive visual quality.

Critical strengths

The paper proposes methods to improve long-horizon video generation, especially in terms of dynamics. The rollouts exhibit improvements compared to the baselines.

Critical weaknesses

The baseline methods and the proposed method are trained on different data. Although they are compared using the same metric, the results are unfair. Specifically, the proposed method is initialized from the second baseline in Table 1 and trained on additional data, which makes it highly likely to rank first. As seen in “We initialize Long-Horizon TF from the released chunk-wise AR diffusion checkpoint of Causal Forcing (Zhu et al., 2026), based on Wan2.1-T2V-1.3B (Wan et al., 2025). After 3,000 additional updates, the resulting model initializes the student and serves as the frozen Long-Horizon AR teacher.”.

Questions for authors
  1. The description in line 178, “Long-Horizon TF to learn long-horizon generation from real long videos,” is inaccurate, as described in line 347: “Long-Horizon TF uses 43K video–text pairs, including OpenVidHD clips (Nan et al., 2025), and 3,000 updates to supervise generation up to 30s.”
  2. The paper lacks definitions of abbreviations when they are first introduced, such as CD.
  3. Why does combining additional data with the proposed methods decrease Subject Consistency and Aesthetic Quality despite greatly improving Dynamic Degree? Is this due to loss removal and conflicts during training?
  4. Figure 4 lacks a reference.
  5. Figure 6 lacks punctuation.
  6. It is recommended that the method figure be refined to make it self-explanatory.
3 comments
AuthorsOpenReview ↗

Answer to Reviewer rAtd (1/2)

We sincerely appreciate the reviewer's interest in our work and recognition of its contributions. We have addressed the concerns raised by the reviewer below.


1. Additional data and training.

  • We would like to clarify that LongTake starts from an earlier checkpoint of [1], rather than from the final model evaluated in Table 1. [1] builds its model in three stages. Stage 1 trains a multi-step AR diffusion model with teacher forcing (TF), Stage 2 distills it into a few-step model by ODE distillation or consistency distillation (CD), and Stage 3 refines the few-step model with DMD into the model evaluated in Table 1.

  • The earlier checkpoint is the released Stage 1 model, which is the AR diffusion checkpoint in the sentence the reviewer quotes. Figure 5 and the tables below denote this model as Short TF. LongTake trains this model with Long TF on our data in place of Stage 2 and then applies DMD. Long TF thus replaces Stage 2 rather than adding training on top of the model in Table 1, and LongTake costs less in total than the pipeline with CD, as the GPU-hours table below shows. We have added this table to Appendix C.1.

Table R1. B200 GPU-hours of each variant.

VariantShort TFLong TFCDSelf-rollout DMDTotal
Short TF + CD + joint DMD139.8–381.327.9549.0
Long TF + joint DMD139.8338.5–27.9506.2
Long TF + Hybrid DMD139.8338.5–42.9521.2
  • We note that the added data is small, since Long TF uses 43K video–text pairs for 3,000 updates whereas the pretrained backbone [2] was trained on billions of images and videos, indicating that the observed gains are unlikely to be explained solely by the additional training. One of our baselines, also trains on additional long videos at a similar scale [3], yet its dynamic degree stays below that of several baselines in Table 1 trained only on short videos. This partially supports our view that additional long videos alone do not explain the gain of LongTake.

  • Training on additionally curated data is also common, as in [4], and we regard which data to add and how to use it in the training pipeline as part of our contribution.

  • To address the reviewer's concern directly, we checked whether the gain simply comes from fine-tuning on the additional data. We trained a short-window TF on top of Short TF, denoted additional Short TF, that is identical to Long TF in every setting, including the data and the 3,000 updates, and differs only in training on the first 81 frames of each video without a video prefix (i.e., s = 0 in Algorithm 1). We applied the same DMD to each initialization and evaluated the results on 30s rollouts as in Figure 5.

  • The initialization table below shows that additional Short TF raises dynamic degree well above Short TF, so the added data and updates do contribute. Additional Short TF still stays below Short TF + CD in dynamic degree.

  • Under the identical setting, Long TF raises dynamic degree well above additional Short TF at comparable subject consistency and aesthetic quality, and it exceeds Short TF + CD in both dynamic degree and aesthetic quality.

Table R2. Initializations followed by the same 5s joint DMD and evaluated on 30s rollouts as in Figure 5.

InitializationSubject Consistency ↑Dynamic Degree ↑Aesthetic Quality ↑
Short TF98.1951.3063.90
Short TF + CD96.0481.7258.82
Additional Short TF97.7473.1263.22
Long TF96.8390.7362.51
  • These results show that the gain comes from supervising the later frames conditioned on long video prefixes, rather than from fine-tuning on the additional data alone. We have added them to the Ablation Study in Section 5.1 and the training details of additional Short TF to Appendix C.1.

2. "Real long videos" in line 178.

  • We clarify that, by long, we mean longer than the 5s clips used in standard AR training. As detailed in Appendix C.2, Long TF uses 40K real videos of up to 30s from OpenVidHD and 3K synthetic 5s clips from [1], so all supervision beyond the 5s horizon comes from real videos. Since Long TF starts from Short TF, the synthetic clips keep injecting its original TF signal without a video prefix (i.e., s = 0 in Algorithm 1) to prevent forgetting.
AuthorsOpenReview ↗

Answer to Reviewer rAtd (2/2)

3. Lower subject consistency and aesthetic quality.

  • We thank the reviewer for this question. The modest decreases in subject consistency and aesthetic quality are consistent with a trade-off observed as dynamics increase. Subject consistency measures similarity across frames, which favors nearly static videos [5, 6]. Greater subject motion or a change in viewpoint can therefore lower the score even when the subject retains its identity. The slight decline in aesthetic quality also follows the trend reported in [5].

  • As in our response above, the same trade-off appears between additional Short TF and Long TF, which both skip CD and use the same joint DMD objective. This supports our interpretation that the decreases are associated with stronger dynamics, not with the omission of CD or the combination of joint and conditional DMD losses.

  • The per-window evaluation of 60s rollouts that we added in Appendix D.3 shows the same pattern. Several methods with low dynamic degree keep high subject consistency and aesthetic quality, while LongTake keeps aesthetic quality close to theirs with slightly lower subject consistency at a far higher dynamic degree in every window.


4. Minor corrections.

  • We thank the reviewer for pointing these out (the definition of CD, the reference to Figure 4 and the punctuation of the Figure 6 caption), and we have checked the rest of the paper for similar issues and corrected all of them in the revised paper.

  • We agree that the method figure (Figure 1) can be made easier to understand on its own, and we have revised its caption to state the starting model of each stage and the frozen teachers that supervise the student.


[1] Zhu et al., Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation.
[2] Wan et al., Wan: Open and Advanced Large-Scale Video Generative Models.
[3] Chen et al., Context Forcing: Consistent Autoregressive Video Generation with Long Context.
[4] Zhao et al., Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation.
[5] Liao et al., Evaluation of Text-to-Video Generation Models: A Dynamics Perspective.
[6] Huang et al., VBench: Comprehensive Benchmark Suite for Video Generative Models.
AuthorsOpenReview ↗

Gentle Reminder

We were wondering whether our response has sufficiently addressed the reviewer's questions as the discussion period nears its end. If so, we would greatly appreciate it if the reviewer could consider updating the score to reflect this.

If the reviewer has any additional comments or questions, please let us know, and we will do our utmost to address them before the deadline. Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We sincerely thank the reviewers for their thoughtful feedback. We are pleased that the reviewers recognized the improvements of LongTake over the baselines in sustaining dynamics (Reviewer rAtd, Kmkk, NMi7), found our initialization result useful (Reviewer Kmkk, NMi7), found its motion–quality trade-off favorable (Reviewer rAtd, NMi7), and found the problem well motivated and the paper well written (Reviewer Kmkk).

We believe LongTake makes a meaningful contribution by showing that supervising the later frames conditioned on real long video prefixes strengthens the initialization of self-rollout DMD, without a separate few-step initialization stage.

The reviewers raised two shared concerns, namely (1) whether the gain comes from the additional data and training (Reviewer rAtd, NMi7) and (2) whether the high dynamic degree reflects sustained, prompted motion in later frames rather than broken generations (Reviewer Kmkk, NMi7). In response to their valuable feedback, we provide general responses below.


1. Source of the gain.

  • As the reviewers pointed out, the initialization comparison in Figure 5 did not separate the effect of the long video prefixes from that of the added data and updates. We therefore added a short-window TF control (additional Short TF) that uses the same data and the same number of updates as Long TF but no video prefix.

  • Under the same DMD, the long video prefixes raise dynamic degree beyond what the added data and updates provide, which shows that the additional gain comes from supervising the later frames conditioned on long video prefixes. We have added this comparison to the Ablation Study in Section 5.1.


2. Sustained motion in later frames.

  • For LongTake and all baselines, we report subject consistency, dynamic degree and aesthetic quality in each 10s window of the 60s rollouts (Appendix D.3). LongTake keeps a high dynamic degree in every window, including those beyond the 30s supervision horizon, while its aesthetic quality stays close to or above that of the baselines with low dynamic degree.

  • Using a VLM judge in place of a human evaluation, we check whether the prompted action appears in the first and last 10s of each 60s rollout for the 72 dynamic-degree prompts (Appendix D.4). In the last 10s, LongTake shows the prompted action about as often as the strongest baselines and as often as in its own first 10s.


Notable changes are highlighted in magenta in the revised manuscript. We have made every effort to address all concerns raised, as detailed in the individual responses below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.