acceptodds
Under review as a conference paper at ICLR 2027

Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

Abstract

Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
autoregressive video generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mSZ8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Rollout-Marginal Distillation (RMD), which scores generated chunks independently and subsequently applies video-level distillation to improve temporal coherence. It addresses error accumulation in long-horizon autoregressive video generation, where joint video supervision can reinforce artifacts in imperfect generated context. Trained on five-second rollouts, RMD outperforms Self Forcing and Causal Forcing on 60-second VBench-Long evaluations, with an additional qualitative example extending to 500 seconds.

Critical strengths
  1. A clear distinction between generation context and scoring context. Equations (4)–(5) explicitly formulate the rollout-induced chunk marginal while preserving conditional autoregressive generation. This provides a concrete approach to separating local visual-quality supervision from the influence of imperfect generated context. Subsequent video-level refinement addresses the temporal relationships omitted by independent scoring (Sections 3.2–3.4).

  2. Strong long-horizon results supported by relevant ablations. Evaluation covers 944 prompts with five generations per prompt. With chunk size one, the 60-second Total score exceeds both evaluated baselines by more than ten points; improvements also hold with chunk size three. Marginal-only, video-DMD-only, and random-exit comparisons, together with temporal-consistency metrics, support the complementary roles of independent quality supervision and temporal refinement. Inference retains the same architecture and sliding-window protocol without additional modules (Section 4, Tables 1–2).

Critical weaknesses
  1. Incomplete training details limit attribution of the improvements. The video-DMD-only control is not explicitly related to full RMD in terms of total generator updates, fake-score updates, learning-rate schedules, and computational budget. The purpose and data source of continuation-teacher adaptation are explained, but adaptation sample counts, LoRA rank, and training schedules remain unspecified. These details are necessary to assess the contribution of the objective change separately from other training choices and to estimate reproduction cost (Sections 3.3–4.1, Table 2).

  2. The explanation for artifact accumulation remains supported primarily by illustrative and indirect evidence. Equation (3) establishes that joint scoring depends on the surrounding sequence, and Figure 2 presents an example consistent with the proposed mechanism. However, these observations do not directly isolate how historical artifacts alter the teacher’s correction direction or establish how much this mechanism accounts for the long-horizon gains. A controlled analysis of scoring context, or a more explicit qualification of the causal interpretation, would strengthen the connection between the motivation and the experimental findings (Section 3.1, Figure 2).

Questions for authors
  1. Does video-DMD-only match full RMD in generator updates, fake-score updates, and learning-rate schedules? Please specify the corresponding configurations.

  2. What sample count, LoRA rank, training schedule, and additional computational cost are used for continuation-teacher adaptation?

  3. With the generated chunk and scoring noise fixed, how does changing artifacts in the scoring context affect the teacher’s correction direction? What evidence distinguishes this mechanism from other possible explanations for the observed improvements?

1 comment

Thank you for the constructive review and for recognizing our clear distinction between generation and scoring contexts, as well as the strength of our long-horizon results. The additional experimental settings you suggested help further substantiate our claims. We address your concerns below.

W1 & Q1: Budget-Matched Ablations for the Training Objective

Our original video-DMD-only ablation was designed to measure the impact of omitting the initial rollout-marginal training phase, which inherently resulted in fewer total updates. To isolate the contribution of the training objective under equal compute, we added two budget-matched ablations evaluated on the 60-second VBench-Long protocol:

SettingGenerator / fake-score updatesGenerator / fake-score LRTotal ↑Quality ↑Semantic ↑
Original ablation (video-DMD only)160 / 8002e-6 / 4e-772.4478.6847.50
Extended video-DMD (matched updates)~300 / 1,5002e-6 / 4e-774.0879.4752.49
Video-DMD with RMD's sequential LR schedule310 / 1,5502e-5 / 2e-5 → 2e-6 / 4e-762.1771.4924.89
Full RMD (ours)310 / 1,5502e-5 / 2e-5 → 2e-6 / 4e-781.2684.5268.24

These ablations confirm that RMD's performance gains do not simply come from extended training compute. Extending video-DMD at its original learning rate provides only a marginal gain (Total: 74.08). Furthermore, forcing video-DMD into our sequential learning rate schedule destabilizes the model (Total collapses to 62.17). This reinforces our original conclusion: the rollout-marginal objective itself is the primary driver of the long-horizon improvements.

W1 & Q2: Continuation-Teacher Adaptation Details and Cost

To detail the adaptation process, the continuation-teacher and fake-score models are one-time offline initializations.

  • Data: A fixed pool of 70,000 model-generated (not real) latents.
  • LoRA Rank: 128 (applied to attention projections and FFN).
  • Training Schedule: The teacher LoRA requires 3,800 optimizer steps (effective batch size 8), and the fake-score initialization takes 5,100 steps.

Because this is purely an offline pre-training setup to prepare the scoring targets, student generation during inference remains standard, incurring zero additional modules, memory, or computational cost.

W2 & Q3: Controlled Analysis of the Scoring Context

We agree that a controlled analysis of the scoring context strengthens the connection between our motivation and experimental findings. Following your suggestion to fix the generated chunk and scoring noise, we conducted this targeted evaluation.

For each of six prompts, we extracted a 9-latent-frame window. Across all conditions, the 3-frame target chunk, diffusion timestep, and scoring noise were kept identical; only the 6-frame history shown to the scoring models was modified. We introduced two types of history interventions:

  1. Synthetic artifacts: Increasing saturation (1.5× to 2.5×) and contrast (1.15× to 1.45×) of the clean history.
  2. Degraded-statistics transfer: Shifting the per-channel mean and variance of the clean history toward those of a severely degraded video-DMD rollout (at ~40 seconds), transferring feature statistics without copying exact pixels.

We then measured the relative change and directional shift (cosine similarity) of the diagnostic DMD correction signal, g=x0fake−x0realg = x_{0_fake} - x_{0_real}:

History Intervention (Target and noise fixed)Relative change in ggDirectional cosine similarity to clean gg
Mild color artifact (saturation 1.5×, contrast 1.15×)55%0.85
Strong color artifact (saturation 2.5×, contrast 1.45×)72%0.75
50% transfer of degraded-history feature statistics61%0.82
100% transfer of degraded-history feature statistics80%0.70

These results verify our theoretical mechanism. Progressively degrading the generated history substantially changes the score-difference signal, even when the target chunk and scoring noise are held identical.

Reviewer_QAsiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds on the observation that, when correcting errors in autoregressive rollouts, video-level DMD can prioritize temporal consistency at the expense of high-quality score supervision. It proposes a two-stage training approach: the first stage uses independent chunk-level supervision to focus the teacher’s corrective signal on visual quality, while the second stage restores standard video-level DMD to improve temporal coherence. The experiments demonstrate the approach’s effectiveness in improving the visual quality of long-horizon video generation.

Critical strengths
  1. The observations about the DMD teacher’s score predictions offer useful insight into long-horizon degradation. The proposed method is simple and effective in the reported experiments, providing a practical way to improve long-horizon visual quality.

  2. The problem and motivation are clearly presented. The method is easy to understand, and the experimental settings are clearly described.

Critical weaknesses
  1. Limited baseline coverage. The baseline comparison is not sufficiently comprehensive. The reviewer understands the authors’ choice to compare against methods that match their sink-free inference setting. However, the strong long-horizon results reported by other approaches suggest that the DMD teacher’s trade-off between visual quality and temporal consistency can be mitigated, or may not be the dominant bottleneck in practice. Although these methods do not exactly match the proposed setting, the current comparisons leave the reviewer uncertain about how broadly the paper’s diagnosis applies. Comparisons with representative recent approaches, together with a discussion of the differences in their settings, would help clarify this point.

  2. Limited evidence for the benefit of video-level refinement. The aggregate results of the marginal-only and full RMD variants in Table 2 are relatively close. Although the reported temporal metrics improve with refinement, the reviewer would appreciate side-by-side ablation videos to assess the perceptual differences, particularly flickering and cross-chunk temporal coherence.

Questions for authors
  1. Could the authors compare with more recent baselines or clarify why their settings are not comparable?

  2. Could the authors provide side-by-side videos of marginal-only and full RMD with matched prompts and seeds to demonstrate the benefit of video-level refinement?

1 comment

We sincerely thank the reviewer for recognizing the value of our observations and finding our approach simple, effective, and clearly presented. Your questions prompt a highly valuable discussion on the broader context of our approach, which we address directly below.

W1 & Q1: Baseline coverage and the DMD scoring bottleneck

You raised a very fair question: given the strong long-horizon results of other recent approaches, is the DMD scoring trade-off truly the dominant bottleneck in practice?

Our initial evaluation focused exclusively on sink-free methods to strictly isolate the impact of training objectives, deliberately excluding inference-time memory variables. However, to answer your question directly, we have now evaluated RMD against LongLive [1], Rolling Forcing [2], and Rolling Sink [3] on the full 60-second VBench-Long protocol.

MethodFrame sinkTotal ↑Quality ↑Semantic ↑
Self ForcingNo77.4481.9959.22
Causal ForcingNo74.8880.5352.27
LongLiveYes80.4882.9370.65
Rolling ForcingYes80.3182.8170.32
Rolling SinkYes81.1083.7170.66
RMD (ours)No81.4885.0367.24

Rather than negating the DMD bottleneck, these results reveal two distinct, highly effective avenues for scaling video generation. Sink-based baselines bypass error accumulation by altering the inference architecture—explicitly anchoring generation to initial frames, which excels at maintaining semantic alignment.

In contrast, RMD tackles the problem purely through the training objective. By resolving the context-coupled scoring bottleneck, RMD achieves the highest Quality (85.03) and Total scores (81.48) in a strictly sink-free inference setting. This confirms that while architectural memory (sinks) is a powerful tool for semantic preservation, the DMD scoring coupling remains the critical bottleneck for maximizing pure visual fidelity.

W2 & Q2: Visual evidence for video-level refinement (Marginal-only vs. Full RMD)

You are absolutely right that aggregate temporal metrics do not fully convey the perceptual difference made by the refinement stage. To provide the direct visual evidence you requested, we generated side-by-side comparisons of marginal-only training versus full RMD using identical prompts and seeds:

Anonymous Link: Marginal-only vs. Full RMD

These visual results clearly validate the necessity of the two-stage design. Independent marginal scoring successfully injects high visual realism, but lacking cross-chunk constraints, it naturally leads to temporal flickering. The full RMD videos demonstrate exactly how the subsequent video-DMD refinement visually smooths these transitions and structurally restores cross-chunk coherence.

References

[1] LongLive: Real-time Interactive Long Video Generation

[2] Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

[3] Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion

Reviewer_xwKROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Rollout-Marginal Distillation (RMD) to address error accumulation and visual quality degradation in long-horizon autoregressive video generation.

The authors argue that existing video-level Distribution Matching Distillation (DMD) couples the supervision of each generated chunk with its surrounding temporal context. When previously generated chunks contain visual artifacts, the teacher may preserve these artifacts to maintain temporal consistency, thereby limiting its ability to correct the visual quality of subsequent chunks.

To address this issue, RMD adopts a two-stage training strategy. First, it independently matches the marginal distributions of generated chunks against an adapted chunk-level teacher, providing context-independent visual quality supervision. Second, it applies video-level DMD refinement to restore temporal coherence across chunks.

Experiments based on Wan 1.3B and Wan 14B show substantial improvements over Self Forcing and Causal Forcing on VBench-Long, with quantitative evaluations up to approximately 60 seconds and a qualitative example extending to 500 seconds.

Critical strengths

S1. Important problem and clear motivation.

Error accumulation in long-horizon autoregressive video generation is a significant and practically relevant challenge. The paper provides an interesting perspective on the potential conflict between local visual quality correction and temporal consistency under video-level distribution matching.

S2. Intuitive and practical framework.

The proposed two-stage approach provides a straightforward way to separate local visual quality supervision from temporal coherence refinement. It does not require changes to the generator architecture or introduce additional inference-time computational overhead.

S3. Substantial improvements in long-horizon generation.

RMD demonstrates clear improvements on VBench-Long across two chunk configurations. For chunk size 1, the total score increases from 70.94 with Self Forcing to 81.26 with RMD. Figure 5 further shows that RMD maintains relatively stable evaluation scores as the generation horizon increases from 10 to 60 seconds, while the baselines experience considerable degradation.

S4. Useful ablation studies.

The comparisons between marginal-only training, video-level DMD only, random denoising exits, and the full RMD pipeline provide useful evidence for the proposed design. In particular, the results suggest that marginal supervision improves local visual quality, while subsequent video-level refinement helps restore temporal coherence.

Critical weaknesses

W1. Lack of human evaluation to validate perceptual improvements.

A major concern is the absence of systematic human evaluation. Although RMD achieves substantial improvements on VBench-Long, it remains unclear whether these numerical gains translate into perceptually meaningful improvements in long-horizon video generation.

Long-form video quality involves not only local visual fidelity but also natural motion, subject consistency, meaningful scene evolution, and the absence of repetitive or nearly static content. Improvements in automated metrics do not necessarily guarantee corresponding improvements in these perceptual aspects.

The qualitative examples are encouraging but insufficient to establish consistent human preference at scale.

I would encourage the authors to conduct a pairwise human GSB (Good/Same/Bad) evaluation against Self Forcing and Causal Forcing, assessing overall visual quality, temporal consistency, and motion naturalness separately. Such an evaluation would substantially strengthen the practical significance of the reported improvements.

W2. Insufficient systematic evidence for extremely long-horizon generation.

One of the most compelling claims of the paper is that RMD can maintain high visual quality over 500-second rollouts despite being trained on approximately five-second sequences. However, the quantitative evaluations only cover generation durations of approximately 60 seconds, while the 500-second result is presented solely as a qualitative demonstration in Figure 1.

Although the 60-second results provide strong evidence for improved extrapolation beyond the training horizon, they do not necessarily establish that similar robustness holds over several hundred seconds.

Moreover, the extremely long examples shown in Figure 1 primarily involve relatively simple or repetitive dynamics, such as ocean waves, desert landscapes, and flames. It remains unclear whether the method can maintain comparable quality under complex object interactions, sustained character motion, or evolving scene dynamics.

A more systematic evaluation at longer horizons, such as 120, 300, and 500 seconds, across diverse scenarios would be necessary to substantiate the broader long-horizon claims.

W3. Insufficient isolation of the marginal-matching objective from additional teacher adaptation.

The paper attributes its improvements primarily to replacing joint video-level supervision with independent chunk-level marginal matching.

However, the practical implementation also involves adapting the Wan 14B teacher through LoRA on continuation chunks extracted from videos generated by the original bidirectional model.

Consequently, the proposed method differs from conventional video-level DMD not only in its supervision granularity but also in its teacher adaptation procedure, target distribution, and additional training process.

While the existing ablations demonstrate the effectiveness of the complete RMD pipeline, they do not fully isolate the contributions of independent chunk scoring, teacher adaptation, and additional optimization.

Furthermore, the proposed explanation that video-level teachers preserve historical artifacts is supported primarily by qualitative examples in Figure 2. More systematic evidence would be valuable to establish how frequently this failure occurs and how effectively independent scoring resolves it.

Additional controlled experiments that isolate these factors would strengthen the causal interpretation of the reported improvements.

W4. Marginal matching does not directly constrain long-term dynamics and may introduce a visual-quality versus motion trade-off.

Equation (4) defines the continuation-chunk marginal by pooling chunks across different autoregressive positions and generated histories. While this formulation provides context-independent visual quality supervision, it does not directly enforce temporal relationships, long-term semantic evolution, or meaningful action progression across chunks.

A potential concern is that a generator could achieve high local visual quality while producing limited motion, repetitive actions, or insufficient scene evolution.

Although the subsequent video-level DMD refinement helps restore temporal coherence, it is unclear whether this stage is sufficient to preserve meaningful long-term dynamics, especially far beyond the training horizon.

Interestingly, Table 2 shows that the marginal-only variant achieves a slightly higher quality score than the full RMD pipeline (84.90 versus 84.52), while the full pipeline improves temporal consistency metrics. This suggests a trade-off between local visual quality and temporal coherence.

Further evaluations of dynamic degree, motion quality, action progression, repetition, and long-term semantic consistency would help establish whether RMD improves long-form video generation beyond maintaining visually plausible individual chunks.

W5. Limited baseline coverage and incomplete computational fairness analysis.

The main quantitative comparisons focus on Self Forcing and Causal Forcing. While these are relevant baselines, the paper also discusses other recent approaches related to long-horizon generation and teacher supervision, including LongLive, OPSD-V, and Context-Matched Distillation, without providing direct experimental comparisons.

Additionally, the full RMD pipeline involves extra teacher adaptation, rollout-marginal training, and subsequent video-level refinement. Although the compared methods use the same sliding-window inference protocol, the paper does not sufficiently establish their performance under matched total training compute.

Therefore, while the current results demonstrate a clear advantage over the selected baselines, they do not fully establish whether the gains can be attributed specifically to the proposed distillation objective rather than additional training or teacher adaptation.

More comprehensive baseline comparisons and reporting of training FLOPs, GPU-hours, and memory consumption would strengthen the empirical conclusions.

Questions for authors

Q1. Human Evaluation.

Have the authors conducted any human preference evaluations to verify whether the improvements on VBench-Long translate into perceptually meaningful gains? Could the authors provide pairwise GSB comparisons against Self Forcing and Causal Forcing, separately assessing visual quality, temporal consistency, and motion naturalness?

Q2. Extremely Long-Horizon Evaluation.

The paper presents a 500-second qualitative example, while the quantitative evaluations are limited to approximately 60 seconds. Could the authors provide systematic evaluations over longer horizons, such as 120, 300, and 500 seconds, across more diverse scenarios? In particular, how do visual quality, subject consistency, and motion dynamics evolve over these longer rollouts?

Q3. Contribution of Marginal Matching.

Since RMD introduces an additional LoRA-adapted chunk teacher, could the authors provide more controlled experiments to isolate the contribution of independent chunk scoring from teacher adaptation? How much of the improvement can be attributed specifically to the marginal-matching objective under matched teacher configurations and computational budgets?

Q4. Long-Term Motion and Semantic Evolution.

Because the marginal objective in Eq. (4) does not explicitly model cross-chunk temporal dependencies, how does RMD avoid maintaining high visual quality through reduced motion, repetitive content, or nearly static scenes? Could the authors provide additional evaluations of motion quality, action progression, and long-term semantic consistency?

Q5. Baselines and Training Efficiency.

Could the authors compare RMD with additional recent long-horizon autoregressive video generation approaches, ideally under a unified evaluation and training setup? Additionally, what are the actual training GPU-hours, peak memory requirements, and additional computational costs associated with the LoRA-adapted chunk teacher and the two-stage training pipeline?

2 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part I)

Thank you for the comprehensive review and for recognizing the importance of our motivation, the practicality of our two-stage framework, and the strength of our long-horizon results. We address your concerns below.

W1 & Q1: Human Evaluation on Perceptual Quality

Following your suggestion, we conducted a pairwise human Good/Same/Bad (GSB) evaluation of RMD against Self Forcing and Causal Forcing, separately assessing overall visual quality, temporal consistency, and motion naturalness. We used matched-prompt, matched-seed 60-second outputs (180 videos in total), with each pair of full-length videos played simultaneously in 60-second loops.

20 participants each evaluated eight pairs, yielding 160 pairwise comparisons in total. Their responses were recoded as Good/Same/Bad relative to RMD.

BaselineDimensionG: RMD betterS: SameB: Baseline better
Self ForcingVisual quality96.25%1.25%2.50%
Self ForcingTemporal consistency71.25%16.25%12.50%
Self ForcingMotion naturalness61.25%16.25%22.50%
Causal ForcingVisual quality93.75%6.25%0.00%
Causal ForcingTemporal consistency68.75%20.00%11.25%
Causal ForcingMotion naturalness65.00%18.75%16.25%

(Participant-clustered bootstrap resampling with 5,000 resamples confirmed a positive G-minus-B preference margin in all six comparisons.)

These results confirm a particularly strong perceptual advantage in visual quality (over 93% preference) and a clear human preference for RMD in temporal consistency and motion naturalness against both baselines.

W2 & Q2: Extremely Long-Horizon Evaluation (500 seconds)

We conducted a 500-second evaluation of chunk-1 RMD, Self Forcing, and Causal Forcing. We used 40 stratified prompts selected to cover the complex scenarios you highlighted, including sustained human actions, multiple objects, and evolving scenes. Each video was generated once to 500 seconds, and we evaluated the 60-, 120-, 300-, and 500-second prefixes of the same rollout.

MethodTotal, 60 sTotal, 120 sTotal, 300 sTotal, 500 s
RMD83.6783.5082.6181.85
Self Forcing69.7667.9266.9866.50
Causal Forcing71.1268.8166.7165.80

As shown in the table, RMD maintains the highest Total score at every evaluated duration.

AuthorsOpenReview ↗

Official Comment by Authors (Part II)

W3 & Q3: Contribution of Marginal Matching vs. Teacher Adaptation

To isolate the contribution of independent chunk scoring from teacher adaptation, we evaluated an ablation that removes the rank-128 continuation LoRA and instead uses the unmodified Wan2.1-T2V-14B teacher in RMD. All other hyperparameters and training lengths were kept fixed.

SettingTotalQualitySemanticSubject consistency
Marginal-only, adapted teacher80.6484.9063.5997.29
Marginal-only, base teacher74.6279.6554.4988.23
Full RMD, adapted teacher81.2684.5268.2498.05
Full RMD, base teacher79.7183.7663.5394.14

While teacher adaptation provides a measurable boost, Full RMD, base teacher still achieves a Total score of 79.71, outperforming the Video-DMD only control (72.44). We also conducted update-matched controls showing that extending video-DMD with its original learning rates or applying RMD's two-stage learning-rate schedule does not close this gap (the best result is 74.08). These results support the importance of the marginal-matching objective beyond teacher adaptation or additional optimization updates alone.

W4 & Q4: Long-Term Motion and Semantic Evolution

RMD uses two complementary mechanisms to avoid optimizing only isolated, visually strong chunks. First, generation remains autoregressively conditioned on the generated history. During Stage 1, only the first block uses a randomly selected denoising exit, while every subsequent chunk is trained at the final denoising step. This design avoids directly optimizing the earlier denoising steps, helping retain their temporal priors while concentrating marginal supervision on appearance refinement at the final step. Second, Stage-2 video-level refinement evaluates generated chunks together with their history, reintroducing cross-chunk temporal and semantic constraints. Accordingly, refinement increases the Semantic score from 63.59 to 68.24, subject consistency from 97.29 to 98.05, and motion smoothness from 98.79 to 98.96.

W5 & Q5: Baselines and Training Efficiency

We expanded our evaluation to include LongLive [1], Rolling Forcing [2], and Rolling Sink [3] under the same 60-second VBench-Long protocol. These methods rely on inference-time frame sinks, which retain fixed anchor frames to mitigate long-horizon drift. In contrast, RMD is sink-free and uses standard sliding-window autoregressive inference without retaining fixed anchor frames.

MethodTotalQualitySemantic
LongLive80.4882.9370.65
Rolling Forcing80.3182.8170.32
Rolling Sink81.1083.7170.66
RMD, chunk 381.4885.0367.24

Despite operating without frame sinks, RMD achieves the highest Total and Quality scores in this comparison, while the sink-based baselines retain higher Semantic scores.

Regarding efficiency, the continuation-teacher adaptation uses a fixed offline split of 70,000 model-generated video-latent records. The teacher LoRA requires 3,800 optimizer steps on eight GPUs with a global batch size of 8, and the fake-score initialization takes 5,100 steps. Full RMD uses exactly 310 generator updates and 1,550 fake-score updates.

References

[1] LongLive: Real-time Interactive Long Video Generation

[2] Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

[3] Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion

Comments on the paper

1 comment

Dear reviewers,

As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. If you have not yet responded, please indicate whether your concerns have been adequately addressed. Many thanks!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.