acceptodds
Under review as a conference paper at ICLR 2027

Glance Forcing: Making Bidirectional Diffusion Autoregressive at a Glance

Abstract

Autoregressive generation underlies real-time, causal video synthesis and is the backbone of video world models. Today's dominant approach distills a bidirectional diffusion model into a causal few-step student, but existing pipelines make this conversion prohibitively expensive, requiring offline Probability Flow ODE (PF-ODE) trajectories that cost tens of thousands of GPU-hours and terabytes of storage. We show it can be made dramatically more efficient. Pushing this efficiency to its extreme, we reduce trajectory supervision to a single sample, the strictest setting possible, and find that it causes severe quality collapse. We trace this to the teacher's trajectory geometry: the high-noise regime, governing semantic layout and motion, and the low-noise regime, governing fine texture exhibit fundamentally different curvature and optimization behavior, so a single monolithic student cannot jointly fit both, causing severe representation interference. Motivated by this, we propose Glance Forcing, a lightweight distillation framework that decouples the denoising trajectory into two complementary slow and fast experts, each guided by a tailored, stage-specific objective, while keeping the pretrained teacher entirely frozen. This decoupling further reveals a general mechanism underlying few-step distillation: PF-ODE distillation mainly straightens the few-step trajectory geometry, while the subsequent distribution-matching stage independently governs distributional quality. Using only one PF-ODE trajectory, Glance Forcing already surpasses the fully-trained state-of-the-art baseline, raising the VBench Total score from 84.04 to 84.22 while cutting trajectory-collection cost from thousands of GPU-hours to about one A100-GPU-hour, with markedly larger gains in dynamic degree, VisionReward, and instruction-following, and consistent preference from human raters over strong baselines.

Primary area:
generative models
Keywords:
Causal Video Generation, Diffusion Distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uaEjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

When distilling a bidirectional diffusion model into a few-step AR student model, the trajectory supervision for the PF-ODE stage can be reduced to a single trajectory, by splitting the denoising trajectory between two different experts based on noise levels.

Each of the two experts is supervised using a loss function with a different granularity, and the final results show improvements over previous baselines, demonstrating the effectiveness of this simple approach.

Critical strengths
  1. The diagnostic is well supported in the motivation section. Around 92pp of the total curvature mass at t >= 0.9 is a surprising but solid discovery.
  2. The huge improvement in Initialization time cost and extra data storage cost is stunning. If this advantage can be transferred to bigger backbones like LTX 2.3 / MiniMax H3, it will significantly reduce the computational resources required to distill these models.
  3. The figures in the paper are concise and easy to understand.
Critical weaknesses
  1. Question on the novelty. The Glance Forcing framework closely follows DCM (Lv et al., ICCV 2025)[1], which the paper does not cite. In general, both work split the ODE trajectory into a high-noise expert and a low-noise expert. In detail, the two papers also share many similarities, including:
    a. Sec. 3.3's sub-trajectory definition (L257-L263) tracks DCM Sec. 3.2 almost sentence by sentence
    b. Eqs. (9) and (10) are identical to DCM Eqs. (6) and (7) apart from renaming the experts (F_SemE, F_DetE → F_Slow, F_Fast).
    c. Highly similar drawing styles in the figures. For example, Fig.5 (a) (b) v.s. DCM Fig.3 (a) (b); Fig.10 v.s. DCM Fig. 10. In addition, Fig. 5's caption and DCM Fig. 3’s are also close paraphrases.

    Given so many similarities, it is strange that DCM is not cited. The author should re-examine the novelty of his contributions.

  2. The user study involved only 3 participants, so the results may lack statistical significance. Additionally, the appendix mentions that human evaluation covered four aspects, while the authors did not report the win rate for each.

  3. In supplementary video3 (train video), the movement of the train and the shape of the smoke are clearly unnatural.

[1]. Lv, Zhengyao et al. “Dual-Expert Consistency Model for Efficient and High-Quality Video Generation.” 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (2025): 14983-14993.

Questions for authors
  1. Scope of the efficiency claim. Fig. 1 says “GF reduces training cost by 11600×”, but how much does GF improve efficiency across the entire training process?
    a. Please report the cost of the three stages separately in a consistent unit, ideally GPU-hours, so that the share corresponding to the 11,600x reduction is visible.
    b. State whether the baseline rows in Tab. 1 were re-run under this pipeline or quoted from the original papers. If re-run, report their measured GPU hour cost on the same hardware; if quoted, label them as such.

  2. Please clarify the reasons for the similarities between your paper and DCM, and restate the originality of your contributions.

3 comments

Q1(a). Three Stages Cost

We report the training cost and extra storage of each stage to provide a comprehensive comparison of the entire pipeline:

ModelAR DiffusionODEDMDTotal
Training CostExtra StorageTraining CostExtra StorageTraining CostExtra StorageTraining CostExtra Storage
Causal Forcing2300011600190029000168001900
Glance Forcing23000112900052011

Note: Training Cost is measured in GPU-hours, and Extra Storage is measured in GB.

As shown in the table, the ODE distillation stage is the primary computational bottleneck in Causal Forcing, accounting for approximately 70% of its total training cost. By leveraging our single-trajectory PF-ODE initialization, Glance Forcing reduces the cost of this stage from 11,600 to just 1 GPU-hour.

Notably, the subsequent DMD stage is retained in Glance Forcing with the same training cost as in Causal Forcing. Since DMD distills directly from online teacher predictions using 25k text prompts, it eliminates the need to pre-generate or store a large-scale text-video pair dataset, resulting in zero extra storage."

Consequently, the overall training cost of the entire pipeline is reduced by approximately 70%, while the extra storage requirement drops by about 99%.

Q1(b). Setup

Table 1 results are quoted from the original papers, mostly from Causal Forcing [1]. For a fair comparison, we follow its hardware (H100 GPUs) and evaluation protocols.

Q2. Contributions

Thank you for raising this important concern. We acknowledge that DCM is closely related to our work, and that its omission from the submitted manuscript was an oversight. In fact, DCM was discussed in an earlier version of our Related Work section on phase-aware diffusion modeling, but this discussion was removed accidently during revision. We sincerely apologize.

We have updated the PDF (changes in blue). For more details, please refer to the Phase-aware Denoising section on page 10 of the main manuscript.

Regarding the originality, we would like to clarify fundamental differences from DCM:

  1. Motivation and timestep allocation. DCM adopts uniform timestep allocation across phases. In contrast, Glance Forcing is motivated by the varying sensitivity of different denoising phases and employs an asymmetric training schedule, densely sampling four steps in the Slow phase while sparsely sampling four steps in the Fast phase.

  2. Training strategy. DCM trains a full-parameter high-noise expert to initialize a low-noise LoRA expert. In contrast, Glance Forcing independently trains two LoRA experts, treating the phases as distinct tasks. Experiments confirm that this strategy achieves better performance (see Q3 of Reviewer 1Tvy for details).

  3. Distillation objective. Glance Forcing addresses representation interference under single-trajectory supervision, enabling extremely low-cost AR diffusion distillation, while DCM requires tens of thousands of text-video pairs for training.

We will include DCM in Related Work and clarify these distinctions.

W2. User Study

We acknowledge that the original user study was limited in scale. To align with Causal Forcing [1], which involved 10 annotators, we recruited seven additional participants. All newly recruited participants hold or are currently pursuing master's or doctoral degrees and have substantial experience in generative modeling, ensuring a level of expertise comparable to that of the original annotators.

The results are summarized below:

BaselineOurs winswinsTotalWin rate
Self Forcing4346650086.8%
Causal Forcing32717350065.4%

The expanded study yields results consistent with our original findings, showing that our method is consistently preferred over both baselines.

Regarding the missing aspect-wise win rates, we would like to clarify that the four aspects mentioned in the appendix were considered jointly by annotators when making judgment. They were not designed as four independently rated evaluation dimensions. Therefore, Table 2 reports the aggregated overall preference results.

W3. Train Video

To further investigate this issue, we have updated the supplementary materials with additional comparisons in Demo/Train, including Causal Forcing, Self Forcing, and CausVid, on the same train example.

These methods exhibit similar unnatural motion and smoke artifacts, suggests that such artifacts are primarily inherited from the base model's generation limitations, rather than being introduced by our framework.

References

[1] Zhu, Hongzhou, et al. "Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation." arXiv preprint arXiv:2602.02214 (2026).

We are grateful to the reviewer for the thoughtful feedback. We have carefully considered all the comments and provided detailed responses in rebuttal. If there are any remaining questions that require further clarification, we would be glad to address them.

I appreciate the authors' addition of the supplementary experiments.

However, the authors have not provided the necessary clarification regarding the similarities noted in points a., b., and c. of W1 concerning the two papers mentioned. So I have decided to maintain my score.

Reviewer_nYbFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Glance Forcing, an efficient framework for converting bidirectional video diffusion models into few-step autoregressive generators. The key observation is that high-noise and low-noise portions of the denoising trajectory exhibit substantially different geometry and optimization behavior, making a single student difficult to train under extremely sparse trajectory supervision. The proposed method therefore uses two LoRA experts, a Slow-LoRA for high-noise layout and motion modeling and a Fast-LoRA for low-noise detail refinement, together with temporal-frequency and spatial high-pass objectives. The paper further argues that PF-ODE initialization mainly serves to straighten the few-step trajectory, while subsequent DMD is responsible for distributional refinement. Using only one PF-ODE trajectory for initialization, the resulting model reaches a final VBench score of 84.22 versus 84.04 for Causal Forcing, while greatly reducing trajectory-collection cost.

Critical strengths
  1. The paper targets an important bottleneck in autoregressive video-diffusion distillation, constructing and storing large PF-ODE datasets. Reducing this stage to a single trajectory while maintaining competitive final generation quality is practically significant.
  2. The trajectory analysis shows strong curvature concentration in the high-noise regime, motivating separate experts for different denoising phases. The slow/fast, temporal-frequency, and high-pass ablations consistently improve the final result, and the human study further supports the perceptual gains.
Critical weaknesses
  1. Only the PF-ODE initialization uses one trajectory; the final 84.22 result still corresponds to a subsequent DMD stage using 25k training samples. So the reported orders-of-magnitude reduction applies primarily to trajectory collection rather than to the cost of the complete distillation pipeline. This distinction is important when assessing the practical efficiency claim.
Questions for authors

see weaknesses

3 comments

We thank the reviewer for pointing out this important distinction. To provide a more comprehensive assessment, we report the training cost and extra storage of all three stages separately:

ModelAR DiffusionODEDMDTotal
Training CostExtra StorageTraining CostExtra StorageTraining CostExtra StorageTraining CostExtra Storage
Causal Forcing2300011600190029000168001900
Glance Forcing23000112900052011

Note: Training Cost is measured in GPU-hours, and Extra Storage is measured in GB.

As shown in the table, the ODE distillation stage is the primary computational bottleneck in Causal Forcing, accounting for approximately 70% of its total training cost. By leveraging our single-trajectory PF-ODE initialization, Glance Forcing reduces the cost of this stage from 11,600 to just 1 GPU-hour.

Notably, the subsequent DMD stage is retained in Glance Forcing with the same training cost as in Causal Forcing. Since DMD distills directly from online teacher predictions using 25k text prompts, it eliminates the need to pre-generate or store a large-scale text-video pair dataset, resulting in zero extra storage."

Consequently, the overall training cost of the entire pipeline is reduced by approximately 70%, while the extra storage requirement drops by about 99%.

We will clarify this distinction and update the efficiency discussion in the revised manuscript.

I thank the authors for their detailed responses and additional experimental results, which help clarify my concerns and provide further insights into the proposed method. I will maintain my original score.

Many thanks once again for your time and thoughtful review of our manuscript.

Sincerely,

The Glance Forcing Authors

Reviewer_1TvyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how to reduce the cost of distilling bidirectional video diffusion models into few-step autoregressive models, with a particular focus on reducing the number of trajectories required for ODE initialization. The authors propose Glance Forcing, which uses two LoRA adapters on top of a frozen shared backbone to model the high-noise and low-noise denoising stages separately. The method further introduces a temporal frequency spectrum loss and a spatial high-pass loss, followed by standard DMD for distributional refinement.

Critical strengths
  1. Generating and storing large-scale ODE trajectories introduces computational and storage overhead in existing distillation pipelines.
  2. Despite reducing the number of trajectories used for ODE initialization, the proposed method maintains competitive generation quality.
Critical weaknesses
  1. The proposed division between a high-noise expert for layout and motion and a low-noise expert for detail refinement appears closely related to the expert specialization in Wan2.2 A14B.
  2. The single-sample setting applies only to ODE initialization, while the DMD stage still relies on a larger training dataset. The reported data efficiency should be interpreted in terms of ODE trajectory supervision rather than the overall distillation pipeline.
  3. There are several numerical inconsistencies between the main text and the tables (e.g., 84.58 in Section 4.2 vs. 84.33 in Table 1). The authors need to proofread the manuscript.
Questions for authors
  1. How many candidate samples were considered, and was the final sample selected based on validation performance or preliminary experiments?
  2. Under a matched trainable parameter budget, how does the dual-LoRA design compare with a single-LoRA baseline?
  3. Could similar improvements be achieved by separating the distillation objectives across noise stages while sharing the same trainable parameters?
2 comments

Q1. Candidate Samples

In total, we considered 11 candidate samples, including the six alternative samples evaluated in the controlled ablation study in Section 4.4 and four additional samples (old house, country map, cat, and cheering crowds) used to evaluate sensitivity to sample selection.

We selected Boy as the default sample because it achieved the best overall quantitative and qualitative performance (Table 4 and Figure 8). Notably, the old house and cat samples achieved VBench total scores of 83.26 and 83.61, respectively, further suggesting that our method is not strongly dependent on the choice of training sample.

Q2. Single-LoRA Baseline

We thank the reviewer for raising this important question. We agree that a fair comparison under a matched parameter budget is essential, and we have investigated this in Section 4.3 (Table 6). Specifically, S2–F2 represents our default Dual-LoRA configuration, while S4 and F4 denote single-LoRA baselines trained on the High-Noise and Low-noise denoising trajectory, respectively. To provide a more comprehensive comparison, we further introduce M4, which trains a single LoRA along a uniform 4-step ODE denoising trajectory.

Under a matched trainable parameter budget, S2–F2 achieves the best performance across all evaluation metrics. In contrast, M4 yields the lowest performance across all metrics. These results further prove that forcing a single monolithic network to capture two conflicting learning regimes leads to severe representation interference.

ModelTotal↑Quality↑Semantic↑Dynamic↑Vision↑Instruct↑
Dual-LoRA
S2–F2 (default)84.3385.0181.64786.92169
Single-LoRA
S483.4284.8077.90552.45043
F483.1584.2078.95721.31031
M481.0282.3175.84430.98225

Q3. Parameter Sharing

We explicitly evaluate parameter sharing under the same distillation objectives. Specifically, we keep Slow-LoRA training unchanged, while initializing Fast-LoRA from the Slow-LoRA weights and further training it with the same Fast-stage distillation objective.

The results show that parameter sharing leads to substantially worse performance across all metrics, as summarized below:

ModelTotal↑Quality↑Semantic↑Dynamic↑Vision↑Instruct↑
Seperate (default)84.3385.0181.64786.92169
Shared82.3183.2378.62573.53047

These suggest that different noise stages exhibit conflicting learning requirements. Sharing trainable parameters introduces interference between these learning regimes.

We would also appreciate it if the reviewer could further clarify the intended parameter-sharing setup, so that we can better address this question and continue the discussion.

W1. Expert Specialization

Although both approaches employ experts operating at different noise regimes, their motivations and training objectives differ fundamentally. Wan2.2 focuses on expert specialization for different generation tasks, whereas our goal is non-uniform distillation: applying minimal speedups to semantically critical steps (Slow) and maximal speedups to redundant steps (Fast), thereby optimizing distillation where it matters most.

We have updated the PDF (changes in blue). For more details, please refer to the “Phase-aware Denoising” section on page 10 of the main manuscript.

W2. Distillation Cost

To provide a more comprehensive assessment, we report the training cost and extra storage of all three stages separately:

ModelAR DiffusionODEDMDTotal
Training CostExtra StorageTraining CostExtra StorageTraining CostExtra StorageTraining CostExtra Storage
Causal Forcing2300011600190029000168001900
Glance Forcing23000112900052011

Note: Training Cost is measured in GPU-hours, and Extra Storage is measured in GB.

Notably, since DMD distills directly from online teacher predictions using 25k text prompts, it eliminates the need to pre-generate or store a large-scale text-video pair dataset, resulting in zero extra storage."

Consequently, the overall training cost of the entire pipeline is reduced by approximately 70%, while the extra storage requirement drops by about 99%.

We will clarify this distinction and update the efficiency discussion in the revised manuscript.

W3. Value Inconsistency

Thank you for catching this inconsistency. This is indeed a typo in Section 4.2, and Table 1 presents the correct results. We will correct this discrepancy and carefully proofread the entire manuscript in the revision.

We sincerely thank the reviewer for their valuable feedback. We have carefully addressed each point raised in the review and provided detailed responses in rebuttal. Should the reviewer have any further questions, we would be more than happy to provide additional clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.