- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this work, the authors turn a pretrained bidirectional video diffusion model into a causal world model that follows camera controls and prompts that change mid-rollout. The authors argue that standard teacher forcing (TF) degrades visual quality because its attention pattern and noise setup differ from pretraining. Mix Forcing appends a second noisy copy of each clip to the TF sequence and trains it with the original bidirectional objective, using an attention mask so that both losses come from one forward pass. In cross-attention, each chunk sees an environment prompt plus its own local prompt, so a prompt update only affects later chunks and the KV cache is reused. The full system (Wan2.1-I2V-14B, then consistency distillation and DMD to four steps) reports the best visual quality and event following on an internal benchmark and on SANA-WM-Bench. The ablations indicate that Mix Forcing is a better starting point for distillation than TF.
Critical strengths
- This work presents a simple fix for a real problem in recent causal pre-training pipelines, with no inference cost.
- Table 5 is the most useful result. With distillation held fixed, starting from Mix Forcing instead of TF gives a much better four-step model. The pre-training objective therefore still matters after distillation.
- The chunk-wise prompt design is practical, and the on-time metric captures responsiveness that single-prompt benchmarks ignore
Critical weaknesses
- The main comparisons may not attribute directly the gains to Mix Forcing. The final model uses LingBot-World-Base as its DMD teacher, whereas InBench uses the method's own prompt format.
- Novelty is modest. Training one model on several objectives by switching attention masks is a known recipe (e.g., UniLM), and LongLive already supports prompt switching.
- Several details are missing: the long-horizon inference setup, how the per-chunk captions were produced, which VLM judges EA/OT, how baselines were prompted, and inference latency.
Questions for authors
- Can you separate attention pattern from noise level, for example with a causal variant trained at a shared noise level?
- Does the TF-only arm in Table 5 use chunk-wise or global prompts, and how does the 50-step TF-only model score?
- How is LingBot-World-Base conditioned on the environment and local prompts (Eq. 9), and how much of the final quality comes from it?
- What cache window and positional scheme do you use for one-hour rollouts, and how does "without quality degradation" square with Table 2?
- How were baselines prompted on InBench? Which VLM judges EA/OT, and what is the inference latency of the four-step model?
4 comments
Response (Part 2/3)
3.1 Contribution of Mix Forcing and LingBot-World-Base (W1, Q3).
In Eq. (9), LingBot-World-Base serves as the bidirectional DMD teacher. Both the real and fake score models are conditioned on the camera poses and the global prompt for the same video. Our autoregressive generator receives the camera poses, the shared environment prompt, and the corresponding chunk-wise local prompts. We have made these conditioning sets explicit in the revised equation and accompanying text.
Fully isolating the teacher's contribution would require post-training with a different teacher. Instead, the following two comparisons hold the teacher fixed.
In our system-level comparisons, LingBot-World-Fast also uses LingBot-World-Base as its DMD teacher. Our model achieves higher overall Quality on InBench (75.1 vs. 72.4), both splits of SANA-WM-Bench (74.8 vs. 70.1 on Simple and 74.7 vs. 69.9 on Hard), and WBench (78.5 vs. 75.3). These comparisons therefore include a baseline that also benefits from LingBot-World-Base supervision.
To evaluate the contribution of causal pre-training, Table 5 compares the four-step TF-only and Mix Forcing models under the same chunk-wise prompts, CD/DMD post-training pipeline, and LingBot-World-Base DMD teacher. Under this shared setup, Mix Forcing improves Quality from 64.1 to 70.9, EA from 33.7 to 55.1, and [email protected] from 19.8 to 64.3, while reducing CMC error from 1.331 to 0.593. These results support Mix Forcing as a stronger causal initialization whose benefits persist through the same post-training process.
Regarding the InBench prompt format, we evaluate each baseline under two prompting settings and report its best result in Table 1 (see Response 5.1).
4.1 One-hour rollout settings and quality claims (W3, Q4).
Each inference chunk contains four latent frames. During one-hour generation, we use a fixed-size KV cache window: the first latent chunk is retained as an attention sink together with the three most recent latent chunks. To stay consistent with the 561-frame clips used in training, we additionally reset the KV cache every 561 generated frames, keeping only the attention-sink chunk and clearing all other entries. We extend the precomputed RoPE positional range to support long-horizon rollout.
The term “quality” in this claim refers to overall Quality, which aggregates AQ, IQ, TFlk, DD, and HPSv3, whereas IQ in the original Table 2 is only one component of this aggregate. The temporal IQ results are now reported in supplementary Table S3. To clarify the temporal behavior of overall Quality, we further report the overall Quality of all models over the first and last ten seconds, denoted by Quality₀–₁₀ and Quality₅₀–₆₀, respectively, in the table below.
- Simple trajectory splits
| Method | Quality₀–₁₀ ↑ | Quality₅₀–₆₀ ↑ | SQ ↑ |
|---|---|---|---|
| HY-WorldPlay | 60.84 | 60.35 | 60.59 |
| JoyAI-Echo-WM | 70.23 | 64.45 | 67.22 |
| Matrix-Game 3.0 | 76.55 | 74.81 | 75.67 |
| SANA-WM-Streaming | 74.29 | 66.27 | 70.05 |
| LingBot-World-Fast | 74.63 | 70.11 | 72.30 |
| Mix Forcing (Ours) | 78.37 | 75.39 | 76.85 |
- Hard trajectory splits
| Method | Quality₀–₁₀ ↑ | Quality₅₀–₆₀ ↑ | SQ ↑ |
|---|---|---|---|
| HY-WorldPlay | 62.35 | 65.48 | 63.88 |
| JoyAI-Echo-WM | 72.94 | 66.38 | 69.51 |
| Matrix-Game 3.0 | 76.34 | 75.01 | 75.67 |
| SANA-WM-Streaming | 74.89 | 67.17 | 70.82 |
| LingBot-World-Fast | 75.81 | 72.89 | 74.32 |
| Mix Forcing (Ours) | 78.36 | 75.11 | 76.70 |
We additionally report SQ, defined as the harmonic mean of the split-level mean Quality scores in the two windows, where SQ = 2 × Quality₀–₁₀ × Quality₅₀–₆₀ / (Quality₀–₁₀ + Quality₅₀–₆₀). Our model achieves the highest Quality in both windows on both splits, while its Quality declines by about 3 points from the first to the last ten seconds (78.37 → 75.39 on Simple and 78.36 → 75.11 on Hard). Over one-hour rollouts, mean Quality is 71.0 in the first five-minute interval and 69.3 in the last (Figure S12). We agree that "without quality degradation" overstates these results and have revised it to "without significant quality degradation" in the abstract. We have added the Quality₀–₁₀ and Quality₅₀–₆₀ scores and SQ to the revised Table 2, with the SQ definition in Appendix B.1.
Response (Part 3/3)
5.1 Baseline prompting, caption construction, and EA/OT evaluation (W1, W3, Q5).
-
Baseline prompting. All variants in Table 3 use global prompts to isolate the effects of the self-attention design, while both TF-only and Mix Forcing in Table 5 use chunk-wise prompts. For the baseline world models on InBench, we test two prompting settings: (i) using a global prompt throughout generation; and (ii) starting with the environment prompt and concatenating the environment prompt with the corresponding local prompt when each event occurs. We report the best results obtained for each world model across these two settings in Table 1. All methods use the same event definitions and evaluation videos. Table 1 evaluates the complete systems, while Table 4 provides the controlled comparison of global and chunk-wise conditioning within our model. We have clarified the baseline prompting protocol in Appendix B.2.
-
Caption construction. We use Qwen3-VL to generate a local caption for each one-second segment of a 16 FPS video. Each segment contains 16 video frames, corresponding to four latent frames and matching our inference chunk size. The resulting caption serves as the local prompt for that chunk. We have added this to Appendix A.1.
-
EA/OT evaluation. We use GPT-6 Astra as the vision-language model judge for EA and OT. We have added this to Appendix B.2.
5.2 Four-step inference latency (W3, Q5).
The four-step model has an end-to-end latency of 0.559 s per chunk at 832×480 resolution, measured with batch size 1 and BF16 precision on 8 H800 GPUs. These measurements exclude the first four chunks of the one-hour rollout. Each inference step generates one chunk of four latent frames, corresponding to 16 video frames (1 s of video at 16 FPS). The end-to-end latency is therefore 0.035 s per frame, or about 28.6 FPS. We use Ulysses sequence parallelism with a degree of 8 for the four-step generator and patch-parallel VAE decoding [1].
| Latency | 4-step generator | VAE decoding | End-to-end |
|---|---|---|---|
| s/chunk | 0.525 | 0.092 | 0.559 |
| s/frame | 0.033 | 0.006 | 0.035 |
End-to-end latency measures the time from starting the generation of each latent chunk to receiving its decoded video frames. We have added the measurement settings and latency breakdown to supplementary Table S6 and the accompanying text.
[1] Fang et al. xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism. arXiv:2411.01738, 2024.
6.1 Distinction from UniLM and LongLive (W2).
- UniLM. UniLM unifies natural language understanding and generation in a shared model, where each attention mask corresponds to a capability used at inference: bidirectional masks for understanding and unidirectional masks for generation. In Mix Forcing, inference is purely causal. The BID objective is never used at inference; instead, it anchors the model to its pretraining configuration, i.e., bidirectional attention with a shared noise level, so that pretrained visual priors are preserved while TF learns causal next-chunk prediction. Moreover, in the diffusion setting, each attention mask must be paired with a matching noise configuration, which has no counterpart in UniLM. Our ablations show that this pairing, rather than mask switching itself, drives the gains: adding a DF objective lowers Quality from 68.9 to 65.6, whereas adding BID raises it to 70.8 (Table 3); with identical attention masks, changing only the TF noise configuration lowers Quality from 70.8 to 68.3 (Response 1.1).
- LongLive. LongLive conditions the retained history on the current prompt. When the prompt switches, it recomputes the KV cache of the retained frames under the new prompt, so past frames are re-encoded with a prompt that did not generate them, and each switch incurs an extra recomputation. Mix Forcing instead introduces chunk-wise conditioning during causal pre-training: each chunk is conditioned on the shared environment prompt and its own local prompt, so cached chunks remain consistent with the prompts under which they were generated. A new local prompt affects only subsequent chunks, and the KV cache is reused without recomputation. Within our model, this design raises EA from 23.4 to 50.0 and [email protected] from 18.3 to 62.1 compared with global prompting (Table 4).
Dear Reviewer,
Thank you again for your time and thoughtful feedback. We have posted our responses to the points raised in your review. Due to the limited time remaining in the discussion period, we would sincerely appreciate it if you could review them and let us know whether they address your concerns. We welcome any further discussion if needed.
Best regards
Response (Part 1/3)
We sincerely appreciate your careful review and constructive comments. We address each concern below.
Following your suggestion, we train a causal variant at a shared noise level. Starting from TF+BID in Table 3 (one latent frame per chunk, global prompts), we keep all attention masks unchanged and share a single noise level across all TF chunks, so that the causal branch matches the noise configuration of pretraining but not its attention pattern. BID uses a shared noise level in both variants.
Sharing the noise level under causal attention lowers Quality from 70.8 to 68.3, below even TF-only (68.9 in Table 3), and more than doubles the Rot and CMC errors (0.449 → 0.981 and 0.647 → 1.383). Matching the pretraining noise level alone therefore does not reproduce the benefit of BID; this benefit relies on bidirectional attention.
This result is consistent with our motivation for independent noise sampling, following Diffusion Forcing [1]. Since TF predicts each noisy chunk from clean historical context, independent sampling lets different chunks of the same video provide supervision at different denoising timesteps within a single update, whereas a shared noise level reduces this within-video timestep diversity. We have added this ablation as Table S5 and clarified the noise-level configurations in the revised supplementary material.
[1] Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. Advances in Neural Information Processing Systems, 37:24081–24125, 2024.
In Table 5, the TF-only baseline uses the same chunk-wise prompting scheme and CD/DMD post-training pipeline as Mix Forcing, ensuring consistent prompting across the comparison. The 50-step TF-only model performance is as follows:
Before few-step post-training, Mix Forcing achieves Quality of 71.1 versus 66.8 for TF-only; after the same CD/DMD pipeline, the corresponding scores are 70.9 and 64.1. The benefit is already present before DMD and remains in the four-step models. We have added the 50-step TF-only results to Table 5 and clarified that all models in the table use chunk-wise prompt conditioning.