- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper diagnoses a seriality gap in bidirectional diffusion using hard-sphere and Turmite prediction in video and simulator-state representations. Longer event dependencies increase prediction difficulty, additional denoising steps do not close the measured gap, and task-aligned temporal factorization and greater backbone depth improve accuracy. For fully observed deterministic prediction, Proposition 4.1 shows that a TC0 exact-score network permits state prediction through one network evaluation, under the stated finite-precision and recovery assumptions.
Critical strengths
-
The empirical gap survives substantial controls. At 49 frames and width 1536, hard-sphere rollout-5 error falls from 0.883 for bidirectional generation to 0.607 for AR, with a paired gap interval of [0.262, 0.290] (Tables 3–4). The five-ball separate-lane control nearly eliminates this local-error gap (p.36). Direct-state, extractor-sensitivity, and three-seed experiments further support robustness (Appendices B.4.2, C.1, F.3).
-
Dependency and shared-weight tests strengthen the finding. At fixed event count, two-agent Turmites show decreasing bidirectional accuracy with dependency depth at all four tested counts; all eight within-seed bootstrap intervals are below zero (Table 17). A shared checkpoint trained on all four factorizations preserves AR > Block-5 > Block-15 > bidirectional generation (Appendix F.1).
Critical weaknesses
- The Turmite state comparison confounds temporal factorization with projection frequency. It supports Observation 3 on serial generation (§3.1, ll.310–317), but AR and blockwise generation apply argmax and feed back one-hot states after every block; bidirectional generation projects only after denoising the full future (Appendix C.2, ll.1127–1131). Smaller blocks also mean more frequent projection. The shared-checkpoint control retains this difference, leaving its contribution to the AR benefit unresolved (Appendix F.1).
Questions for authors
- How much of the Turmite AR/blockwise advantage comes from repeated discrete projection? Compare hard and soft prefix feedback under matched training to separate projection from temporal factorization.
- How does the exact-score result relate to transformer CoT expressivity results? Which computational limitation is shared with CoT, and what is specific to video diffusion? What does the theorem imply for intermediate visual/latent computation before the first output frame?
- Can a fixed visual model solve longer, unseen dependency chains by arbitrarily increasing internal steps, without retraining or lengthening the output video? Specify candidate intermediate visual states, per-step updates, and supervision. Can a purely visual CoT be learned from video alone, or does it require explicit symbolic states or supervision?
3 comments
Uk4x rebuttal (2/2)
Q2
How does the exact-score result relate to transformer CoT expressivity results? Which computational limitation is shared with CoT, and what is specific to video diffusion?
Our paper does not study explicit scratchpad CoT before the first output frame, so we interpret “CoT” here as predicting intermediate frames autoregressively, as in our autoregressive models.
The connection is that our exact-score result shows that serial computation must come from the backbone rather than additional diffusion steps, under our assumptions of determinism, full observability, and an exact score.
Intuitively, autoregressive diffusion can distribute computation across successive frames for state prediction problems. This is analogous to how transformer CoT can go beyond TC⁰ through sufficiently long intermediate generation under the assumptions of Merrill and Sabharwal (2024). Thus, autoregressive generation can help bridge the Seriality Gap observed with bidirectional diffusion, even though each fixed-depth backbone evaluation remains in TC⁰ under the log-precision assumptions.
What does the theorem imply for intermediate visual/latent computation before the first output frame?
Under our assumptions of determinism, full observability, and an exact score, a single score-network evaluation already contains enough information to recover the complete future. Thus, the prediction burden rests on the backbone, not denoising steps.
Autoregressive frame generation provides serial computation across successive frames, but later frames cannot supply computation needed for earlier frames. Therefore, if producing the first frame already requires substantial computation, the model needs sufficient backbone capacity or additional computation before that frame, such as recurrence, looping, or scratchpad steps. This is why latent-frame autoregression is not a universal remedy (Lines 123–124).
Q3
Can a fixed visual model solve longer, unseen dependency chains by arbitrarily increasing internal steps, without retraining or lengthening the output video? Specify candidate intermediate visual states, per-step updates, and supervision. Can a purely visual CoT be learned from video alone, or does it require explicit symbolic states or supervision?
This question goes beyond the scope of our current paper: we study the Seriality Gap and interventions that reduce it, rather than develop a model that resolves arbitrarily long, unseen dependency chains. Nevertheless, we think this is an interesting direction.
For the first part, a natural candidate is a looped backbone that repeatedly updates a hidden latent state using shared weights. Additional loops would provide more computation without requiring more output frames. Looped language models provide evidence that this approach can support reasoning (Saunshi et al., 2025), and looping has also been explored for image and video generation (Goyal et al., 2026). However, these results do not establish that arbitrarily increasing internal steps will resolve longer, unseen dependency chains, especially in videos.
For the second part, we interpret a purely visual CoT as an internal visual scratchpad generated before predicting each output frame, without adding frames to the output video. The training videos would not directly supervise these scratchpad states; instead, the model could learn them end-to-end through the video-prediction objective. We are optimistic that useful intermediate computation can be learned this way. However, whether the learned scratchpad remains visually meaningful, rather than becoming an opaque latent representation, is an interesting empirical question beyond our current study.
The authors' reply address my major concerns. Thus, I maintain the positive score.
Uk4x rebuttal (1/2)
We thank the reviewer for highlighting the strength of our experimental controls. We also welcome their questions connecting our findings to CoT expressivity and intermediate visual computation.
W1 / Q1
We thank the reviewer for pointing out this subtlety. Below we show that repeated discrete projection is unnecessary for the observed serialization advantage.
To isolate the contribution of discrete projection, we re-evaluate the three shared checkpoints from Appendix F.1 with continuous predictions fed back between blocks. We hold each checkpoint and all other evaluation settings fixed, changing only the prefix feedback.
Final-grid cell accuracy ↑; hard one-hot / continuous prefix feedback. Hard-feedback evaluations reproduce all 12 previously reported scores exactly.
Removing discrete projection preserves AR > Block-5 > Block-15 > Bidir at every horizon. AR and Block-5 final-grid accuracies are unchanged; Block-15 changes by at most 0.0008, with paired 95% bootstrap confidence intervals over held-out examples including zero at all three horizons.