- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper notes the difficulty that world models face in generating videos that adhere to complex physical laws. To address this, it proposes injecting temporally grounded physical register slots into DiTs. By learning within a representation space aligned with JEPA, the model is endowed with the ability to generate scene evolutions that conform to physical laws.
Critical strengths
- The paper presents a clear methodological design and extensive experiments, demonstrating performance improvements across multiple benchmarks and base models.
- The visualized video generation results are clear, showing that state prediction based on temporally grounded representations enables DiT-based video generation to better adhere to the laws of physical evolution.
Critical weaknesses
- Although the method proposed in the paper introduces only a small number of new parameters, it leads to a significant increase in token sequence length; however, the paper fails to provide clear comparative experiments conducted under similar computational cost constraints.
- Furthermore, when compared to "register" methods, the proposed approach incurs a substantially higher computational cost. While the authors claim that aligning with JEPA representations enhances adherence to physical laws, they offer no theoretical explanation for why this leads to performance improvements. This argument is further weakened by the disparity in computational costs. The paper lacks direct empirical evidence regarding how the model accesses or retrieves physical information, making it impossible to determine whether the observed improvement in adherence to physical laws stems from the method's design or simply from an increase in computational expenditure.
- Additionally, the selection strategy which chooses candidates that deviate the most within the JEPA representation space lacks a solid theoretical grounding in physical principles. Moreover, as shown in Table 12, the performance of the PRS selection strategy is not significantly better than that of the WMReward-based selection.
Questions for authors
- Could you provide a qualitative and quantitative comparison of the physical registers against the baseline, while keeping the computational cost constant?
- Could you offer a more detailed explanation as to why selecting candidates that deviate the most from the initial JEPA distribution leads to a better ability to adhere to physical laws?
- Could you provide additional zero-shot examples to demonstrate the method's generalizability, and include further experiments to show that video generation performance does not degrade in out-of-distribution scenarios?
4 comments
Response to Reviewer BjLo#2
W2-3: Direct evidence (new, Table R2). To address the lack of "direct empirical evidence regarding how the model accesses or retrieves physical information," we changed only the forecast that conditions the registry of the trained models at inference.
-
Static context: copies the observed context feature into all seven future slots.
-
Shuffled forecast: randomly reorders the seven future slots so that none stays at its own time step (one fixed order per scene and seed, shared across backbones).
-
Takeaway: both score below the predicted forecast in all three settings, which suggests that generation uses the predicted dynamics and their temporal order.
Table R2. Forecast-content interventions (Physics-IQ Verified). Mean ± std over two seeds (Δ vs. Physical Registry).
| Registry input | Nano 16B I2V | Nano 16B V2V | WAN I2V-A14B I2V |
|---|---|---|---|
| Physical Registry (predicted forecast) | 36.01 ± 0.95 | 45.12 ± 0.94 | 38.44 ± 1.06 |
| Static context | 35.22 ± 0.52 (−0.79) | 44.67 ± 0.09 (−0.45) | 36.73 ± 1.31 (−1.71) |
| Shuffled forecast | 35.48 ± 0.38 (−0.53) | 44.81 ± 0.04 (−0.31) | 37.52 ± 1.28 (−0.92) |
| Zero mask (Table 4a) | 16.78 ± 0.01 (−19.23) | 24.15 ± 0.32 (−20.97) | 35.88 ± 0.22 (−2.56) |
The following analyses in the paper point the same way:
- Forecast ablation (Table 4a): zeroing or perturbing the forecast lowers the Nano score.
- Motion transfer (Appendix G): another video's forecast changes the ball's upward return from 80.1 to 3.6 pixels (Table 18).
- Registry activation (Figure 5): attention and activation concentrate on the region undergoing physical change.
To make the role of shared attention visible in the method section, we will state how the DiT receives the forecast and updates the forecast toward the realized future.
- Revision: Section 3.4 (L184–L186) — add one sentence stating that the DiT receives the forecast together with the video tokens and, through the alignment, updates the forecast to match the features of the actual future.
W3 & Q2 — Why select the largest departure
PRS selects the candidate whose generation refined the shared forecast the most, read from the model's own registry during denoising (Q2).
- One shared forecast: all candidates start from the same forecast, a single estimate of several possible futures [1, 6].
- Candidate-specific update: trained toward the realized future (Eq. 11), the registry moves from the shared forecast toward each candidate's own video during denoising (readout cosine 0.764 → 0.974; Figure 12a, L1559–L1561).
- Largest departure: departure (Eq. 12) measures how far each candidate's registry has moved from the shared forecast toward its own video, and PRS selects the candidate updated the most. Selecting the largest departure is +8.3 above the reversed ranking and +3.6 above random selection on average over the six settings (Figure 12c; L1569–L1572).
- W3: Position of PRS. PRS offers two properties.
- The registry is a readable internal state: selecting from the registry without an external reward model shows that the registry can serve as a readout of the generation in progress.
- Early selection: PRS selects before the videos are fully generated and re-encoded for scoring, at 2.84–4.07× lower cost than best-of-N with WMReward (Table 17; Appendix F.4).
We agree that the submitted wording of Section 4.4 compares PRS on Physical Registry with WMReward on the backbone, and that on the same model the two are comparable in quality (Table 12 in the submission, now Table 16; L1593–L1596). In the revision, we will state PRS as comparable to WMReward at lower cost, and add the three steps above to the method section. Thank you for suggesting the same-model comparison.
- Revision: Section 3.6 (L222–L229) — add the three steps above as the rationale for selecting the largest departure.
- Revision: Section 4.4 (L371–L376) — replace "PRS on Physical Registry exceeds WMReward on the corresponding backbone" with the same-model comparison: PRS is comparable to WMReward in quality and costs 2.84–4.07× less.
Response to Reviewer BjLo#3
Q3 — Zero-shot examples and out-of-distribution performance
To address the request to show that "video generation performance does not degrade in out-of-distribution scenarios," we evaluate on the WorldModelBench domains that never appear in our Enlarge fine-tuning data (driving, industrial, gaming, and animation; 200 prompts, Table 5), where the total score rises on all four backbones (Q3; Table R3).
- Common sense: maintained or slightly higher on all four backbones in these domains, suggesting that generation quality is preserved outside the fine-tuning domains.
Table R3. WorldModelBench on the four domains absent from Enlarge. Each cell shows the Physical Registry (PR) score as mean ± std, with the difference from the backbone in parentheses. Bold marks the higher of the two: the PR score when PR is higher, and the difference when the backbone is higher.
| Model | Instruction (0–3) | Physical (0–5) | Common sense (0–2) | Total (0–10) | Δ Total, 95% CI |
|---|---|---|---|---|---|
| Cosmos 3 Nano 16B + PR | 2.00 ± 0.05 (−0.11) | 4.86 ± 0.03 (+0.40) | 1.83 ± 0.03 (+0.54) | 8.69 ± 0.06 (+0.83) | [+0.68, +0.98] |
| Cosmos v2.5 2B + PR | 2.23 ± 0.11 (−0.01) | 4.83 ± 0.06 (+0.08) | 1.78 ± 0.06 (+0.07) | 8.84 ± 0.19 (+0.15) | [+0.02, +0.31] |
| WAN I2V-A14B + PR | 2.50 ± 0.07 (+0.07) | 4.83 ± 0.05 (+0.02) | 1.86 ± 0.07 (+0.04) | 9.19 ± 0.14 (+0.12) | [+0.01, +0.26] |
| WAN TI2V-5B + PR | 2.38 ± 0.09 (+0.13) | 4.79 ± 0.09 (+0.02) | 1.76 ± 0.10 (+0.04) | 8.93 ± 0.16 (+0.19) | [+0.09, +0.30] |
- Zero-shot examples (Q3): additional Physical Registry generations on WorldModelBench industrial, video-game, and nature prompts are in Figures 23–25 (Appendix I.2).
We sincerely appreciate the helpful review and hope that these results and revisions address your concerns. If you have any further questions or would like additional discussion, please feel free to leave a comment at any time.
References
[1] Assran et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv:2506.09985, 2025.
[2] Mur-Labadia et al. V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning. arXiv:2603.14482, 2026.
[3] Yu et al. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. ICLR 2025.
[4] Wang et al. Learning Diffusion Models with Flexible Representation Guidance. NeurIPS 2025.
[5] Darcet et al. Vision Transformers Need Registers. ICLR 2024.
[6] Denton and Fergus. Stochastic Video Generation with a Learned Prior. ICML 2018.
I thank the authors for their efforts during the rebuttal period. My concerns have been largely resolved, and I will maintain my original rating.
Response to Reviewer BjLo#1
We thank Reviewer BjLo for noting the clear methodological design and the extensive experiments across multiple benchmarks and base models. We also appreciate the detailed feedback on computational cost (W1, W2-1 & Q1), the theoretical basis and evidence of forecast use (W2-2, W2-3), the departure rule (W3 & Q2), and zero-shot generalization (Q3).
W1, W2-1 & Q1 — Comparisons at the same computational cost
To address the concern about "comparative experiments conducted under similar computational cost constraints," we compare at the same inference cost, quantitatively (new, Table R1) and qualitatively (Figures 8 and 9), and at the same training cost.
Table R1. Matched inference compute (Physics-IQ Verified; two seeds). * Published score.
So that readers can weigh the added cost against the gain, we will report the matched-compute comparison.
W2-2 & W2-3 — Theoretical basis and direct evidence of forecast use
W2-2: Theoretical basis. We explain why aligning with JEPA representations can improve physical adherence.