- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes physical language to represent world-state transitions. To learn it, they use a physical language tokenizer. Then, they build PhiZero on top of the physical language. They use a reason-then-render paradigm, which first infer future world evolution in physical langauge space and then render the inferred transitions into videos. Experiment shows that Phizero achieves good performance on the listed benchmarks. Ablation studies show the proposed components contribute positively. Authors also show potential use of such model.
Critical strengths
-
The paper tackles a critical problem in video world models: physics. The paper proposes physical language and uses a discretized tokenizer to learn the tokens to represent it.
-
Experiment in Sec 4.4 shows that learned visual tokens have clear clusters under PCA, which supports the claim of physical language. A suggestion is that authors may consider use numeric numbers, e.g., classification result to further show the learned semantics of physical language.
-
Experiment shows PhiZero demonstates optimal performance over the demonstrated method on aggregated metrics (e.g., Total, Avg. Error). Ablation study shows the design may be useful for improving physics understanding.
Critical weaknesses
-
The paper compares PhiZero with previous work, stating that PhiZero explicitly represents state transitions as a compact discrete physical language. But specific reason for why this paradigm is better than "evolution in pixel spaces" is not clearly discussed.
-
Further, the paper lacks baselines, e.g., VideoWorld 2 and DisMo. The paper mentions these baselines in related works but does not have a comprehensive experimental comparison with them. While the paper discusses differences in scope and objectives, it does not explain why these methods are excluded from the empirical comparisons. The Wan2.2 baseline in Table 9 is valuable, but does not establish the benefit of the proposed representation over existing latent-dynamics approaches.
-
The ablation study in table 9 is not sufficient. It does not isolate the contribution of the proposed physical-language representation from that of the additional pretrained Qwen3-VL reasoner. Separately, table 4 includes several general-purpose VLMs but omits Qwen3-VL-4B, the backbone used to initialize the reasoner. It is helpful to include this experiment to understand where the improvement comes from.
Questions for authors
None
8 comments
Response to Reviewer V7Ca (Part 2/2)
[W2] Lack of Comparisons with Existing Latent-Dynamics Methods
We appreciate your suggestion to compare PhiZero with existing latent-dynamics methods, as it helps us more clearly establish the contributions of physical language. We construct two baseline models following the dLDM approach of VideoWorld 2 and the motion representation learning approach of DisMo, respectively, and conduct comparisons across three complementary tasks:
(1) Physical Future Prediction. We evaluate their physical generation performance on Physics-IQ Verified (IQ) and PhyGround (PG).
(2) Physical Motion Transfer. To evaluate the transferability of different representations, we assess cross-scene physical motion transfer quality using PSNR, SSIM, and LPIPS on PhysInOne data.
(3) Physical Consequence Probing. To investigate the physical information encoded in different representations, we construct two probing tasks based on PhysInOne data. For Outcome Classification, we train a linear classifier to distinguish three interaction outcomes: stationary, translation, and tipping. For Magnitude Classification, we evaluate whether representations can distinguish short and long displacements, using the median displacement in the training set as the threshold.
The results across all three tasks are summarized below:
| Method | Future Prediction | Motion Transfer | Consequence Probes |
|---|---|---|---|
| Metrics | IQ↑/PG↑ | PSNR↑/SSIM↑/LPIPS↓ | Outcome↑/Magnitude↑ |
| DisMo | 37.2/2.84 | 19.03/0.722/0.467 | 70.40%/68.80% |
| VideoWorld 2 | 38.4/2.90 | 18.42/0.713/0.489 | 72.80%/70.60% |
| PhiZero (Ours) | 41.2/2.97 | 20.89/0.781/0.436 | 85.70%/81.90% |
For physical future prediction, PhiZero demonstrates the effectiveness of physical language in modeling future physical evolution. Its superior motion transfer quality also indicates that physical language better preserves fine-grained state-transition information. The consequence probing results further demonstrate its ability to encode meaningful physical interactions. Together, these results demonstrate the advantages of physical language over existing latent-dynamics representations in physical future prediction, cross-scene motion transfer, and physical consequence understanding.
[W3] Insufficient Disentanglement of the Contributions of Physical Language and the Pretrained VLM
We thank you for highlighting the need to disentangle the contributions of physical language and the pretrained VLM, which motivates us to further investigate the sources of PhiZero's improvements. We conducted controlled experiments examining two factors: (i) Predictor Initialization: a randomly initialized 4B model versus a pretrained Qwen3-VL-4B; and (ii) Prediction Strategy: direct video prediction versus explicit physical-language prediction. For direct video prediction, learnable queries extract hidden states from the 4B model to condition the diffusion decoder for video generation. Furthermore, to investigate whether PhiZero's improvements arise from extending the VLM's capabilities into the learned physical-language space, we introduce a natural-language enhancement baseline. Specifically, we train Qwen to generate physics-enriched natural-language descriptions, which are then used as enhanced textual conditions for future video generation. The results are summarized below:
| Predictor Initialization | Prediction Strategy | Physics-IQ↑ | PhyGround↑ |
|---|---|---|---|
| Random Init. | Direct Video | 30.4 | 2.72 |
| Random Init. | Physical Language | 38.1 | 2.92 |
| Qwen3-VL-4B | Direct Video | 33.6 | 2.84 |
| Qwen3-VL-4B | Natural-Language Enhancement | 36.5 | 2.89 |
| Qwen3-VL-4B | Physical Language (Ours) | 41.2 | 2.97 |
As shown in the table, physical-language prediction substantially improves physical generation performance regardless of whether pretrained Qwen3-VL weights are used, demonstrating that the advantages of our paradigm do not solely depend on the pretrained VLM. Moreover, while natural-language enhancement improves generation performance over direct video prediction, it still falls short of PhiZero. This suggests that physical language provides the VLM with a prediction space more closely aligned with fine-grained physical world evolution than natural language.
Finally, following your suggestion, we further evaluated the physical plausibility understanding of the original Qwen3-VL-4B on IntPhys2.
| Model | Easy↑ | Medium↑ | Hard↑ | Overall↑ |
|---|---|---|---|---|
| Qwen2.5-VL | 50.96 | 53.25 | 51.49 | 52.27 |
| Qwen3-VL-4B | 55.77 | 56.5 | 52.08 | 54.64 |
| PhiZero (Ours) | 60.98 | 60.5 | 52.38 | 56.34 |
As shown in the table above, PhiZero outperforms the original Qwen3-VL-4B across all IntPhys2 subsets, providing further evidence that modeling state transitions through physical language benefits physical understanding.
Thank you for the additional experiments. The experiments in W1, new comparisons with VideoWorld 2 and DisMo, and the ablation on Qwen initialization address several of my concerns. For the experiments above, I have some additional questions:
For W2: Could the authors clarify how the VideoWorld 2 and DisMo baselines were adapted and whether their training data, backbone initialization, and compute budgets were comparable to PhiZero’s?
For W3: On IntPhys2, PhiZero improves over the original Qwen3-VL-4B, but it also receives additional video training. Do the authors have a comparison in which the same backbone (Qwen3-VL) is trained on the same videos? This could help distinguish if the improvement is from the addtional training data or the proposed physical language
Response to Reviewer V7Ca (Part 1/2)
We sincerely appreciate your recognition of our work as "tackles a critical problem in video world models". In the revised PDF, changes made in response to your feedback appear in green. Below, we address your Suggestions(S) and weaknesses (W) in detail.
[S] Quantitative Evaluation of the Semantic Properties of Learned Physical Language
Thank you for suggesting a quantitative evaluation of physical-language semantics, which provides a valuable complement to our existing qualitative analysis. We conducted additional linear probing experiments, training linear classifiers on extracted physical-language representations to evaluate semantic representation at two levels: (1) Action Category Recognition: We classify four driving behaviors (left turn, right turn, straight, and stationary), and four manipulation patterns (gripper moving downward while closing, upward while closing, downward while opening, and sweeping). (2) Motion Magnitude Recognition: We further distinguish the magnitude of lateral displacement for turns and travel distance for gripper. Labels are obtained through a median split of displacement values into short- and long-range movements.
As shown in the table, physical language demonstrates its ability to encode meaningful state-transition semantics and preserve fine-grained motion magnitude information.
[W1] Insufficient Explanation of Why the Physical-Language is Preferable to Pixel-Space Modeling of World Evolution
We appreciate your insightful question about the advantages of physical-language modeling over direct pixel-space prediction, as it motivates us to further validate our design through further experiments. Our core motivation is that physical world modeling requires accurately predicting how world states evolve over time. Conventional pixel-space methods typically model physical dynamics implicitly within the visual generation process. In contrast, PhiZero explicitly decouples state-transition reasoning from visual rendering, providing a well-defined and predictable intermediate modeling target for physical world evolution.
To further validate this motivation, we conducted two experiments to investigate the contributions of intermediate supervision and explicit intermediate representation prediction. First, we constructed a Qwen-based direct video prediction baseline, where learnable queries extract hidden states from Qwen to condition the diffusion decoder for video generation. Building on this baseline, we introduced two types of supervision on the intermediate query representations: (1) SigLIP features extracted from future videos, and (2) the learned physical language. We further compared two approaches that explicitly predict intermediate representations: (1) SigLIP Feature Prediction, where Qwen predicts SigLIP features to condition the diffusion decoder; and (2) Physical Language Prediction (Ours), where Qwen predicts physical-language tokens to condition the diffusion decoder.
As shown in the table, introducing either SigLIP or physical-language intermediate supervision improves physical generation performance over direct video prediction, with physical-language supervision yielding substantially larger gains. This demonstrates that providing an explicit intermediate learning target benefits physical world modeling, while the learned physical language serves as a more effective target. Furthermore, explicitly predicting physical language yields additional improvements across all metrics, highlighting the effectiveness of treating world-state transitions as an independent prediction target and explicitly reasoning over them in physical-language space.