acceptodds
Under review as a conference paper at ICLR 2027

PhiZero: A World Model Built Around Physical Language

17 views

Abstract

We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and cross-embodiment and sim-to-real transfer.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
world model, video generation, self-supervised learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_V7CaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes physical language to represent world-state transitions. To learn it, they use a physical language tokenizer. Then, they build PhiZero on top of the physical language. They use a reason-then-render paradigm, which first infer future world evolution in physical langauge space and then render the inferred transitions into videos. Experiment shows that Phizero achieves good performance on the listed benchmarks. Ablation studies show the proposed components contribute positively. Authors also show potential use of such model.

Critical strengths
  • The paper tackles a critical problem in video world models: physics. The paper proposes physical language and uses a discretized tokenizer to learn the tokens to represent it.

  • Experiment in Sec 4.4 shows that learned visual tokens have clear clusters under PCA, which supports the claim of physical language. A suggestion is that authors may consider use numeric numbers, e.g., classification result to further show the learned semantics of physical language.

  • Experiment shows PhiZero demonstates optimal performance over the demonstrated method on aggregated metrics (e.g., Total, Avg. Error). Ablation study shows the design may be useful for improving physics understanding.

Critical weaknesses
  • The paper compares PhiZero with previous work, stating that PhiZero explicitly represents state transitions as a compact discrete physical language. But specific reason for why this paradigm is better than "evolution in pixel spaces" is not clearly discussed.

  • Further, the paper lacks baselines, e.g., VideoWorld 2 and DisMo. The paper mentions these baselines in related works but does not have a comprehensive experimental comparison with them. While the paper discusses differences in scope and objectives, it does not explain why these methods are excluded from the empirical comparisons. The Wan2.2 baseline in Table 9 is valuable, but does not establish the benefit of the proposed representation over existing latent-dynamics approaches.

  • The ablation study in table 9 is not sufficient. It does not isolate the contribution of the proposed physical-language representation from that of the additional pretrained Qwen3-VL reasoner. Separately, table 4 includes several general-purpose VLMs but omits Qwen3-VL-4B, the backbone used to initialize the reasoner. It is helpful to include this experiment to understand where the improvement comes from.

Questions for authors

None

8 comments
AuthorsOpenReview ↗

Response to Reviewer V7Ca (Part 1/2)

We sincerely appreciate your recognition of our work as "tackles a critical problem in video world models". In the revised PDF, changes made in response to your feedback appear in green. Below, we address your Suggestions(S) and weaknesses (W) in detail.

[S] Quantitative Evaluation of the Semantic Properties of Learned Physical Language

Thank you for suggesting a quantitative evaluation of physical-language semantics, which provides a valuable complement to our existing qualitative analysis. We conducted additional linear probing experiments, training linear classifiers on extracted physical-language representations to evaluate semantic representation at two levels: (1) Action Category Recognition: We classify four driving behaviors (left turn, right turn, straight, and stationary), and four manipulation patterns (gripper moving downward while closing, upward while closing, downward while opening, and sweeping). (2) Motion Magnitude Recognition: We further distinguish the magnitude of lateral displacement for turns and travel distance for gripper. Labels are obtained through a median split of displacement values into short- and long-range movements.

RepresentationDriving Action Acc.↑Manipulation Action Acc.↑Turning Magnitude Acc.↑Gripper Motion Magnitude Acc.↑
Random Chance25.00%25.00%50.00%50.00%
Physical Language (Ours)92.17%90.42%88.00%81.50%

As shown in the table, physical language demonstrates its ability to encode meaningful state-transition semantics and preserve fine-grained motion magnitude information.

[W1] Insufficient Explanation of Why the Physical-Language is Preferable to Pixel-Space Modeling of World Evolution

We appreciate your insightful question about the advantages of physical-language modeling over direct pixel-space prediction, as it motivates us to further validate our design through further experiments. Our core motivation is that physical world modeling requires accurately predicting how world states evolve over time. Conventional pixel-space methods typically model physical dynamics implicitly within the visual generation process. In contrast, PhiZero explicitly decouples state-transition reasoning from visual rendering, providing a well-defined and predictable intermediate modeling target for physical world evolution.

To further validate this motivation, we conducted two experiments to investigate the contributions of intermediate supervision and explicit intermediate representation prediction. First, we constructed a Qwen-based direct video prediction baseline, where learnable queries extract hidden states from Qwen to condition the diffusion decoder for video generation. Building on this baseline, we introduced two types of supervision on the intermediate query representations: (1) SigLIP features extracted from future videos, and (2) the learned physical language. We further compared two approaches that explicitly predict intermediate representations: (1) SigLIP Feature Prediction, where Qwen predicts SigLIP features to condition the diffusion decoder; and (2) Physical Language Prediction (Ours), where Qwen predicts physical-language tokens to condition the diffusion decoder.

MethodPhysics-IQ↑PhyGround↑
Wan2.2-5B Direct Fine-tuning30.32.81
Direct Video Prediction33.62.84
w/ SigLIP Intermediate Supervision34.52.88
w/ Physical Language Intermediate Supervision37.22.92
SigLIP Feature Prediction35.42.86
Physical Language Prediction (Ours)41.22.97

As shown in the table, introducing either SigLIP or physical-language intermediate supervision improves physical generation performance over direct video prediction, with physical-language supervision yielding substantially larger gains. This demonstrates that providing an explicit intermediate learning target benefits physical world modeling, while the learned physical language serves as a more effective target. Furthermore, explicitly predicting physical language yields additional improvements across all metrics, highlighting the effectiveness of treating world-state transitions as an independent prediction target and explicitly reasoning over them in physical-language space.

AuthorsOpenReview ↗

Response to Reviewer V7Ca (Part 2/2)

[W2] Lack of Comparisons with Existing Latent-Dynamics Methods

We appreciate your suggestion to compare PhiZero with existing latent-dynamics methods, as it helps us more clearly establish the contributions of physical language. We construct two baseline models following the dLDM approach of VideoWorld 2 and the motion representation learning approach of DisMo, respectively, and conduct comparisons across three complementary tasks:

(1) Physical Future Prediction. We evaluate their physical generation performance on Physics-IQ Verified (IQ) and PhyGround (PG).

(2) Physical Motion Transfer. To evaluate the transferability of different representations, we assess cross-scene physical motion transfer quality using PSNR, SSIM, and LPIPS on PhysInOne data.

(3) Physical Consequence Probing. To investigate the physical information encoded in different representations, we construct two probing tasks based on PhysInOne data. For Outcome Classification, we train a linear classifier to distinguish three interaction outcomes: stationary, translation, and tipping. For Magnitude Classification, we evaluate whether representations can distinguish short and long displacements, using the median displacement in the training set as the threshold.

The results across all three tasks are summarized below:

Method    Future Prediction        Motion Transfer        Consequence Probes    
MetricsIQ↑/PG↑PSNR↑/SSIM↑/LPIPS↓Outcome↑/Magnitude↑
DisMo37.2/2.8419.03/0.722/0.46770.40%/68.80%
VideoWorld 238.4/2.9018.42/0.713/0.48972.80%/70.60%
PhiZero (Ours)41.2/2.9720.89/0.781/0.43685.70%/81.90%

For physical future prediction, PhiZero demonstrates the effectiveness of physical language in modeling future physical evolution. Its superior motion transfer quality also indicates that physical language better preserves fine-grained state-transition information. The consequence probing results further demonstrate its ability to encode meaningful physical interactions. Together, these results demonstrate the advantages of physical language over existing latent-dynamics representations in physical future prediction, cross-scene motion transfer, and physical consequence understanding.

[W3] Insufficient Disentanglement of the Contributions of Physical Language and the Pretrained VLM

We thank you for highlighting the need to disentangle the contributions of physical language and the pretrained VLM, which motivates us to further investigate the sources of PhiZero's improvements. We conducted controlled experiments examining two factors: (i) Predictor Initialization: a randomly initialized 4B model versus a pretrained Qwen3-VL-4B; and (ii) Prediction Strategy: direct video prediction versus explicit physical-language prediction. For direct video prediction, learnable queries extract hidden states from the 4B model to condition the diffusion decoder for video generation. Furthermore, to investigate whether PhiZero's improvements arise from extending the VLM's capabilities into the learned physical-language space, we introduce a natural-language enhancement baseline. Specifically, we train Qwen to generate physics-enriched natural-language descriptions, which are then used as enhanced textual conditions for future video generation. The results are summarized below:

Predictor InitializationPrediction StrategyPhysics-IQ↑PhyGround↑
Random Init.Direct Video30.42.72
Random Init.Physical Language38.12.92
Qwen3-VL-4BDirect Video33.62.84
Qwen3-VL-4BNatural-Language Enhancement36.52.89
Qwen3-VL-4BPhysical Language (Ours)41.22.97

As shown in the table, physical-language prediction substantially improves physical generation performance regardless of whether pretrained Qwen3-VL weights are used, demonstrating that the advantages of our paradigm do not solely depend on the pretrained VLM. Moreover, while natural-language enhancement improves generation performance over direct video prediction, it still falls short of PhiZero. This suggests that physical language provides the VLM with a prediction space more closely aligned with fine-grained physical world evolution than natural language.

Finally, following your suggestion, we further evaluated the physical plausibility understanding of the original Qwen3-VL-4B on IntPhys2.

ModelEasy↑Medium↑Hard↑Overall↑
Qwen2.5-VL50.9653.2551.4952.27
Qwen3-VL-4B55.7756.552.0854.64
PhiZero (Ours)60.9860.552.3856.34

As shown in the table above, PhiZero outperforms the original Qwen3-VL-4B across all IntPhys2 subsets, providing further evidence that modeling state transitions through physical language benefits physical understanding.

Thank you for the additional experiments. The experiments in W1, new comparisons with VideoWorld 2 and DisMo, and the ablation on Qwen initialization address several of my concerns. For the experiments above, I have some additional questions:

For W2: Could the authors clarify how the VideoWorld 2 and DisMo baselines were adapted and whether their training data, backbone initialization, and compute budgets were comparable to PhiZero’s?

For W3: On IntPhys2, PhiZero improves over the original Qwen3-VL-4B, but it also receives additional video training. Do the authors have a comparison in which the same backbone (Qwen3-VL) is trained on the same videos? This could help distinguish if the improvement is from the addtional training data or the proposed physical language

Reviewer_zpxrOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

PhiZero learns a compact discrete “physical language” of video state transitions. A Qwen3-VL-based reasoner predicts these tokens from an initial image and action condition, and a Wan2.2-based decoder renders the future video. The paper validates the approach on six generation and understanding benchmarks, with additional demonstrations of action conditioning and motion transfer.

Critical strengths
  1. The reason-then-render framework is elegant and well motivated. A shared transition representation connects video generation, understanding, and motion transfer in a coherent way.
  2. The paper is exceptionally clear and provides broad empirical validation. The same-data, same-compute baseline and prompt-enhancement comparison strengthen the evidence for the proposed framework. Its methodological clarity and convincing results lead me to strongly recommend acceptance.
Critical weaknesses

No critical weakness for me. I raised a number of questions after reading about the primary methodology, but the extensive comparative experiments presented in the paper fully addressed them.

Questions for authors

The analysis shows that the learned tokens capture motion and can transfer it across scenes. What evidence shows that they also capture physical rules and generalize to new physical situations?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zpxr

We sincerely appreciate your recognition of our work as "elegant and well motivated" and "methodological clarity and convincing results lead me to strongly recommend acceptance". In the revised PDF, changes made in response to your feedback appear in red. Below, we address your question (Q) in detail.

[Q] Can Physical Language Capture Physical Rules and Generalize to New Physical Situations?

We appreciate your emphasis on whether physical language captures transferable physical regularities, as it encourages us to investigate its capabilities beyond motion representation and transfer. We investigate whether physical language encodes underlying physical regularities and whether such knowledge can generalize to new physical situations. To this end, we conduct two complementary quantitative experiments.

(1) Physical Property Probing. We first investigate whether physical language captures information about underlying physical properties. Using the physical property annotations provided by PhysInOne data, we extract physical-language representations and train linear classifiers to predict three physically meaningful properties: (i) Coefficient of Restitution, which characterizes the recovery of relative velocity after collisions; (ii) Young's Modulus, which measures a material's resistance to elastic deformation; and (iii) Viscosity, which characterizes a fluid's resistance to flow. Each property is divided into three ranges (low, medium, and high), forming a three-class classification task. We compare against DINOv2 and Wan2.2 VAE features.

RepresentationRestitution↑Young's Modulus↑Viscosity↑Average↑
Random Chance33.3%33.3%33.3%33.3%
DINOv2 Features51.6%54.3%49.8%51.9%
Wan2.2 VAE Features61.2%59.4%57.6%59.4%
Physical Language (Ours)72.5%68.3%65.7%68.8%

As shown in the table, physical language consistently achieves the best performance across all three physical properties, demonstrating that it captures meaningful information about underlying physical properties.

(2) Generalization of Physical Rules to New Situations. We further investigate whether the physical regularities captured by physical language can generalize to new physical situations. We construct a cross-scenario physical violation detection task, examining whether learned representations can identify violations of physical constraints in unseen scenes.

Specifically, we consider four types of physical constraints: Spatial Continuity, Energy Consistency, Mass Conservation, and Optical Consistency. We train a lightweight linear scorer on physical-language representations to distinguish physically valid from invalid state transitions. Each valid–invalid video pair shares the same initial scene but differs in whether its subsequent evolution satisfies the corresponding physical constraint. We report pairwise accuracy, defined as the percentage of video pairs in which the physically valid video receives a higher plausibility score than the invalid one. The evaluation is conducted on objects and scene configurations held out from training set, allowing us to assess whether the learned physical consistency information transfers to new situations. We compare against DINOv2 and Wan2.2 VAE representations.

RepresentationSpatial Continuity↑Energy Consistency↑Mass Conservation↑Optical Consistency↑Average↑
DINOv2 Features59.2%57.7%51.9%58.1%56.7%
Wan2.2 VAE Features63.2%59.1%52.2%60.4%58.7%
Physical Language (Ours)82.4%75.3%65.4%76.1%74.8%

As shown in the table, physical language consistently outperforms general-purpose visual representations across all four physical constraints. Its ability to identify physical violations in unseen situations provides evidence that the learned physical language captures transferable physical regularities. In future work, we plan to further explore the generalization of physical language to unseen physical phenomena and interaction types, while expanding the diversity of physical experiences to enhance its ability to capture transferable physical regularities.

Thank you for the thorough response. The new probing and violation detection experiments fully address my question, and I encourage the authors to include them in the final version. I maintain my rating of clear acceptance.

AuthorsOpenReview ↗

Response to Reviewer zpxr

We sincerely thank you for your positive feedback and for confirming that our additional experiments have fully addressed your question. We appreciate your suggestion to include these experiments in the final version and will incorporate the corresponding analyses and discussions into the revised manuscript. Thank you again for your thoughtful feedback and strong support of our work.

Reviewer_jF7COpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PhiZero proposes a world model built around Physical Language, a compact discrete representation of physical state transitions, to improve the physical consistency of video generation. Its framework follows a reason-then-render paradigm: a Physical Language Tokenizer encodes transitions between video frames into discrete tokens, while a vision-language model predicts future token sequences conditioned on the initial frame and textual instructions. A pretrained video diffusion model then uses these tokens to generate future frames. Experiments show promising results on physics-oriented video generation benchmarks, while the learned representation also supports action-conditioned generation, interactive rollouts, and cross-embodiment motion transfer.

Critical strengths
  1. This paper employs a simple yet novel approach to enhance world models, enabling them to generate videos that better adhere to physical plausibility. It has also achieved strong results across various benchmarks focused on physical plausibility.

  2. This paper devotes a significant portion to broader applications, demonstrating that this world model can be utilized across various downstream tasks. This highlights the meaningful nature of the problem the paper aims to address and contributes to a more comprehensive and well-rounded piece of work.

  3. This paper is logically coherent, clear, and easy to understand. It also features a wealth of visualizations that allow for a more direct view of the final results.

Critical weaknesses
  1. Comparing a model enhanced with physical priors against one without only demonstrates that this approach improves physical reasoning capabilities relative to the baseline. Given that much ongoing research focuses on enhancing the physical plausibility of video generation, I believe it is also necessary to include comparisons with other physics-enhanced methods, such as PhysisForcing[1], VideoREPA[2], and PhyWorld[3].

  2. In the experiment section, there is insufficient analysis regarding areas where the model underperforms or specific failure cases. For instance, in Table 5, the model’s performance on the "fluid" category within the LikePhys benchmark ranks among the worst, and I would suggest providing further analysis on this.

  3. In the ablation studies, I did not see any analysis regarding the selection of 'q' within the Q-Former. Why was 32 chosen? Would a larger number yield better results? Furthermore, is there any connection between the choices made for 'q' in the context of reconstruction versus generation tasks when using these physical tokens?

[1] Zhang, Peiwen, et al. "PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation." arXiv preprint arXiv:2606.28128 (2026). [2] Zhang, Xiangdong, et al. "Videorepa: Learning physics for video generation through relational alignment with foundation models." Advances in Neural Information Processing Systems 38 (2026): 122647-122676. [3] Zhao, Pu, et al. "PhyWorld: Physics-Faithful World Model for Video Generation." arXiv preprint arXiv:2605.19242 (2026).

Questions for authors
  1. When training the Physical Language Tokenizer, the task performed is video reconstruction, which relies on a pixel-level loss, so how is it able to learn latent representations of state transitions?

  2. Why choose to discretize physical tokens rather than using continuous tokens directly as physical conditions?

7 comments
AuthorsOpenReview ↗

Response to Reviewer jF7C (Part 1/2)

We sincerely appreciate your recognition of our work as "a simple yet novel approach" and "devotes a significant portion to broader applications". In the revised PDF, changes made in response to your feedback appear in blue. Below, we address your weaknesses (W) and questions (Q) in detail.

[W1] Comparisons with Other Physics-Enhanced Methods

We appreciate your suggestion to include comparisons with physics-enhanced methods, as it helps us provide a more comprehensive evaluation of PhiZero. We have added comparisons with four representative physics-enhanced methods: VideoREPA [1], which distills physics-related knowledge from video foundation models into generative models; PhysisForcing [2], which improves physical consistency through hierarchical alignment objectives; PhyWorld [3], which optimizes physical plausibility using physics-preference data; and PhysRAG [4], which retrieves reference videos exhibiting similar underlying physical processes to guide generation. We evaluate these methods on Physics-IQ Verified and PhyGround, with the results summarized below:

MethodPhysics-IQ↑PhyGround General↑PhyGround Physics↑PhyGround Overall↑
VideoREPA34.52.792.872.83
PhysRAG36.72.762.722.74
PhysisForcing37.22.922.782.85
PhyWorld38.52.872.932.9
PhiZero (Ours)41.22.933.012.97

As shown in the table, PhiZero consistently outperforms existing physics-enhanced methods across all four metrics, demonstrating the effectiveness of our approach in improving physical plausibility.

[W2] Analysis of Underperformance on the Fluid Category of LikePhys

Thank you for highlighting the relatively weak performance on fluid dynamics, which motivated us to investigate its underlying causes and potential improvements. To investigate the relatively weak performance on the fluid category of LikePhys, we found that among the 1M videos used for Reasoner fine-tuning, only around 1.4K (0.14%) involve fluid dynamics. This substantial underrepresentation of fluid-related state transitions may limit the model's ability to distinguish physically plausible fluid behaviors. To validate this hypothesis, we collected approximately 5K additional fluid videos for targeted fine-tuning. We also conducted an experiment using additional general physical videos to distinguish the benefits of targeted data augmentation from those of general continued training.

MethodFluid PPE↓Rigid PPE↓Optical PPE↓
PhiZero (Original)53.15%29.14%37.50%
w/ General Physical Data51.64%28.53%35.50%
w/ Fluid-specific Data45.97%29.40%36.50%

As shown in the table, targeted fluid-data augmentation substantially improves PhiZero's performance on the fluid category of LikePhys, with a much larger gain than general continued training. These results suggest that the current performance bottleneck in fluid scenarios is closely related to the insufficient coverage of relevant data. This also highlights the potential of expanding the coverage of physical phenomena to further improve physical-language reasoning.

[W3] Ablation Study on the Number of Q-Former Queries

Your suggestion to examine the number of Q-Former queries is particularly valuable, as it allows us to better understand the trade-off between reconstruction quality and downstream generation performance. We conducted additional experiments with q = 16, 32, and 64, evaluating both reconstruction quality and downstream generation performance. The results are summarized below:

qPSNR↑SSIM↑LPIPS↓Physics-IQ↑PhyGround↑
1627.90.8790.09838.62.87
32 (Ours)28.90.9030.08741.22.97
6429.10.9070.08440.12.93

As shown in the table, increasing q from 16 to 32 improves both reconstruction quality and generation performance. However, further increasing q to 64 yields only marginal improvements in reconstruction quality while degrading downstream generation performance. This indicates that higher tokenizer reconstruction fidelity does not necessarily translate into better generation performance. For reconstruction, a larger q allows the tokenizer to preserve more fine-grained dynamic information. For generation, however, a larger q may introduce additional details that are difficult to predict from the initial state, increasing the modeling difficulty for the Reasoner. This observation is also consistent with the findings of When Worse is Better [5] regarding the trade-off between tokenizer compression and downstream generation performance. Therefore, we choose q = 32 as it achieves a better balance between preserving sufficient state-transition information and maintaining a compact, predictable physical language.

AuthorsOpenReview ↗

Response to Reviewer jF7C (Part 2/2)

[Q1] How Does the Tokenizer Learn Representations of State Transitions Through Video Reconstruction?

This is an important question about how reconstruction supervision enables the learning of state-transition representations, and we appreciate the opportunity to clarify our design. In our design, the Tokenizer's decoder is explicitly conditioned on the first frame when reconstructing future videos. This allows appearance information to be provided directly by the first frame, encouraging the physical-language tokens to focus on state transitions that cannot be inferred from the first frame alone. Thus, even with video reconstruction supervision alone, the Tokenizer can learn physical language that captures meaningful state-transition semantics while remaining relatively disentangled from object appearance.

To further validate the disentanglement between state transitions and object appearance, we constructed a video dataset containing diverse object instances and state-transition patterns. We evaluated two metrics: (1) Cross-Object Transition Accuracy (↑), which measures the ability of physical-language tokens to recognize state transitions; and (2) Cross-Transition Object Accuracy (↓), which measures how much object identity information is retained in physical-language tokens, where lower accuracy indicates less object-specific information. We tested Tokenizer under three decoder-conditioning settings: without the first frame, with a grayscale first frame, and with an RGB first frame. The results are summarized below:

Decoder ConditionCross-Object Transition Acc.↑Cross-Transition Object Acc.↓
Without First Frame56%68%
Grayscale First Frame71%32%
RGB First Frame (Ours)86%11%

As shown in the table, providing the decoder with richer initial appearance information consistently improves the ability of physical-language tokens to recognize state transitions across different objects, while substantially reducing the recoverability of object identity. These results provide further evidence that our design encourages the limited token capacity to focus on representing dynamic state changes rather than object-specific appearance information, even when trained solely through video reconstruction.

[Q2] Why Choose Discrete Physical Tokens Instead of Continuous Tokens?

We value this question about discrete versus continuous representations, as it motivates a more direct examination of our design choice. We adopt discrete physical-language tokens primarily to enable physical language to be modeled by VLMs through their native autoregressive prediction paradigm.

To validate this design, we conducted additional experiments comparing continuous and discrete representations, evaluating both reconstruction quality and downstream generation performance. For the continuous variant, we remove quantization and introduce a VAE-style KL loss. In the Reasoner, we append learnable query tokens to the input context and use the resulting VLM hidden states with a regression head to predict continuous representations.

RepresentationPSNR↑SSIM↑LPIPS↓Physics-IQ↑PhyGround↑
Continuous29.10.9080.08339.42.93
Discrete (Ours)28.90.9030.08741.22.97

As shown in the table, continuous representations achieve slightly better reconstruction quality, as they avoid quantization errors. However, discrete representations achieve superior downstream physical generation performance. By mapping state transitions into a finite symbolic vocabulary, discretization provides a prediction target naturally suited to autoregressive VLMs, allowing the Reasoner to better leverage the pretrained VLM's autoregressive modeling capabilities.

[1] Zhang X, Liao J, Zhang S, et al. Videorepa: Learning physics for video generation through relational alignment with foundation models. Advances in Neural Information Processing Systems, 2025, 38: 122647-122676.

[2] Zhang P, Deng Y, Sun S, et al. PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation. arXiv preprint arXiv:2606.28128, 2026.

[3] Zhao P, Lin J, Rupprecht T, et al. PhyWorld: Physics-Faithful World Model for Video Generation. arXiv preprint arXiv:2605.19242, 2026.

[4] Cheng K, Liu Z, Gao M, et al. PhysRAG: Enhancing physics-awareness in video generation via retrieval-augmented generation. European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 558-578.

[5] Ramanujan V, Tirumala K, Aghajanyan A, et al. When Worse is Better: Navigating the Compression Generation Trade-off In Visual Tokenization. Advances in Neural Information Processing Systems, 2025, 38: 125351-125378.

Thank you to the authors for the detailed rebuttal and for conducting numerous experiments that addressed most of my concerns. Regarding Q1, however, I would like to raise a follow-up point: the current experiments have not fully convinced me that the representations truly contain information about state transitions. Is it possible that the underlying autoregressive VLM is so powerful that the same result could be achieved by jointly training just the VLM with the subsequent diffusion model?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Brief summary and sincere thanks to all reviewers

We sincerely thank all three reviewers for their careful reading of our paper and their thoughtful feedback. We are particularly encouraged by their positive assessments of our work. Reviewer jF7C described PhiZero as “a simple yet novel approach” that “devotes a significant portion to broader applications”. Reviewer zpxr recognized our framework as “elegant and well motivated” and stated that its “methodological clarity and convincing results lead me to strongly recommend acceptance”. Reviewer V7Ca also acknowledged that “the overall quality of the paper is good”. We are also grateful that all three reviewers acknowledged that our additional experiments and clarifications had addressed most or all of their concerns.

  • Reviewer jF7C raised important questions about the empirical validation of PhiZero and whether physical language genuinely captures state transitions. We strengthened our evaluation through additional physics-enhanced baselines, failure analysis, and tokenizer ablations. More importantly, comparisons with Joint VLM–Diffusion learning and representation-level probes demonstrate that Physical Language Prediction achieves better physical generation performance while learning richer and more appearance-disentangled state-transition representations, directly addressing the reviewer's central concern.

  • Reviewer zpxr emphasized the importance of capturing physical regularities. We introduced physical-property probing for restitution, Young's modulus, and viscosity, together with physical violation detection on unseen objects and scenes. We especially appreciate the reviewer's confirmation that these experiments fully addressed the question and the continued strong recommendation for acceptance.

  • Reviewer V7Ca questioned the advantages of physical language over direct video prediction and existing latent-dynamics approaches. We added comparisons with VideoWorld 2 and DisMo, intermediate-supervision baselines, and experiments disentangling physical-language prediction from VLM pretraining and additional video training. These experiments demonstrate the advantages of physical language over alternative modeling approaches, beyond the effects of VLM pretraining and additional video training.

Together, these additional experiments provide stronger evidence that physical language captures meaningful and transferable state-transition information, while further demonstrating the advantages of explicitly modeling world evolution through physical language. The additional analyses and clarifications are documented in our revised manuscript and detailed responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.