- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents PhysiCalWorld to help world models generalize to new environments by trying a few actions and seeing what happens. For example, a robot can learn that an object slides more easily on one surface than another. Instead of retraining the whole model, it uses a context encoder to turn observations and actions into a compact physics descriptor to make better predictions. This allows the model to make better predictions and plan actions without retraining or needing explicit physics labels.
Critical strengths
-
This paper tackles physics ambiguity that can cause substantial errors in world-model prediction and planning, it's one of the hardest problem, and the paper proposed to use simple yet convincing technic by leveraging a compact descriptor without analytical physics models, explicit parameter labels, or test-time optimization.
-
Several result show strong improvement, e.g., planning success improves from 53.3% to 62.7% in Push-T and from 26.7% to 56.0% in FetchSlide over DINO-WM. Exhaustive experimental results are shown in four simulated environments and two world-model backbones, including out-of-distribution experiments.
-
Also strong computational efficiency where the descriptor reuse provides inference efficiency close to the original backbone while avoiding expensive raw-context conditioning.
Critical weaknesses
-
Several results are worse than baseline, e.g., context-prefix conditioning achieves 82.7% success in BeerPong vs. 80.0% in the proposed method (2.7% worse).
-
Some context-prefix baselines receive fewer training epochs, which may lead to unfair comparison.
-
While the paper shows that the descriptor tokens for nearby center-of-mass configurations exhibit high cosine similarity, while some distant configurations have negative similarity. This is not a direct evidence of the interpretability claim (it only suggested that there is some latent structures).
-
All experiments seem to be evaluated on simulated data (not real-world data).
Questions for authors
-
Does the number of training epoch impact the result (my concern is: if yes, some experiments are unfair)? Or perhaps a better ask is: could the author provide the training curve to see if the training is converged?
-
How is the performance on real-world data?
-
Would the experimental seed impacts that result (running the same experiments several times change the number)? My concern is: this could impact some results with smaller gains, e.g., in BeerPong.
6 comments
Author Response (2/4)
FetchSlide:
| Method | Train # of Epochs | Model Train Time | Success Rate | Planning Time (per task) | PSNR↑ | SSIM↑ | LPIPS↓ | IoU(%)↑ |
|---|---|---|---|---|---|---|---|---|
| Context Prefix | 3 | 27 h | 55.6±0.8 | 37.21 s | 26.78 | 0.990 | 0.0054 | 70.1 |
| Context Prefix | 5 | 45 h | 58.2±3.4 | 37.21 s | 27.14 | 0.991 | 0.0048 | 71.5 |
| Ours | 5 | 26 h | 54.7±4.0 | 3.91 s | 27.87 | 0.991 | 0.0047 | 73.8 |
The tables above show that our method yields predictions of better visual quality (PSNR, IoU, etc.) across all four simulation environments. However, in some environments, it achieves lower planning success rates than the context-prefix baseline. One possible reason is the model's sensitivity to noisy action sequences, as CEM initializes candidate actions by sampling from a Gaussian distribution. Nevertheless, our method enables substantially more efficient planning, requiring approximately one-tenth of the time of the context-prefix baseline, for example, 32 seconds versus 323 seconds per task in Push-T. We have also included these results in Appendix D.3 of our updated manuscript.
W3. While the paper shows that the descriptor tokens for nearby center-of-mass configurations exhibit high cosine similarity, while some distant configurations have negative similarity. This is not a direct evidence of the interpretability claim (it only suggested that there is some latent structures).
Thank you for the comment. We have updated paper and replace the 'interpretability' claim with 'latent structures'. Please see Page 9 of the updated manuscript.
To further evaluate the information encoded in the physics descriptor, we conduct an additional linear probing experiment in Push-T. Specifically, we freeze the descriptor encoder and train a single linear layer to predict the T-block’s normalized center-of-mass (CoM) coordinates from . As a reference, we train a separate Physics Estimator with approximately 20 million parameters from scratch to predict the same target.
| Probe input | Type | Model Param | Trainable Param | CoM error |
|---|---|---|---|---|
| Physics Estimator | Full Model Train from Scratch | 19.7M | 19.7M | 0.092 |
| Ours | Linear Probe | 19.7M | 0.00013M | 0.129 |
The linear probe achieves a CoM mean square error of 0.129 while training only 130 parameters (the context encoder that generate the descriptors is frozen), compared with 0.092 for the fully trained Physics Estimator as reference. Although the dedicated estimator achieves slightly lower error, this result shows that CoM information is accessible through a simple linear probing of the learned descriptors. This also indicates that the physics descriptors form some effective physics-related latent structure. We have also included these results in Appendix D.5 of our updated manuscript.
W4. All experiments seem to be evaluated on simulated data (not real-world data).
Thank you for raising this concern. We are currently setting up and conducting a real robot version of our FetchSlide environment, where we adjust the surface friction and the mass of the puck. However, due to time constraints, we probably cannot present the real robot results within the rebuttal period. But we promise to include real robot experiment results in our camera-ready version.
Author Response (3/4)
Q1. Does the number of training epoch impact the result (my concern is: if yes, some experiments are unfair)? Or perhaps a better ask is: could the author provide the training curve to see if the training is converged?
Thank you for the question. For the experiments shown in our original manuscript, we explicitly align the number of total training GPU hours between context-prefix baseline and ours. Because the per epoch train time of context-prefix baseline is much longer, it hence receive less training epochs.
In the response to W1 and W2 (please see the four tables above), we provide side-by-side comparison of 'matched epoch' and 'matched training hours' between context-prefix baseline and our method, where we observe that our method consistently outperforms context-prefix baseline in visual metrics, highlighting the prediction accuracy of our method. We also notice that in some environments, our method falls behind context-prefix baseline on planning success rate. This may be due to the model's sensitivity to noisy action inputs (the initial action of CEM planning are randomly sampled Gaussian distribution). However, planning with our method is much more efficient than the context-prefix baseline, which typically requires 10 times more time than ours, e.g., in Push-T ours(32 seconds) vs context-prefix(323 seconds).
We also provide the training curve in the paper, shown in Figure A2 on Page 29. From the training curve, we observe that, for example, the models in Push-T may not completely converged. Hence, we further study the model performance with more training steps. Due to time and compute limit, we train these two models until 16 epochs, and compare their performance under various training epoch: 6,8,10,12,14,16. (Each training epoch of the +Context Prefix model requires approximately 11 hours on 4xL40S GPU and each planning run of +Context Prefix needs 24 hours on a single L40S).
In the table below, we present the evaluation performance of the context-prefix baseline and our method over different numbers of training epoch: from epoch 6 to epoch 16, in Push-T environment. Both + Context Prefix and + PhysiCal (Ours) are based on DINO-WM. We have also included these results in Appendix D.4 of our updated manuscript.
| Method | Epoch | PSNR↑ | SSIM↑ | LPIPS↓ | IoU(%)↑ | SR(%)↑ |
|---|---|---|---|---|---|---|
| Context Prefix | 6 | 22.80 | 0.961 | 0.0304 | 60.3 | 59.1±6.6 |
| PhysiCal (Ours) | 6 | 23.17 | 0.962 | 0.0290 | 62.2 | 63.6±0.8 |
| Context Prefix | 8 | 23.11 | 0.961 | 0.0308 | 60.8 | 61.3±4.0 |
| PhysiCal (Ours) | 8 | 23.23 | 0.962 | 0.0295 | 62.2 | 58.2±5.4 |
| Context Prefix | 10 | 23.13 | 0.962 | 0.0300 | 61.5 | 59.6±2.8 |
| PhysiCal (Ours) | 10 | 23.83 | 0.964 | 0.0267 | 65.1 | 61.8±4.1 |
| Context Prefix | 12 | 23.09 | 0.961 | 0.0316 | 60.3 | 61.3±2.7 |
| PhysiCal (Ours) | 12 | 23.15 | 0.961 | 0.0322 | 59.8 | 65.8±4.7 |
| Context Prefix | 14 | 22.88 | 0.960 | 0.0337 | 58.5 | 62.2±4.7 |
| PhysiCal (Ours) | 14 | 23.69 | 0.963 | 0.0295 | 63.2 | 62.7±2.7 |
| Context Prefix | 16 | 23.33 | 0.962 | 0.0317 | 60.6 | 64.9±4.1 |
| PhysiCal (Ours) | 16 | 23.60 | 0.963 | 0.0296 | 62.3 | 68.9±4.7 |
From the table, we can see that the evaluation performance fluctuate after epoch 10, even though the training loss can still decrease. In addition, we can see that our method generally outperform the context-prefix baseline in both visual metrics and planning success rate.
Q2. How is the performance on real-world data?
Thank you for the question. We are currently setting up and conducting a real robot version of our FetchSlide environment, where we adjust the surface friction and the mass of the puck. However, due to time constraints, we probably cannot present the real robot results within the rebuttal period. But we promise to include real robot experiment results in our camera-ready version.
Author Response (1/4)
We sincerely thank reviewer Ysz4 for their thoughtful review and constructive feedback. We have run additional experiments and revised our manuscript accordingly. Please see our response to each concern below.
Thank you for raising these concerns. We provide our responses along with additional experiments results for Weakness 1 and Weakness 2 together in the four tables below.
Weakness 1. In terms of planning success rate in BeerPong environment, we do notice that our method is slightly behind (2.7% worse) in our original manuscript. However, our method is significantly more compute efficient, in both training and inference. For example, in BeerPong environment, the planning time of our method is only 9 seconds while Context Prefix needs 76 seconds. For training, the per epoch time of our method is around 9 hours while Context Prefix needs more than 18 hours. In addition, with the matched training budget, our method is able to achieve better visual accuracy (e.g., IoU) over the context prefix in all environments, for example, ours (65.1%) vs context-prefix (61.4%) in Push-T. Please see the detailed training and inference speed comparison in the provided tables below.
Weakness 2. (regarding unfair comparison), we would like to note that both ours and the context-prefix baseline use the same amount of training budget, measured by total GPU hours. Due to the slow training speed of the context-prefix baseline, using similar GPU hours results in much less training epochs compared with ours. For example, in Push-T, as shown in the table below, context-prefix baseline is only trained for 5 epochs, but it requires 52 hours of training using 4xL40S GPUs, while our method only needs 50 hours to reach 10 epochs with the same 4xL40S GPU setup.
For a complete study, we provide two comparisons in the table belows, "matched training cost" (measured by similar GPU hours, compare row 1 and row 3 of each table) and "matched training epoch" (compare row 2 and row 3 of each table). Note that for "matched training epoch", Context Prefix uses significantly more GPU hours than Ours. For the planning results, we report the mean and standard deviation over 3 random seeds. Note that for Reacher and BeerPong, the results are slightly different with the ones reported in the initial manuscript, since we re-train the models using the same 4xL40S setup as our method for an apple-to-apple comparison. Total GPU-hours are four times these values because all models below are trained on four L40S GPUs.
Push-T:
Reacher:
BeerPong: