- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DR-WM for predicting robot motion and future observations in outdoor quadruped navigation. It combines terrain-property prediction, physics simulation calibrated against real robot responses, and video generation. The predicted videos are also used to help a VLM choose actions. Experiments evaluate video quality, motion prediction, and action selection.
Critical strengths
- The motivation is clear. Different terrain conditions can lead to different outcomes for the same robot command, making this a relevant prediction problem.
- The physical model is calibrated against real robot responses. Tables 4 and 13 show improved motion prediction over the default and uniform-parameter settings.
- The supplementary experiments are useful. They include comparisons with lookup-based methods, video-generation ablations, and multi-step predictions without future real observations (Appendices C, E, and H).
Critical weaknesses
- The contribution of PhyMap is not fully established across the evaluation stages. Table 9 compares physical-prior prediction, Tables 4 and 13 evaluate motion prediction, and Table 5 evaluates action selection. However, the paper does not clearly connect these results or show that the advantage in physical-prior prediction leads to the downstream gains. The training and adaptation protocols of the Table 9 baselines also need clarification.
- The physical-property evaluation measures agreement with proxy targets. Since the targets are derived from semantic and language-model priors, Table 9 can compare prediction agreement, but it does not independently validate the absolute correctness of the physical properties. The paper should distinguish proxy-target accuracy from recovery of real terrain properties.
- The evaluation protocol in Table 5 limits the interpretation of the action-selection result. Section 4.4 describes candidate-wise rollouts and reports metrics for the VLM-selected action, but does not clearly explain how candidate actions and the selected-action outcome are obtained. This makes it difficult to assess what the reported gain represents.
Questions for authors
- For Table 9, did the compared methods use the same physical-prior targets, data, test split, and adaptation protocol as PhyMap? Were they evaluated zero-shot or trained/fine-tuned on Physcene?
- How do the improvements in physical-prior prediction and motion prediction translate into action-selection performance?
- How are candidate actions generated, and how is the outcome of the VLM-selected action in Table 5 obtained?
- What independent evidence supports the physical-prior values beyond agreement with the lookup-derived targets?
4 comments
Response to Question 2
Your detailed feedback is greatly appreciated. PhyMap estimates terrain contact properties, which Genesis uses to predict how the robot will move under each candidate action. The resulting future frames help the VLM compare progress, drift, and stability across candidates. Tables 4 and 13 show the gains in motion prediction. Table 5 shows that, with the full DR-WM rollouts, the VLM selects actions that make more progress with less tilt and slip in simulation. We explain this connection in the newly added Appendix I.
Response to Question 3
Thank you for the time and effort. A VLM proposes candidate maneuvers based on the current image and navigation goal. Each candidate is paired with a locomotion command sequence, and all settings use the same candidates. The selection VLM returns the ID of its chosen candidate. For Table 5, we compute the metrics from that candidate’s simulation logs, recorded during execution by the official locomotion policy. The newly added Appendix I describes this procedure and provides example prompts.
Response to Question 4
We are grateful for your careful reading and valuable comments. Tables 4 and 13 provide evidence independent of the lookup table labels. Both experiments compare simulated motion with real robot motion from test scenes held out from calibration. In Table 4, Full PhyMap has lower motion prediction errors than Uniform PhyMap. Both use the same scene geometry, control commands, and mean contact parameter values. Table 13 shows similar gains over longer rollouts.
DR-WM uses physical priors inferred from images to predict how a given robot will interact with the terrain. For example, the coefficient of friction depends on the foot material, the terrain surface, and the contact conditions. This makes it very difficult to estimate actual contact parameter values from images alone. We use real robot action and response data to calibrate these priors and obtain effective contact parameters in Genesis. We then evaluate the calibrated priors in motion prediction experiments.
Response to Question 1
We sincerely appreciate the time and care you devoted to reviewing our work and providing constructive feedback. All VLM-based baselines were evaluated in a zero-shot setting on Physcene, without additional fine-tuning. To maintain consistent physical prior definitions, their prompts included the same 32-class mapping to friction, roughness, and stiffness used to construct PhyMap’s dense supervision. Accordingly, PhyMap learned physical prior from supervised training on Physcene. These evaluation settings are clarified in the revised Appendix C.