- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces DreamPolicy: a method for humanoid locomotion on rough terrain, which combines a terrain-conditioned diffusion planner with a whole-body RL tracking controller. The planner is trained from an offline dataset, which contains transitions gathered from individual expert RL policies on five specific terrains (Flat, Uneven, Gap, Stair, Bridge). A whole-body RL tracking controller is then trained to track the planner’s output using tracking rewards and an adversarial objective with references generated by the planner. In simulation, the paper reports improvements over DAgger distillation and joint-multiterrain PPO on four unseen and five combined terrains. Furthermore, the authors show zero-shot sim-to-real transfer on a Unitree G1 robot.
Critical strengths
Meaningful hardware results. The paper presents a success rate increase for both the Gap experiment as well as the Balancing Beam experiment, compared to distillation. Additionally, they appropriately report no statistically significant difference on the Slope and Stair experiment.
Critical weaknesses
-
Unclear baseline strength and insufficiently controlled ablations. The authors do not compare their method to an existing diffusion-planner and tracker method (for example: PARC (SIGGRAPH, 2025)), and therefore do not show the benefit of their method over established diffusion-planner-tracker approaches. It is also unclear whether the joint multi-terrain PPO baseline is competitive with strong perceptive locomotion baselines such as Attention-Based Map Encoding for learning generalized legged locomotion (Science Robotics, 2025) or BeamDojo (RSS, 2025). The w/o GCRL ablation achieves only 0.06% and 35.08% success rate on Uneven and Stair respectively, despite being trained on expert data from those terrains (Table A9). The experts achieve a success rate higher than 98% on those terrains. This unexpectedly poor performance on training terrains needs explanation. Finally, it is stated in Appendix E1 that the tracking controller has no terrain input (Eq. 14). Therefore, it is unclear if the ablation w/o Diffusion Planner is a blind policy. If so, this ablation demonstrates the value of terrain information and not the value of a diffusion planner.
-
Limited evidence for generalization claim. One of the central claims in this paper is generalization beyond the expert source policies. Out of the four unseen terrains, only Slope Bridge and Balancing Beam deliver distinguishable results. Appendix B1 describes Balancing Beam as a gap with platforms of bridge width. Slope Bridge is a bridge with a sine height profile. Both therefore combine features from the expert terrains. The results therefore establish compositional generalization. However, the paper does not report the performance of the source experts on these terrains. A claim of generalization beyond those policies is therefore unsupported. Appendix E2 also acknowledges that the ability to compose the traversal of different terrains stops outside the expert data union.
Questions for authors
- Diffusion-planner baseline. What benefit does the proposed method give compared to established diffusion-planner-tracker methods?
- Multi-terrain PPO baseline. How were the architecture, training budget and hyperparameters selected and tuned?
- Ablations. Why does the w/o GCRL baseline perform poorly on the terrains Uneven and Stair, despite training on expert data from those terrains? Does w/o Diffusion Planner receive terrain observations? If not, how does it perform when it receives the same terrain information as the diffusion-planner does?
- Generalization. How do the source experts perform on the unseen and combined terrains? A clarification on which terrain combinations are used during the different training stages (expert training, planner-data collection, tracker training) would be helpful as well.
3 comments
Response to Reviewer LiBZ (Part2, Questions)
Questions
Q1: Diffusion-planner baseline. What benefit does the proposed method give compared to established diffusion-planner-tracker methods?
QA1: The benefit is the terrain-conditioned diffusion planner paired with Humanoid Motion Imagenation-conditioned policy. Established diffusion-planner-tracker methods condition on text or a target position, not on the local height map. Our planner takes the height-map embedding and the state history at every replan, and generates Humanoid Motion Imagenation(HMI). The policy tracks the HMI with four tracking rewards, and an AMP discriminator uses planner-generated transitions as positive samples. The policy therefore gets a tracking target from the planner. The policy itself does not take the height map; the terrain enters through the reference. One controller then runs on terrains and combinations it never trained on.
Q2: Multi-terrain PPO baseline. How were the architecture, training budget and hyperparameters selected and tuned?
QA2: The joint multi-terrain policy uses the following configuration trained for 50,000 iterations on NVIDIA RTX 4090 (22 hours wall time approximately) (Appendix C.1; Appendix B.4).
| Term | Configuration |
|---|---|
| Actor | MLP with hidden layers [512, 256, 128] |
| Critic | MLP with hidden layers [512, 256, 128] |
| Scandot encoder | MLP with hidden layers [128, 64, 32] |
| History encoder | 1-layer RNN, hidden size 512 |
| Observation | Proprioception, state history, command, terrain scandots |
| Training | PPO, 4096 environments |
| PPO settings | clip 0.2, GAE 0.95, lr 2e-4, 24 steps per batch, 5 epochs, 4 mini-batches |
| Terrain | Mixed five primitive terrains |
The hyperparameters are fixed across terrains.
Q4: Generalization. How do the source experts perform on the unseen and combined terrains? A clarification on which terrain combinations are used during the different training stages (expert training, planner-data collection, tracker training) would be helpful as well.
QA4: We added the experiment: five source experts were tested on the four unseen terrains and the five combined terrains (table [4], [5]). No expert reaches 40% success on any unseen terrain; the best is 37.42% (Uneven on Wave), against 92.77–99.59% for DreamPolicy. On combined terrains no expert reaches 32%; the best is 31.82% (Stair on Stair&Bridge), against 84.78–95.79% for DreamPolicy.
During the training stage:
- expert RL trains one policy per primitive terrain (Flat, Uneven, Gap, Stair, Bridge).
- Planner data are their rollouts.
- The unified policy trains online on the mixed terrain built from these primitive terrains. The four unseen and five combined terrains are evaluation-only.
[4] Unseen terrains (success / completion, %)
| Trained on | Wave | Slope | Balancing Beam | Slope Bridge |
|---|---|---|---|---|
| Flat | 2.73 / 9.46 | 0.84 / 4.37 | 0.19 / 3.62 | 0.06 / 1.28 |
| Uneven | 37.42 / 66.81 | 24.16 / 49.73 | 6.38 / 19.24 | 2.47 / 11.86 |
| Gap | 11.57 / 27.43 | 6.21 / 18.75 | 21.94 / 43.68 | 1.83 / 8.92 |
| Stair | 8.64 / 23.17 | 4.58 / 15.36 | 4.27 / 12.81 | 1.12 / 6.47 |
| Bridge | 13.29 / 31.72 | 5.73 / 17.48 | 27.86 / 52.41 | 9.67 / 26.34 |
| DreamPolicy | 99.59 / 99.72 | 99.54 / 99.70 | 96.35 / 98.33 | 92.77 / 96.80 |
[5] Combined terrains (success / completion, %)
| Trained on | Stair&Uneven | Bridge&Uneven | Stair&Gap | Stair&Bridge | Bridge&Gap |
|---|---|---|---|---|---|
| Flat | 1.36 / 7.84 | 1.92 / 10.57 | 2.48 / 13.29 | 0.43 / 4.16 | 2.07 / 11.73 |
| Uneven | 17.83 / 41.26 | 22.47 / 48.93 | 5.31 / 16.84 | 3.76 / 12.57 | 7.92 / 24.18 |
| Gap | 6.74 / 19.35 | 4.28 / 13.62 | 25.63 / 50.47 | 2.91 / 10.84 | 28.14 / 54.26 |
| Stair | 27.49 / 52.83 | 9.16 / 25.74 | 28.37 / 55.16 | 31.82 / 59.47 | 10.58 / 28.31 |
| Bridge | 12.64 / 33.18 | 27.31 / 53.62 | 8.47 / 23.95 | 18.26 / 40.73 | 30.47 / 57.84 |
| DreamPolicy | 88.46 / 96.23 | 84.78 / 90.73 | 95.79 / 98.78 | 91.45 / 95.60 | 94.68 / 96.67 |
Kind Follow-up to Reviewer LiBZ Before the Discussion Deadline
Dear Reviewer LiBZ,
Thank you again for your detailed review and insightful questions. We have made substantial efforts to address your concerns, including clarifying the comparisons with existing baselines, explaining both ablation settings, and adding a comprehensive evaluation of all five source experts on unseen and combined terrains. In particular, the new results directly address your concern about whether the unified policy generalizes beyond the capabilities of individual source experts. These additions are documented in the revised PDF, with all updates highlighted in blue. We kindly encourage you to reconsider our submission based on these additional results and clarifications, and would greatly appreciate your feedback before the discussion period closes.
Response to Reviewer LiBZ (Part1, Weaknesses)
We thank the reviewer for the detailed comments. Below we address the baselines, ablations, generalization evidence, and deployment questions.
Weaknesses
A1a: PARC targets simulated human characters rather than physical humanoid robots. Although it incorporates physics-based tracking, its controller is trained under character-specific dynamics and simulation interfaces, without accounting for real robot-specific actuator constraints and hardware dynamics. Direct deployment on real humanoids therefore requires substantial adaptation, including motion retargeting and robot-specific controller retraining. Consequently, PARC cannot be directly applied to our real-world humanoid locomotion setting.
A1b: Actually, we built our joint multi-terrain PPO baseline using newer SOTA designs: Extreme Parkour[1], TTT-Parkour[2], and Hiking in the Wild[3] (Sec. 3.2, Appendix C.1). Some newer SOTA work already compares against these perceptive baselines, and neither AME nor BeamDojo is officially open-sourced for a matched run. On the five training terrains our baseline reaches 86.83–93.18% success, against 98.46–99.65% for DreamPolicy. On Wave/Slope it reaches 99.40/99.12%, against 99.59/99.54%.
[1] Extreme parkour with legged robots. ICRA 2024, pp. 11443–11450. IEEE, 2024.
[2] TTT-Parkour: Rapid test-time training for perceptive robot parkour. CoRL 2026.
[3] Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids. arXiv 2601.07718, 2026.
A1c-QA3a: The poor performance comes from the difference in training paradigms. The ablation is a DiffuseLoco[3]-style direct-action diffusion policy, it relies purely on offline diffusion training to imitate expert actions, without subsequent RL optimization through environment interaction. Such direct action generation is insufficient for robust control of highly dynamic humanoid locomotion. It can handle relatively simple flat terrain (96.34%), but struggles significantly on complex terrains such as Stair (35.08%) and Uneven (0.06%).
[3] DiffuseLoco: Real-time legged locomotion control with diffusion from offline datasets. CoRL 2024.
A1d-QA3b: No, it is not blind. In the w/o-planner ablation, the controller receives the same 187-D terrain observation. The ablation removes the diffusion planner and the four tracking rewards tied to the planner, and uses offline AMP (Appendix C.1). It reaches 0.08/20.14 on Slope Bridge/Balancing Beam against 92.77/96.35, while the terrain-aware joint policy reaches 31.24/41.36. Terrain input alone does not explain the gap.
A2: Thanks for pointing this out. We have added the corresponding experiment: five experts tested on four unseen and five combined terrains (shown in table [4], [5]). The best expert reaches 37.42% success on the unseen terrains (Uneven on Wave), against 92.77–99.59% for DreamPolicy. On the combined courses the best expert reaches 31.82% (Stair on Stair&Bridge), against 84.78–95.79% for DreamPolicy. These results support the claim that DreamPolicy generalizes beyond the source policies. The experts fail on these terrains. One unified policy succeeds. The combined courses reuse features from the training terrains, but the experts still fail on them, so composing those features is already beyond what the source policies can do. We have revised the PDF to add this evaluation in Appendix C.2 (lines 1278–1282; Tables A10–A11, highlighted in blue).