acceptodds
Under review as a conference paper at ICLR 2027

DreamPolicy: Scaling Humanoid Locomotion through Terrain-Aware Generative Planning

Abstract

Achieving versatile humanoid locomotion with a single policy remains a key scalability challenge. Existing approaches often distill multiple terrain-specific policies into a unified policy, but such distillation mainly transfers individual locomotion primitives and provides limited ability to compose them when encountering novel and structurally different terrains. We present DreamPolicy, a scalable framework for humanoid locomotion through terrain-aware generative planning. We first construct a modular multi-terrain benchmark with over a dozen primitive terrains and their combinations, and collect 75M transitions of expert rollouts from specialized policies, providing diverse locomotion primitives and terrain-conditioned behaviors for learning. On this dataset, we train a terrain-aware autoregressive diffusion planner to generate a humanoid motion imagination (HMI), a physically plausible future state trajectory conditioned on state history and terrain information. A unified HMI-conditioned policy then tracks these generated future states, allowing locomotion behaviors to be dynamically composed according to the encountered terrain without terrain-specific reward design. DreamPolicy scales consistently with offline data and generalizes beyond the terrains covered by the source policies. Experiments show substantial improvements over strong baselines on unseen and composite terrains, together with zero-shot transfer to a Unitree G1. These results demonstrate that terrain-aware generative planning provides a scalable approach to humanoid locomotion beyond the "one task, one policy" regime.

Primary area:
reinforcement learning
Keywords:
Humanoid Locomotion, Generative Planning, Diffusion Models, Terrain Generalization, Motion Imagination

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LiBZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DreamPolicy: a method for humanoid locomotion on rough terrain, which combines a terrain-conditioned diffusion planner with a whole-body RL tracking controller. The planner is trained from an offline dataset, which contains transitions gathered from individual expert RL policies on five specific terrains (Flat, Uneven, Gap, Stair, Bridge). A whole-body RL tracking controller is then trained to track the planner’s output using tracking rewards and an adversarial objective with references generated by the planner. In simulation, the paper reports improvements over DAgger distillation and joint-multiterrain PPO on four unseen and five combined terrains. Furthermore, the authors show zero-shot sim-to-real transfer on a Unitree G1 robot.

Critical strengths

Meaningful hardware results. The paper presents a success rate increase for both the Gap experiment as well as the Balancing Beam experiment, compared to distillation. Additionally, they appropriately report no statistically significant difference on the Slope and Stair experiment.

Critical weaknesses
  • Unclear baseline strength and insufficiently controlled ablations. The authors do not compare their method to an existing diffusion-planner and tracker method (for example: PARC (SIGGRAPH, 2025)), and therefore do not show the benefit of their method over established diffusion-planner-tracker approaches. It is also unclear whether the joint multi-terrain PPO baseline is competitive with strong perceptive locomotion baselines such as Attention-Based Map Encoding for learning generalized legged locomotion (Science Robotics, 2025) or BeamDojo (RSS, 2025). The w/o GCRL ablation achieves only 0.06% and 35.08% success rate on Uneven and Stair respectively, despite being trained on expert data from those terrains (Table A9). The experts achieve a success rate higher than 98% on those terrains. This unexpectedly poor performance on training terrains needs explanation. Finally, it is stated in Appendix E1 that the tracking controller has no terrain input (Eq. 14). Therefore, it is unclear if the ablation w/o Diffusion Planner is a blind policy. If so, this ablation demonstrates the value of terrain information and not the value of a diffusion planner.

  • Limited evidence for generalization claim. One of the central claims in this paper is generalization beyond the expert source policies. Out of the four unseen terrains, only Slope Bridge and Balancing Beam deliver distinguishable results. Appendix B1 describes Balancing Beam as a gap with platforms of bridge width. Slope Bridge is a bridge with a sine height profile. Both therefore combine features from the expert terrains. The results therefore establish compositional generalization. However, the paper does not report the performance of the source experts on these terrains. A claim of generalization beyond those policies is therefore unsupported. Appendix E2 also acknowledges that the ability to compose the traversal of different terrains stops outside the expert data union.

Questions for authors
  • Diffusion-planner baseline. What benefit does the proposed method give compared to established diffusion-planner-tracker methods?
  • Multi-terrain PPO baseline. How were the architecture, training budget and hyperparameters selected and tuned?
  • Ablations. Why does the w/o GCRL baseline perform poorly on the terrains Uneven and Stair, despite training on expert data from those terrains? Does w/o Diffusion Planner receive terrain observations? If not, how does it perform when it receives the same terrain information as the diffusion-planner does?
  • Generalization. How do the source experts perform on the unseen and combined terrains? A clarification on which terrain combinations are used during the different training stages (expert training, planner-data collection, tracker training) would be helpful as well.
3 comments
AuthorsOpenReview ↗

Response to Reviewer LiBZ (Part1, Weaknesses)

We thank the reviewer for the detailed comments. Below we address the baselines, ablations, generalization evidence, and deployment questions.

Weaknesses

W1a: The authors do not compare their method to an existing diffusion-planner and tracker method (for example: PARC (SIGGRAPH, 2025)), and therefore do not show the benefit of their method over established diffusion-planner-tracker approaches.

A1a: PARC targets simulated human characters rather than physical humanoid robots. Although it incorporates physics-based tracking, its controller is trained under character-specific dynamics and simulation interfaces, without accounting for real robot-specific actuator constraints and hardware dynamics. Direct deployment on real humanoids therefore requires substantial adaptation, including motion retargeting and robot-specific controller retraining. Consequently, PARC cannot be directly applied to our real-world humanoid locomotion setting.

W1b: It is also unclear whether the joint multi-terrain PPO baseline is competitive with ...... or BeamDojo (RSS, 2025).

A1b: Actually, we built our joint multi-terrain PPO baseline using newer SOTA designs: Extreme Parkour[1], TTT-Parkour[2], and Hiking in the Wild[3] (Sec. 3.2, Appendix C.1). Some newer SOTA work already compares against these perceptive baselines, and neither AME nor BeamDojo is officially open-sourced for a matched run. On the five training terrains our baseline reaches 86.83–93.18% success, against 98.46–99.65% for DreamPolicy. On Wave/Slope it reaches 99.40/99.12%, against 99.59/99.54%.

[1] Extreme parkour with legged robots. ICRA 2024, pp. 11443–11450. IEEE, 2024.
[2] TTT-Parkour: Rapid test-time training for perceptive robot parkour. CoRL 2026.
[3] Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids. arXiv 2601.07718, 2026.

W1c: The w/o GCRL ablation achieves only 0.06% and 35.08% success rate on Uneven and Stair respectively, despite being trained on expert data from those terrains (Table A9). The experts achieve a success rate higher than 98% on those terrains. This unexpectedly poor performance on training terrains needs explanation.

Q3a: Ablations. Why does the w/o GCRL baseline perform poorly on the terrains Uneven and Stair, despite training on expert data from those terrains?

A1c-QA3a: The poor performance comes from the difference in training paradigms. The ablation is a DiffuseLoco[3]-style direct-action diffusion policy, it relies purely on offline diffusion training to imitate expert actions, without subsequent RL optimization through environment interaction. Such direct action generation is insufficient for robust control of highly dynamic humanoid locomotion. It can handle relatively simple flat terrain (96.34%), but struggles significantly on complex terrains such as Stair (35.08%) and Uneven (0.06%).

[3] DiffuseLoco: Real-time legged locomotion control with diffusion from offline datasets. CoRL 2024.

W1d: Finally, it is stated in Appendix E1 that the tracking controller has no terrain input (Eq. 14). Therefore, it is unclear if the ablation w/o Diffusion Planner is a blind policy. If so, this ablation demonstrates the value of terrain information and not the value of a diffusion planner.

Q3b: Does w/o Diffusion Planner receive terrain observations? If not, how does it perform when it receives the same terrain information as the diffusion-planner does?

A1d-QA3b: No, it is not blind. In the w/o-planner ablation, the controller receives the same 187-D terrain observation. The ablation removes the diffusion planner and the four tracking rewards tied to the planner, and uses offline AMP (Appendix C.1). It reaches 0.08/20.14 on Slope Bridge/Balancing Beam against 92.77/96.35, while the terrain-aware joint policy reaches 31.24/41.36. Terrain input alone does not explain the gap.

W2: Limited evidence for generalization claim...... Appendix E2 also acknowledges that the ability to compose the traversal of different terrains stops outside the expert data union.

A2: Thanks for pointing this out. We have added the corresponding experiment: five experts tested on four unseen and five combined terrains (shown in table [4], [5]). The best expert reaches 37.42% success on the unseen terrains (Uneven on Wave), against 92.77–99.59% for DreamPolicy. On the combined courses the best expert reaches 31.82% (Stair on Stair&Bridge), against 84.78–95.79% for DreamPolicy. These results support the claim that DreamPolicy generalizes beyond the source policies. The experts fail on these terrains. One unified policy succeeds. The combined courses reuse features from the training terrains, but the experts still fail on them, so composing those features is already beyond what the source policies can do. We have revised the PDF to add this evaluation in Appendix C.2 (lines 1278–1282; Tables A10–A11, highlighted in blue).

AuthorsOpenReview ↗

Response to Reviewer LiBZ (Part2, Questions)

Questions

Q1: Diffusion-planner baseline. What benefit does the proposed method give compared to established diffusion-planner-tracker methods?

QA1: The benefit is the terrain-conditioned diffusion planner paired with Humanoid Motion Imagenation-conditioned policy. Established diffusion-planner-tracker methods condition on text or a target position, not on the local height map. Our planner takes the height-map embedding and the state history at every replan, and generates Humanoid Motion Imagenation(HMI). The policy tracks the HMI with four tracking rewards, and an AMP discriminator uses planner-generated transitions as positive samples. The policy therefore gets a tracking target from the planner. The policy itself does not take the height map; the terrain enters through the reference. One controller then runs on terrains and combinations it never trained on.

Q2: Multi-terrain PPO baseline. How were the architecture, training budget and hyperparameters selected and tuned?

QA2: The joint multi-terrain policy uses the following configuration trained for 50,000 iterations on NVIDIA RTX 4090 (22 hours wall time approximately) (Appendix C.1; Appendix B.4).

TermConfiguration
ActorMLP with hidden layers [512, 256, 128]
CriticMLP with hidden layers [512, 256, 128]
Scandot encoderMLP with hidden layers [128, 64, 32]
History encoder1-layer RNN, hidden size 512
ObservationProprioception, state history, command, terrain scandots
TrainingPPO, 4096 environments
PPO settingsclip 0.2, GAE 0.95, lr 2e-4, 24 steps per batch, 5 epochs, 4 mini-batches
TerrainMixed five primitive terrains

The hyperparameters are fixed across terrains.

Q4: Generalization. How do the source experts perform on the unseen and combined terrains? A clarification on which terrain combinations are used during the different training stages (expert training, planner-data collection, tracker training) would be helpful as well.

QA4: We added the experiment: five source experts were tested on the four unseen terrains and the five combined terrains (table [4], [5]). No expert reaches 40% success on any unseen terrain; the best is 37.42% (Uneven on Wave), against 92.77–99.59% for DreamPolicy. On combined terrains no expert reaches 32%; the best is 31.82% (Stair on Stair&Bridge), against 84.78–95.79% for DreamPolicy.

During the training stage:

  • expert RL trains one policy per primitive terrain (Flat, Uneven, Gap, Stair, Bridge).
  • Planner data are their rollouts.
  • The unified policy trains online on the mixed terrain built from these primitive terrains. The four unseen and five combined terrains are evaluation-only.

[4] Unseen terrains (success / completion, %)

Trained onWaveSlopeBalancing BeamSlope Bridge
Flat2.73 / 9.460.84 / 4.370.19 / 3.620.06 / 1.28
Uneven37.42 / 66.8124.16 / 49.736.38 / 19.242.47 / 11.86
Gap11.57 / 27.436.21 / 18.7521.94 / 43.681.83 / 8.92
Stair8.64 / 23.174.58 / 15.364.27 / 12.811.12 / 6.47
Bridge13.29 / 31.725.73 / 17.4827.86 / 52.419.67 / 26.34
DreamPolicy99.59 / 99.7299.54 / 99.7096.35 / 98.3392.77 / 96.80

[5] Combined terrains (success / completion, %)

Trained onStair&UnevenBridge&UnevenStair&GapStair&BridgeBridge&Gap
Flat1.36 / 7.841.92 / 10.572.48 / 13.290.43 / 4.162.07 / 11.73
Uneven17.83 / 41.2622.47 / 48.935.31 / 16.843.76 / 12.577.92 / 24.18
Gap6.74 / 19.354.28 / 13.6225.63 / 50.472.91 / 10.8428.14 / 54.26
Stair27.49 / 52.839.16 / 25.7428.37 / 55.1631.82 / 59.4710.58 / 28.31
Bridge12.64 / 33.1827.31 / 53.628.47 / 23.9518.26 / 40.7330.47 / 57.84
DreamPolicy88.46 / 96.2384.78 / 90.7395.79 / 98.7891.45 / 95.6094.68 / 96.67
AuthorsOpenReview ↗

Kind Follow-up to Reviewer LiBZ Before the Discussion Deadline

Dear Reviewer LiBZ,

Thank you again for your detailed review and insightful questions. We have made substantial efforts to address your concerns, including clarifying the comparisons with existing baselines, explaining both ablation settings, and adding a comprehensive evaluation of all five source experts on unseen and combined terrains. In particular, the new results directly address your concern about whether the unified policy generalizes beyond the capabilities of individual source experts. These additions are documented in the revised PDF, with all updates highlighted in blue. We kindly encourage you to reconsider our submission based on these additional results and clarifications, and would greatly appreciate your feedback before the discussion period closes.

Reviewer_nJE5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

DreamPolicy learns a generative model of future humanoid motions and trains a unified controller to track those generated trajectories. Their main contribution is the combination of terrain-conditioned autoregressive motion generation with a unified RL tracking policy using large-scale expert trajectories as the motion prior.

Critical strengths
  1. Learning a unified humanoid controller that adapts across distinct terrain structures without switching among task-specific controllers is an important problem
  2. The authors introduce 75M-transition expert dataset, modular terrain environments, Isaac Gym/Isaac Lab support
  3. The paper shows promising zero-shot sim-to-real transfer to Unitree G1, with significant improvements on gaps and balancing beams.
Critical weaknesses
  1. Unseen terrains used share characteristics with training terrains, so generalization to fundamentally different locomotion behaviors is unexplored.
  2. Existing ablations do not show whether diffusion is necessary when compared with deterministic future-state prediction using the same tracking controller.
  3. Several existing works like DiffuseLoco(https://proceedings.mlr.press/v270/huang25a.html); BeyondMimic(https://arxiv.org/abs/2508.08241); ReactiveMimic(https://arxiv.org/abs/2606.30362); T-GMP(https://arxiv.org/abs/2606.06944) already explore using generative motion models for control. The authors didn't clearly establish which of their contribution accounts for its reported improvements.
Questions for authors
  1. How much of the performance improvement can be due to diffusion-based trajectory generation rather than because of future-state references or the goal-conditioned tracking architecture? Could the authors compare against a deterministic future-state prediction model paired with the same tracker?
  2. Can the planner generate motion transitions that are not present in the source demonstrations, and how do these compare with deterministic trajectory prediction or expert-motion retrieval?
  3. Could the authors specify the diffusion sampling steps, prediction horizon, replanning frequency, and end-to-end inference latency on hardware? How are stale predictions handled when the controller executes faster than the planner or terrain-map updates, and is the terrain embedding recomputed at every replanning step?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nJE5 (Part1, Weaknesses & Questions 1)

Thank you for your careful review. The review raises three main points: unseen-terrain scope, deterministic prediction, and related-work distinctions. Each is answered below.

Weaknesses

W1: Unseen terrains used share characteristics with training terrains, so generalization to fundamentally different locomotion behaviors is unexplored.

A1: We agree that some of the unseen terrains share characteristics with the training terrains, but not all. In particular, wave is a periodic height profile that lies outside the five primitives, and no expert was trained on it. DreamPolicy still reaches 99.59% success on Wave, against 37.42% for the best single expert (Table 1; [4]).

W2: Existing ablations do not show whether diffusion is necessary when compared with deterministic future-state prediction using the same tracking controller.

Q1: How much of the performance improvement can be due to diffusion-based trajectory generation rather than because of future-state references or the goal-conditioned tracking architecture? Could the authors compare against a deterministic future-state prediction model paired with the same tracker?

A2–QA1: Thanks for pointing this out. We have added a matched deterministic predictor experiment, as shown in Table [1], [2]. This experiment keeps the same transformer backbone, the same input/output, and the same training data. We replace the diffusion sampler with a one-pass predictor trained by MSE loss, and train a policy on this predictor. Evaluation uses 4,096 parallel environments for 10 episodes, 5 rounds (40,960 episodes per round).
It reaches 71.34/78.91 on Slope Bridge/Balancing Beam against 92.77/96.35 for DreamPolicy, and 75.18/71.93/84.72 on Stair&Uneven/Bridge&Uneven/Stair&Bridge against 88.46/84.78/91.45.
These results show that the performance improvement cannot be attributed solely to future-state references or to the goal-conditioned tracking architecture; diffusion-based trajectory generation itself plays an important role. MSE regression tends to average multiple plausible future motions into a single prediction, whereas diffusion models their multimodal distribution and can generate distinct feasible motion modes. This is particularly important for composing locomotion skills across different terrain configurations. We have revised the PDF to add this control to Sec. 5.1.1 and Appendix C.1 and to report it in Tables 1–2 (revised PDF, lines 363–365, 393 and 437; Appendix C.1, lines 1269–1274, highlighted in blue).

[1] Unseen terrains (success / completion, %)

MethodSlope BridgeBalancing BeamWaveSlope
Deterministic transformer + tracker71.34 / 85.0778.91 / 90.6299.46 / 99.6699.38 / 99.58
DreamPolicy92.77 / 96.8096.35 / 98.3399.59 / 99.7299.54 / 99.70
Joint multi-terrain PPO31.24 / 37.9641.36 / 48.7399.40 / 99.7099.12 / 99.55

[2] Composite terrains (success / completion, %)

MethodStair&UnevenBridge&UnevenStair&GapStair&BridgeBridge&Gap
Deterministic transformer + tracker75.18 / 86.4771.93 / 83.1596.91 / 99.1284.72 / 92.1894.44 / 96.51
DreamPolicy88.46 / 96.2384.78 / 90.7395.79 / 98.7891.45 / 95.6094.68 / 96.67
Joint multi-terrain PPO42.15 / 60.0454.62 / 68.0978.35 / 85.3775.46 / 84.0486.52 / 94.95

W3: Several existing works like DiffuseLoco; BeyondMimic; ReactiveMimic; T-GMP already explore using generative motion models for control. The authors didn't clearly establish which of their contribution accounts for its reported improvements.

A3: DiffuseLoco is designed and evaluated primarily for quadrupeds. Our w/o GCRL variant serves as the inherent adaptation of DiffuseLoco for humanoid locomotion. We did not run the other three for the reason below:

  • BeyondMimic is motion-tracking on human motion capture: no terrain input, flat ground; the paper reuses only its body-tracking reward.
  • ReactiveBFM (ReactiveMimic) is an autoregressive motion diffusion planner with a behavioral-foundation-model tracker, conditioned on text or a target position, no terrain input, trained on retargeted motion, not terrain-specialist RL transitions.
  • T-GMP is a terrain-conditioned CVAE prior used at training time, not an online planner.

The ablation isolates the two design choices: removing the reference (w/o planner) or the tracker (w/o GCRL) collapses the discriminative terrains (92.77/96.35 → 0.08/20.14 / 0.05/0.12).

AuthorsOpenReview ↗

Response to Reviewer nJE5 (Part2, Questions 2, 3)

Questions

Q2: Can the planner generate motion transitions that are not present in the source demonstrations, and how do these compare with deterministic trajectory prediction or expert-motion retrieval?

QA2: Yes. Diffusion models the multi-modal distribution of future states, so it can sample transitions absent from the source demonstrations. Tables [1], [2] show that the deterministic predictor trails by 17.4–21.4 points on Slope Bridge and Balancing Beam. DreamPolicy reaches 84.78–95.79% on the unseen and combined terrains.

Q3: Could the authors specify the diffusion sampling steps, prediction horizon, replanning frequency, and end-to-end inference latency on hardware? How are stale predictions handled when the controller executes faster than the planner or terrain-map updates, and is the terrain embedding recomputed at every replanning step?

QA3: The deployment parameters are below (Sec. 5.2; Appendix C.4).

ParameterValue
Diffusion sampling steps5
Prediction horizon15
Replanning interval4 prediction windows
Control frequency50 Hz
Perception / elevation mapping≈20 Hz
Planner inference (external PC)≈20 ms

The asynchronous buffer lets the 50 Hz controller run without waiting for new perception or planner results. The controller follows the latest plan, and each new plan uses the measured state history.

Yes, the terrain embedding is recomputed from the current height map at every replanning step.

AuthorsOpenReview ↗

Kind Follow-up to Reviewer nJE5 Before the Discussion Deadline

Dear Reviewer nJE5,

Thank you again for your constructive review. In particular, we have added a matched deterministic-prediction baseline using the same backbone, inputs, outputs, and training data, providing direct experimental evidence for the contribution of diffusion-based generation. We have also clarified the scope of our generalization claims and the distinctions from related methods. These new results and clarifications are included in the revised PDF, with all updates highlighted in blue. We kindly encourage you to reconsider our submission in light of this additional evidence and would sincerely appreciate your feedback before the discussion period closes.

Reviewer_c2m4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DreamPolicy, which is a framework for reusing terrain-specific locomotion skills in a single humanoid controller. It contains a terrain-conditioned diffusion planner that learn to predict future robot states from expert rollouts, with an RL controller tracks predictions from the planner. The paper reports improved performance in simulation, and also show real deployment experiments.

Critical strengths

The overall goal is practical: reusing terrain-specific skills in one controller. The models came from this idea is experimented on real robot with a useful gain on gap traversal, 16/20 successes comparing 8/20 for distillation.

Critical weaknesses
  1. The claim of "zero-shot" is a bit confusing, what exactly is it, how to support the claim? In figure 3, it calls the combined and unseen terrains zero-shot tests, but the terrains used for the unified controller's online RL are not listed. The understanding is a terrain being new to the source experts does not mean it was actually held out from controller training, this needs more clarification.
  2. Paper CLoSD already combines diffusion planning with RL tracking in a different task. Here in this work, more interesting claim could be terrain-aware skill composition. In Table 1, removing the planner also switches to offline AMP, and the paper doesn't say how the planner-based tracking rewards are handled. The drop here could be contributed by other reasons, say come from changing the controller's learning objective, so this ablation alone doesn't fully explain the gains.

CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control. ICLR 2025

  1. This paper emphasizes scaling with expert data, but adding more experts and using their experience more efficiently are essentially different things. More experts also bring more skills and terrain coverage. I don't see a clear data-efficiency advantage over distillation when both expert experience and online training effort are taken into account.

  2. For me the practical value of skill reuse is to reduce the work needed when moving to new terrains. The problem is real, but the contribution here feels more like another way to provide motion references to a controller. I am not yet convinced this is a substantial step towards reusing skills with less terrain-specific training and controller design.

Questions for authors

For the version without the planner, what goes into the controller and what happens to the four tracking rewards? Is it removed or have a different reference?

In figure 5, when changing the data size, are all terrain types kept in the dataset? how much data and online training does each method get at each expert count?

3 comments
AuthorsOpenReview ↗

Response to Reviewer c2m4

Thank you for the detailed review. The zero-shot definition, ablation, scaling, and skill reuse are addressed below.

Weaknesses

W1: The claim of "zero-shot" is a bit confusing, what exactly is it, how to support the claim? In figure 3, ...... this needs more clarification.

A1: Thanks for pointing this out. We have revised the PDF to state this stage order in Sec. 3 (revised PDF, lines 204–205, 252–253 and 307–308, highlighted in blue). For training and evaluation, zero-shot means the unified policy is evaluated on unseen and combined terrains without training on them. It reaches 92.77–99.59% on unseen terrains and 84.78–95.79% on combined terrains (Tables 1–2). Terrains used during training and evaluation are:

  • Expert policy: one policy per primitive terrain (Flat, Uneven, Gap, Stair, Bridge).
  • Planner data: rollouts from these five experts.
  • Unified policy: online training on the mixed terrain of the five primitive terrains, evaluating on four unseen and five combined terrains.

For hardware and deployment, zero-shot means the policy runs on the real robot without real-world fine-tuning (Sec. 5.2).

W2: Paper CLoSD already combines diffusion planning with RL tracking in a different task. Here in this work, ...... say come from changing the controller's learning objective, so this ablation alone doesn't fully explain the gains

A2: We agree that terrain-conditioned composition of learned locomotion primitives is the more relevant contribution. Regarding the ablation:

  1. The four tracking rewards are removed because they are defined on planner-generated future states, they are unavailable without the planner.
  2. These changes are necessary because both the tracking rewards and online AMP supervision depend on the planner. Therefore, removing the planner requires removing or replacing these terms. This is a fair ablation of the planner-guided framework.

The resulting success rates drop from 92.77/96.35% to 0.08/20.14% on unseen terrains like Slope Bridge/Balancing Beam. We have revised the PDF to document these configurations in Appendix C.1 (lines 1238–1241 and 1260–1266, highlighted in blue).

W3: This paper emphasizes scaling with expert data, but ...... both expert experience and online training effort are taken into account.

A3: Fig. 5 has demonstrating more effective utilization of expert data. With the same expert datasets and online training budget at each expert count shown , DreamPolicy achieves 96.35% success with five experts, compared with 74.36% for distillation; with three experts, DreamPolicy achieve 30.37%, compared with 2.87% for distillation.
DreamPolicy improve data-efficiency through the generative planner models the multimodal distribution of expert motions and composes terrain-relevant motion modes online, whereas distillation tends to merge different expert behaviors into a single policy.

W4: For me the practical value of skill reuse is to reduce the work needed when moving to new terrains. The problem is real, but the contribution here feels more like another way to provide motion references to a controller. I am not yet convinced this is a substantial step towards reusing skills with less terrain-specific training and controller design.

A4: DreamPolicy enables skill reuse on new terrains. In experiments, DreamPolicy beats distillation and joint multi-terrain policy on unseen and combined terrains under the same data and training budget: 92.77–99.59% vs 65.48–74.36% on unseen (Table 1), and 84.78–95.79% vs 50.48–91.34% for distillation and 42.15–86.52% for joint multi-terrain policy on combined (Table 2). Fig. 5 also shows the same trend as expert data grows.
The diffusion planner learns the multi-modal distribution of the five expert datasets. The controller tracks the reference; and the controller needs less terrain-specific reward or observation.

Questions

Q1: For the version without the planner, what goes into the controller and what happens to the four tracking rewards? Is it removed or have a different reference?

QA1: Without the planner, the controller receives proprioception, the observation history, the command, terrain observation and the previous action. The four tracking rewards are removed (Appendix C.1). The remaining reward terms stay, and the AMP discriminator uses offline expert transitions.

Q2: In figure 5, when changing the data size, are all terrain types kept in the dataset? how much data and online training does each method get at each expert count?

QA2: Yes. Panel (a) keeps all 5 terrain types and varies total transitions from 15M to 75M; at 15M each expert contributes 3M, at 75M each contributes 15M. Panel (b) fixes each expert at 15M and varies the number of terrain types from 1 to 5; total transitions grow with the expert count. At each point both methods receive the same datasets and the same online training budget.

AuthorsOpenReview ↗

Kind Follow-up to Reviewer c2m4 Before the Discussion Deadline

Dear Reviewer c2m4,

Thank you again for your careful review and constructive questions. We have addressed your main concerns by clarifying the zero-shot evaluation protocol, explaining the planner ablation and its associated reward terms, and providing a detailed breakdown of the data budgets in Fig. 5. These clarifications and experimental settings have also been incorporated into the revised PDF, with all updates highlighted in blue. We kindly encourage you to revisit our responses and the revised manuscript, and would greatly appreciate your feedback on whether the concerns have been adequately addressed before the discussion closes.

Thank you for the reply and added experiments. These address part of my earlier concerns, but I still have some reservations about the overall contribution, especially about skill reuse. CLoSD already uses generated motion references to guide a unified controller without requiring a new reward design for each task. I see the work mainly as adapting that approach to terrain-aware locomotion. After a overall consideration, I prefer to keep my original rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global comment by Authors

We thank the reviewers and the Area Chair for their careful reading and constructive feedback on this submission. The reviews recognize several strengths that motivated this work:

  • A practical goal: one controller that reuses terrain-specific skills across distinct terrains.
  • Strong hardware evidence: on Gap and Balancing Beam, DreamPolicy improves success over distillation (16/20 vs 8/20 and 17/20 vs 9/20), with an honest report of no significant difference on Slope and Stair.
  • A large-scale asset: 75M expert transitions, a modular terrain benchmark, and Isaac Gym/Isaac Lab support.
  • Promising zero-shot sim-to-real transfer to a Unitree G1, with clear gains on gaps and balancing beams.

We appreciate these assessments. The common questions concern scope, ablations, and positioning; our responses are summarized below.

Zero-shot and generalization

The paper uses zero-shot in two senses. For terrain, the unified policy is evaluated on terrains it did not train on. Expert RL trains one policy per primitive terrain; planner data are their rollouts; the unified policy trains online on the mixed terrain of the same five. The four unseen and five combined terrains are evaluation-only. For the real robot, zero-shot means no real-world fine-tuning. Wave is a periodic height profile outside the five primitives; no expert was trained on it, and DreamPolicy reaches 99.59% success. The new source-expert table shows the best expert at 37.42% on unseen and 31.82% on combined, against 92.77–99.59% and 84.78–95.79% for DreamPolicy. The claim is composition within the support of the expert data, not arbitrary new locomotion primitives.

Ablations and comparisons

The w/o-planner row keeps terrain observation. It removes the planner and the four tracking rewards defined on its predicted states, and uses offline AMP. It is a pathway ablation, not a blind policy: 0.08/20.14 on Slope Bridge/Balancing Beam against 92.77/96.35; the terrain-aware Joint PPO baseline reaches 31.24/41.36. The w/o-GCRL row is a direct-action diffusion policy; its failure is a training-paradigm gap, not a data gap. We also added a matched deterministic transformer, with the same backbone, input/output and data, and trained a policy on its predictions. It trails by 17–21 points on Slope Bridge/Balancing Beam, and is close on saturated terrains. Diffusion is better where several feasible futures must be represented; MSE regression averages them, and that average can be an infeasible reference.

Scaling, skill reuse, and positioning

Fig. 5(b) compares the two methods at the same expert count, with the same datasets and online budget. At five experts DreamPolicy reaches 96.35% against 74.36% for distillation; at three experts, 30.37% against 2.87%. The advantage remains large at every expert count. For skill reuse, the planner generates the reference and the controller tracks it without terrain-specific reward or observation; the same controller runs on unseen and combined terrains. The planner–tracker loop is prior work; our contribution is the terrain-conditioned state-space diffusion planner and the unified tracker trained on multi-terrain expert rollouts. PARC targets simulated characters; CLoSD and ReactiveBFM condition on text or a target position, not the local height map; DiffuseLoco generates actions directly; T-GMP is a training-time CVAE prior. Deployment details (N=5, H=15, 4 prediction windows, 50 Hz control, ≈20 Hz perception, ≈20 ms planner inference, asynchronous buffer) are in the response tables.

Revised PDF

We have updated the revised PDF as follows. Sec. 3 now states the training stages across the data-collection, planner and unified-policy subsections. Sec. 2 now distinguishes the CLoSD/ReactiveBFM conditioning and the PARC setting. Sec. 5.1.4 now defines the data budgets, and Tables 1–2 include the deterministic-predictor row. Appendix C.1 now documents the w/o-planner, w/o-GCRL and deterministic-predictor configurations. Appendix C.2 adds the source-expert evaluation with Tables A10–A11. Appendix C.4 now gives the deployment details. Appendix B.4 reports 4,096 environments, consistent with Sec. 4.1. These changes clarify the claims and add the requested evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.