acceptodds
Under review as a conference paper at ICLR 2027

BGDL: Learning Whole-Body Humanoid Control from Demonstrations with Physical Action Geometry

Abstract

Demonstrations are useful for whole-body humanoid control, but their value is not uniform along a trajectory. In permissive states, many actions can still complete the task, whereas near foothold limits, balance boundaries, or required contacts, small deviations can change the outcome. We introduce **Boundary Guided Demonstration Learning (BGDL)**, which uses this variation in local action freedom to determine where demonstration guidance should be strongest. BGDL derives state-dependent guidance from physical margins, proximity to required interactions, and near-term failure risk, strengthening imitation where action choices are more restricted while leaving greater freedom for reinforcement learning elsewhere. We further characterize how allowable deviation from a demonstrated action depends on margin reserve, local action sensitivity, and margin-estimation error, giving a local sufficient condition for preserving a one-step physical-margin condition. We evaluate BGDL on five whole-body humanoid tasks: stair climbing, curb traversal, ground object pick-up, chair sitting, and object carrying, and transfer the learned policies to a Unitree G1 without policy fine-tuning. BGDL improves average success from 86.0% to 91.2% over reference tracking while reducing falls from 12.8% to 8.0%. With matched demonstration data, total imitation weight, and environment interactions, BGDL improves success by 3.4 percentage points over random assignment of the same guidance weights. This suggests that the placement of demonstration supervision matters beyond its total amount.

Primary area:
reinforcement learning
Keywords:
Humanoid robots, reinforcement learning, learning from demonstrations, whole-body control, loco-manipulation, physical action geometry

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YcqmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BGDL combines PPO with a demonstration loss whose weights depend on physical margins, required interactions, and predicted failure. It evaluates five humanoid tasks and includes a local theoretical analysis and Unitree G1 trials.

Critical strengths
  1. The paper addresses a useful problem: how to use demonstrations more effectively when precision requirements vary across a task. Giving more supervision to critical states is a reasonable design, and the guidance modules add no computation at deployment.

  2. The experiments provide credible evidence of the overall improvement. BGDL outperforms random weighting (91.2% vs. 87.8%) with the same total imitation weight, helping distinguish better supervision placement from simply using more imitation. Five training runs per task further strengthen this comparison.

Critical weaknesses
  1. Table 5 reports 91.2% success with all cues and 87.0% without the physical-margin cue, but does not state whether total imitation weight is matched. As reported, this comparison does not separate better state selection from a change in supervision strength. The 1.4-point gain over phase shuffling also lacks an interval for the difference, and the within-trajectory control described on page 29 has no clearly identified result. These gaps limit the evidence for the specific state-selection rule, despite the supported overall gain.

  2. Two descriptions need clarification. Section 3.1 and the supplied code score raw policy commands in the demonstration loss, while Appendix D.2 says executed actions are used. Figure 2(d) shows tabletop pick-up, whereas Section 4 lists ground pick-up and sitting among the evaluated tasks. The paper should specify the action representation used in the experiments and whether tabletop pick-up was evaluated. I treat both as minor reporting issues.

Questions for authors
  1. Could the authors clarify how the method performs when deployed on a real robot, particularly whether the gains observed in simulation carry over to real-world deployment?

  2. Was the tabletop pick-up in Figure 2(d) included in the reported experiments, or is it only an illustration? Please clarify its relation to the task list in Section 4.

  3. In Table 5, did the all-cue (91.2%) and event-plus-failure (87.0%) variants use the same total imitation weight and reward shaping? If not, could the authors report a comparison with these held fixed to isolate the physical-margin cue’s contribution?

1 comment
Area_Chair_rW7wOpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

Reviewer_Bo1cOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper argues that demonstration importance varies along a trajectory: near foothold limits, balance boundaries, or required contacts, the set of acceptable actions is smaller and demonstration guidance should be stronger. BGDL computes a guidance weight for each demonstration sample from physical margins, required-interaction distance, and a short-term failure predictor, then combines weighted imitation with PPO, physical costs, and potential shaping. Across five whole-body humanoid tasks, BGDL reaches 91.2% average success versus 86.0% for reference tracking and reduces falls from 12.8% to 8.0%. Under combined distribution shift, it reaches 79.4% success. On a Unitree G1 without policy fine-tuning, BGDL succeeds in 41 of 50 trials, compared with 36 of 50 for reference tracking.

Critical strengths
  1. The central intuition is physically meaningful and the theory is appropriately scoped. The paper connects local action freedom to demonstration weighting and gives a one-step sufficient condition involving margin reserve, local sensitivity, and estimation error. It explicitly states that this is not a full-trajectory safety guarantee.
  2. The matched-control design is persuasive. Uniform, random, and phase-shuffled controls preserve the demonstration data and the total imitation weight, yet perform worse than BGDL. This directly supports the claim that the placement of supervision matters.
Critical weaknesses
  1. The training signals are highly task-specific and rely on privileged information. Foothold, collision, balance, and required-event rules require detailed simulator state and manual task definitions. It is unclear how well the method transfers to unmodeled contact modes or open-ended tasks.
  2. The risk-channel ablation does not isolate the source of the gain. The failure predictor is used both for demonstration weighting and for rollout potential shaping. Removing the risk channel therefore cannot distinguish the effect of selective imitation from the effect of reward shaping or their interaction.
Questions for authors

See weakness

1 comment
Area_Chair_rW7wOpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

Reviewer_gAMfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BGDL combines PPO with weighted likelihood training on a fixed demonstration set. Each demonstration state receives the maximum of three gates based on normalized physical margins, proximity to active required interactions, and predicted near-term failure. The rollout objective additionally includes physical costs and a frozen shaping potential. The paper presents a conditional one-step action-deviation bound and evaluates five humanoid tasks using five training runs per task, matched-weight controls, restored-state action perturbations, and Unitree G1 trials.

Reported simulation success is 91.2%, compared with 86.0% for reference tracking, 87.8% for random matched weights, 89.0% for Q-filtered PPO, and 89.8% for phase-shuffled weights. Hardware success is 41/50 versus 36/50 for reference tracking. The proposed contribution concerns the allocation of demonstration supervision during training; deployment uses the actor without guidance networks or a runtime safety filter.

Critical strengths
  1. The paper asks a useful controlled question and includes relevant controls. Uniform, random, phase-shuffled, and temporally shifted weight assignments test whether supervision placement matters at fixed demonstration data and imitation weight. Table 12 supports a 3.4 percentage-point gain over random matching; independent resampling of the reported training runs gives an approximate 95% difference interval of [1.44, 5.36]. This is substantially more informative than comparing only against PPO or behavior-cloning initialization.

  2. The training design is coherent and the evaluation has meaningful breadth. The method separates desired interactions from physical violations, freezes the risk predictor during each update cycle, and explicitly handles incomplete outcome windows. Five tasks, predictor checks on restored demonstration states, and limited physical-robot trials support practical interest. The local theorem is valid under its stated assumptions, and the paper appropriately avoids presenting it as a guarantee of trajectory-level safety.

Critical weaknesses
  1. The evidence establishes selective supervision more clearly than the specific physical-action-geometry explanation. Demonstration-augmented policy updates and selective imitation are established ideas, including DAPG and Q-filtered behavior cloning. BGDL's incremental contribution therefore lies primarily in its choice and combination of gates. Theorem 2 is a valid local Lipschitz and estimation-error argument, but the implemented gate does not estimate the reference reserve, action sensitivity, or uniform error bound that determine its admissible radius. The paper acknowledges this limitation; the theorem motivates a heuristic rather than deriving its weighting rule.

The diagnostic evidence does not establish how much the physical and risk cues add after accounting for event proximity. Table 9 reports median correlations of 0.64 for the event gate and 0.69 for the full weight, versus 0.19 for the physical gate and 0.10 for risk. The closest control, phase shuffling, achieves 89.8% against BGDL's 91.2%. Recomputing from Table 12 under the stated independent run-resampling protocol gives an approximate 95% interval of [-0.56, 3.36] percentage points for this 1.4-point difference. This does not establish equivalence or zero effect, but leaves the incremental within-phase benefit uncertain.

  1. The action likelihood and controlled comparison protocol need a more precise specification. Section D.2 says action limits are applied after policy output and demonstration losses are computed in executed-action coordinates, while D.3 defines an ordinary Gaussian with learned variance. If executed actions are hard-clipped, their distribution has boundary masses; an ordinary Gaussian density at a clipped target is not that executed-action likelihood. Such a density may still be a deliberate behavior-cloning surrogate. Likewise, retaining pre-clipping actions for PPO can be valid. The paper needs to distinguish these cases, identify the action stored and scored in each loss and PPO ratio, and report saturation frequency. Appendix C.8 recognizes the theoretical caveat but does not resolve the implementation.

The matched controls preserve data, architecture, interaction counts, and weight statistics, which is a strength. However, Table 10 needs an explicit account of each selector's rollout reward, physical cost, shaping potential, predictor dataset, and scoring schedule. It is unclear whether adaptive weight vectors are shared from another run or recomputed under each control's collection policy, and how any shared scoring data are budgeted. Q-filtered PPO is a relevant baseline, but its separate Q-estimator targets, data coverage, fitting schedule, and tuning procedure are not specified in the PDF. The temporal-shift study explicitly holds rollout terms fixed; that clarity should extend to the principal selector comparison.

Questions for authors
  1. Action representation and likelihood. Does executed-action coordinates mean only common scaling, or post-clipping targets? Please specify the exact action stored and scored by imitation and PPO, clipping or squashing transformations, learned-variance handling, and saturation rates. If Gaussian likelihood is a surrogate rather than the executed distribution, please state this explicitly.

  2. Controlled objectives and baseline implementation. For each selector in Table 10, which rollout rewards, costs, potentials, predictor datasets, initialization, and scoring schedules are identical? How are adaptive weight vectors obtained and shared-data costs counted? Please give the Q-estimator targets, training data, update schedule, and validation/tuning protocol.

  3. Evidence for the geometry contribution. Please report a direct BGDL-minus-phase-shuffled difference interval and locate the separately reported within-trajectory control using twenty three-second trajectories in Section F.2. Can you provide an event-distance or phase-constant control with matched imitation weight and unchanged rollout objectives, and separate risk-in-imitation from risk-in-shaping? Separate physical probe rejection from event misses.

  4. Reproducibility and reporting. Please identify the configurations for task rewards, demonstration generation, physical normalizers, event targets/scales, phase rules, and perturbation scaling and continuation controller. How are Table 1's aggregate standard deviations obtained from Table 12? Please distinguish per-run T80 from the threshold crossing of the mean learning curve.

  5. Shifted and hardware evaluation. How are expanded test ranges sampled, and what performance is obtained on configurations strictly outside the training ranges? How were hardware checkpoints selected, trial order and initial conditions controlled, and terrain/object/chair geometry sensed? Are comparable phase-shuffled or Q-filtered hardware trials available? The existing hardware data support tested-condition transfer feasibility; a stronger comparative claim would require additional evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.