- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces P-BFM, a perceptive behavioral foundation model for humanoid control that realizes shared behavioral intents across different terrains without terrain-matched demonstrations. Built on forward–backward representations, it maps motion references, target body states, and reward functions into a terrain-agnostic intent space. A depth-conditioned policy then adapts execution to the surrounding terrain. Explicit Root Instructions provide heading and planar-velocity targets to improve global motion precision. A single frozen policy supports motion tracking, goal reaching, reward inference, and teleoperation.
Critical strengths
-
The central design is clear and offers an element of novelty. Separating behavioral intent from terrain-conditioned execution allows the policy to preserve the intended behavior while adjusting step height, footholds, posture, and contacts. This decomposition is well aligned with the requirements of humanoid control across terrains.
-
Reducing reliance on terrain-matched demonstrations has practical value. The method uses flat-ground motions as behavioral prompts and motion priors, while learning terrain adaptation through autonomous interaction. This reduces the need to construct reference trajectories for individual motion–terrain combinations.
-
ERI is well motivated and supported by direct ablations. The paper identifies that behavioral latents may preserve local motion patterns without ensuring accurate heading or forward progress. Explicit root instructions provide a simple complement, and the corresponding ablations support their contribution.
-
The unified policy supports flexible task specification. Motion tracking, goal reaching, reward inference, and teleoperation share the same behavioral representation and execution policy without task-specific policy optimization.
Critical weaknesses
The ERI and perception ablations demonstrate the value of instructions and visual inputs, but do not directly establish whether excluding terrain information from the backward representation is important for the performance. A comparison in which only the backward representation’s access to terrain information changes would help isolate the benefit of terrain-agnostic intent encoding. This would strengthen the experimental support for decomposition of intent and terrain-conditioned execution.
Questions for authors
Can the contribution of terrain-agnostic intent encoding be evaluated directly? With the actor, perceptual inputs, ERI, and training settings held consistent, how would allowing the backward representation to receive terrain information affect motion fidelity and terrain adaptation?
2 comments
Thank you for your added experiments. This additional study on terrain-agnostic intent encoding has addressed my main concern. I'm raising my rating to 4.
Rebuttal: Benefits of Separating Intent Encoding from Terrain Perception
Thank you for your thoughtful feedback and for recognizing our motivation and pipeline design. The response and additional results are also available on the anonymous rebuttal page.
Benefits of Separating Intent Encoding from Terrain Perception
1. Better tracking accuracy and robustness. As shown in the table below, adding depth to the backward encoder B reduces motion-tracking accuracy at inference time and makes the policy less robust. We see two possible reasons.
2. Simpler deployment on real robots. P-BFM lets users specify tasks through rewards or intent latents. With a terrain-agnostic backward encoder B, we can compute the intent latent z offline or provide it asynchronously. A depth-conditioned encoder would need to infer z online from the current depth observation, adding latency and complicating deployment. Differences between simulated and real depth observations would also introduce an additional sim-to-real gap in intent encoding.
We hope these clarifications and additional experiments address your concerns. We will incorporate these clarifications and additional experimental results into the revised manuscript.