acceptodds
Under review as a conference paper at ICLR 2027

Ada-BFM: Adapting Behavior Foundation Models for Humanoid Scene Interaction

Abstract

Behavior Foundation Models (BFMs) have emerged as a promising paradigm for general-purpose humanoid control through pretraining on large-scale behavioral data. However, their proficiency in whole-body coordination does not readily extend to scene interaction, which requires adapting motion to scene-dependent geometric constraints and contact dynamics. To bridge this gap, we introduce Ada-BFM, a post-training framework that adapts pretrained BFMs for humanoid scene interaction. Ada-BFM augments the original architecture with a lightweight adapter, enabling interaction-specific adjustments while leveraging pretrained control capabilities. Rather than producing a separate controller for each task, the framework yields a shared model for each interaction domain, supporting diverse tasks involving terrain or object interaction. Extensive experiments in simulation and the real world demonstrate that Ada-BFM enables diverse scene interaction behaviors, accelerates learning compared with training from scratch, and largely preserves pretrained behavioral competence. These findings establish Ada-BFM as an effective and efficient approach to extending the capabilities of BFMs beyond their pretraining settings while retaining their strengths in general whole-body control.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Behavior Foundation Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RShtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes Ada-BFM, a framework for adapting pretrained behavior foundation models to humanoid scene interaction. The method freezes the pretrained ScaleBFM and introduces depth-conditioned adapters trained with reinforcement learning. It enables the robot to perform various terrain and object interaction tasks while largely preserving its original motion-tracking capabilities.

Critical strengths

See "Questions For Authors".

Critical weaknesses

See "Questions For Authors".

Questions for authors
  1. The proposed method is evaluated exclusively on ScaleBFM, and its adapter design is closely tied to the Transformer's architecture. Although the authors position Ada-BFM as a general framework for adapting BFMs, it remains unclear whether the approach can be effectively transferred to other BFM architectures, such as SONIC. Evaluating Ada-BFM on an additional pretrained BFM would provide stronger evidence for its claimed generality.
  2. If I understand correctly, Ada-BFM still relies on reference motions as control inputs for terrain interaction tasks such as stair climbing and obstacle traversal. Why is this design adopted? If the actual terrain differs from the reference motion (e.g., different stair heights or obstacle positions), how well can the model adapt to these variations? I believe this input design has certain limitations, and it would be more practical if the robot could perform these tasks using simple directional commands, such as moving forward, backward, left, and right.
  3. After watching the provided videos, I was particularly interested in the G1's ability to pick up various objects. However, the paper provides limited details about how many different objects were used during training, whether the test objects were seen during training, and what the reference motions look like. Since the method still relies on reference motions to perform these tasks, I consider this dependency a significant limitation. Therefore, I am more inclined to view this work as primarily focusing on sim-to-real transfer rather than learning generalizable object manipulation capabilities.
  4. I did not find any experiments investigating how changing the reference motions affects the results. I believe this is an important aspect that should be evaluated.
Reviewer_HHJkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Ada-BFM which adapts a pretrained ScaleBFM humanoid controller to scene interaction by adding depth-conditioned gated adapters into the backbone while freezing other parameters. Training combines PPO with an action-preservation loss. It produces two separately adapted models for different purpose, one for terrain interaction and one for object interaction, each supporting multiple behaviors conditioned on reference motions. Evaluation on simulation comparisons, learning-efficiency and motion-retention tests, and qualitative demonstrations on a real G1 robot shows the method has merit.

Critical strengths

The robot performs terrain traversal and object interactions, including lifting varied objects and carrying while turning or jogging. Repeated executions and motions labeled unseen during adaptation provide useful evidence of practical transfer. Sharing a policy across multiple behaviors within each domain strengthens the system contribution.

The evaluation on adaptation and retention is interesting. Evaluating retained competence with adapters enabled addresses an important question: whether new interaction skills can be acquired while preserving general motion control.

Critical weaknesses

The method mainly combines established ideas: a frozen backbone, gated cross-attention adapters, zero initialization, and preservation regularization and the independent contribution of depth perception remains unclear. Future reference motions already provide a strong information for executing each interaction. It might be more informative to include controller receiving identical task-specific training and reference inputs but no depth observations. Thus, the results leave open how much improvement requires real-time perception versus learning to execute the supplied references.

Faster learning than training from scratch demonstrates useful initialization. But it provides less insight into whether broad pretrained behavioral knowledge enables capabilities that a narrower pretrained controller could not support in scene interaction. On the other hand, freezing the prior may impose a ceiling on acquiring new skills. Object interaction requires coordination absent from motion-only pretraining, and the weaker object results raise a question about whether the framework can accommodate increasingly different interaction dynamics, beyond behaviors close to the pretrained data.

Questions for authors

How are reference motions selected and aligned to the physical scene during deployment? Does this require manual alignment or ground-truth terrain/object information?

Does the actor receive reference object trajectories, or only robot-pose targets? Section 5 describes references containing object motion, while the policy interface specifies whole-body poses.

Reviewer_XqbnOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Ada-BFM, a post-training framework for extending pretrained humanoid Behavior Foundation Models (BFMs) to scene-aware physical interaction. The central idea is to reuse the general whole-body control capabilities acquired during pretraining while introducing a lightweight adaptation pathway for perception-conditioned interaction. Specifically, Ada-BFM freezes the pretrained BFM backbone and incorporates depth observations through gated residual adapters, optimized with reinforcement learning and a behavior-preservation objective.

The framework is evaluated on two interaction domains, terrain interaction and object interaction, with a single adapted model supporting multiple behaviors within each domain. The experiments compare Ada-BFM against several adaptation strategies and existing interaction methods, examining interaction success, learning efficiency, and retention of pretrained capabilities. Results demonstrate improved interaction performance, faster learning compared with training from scratch, and substantial preservation of the original model's general motion capabilities.

Critical strengths

1.The problem addressed in this pape, how to adapt pretrained humanoid control models to new domains is both interesting and important. The authors propose a simple yet effective solution to this problem.

2.The proposed adapters are lightweight while achieving strong performance, providing a promising foundation for future extensions and practical applications.

3.The experimental results demonstrate that the proposed approach can effectively leverage pretrained knowledge to acquire new interaction capabilities while largely preserving the original control capabilities of the BFM. In particular, Ada-BFM achieves strong performance on downstream interaction tasks with only a lightweight adaptation pathway, while significantly reducing the degradation of pretrained capabilities compared with conventional fine-tuning methods. This highlights the benefits of decoupling pretrained knowledge from downstream adaptation and provides compelling evidence for the effectiveness of the proposed framework.

Critical weaknesses

1.Although the experimental results demonstrate the effectiveness of Ada-BFM, the paper would benefit from a deeper analysis of why the proposed method outperforms existing adaptation approaches. In particular, more detailed component ablations could help clarify the individual contributions of the gated cross-attention mechanism, the feed-forward branch, and the behavior-preservation loss. Such analyses would provide a better understanding of the underlying mechanisms responsible for the performance improvements and strengthen the technical contribution of the paper.

2.Although the paper presents an effective solution for domain-specific post-training, it remains unclear whether the proposed framework supports compositional generalization across domains. For example, in a task requiring a humanoid to carry a box while climbing stairs, would joint training be necessary, or could independently trained terrain-interaction and object-interaction adapters be combined in a plug-and-play manner? Nevertheless, I consider this an interesting direction for future research rather than a fundamental limitation of the current work, and it does not affect my overall recommendation.

Questions for authors

1.Could the authors provide further analysis or insights into why Ada-BFM outperforms existing adaptation methods? In particular, what are the key factors contributing to its superior performance?

2.Could the authors investigate the scalability of the proposed framework as the number and diversity of tasks within an interaction domain increase? In particular, can a single domain-specific adapter maintain its performance when trained on a larger and more diverse set of interaction tasks, or does interference between tasks emerge? Such an analysis would help assess the scalability of the proposed one-adapter-per-domain design.

Comments on the paper

1 comment

Dear Reviewers,

As the discussion period is coming to a close, please take a moment to review the authors’ responses and revised manuscript. Kindly assess whether your concerns have been adequately addressed or if any questions remain, and update your scores accordingly, if necessary.

Thank you for your time and thoughtful evaluations.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.