acceptodds
Under review as a conference paper at ICLR 2027

ECHO: In-Context Behavior foundation model for Humanoid Whole-Body Control

Abstract

Humanoid behavior foundation models (BFMs) acquire diverse skills through large-scale motion imitation and massively parallel reinforcement learning, yet broad behavioral capability does not ensure precise execution across physical conditions. Without distinguishing the underlying dynamics, policies may favor conservative compromises over environment-specific precision. We introduce ECHO, an in-context BFM that turns interaction experience into environment-specific control, separating general behavioral capability from dynamics-specific execution. Mixed and Annealed Sampling first establishes a behavior prior in a single pretraining stage by balancing targeted practice with broad motion coverage. Building on this prior, Context Representation and Residual Adaptation learns structured dynamics representations from interaction histories to guide residual corrections to the frozen prior, enabling adaptation without test-time parameter updates. Extensive experiments show that ECHO outperforms all baseline methods across diverse motions and physical conditions. Moreover, deployment on the Unitree G1 demonstrates whole-body teleoperation spanning extreme acrobatics and terrain interaction, alongside payload adaptation in manipulation and whole-body control. Page is available at https://echo-bfm.github.io/echoweb/https://echo-bfm.github.io/echoweb/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Humanoid Robot, Behaviour Foundation Model, In Context Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SYQwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ECHO combines a pretrained humanoid motion tracker with context-conditioned residual adaptation. It introduces MIAS for balancing broad motion coverage and difficult-segment practice, and CRA for learning dynamics representations from interaction histories. The framework is evaluated through simulation benchmarks, representation analyses, and real-world Unitree G1 demonstrations.

Critical strengths

The real-world demonstrations are impressive, covering highly dynamic motions, ground interactions, terrain traversal, and payload-conditioned behaviors with a shared control framework.

Separating broad behavioral capability from dynamics-specific execution is well motivated. The frozen-prior and residual-policy design provides a practical mechanism for adaptation without test-time parameter updates.

The evaluation includes tracking performance, adaptation baselines, representation analyses, and hardware experiments. Sharing the frozen backbone across adaptation baselines is a useful experimental control.

Critical weaknesses
  1. The connection between the strongest demonstrations and the central in-context learning contribution remains unclear. The broad terrain and acrobatic demonstrations in Figure 6 are produced by ECHO-PRE, before introducing CRA. These results establish the strength of the behavior prior, but provide limited evidence for the necessity of the proposed adaptation mechanism. Although CRA is also evaluated in simulation, its direct real-world validation is concentrated on payload changes.

  2. The contribution relative to existing BFMs is insufficiently established. The paper does not quantitatively compare with BFM, ScaleBFM, BFM-Zero, or Perceptive BFM. While these methods address different aspects of general-purpose control, a clearer comparison of capabilities, interfaces, and adaptation mechanisms is needed. The RMA and Any2Track comparisons are relevant to dynamics adaptation, but do not resolve the broader positioning of ECHO as a behavior foundation model.

  3. The evidence connecting structured context representations to improved control is incomplete. The representation ablations and context-length experiments primarily measure environment identification, retrieval, and physical-parameter readout. They do not directly establish how the proposed representation objectives or additional context improve downstream tracking. Adaptation speed, cold-start behavior, and sensitivity to outdated context are also insufficiently quantified. Consequently, the necessity and practical benefit of the specific in-context learning design remain difficult to assess.

Questions for authors
  1. Which demonstrated capabilities specifically depend on CRA? Could ECHO-PRE and the full ECHO model be evaluated on identical real-world motions and physical conditions to separate the contributions of the behavior prior and contextual adaptation?

  2. Could the authors report downstream tracking success and errors after removing dynamics prediction or physical-neighborhood alignment? How does the full method compare with direct history-conditioned control under matched history length, model capacity, and training budget?

  3. How does control performance improve as interaction context accumulates? What is the adaptation time from an empty history, and how is accumulated memory handled when a payload is attached or removed within a session?

  4. How does ECHO compare with BFM, ScaleBFM, BFM-Zero, and Perceptive BFM in terms of control interfaces and demonstrated capabilities? Could matched quantitative comparisons be provided where applicable, with explicit discussion of differences in scope and assumptions?

10 comments
AuthorsOpenReview ↗

Response to Reviewer SYQw for question Q1

Thank you very much for carefully reviewing our paper and raising insightful questions that help us improve our work.

Capability of CRA has been shown by ECHO-H. In fact, we have already demonstrated some capabilities of CRA in our original demos, such as precise manipulation under varying payloads. Additionally, Video S2 illustrates the process of adapting to payload changes. However, these demonstrations mainly focus on the capabilities of ECHO-H trained under HDR, without explicitly highlighting the differences between ECHO-PRE and ECHO-N, both of which are trained under NDR. We will further show the capability of ECHO-N comparing with ECHO-PRE.

Interactive simulation demonstration. CRA is not merely a model, but rather a method that enables BFMs to identify and adapt to different environments, thereby enhancing their adaptability to diverse environments, especially extremely challenging ones. ECHO-N exhibits greater robustness in certain extreme environments, for example, on a very smooth surface, ECHO-PRE fails to maintain balance, whereas ECHO-N remains stable. Furthermore, we observed that providing ECHO-N with dismatched context degrades its tracking performance and may even lead to failure. To demonstrate these capabilities, we developed an interactive web demo, available at ECHO_N_vs_ECHO_PRE.

Real world results. To further highlight the differences between ECHO-PRE and ECHO-N, we also conducted additional real-world experiments, as illustrated in Video S1. On a smooth floor, ECHO-PRE fails to maintain balance, whereas ECHO-N succeeds. Specifically, we teleoperated the G1 robot to walk on the smooth floor to collect matched context and on a rough surface (a floor mat) to collect mismatched context.

AuthorsOpenReview ↗

Table list (Table1-3)

We have added 300 highly dynamics motions on the original BONES-SEED-100-HighDynamic to compose BONES-SEED-400-HighDynamic, maintaining the same ratio: 1/3 terrain interaction, 1/3 extreme movements, and 1/3 dance behaviors. What's more, we added ScaleBFM, BFM-Zero and P-BFM as baselines. As illustrated in Table 1, ECHO-N achieved the best overall performance.

Table 1

Motion-tracking evaluation on BONES-SEED-400-HighDynamic. Results are reported as mean ±\pm sample standard deviation over three independent action-noise seeds with σ=0.01\sigma=0.01. Bold and underlined values indicate the best and second-best means within each environment, respectively.

Nominal Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
SONIC v1.10.8750 ± 0.0043383.362 ± 3.34443.706 ± 0.5193.316 ± 0.0118.869 ± 0.027
HEFT0.8833 ± 0.0029270.710 ± 1.60839.906 ± 0.0512.929 ± 0.0167.133 ± 0.027
SCALEBFM-XL0.9275 ± 0.0050249.122 ± 0.96838.601‾\underline{38.601} ± 0.0362.657 ± 0.0026.329 ± 0.010
BFM-ZERO0.2858 ± 0.0151813.942 ± 15.446152.798 ± 1.2873.006 ± 0.05313.555 ± 0.189
P-BFM0.2475 ± 0.0090653.343 ± 20.239126.544 ± 4.3894.576 ± 0.02013.536 ± 0.229
ECHO-PRE0.9200 ± 0.0000218.547 ± 1.82939.082 ± 0.1092.635‾\underline{2.635} ± 0.0106.827 ± 0.017
ECHO-N0.9242‾\underline{0.9242} ± 0.0038219.735‾\underline{219.735} ± 1.38538.118 ± 0.0682.594 ± 0.0086.720‾\underline{6.720} ± 0.018

NDR Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
SONIC v1.10.8417 ± 0.0029416.569 ± 4.22045.345 ± 0.1783.475 ± 0.0039.341 ± 0.015
HEFT0.8633 ± 0.0038339.039 ± 3.28242.549 ± 0.0743.047 ± 0.0117.580 ± 0.018
SCALEBFM-XL0.8975 ± 0.0025279.684 ± 4.90841.005‾\underline{41.005} ± 0.1272.781 ± 0.0166.744 ± 0.046
BFM-ZERO0.2492 ± 0.0063870.781 ± 12.997159.128 ± 3.7252.903 ± 0.04513.160 ± 0.247
P-BFM0.1900 ± 0.0050603.520 ± 13.907127.483 ± 7.2004.749 ± 0.05813.851 ± 0.305
ECHO-PRE0.9075‾\underline{0.9075} ± 0.0000245.557‾\underline{245.557} ± 1.77541.956 ± 0.0392.714‾\underline{2.714} ± 0.0017.188 ± 0.006
ECHO-N0.9167 ± 0.0063236.017 ± 1.41040.342 ± 0.0622.655 ± 0.0046.988‾\underline{6.988} ± 0.020

Table 2

Comparison of and ECHO and ECHO-E2E. ECHO-E2E and ECHO share the same policy architecture, history capacity and training budget.

MetricECHO-E2EECHO
Policy parameters19.150 M19.150 M
Training budget (environment steps)3,932,160,0003,932,160,000
History capacity (steps)350 (7 s at 50 Hz)350 (7 s at 50 Hz)

Table 3

Motion-tracking evaluation on LAFAN40. Results are reported as mean ±\pm sample standard deviation over three seeds, with action-noise standard deviation σ=0.01\sigma=0.01. Bold values indicate the best means within each environment.

Nominal Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
ECHO-E2E0.4083 ± 0.01445299.772 ± 181.732168.213 ± 4.0663.262 ± 0.00619.831 ± 0.091
ECHO-H0.5667 ± 0.01442907.805 ± 43.40330.969 ± 0.0342.801 ± 0.0017.410 ± 0.008

HDR Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
ECHO-E2E0.4021 ± 0.00725586.404 ± 151.949180.759 ± 3.0803.322 ± 0.07419.845 ± 1.108
ECHO-H0.4646 ± 0.00953394.704 ± 160.67139.016 ± 0.7692.986 ± 0.0338.761 ± 0.169
AuthorsOpenReview ↗

Table list (Table4-5)

Table 4

Simulation evaluation of BFMs' in-context ability. ECHO-H effectively exploits interaction context to improve tracking under heavily randomized dynamics (HDR). Bold and underlined values indicate the best and second-best results within each environment, respectively.

Nominal Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
Vanilla0.8892 ± 0.0003194.541 ± 1.58527.286 ± 0.0160.881 ± 0.0082.887 ± 0.006
Any2Track0.8743 ± 0.0008193.623 ± 0.73827.846 ± 0.0020.873 ± 0.0082.883 ± 0.005
RMA-Teacher0.9798‾\underline{0.9798} ± 0.0003189.908 ± 0.22024.062‾\underline{24.062} ± 0.0100.849‾\underline{0.849} ± 0.0072.772‾\underline{2.772} ± 0.004
RMA-Student0.9763 ± 0.0008190.329 ± 0.59024.026 ± 0.0070.851 ± 0.0082.774 ± 0.005
ECHO-PO0.9823 ± 0.0010194.255 ± 0.34425.125 ± 0.0040.90597 ± 0.000082.898 ± 0.001
ECHO-SO0.9792 ± 0.0010202.901 ± 0.91425.299 ± 0.0150.91173 ± 0.000222.934 ± 0.001
ECHO-H0.9748 ± 0.0010190.088‾\underline{190.088} ± 0.71824.496 ± 0.0140.846 ± 0.0082.760 ± 0.006

HDR Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
Vanilla0.8632 ± 0.0008230.001 ± 0.83331.541 ± 0.0181.088 ± 0.0033.701 ± 0.004
Any2Track0.8455 ± 0.0009227.549 ± 0.99931.939 ± 0.0121.087 ± 0.0033.709 ± 0.003
RMA-Teacher0.8827‾\underline{0.8827} ± 0.0021216.826 ± 1.46230.063 ± 0.0171.052‾\underline{1.052} ± 0.0023.591‾\underline{3.591} ± 0.002
RMA-Student0.8700 ± 0.0009224.568 ± 0.89430.542 ± 0.0141.065 ± 0.0023.648 ± 0.001
ECHO-PO0.8718 ± 0.0080216.716‾\underline{216.716} ± 1.78930.466 ± 0.1161.056 ± 0.0053.612 ± 0.025
ECHO-SO0.8653 ± 0.0063221.908 ± 2.86530.266‾\underline{30.266} ± 0.0121.067 ± 0.0043.648 ± 0.017
ECHO-H0.8922 ± 0.0016210.351 ± 1.01730.463 ± 0.0081.039 ± 0.0033.530 ± 0.003

Table 5

Motion-tracking evaluation on AMASS-2000-Random. Results are reported as mean ±\pm sample standard deviation over three runs with action-noise standard deviation σ=0.01\sigma=0.01. Bold and underlined values indicate the best and second-best means within each environment, respectively.

Nominal Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
SONIC v1.10.9633 ± 0.0003316.651 ± 1.64930.411 ± 0.0151.135 ± 0.0023.770 ± 0.006
HEFT0.9785 ± 0.0009261.573 ± 2.35228.524 ± 0.0091.062 ± 0.0013.103 ± 0.004
SCALEBFM-XL0.9838 ± 0.0003178.604 ± 0.62231.179 ± 0.0110.979 ± 0.0052.753 ± 0.009
BFM-ZERO0.6642 ± 0.0013710.468 ± 7.30488.765 ± 0.2311.397 ± 0.0027.179 ± 0.027
P-BFM0.8485 ± 0.0031631.253 ± 3.67689.705 ± 0.3231.620 ± 0.0045.944 ± 0.006
ECHO-PRE0.9832‾\underline{0.9832} ± 0.0012207.995 ± 0.97026.595‾\underline{26.595} ± 0.0070.932‾\underline{0.932} ± 0.0033.036 ± 0.006
ECHO-N0.9827 ± 0.0003203.149‾\underline{203.149} ± 0.28925.577 ± 0.0240.917 ± 0.0032.956‾\underline{2.956} ± 0.007

NDR Environment

MethodSR ↑Eg-MPJPEE_{\mathrm{g\text{-}MPJPE}} ↓EMPJPEE_{\mathrm{MPJPE}} ↓EaccE_{\mathrm{acc}} ↓EvelE_{\mathrm{vel}} ↓
SONIC v1.10.9545 ± 0.0022343.709 ± 6.53331.410 ± 0.0561.196 ± 0.0023.965 ± 0.004
HEFT0.9738 ± 0.0008326.887 ± 10.85329.882 ± 0.1571.123 ± 0.0073.351 ± 0.032
SCALEBFM-XL0.9787‾\underline{0.9787} ± 0.0029245.105 ± 4.94932.691 ± 0.2251.061 ± 0.0043.062 ± 0.008
BFM-ZERO0.6503 ± 0.0029818.227 ± 18.51596.278 ± 0.4261.512 ± 0.0097.680 ± 0.057
P-BFM0.8078 ± 0.0051651.868 ± 12.30393.229 ± 0.7541.778 ± 0.0146.233 ± 0.083
ECHO-PRE0.9775 ± 0.0023222.062‾\underline{222.062} ± 2.19527.780‾\underline{27.780} ± 0.0670.946‾\underline{0.946} ± 0.0043.149 ± 0.012
ECHO-N0.9803 ± 0.0012214.227 ± 0.46227.232 ± 0.0900.943 ± 0.0033.093‾\underline{3.093} ± 0.006
Reviewer_qruxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ECHO, a framework for humanoid whole-body control that uses interaction history to infer environment-specific dynamics and guide action corrections. The approach combines two components: Mixed and Annealed Sampling (MIAS), which learns a broad motion-tracking prior, and Context Representation and Residual Adaptation (CRA), which learns physically informative context representations and uses them to condition a residual policy. The resulting controller adapts its execution to different physical conditions while keeping the pretrained behavior prior fixed and requiring no parameter updates at deployment. The paper evaluates motion tracking, dynamics adaptation, and context representations in simulation, alongside real-world demonstrations on the Unitree G1.

Critical strengths

Interesting and conceptually sound use of interaction context. Using interaction history to generate environment-specific action corrections is an appealing approach to dynamics adaptation. The separation between a shared behavior prior and context-conditioned residual corrections provides a clever way to accommodate different physical conditions while retaining the capabilities of the pretrained controller.

Supportive experiments and compelling demonstrations. The simulation experiments and real-world video demonstrations provide convincing evidence of the approach’s effectiveness. The demonstrations cover diverse whole-body behaviors and payload conditions, while the quantitative comparisons support improvements in tracking and adaptation. The context-representation analysis is also useful for examining whether the learned latent representation captures information about the underlying physical environment.

Critical weaknesses
  1. The training procedure for CRA needs a clearer explanation. Although Section 3.3 and Appendix A.1 provide the mathematical objectives, the paper would benefit from a more accessible explanation of how the context encoder and dynamics predictor are trained. In particular, the main text should introduce the meaning of the neighborhood distributions (P) and (Q) more intuitively. The authors should explain how minimizing the KL divergence between these distributions encourages the context representation to reflect physical conditions consistently across different motions, and how this supervision complements the dynamics-prediction objective.

  2. The residual-policy training procedure also requires clarification. Section 4.1 identifies PPO and the NDR/HDR settings, but the connection between these settings and the residual-training stage could be more explicit. What reward and optimization objective are used for the residual policy? Does this stage reuse the tracking rewards and MIAS sampling procedure from pretraining? How are reference motions, initial states, and physical configurations sampled? A concise training algorithm or a description of the data flow through each stage would make the method easier to understand and reproduce.

  3. Figure 3 needs a more explicit interpretation. The caption identifies colors with environments, but the discussion should explain more clearly how the observed clusters support the intended claim. In particular, readers should understand whether histories from different motions under the same physical configuration remain grouped together, while histories from different configurations remain distinguishable. The relevant evidence is this consistency within environments and separation between environments. A better overall spread of points in the two-dimensional visualization alone does not establish a better representation. Connecting the visual patterns more directly to the retrieval and silhouette results would make the interpretation clearer.

  4. The high-dynamic motion benchmark is relatively small. While AMASS-2000-Random provides a larger evaluation of everyday motions, BONES-SEED-100-HighDynamic contains only 100 motions for assessing challenging behaviors. Given the breadth of the capabilities demonstrated by the system, I would be interested in a larger evaluation covering more high-dynamic motions. This would help establish how consistently the reported improvements hold across extreme movements, ground interactions, and dance behaviors.

I would be inclined to raise my score if the authors address these concerns.

Questions for authors

See weakness.

6 comments
AuthorsOpenReview ↗

Response to Reviewer qrux for weakness Q1

We sincerely thank you for your careful review and valuable feedback. We will explain more about the training procedure of CRA.

(1) Encoder and predictor training.

We jointly train the context encoder and forward predictor on rollouts collected by the frozen ECHO-PRE tracker. As illustrated in Figure 2, the encoder maps completed interaction histories to a context representation ztz_t. Conditioned on ztz_t and the recorded actions, the predictor estimates future states over multiple steps. Both networks are optimized using the dynamics-prediction and neighborhood-alignment objectives.

(2) Neighborhood alignment and its role.

For an anchor environment ii, QijQ_{ij} assigns a neighborhood probability based on physical-parameter similarity, whereas PijP_{ij} assigns one based on the similarity between the current context from ii and an archived context from jj. Crucially, PiiP_{ii} compares nonoverlapping histories from different reference motions under the same physical configuration. Minimizing KL(Qi:∣Pi:)\mathrm{KL}(Q_{i:}\\|P_{i:}) therefore encourages consistency across motions while preserving graded similarities between physical configurations. This complements dynamics prediction because accurate prediction alone does not require histories from different motions under the same physics to have consistent representations; the KL loss supplies this additional constraint.

AuthorsOpenReview ↗

Response to Reviewer qrux for weakness Q2

We detail the training pipeline, residual-policy objective, and sampling and reset procedures below and will incorporate these clarifications into the final version of the manuscript.

(1) Training flow.

We first pretrain ECHO-PRE with MIAS, then jointly train the context encoder and forward predictor on rollouts from the frozen prior. During residual training, we freeze ECHO-PRE and the context encoder and train the residual actor and critic with PPO. The forward predictor is not used in this stage. The applied action is at=atprior+atΔa_t=a_t^{\mathrm{prior}}+a_t^{\Delta}.

(2) Reward and objective.

The residual stage reuses ECHO-PRE's motion-tracking reward and termination rules, without introducing a residual-specific reward. PPO optimizes the expected discounted tracking return under the combined action ata_t.

(3) Sampling and resets.

The residual stage reuses MIAS to sample reference motions and starting frames. At each reset, the robot is initialized from the reference state at the selected frame. Physical configurations are sampled from the NDR ranges for ECHO-N and the HDR ranges for ECHO-H, respectively. Each configuration remains fixed within a physical session across reference-motion changes and episode resets.

AuthorsOpenReview ↗

Response to Reviewer qrux for weakness Q3

(1) Interpretation of Figure 3.

Each color denotes one physical configuration, with points representing histories from different reference motions. ECHO groups histories from the same configuration across motions while keeping different configurations distinguishable; Any2Track shows greater overlap between configurations. We will clarify that the relevant evidence is this within-configuration consistency and between-configuration separation, rather than the overall spread of the t-SNE points.

(2) Quantitative support.

We have conducted an additional representation evaluation of Any2Track to complement the original results in Table 4. Any2Track achieves 18.5±0.218.5\pm0.2\\% cross-motion retrieval mAP and a silhouette score of −0.0271±0.0023-0.0271\pm0.0023, compared with 93.872±1.45093.872\pm1.450\\% and 0.4642±0.01050.4642\pm0.0105 for ECHO, respectively. Retrieval mAP evaluates whether histories from the same configuration remain close across different motions, while silhouette measures within-configuration cohesion relative to between-configuration separation. These newly added results quantitatively support ECHO's stronger cross-motion consistency and clearer configuration separation, corroborating the visual patterns in Figure 3.

Reviewer_YWe1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for humanoid robot controller that takes in history state-actions as context to estimate the physical parameters of the current environments (e.g. payloads) and leverages a residual adaption module to refine the original controller commands with the context considered. The results show improved tracking performance on high domain randomization scenarios and the authors also show that the learned contexts could be correlated to the original environment (i.e., helpful for environment identification).

Critical strengths
  • The problem of identifying physical environments for improving controller behaviors is an important problem to study. The overall tracking performance is good compared to existing baselines.

  • The overall derivation of method and design is clearly illustrated and makes sense. The designed linear environment identification method also shows that the learned context links learned contexts to physical environments

Critical weaknesses
  • The major concern about this paper is the adaptivity of the controller when environment physical parameters changes during a session (e.g. payload changes), as the current model consider both a session-wise memory and a episode-wise history. Such a design is also not clearly ablated since I assume it's the major formulation constraint that limited the adaptivity potential, this makes the proposal of such a design somewhat confusing.

  • Another concern is on the details of soft-alignment loss definition as the current one only describes a conceptual dijd_{ij} for one physical parameter, leaving the combination of parameters as well as what is considered untouched. For example, for payload added to different joints, how is this aligned? Moreover, the current draft needs better listing of all physical parameters considered and their DR application scenarios, from Tab.7 only payload-related is different between HDR and NDR, and friction is the same, but from the experiments, friction is also treated as an important evaluation aspects.

Questions for authors

See the weakness section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer YWe1 for weakness Q1

We thank the reviewer for raising this important question about the rationale for session-wise memory and its implications for adaptation. We address this concern from the following five perspectives.

(1) Adaptation trade-off and practical relevance.
We acknowledge that retaining session-wise context can delay adaptation to abrupt changes in dynamics. Nevertheless, this design remains useful when a robot can gather interactions before demanding tasks. For example, after a payload modification or a change in terrain, simple exploratory behaviors such as walking or arm swinging can provide evidence about the new dynamics before precise or highly dynamic execution.

(2) Motivation for session-wise memory.
The purpose of session-wise memory is to reduce dependence on individual motion clips and support a motion-agnostic representation of the underlying dynamics. Different behaviors reveal complementary physical properties: standing provides limited information about ground friction, whereas walking can make its effects more observable. Aggregating interactions across motions therefore provides richer evidence for dynamics identification.

(3) Evidence from the existing context-length ablation.
Figure 4 in the original submission, evaluates the effect of context length. With a context length of 50, the encoder effectively relies on short, episode-scale history. The improvements obtained with longer context support the benefit of retaining additional interaction evidence across motions, motivating the session-wise design.

(4) Memory updates under changing dynamics.
Session-wise memory is continuously updated through a rolling history. Adding or removing a payload does not clear the memory; instead, new interactions progressively replace older ones. The transition therefore contains evidence from both the previous and current dynamics. As post-change interactions accumulate, the context increasingly reflects the current physical conditions, allowing the policy to adjust accordingly.

(5) Interactive and real-world demonstrations.
We provide an interactive demo, ECHO-H: Changing Payloads, where users can select different motions and add or remove payloads during execution. The accompanying latent visualization shows how the context representation evolves following these changes. We also provide real-world experiments in Video S2: while tracking a T-pose reference, the robot gradually adjusts after a payload is added and recovers over several seconds after its removal. These demonstrations illustrate both the controller's ability to adapt and the transient delay associated with updating its interaction history.

AuthorsOpenReview ↗

Response to Reviewer YWe1 for weakness Q2

We thank the reviewer for pointing out this ambiguity. We clarify how physical parameters are combined and how payload attachment locations are distinguished below, and will include these implementation details in the revised manuscript.

Combining physical parameters.

Each environment ii is represented by a physical parameter vector θi∈R108\boldsymbol{\theta}_i \in \mathbb{R}^{108}. Its coordinates are partitioned into 1616 equally weighted factors, indexed by g∈1,…,16g \in \\{1,\ldots,16\\}. For each factor gg, let Gg\mathcal{G}_g denote the set of coordinate indices belonging to that factor. For example, the joint stiffness factor contains 29 coordinates, whereas each payload mass factor contains one. The grouping is summarized below:

Physical parametersDimensionsFactors
Torso COM offsets (x,y,zx,y,z)33
Torso relative mass and shared foot friction22
Joint KpK_p, KdK_d, and armature scales3×293 \times 293
Payload masses at four attachment sites44
Payload COM offsets at four attachment sites4×34 \times 34
Total10816

Let θi,k\theta_{i,k} denote the kk-th coordinate of the physical parameter vector for environment ii, where k∈1,…,108k \in \\{1,\ldots,108\\}. Each coordinate corresponds to a specific physical quantity, such as the left-hand payload mass or the stiffness scale of a particular joint. We normalize each coordinate using its fixed randomization bounds:

xi,k=θi,k−ℓkuk−ℓkx_{i,k} = \frac{\theta_{i,k} - \ell_k}{u_k - \ell_k}

where ℓk\ell_k and uku_k denote the lower and upper sampling bounds. Let Gg\mathcal{G}_g be the coordinate set of factor gg. The distance used in soft alignment is

dij=[116∑g=1161∣Gg∣∑k∈Gg(xi,k−xj,k)2]1/2.d_{ij} = \left[ \frac{1}{16} \sum_{g=1}^{16} \frac{1}{\lvert \mathcal{G}_g \rvert} \sum_{k \in \mathcal{G}_g} \left(x_{i,k} - x_{j,k}\right)^2 \right]^{1/2}.

Thus, squared normalized differences are averaged within each factor, and the resulting factor contributions are averaged equally. This prevents a factor containing many coordinates, such as the 29 joint stiffness scales, from dominating solely due to its dimensionality.

Distinguishing payload attachment locations.

Payload masses are represented separately for each attachment site. For example, placing the same payload on the left hand versus the right hand changes both hand-specific mass coordinates, yielding a nonzero dijd_{ij} despite identical total payload mass. Soft alignment therefore accounts for both the payload's physical properties and its attachment location.

Also, we thank the reviewer for highlighting the need to clarify the DR settings.

Distinction between NDR and HDR. The primary difference between NDR and HDR lies in payload randomization. Keeping the friction range consistent makes the additional challenge introduced by payload variation easier to assess.

Role of friction randomization. In both settings, friction is independently sampled within [0.3, 2.0]. This range already produces substantial changes in foot–ground interaction and humanoid behavior, motivating our dedicated friction evaluation. Using the same bounds across NDR and HDR therefore preserves a shared but varied set of contact conditions; it does not imply that friction is fixed or unimportant.

Dear Reviewer YWe1,

Thank you again for your time and feedback. As the discussion period is nearing its end, we wanted to briefly follow up on our responses and additional experiments addressing your concerns.

We understand that this is a busy period, and we would appreciate hearing whether our responses have clarified the concerns you raised. If any concerns remain, we would be happy to address them.

Thank you for your consideration.

Best regards, The ECHO Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

A Friendly Follow-up on Our Responses

Dear Reviewers,

Thank you for your time and constructive feedback. We have responded to all the concerns raised in your reviews and provided additional experiments and clarifications.

As the discussion period draws to a close on November 18 (AoE), we would appreciate any further feedback on whether our responses address your concerns. If any questions remain, we would be happy to clarify them during the remaining discussion period.

To make our results easier to explore, we have also provided two interactive demos and supplementary videos (Click directly):

  • ECHO-N vs. ECHO-PRE: Compare the policies under different physical conditions and explore the effects of mismatched context.
  • ECHO-H: Changing Payloads: Add or remove payloads during execution and observe how the policy adapts.
  • Supplementary Videos: Real-world comparisons on a low-friction surface (Video S1) and adaptation to payload changes (Video S2).

Thank you again for your consideration and time.

Best regards, The ECHO Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.