acceptodds
Withdrawn from ICLR 2027

ECA: REWARD-FREE ONLINE EFFERENCE-COPY ADAPTATION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learned policies are widely used to control autonomous systems, but actuator ordering, direction, or gain errors can exceed the tolerance of their feedback. Identifying and inverting the actuator map restores performance; applying an estimated inverse when no such map is present can instead degrade working feedback. We propose Efference-Copy Adaptation (ECA), which augments an identification-and-inversion core with static-map and velocity-coupled hypotheses, a rejection gate, a unit amplification bound, and an identifier hold. ECA acts under the best-supported hypothesis, withdraws when the identity or a delayed identity explains recent transitions better, decides only on informative data, and never amplifies the command it corrects. On two MuJoCo manipulators, it turns the seven-actuator permutation and the orthogonal mix, on which identification alone loses and return, into gains of and . It removes the delay losses of and return that identification causes, and converts a velocity-coupled disturbance that a map-only gate mis-fits by return into a gain of . No conservative identification control in a full grid attains ECA's recovery with a smaller worst mean loss over the four recorded preservation conditions. On a cohort trained after the method was frozen and evaluated once, recovery is met on both plants; a return loss under gain loss leaves preservation unmet on one. On a contact-rich third plant, recovery transfers but preservation does not, and a healthy-body measurement predicts the difference.

Primary area:
applications to robotics, autonomy, planning
Keywords:
online adaptation, test-time adaptation, reward-free adaptation, efference copy, forward model, feedback error learning, adaptive control, fault-tolerant control, robot learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DXViOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ECA adapts a frozen control policy without online rewards by identifying actuator transformations from a learned nominal model's prediction errors. It combines static-map and velocity-coupled hypotheses, rejection against identity and delayed-identity models, bounded inversion, and an estimator hold. MuJoCo experiments separate recovery from actuator faults from interference with otherwise functioning policies, with detailed ablations and additional policy cohorts.

Critical strengths
  1. The paper addresses a useful distinction: recovering from faults is insufficient if adaptation damages working feedback. Comparisons with bounded, partial, and regularized identification controls provide meaningful evidence for the recovery-interference tradeoff on the two manipulators.

  2. The empirical investigation is transparent: it reports component ablations, fresh policy cohorts, full-episode results, and failure traces, including negative transfer to HalfCheetah. The local propositions are explicitly separated from guarantees for the implemented nonlinear algorithm.

Critical weaknesses
  1. The central benefit of selective compensation remains limited to two simulated manipulators. On HalfCheetah, ECA loses 20.3 return on the healthy body and 36.0 under gain loss; ungated identification achieves greater average map recovery and less healthy-body degradation (Table 19). The proposed healthy-command-drift check is informative, but its ability to predict preservation under unseen faults is not established beyond these three plants. This limits the generality of the main contribution.

  2. A consequential initialization detail is missing from the main algorithm. Section 4.1 and Algorithm 1 restrict estimator updates to informative windows, whereas Appendix F.2 describes an exception during the first window. Applying the hold from step zero changes Pusher recovery by up to 93 return and substantially reduces HalfCheetah degradation. This is therefore a defining part of the method, not a minor implementation detail, and the current specification does not clearly determine which algorithm to reproduce.

Questions for authors
  1. Please specify the exact estimator-update and gate-decision rules before the buffer reaches its minimum size, including the first-window exception, and reconcile Algorithm 1 with Appendix F.2. Which reported experiments use each rule?

  2. Can a decision rule based only on healthy calibration data determine when ECA should be enabled, and does a threshold fixed in advance predict preservation on an additional plant or unseen fault family? This would help establish whether the proposed diagnostic supports deployment beyond the two manipulators.

Reviewer_hskROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ECA, a reward-free online adaptation wrapper for frozen control policies under actuator mismatch. It estimates actuator-map hypotheses from a learned nominal consequence model and uses a gate to decide whether compensation should remain active. The paper provides extensive simulation experiments and ablations. My main concerns are the unclear problem definition and motivation, limited novelty, insufficient evaluation of the gating mechanism itself, and the lack of real-robot validation.

Critical strengths
  • The paper provides a very detailed ablation study and failure analysis. Many individual components, including the gate initialization, delayed hypotheses, norm bound, and velocity-coupled hypothesis, are separately examined.

  • The gating mechanism is relatively lightweight and designed for fast online adaptation. The map hypothesis can start making decisions after only a few samples, and the reported overall computation remains within the simulated control periods.

Critical weaknesses
  1. The gating mechanism is a central contribution, but its decision accuracy is not directly evaluated. The paper mainly reports gate-open fractions, while the reported R2R^2 asures the regression accuracy of the nominal consequence model rather than the classification/decision quality of the gate. Since the fault types are known in simulation, precision/recall, false intervention rate, missed detection rate, or equivalent gating metrics should be directly measurable. Without these results, it is difficult to assess whether the proposed gate is actually reliable.

  2. There is no real-robot validation. For a control method targeting actuator mismatch and online adaptation, simulation-only evaluation leaves a substantial gap between the proposed method and practical control. Real systems introduce actuator nonlinearities, sensing noise, timing jitter, friction, and other effects that are difficult to capture with the synthetic faults considered here. This significantly weakens the practical evidence for the method.

Questions for authors
  • The introduction does not clearly define the core problem or distinguish the proposed actuator-mismatch setting from more general tracking/control errors. Can the authors give a clearer definition of the exact problem ECA is intended to solve?

  • In y=P(s,Mu+Kq˙)y=P(s,Mu+K\dot q), why is MM alone insufficient, and what specific role does Kq˙K\dot q play? What is the advantage of modeling uu and q˙\dot q separately?

  • The paper states that the contribution is mainly the selective application of the identification core and acknowledges that hypothesis selection and supervisory switching are established ideas. What is the main methodological novelty beyond combining these existing components?

  • What are the precision/recall or equivalent decision-quality metrics of the gate, and what is its detection/withdrawal latency under each fault type?

  • How much additional computation does the gating mechanism introduce relative to the ungated identification baseline, especially as the action dimension increases?

Reviewer_kuKBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how a frozen control policy can recover performance after deployment when actuator ordering, direction, gain, linear mixing, or delay errors occur, without damaging feedback that is already working. ECA wraps an identification-and-inversion core with static-map and velocity-coupled hypotheses, identity and delayed-identity alternatives, an evidence-based rejection gate, a unit amplification bound, and an identifier hold. The best-supported hypothesis is allowed to compensate online. In MuJoCo Reacher and Pusher experiments, ECA recovers performance under rotations, permutations, and orthogonal mixes while substantially reducing the interference caused by ungated identification under delays, healthy dynamics, and velocity-coupled disturbances. On a third held-out cohort, the recovery criterion is met on both plants, but preservation fails on the Pusher under gain loss; preservation also fails on a third contact-rich plant.

Critical strengths
  1. The objective and evaluation criteria are clearly defined. The paper treats fault recovery and preservation as separate goals and uses Bound ID, ECA-m, and ungated ID to isolate the effects of the unit bound, gate, hypothesis library, and withdrawal behavior.
  2. The evaluation protocol includes development, frozen, and held-out stages. Method choices are made on a development split, followed by new policy cohorts, fresh-episode controls, multiple fault types, and complete-episode checks. This reduces concern that the method was tuned to one trajectory.
Critical weaknesses
  1. The scope is narrow and all experiments are simulated. The method mainly covers fixed linear input maps and a velocity-coupled extension; it does not address inertia changes, nonlinear torques, or complex actuator dynamics. There is no hardware evaluation, and preservation fails on the third contact-rich plant.
  2. The statistical evidence is limited. Each split contains only five policies and each policy is evaluated on five episodes. The authors acknowledge that the bootstrap intervals are approximate and uncorrected for multiple comparisons, while the held-out cohort is evaluated only once.
Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.