acceptodds
Withdrawn from ICLR 2027

TETHER: Task-Evolving Teacher for History-Efficient Regularization in Continual Robot Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In continual robot manipulation, sequential fine-tuning can cause a language-conditioned flow-matching policy to generate similar actions under distinct past task conditions. We introduce TETHER (Task-Evolving Teacher for History-Efficient Regularization), which preserves past condition–action correspondences in a shared policy without storing individual demonstrations. The preceding policy serves as a frozen, task-evolving teacher. Task Statistics Memory (TSM) retains compact state-cluster statistics and local state-to-visual-feature maps to reconstruct paired proprioceptive and visual conditions. The teacher estimates an action coordinate for each reconstructed condition, defining action queries at which the student matches its velocity while learning new tasks through flow matching. Magnitude-Aware Regularization (MAR) gives small teacher pose motions greater relative weight, strengthening directional guidance that popular squared-error matching can underemphasize. We evaluate TETHER on simulated benchmarks and real-robot tasks. TETHER achieves strong performance compared to the state-of-the-art methods, notably with the best Area Under the Curve (AUC) and lowest Negative Backward Transfer (NBT) across various tasks, showing that its regularization effectively mitigates forgetting of previously learned condition–action correspondences.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, lifelong robot learning, imitation learning, robot manipulation, flow matching, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HqWaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

TETHER addresses catastrophic forgetting in continual robot learning without retaining individual demonstrations. It uses compact state-cluster statistics and local state-to-visual-feature mappings to reconstruct past task conditions. A frozen teacher from the preceding learning stage guides the current policy through velocity-field matching, while Magnitude-Aware Regularization assigns greater relative weight to small teacher pose responses. Experiments on LIBERO benchmarks and four real-robot tasks demonstrate improved task retention and reduced historical-data storage.

Critical strengths
  • The paper focuses on an important problem in continual robot learning: acquiring new skills while preserving previously learned behaviors with limited historical data storage.
  • Reconstructing paired state and visual conditions from compact statistics provides a practical way to support teacher-guided retention within a single shared policy.
  • Extensive experiments demonstrate the effectiveness of TETHER.
Critical weaknesses
  • Similar proprioceptive states may correspond to different object configurations or grasp outcomes. Attempted and successful grasps may fall into the same cluster, whose deterministic state-to-feature mapping may fail to preserve these task-relevant distinctions.
  • The one-step teacher prediction approximates the conditional action mean and may inadequately represent multimodal action distributions. Consequently, distillation queries constructed from this estimate may fail to sufficiently cover distinct valid behaviors.
  • Although the authors acknowledge reduced new-task performance in a few settings, the trade-off between retaining old skills and learning new ones remains insufficiently analyzed. The results do not clearly establish whether reduced forgetting reflects more effective knowledge preservation or partly stems from stronger constraints on learning new tasks.
Questions for authors

See weaknesses above.

Reviewer_SbMHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

TETHER is a continual imitation learning method for language-conditioned, flow-matching robot policies. At each learning stage, the previous policy is frozen as a teacher, and the current policy is trained on new task demonstrations while matching the teacher’s velocity predictions on reconstructed past-task conditions. To construct those conditions without retaining past demonstrations, Task Statistics Memory stores per-task state-cluster statistics and local maps from robot state to visual features. For each reconstructed condition, the teacher makes a one-step action estimate that is used to form a noisy-action query for velocity matching. Magnitude-Aware Regularization increases the relative weight of pose-matching errors at smaller teacher pose magnitudes. The performance of TEHER was evaluated and compared on LIBERO benchmarks and a four-task real-robot sequence.

Critical strengths

S1) TETHER offers an alternative to retaining past latent-action examples: it statistically reconstructs past-task conditions and uses a frozen teacher to generate flow-velocity targets on those conditions. This is a coherent design tailored to flow-matching policies.

S2) Although limited to four tasks and a comparison against ER alone, the evaluation includes a physical-robot study. This provides useful hardware evidence that is absent from some recent continual-imitation studies, which evaluate only in simulation.

Critical weaknesses

W1) The paper does not establish the originality of its central method clearly enough. Continual imitation learning already includes multimodal feature and action distillation (e.g., M2Distill) and compact multimodal replay (MLR-IFA). TETHER’s proposed distinction, reconstructing paired conditions statistically and matching flow velocities on teacher-derived queries, may be new, but the paper does not sufficiently situate that combination against these closest methods or show in a matched comparison that it offers a necessary advantage. This leaves the main contribution’s novelty uncertain.

W2) Several leading baseline results, including MLR and MLR-IFA, are taken from prior work using different architectures, pretraining, and training configurations. As far as I understood, the paper reruns some baselines, including ER and CLARE, on its own setup, but those comparisons do not establish superiority over the closest replay-and-distillation methods.

W3) The real-robot study is also limited to four tasks and an ER comparison. This weakens the evidence for the paper’s broader comparative claims.

W4) The diagnostic plots angular-loss coefficients, not measured improvement in teacher–student directional alignment; the authors note that the compared students’ magnitudes also differ. In the anchoring ablation, full TETHER improves AUC and FWT over coupled anchors, but has worse NBT. That makes the specific claim that MAR improves retention less conclusive.

W5) The paper fits its state clusters and state-to-feature maps using demonstrations, then evaluates the reported state coverage and feature variance diagnostics on those same data; the feature variance analysis also covers only one task. These results show how well TSM summarizes its fitting data, but not whether it reconstructs realistic paired state–visual conditions for unseen demonstrations. This leaves the validity of the inputs used for teacher–student matching uncertain.

Questions for authors

Q1) How comparable are the MLR and MLR-IFA results in Table 1 to TETHER’s runs? Please specify the initial policy weights and pretraining data, visual and language encoders, training budget, and evaluation protocol for each. Were MLR/MLR-IFA rerun under TETHER’s setup, or are these values entirely from the cited paper? This would affect how I interpret the claimed advantage over these closest methods.

Q2) Did the authors evaluate TSM’s joint state-visual reconstruction on held-out demonstrations? The reported diagnostics appear to use the data used to fit TSM, and the feature-variance analysis is for one task. If held-out validation was performed, please report it; if not, please clarify how the authors establish that reconstructed pairs are representative enough for teacher–student matching.

Q3) What is the precise novelty claim relative to M2Distill and MLR-IFA? In particular, which part is claimed as new: reconstructing paired conditions from statistics, querying a flow-matching teacher at teacher-derived action coordinates, or MAR? A precise distinction would help assess the contribution.

Q4) A matched comparison with MLR-IFA: using the same backbone (e.g., CLIP, not DINO), initialization, data split, and training budget would more directly test whether TETHER’s reconstructed-condition velocity matching offers an advantage over latent replay and action-policy distillation.

I would reconsider my score especially if the authors (1) provide a matched comparison with both MLR and MLR-IFA by using the same backbone and encoders, initialization and pretraining, task sequence and demonstrations, training budget, seeds, and evaluation protocol and (2) precisely identify which parts of TETHER they claim are novel and how each differs from those prior methods. The current cross-paper results are not enough to establish comparative advantage.

Reviewer_XUpnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TETHER addresses continual robot imitation learning without retaining individual demonstrations. It reconstructs past state–visual conditions from compact statistics and uses the previous policy as a teacher for velocity matching. Magnitude-Aware Regularization gives smaller pose actions greater relative weight. Experiments show improved retention on several LIBERO benchmarks and four real-robot tasks.

Critical strengths
  • The method achieves strong retention, particularly on LIBERO-50 and Spatial.
  • Statistical memory reduces storage compared with raw demonstration replay.
  • The final system uses a single shared policy without additional inference modules
Critical weaknesses
  • The practical advantage over compressed replay remains unclear. TETHER still stores task statistics and a teacher model. Comparisons against latent replay under matched total storage and computation budgets are needed to justify this design.
  • The ablations do not clearly isolate the contributions. In particular, the paper should evaluate the complete TSM without MAR and separately measure the benefit of the local state-to-feature maps.
  • Better retention comes with reduced new-task acquisition in several settings. For example, on LIBERO-50, acquisition success drops from ER’s 73.3 to 62.9. A regularization-strength sweep would help establish whether TETHER offers a better retention–acquisition trade-off.
  • The explanation for MAR is not fully supported. The diagnostics show changes in loss coefficients rather than directly demonstrating improved directional alignment or better performance on fine manipulation steps.
Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.