acceptodds
Withdrawn from ICLR 2027

TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large-scale action pretraining equips Embodied Foundation Models (EFMs) with strong manipulation capabilities, with particular strengths in generalization and closed-loop robustness, but reproducing such pretraining requires substantial robot data and computation. To transfer these capabilities from pretrained EFMs to policies without action pretraining, we introduce TILDE (**T**eacher-**I**n-the-**L**oop **D**istillation from **E**FMs), which synergizes on-policy distillation at student-induced states with learning from teacher-executed recovery trajectories. We further propose AutoDAgger, an autonomous data aggregation pipeline in which a robotic reward model monitors task progress and triggers teacher intervention, thereby collecting recovery experience without human monitoring or manual control. AutoDAgger accelerates recovery-trajectory collection by more than compared with the human-in-the-loop method. By unifying learning from demonstrations, on-policy teacher supervision, and teacher's corrective behavior, TILDE improves task performance by up to over behavior cloning across three real-world tasks and supports effective cross-architecture distillation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Foundation Model, Policy Distillation, Interactive Imitation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Wj7ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposed TILDE and AutoDAgger where they use an off-the-shelf reward function to monitor progress of a student VLA, and when it gets stuck, uses a teacher VLA that was trained on large scale action data to guide the student VLA. The experiments focuses on distilling this large scale action data pertaining into the student VLA. Experiments cover real world tasks, cross architecture transfer, and comparison with HG-DAgger. It shows TILDE achieves better success rate compared to other distillation baselines and needs less teacher supervision compared to HG-Dagger.

Critical strengths
  1. Idea is very straightforward and figure 1 shows a motivation.
  2. The monitoring reward function is an off-the-shelf module so it doesn't need further fine-tuning or extra components.
  3. Shows good performance on real world tasks and shows scaling capabilities.
Critical weaknesses
  1. Distilling from a teacher is a very common idea (RPD, VLA-OPD) so the real contribution is the auto monitoring loop. But there is no ablation on the reward model choices. This also feels like a very limited contribution.
  2. The current formulation only works for quasi-static tasks and would be hard to implement on more dynamic tasks or tasks where progress is hard to judge (e.g., tasks that requires waiting where the robot does nothing while waiting).
  3. More in the questions section below
Questions for authors
  1. How does the quality of the monitoring reward function impact the distillation process? I imagine as long as it picks sufficient coverage from the student's distribution then it would suffice? What if we replace the entire procedure of AutoDAgger to be randomly sampling from the student's distribution or have the teacher randomly intervene?
  2. For figure 1(d), both yellow and red lines are generated by the same teacher VLA? the current color coding is a little confusing.
  3. The notation in equation 3 is confusing. After we sample the observations do we assign a score to each of the observation or only one score as a whole? How are the observations sampled? at random, equally spaced? why not just always evaluate the last K_t frames to get the latest progress?
  4. For the comparisons in Table 4, how much of the efficiency improvements comes from the fact that humans have longer reaction times / potentially suboptimal performance than teacher VLAs?
  5. One of the claim is "large-scale action pretraining can be transferred", but is there any experiment shows the action pretraining knowledge outside of the demonstrated task or other task variations? Right now the experiments are only strong enough to conclude that "large-scale action pretraining knowledge related to this task can be transferred".
  6. One of the motivation is to avoid large scale action data pertaining, but the teacher with the action data pertaining still outperforms the trained student by a considerable margin. Additionally, large scale action data pertaining typically uses SFT without the need of on-policy rollouts, so it can scale very well as compute grows. How does the actual training time compare? How much time would the proposed TILDE and AutoDAgger actually save at the cost of the performance degradation?
Reviewer_uiCFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

TILDE proposes a teacher-in-the-loop distillation method to transfer ability from a pretrained EFM to a untrained model. The authors also propose AutoDagger to make the data collection more efficient.

Critical strengths

The paper is easy to follow, and the presentation is quite clear.

Critical weaknesses

I have a few concerns about the clarity and motivation of this paper:

  1. The two components proposed in the method are disconnected. They can be treated as two different work. Can the authors explain the neccessartiy of combining these two components together?
  2. Another confusing design is the teacher and the student models are exactly same architecture. My quesiton is: if we already have a strong teacher model here, why do we have to clone its ability to another student model? This question should be answered properly.
  3. The current ablation missed KD-only cases.
Questions for authors

I've asked my main two questions in the first and second points of weaknesses, please answer them properly.

Reviewer_WGzHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a DAgger-like way of distilling pretrained foundation models (for robotics) into student policies.

Critical strengths

The results are positive on real-robot manipulation tasks. The idea overall makes sense and intuitive.

Critical weaknesses

The proposed method mainly integrates existing components. The analogy between action distillation and imitation learning is already an obvious one. Using ideas from imitation learning, in this case DAgger, for distillation is therefore not conceptually new. Perhaps the automation of when the teacher will be queried is novel, but the method there is mainly some heuristics, and it is not clear if those heuristics are the best.

Another reason why I don't see the latter part as a vital contribution is because whether interventions occur at appropriate times was not tested. So even if there may be some novelty there, that part is not assessed in a sufficient way.

Finally, if my understanding is correct, the reported more than 2x speedup measures successful recovery execution time. But total data collection time, including stagnation detection, failed attempts, and resets, is also important (probably more important). Is there a more than 2x speedup there, too?

Questions for authors

See the weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.