acceptodds
Withdrawn from ICLR 2027

Hindsight Self-Distillation: Teaching Robots to Act from the Present

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Robot demonstrations reveal the consequences of actions, but a deployed policy must act before those consequences are observed. How can hindsight shape what a policy sees before it acts? We introduce Hindsight Self-Distillation (HSD), which uses training-only future observations to shape the visual summaries used for action generation. A privileged teacher and a current-observation student share video and action experts, with each action branch receiving visual information exclusively through its own summaries. Ground-truth action supervision trains the teacher, which transfers both action-flow predictions and summary representations to the student. An interface-aware analysis separates teacher dependence on unavailable future evidence, information lost through the student interface, and student approximation error. At deployment, the student computes its visual summaries once and reuses their keys and values throughout action denoising. On LIBERO and RoboTwin 2.0, HSD achieves average success rates of 98.50% and 92.31%, respectively.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, imitation learning, self-distillation, privileged information, world action models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_854TOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Hindsight Self-Distillation (HSD), a training framwork that leverages future observation hindsight from demonstrations to train a deployable, current-observation robot policy. HSD employs shared Video and Action DiT backbones with distinct query banks for a privileged teacher and a student. The teacher is trained on ground-truth actions, which distills both action-flow predictions and visual summary representations to the student. The authors also provide an interface-aware theoretical decomposition separating distillation mismatch into unavailable future evidence, representation loss, and student approximation error.

Critical strengths
  • The idea of leveraging hindsight observation is simple, intuitive, and well-motivated, and the experiment also shows its effectiveness.
  • The shared model parameter and summary level transfer effectively improves the alignment between teacher and student model.
  • The experiments show consistent gain across both simulated and real world environments.
  • HSD demonstrates comparable inference latency as Fast-WAM.
Critical weaknesses
  • The theoretical analysis minimizes the KL divergence between teacher and student, but the training loss is a linear combination of action-flow and summary MSE loss, which introduces a gap between the theoretical analysis and the actual training.
  • The value of observation horizon is not listed in the paper, and the similarity of teacher's and student's summary depends on this parameter. If it is much smaller than action chunk, how the teacher prediction is mixed with ground truth action and how student model is leanred to generalize to a larger action chunk?
Questions for authors
  • What is the key difference between action level transfer and other IDM methods? Since the dataset already contains the ground truth action, what is the advantage of using teacher model to predict the action?
  • What is the design choice of the summary loss? It is clear that the teacher's summary contains many future information depending on the observation horizon, will this affect the representation alignment? Can you compare with other alignment methods, such as an extra summary prediction head with student's summary and action as input?
  • How many seeds are used for the experiments? In the ablation study, most difference comes from the long horizon task, can you provide an explanation of why the action level and summary level transfer improves the performance in long horizon task?
Reviewer_BpoQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents Hindsight Self-Distillation (HSD), a self-distillation framework for world action models in which a teacher is provided with future observations to produce outputs that are used to supervise a deployable student branch. The teacher provides supervision signals at both visual summary and action level by conditioning on privileged future information. The proposed distillation technique improves the base FastWAM on both simulated and real-robot tasks, and achieves strong performance compared to the baselines. Ablation shows how summary- and action-level transfers, as well as the number of summary tokens, affect task performance.

Critical strengths
  1. The proposed method distills hindsight information for action generation in a world action model, improving task performance while maintaining modest inference cost.
  2. The experiment includes broad evaluation settings across both simulated and real-robot environments.
Critical weaknesses
  1. The proposed method (summary-level transfer in particular) is closely related to several prior works that align future representation targets[1][2], which also provide latent features encoded from future observations as "privileged information” during training. Nevertheless, the experiments include no direct comparisons with these conceptually close baselines.
  2. The shown improvements on both benchmarks look "marginal" when compared to the base FastWAM model, it remains unclear whether those improvements fall within the uncertainty range of the results (e.g., standard deviation/error). Similarly, in the ablation, “No explicit transfer” is better than “with summary-level transfer” on average performance, and 16 summary tokens are worse than 8/32 summary tokens on average. It’d be essential to include confidence intervals for the reported numbers to draw robust conclusions.
  3. The writing of the paper requires major revision to improve readability. Many terms in the abstract or introduction, such as “training-only future observations”, “visual summaries”, "query banks", “parameter snapshot”, and “interface-aware”, are used without formal definition and elaboration, making it difficult to infer their meanings and follow the overall pitch in the first place without carefully reading the method.

[1] Zheng et al. FLARE: Robot Learning with Implicit World Modeling. 2025.

[2] Zhao et al. FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment, 2026.

Questions for authors
  1. Currently, all weights in Eq. (8) are set equally to 1. How would varying the weights for each loss in Eq. (8) affect the performance?
  2. What is the horizon of future observations hh used in the experiment? Is it the same as the action horizon HH?
Reviewer_gYBeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a method to supervise action prediction using the information contained in the future frames of the trajectory, using student teacher distillation to get a deployable student model that does not require this future frame conditioning. They do this by using a privileged teacher model that shares a video and action backbone but uses separate branches for learnable query tokens that provide action prediction relevant context. The teacher supervises both these representational alignment as well as direct action flow supervision. They evaluate their method on real world robotics tasks, as well as robotwin and libero.

Critical strengths

This paper has a well motivated approach, and a strong set of both simulation and real world robotics tasks. It also maintains the benefits of fast inference due to discarding the teacher branch performing around the same inference time speed as fast wam.

Critical weaknesses

In table 3’s ablation, compared to the no explicit transfer baseline, action level supervision provides only a .15% improvement, while summary level distillation alone reduces performance by .25%. The combination of the two seems to only help significantly on libero long, making it hard to understand the performance gains over baselines. Can you add a student only baseline to these results to show a performance floor as well as add statistical significance analysis to these and the other results to show if the gains are meaningful?

Differing forms of student/teacher or prior / posterior world model supervision appears in prior work mentioned in the paper such as being h07. Can you clarify the distinction between past works like being-ho7 which learn latent query tokens in a prior branch supervised by true next frame representations in a posterior branch? I also think including this method as a baseline due to its similarity would strengthen this work.

Questions for authors

Why do you supervise the final action flow with both the gt actions as well as the teachers predictions?

Can you clarify what the theoretical results establish specifically about your objectives and how they justify the proposed distillation methods?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.