acceptodds
Under review as a conference paper at ICLR 2027

Twinsight: Guiding Human Teaching and Policy Adaptation with Learner Forecasts

Abstract

Successful demonstrations can still leave a robot policy's weaknesses unaddressed. Without seeing how the learner would continue, human teachers may miss useful corrections, and data selection cannot recover examples that were never demonstrated. We introduce Twinsight, a framework that connects human teaching and policy adaptation through learner forecasts. During human control, a learner-specific predictive model forecasts end-effector motion beyond the current action chunk, helping teachers anticipate later difficulties before committing to an approach. Persistent teaching bubbles record local differences between forecast and demonstrated motion and link them to coherent human action segments for subsequent selection and training. Across six simulated and three real-robot manipulation tasks, Twinsight achieves mean success rates of 74.0% and 80.7%, exceeding the strongest evaluated baselines by 5.3 and 6.7 percentage points, respectively. On shared simulation datasets, segment selection achieves similar mean success to training on all retained data with 30.7% fewer training steps.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Learning from Demonstration, Vision-Language-Action Models, World Models, Human-in-the-Loop Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XzsmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TwinSight, a framework that uses learner-specific predictions to guide human demonstration collection and subsequent policy adaptation. It identifies three challenges and proposes corresponding solutions. To address the limited foresight of short-horizon action previews, it trains a learner-specific world model to forecast longer-horizon end-effector motion and potential failures. To preserve early corrective actions whose consequences emerge later, it introduces teaching bubbles that record local learner–human discrepancies and recover temporally coherent demonstration segments. To prioritize segments that may benefit policy adaptation, it ranks them by their average discrepancies for subsequent selection and fine-tuning. Experiments on six simulated manipulation tasks and three real-robot tasks demonstrate performance improvements over baselines including HG-DAgger and RoboPocket-style action previews, with additional ablation studies examining the contributions of longer-horizon forecasts, teaching bubbles, and segment selection.

Critical strengths

1. Intuitive visualization through teaching bubbles. The teaching-bubble visualization is an interesting design choice. By spatially grouping learner–human discrepancies into persistent visual markers, the method may provide clearer and more interpretable guidance during human demonstrations.

2. Relatively comprehensive experimental evaluation. The paper evaluates its method on six simulated manipulation tasks and three real-robot tasks, demonstrating performance improvements over the evaluated baselines. This provides encouraging empirical evidence for the effectiveness of the proposed framework.

Critical weaknesses

Major Weaknesses

1. The learner-specific world model introduces strong assumptions.

  • (1) Modeling complexity: The accuracy and reliability of the proposed learner-specific world model are concerning. Unlike a general world model, the predictive twin must implicitly capture both environment dynamics and the pretrained policy's future decisions. This introduces additional modeling complexity and raises concerns about prediction accuracy. A natural alternative would be to combine a general world model with the pretrained policy through recursive rollouts, but this comparison is missing.

  • (2) Model reliability: Even learning an accurate general world model is challenging. Model mismatch, especially when physical properties or environmental conditions differ from the training data, may lead to inaccurate future predictions. The robustness of the proposed predictive twin remains unclear.

  • (3) Alternative approaches: If a sufficiently accurate world model can be learned, it is unclear why the authors do not directly leverage it for model-based planning or reinforcement learning to improve the policy, rather than using it primarily to guide human demonstrations.

2. Teaching bubbles do not guarantee the retention of useful corrective actions.

  • (1) Insufficiently addressing Challenge 2: To my understanding, Challenge 2 concerns identifying and retaining useful demonstration segments when important corrective decisions and their consequences occur at different times. However, the proposed method primarily relies on short-horizon learner–human discrepancies and fixed context windows to construct training segments. While this provides temporal localization of behavioral differences, it does not explicitly address causal credit assignment or establish whether the selected segments contain the earlier decisions responsible for preventing future failures. Therefore, it remains unclear whether the proposed solution sufficiently addresses the stated challenge.

  • (2) Visualization scalability: As the number of demonstrations increases or task trajectories overlap significantly, teaching bubbles may become increasingly dense and potentially obstruct the human teacher's view. The paper does not clearly explain how bubble accumulation is managed across demonstrations. Without appropriate visualization controls, excessive bubbles could obscure the robot's workspace and interfere with human teaching.

3. Discrepancy-based segment ranking does not necessarily target the learner's weaknesses.

Challenge 3 concerns whether a demonstration segment relevant to a learner's weakness actually improves policy performance. However, the proposed solution simply ranks segments by learner–human behavioral discrepancy. A larger discrepancy does not necessarily indicate greater training utility, as different behaviors may be equally valid, or the demonstration may provide little additional learning value. Therefore, the proposed discrepancy-based heuristic does not directly address the stated challenge, and the connection between behavioral discrepancy and actual policy improvement remains insufficiently justified.

Minor Weaknesses

1. Limited ability to model multimodal future trajectories and predictive uncertainty.

The proposed predictive twin generates a single future end-effector position at each horizon using an MSE objective. However, in robotic manipulation, multiple valid future trajectories may exist, each potentially leading to different subsequent behaviors. Point predictions may fail to capture such multimodality. Moreover, the framework does not explicitly model predictive uncertainty, which could help human teachers assess forecast reliability and understand the learner's possible future behaviors. It would be valuable to discuss this limitation and investigate whether probabilistic or multimodal predictions could improve the framework.

Questions for authors

Please refer to the weaknesses section above

1 comment
Area_Chair_Cwf1OpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

Reviewer_HKTEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission introduces Twinsight, a framework to guide human intervention collection for learned visuomotor policies. The core idea is to train a predictive model from policy's rollout data. The predictive model is tasked to predict future image frame change in feature space, and end-effector changes. The predictive model is then co-deployed with the policy for data collection. Candidate score is then derived for data selection purpose. Experiments on simulation benchmark and real-world tasks show that the proposed framework outperforms various baselines that use human correction data.

Critical strengths
  • The learned predictive model seems to provide useful visual guidance.
  • Discrepancy-based score quantifies the utility of human intervention data.
  • The proposed method achieves the best performance across all tasks compared to all baselines.
Critical weaknesses
  • Training the predictive model requires extra compute and time. However, baseline methods don't have such extra requirement.
  • To project the predicted end-effector movements back to the image frame, it assumes a fixed third-view camera with known transformation. This condition is not easy to satisfy for humanoid mobile manipulation or in-the-wild settings.
  • The performance improvement is marginal.
  • Tasks are relatively easy. It's unclear how the proposed method works with more challenging manipulation tasks or more sophisticated robot embodiments.
Questions for authors
  • The predictive model is trained with data collected by the same policy (which is frozen). So it is distributionally identical to the policy. How can we expect samples from the distribution induced by the predictive model can substantially provide more information to learn, compare to some vanilla approaches, e.g., resample the action multiple time from the policy?
  • L157, the forecast is provided for the full task horizon in an open-loop manner, rather than re-forecast every certain interval (closed loop). Therefore, I would expect the proposed method won't work well with quasi-static or dynamics tasks. Can authors show any results on such tasks?
  • How does the method compare to iteratively performing DAgger?
  • What is the exact model used for the predictive model?
  • How many evaluation episodes?
Reviewer_8MDSOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper describes a way of evaluating the informativeness of steps in a human demonstration of a task for a robot, and then using this to reduce the number of steps needed to FT a robot for that task.

Critical strengths
  • The paper is fairly clearly written with good examples.
  • The experiments are extensive, given the human-subject constraints.
  • The results show solid improvement over strong baselines.
Critical weaknesses
  • There are no error bars for the experimental results (and the test sizes seem to be small)
  • The method only reduces the computational cost of training, and doesn't affect the cost of obtaining human demonstrations (which I assume is relatively larger). There is also an overhead to using the predictive model, so the end improvement in performance is modest.
Questions for authors
  • I'm guessing these improvements are statistically significant, since they seem consistent, but can you verify that? are the results better for all human operators, or only some, or only in the aggregate?
1 comment
Area_Chair_Cwf1OpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.