- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TwinSight, a framework that uses learner-specific predictions to guide human demonstration collection and subsequent policy adaptation. It identifies three challenges and proposes corresponding solutions. To address the limited foresight of short-horizon action previews, it trains a learner-specific world model to forecast longer-horizon end-effector motion and potential failures. To preserve early corrective actions whose consequences emerge later, it introduces teaching bubbles that record local learner–human discrepancies and recover temporally coherent demonstration segments. To prioritize segments that may benefit policy adaptation, it ranks them by their average discrepancies for subsequent selection and fine-tuning. Experiments on six simulated manipulation tasks and three real-robot tasks demonstrate performance improvements over baselines including HG-DAgger and RoboPocket-style action previews, with additional ablation studies examining the contributions of longer-horizon forecasts, teaching bubbles, and segment selection.
Critical strengths
1. Intuitive visualization through teaching bubbles. The teaching-bubble visualization is an interesting design choice. By spatially grouping learner–human discrepancies into persistent visual markers, the method may provide clearer and more interpretable guidance during human demonstrations.
2. Relatively comprehensive experimental evaluation. The paper evaluates its method on six simulated manipulation tasks and three real-robot tasks, demonstrating performance improvements over the evaluated baselines. This provides encouraging empirical evidence for the effectiveness of the proposed framework.
Critical weaknesses
Major Weaknesses
1. The learner-specific world model introduces strong assumptions.
-
(1) Modeling complexity: The accuracy and reliability of the proposed learner-specific world model are concerning. Unlike a general world model, the predictive twin must implicitly capture both environment dynamics and the pretrained policy's future decisions. This introduces additional modeling complexity and raises concerns about prediction accuracy. A natural alternative would be to combine a general world model with the pretrained policy through recursive rollouts, but this comparison is missing.
-
(2) Model reliability: Even learning an accurate general world model is challenging. Model mismatch, especially when physical properties or environmental conditions differ from the training data, may lead to inaccurate future predictions. The robustness of the proposed predictive twin remains unclear.
-
(3) Alternative approaches: If a sufficiently accurate world model can be learned, it is unclear why the authors do not directly leverage it for model-based planning or reinforcement learning to improve the policy, rather than using it primarily to guide human demonstrations.
2. Teaching bubbles do not guarantee the retention of useful corrective actions.
-
(1) Insufficiently addressing Challenge 2: To my understanding, Challenge 2 concerns identifying and retaining useful demonstration segments when important corrective decisions and their consequences occur at different times. However, the proposed method primarily relies on short-horizon learner–human discrepancies and fixed context windows to construct training segments. While this provides temporal localization of behavioral differences, it does not explicitly address causal credit assignment or establish whether the selected segments contain the earlier decisions responsible for preventing future failures. Therefore, it remains unclear whether the proposed solution sufficiently addresses the stated challenge.
-
(2) Visualization scalability: As the number of demonstrations increases or task trajectories overlap significantly, teaching bubbles may become increasingly dense and potentially obstruct the human teacher's view. The paper does not clearly explain how bubble accumulation is managed across demonstrations. Without appropriate visualization controls, excessive bubbles could obscure the robot's workspace and interfere with human teaching.
3. Discrepancy-based segment ranking does not necessarily target the learner's weaknesses.
Challenge 3 concerns whether a demonstration segment relevant to a learner's weakness actually improves policy performance. However, the proposed solution simply ranks segments by learner–human behavioral discrepancy. A larger discrepancy does not necessarily indicate greater training utility, as different behaviors may be equally valid, or the demonstration may provide little additional learning value. Therefore, the proposed discrepancy-based heuristic does not directly address the stated challenge, and the connection between behavioral discrepancy and actual policy improvement remains insufficiently justified.
Minor Weaknesses
1. Limited ability to model multimodal future trajectories and predictive uncertainty.
The proposed predictive twin generates a single future end-effector position at each horizon using an MSE objective. However, in robotic manipulation, multiple valid future trajectories may exist, each potentially leading to different subsequent behaviors. Point predictions may fail to capture such multimodality. Moreover, the framework does not explicitly model predictive uncertainty, which could help human teachers assess forecast reliability and understand the learner's possible future behaviors. It would be valuable to discuss this limitation and investigate whether probabilistic or multimodal predictions could improve the framework.
Questions for authors
Please refer to the weaknesses section above
Reminder to respond to the authors’ rebuttal
With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.
Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.
Best regards,
Area Chair