- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
LoopWAM learns from expert demonstrations and mixed-quality deployment rollouts through joint video-action denoising. Source-dependent noise sampling lets relatively clean rollout actions condition consequence prediction, while quality labels condition both streams and weight action imitation. Failed trajectories retain video supervision but receive no direct action-imitation loss. At deployment, the model uses the expert-quality condition. Across separately trained policies for four robot embodiments, three deployment-and-training updates raise Table30-V2 success from 23.67% to 43.33%. Shirt-folding success rises from 13.20% to 55.56% over two updates after task-specific expert training. RoboTwin 2.0 and RoboCasa365 provide additional simulation results. The repeated real-robot gains support the complete pipeline; the individual contributions of failure data, bidirectional interaction, and numerical quality scoring are less clearly established.
Critical strengths
-
Repeated improvement on substantial real-robot evaluations. Table30-V2 covers 30 tasks across four robot embodiments, with a separate multitask policy for each embodiment. Overall success improves at all three updates, from 23.67% to 28.00%, 37.67%, and 43.33%. The separate shirt-folding study also improves from 13.20% to 55.56% over two updates after task-specific expert training. These results provide substantial evidence that deployment-driven learning is useful across varied manipulation tasks.
-
Practical evidence for quality-aware training. With rollout action supervision disabled, numerical quality conditioning raises RoboTwin success from 72.3% to 92.7%; simple low/medium/high labels reach 92.8%. Adding weighted rollout action supervision to the numerical scheme raises success to 94.5% (Table 4). Thus quality conditioning and selective imitation are useful parts of the recipe, and simple quality categories already work well in this setting.
Critical weaknesses
-
Limited methodological novelty. UWM already couples video and action diffusion with independent noise levels; FACT learns failed-action consequences while masking their imitation loss; and uses quality metadata to learn from mixed-quality data. LoopWAM cites these precedents and adds a combination of source-dependent noise sampling, bidirectional denoising, and quality-aware supervision. The real-robot results support the integrated approach, but the learning advantage provided by this particular combination, especially the noise design and bidirectional interaction, is less directly established.
-
The distinct value of failure data is unclear. Each deployment update adds successful rollouts as well as failures. Table 4 varies quality conditioning and rollout action supervision, while retaining the same video-supervision objective on expert and rollout data in every configuration. It therefore does not establish what failed outcomes add beyond the useful behavior in successful rollouts. The repeated gains support the complete pipeline, but provide less direct support for the central claim that learning from failure experience itself improves the policy.
-
The meaning of numerical quality scores needs clarification. All training segments from an episode share one score. Under coarse annotation, successful rollouts from the same policy also share a score. These scores can reflect overall policy performance, but may not capture the quality of each action. For example, useful early actions receive no direct imitation supervision if the episode fails.Table 4 shows that broad labels already work well. Without rollout action supervision, they reach 92.8%, versus 92.7% for numerical scores. With this supervision, broad labels reach 93.6% and numerical scores reach 94.5%.It remains unclear what numerical scores capture beyond these categories. The effect of label errors on conditioning and action weighting is also unclear.
Questions for authors
-
What learning advantage or new capability does this combination provide beyond existing failure-aware WAM training and quality-conditioned policies? Which design choice is central to that advantage?
-
What evidence shows that failed trajectories add useful learning signals beyond successful rollouts and quality conditioning?
-
The paper highlights bidirectional interaction as a way for learned consequences to inform action generation. What evidence supports the added benefit of this design when learning from deployment rollouts?
-
Appendix A.1 describes completion-based episode ratings and source-policy scores based on success and inspected motion quality. Could you clarify how these criteria translate into numerical scores and illustrate this with a few existing trajectories and short rationales? This would help explain what numerical scores capture beyond the low/medium/high labels in Table 4.
3 comments
Question 1: Learning advantage over existing approaches and the central design choice
The central design connects action-conditioned consequence learning with expert-directed action generation, as tested above. We additionally compare with FACT and RECAP-style conditioning, using the same expert demonstrations as FACT and matching its rollout count (1.3K trajectories) and policy optimization budget:
| Method / variant | Additional rollout data | Average success (%) |
|---|---|---|
| LoopWAM (FACT-matched count) | 1.3K failed | 92.3 |
| FACT (reported) | 1.3K failed | 87.5 |
| LoopWAM (quality steering) | 1.3K mixed | 93.8 |
| LoopWAM + RECAP-style conditioning | Same 1.3K mixed | 93.1 |
FACT: Its result is taken from the paper. We sample failures from our corpus because we could not locate a public release of FACT's specific failure-rollout set. The 4.8-point improvement, at matched failure-data quantity and policy optimization budget, supports the complete LoopWAM training design at this data scale.
RECAP-style conditioning: The mixed rollout samples and all other LoopWAM components are identical. A Gemma-based VLM value function trained on those rollouts supplies binary advantage indicators in place of quality text; inference uses the positive indicator. Quality steering performs 0.7 points better without additional value-model training. That extra training is excluded from the matched policy optimization budget.
Together, these comparisons support the complete design for learning from deployment experience and quality steering within that design. Alongside the component ablations, they support connecting consequence learning with action generation guided toward the expert reference (Appendix F.5).
Question 2: Evidence beyond successful rollouts and quality conditioning
The controlled experiment in Weakness 2 changes only failed-rollout WM loss, retaining successful-rollout supervision and quality conditioning. Its 1.4-point gain directly supports an additional learning signal from failures (Appendix F.3).
Question 3: Evidence for bidirectional video–action attention
The one-way variant lets action tokens attend to video tokens but prevents video tokens from attending to action tokens. With data and optimization unchanged, it achieves 89.5% versus 94.5% for full LoopWAM. This removes the video branch's direct access to action tokens for consequence prediction. Bidirectional attention restores this connection and lets action-dependent video representations participate in action denoising (Appendix F.2).
Question 4: How scores are assigned, with trajectory examples
Thank you for suggesting that we include concrete examples. We have included visualized rollout examples from both Table30-V2 and RoboTwin 2.0 in Appendix G: Qualitative Examples of Trajectory Quality, covering episode-level and coarse-level annotation. The visualizations and accompanying explanations present actual executions, their assigned scores, and the criteria behind those scores.
- Table30-V2 (episode-level annotation): Trained platform evaluators score task progress and execution quality out of 10. Unfinished steps reduce the score, and each retry incurs a 0.5-point deduction. We retain the platform scores for both successful and failed rollouts; the examples illustrate how completed steps and retries affect the ratings.
- RoboTwin 2.0 (coarse-level annotation): Expert demonstrations receive q = 5, failures q = 1, and successful rollouts inherit fixed source-policy grades (q = 3 or 4). These grades reflect human judgments of overall policy reliability and inspected motion quality. The examples include trajectory visualizations and action traces illustrating this assessment.
Further discussion of the scoring schemes and their use in training is provided in Appendix F.4.
If you have any further questions, please let us know. Thank you again for your careful review and helpful suggestions!
Dear Reviewer,
Thank you for your detailed and thoughtful review. We have now posted our responses and revised the manuscript with additional experiments and clarifications addressing your concerns. Your feedback has been valuable in strengthening our work, and we would be grateful if you could reconsider your evaluation in light of the new evidence and explanations. We hope these additions have addressed the issues you raised. If any questions or concerns remain, we would welcome the opportunity to discuss them and provide further clarification. Thank you again for the time and care you have devoted to reviewing our manuscript.
Best regards,
The Authors
Thank you for your careful reading and constructive feedback. We have revised the manuscript PDF by appending Appendix F: Rebuttal Material after the original supplementary material, adding controlled experiments and annotation details. Trajectory visualizations and scoring examples appear in Appendix G.
Weakness 1: Methodological contribution and evidence for the training design
We appreciate this concern. Building on prior WAM and quality-conditioning research, LoopWAM targets closed-loop self-improvement in the physical world from mixed-quality deployment experience. Source-dependent noise sampling keeps recorded rollout actions relatively clean for action-conditioned world modeling; bidirectional video–action attention connects consequence prediction with action generation; and quality steering and quality-aware supervision guide behavior toward the expert reference while retaining predictive learning across qualities.
We test the noise and attention designs on RoboTwin 2.0.
In the first two rows, video and action noise remain independently sampled; adapting their distributions to the data source improves success by 5.6 points. Expert demonstrations support high-quality generation, while relatively clean rollout actions support prediction of their observed consequences. Bidirectional attention lets these action-dependent video representations participate in action denoising, with a 5.0-point benefit over one-way attention (Appendix F.2).
Weakness 2: The distinct contribution of failed trajectories
We agree that adding mixed outcomes alone does not isolate the contribution of failures.
The last two rows hold all trajectories, sampling, and optimization updates fixed, changing only failed-rollout WM supervision. Failed-rollout action loss remains zero; expert and successful-rollout supervision and quality conditioning are unchanged. The 1.4-point gain therefore isolates a benefit from learning failed-action consequences beyond successful-rollout supervision, without imitating failed actions.
Adding only failed rollouts also improves over expert-only training (92.7% vs. 91.4%), and the full mixture outperforms successful-only training. The fixed-data WM comparison isolates the contribution of failed-rollout supervision while holding the number of optimization updates and the training-data sampling scheme constant (Appendix F.3).
Weakness 3: Meaning of numerical quality scores and sensitivity to label errors
We agree that a score shared by all segments of a trajectory cannot capture each action's quality. Useful early actions in a failed episode receive no direct imitation supervision, although their transitions remain available for WM learning.
On RoboTwin, low/medium/high labels separate failures, successes, and experts; numerical scores further distinguish source-policy quality within successful rollouts. In Table 4 of the original manuscript, categorical and numerical conditioning achieve 92.8% and 92.7% without rollout action supervision, and 93.6% and 94.5% with quality-weighted supervision. These results show that finer-grained quality annotations can yield additional gains when combined with action weighting. We therefore hypothesize that reliable action-level quality annotations could further improve learning, but their higher annotation cost would require a trade-off between performance and labeling effort.
We randomly corrupt 10% or 30% of rollout labels, uniformly replacing each selected score with another allowed value, fixed before training; expert labels remain unchanged. Modified scores control both conditioning and action weights, without a separate mask based on the original success/failure outcomes. A mislabeled failure can therefore receive nonzero imitation weight.
LoopWAM shows some robustness at 10% corruption, but degrades substantially at 30%. Errors disrupt both conditioning and imitation weights, highlighting the importance of quality steering and quality-aware supervision for mixed-quality learning (Appendix F.4).
To be continued in next reply.