- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes R²-WAM, a two-stage offline post-training framework for world action models built on Fast-WAM. Both stages combine demonstration-based flow matching with the existing DiffusionNFT negative fine-tuning objective. First, the video expert is refined by penalizing generated futures whose robot motion disagrees with demonstrated motion, measured through kinematic alignment and point tracking. The repaired video expert is then frozen and used to predict outcomes for sampled and demonstrated actions. A STEAM-style progress scorer identifies sampled actions with inferior predicted progress, which become negatives for action-policy fine-tuning. The framework requires no additional environment interaction during post-training and retains the original inference procedure. Evaluation covers 50 RoboTwin 2.0 tasks and two real robot tasks.
Critical strengths
- Strong empirical results. Evaluation spans 50 simulated tasks and two real robot tasks, including unseen objects and positions. The reported shirt-folding improvement is substantial.
- Meaningful ablations. The experiments separately examine video repair, action rejection, negative selection, and alternative repair scores, supporting the contribution of both stages.
- Practical deployment. Post-training reuses existing demonstrations without additional environment interaction, and deployment retains the original inference procedure.
- Direct evaluation of prediction reliability. The paper compares imagined consequences with actual simulator executions, providing evidence that repair improves the reliability of action-negative selection.
Critical weaknesses
- Limited methodological novelty. The framework applies the existing DiffusionNFT objective to Fast-WAM’s video and action experts, alongside an existing progress-scoring approach. The contribution is primarily the negative-selection rules and their sequential application.
- Unclear benefit relative to training cost. The method adds video generation, tracking, and progress scoring. Comparisons with equal optimizer updates do not establish an advantage under equal computational budgets.
- Remaining errors in training feedback. Negative selection depends on both generated videos and a learned scorer. Reported selection precision is 69.8% at the simulation training threshold, leaving substantial disagreement with execution-based judgments.
- Potential bias toward demonstrated short-term progress. Actions with lower immediate progress than the demonstration are penalized, even though slower or alternative strategies may ultimately succeed. This limitation is not sufficiently examined.
- Incomplete justification of the training procedure. Component ablations support combining repair and rejection, but do not establish whether the sequential schedule is preferable to joint or alternating training with the same objectives.
Questions for authors
-
Methodological contribution. What is the main new insight beyond applying DiffusionNFT to Fast-WAM’s video and action experts with different negative-selection rules? What specifically motivates NFT for these two stages?
-
Training schedule. Does sequential repair followed by rejection outperform joint or alternating training with the same objectives? The component ablations support using both stages, but do not isolate the benefit of their ordering and freezing strategy.
-
Computational efficiency. What is the total training cost, including candidate generation, tracking, and progress scoring? Would the gains remain under a comparison that gives ordinary demonstration training the same computational budget?
-
Reliability of negative labels. At the simulation training threshold, negative-selection precision is 69.8% against execution-based STEAM judgments. What is the effect of incorrectly selected negatives on policy training? Is there evidence that these judgments agree with eventual task success, beyond agreement with the same progress scorer?
-
Bias toward immediate progress. Can actions that make less short-term progress than the demonstration still lead to successful completion? Could necessary preparatory movements or alternative strategies be incorrectly penalized, particularly in long-horizon tasks?
-
Threshold selection. What determines the rejection margins and weighting scales, including their different values in simulation and real-world experiments? Beyond the selection metrics reported at several margins, how sensitive is final policy success to these choices?
5 comments
Author response to Reviewer FX3T - Part 2
“Does sequential repair followed by rejection outperform joint or alternating training with the same objectives? The component ablations support using both stages, but do not isolate the benefit of the ordering and freezing strategy.”
Response: We add comparisons using the same repair and rejection objectives, with two epochs allocated to each expert (Section 5.3, Table 5; Appendix E.3):
| Training schedule | Clean (%) | Randomized (%) |
|---|---|---|
| Joint | 92.7 | 92.4 |
| Alternating | 93.2 | 93.3 |
| Sequential | 93.8 | 93.7 |
Joint training updates both experts concurrently; alternating training switches between repair and rejection after each epoch; sequential training completes repair before rejection and freezes the video expert during rejection (Appendix E.3).
Sequential training yields higher success in both settings. It allows action rejection to use the repaired predictor from the outset and keeps that predictor fixed throughout action post-training. These results support the sequential schedule among the evaluated alternatives.
“What is the total training cost, including candidate generation, tracking, and progress scoring? Would the gains remain under a comparison that gives ordinary demonstration training the same computational budget?”
Response: We agree that equal optimizer updates alone are insufficient. We therefore extend GT FM to 23 epochs (Section 5.3, Table 6):
| Training strategy | Estimated GPU-hours | Clean (%) | Randomized (%) |
|---|---|---|---|
| GT FM (23 epochs) | 4,451.60 | 90.8 | 90.9 |
| R²-WAM | 4,302.30 | 93.8 | 93.7 |
The reported R²-WAM training cost includes the five-epoch GT FM initialization and both post-training stages, including feedback generation, tracking, progress scoring and parameter updates. Predictive repair and action rejection account for 1,429.0 and 1,905.6 GPU-hours, respectively, on 16 NVIDIA A800 GPUs (Appendix E.3, Table 11).
Extending joint GT FM from 7 to 23 epochs increases success from 90.6%/90.4% to 90.8%/90.9%, an improvement of 0.2/0.5 percentage points (Tables 1 and 6). R²-WAM exceeds the 23-epoch baseline by 3.0/2.8 percentage points with slightly lower estimated computation. Thus, the gains from repair and rejection are not matched by extending GT FM to a comparable computational budget in this comparison.
Author response to Reviewer FX3T - Part 3
“At the simulation training threshold, negative-selection precision is 69.8% against execution-based STEAM judgments. What is the effect of incorrectly selected negatives on policy training? Is there evidence that these judgments agree with eventual task success, beyond agreement with the same progress scorer?”
Response: We acknowledge the remaining disagreement. At the simulation margin of 0.08, 382 of 547 selected candidates also meet the rejection criterion under execution-based STEAM judgments, giving 69.8% precision; the remaining 165 do not (Section 5.2; Appendix E.6, Table 17). These labels measure relative progress on actual executions, not eventual task success. The diagnostic uses newly collected trajectories excluded from scorer training and policy post-training.
Erroneous selections can suppress suitable actions. We retain GT FM and cap the action negative-loss contribution at 15% of GT FM, but this does not eliminate false rejection. With the repaired video expert fixed, adding action rejection increases success from 91.9%/91.4% to 93.8%/93.7% (Table 3). This demonstrates a net policy benefit, not an isolated assessment of erroneous selections. Section 6 now states this distinction explicitly.
“Can actions that make less short-term progress than the demonstration still lead to successful completion? Could necessary preparatory movements or alternative strategies be incorrectly penalized, particularly in long-horizon tasks?”
Response: We agree that lower progress within the prediction horizon need not imply lower long-term action value. Our criterion compares candidate and demonstrated progress over a fixed prediction window. It may therefore penalize preparatory motions or slower alternatives whose benefits emerge beyond that window, even when they ultimately succeed. Predictive repair improves the consistency of predicted consequences with the conditioning actions, but does not resolve this mismatch between local progress and eventual task success. We have added this limitation to Section 6.
“What determines the rejection margins and weighting scales, including their different values in simulation and real-world experiments? Beyond the selection metrics reported at several margins, how sensitive is final policy success to these choices?”
Response: The rejection margin and weighting scale were selected empirically through small-scale pilot experiments conducted separately in simulation and on the real robot, then fixed across tasks and evaluation conditions within each domain. Appendix G, Tables 23–24 report sensitivity in final policy success.
With the repaired video expert and action initialization fixed, we vary one action-rejection parameter at a time around the original simulation configuration . Testing and yields success rates of 93.4–94.1% in clean environments and 93.2–93.9% under randomization. Every tested configuration exceeds GT FM with the repaired video expert (91.9%/91.4%), indicating that the benefit is not confined to the original parameter values.
On Fold Shirt, each configuration is evaluated over 80 autonomous trials across the four existing conditions:
| Success (%) | ||
|---|---|---|
| 0.05 | 0.04 | 87.50 |
| 0.08 | 0.04 | 85.00 |
| 0.05 | 0.10 | 81.25 |
| 0.08 | 0.10 | 82.50 |
Across the four configurations, success ranges from 81.25% to 87.50%. Directly using the simulation values and achieves 82.50% success on Fold Shirt.
Author response to Reviewer FX3T - Part 1
Real-robot videos are available at the following URL, also included in the revised abstract:
https://anonymous.4open.science/r/submission-30413-videos-8039/index.html
We thank the reviewer for the constructive feedback. The revised manuscript incorporates additional comparisons and an explicit discussion of limitations, with revisions marked in blue.
“What is the main new insight beyond applying DiffusionNFT to Fast-WAM’s video and action experts with different negative-selection rules? What specifically motivates NFT for these two stages?”
Response: The central insight is that improving action consistency in video prediction can support action learning through both the representations used for action generation and the consequence feedback used for action post-training. We implement this through reference-based negative-selection rules that define the video and action post-training objectives, applied sequentially as predictive repair followed by action rejection (Sections 4.1–4.2). This design uses observed demonstration outcomes to repair action-inconsistent predictions before employing the repaired predictor to construct feedback for policy-sampled actions.
Predictive repair uses observed demonstration futures to identify generated motion inconsistent with the conditioning action. Action rejection then uses the repaired predictor to select policy samples whose predicted progress falls below the demonstrated reference by more than a prescribed margin. We follow STEAM to train the progress evaluator and keep it frozen during action post-training.
With the action objective fixed to GT FM, repair increases success from 90.8%/90.5% to 91.9%/91.4% in clean/randomized environments (Section 5.3, Table 3), consistent with improved representations for action generation. On fixed policy candidates, negative-selection precision increases from 64.6% to 69.8% and recall from 35.5% to 50.6% at the training margin of 0.08 (Section 5.2, Table 2). Action rejection yields gains of 1.9/2.3 percentage points after repair, versus 0.3/0.2 without repair (Table 3), supporting the value of prediction repair for consequence-based policy refinement.
Why NFT in both stages?
Both stages retain observed demonstrations as positive GT FM targets. A high AF score assesses robot-motion alignment but does not establish correct object dynamics or other generated content. Likewise, a high imagined progress score depends on both the video predictor and STEAM; promoting such actions would make positive supervision depend on this imperfect evaluation chain. We therefore use these scores for selective negative updates, implemented through DiffusionNFT’s implicit negative parameterization within the flow-matching objectives of both experts. With GT FM retained in all variants, positive-only and positive–negative fine-tuning achieve 92.1%/91.7% and 93.3%/92.9%, respectively, compared with 93.8%/93.7% for selective negative NFT (Appendix E.2, Table 10).