acceptodds
Under review as a conference paper at ICLR 2027

-WAM: Repair-and-Reject Post-Training for World Action Models

Abstract

World Action Models (WAMs) emerge as a promising foundation for policy refinement by predicting the consequences of sampled actions. However, visually plausible predictions can mislead policy refinement if they fail to reflect the input actions. To address this mismatch, we introduce -WAM, a two-stage repair-and-reject post-training framework that first improves the consistency of predicted futures with input actions, then uses these futures to select inferior action samples for negative fine-tuning. The repair stage grounds imagination in observed robot behavior through a kinematic alignment score that measures agreement between predicted and demonstrated motion, enabling the predicted video to faithfully reflect its input actions. Using the repaired video model, the rejection stage compares imagined outcomes of sampled and demonstrated actions, selectively applying negative fine-tuning to samples whose predicted task progress falls below the demonstrated reference by a prescribed margin. Together, the two stages extend video prediction from representation learning to consequence-based policy refinement without additional environment interaction or changes to the inference procedure. -WAM achieves 93.8% average success on RoboTwin 2.0 across clean and randomized settings. On the long-horizon real-world Fold Shirt task, it achieves 87.5% average success, compared with 0% for Fast-WAM.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, world action models, post-training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FX3TOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes R²-WAM, a two-stage offline post-training framework for world action models built on Fast-WAM. Both stages combine demonstration-based flow matching with the existing DiffusionNFT negative fine-tuning objective. First, the video expert is refined by penalizing generated futures whose robot motion disagrees with demonstrated motion, measured through kinematic alignment and point tracking. The repaired video expert is then frozen and used to predict outcomes for sampled and demonstrated actions. A STEAM-style progress scorer identifies sampled actions with inferior predicted progress, which become negatives for action-policy fine-tuning. The framework requires no additional environment interaction during post-training and retains the original inference procedure. Evaluation covers 50 RoboTwin 2.0 tasks and two real robot tasks.

Critical strengths
  • Strong empirical results. Evaluation spans 50 simulated tasks and two real robot tasks, including unseen objects and positions. The reported shirt-folding improvement is substantial.
  • Meaningful ablations. The experiments separately examine video repair, action rejection, negative selection, and alternative repair scores, supporting the contribution of both stages.
  • Practical deployment. Post-training reuses existing demonstrations without additional environment interaction, and deployment retains the original inference procedure.
  • Direct evaluation of prediction reliability. The paper compares imagined consequences with actual simulator executions, providing evidence that repair improves the reliability of action-negative selection.
Critical weaknesses
  • Limited methodological novelty. The framework applies the existing DiffusionNFT objective to Fast-WAM’s video and action experts, alongside an existing progress-scoring approach. The contribution is primarily the negative-selection rules and their sequential application.
  • Unclear benefit relative to training cost. The method adds video generation, tracking, and progress scoring. Comparisons with equal optimizer updates do not establish an advantage under equal computational budgets.
  • Remaining errors in training feedback. Negative selection depends on both generated videos and a learned scorer. Reported selection precision is 69.8% at the simulation training threshold, leaving substantial disagreement with execution-based judgments.
  • Potential bias toward demonstrated short-term progress. Actions with lower immediate progress than the demonstration are penalized, even though slower or alternative strategies may ultimately succeed. This limitation is not sufficiently examined.
  • Incomplete justification of the training procedure. Component ablations support combining repair and rejection, but do not establish whether the sequential schedule is preferable to joint or alternating training with the same objectives.
Questions for authors
  1. Methodological contribution. What is the main new insight beyond applying DiffusionNFT to Fast-WAM’s video and action experts with different negative-selection rules? What specifically motivates NFT for these two stages?

  2. Training schedule. Does sequential repair followed by rejection outperform joint or alternating training with the same objectives? The component ablations support using both stages, but do not isolate the benefit of their ordering and freezing strategy.

  3. Computational efficiency. What is the total training cost, including candidate generation, tracking, and progress scoring? Would the gains remain under a comparison that gives ordinary demonstration training the same computational budget?

  4. Reliability of negative labels. At the simulation training threshold, negative-selection precision is 69.8% against execution-based STEAM judgments. What is the effect of incorrectly selected negatives on policy training? Is there evidence that these judgments agree with eventual task success, beyond agreement with the same progress scorer?

  5. Bias toward immediate progress. Can actions that make less short-term progress than the demonstration still lead to successful completion? Could necessary preparatory movements or alternative strategies be incorrectly penalized, particularly in long-horizon tasks?

  6. Threshold selection. What determines the rejection margins and weighting scales, including their different values in simulation and real-world experiments? Beyond the selection metrics reported at several margins, how sensitive is final policy success to these choices?

5 comments
AuthorsOpenReview ↗

Author response to Reviewer FX3T - Part 1

Real-robot videos are available at the following URL, also included in the revised abstract:

https://anonymous.4open.science/r/submission-30413-videos-8039/index.html

We thank the reviewer for the constructive feedback. The revised manuscript incorporates additional comparisons and an explicit discussion of limitations, with revisions marked in blue.

“What is the main new insight beyond applying DiffusionNFT to Fast-WAM’s video and action experts with different negative-selection rules? What specifically motivates NFT for these two stages?”

Response: The central insight is that improving action consistency in video prediction can support action learning through both the representations used for action generation and the consequence feedback used for action post-training. We implement this through reference-based negative-selection rules that define the video and action post-training objectives, applied sequentially as predictive repair followed by action rejection (Sections 4.1–4.2). This design uses observed demonstration outcomes to repair action-inconsistent predictions before employing the repaired predictor to construct feedback for policy-sampled actions.

Predictive repair uses observed demonstration futures to identify generated motion inconsistent with the conditioning action. Action rejection then uses the repaired predictor to select policy samples whose predicted progress falls below the demonstrated reference by more than a prescribed margin. We follow STEAM to train the progress evaluator and keep it frozen during action post-training.

With the action objective fixed to GT FM, repair increases success from 90.8%/90.5% to 91.9%/91.4% in clean/randomized environments (Section 5.3, Table 3), consistent with improved representations for action generation. On fixed policy candidates, negative-selection precision increases from 64.6% to 69.8% and recall from 35.5% to 50.6% at the training margin of 0.08 (Section 5.2, Table 2). Action rejection yields gains of 1.9/2.3 percentage points after repair, versus 0.3/0.2 without repair (Table 3), supporting the value of prediction repair for consequence-based policy refinement.

Why NFT in both stages?

Both stages retain observed demonstrations as positive GT FM targets. A high AF score assesses robot-motion alignment but does not establish correct object dynamics or other generated content. Likewise, a high imagined progress score depends on both the video predictor and STEAM; promoting such actions would make positive supervision depend on this imperfect evaluation chain. We therefore use these scores for selective negative updates, implemented through DiffusionNFT’s implicit negative parameterization within the flow-matching objectives of both experts. With GT FM retained in all variants, positive-only and positive–negative fine-tuning achieve 92.1%/91.7% and 93.3%/92.9%, respectively, compared with 93.8%/93.7% for selective negative NFT (Appendix E.2, Table 10).

AuthorsOpenReview ↗

Author response to Reviewer FX3T - Part 2

“Does sequential repair followed by rejection outperform joint or alternating training with the same objectives? The component ablations support using both stages, but do not isolate the benefit of the ordering and freezing strategy.”

Response: We add comparisons using the same repair and rejection objectives, with two epochs allocated to each expert (Section 5.3, Table 5; Appendix E.3):

Training scheduleClean (%)Randomized (%)
Joint92.792.4
Alternating93.293.3
Sequential93.893.7

Joint training updates both experts concurrently; alternating training switches between repair and rejection after each epoch; sequential training completes repair before rejection and freezes the video expert during rejection (Appendix E.3).

Sequential training yields higher success in both settings. It allows action rejection to use the repaired predictor from the outset and keeps that predictor fixed throughout action post-training. These results support the sequential schedule among the evaluated alternatives.

“What is the total training cost, including candidate generation, tracking, and progress scoring? Would the gains remain under a comparison that gives ordinary demonstration training the same computational budget?”

Response: We agree that equal optimizer updates alone are insufficient. We therefore extend GT FM to 23 epochs (Section 5.3, Table 6):

Training strategyEstimated GPU-hoursClean (%)Randomized (%)
GT FM (23 epochs)4,451.6090.890.9
R²-WAM4,302.3093.893.7

The reported R²-WAM training cost includes the five-epoch GT FM initialization and both post-training stages, including feedback generation, tracking, progress scoring and parameter updates. Predictive repair and action rejection account for 1,429.0 and 1,905.6 GPU-hours, respectively, on 16 NVIDIA A800 GPUs (Appendix E.3, Table 11).

Extending joint GT FM from 7 to 23 epochs increases success from 90.6%/90.4% to 90.8%/90.9%, an improvement of 0.2/0.5 percentage points (Tables 1 and 6). R²-WAM exceeds the 23-epoch baseline by 3.0/2.8 percentage points with slightly lower estimated computation. Thus, the gains from repair and rejection are not matched by extending GT FM to a comparable computational budget in this comparison.

AuthorsOpenReview ↗

Author response to Reviewer FX3T - Part 3

“At the simulation training threshold, negative-selection precision is 69.8% against execution-based STEAM judgments. What is the effect of incorrectly selected negatives on policy training? Is there evidence that these judgments agree with eventual task success, beyond agreement with the same progress scorer?”

Response: We acknowledge the remaining disagreement. At the simulation margin of 0.08, 382 of 547 selected candidates also meet the rejection criterion under execution-based STEAM judgments, giving 69.8% precision; the remaining 165 do not (Section 5.2; Appendix E.6, Table 17). These labels measure relative progress on actual executions, not eventual task success. The diagnostic uses newly collected trajectories excluded from scorer training and policy post-training.

Erroneous selections can suppress suitable actions. We retain GT FM and cap the action negative-loss contribution at 15% of GT FM, but this does not eliminate false rejection. With the repaired video expert fixed, adding action rejection increases success from 91.9%/91.4% to 93.8%/93.7% (Table 3). This demonstrates a net policy benefit, not an isolated assessment of erroneous selections. Section 6 now states this distinction explicitly.

“Can actions that make less short-term progress than the demonstration still lead to successful completion? Could necessary preparatory movements or alternative strategies be incorrectly penalized, particularly in long-horizon tasks?”

Response: We agree that lower progress within the prediction horizon need not imply lower long-term action value. Our criterion compares candidate and demonstrated progress over a fixed prediction window. It may therefore penalize preparatory motions or slower alternatives whose benefits emerge beyond that window, even when they ultimately succeed. Predictive repair improves the consistency of predicted consequences with the conditioning actions, but does not resolve this mismatch between local progress and eventual task success. We have added this limitation to Section 6.

“What determines the rejection margins and weighting scales, including their different values in simulation and real-world experiments? Beyond the selection metrics reported at several margins, how sensitive is final policy success to these choices?”

Response: The rejection margin and weighting scale were selected empirically through small-scale pilot experiments conducted separately in simulation and on the real robot, then fixed across tasks and evaluation conditions within each domain. Appendix G, Tables 23–24 report sensitivity in final policy success.

With the repaired video expert and action initialization fixed, we vary one action-rejection parameter at a time around the original simulation configuration (ma,τa)=(0.08,0.10)(m_a,\tau_a)=(0.08,0.10). Testing ma∈{0.04,0.08,0.12}m_a \in \{0.04,0.08,0.12\} and τa∈{0.04,0.10,0.20}\tau_a \in \{0.04,0.10,0.20\} yields success rates of 93.4–94.1% in clean environments and 93.2–93.9% under randomization. Every tested configuration exceeds GT FM with the repaired video expert (91.9%/91.4%), indicating that the benefit is not confined to the original parameter values.

On Fold Shirt, each configuration is evaluated over 80 autonomous trials across the four existing conditions:

mam_aτa\tau_aSuccess (%)
0.050.0487.50
0.080.0485.00
0.050.1081.25
0.080.1082.50

Across the four configurations, success ranges from 81.25% to 87.50%. Directly using the simulation values ma=0.08m_a = 0.08 and τa=0.10\tau_a = 0.10 achieves 82.50% success on Fold Shirt.

Reviewer_AeGLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper post-trains a world action model in two stages. The repair stage samples futures under each demonstrated action, measures their agreement with the observed robot motion according to a kinematic alignment score, and uses predictions that fall short of the observed reference as negatives in a DiffusionNFT-style objective. The rejection stage then freezes the repaired video expert, predicts task progress for sampled and demonstrated actions with a STEAM model, and negatively fine-tunes sampled actions whose imagined progress falls below the demonstration by a margin. Extensive experiments show good results.

Critical strengths
  1. The ablation studies are very comprehensive. With the action objective fixed to demonstration flow matching, repair alone increases success rate from 90.8 to 91.9 in clean scenes. With the video expert fixed to continued flow matching, rejection alone increases it from 90.8 to 91.1. Together they achieve 93.8 and 93.7, a gain of 3.0 and 3.2 over the base. It confirms that the omitted stage still receives two epochs of flow matching, so this is not a comparison against an undertrained baseline.
  2. The paper is well preseted. The paper is very easy to follow with dedicated figures. The methods are clearly illustrated and the experiments prove most of the points that the paper raise in the introduction part.
  3. The real-world evaluation is well-designed. The experiments include four conditions vary position and object instance one factor at a time, every cell contains 20 autonomous trials, success criteria were fixed before evaluation and applied identically. In addition, Table 10 gives Wilson intervals for every cell.
Critical weaknesses
  1. The 3.3-point performance enhancement is measured against a base that is worse than the published model. Table 1 has Fast-WAM at 91.9 and action-conditioned Fast-WAM at 90.5, and the paper says the published model is action-unconditioned. However, the reason why adding action input will damage the Fast-WAM is not explained and it makes me wonder how much is the real contribution that repair/rejction contributes to.
  2. The repair score is 2D, not 3D. To compare a generated future with the demonstration, the paper tracks the arm in the head camera and then reconstructs its 3D position using the depth from the demonstration, not the depth in the generated video (Eq. 15). The two positions therefore always have the same depth, and the score can only see errors that move the arm sideways in the image and cannot see its actual position in the 3D space.
  3. The experiments that are related to the Repair are not shown in the real robot. Table 6’s tracking success and action-following failure rate exist only for RoboTwin validation. In addition, the hyperparameters for rejection are not the same for simulator and real robot. It seems like that you need to set a specific number for various conditions, which weakens its application scope. And the authors do not make any explanation, nor any sensitive analysis.
Questions for authors
  1. Why is action-conditioned Fast-WAM 1.4 points below the published Fast-WAM?
  2. Can you explain why the repair score can reflect the point you make as it has some shortages in the 3D space?
  3. PSNR-threshold repair already reaches 92.1 and 91.8, and the kinematic score adds 1.7 and 1.9. Given the tracker, kinematic model and camera calibration the score needs, what does the extra 1.7 buy?
  4. On the real robot, what are the tracking success rate and AF failure rate for the continued-FM and repaired video experts on Fold Shirt and Clean Table?
  5. Can you break the RoboTwin result down by task? How many of the 50 tasks improve over action-conditioned Fast-WAM, how many are flat, how many regress, and is the gain concentrated in tasks with large wrist displacement?
3 comments
AuthorsOpenReview ↗

Author response to Reviewer AeGL - Part 1

Real-robot videos are available at the following URL, also included in the revised abstract:

https://anonymous.4open.science/r/submission-30413-videos-8039/index.html

We thank the reviewer for the constructive assessment. The revised manuscript includes the additional experiments below, with revisions marked in blue.

“Why is action-conditioned Fast-WAM 1.4 points below the published Fast-WAM?”

Response: The 91.9% average for Fast-WAM is quoted from the original publication, whereas the 90.5% action-conditioned result was obtained under our training configuration and computational budget (Appendix C). The 1.4-point gap may therefore reflect differences between the two experimental setups, in addition to any effect of action conditioning. We have added a Fast-WAM reproduction to enable comparison within our experimental setup (Table 1):

MethodClean (%)Randomized (%)
Fast-WAM (reported)91.991.8
Fast-WAM (reproduced)90.890.6
Action-conditioned Fast-WAM90.690.4
R²-WAM93.893.7

Within our experimental setup, average success is 90.7% for reproduced Fast-WAM and 90.5% for action-conditioned Fast-WAM, a difference of 0.2 percentage points. R²-WAM improves upon our Fast-WAM reproduction by 3.0/3.1 percentage points, indicating gains beyond recovering this baseline difference.

The component ablation provides a controlled assessment of repair and rejection with the action-conditioned architecture, initialization and sequential training schedule held fixed (Section 5.3, Table 3). GT FM / GT FM achieves 90.8%/90.5%, compared with 93.8%/93.7% for Repair / Rejection. The resulting improvements of 3.0/3.2 percentage points support the contribution of the proposed post-training objectives.

“Can you break the RoboTwin result down by task? How many of the 50 tasks improve over action-conditioned Fast-WAM, how many are flat, how many regress, and is the gain concentrated in tasks with large wrist displacement?”

Response: Appendix H, Table 25 reports results for all 50 tasks. Relative to action-conditioned Fast-WAM, the numbers of improved, unchanged and regressed tasks are:

SettingImprovedUnchangedRegressed
Clean32171
Randomized34142
Average36122

Tasks involving substantial wrist motion also improve. Relative to action-conditioned Fast-WAM, average success increases from 41.0% to 66.0% on Open Microwave and from 59.5% to 71.0% on Hanging Mug, corresponding to gains of 25.0 and 11.5 percentage points, respectively. The two tasks with lower average success are Place Mouse Pad (87.0% to 83.5%) and Turn Switch (58.5% to 56.5%).

AuthorsOpenReview ↗

Author response to Reviewer AeGL - Part 2

“Can you explain why the repair score can reflect the point you make as it has some shortages in the 3D Space?”

Response: We agree that reference-depth lifting does not independently recover depth from the generated video. Equation 15 uses kinematic reference depths to express tracked projections in metric camera coordinates. It detects depth-related motion errors when they change landmark projections, but cannot distinguish 3D discrepancies that preserve those projections. We now explicitly state this limitation in Section 4.1 and Appendix A.2.

We additionally evaluate repair using Depth Anything 3 estimates instead of reference depths. A fixed additive offset aligns each landmark’s estimated depth with its kinematic depth at initialization; subsequent depth changes come from the video. All other repair and action post-training settings remain unchanged (Appendix A.2, Table 7).

Repair scoringClean (%)Randomized (%)
Reference depth93.893.7
Estimated depth93.893.8

The alternative depth treatment retains similar policy success while allowing visually estimated depth changes to enter the score.

“PSNR-threshold repair already reaches 92.1 and 91.8, and the kinematic score adds 1.7 and 1.9. Given the tracker, kinematic model and calibration the score needs, what does the extra 1.7 buy?”

Response: PSNR measures overall image similarity, whereas the kinematic score evaluates agreement between predicted robot motion and the conditioning action. In Table 12, AF failure rates are 18.6% for PSNR repair and 6.8% for kinematic repair. The following table compares policy success after applying the same action rejection procedure to both repaired models (Appendix E.4, Table 13; Appendix F.7, Table 22):

Repair scoreRoboTwin Clean (%)RoboTwin Randomized (%)Fold Shirt (%)Clean Table (%)
PSNR92.191.87.548.75
Kinematic alignment93.893.787.576.25

With the original aggregation rule used for predictive repair, kinematic repair improves policy success over PSNR repair by 1.7/1.9 percentage points in clean/randomized RoboTwin environments and by 80.0/27.5 percentage points on Fold Shirt/Clean Table, respectively.

The additional tracking, kinematic scoring and camera calibration are required for offline training during predictive repair. They introduce no additional computation during policy deployment, which retains the original inference procedure.

AuthorsOpenReview ↗

Author response to Reviewer AeGL - Part 3

“On the real robot, what are the tracking success rate and AF failure rate for the continued-FM and repaired video experts on Fold Shirt and Clean Table?”

Response: We evaluate both video experts on the same 605 Fold Shirt clips and 2,426 Clean Table clips, selecting one action chunk from each training episode and conditioning on identical initial observations and demonstrated actions (Appendix F.5, Table 20).

TaskVideo expertTracking success (%)AF failure (%)
Fold ShirtContinued FM94.230.4
Fold ShirtRepair98.78.3
Clean TableContinued FM94.836.2
Clean TableRepair99.18.5

Using the original AF definitions, these diagnostics show improved trackability and action consistency on real-world training demonstrations; they do not establish generalization to held-out trajectories.

“In addition, the hyperparameters for rejection are not the same for simulator and real robot. It seems like that you need to set a specific number for various conditions, which weakens its application scope. And the authors do not make any explanation, nor any sensitive analysis.”

Response: The rejection margin and weighting scale were selected empirically through small-scale pilot experiments conducted separately in simulation and on the real robot, then fixed across tasks and evaluation conditions within each domain. Appendix G, Tables 23–24 report sensitivity in final policy success.

We repeat action post-training with the repaired video expert and action initialization fixed. In simulation, we vary one parameter at a time, testing mam_a at 0.04, 0.08 and 0.12, and τa\tau_a at 0.04, 0.10 and 0.20. Success ranges from 93.4% to 94.1% in clean environments and from 93.2% to 93.9% under randomization. Every tested configuration exceeds GT FM with the repaired expert (91.9%/91.4%), showing that the benefit is not confined to the original configuration.

On Fold Shirt, each configuration receives 80 autonomous trials across the four evaluation conditions:

mam_aτa\tau_aSuccess (%)
0.050.0487.50
0.080.0485.00
0.050.1081.25
0.080.1082.50

Across the four configurations, success ranges from 81.25% to 87.50%. Directly using the simulation values ma=0.08m_a = 0.08 and τa=0.10\tau_a = 0.10 achieves 82.50% success on Fold Shirt.

Reviewer_jNKcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces R2R^2-WAM, a two-stage post-training framework for WAMs that improves a policy using only offline demonstrations, without additional environment interaction or changes at inference time. The method has two stages. The first stage checks whether the video expert’s predicted futures actually follow the input actions by comparing the robot motion tracked in generated videos with the motion recorded in the demonstrations. The second stage freezes the repaired video expert and uses it to imagine the outcomes of both policy-sampled and demonstrated actions. Experiments on RoboTwin 2.0 and two real-world bimanual tasks show improved success over both VLA and WAM baselines.

Critical strengths
  • Clear motivation. The point that visually plausible predictions can become misleading training signals when they fail to reflect the input actions is valid. The idea of using the observed futures of demonstrations as a test of action fidelity is simple and practical.
  • Economical design. The method reuses the WAM’s own video expert both to supply world representations and to predict imagined consequences, without additional environment interaction or changes to the inference procedure.
  • Clear writing. The paper is written intuitively and is easy to understand.
Critical weaknesses
  • Missing baselines. A baseline that selects negatives without a world model is needed but missing. Because the reference is always the demonstrated action from the same initial observation, ΔAi,k\Delta A_{i,k} is likely to be strongly correlated with ∥a^i,k−ai⋆∥2\lVert \hat{a}_{i,k} - a_i^{\star} \rVert_2. Without a baseline that rejects sampled actions far from the demonstrated action in action space, the contribution of the costly video imagination cannot be assessed. In addition, the paper does not compare with ways of using the imagined consequences other than as negatives. For instance, possible alternatives include fine-tuning that uses sampled actions predicted to outperform the demonstrated reference as positives, DPO-style methods, and imagination RL with the same world model-based rewards.
  • Indirect validation of the central claim. The authors claim that predictive repair is “a source of more reliable consequence feedback for negative action fine-tuning,” but predictive repair is trained and evaluated only on demonstrated actions. Action rejection, however, relies on predictions for policy-sampled actions that may deviate from the demonstrations.
  • Bias in the action-following score. RAFobsR^{\text{obs}}_{\text{AF}} uses the reliable landmark with the highest Si,jrelS^{\text{rel}}_{i,j}, whereas RAFgenR^{\text{gen}}_{\text{AF}} uses a reliability-weighted mean, so ΔRi,k\Delta R_{i,k} may be biased toward positive values. Even when a generated future is identical to the observed future, the difference in the relative-trajectory component is non-negative, so ΔRi,k>mv\Delta R_{i,k} > m_{\text{v}} can hold and the generated future can be selected as a negative. Since the same bias may also affect the AF failure rate in Table 6, the authors should report the distribution of ΔRi,k\Delta R_{i,k} when y^i,k=yi⋆\hat{y}_{i,k} = y_i^{\star} (noise floor).
  • Missing real-world robot videos. The authors must include videos of the real-robot experiments.
Questions for authors

See major concerns above. The authors must provide real-robot videos in a manner permitted by the ICLR guidelines. Please also fix the following typos:

  • “Action Geneneration” in Figure 2.
  • “comprising … comprising” in Line 59-60.
4 comments
AuthorsOpenReview ↗

Author response to Reviewer jNKc - Part 1

We thank the reviewer for the constructive feedback. We have incorporated the additional experiments and clarifications into the revised manuscript, with textual changes marked in blue.

“The authors claim that predictive repair is ‘a source of more reliable consequence feedback for negative action fine-tuning,’ but predictive repair is trained and evaluated only on demonstrated actions. Action rejection, however, relies on predictions for policy-sampled actions that may deviate from the demonstrations.”

Response: The evaluation supporting our consequence-feedback claim is not restricted to demonstrated actions. Although predictive repair is trained on demonstrated actions, our Action Negative Selection experiment directly evaluates its effect on feedback reliability for policy-sampled actions.

Across 500 contexts from all 50 RoboTwin tasks, we compare the continued FM and repaired video experts using the same 2,000 policy-sampled candidate actions. Each candidate and its demonstrated reference are also executed from the same restored simulator state. The same frozen STEAM evaluator scores imagined and actual consequences, providing execution-based reference labels for negative selection. These executions are used only for evaluation, not post-training.

At the training margin of 0.08, repair improves precision from 64.6% to 69.8% and recall from 35.5% to 50.6%. With candidate actions and the evaluator held fixed, these results directly support improved consequence feedback for negative selection on the evaluated policy samples. We have clarified this distinction in Section 5.2, Table 2, and Appendix E.6.

“The authors must provide real-robot videos in a manner permitted by the ICLR guidelines.”

“Please also fix the following typos:”

  • “Action Geneneration” in Figure 2.
  • “comprising … comprising” in Line 59-60.

Response: We apologize for omitting the real-robot videos from the original submission. They are now available at the following URL, also included in the revised abstract:

https://anonymous.4open.science/r/submission-30413-videos-8039/index.html

We have corrected “Action Geneneration” to “Action Generation” in Figure 2 and removed the repeated “comprising” in the Introduction.

AuthorsOpenReview ↗

Author response to Reviewer jNKc - Part 2

“A baseline that selects negatives without a world model is needed but missing.”

“In addition, the paper does not compare with ways of using the imagined consequences other than as negatives.”

Response: We compare action-distance rejection and alternative objectives using imagined feedback. All variants retain GT FM and use the same repaired video expert. Results and implementation details are provided in Section 5.3 and Appendix E.2, Tables 9–10.

Action objectiveClean (%)Randomized (%)
GT FM91.991.4
Action-distance rejection (mD=0.02m_D = 0.02)92.592.1
Positive-only fine-tuning92.191.7
Positive–negative fine-tuning93.392.9
DPO-FM92.492.5
RECAP + STEAM92.692.3
Selective negative NFT (ours)93.893.7

Action-distance rejection applies unit-weight NFT to samples whose RMSE from the demonstrated action exceeds a threshold, computed on normalized actions. For this baseline, we report the best result among the tested thresholds of 0.02, 0.05 and 0.08. Our action rejection achieves success rates 1.3/1.6 percentage points higher in clean/randomized environments. With the repaired video expert held fixed, this comparison supports the value of predicted consequences beyond action distance for negative selection.

Positive-only fine-tuning applies FM to samples predicted to outperform the demonstrated reference; positive–negative fine-tuning additionally suppresses inferior samples through NFT. DPO-FM implements progress-ranked preference optimization through flow-matching errors, while RECAP + STEAM uses progress-derived advantage conditioning (Section 5.3; Appendix E.2).

Selective negative NFT achieves the highest success among the evaluated alternatives, supporting consequence-based rejection with demonstrated actions retained as positive targets in this setting.

AuthorsOpenReview ↗

Author response to Reviewer jNKc - Part 3

“Even when a generated future is identical to the observed future, the difference in the relative-trajectory component is non-negative, so ΔRi,k>mv\Delta R_{i,k} > m_v can hold and the generated future can be selected as a negative.”

“Since the same bias may also affect the AF failure rate in Table 6, the authors should report the distribution of ΔRi,k\Delta R_{i,k} when y^i,k=yi⋆\hat{y}_{i,k} = y_i^\star (noise floor).”

Response: We thank the reviewer for identifying this issue. To clarify, both AF scores combine absolute-position and relative-trajectory consistency with equal weights. For the observed reference, the absolute-position component uses the score of the reliable landmark with the highest tracking reliability, which need not have the highest absolute-position consistency score. The relative-trajectory component instead uses the highest relative-trajectory consistency score among reliable landmarks. For generated futures, each component uses a reliability-weighted mean over reliable landmarks. Thus, the total self-comparison offset need not be nonnegative, but asymmetric aggregation can still select an identical prediction for negative fine-tuning. We acknowledge this limitation in Section 4.1 and examine its effects in Appendix E.5.

Reference self-comparison (Table 14).

For the 500 reference videos in the Table 12 evaluation set, we compute bib_i as the difference between the observed-reference and generated-future scoring functions applied to the same reference video, using identical tracking results.

StatisticValue
Mean0.0154
Median0.0079
95th percentile0.0484
Fraction with bi>mvb_i > m_v, where mv=0.02m_v = 0.027.20%

The self-comparison offset exceeds the repair margin of 0.02 for 7.2% of these references, even though the two scoring functions receive identical videos and tracking results. This proportion does not represent the misselection rate among actual training candidates.

Symmetric evaluation (Table 15).

We recompute AF failure using reliability-weighted landmark averages for each score component in both observed and generated futures, retaining the equal weighting of the two components. The evaluation set, tracking results and failure threshold remain unchanged.

Video expertOriginal AF failure (%)Symmetric AF failure (%)
Continued FM32.025.2
Repair with PSNR18.615.8
Without relative consistency17.011.4
Without absolute consistency10.66.6
Full repair6.83.2

Symmetric aggregation lowers the reported AF failure rates but preserves the model ranking, with full repair retaining a lower failure rate than continued FM (3.2% versus 25.2%).

Symmetric training (Table 16).

We additionally train predictive repair with this symmetric aggregation rule, followed by action rejection.

Training configurationClean (%)Randomized (%)
GT FM / GT FM90.890.5
Symmetric repair / Rejection92.691.9
Original repair / Rejection93.893.7

Symmetric repair followed by action rejection improves success over GT FM / GT FM by 1.8/1.4 percentage points, while success remains 1.2/1.8 percentage points below that obtained with the original aggregation. Policy improvements therefore persist under symmetric aggregation, although the original aggregation yields higher success at the same repair margin.

Reviewer_jNKcOpenReview ↗

Response to Rebuttal

Thank you for the response. However, I am still uncertain whether the best results come from the proposed method itself or partly from the biased scoring rule. Once the bias is removed, the gains shrink noticeably, and it is not clear why the bias helps or whether the advantage over simpler baselines would still hold. Given this uncertainty, I will keep my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Updates for R²-WAM

We thank the Area Chair and all three reviewers for their feedback. R²-WAM first uses observed demonstrations to repair video predictions that are inconsistent with the conditioning action. It then uses the repaired predictor to compare the consequences of sampled and demonstrated actions and select negatives for action post-training. We summarize below what the revised manuscript and subsequent discussion add to the evidence for these two stages. Paired RoboTwin results are reported in clean/randomized order.

1. Policy gains under stronger baselines and comparable training cost (jNKc, AeGL, FX3T). Under our training setup, reproduced Fast-WAM and action-conditioned Fast-WAM differ by only 0.2 percentage points in average success (90.7% vs. 90.5%). R²-WAM exceeds the reproduced model by 3.0/3.1 points. Relative to action-conditioned Fast-WAM, average success improves on 36 of 50 tasks, is unchanged on 12, and decreases on 2 (Tables 1 and 25).

With the repaired video expert held fixed, consequence-based rejection reaches 93.8%/93.7%, compared with 92.5%/92.1% for the best tested action-distance baseline. It also achieves higher success than the positive-only, positive–negative, DPO-FM, and RECAP + STEAM alternatives tested (Tables 9–10). With the same repair and rejection objectives, sequential training outperforms joint and alternating training. Extending demonstration flow matching to 23 epochs yields 90.8%/90.9% at 4,451.6 estimated GPU-hours, versus 93.8%/93.7% at 4,302.3 GPU-hours for R²-WAM, including feedback generation and scoring (Tables 5–6 and 11).

2. Reliability of feedback on policy-sampled actions (jNKc, FX3T). Although repair is trained on demonstrations, its effect on negative selection is evaluated on 2,000 fixed policy-sampled actions. Against STEAM judgments on actual executions, repair raises precision from 64.6% to 69.8% and recall from 35.5% to 50.6% (Table 2). The downstream comparison shows why the stages are linked: action rejection improves policy success by 0.3/0.2 points without repair and by 1.9/2.3 points after repair (Table 3).

In our follow-up to FX3T, we evaluated the same candidates using RoboTwin’s terminal success criterion, independently of STEAM. With a frozen R²-WAM policy continuing each rollout where needed, success was 90.3% for candidates selected for rejection and 93.0% for those not selected. Even when both predicted and executed progress met the rejection criterion, 89.3% of rollouts succeeded. Rejection thus identifies a shortfall in progress relative to the demonstration, not an action from which task completion is impossible.

3. What the kinematic repair score measures (jNKc, AeGL). We quantified the offset introduced by asymmetric landmark aggregation: 7.2% of 500 reference videos exceed the repair margin when scored against themselves. Applying symmetric aggregation to the same predictions preserves the model ranking. Training with symmetric aggregation also improves policy success over sequential demonstration flow matching (92.6%/91.9% vs. 90.8%/90.5%), though it falls below the original rule (Tables 14–16). We also clarified that reference-depth scoring cannot distinguish 3D errors that preserve landmark projections. Substituting video-estimated depth yields similar policy success (93.8%/93.8%; Table 7), without establishing full depth observability.

4. Real-robot evidence and parameter sensitivity (jNKc, AeGL, FX3T). On real-robot training demonstrations, repair reduces action-following failure from 30.4% to 8.3% on Fold Shirt and from 36.2% to 8.5% on Clean Table (Table 20). In separate autonomous trials with the same action rejection procedure, kinematic repair achieves 87.5% and 76.25% success on these tasks, respectively, compared with 7.5% and 48.75% for PSNR repair (Table 22). The additional tracking and kinematic scoring are used during offline training, not deployment.

Rejection parameters were selected in small pilot experiments separately in simulation and on the real robot, then fixed across tasks within each domain. Across the tested settings, success ranges from 93.4–94.1% in clean simulation and 93.2–93.9% under randomization. On Fold Shirt, success ranges from 81.25–87.50%, with 82.50% obtained by directly transferring the simulation parameters (Tables 23–24). The revised abstract links the real-robot videos, and we corrected the typographical errors in Figure 2 and the Introduction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.