acceptodds
Under review as a conference paper at ICLR 2027

ActForge: Action-Faithful Robot World Model with Counterfactual Feedback

Abstract

We present ActForge, a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. Training such a model on existing robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline geometric calibration to align these conditions with the target videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired ground-truth futures, we construct a human-annotated video dataset covering robot, object, and interaction defects and use it to train an embodied video reward model. Its scores guide reinforcement-learning post-training toward more physically plausible interaction outcomes. On AgiBot, ActForge attains state-of-the-art action following, while reducing the human-assessed interaction defect rate from from 48.12% to 6.25%. Notably, our model ranks first in the world model track of the AgiBot World Challenge 2026.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot World Models, Action Conditioning, Video Rewards

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qUPGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ActForge is a multi-view robot video world model trained across four datasets and five arm types. It first refines camera geometry from existing recordings and represents actions using calibrated robot renderings and camera-ray features. It then post-trains on recorded and counterfactual action conditions using a video reward model learned from human annotations of embodiment, object, and interaction defects. On AgiBot, action following reaches an nDTW of 0.8772, while post-training reduces human-assessed interaction defects from 48.12% to 6.25%. An additional RoboTwin study reports improved prediction of policy outcomes. The work addresses useful problems in action conditioning and interaction modeling, but its strongest evidence concerns visual consistency rather than independently verified counterfactual dynamics.

Critical strengths
  1. The geometric calibration component addresses a concrete obstacle to learning from heterogeneous robot data. Misalignment between rendered action conditions and recorded observations can directly undermine action following. The paper evaluates calibration at substantial scale and separates inference-only refinement from refinement during both training and inference. The resulting improvements in trajectory alignment and reconstruction support the practical value of this component, and the proposed release of calibration parameters could benefit subsequent work.
  2. The human evaluation provides credible evidence of reduced visible defects. All 960 counterfactual test videos receive three independent, blinded assessments, with agreement statistics and confidence intervals reported in the appendix. The reduction from 48.12% to 6.25% in interaction defects relative to Stage I is substantial. The study also includes recorded-action fidelity, a separate simulated outcome-prediction evaluation, and a large annotation resource, giving the contribution value beyond selected qualitative examples.
Critical weaknesses
  1. The evaluation does not sufficiently establish physical fidelity under counterfactual controls. The reward model receives only the generated video and a fixed prompt, without the commanded action condition (Appendix B.1). It can therefore encourage visibly coherent interactions without directly verifying that they are the outcomes of the supplied actions. The AgiBot counterfactual benchmark likewise lacks paired physical futures, and nDTW evaluates gripper-trajectory agreement rather than object-state correctness. The RoboTwin policy-outcome study is useful, but covers three tasks and replays complete action sequences obtained from simulator executions; it does not test closed-loop policies acting on generated observations. The real-world OOD examples are predictions from recordings rather than controlled hardware interventions. Consequently, the evidence supports improved visual plausibility more strongly than the broader use of the model as a reliable physical simulator.
  2. The experiments do not isolate the claimed contribution of counterfactual feedback. Stage II jointly introduces counterfactual conditions, multiple defect rewards, and recorded-action PSNR rewards, but no matched recorded-action-only post-training control or reward-channel ablation establishes which changes produce the gains. The policy-outcome study compares the full model with Ctrl-World without including Stage I, so it also cannot attribute the downstream improvement to counterfactual post-training. Furthermore, the strongest external baseline already has a 7.50% interaction-defect rate versus 6.25% for ActForge, with overlapping reported confidence intervals; this requires the paired comparison rather than a claim based on the large Stage-I reduction. These gaps leave the distinctive benefit of the proposed training strategy insufficiently established, leading me to weak rejection.
Questions for authors
  1. Can you validate predicted object outcomes against actual executions of matched counterfactual actions, including near-miss grasps and partial contacts? A controlled hardware subset would clarify the relevance of the visual defect results to real interactions.
  2. Under matched data and optimization budgets, how do recorded-action-only post-training, recorded-plus-counterfactual post-training, and removal of individual reward channels compare?
  3. Can you include Stage I in the policy-outcome evaluation and report the paired statistical comparison with GE-Sim 2.0 on interaction defects? These controls would help separate the value of calibration, general post-training, and counterfactual feedback.
2 comments
AuthorsOpenReview ↗

Response to Reviewer qUPG (Part 1)

Response to Reviewer qUPG (Part 1)

We thank you for the constructive suggestions on validating predicted outcomes and separating the effects of post-training. We added controlled real-world validation, training ablations, and a Stage-I policy-outcome comparison, and report the paired defect analysis with GE-Sim 2.0. We address Q1-Q2 below and Q3 in Part 2.

Q1: Real-world validation under matched counterfactual actions.

Can you validate predicted object outcomes against actual executions of matched counterfactual actions, including near-miss grasps and partial contacts? A controlled hardware subset would clarify the relevance of the visual defect results to real interactions.

We added a controlled hardware evaluation to compare predicted lift outcomes with actual executions under the same perturbed actions (Table 4). The results support more accurate prediction of action-dependent grasp outcomes after post-training, particularly when the robot fails to lift the object. We evaluate three tasks, each containing 10 lift and 10 non-lift cases, for a total of 60 cases. The cases cover failure modes including missed grasps and partial contacts. Different action variants are executed from matched initial states, and both models are evaluated on exactly the same cases.

MethodAccuracy on non-lift cases (%)Accuracy on lift cases (%)Overall accuracy (%)
Ours (w/o RL)60.00100.0080.00
Ours (full)100.0096.6798.33

The model without RL incorrectly predicts lifting in 12 of the 30 actual non-lift cases. The full model correctly predicts all 30 non-lift cases and 29 of the 30 lift cases. We also provide qualitative examples of contact-induced object displacement when the gripper contacts and pushes objects during closure (Figures 17 and 18). The controlled outcome comparison and contact examples provide execution-based evidence supporting the physical relevance of the visual-defect improvements.

Q2: Recorded-only training, counterfactual training, and reward ablations.

Under matched data and optimization budgets, how do recorded-action-only post-training, recorded-plus-counterfactual post-training, and removal of individual reward channels compare?

Thank you for suggesting these controls. We added two recorded-action-only variants to clarify the roles of reward supervision and counterfactual action coverage (Table 8). Embodied feedback improves interaction quality on recorded actions, with substantially greater gains when this feedback is extended to counterfactual actions, while preserving gripper action following.

All RL variants use the same source recordings, Stage-I checkpoint, training steps, batch size, and other configurations, differing only in action sampling and rewards. All models are evaluated on the same 160 counterfactual conditions. The embodied reward combines robot-embodiment, object-consistency, and interaction-plausibility feedback; PSNR applies only to recorded actions with paired ground-truth futures. These controls isolate the embodied reward as a whole, rather than the individual effects of its three channels.

SettingActions used for RLRewardObject defects (%) ↓Interaction defects (%) ↓nDTW ↑
Stage I (w/o RL)——31.8848.120.8735
Recorded-only RLRecordedPSNR30.0045.000.8821
Recorded-only RLRecordedPSNR + embodied24.3842.500.8792
Full modelRecorded + counterfactualPSNR + embodied3.126.250.8831

Embodiment defects are 0.00% throughout.

(1) Embodied feedback. On recorded actions, adding embodied feedback to PSNR reduces object defects from 30.00% to 24.38% and interaction defects from 45.00% to 42.50%, supporting its benefit under fixed action coverage.

(2) Counterfactual training. With the reward definitions fixed, extending training to counterfactual actions further reduces object defects from 24.38% to 3.12% and interaction defects from 42.50% to 6.25%, while maintaining gripper-trajectory nDTW (0.8792 → 0.8831). This supports the value of applying embodied feedback to a broader range of actions and contact configurations.

AuthorsOpenReview ↗

Response to Reviewer qUPG (Part 2)

Response to Reviewer qUPG (Part 2)

Q3: Stage-I policy-outcome evaluation and paired comparison with GE-Sim 2.0.

Can you include Stage I in the policy-outcome evaluation and report the paired statistical comparison with GE-Sim 2.0 on interaction defects? These controls would help separate the value of calibration, general post-training, and counterfactual feedback.

We have added Stage I to the policy-outcome evaluation and expanded the interaction-defect comparison with GE-Sim 2.0. The Stage-I comparison supports improved outcome prediction and success-rate calibration after post-training; the paired analysis reports both the defect differences and their statistical strength.

(1) Policy-outcome evaluation with Stage I.

Compared with Stage I, post-training brings predicted outcomes into closer agreement with actual policy executions and reduces overestimation of policy success. We evaluate Stage I and the full model on the same 240 policy trajectories across 24 task-policy-setting combinations in RoboTwin. Both models share the same calibrated Stage-I starting point, so this comparison measures the incremental benefit of the complete Stage-II training recipe (Figure 8 and Table 12).

MethodSuccess-rate MAE (pp) ↓Signed bias (pp)Outcome accuracy (%) ↑F1 ↑Pearson r ↑
Ours (w/o RL)9.17+2.5087.500.8480.936
Ours (full)7.08+0.4290.420.8810.964

Post-training reduces success-rate MAE by 22.8% and brings the aggregate prediction bias closer to zero, indicating more accurate estimates and less overestimation of policy success. Outcome accuracy and F1 also improve. These results assess agreement with actual simulator outcomes under fixed action trajectories.

(2) Paired interaction-defect comparison with GE-Sim 2.0.

ActForge has fewer interaction defects than GE-Sim 2.0 under both annotation criteria, including half as many unanimously identified defects. We report two-sided exact McNemar tests on 160 matched conditions, each evaluated independently by three annotators. We compare both majority-vote and unanimous defect labels. Discordant pairs are conditions where only one model is labeled defective.

Defect criterionActForgeGE-Sim 2.0Discordant pairs favoring ActForge / GE-Sim 2.0Exact p-value
Majority vote (≥2/3)10/160 (6.25%)12/160 (7.50%)9 / 70.804
Unanimous (3/3)4/160 (2.50%)8/160 (5.00%)7 / 30.344

The paired tests do not establish statistically significant superiority on these 160 conditions.

We thank you again for the helpful suggestions and hope that these additional results and clarifications address your concerns.

Reviewer_zc3AOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

ActForge conditions multi-view video prediction on calibrated robot renderings, then uses learned visual-defect rewards to post-train on recorded and counterfactual actions. It evaluates action following, interaction plausibility, and policy-outcome prediction.

Critical strengths

The calibration ablation connects data quality to action fidelity. Table 4 separately tests raw calibration, inference-only refinement, and refinement during training and inference. AgiBot nDTW rises from 0.8159 to 0.8243 to 0.8711, while synchronous position error falls from 7.73 to 5.77 to 4.60 pixels. These controls establish that alignment improves both conditioning and learning.

Independent human evaluation supports the post-training gains. On the same 160 counterfactual conditions, post-training reduces interaction defects from 48.12% to 6.25%, while nDTW improves from 0.8735 to 0.8831 (Table 1). Three independent annotators assess the outputs (Section 4.1), and Section 4.6 separately validates the reward model on 4,893 held-out clips.

Critical weaknesses

The experiments do not isolate the contribution of counterfactual actions. Section 3.4 introduces edited actions and reward optimization together. Table 1 establishes that this combined procedure substantially improves interaction quality, but does not show whether edited-action coverage adds value beyond applying the same reward training to recorded actions.

The policy-evaluation comparison leaves baseline adaptation unclear. ActForge reduces success-rate MAE from Ctrl-World’s 12.92 to 7.08 percentage points (Section 4.4; Figure 8). ActForge trains on RoboTwin (Section 4.1), but the paper does not specify Ctrl-World’s checkpoint or RoboTwin adaptation. This missing training context limits how readers can assess and reproduce the cross-model comparison.

Questions for authors

What interaction-defect rate and nDTW does recorded-action-only reward training achieve on the existing 160 conditions, using the same starting checkpoint, reward definitions, and update budget as the full method?

Which Ctrl-World checkpoint and RoboTwin adaptation protocol produced Figure 8, and how does its training exposure compare with ActForge’s?

1 comment
AuthorsOpenReview ↗

Response to Reviewer zc3A

Response to Reviewer zc3A

We thank you for the constructive suggestions on isolating the contribution of counterfactual actions and clarifying the baseline adaptation. We provide the controlled comparison and the Ctrl-World training protocol below.

W1 / Q1: Value of counterfactual actions beyond recorded-action reward training.

What interaction-defect rate and nDTW does recorded-action-only reward training achieve on the existing 160 conditions, using the same starting checkpoint, reward definitions, and update budget as the full method?

To address this concern, we added recorded-action-only RL controls with PSNR alone and with PSNR plus embodied feedback (Table 8). The results show that embodied feedback improves interaction quality on recorded actions, with substantially greater gains when this feedback is extended to counterfactual actions, while preserving gripper action following.

All RL variants use the same Stage-I checkpoint, training steps, batch size, and other configurations, differing only in action sampling and rewards. The recorded-only PSNR + embodied variant and the full model use identical reward definitions. The embodied reward combines robot-embodiment, object-consistency, and interaction-plausibility feedback; PSNR applies only to recorded actions with paired ground-truth futures. All models are evaluated on the same 160 counterfactual conditions.

SettingActions used for RLRewardObject defects (%) ↓Interaction defects (%) ↓nDTW ↑
Stage I (w/o RL)——31.8848.120.8735
Recorded-only RLRecordedPSNR30.0045.000.8821
Recorded-only RLRecordedPSNR + embodied24.3842.500.8792
Full modelRecorded + counterfactualPSNR + embodied3.126.250.8831

Embodiment defects are 0.00% throughout.

(1) Contribution of counterfactual actions. With the reward definitions fixed, extending training to counterfactual actions reduces interaction defects from 42.50% to 6.25% and object defects from 24.38% to 3.12%, while maintaining gripper-trajectory nDTW (0.8792 → 0.8831). This directly supports the value of applying embodied feedback to a broader range of actions and contact configurations.

(2) Contribution of embodied feedback. On recorded actions, adding embodied feedback to PSNR reduces interaction defects from 45.00% to 42.50% and object defects from 30.00% to 24.38%. This additional control establishes the benefit of the embodied reward under fixed action coverage, while the comparison above shows the further gains from counterfactual training.

W2 / Q2: Ctrl-World checkpoint, RoboTwin adaptation, and training exposure.

Which Ctrl-World checkpoint and RoboTwin adaptation protocol produced Figure 8, and how does its training exposure compare with ActForge's?

To make the comparison in Figure 8 as fair as possible, Ctrl-World and ActForge used the same 572-hour RoboTwin training corpus with identical sampling proportions across clean, randomized, and failure trajectories. We initialized Ctrl-World from its officially released, DROID-pretrained checkpoint and performed full-parameter fine-tuning using the official training implementation for 100k optimization steps with a batch size of 64, matching the training-step budget and batch size reported in the original Ctrl-World paper. Both models received the same initial observations and policy-executed action trajectories, and their predicted outcomes were evaluated against the same ground-truth simulation results.

We thank you again for the helpful suggestions and hope that these additional results and clarifications address your concerns.

Reviewer_c2GeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ActForge, a multi-view, cross-embodiment robot world model designed to improve action following and physical interaction plausibility. The method consists of three main components:

  1. An offline geometric calibration pipeline that refines camera parameters and robot mounting offsets using existing robot demonstrations.
  2. A unified action-conditioned video generation model that converts robot trajectories from different embodiments into image-space representations.
  3. A counterfactual post-training framework that uses human-annotated video defects to train an embodied reward model, followed by reinforcement learning to improve generated interaction outcomes. The model is trained on 2,232 hours of robot data across multiple datasets. Experiments demonstrate strong action following, substantially reduced interaction defects, and improved policy outcome estimation compared with existing world models.
Critical strengths
  1. A practical and well-validated framework for action-faithful video prediction across robot embodiments. The offline geometric calibration and unified image-space action representation provide a practical solution to heterogeneous robot data integration. The calibration experiments demonstrate improvements in both alignment accuracy and downstream action following. The large-scale multi-dataset training and evaluation further support the effectiveness of this approach.
  2. Effective reward-guided post-training with meaningful downstream validation. The proposed counterfactual post-training framework addresses an important limitation of robot world models: physically implausible object interactions under actions outside the demonstration distribution. The reduction in human-assessed interaction defects from 48.12% to 6.25% is substantial. Importantly, the paper also evaluates policy outcome prediction on 240 simulated execution trajectories, reducing success-rate estimation error from 12.92 to 7.08 percentage points. This provides stronger evidence of practical utility than visual quality metrics alone.
Critical weaknesses
  1. Insufficient ablations to isolate the contributions of counterfactual data augmentation and reward-guided reinforcement learning. The main comparison between Stage I and Stage II demonstrates a substantial overall improvement. However, Stage II simultaneously introduces counterfactual action sampling, embodied reward supervision, and reinforcement learning. The paper does not adequately isolate the individual contributions of these components. In particular, it remains unclear whether the performance gains primarily arise from counterfactual action coverage, reward optimization, or their combination. Additional ablations comparing reward-guided optimization with and without counterfactual actions, as well as evaluating the contributions of different reward components, would substantially strengthen the methodological claims.
  2. Physical plausibility is not equivalent to counterfactual prediction accuracy. The embodied reward model evaluates generated videos without access to the corresponding action conditions or ground-truth counterfactual outcomes. Consequently, it primarily assesses observable physical plausibility rather than whether the predicted object behavior is the correct consequence of the specified action. Although human evaluations demonstrate a significant reduction in visual interaction defects, they do not directly establish the accuracy of counterfactual physical dynamics. The downstream policy evaluation provides useful additional evidence, but it is conducted in RoboTwin simulation rather than on real-world executions. Moreover, the unseen-embodiment and unseen-scene evaluations are primarily qualitative. Therefore, the evidence supports improved visual interaction consistency and simulated policy evaluation, but claims about general physical prediction accuracy across unseen real-world conditions remain less conclusive.
Questions for authors
  1. [Critical experiment] Can the authors provide an ablation separating the effects of counterfactual action augmentation and reward-guided RL? In particular, how does RL using only recorded actions compare with RL using both recorded and counterfactual actions? How much does each reward component contribute to the improvement?
  2. [Critical clarification] Since the embodied reward model receives only the generated video without the corresponding action condition, how does the method prevent optimization toward visually plausible but action-inconsistent object behavior? Can the authors provide evidence that the improvements reflect more accurate counterfactual outcomes rather than merely fewer visible defects?
  3. [Validation] Can the authors evaluate counterfactual prediction accuracy against real-world executions involving deliberately perturbed actions, especially missed grasps and contact failures? Even a small-scale controlled evaluation would help establish the physical validity of the predicted outcomes beyond simulation.
2 comments
AuthorsOpenReview ↗

Response to Reviewer c2Ge (Part 1)

Response to Reviewer c2Ge (Part 1)

We thank you for the careful review and constructive suggestions on isolating the Stage-II contributions and validating counterfactual outcomes. We have added controlled post-training ablations and real-world outcome validation to address these concerns. We discuss W1 / Q1 below and W2 / Q2-Q3 in Part 2.

W1 / Q1: Contributions of counterfactual actions and reward-guided RL.

Can the authors provide an ablation separating the effects of counterfactual action augmentation and reward-guided RL? In particular, how does RL using only recorded actions compare with RL using both recorded and counterfactual actions? How much does each reward component contribute to the improvement?

Thank you for suggesting these controls. We added two ablations to clarify how reward supervision and counterfactual action coverage contribute to Stage II (Table 8). The results show that embodied feedback improves interaction quality on recorded actions, with substantially greater gains when this feedback is extended to counterfactual actions.

All RL variants use the same Stage-I checkpoint, training steps, batch size, and other configurations, differing only in action sampling and rewards. All models are evaluated on the same 160 counterfactual conditions. The embodied reward combines robot-embodiment, object-consistency, and interaction-plausibility feedback, evaluated jointly; PSNR applies only to recorded actions with paired ground-truth futures.

SettingActions used for RLRewardObject defects (%) ↓Interaction defects (%) ↓nDTW ↑
Stage I (w/o RL)——31.8848.120.8735
Recorded-only RLRecordedPSNR30.0045.000.8821
Recorded-only RLRecordedPSNR + embodied24.3842.500.8792
Full modelRecorded + counterfactualPSNR + embodied3.126.250.8831

Embodiment defects are 0.00% throughout.

(1) Effect of the embodied reward.

With training restricted to recorded actions, adding the embodied reward to PSNR reduces object defects from 30.00% to 24.38% and interaction defects from 45.00% to 42.50%. This isolates the incremental benefit of embodied feedback under the same action coverage.

(2) Effect of reward-guided counterfactual training.

Extending the same reward-guided training to counterfactual actions further reduces object defects from 24.38% to 3.12% and interaction defects from 42.50% to 6.25%. Gripper-trajectory nDTW is maintained (0.8792 → 0.8831). These results demonstrate the benefit of extending embodied-reward-guided optimization to a broader range of actions and contact configurations.

AuthorsOpenReview ↗

Response to Reviewer c2Ge (Part 2)

Response to Reviewer c2Ge (Part 2)

W2 / Q2: Action consistency and evidence of outcome accuracy.

Since the embodied reward model receives only the generated video without the corresponding action condition, how does the method prevent optimization toward visually plausible but action-inconsistent object behavior? Can the authors provide evidence that the improvements reflect more accurate counterfactual outcomes rather than merely fewer visible defects?

The video-only reward can assess whether contact and object motion are coherent, but it cannot independently verify whether they match the input action. We address this risk through action conditioning and supervision from real future frames. In Stage I, the model learns to follow actions represented by calibrated robot renderings. Stage II retains this action input and continues training on recorded actions with a PSNR reward against their paired ground-truth future frames. These constraints help preserve action following while the video-only reward improves interaction coherence.

To check whether the improvements go beyond fewer visible defects, we provide two complementary checks:

  • Outcome prediction against actual executions. On the same 240 policy trajectories in RoboTwin, RL post-training reduces the mean absolute error in predicted success rates from 9.17 to 7.08 percentage points and the overall prediction bias from +2.50 to +0.42 percentage points, indicating more accurate estimates and less overestimation (Figure 8 and Table 12). In the controlled real-world evaluation, agreement between predicted and actual lift outcomes increases from 80.00% to 98.33% (Table 4); the protocol and detailed results are provided in Q3 below.
  • Action following under counterfactual conditions. On the same 160 counterfactual conditions, gripper-trajectory nDTW remains comparable (0.8735 → 0.8831), supporting preserved gripper action following. The execution-based evaluations above provide the direct evidence of outcome accuracy.

Together, these results support that RL post-training preserves gripper action following and improves interaction-outcome prediction beyond merely reducing visible defects.

W2 / Q3: Controlled real-world validation with perturbed actions.

Can the authors evaluate counterfactual prediction accuracy against real-world executions involving deliberately perturbed actions, especially missed grasps and contact failures? Even a small-scale controlled evaluation would help establish the physical validity of the predicted outcomes beyond simulation.

Our post-training improves prediction of action-dependent interaction outcomes in both simulation and controlled real-world executions. To validate this on hardware, we evaluate three tasks under perturbed actions, each containing 10 lift and 10 non-lift cases, for a total of 60 cases. The cases cover failure modes including missed grasps and partial contacts. Different action variants are executed from matched initial states, and both models are evaluated on exactly the same cases. We compare predicted lift outcomes with the corresponding real-world executions under the same action conditions (Table 4).

MethodAccuracy on non-lift cases (%)Accuracy on lift cases (%)Overall accuracy (%)
Ours (w/o RL)60.00100.0080.00
Ours (full)100.0096.6798.33

The model without RL incorrectly predicts lifting in 12 of the 30 actual non-lift cases. The full model correctly predicts all 30 non-lift cases and 29 of the 30 lift cases, eliminating false lift predictions on this subset. We also provide qualitative examples of contact-induced object displacement when the gripper contacts and pushes objects during closure (Figures 17 and 18).

This controlled evaluation supports the physical relevance of the improvements for perturbed grasp-and-lift actions.

We thank you again for the constructive suggestions and hope that these clarifications and additional experiments address your concerns.

Comments on the paper

2 comments
Area_Chair_snoeOpenReview ↗

Discussion

Dear reviewers,

Currently, we have the following review ratings

  • c2Ge: 3: Weak acceptance
  • zc3A: 3: Weak acceptance
  • qUPG: 3: Weak acceptance

While all reviewers have agreed ratings toward acceptance, still, we also have weaknesses pointed out and questions that demand answers. Could you take a look at each other's reviews and the author responses and initiate a discussion?

  • Were your concerns resolved and questions answered properly?
  • How do you weigh the strengths and weaknesses?
  • Are there any issues or agenda you want to highlight?

Please note that ICLR 2027 has Policy on LLM use.

Best, Your AC

AuthorsOpenReview ↗

Brief summary and sincere thanks to all reviewers

Thank you to the Area Chair and all three reviewers for taking the time to read our paper and share thoughtful feedback. We appreciate the positive ratings from all three reviewers and your recognition of the value of geometric calibration and counterfactual evaluation for robot world models. Your questions helped us clarify what drives the post-training gains and test how well ActForge predicts the outcomes of perturbed actions.

  • Reviewer c2Ge highlighted the value of calibrated action conditioning across robot embodiments and asked whether visually plausible predictions agree with actual outcomes. We clarified how calibrated action inputs and supervision from recorded future frames help preserve action following. In a controlled study of 60 hardware executions across three tasks, including missed grasps and partial contacts, post-training improved agreement with actual lift outcomes from 80.00% to 98.33%.

  • Reviewer zc3A highlighted the calibration ablation and independent human evaluation, and asked us to isolate the contribution of counterfactual actions and clarify the Ctrl-World baseline. With the reward definitions and optimization budget fixed, adding counterfactual actions reduced interaction defects from 42.50% to 6.25% across 160 conditions while preserving gripper action following. We also documented Ctrl-World’s starting checkpoint and fine-tuning protocol on the same RoboTwin training corpus.

  • Reviewer qUPG recognized the value of the blinded human evaluation and asked for a direct comparison with Stage I on policy-outcome prediction. On the same 240 RoboTwin policy trajectories, the full Stage II post-training reduced the mean absolute error in predicted success rates from 9.17 to 7.08 percentage points. In the requested paired comparison with GE-Sim 2.0 across 160 matched conditions, ActForge had numerically fewer interaction defects under both annotation criteria, with half as many cases flagged by all three annotators (4 vs. 8).

The revised manuscript and our detailed responses describe these experiments and clarifications. Thank you again for the care you put into reviewing ActForge and for suggesting experiments that strengthened the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.