acceptodds
Under review as a conference paper at ICLR 2027

ActEffect: From Predictive Foresight to Consequence Feedback in Robot Learning

Abstract

Robot actions are interventions on the physical world, and their quality is ultimately determined by the state transitions and task outcomes they induce. However, most robot policies are trained to match demonstrated actions, while predictive world models typically provide foresight for action generation rather than feedback on the outcomes of generated actions. We present ActEffect, which augments policy learning with a world model that predicts the consequences of proposed actions and feeds them back to guide policy optimization, internalizing awareness of action outcomes into the policy itself. To make this feedback effective for policy refinement, we develop a decoupled consequence modeling framework that predicts future transitions in a separate DINOv3 latent space, preserving spatial and structural information while reducing interference from task-specific semantics. We then introduce a comparative consequence refinement objective that uses the predicted consequences of coarse-to-fine action proposals from our policy to construct a relative feedback signal. By comparing the refined proposal against the best auxiliary consequence, ActEffect guides policy refinement toward actions whose predicted futures better align with the demonstrated outcome. The world model is used only during training and removed at deployment, introducing no inference overhead. Experiments on simulation benchmarks demonstrate that ActEffect achieves state-of-the-art performance while delivering a 2.5 inference speedup over the strongest baseline, reaching average success rates of 99.0% on LIBERO and 67.5% on RoboCasa-GR1. On real-world contact-rich manipulation tasks, ActEffect further outperforms the baselines by 13.5 percentage points in success rate while demonstrating strong robustness under perturbations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Vision Language Action Models, World Models, Contact-rich Manipulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aa8XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ActEffect uses a world model to provide feedback during robot-policy training. The policy generates feedforward, coarse, and refined action proposals. The world model predicts their terminal outcomes in frozen DINOv3 feature space. Comparative consequence refinement (CCR) uses the best auxiliary proposal as a reference to weight feedback on the refined proposal. The world model is removed at deployment.

The paper reports 99.0% success on LIBERO, 67.5% on RoboCasa-GR1, and 91.5% on one real-robot USB task under four conditions, together with lower inference latency than the listed baselines. The main concerns are the rationale for CCR's comparison rule, prediction accuracy for policy-generated actions, reliability of early feedback, and the breadth of physical-task validation.

Critical strengths
  1. The results support using predicted action outcomes as supervision beyond action imitation. This feedback improves task success and gives the policy a useful additional learning signal.

  2. The policy retains the benefits of consequence supervision after the world model is removed. This matters for deployment: actions can be generated directly, without waiting for predictions and comparisons of candidate outcomes. The reported gains in task performance therefore come with low inference latency.

  3. The real-robot results are encouraging on USB insertion, where small pose errors can cause failure. The policy outperforms the tested baselines under changes in starting pose and visual conditions.

Critical weaknesses
  1. The rationale for CCR's feedback weighting remains unclear. CCR gives stronger consequence feedback when the refined action is predicted to be worse than its reference, and weaker feedback when it is predicted to be better. The ablations show gains, but it remains unclear why outperforming a poor reference should lead to weaker supervision when the refined action's predicted outcome is still far from the demonstration.

  2. The world model's predictions for policy-generated actions need stronger validation. The world model learns from demonstrated actions and their observed outcomes. CCR then uses it to evaluate policy-generated actions, which may deviate from the demonstrations. The retrieval test shows that the model can identify the demonstrated action, but does not verify that its predictions for other actions match what would actually happen. Figure 5 and the frozen-model ablation offer encouraging evidence, but do not directly establish prediction accuracy for policy-generated actions. This matters because CCR relies on these predictions to guide policy learning.

  3. Physical-task validation is narrow. The real-robot study covers one USB removal-and-insertion task under four conditions. This is a challenging contact task, and the reported gains under initial-pose, background/plug-color, and lighting changes are useful. However, these variations do not establish breadth across physical tasks. Figure 6 provides one qualitative recovery comparison. More complex and varied physical tasks are needed to support the broader manipulation and correction claims.

  4. The reliability of feedback early in training is unclear. The dynamics predictor is randomly initialized, and the main objective jointly optimizes it with the policy (Section 4.4; Appendix B.3). If CCR is active from the start, initially inaccurate predictions could misdirect policy updates. The decreasing training errors in Figure 2 and the pretrained, frozen predictor in Table 8 are useful evidence, but do not explain how early feedback is handled in the main joint-training setup. Its effect on early policy learning and training stability remains uncertain.

Questions for authors
  1. Why is relative consequence quality against the auxiliary references an appropriate guide for feedback strength when the refined proposal still predicts a poor absolute outcome? The absolute-loss comparison is encouraging; what explains the added value of CCR's comparison rule?

  2. What evidence shows that the world model's predictions reflect the actual outcomes of actions generated by the policy?

  3. Can you evaluate the method on more complex and varied real-world tasks? This would help show whether the benefits extend beyond the USB setup.

  4. Is CCR active from the start of joint training? How is feedback handled before the world model has learned useful dynamics, and what evidence supports reliable policy learning during this stage?

I would consider raising my rating if the authors' responses satisfactorily address the concerns above.

5 comments

W1 & Q1: Rationale for Relative Consequence Feedback

Thank you for this question. We clarify that the auxiliary reference controls the strength of additional consequence feedback, while the demonstrated future remains the absolute optimization target. As derived in Appendix A.2,

∂LCCR∂d1=σ!(d1−dref+mT)>0.\frac{\partial \mathcal{L}_{\mathrm{CCR}}}{\partial d_1}=\sigma\\!\left(\frac{d_1-d_{\mathrm{ref}}+m}{T}\right)>0.

Thus, CCR preserves the direction of reducing the absolute consequence discrepancy. Outperforming the reference attenuates this feedback; it does not establish that the predicted outcome is already satisfactory.

The comparison rule is motivated by the role of CCR as a supplement to action imitation. All three proposals continue to receive direct supervision from demonstrated actions, while CCR emphasizes samples where the refined proposal underperforms an available auxiliary alternative. Moreover, the reference is continually updated as the auxiliary proposals improve. If the reference improves while the refined proposal remains unchanged, the relative gap increases and CCR strengthens its feedback again. This provides an evolving refinement signal rather than a fixed, potentially poor target. Figure 2(a) further shows that both the reference and refined proposal’s absolute consequence discrepancies decrease during training.

The absolute loss comparison in Table 5 supports the practical benefit of this feedback allocation: CCR achieves 99.0% versus 97.9% on LIBERO and 67.5% versus 59.0% on RoboCasa-GR1. Our additional USB ablation also shows 91.5% versus 85.5% (see W2 in our response to Reviewer h4tv).

W2 & Q2: Prediction Accuracy for Policy-generated Actions

Thank you for this suggestion. To directly validate predictions for policy-generated actions, we conduct an execution-based evaluation in LIBERO. We sample 640 states from held-out demonstration trajectories. For each state, we generate the feedforward, coarse, and refined action proposals and execute each proposal from the same restored simulator state. We then compare the world model's predicted DINO features with those extracted from the actual terminal observation produced by that proposal.

The table below reports the mean feature prediction error. We include persistence prediction, which uses the current DINO features as the predicted future, as a reference:

Action ProposalWorld-model Prediction MSE ↓\downarrowPersistence MSE ↓\downarrow
Feedforward0.00790.0178
Coarse0.00730.0181
Refined0.00680.0184

The world model reduces feature prediction MSE by 55.6%-63.0% relative to persistence across the three types of policy-generated action proposals, supporting its ability to predict their actual execution outcomes.

We further examine whether the predicted consequence comparisons agree with actual execution outcomes. For each state, we rank the proposals by their predicted distance to the demonstrated future and compare this ranking with that obtained from their actual terminal observations. The resulting pairwise ranking agreement is 89.4% across 640 states. This directly evaluates whether the world model's comparative feedback reflects the relative quality of executed actions, as required by CCR.

We have added these results to Section 5.4 (Table 6) and the evaluation protocol to Appendix C.3.

W3 & Q3: Broader Real-world Task Coverage

Thank you for this suggestion. We have expanded our real-world evaluation to a complex Set Parts Supply (SPS) sorting task, which requires reliable grasping and placement of diverse industrial parts, including springs, flanges, valves, sealing strips, etc., under randomized object poses and cluttered layouts. This task introduces variation in object geometry and grasping requirements, complementing the contact-sensitive USB manipulation task. We compare ActEffect with π0.5\pi_{0.5}, the strongest baseline in our original real-world evaluation:

MethodSPS Success Rate (%)
π0.5\pi_{0.5}62
ActEffect78

ActEffect also achieves better performance than the baseline on this task, extending the evidence for its effectiveness to industrial-part sorting. In addition, our new component ablations on the real-world USB task show that removing consequence feedback reduces success from 91.5% to 78.5% (see W2 in our response to Reviewer h4tv), supporting its contribution to physical task performance.

We have added the SPS results and USB ablations to Tables 3 and 5, respectively, with the SPS task illustrated in Figure 5 and the experimental details provided in Appendix C.2.

W4 & Q4: Feedback Reliability in Early Training

Thank you for raising this question. We clarify that CCR is enabled after an initial warmup period, rather than from the start of training. During the first 5k steps on LIBERO (15k on RoboCasa-GR1 and 1k on real-world tasks, aligned with the warmup schedules in Appendix B.3), we set λCCR=0\lambda_{\mathrm{CCR}}=0 and optimize only the action-imitation and world-model prediction objectives. All three action proposals receive direct supervision from demonstrated actions, while the world model learns from observed demonstration transitions with frozen DINOv3 targets. Consequently, policy updates during this stage receive no gradients from the initially inaccurate predictor.

After warmup, we activate CCR with λCCR=1.0\lambda_{\mathrm{CCR}}=1.0 and continue jointly optimizing the policy and world model alongside both supervised objectives. This allows the predictor to learn from demonstrated transitions before its predictions are used to guide policy refinement.

The training trajectories provide supporting evidence for this learning process. Figure 2(a) shows that demonstrated-transition and proposal-conditioned errors decrease during training. Figure 2(b) further shows an early increase in the fraction of refined proposals that outperform their auxiliary references under the learned consequence metric. These diagnostics are consistent with improving consequence prediction and proposal refinement during training.

The main results reported in the paper use this delayed-activation schedule. We have specified the warmup duration and CCR activation schedule in Appendix B.3 and Table 7.

Thank you for the clarifications and additional experiments. The CCR explanation, execution-based validation, and warmup clarification are helpful. However, I still find the evidence for broader real-world applicability and corrective behavior limited. These remaining concerns continue to affect my overall assessment, so I am maintaining my current rating at this stage.

Could you also share complete execution videos of the newly added SPS task? These would help me assess its complexity and how much it broadens the physical validation beyond the USB setup.

Reviewer_CBMCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ActEffect, which improves policy learning with a world model that can predict the consequences of proposed actions and feeds them back to guide policy optimization, incorporating the prediction of action outcomes within the robot policy. Then during inference time, the world model is removed so no inference overhead, The experiments are done extensively, including LIBERO, RoboCasa-GR1, and real-world USB insertion.

Critical strengths
  1. The proposed framework is effective and combining the vla with the world modeling. Also no inference overhead is pivotal.
  2. The experiments are done over a range of tasks, illustrating the effectiveness of the method.
Critical weaknesses
  1. Joint optimization may use the world model as a way to purely improve the metric rather than learn accurate consequences. Like how this is separated with adding a larger architecture with vla? How to show the world modeling is helping because it's predicting about the future.
  2. The results should have more statistical meaning by having multiple seeds.
  3. Please illustrate more on the comparative objective and why it's not an adaptive weighting of an absolute consequence loss
Questions for authors

Please resolve the questions in weakness.

4 comments

W1: Joint Optimization and the Role of World Modeling

Thank you for raising this concern. We clarify that the w/o Feedback variant in the ablation study in Section 5.3 retains the same policy architecture as full ActEffect while removing the training-only world model and the consequence-feedback objective. The improvements reported in Table 5 therefore support the contribution of consequence feedback under fixed policy capacity. Our additional ablation on the real-world USB manipulation task (see W2 in our response to Reviewer h4tv) shows the same pattern, with success increasing from 78.5% to 91.5% when consequence feedback is included.

We further examine whether the world model captures action-dependent future information. In Figure 2(c-d), the demonstrated action produces substantially lower prediction error against the observed future than zero, shuffled, or negated actions, and is identified among eight candidates with 95.8% accuracy, compared with 12.5% random chance.

Figure 6 further visualizes how the predicted future changes with the action while keeping the current observation fixed: demonstrated and policy-generated actions produce similar feature-transition patterns, whereas negated actions produce changes in the opposite direction and zero actions produce only weak changes. These quantitative and qualitative results support that the world model learns action-sensitive future information, complementing the policy improvements demonstrated by the ablations above.

W2: Statistical Reliability across Random Seeds

Thank you for this suggestion. Most baseline results in our original comparison are taken from their respective publications. To directly assess the statistical reliability of our results, we have additionally trained ActEffect and its base policy, consisting of the Qwen3-VL backbone and MIP action head, with five independent random seeds on LIBERO and RoboCasa-GR1. We report the mean and standard deviation of success rates under the same evaluation protocol:

MethodLIBERO (%)RoboCasa-GR1 (%)
Qwen3-VL + MIP96.8 ±\pm 0.748.6 ±\pm 2.8
ActEffect98.9 ±\pm 0.367.2 ±\pm 2.1

ActEffect improves over the base policy on both benchmarks while maintaining the highest average success rates among the compared baselines. These results support the effectiveness of ActEffect across repeated training runs. We report ActEffect's five-seed results in Appendix C.4, which closely match the single-seed results in the main paper.

W3: Rationale for Comparative Consequence Refinement

Thank you for this question. We clarify that CCR admits an adaptive-weighting interpretation at the gradient level, as shown in Proposition 2 of Appendix A.2: ∇θLCCR=σ!(d1−dref+mT)∇θd1,\nabla_\theta \mathcal{L}_{\mathrm{CCR}} = \sigma\\!\left(\frac{d_1-d_{\mathrm{ref}}+m}{T}\right)\nabla_\theta d_1, where d1d_1 is the absolute consequence discrepancy between the refined proposal's predicted future and the demonstrated future. CCR preserves this optimization direction while determining the feedback strength through a comparison with the auxiliary proposals.

Specifically, the better of the coarse and feedforward proposals provides a reference for each sample. The refined proposal receives stronger feedback when it underperforms this reference, and weaker feedback as it improves upon it. This directs additional consequence supervision toward samples where refinement remains insufficient, alongside the ongoing action supervision. The demonstrated future remains the optimization target, while stop-gradient on the reference prevents the objective from directly encouraging degradation of the auxiliary proposals.

The ablations in Table 5 support this design: replacing CCR with an absolute consequence loss reduces success from 99.0% to 97.9% on LIBERO and from 67.5% to 59.0% on RoboCasa-GR1. Our additional USB ablation shows the same pattern, with 91.5% for CCR versus 85.5% for absolute loss (see W2 in our response to Reviewer h4tv). These results support the effectiveness of the proposed relative weighting rule.

Dear Reviewer CBMC,

Thank you again for your constructive suggestions. We clarified that the feedback ablation keeps the policy architecture unchanged and discussed evidence that the world model captures action-dependent future information. We also added five-seed comparisons with the base policy on LIBERO and RoboCasa-GR1, showing consistent gains. Finally, we clarified CCR’s relationship to adaptive weighting and how the auxiliary reference controls feedback strength, supported by comparisons with absolute consequence loss in simulation and real-world USB manipulation.

We would greatly appreciate it if you could let us know whether these additions address your concerns.

Best regards,

The Authors

Dear Reviewer CBMC,

Thank you once more for your time and valuable comments. We would be grateful to know whether our response has resolved your concerns, or whether there are any points we could clarify further.

Best regards, The Authors

Dear reviewer,

As the end of discussion between authors and reviewers are about to end, please take a look at authors rebuttal to see if these have addressed your concerns.

Reviewer_h4tvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ActEffect, a training scheme for World-Action Models (WAMs) based on an independent world model. Departing from prevailing approaches, ActEffect employs feedback mechanism for future prediction. Meanwhile, an independent world model is trained to predict subsequent states within a latent space; by incorporating the outputs of a Minimal Iterative Policy (MIP) and a MLP as action inputs, the world model effectively provides an additional intrinsic loss (LCCR\mathcal L_{CCR}) for supervised training within the latent state space. Experimental results indicate that this approach not only enhances the model's capabilities but also demonstrates strong generalization performance.

Critical strengths
  1. The method is novel in adopting a feedback-based formulation for world-action modeling with a separate world model. This introduces additional supervision in latent space during training and offers a promising direction for policy learning.
  2. The framework is well designed and clearly presented, with comprehensive ablations supporting the contributions of its key components.
  3. The experimental results are strong, with particularly substantial improvements on RoboCasa-GR1.
Critical weaknesses
  1. Existing model-based approaches for embodied policy learning, such as VLA-MBPO[1] and WMPO[2]. Specifically, they use trajectory outcomes generated by world models as feedback for policy optimization. The paper lacks comparisons with these methods, or at least a discussion of their relative advantages and limitations.
  2. Real-world evaluation is limited to one USB manipulation task, while quantitative consequence-model analyses are simulation-only. Broader task coverage or real-world feedback ablations would strengthen the evidence.

[1] Zhang et al. Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models, 2026.

[2] Zhu et al. WMPO: World Model-based Policy Optimization for Vision-Language-Action Models, 2025.

Questions for authors

Following up on Weakness 1: the previously mentioned approaches primarily rely on world models for exploration (e.g., VLA-MBPO), whereas ActEffect functions more like imitation learning-- does this limit the model's generalization capability?

2 comments

W1 & Q1: Comparison with Model-based RL and Generalization

Thank you for highlighting these related works. First, we clarify that ActEffect focuses on the architectural design and imitation learning of VLA/WAM policies, using predicted action consequences in a separate DINOv3 latent space as feedback for policy refinement. VLA-MBPO and WMPO use world models as simulators, generating imagined trajectories for reinforcement learning (RL). ActEffect therefore avoids the computational cost of high-fidelity pixel-space world modeling, reward modeling, and imagined RL rollouts, providing consequence supervision through a lightweight latent predictor.

To provide a controlled comparison, we adapt WMPO and VLA-MBPO to the same Qwen3-VL backbone and MIP action head, while retaining their respective world-model RL procedures. We evaluate all methods on LIBERO under the same evaluation protocol and directly transfer the resulting policies to LIBERO-PLUS without additional training. The results are shown below:

MethodLIBEROLIBERO-PLUS
Qwen3-VL + MIP97.072.8
+ WMPO98.174.2
+ VLA-MBPO97.577.4
ActEffect99.080.3

ActEffect achieves the best performance on both benchmarks. Regarding generalization, it reaches 80.3% on LIBERO-PLUS without additional training, outperforming both model-based RL baselines and demonstrating robustness under distribution shifts.

Finally, the learned policy can serve as an initialization for subsequent RL fine-tuning, allowing reward-driven exploration to complement consequence-aware imitation. We will explore this integration in future work.

W2: Broader Real-world Evaluation and Feedback Ablations

Thank you for this suggestion. We have expanded our real-world evaluation to a complex Set Parts Supply (SPS) sorting task, which requires reliable grasping and placement of diverse industrial parts, including springs, flanges, valves, sealing strips, etc., under randomized object poses and cluttered layouts. We compare ActEffect with π0.5\pi_{0.5}, the strongest baseline in our original real-world evaluation. The results are shown below:

MethodSPS Success Rate (%)
π0.5\pi_{0.5}62
ActEffect78

ActEffect also achieves better performance than the baseline on this task. To directly examine the contribution of consequence feedback in physical execution, we also repeat the component ablations from Table 5 on the real-world USB manipulation task, given its sensitivity to small action errors:

Variantw/o Feedbackw/ Absolute Lossw/o Decouplingw/o FF ProposalFull ActEffect
Success Rate (%)78.585.583.088.591.5

Success rates are averaged over the same four evaluation conditions as in Table 3. Removing consequence feedback reduces the success rate from 91.5% to 78.5%, confirming its contribution to real-world manipulation. We have added the SPS results and USB ablations to Tables 3 and 5, respectively, with the SPS task illustrated in Figure 5 and the experimental details provided in Appendix C.2.

I am appreciate to receive your serious answer. My questions are basically solved, and I will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.