- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ActEffect uses a world model to provide feedback during robot-policy training. The policy generates feedforward, coarse, and refined action proposals. The world model predicts their terminal outcomes in frozen DINOv3 feature space. Comparative consequence refinement (CCR) uses the best auxiliary proposal as a reference to weight feedback on the refined proposal. The world model is removed at deployment.
The paper reports 99.0% success on LIBERO, 67.5% on RoboCasa-GR1, and 91.5% on one real-robot USB task under four conditions, together with lower inference latency than the listed baselines. The main concerns are the rationale for CCR's comparison rule, prediction accuracy for policy-generated actions, reliability of early feedback, and the breadth of physical-task validation.
Critical strengths
-
The results support using predicted action outcomes as supervision beyond action imitation. This feedback improves task success and gives the policy a useful additional learning signal.
-
The policy retains the benefits of consequence supervision after the world model is removed. This matters for deployment: actions can be generated directly, without waiting for predictions and comparisons of candidate outcomes. The reported gains in task performance therefore come with low inference latency.
-
The real-robot results are encouraging on USB insertion, where small pose errors can cause failure. The policy outperforms the tested baselines under changes in starting pose and visual conditions.
Critical weaknesses
-
The rationale for CCR's feedback weighting remains unclear. CCR gives stronger consequence feedback when the refined action is predicted to be worse than its reference, and weaker feedback when it is predicted to be better. The ablations show gains, but it remains unclear why outperforming a poor reference should lead to weaker supervision when the refined action's predicted outcome is still far from the demonstration.
-
The world model's predictions for policy-generated actions need stronger validation. The world model learns from demonstrated actions and their observed outcomes. CCR then uses it to evaluate policy-generated actions, which may deviate from the demonstrations. The retrieval test shows that the model can identify the demonstrated action, but does not verify that its predictions for other actions match what would actually happen. Figure 5 and the frozen-model ablation offer encouraging evidence, but do not directly establish prediction accuracy for policy-generated actions. This matters because CCR relies on these predictions to guide policy learning.
-
Physical-task validation is narrow. The real-robot study covers one USB removal-and-insertion task under four conditions. This is a challenging contact task, and the reported gains under initial-pose, background/plug-color, and lighting changes are useful. However, these variations do not establish breadth across physical tasks. Figure 6 provides one qualitative recovery comparison. More complex and varied physical tasks are needed to support the broader manipulation and correction claims.
-
The reliability of feedback early in training is unclear. The dynamics predictor is randomly initialized, and the main objective jointly optimizes it with the policy (Section 4.4; Appendix B.3). If CCR is active from the start, initially inaccurate predictions could misdirect policy updates. The decreasing training errors in Figure 2 and the pretrained, frozen predictor in Table 8 are useful evidence, but do not explain how early feedback is handled in the main joint-training setup. Its effect on early policy learning and training stability remains uncertain.
Questions for authors
-
Why is relative consequence quality against the auxiliary references an appropriate guide for feedback strength when the refined proposal still predicts a poor absolute outcome? The absolute-loss comparison is encouraging; what explains the added value of CCR's comparison rule?
-
What evidence shows that the world model's predictions reflect the actual outcomes of actions generated by the policy?
-
Can you evaluate the method on more complex and varied real-world tasks? This would help show whether the benefits extend beyond the USB setup.
-
Is CCR active from the start of joint training? How is feedback handled before the world model has learned useful dynamics, and what evidence supports reliable policy learning during this stage?
I would consider raising my rating if the authors' responses satisfactorily address the concerns above.
5 comments
W4 & Q4: Feedback Reliability in Early Training
Thank you for raising this question. We clarify that CCR is enabled after an initial warmup period, rather than from the start of training. During the first 5k steps on LIBERO (15k on RoboCasa-GR1 and 1k on real-world tasks, aligned with the warmup schedules in Appendix B.3), we set and optimize only the action-imitation and world-model prediction objectives. All three action proposals receive direct supervision from demonstrated actions, while the world model learns from observed demonstration transitions with frozen DINOv3 targets. Consequently, policy updates during this stage receive no gradients from the initially inaccurate predictor.
After warmup, we activate CCR with and continue jointly optimizing the policy and world model alongside both supervised objectives. This allows the predictor to learn from demonstrated transitions before its predictions are used to guide policy refinement.
The training trajectories provide supporting evidence for this learning process. Figure 2(a) shows that demonstrated-transition and proposal-conditioned errors decrease during training. Figure 2(b) further shows an early increase in the fraction of refined proposals that outperform their auxiliary references under the learned consequence metric. These diagnostics are consistent with improving consequence prediction and proposal refinement during training.
The main results reported in the paper use this delayed-activation schedule. We have specified the warmup duration and CCR activation schedule in Appendix B.3 and Table 7.
Thank you for the clarifications and additional experiments. The CCR explanation, execution-based validation, and warmup clarification are helpful. However, I still find the evidence for broader real-world applicability and corrective behavior limited. These remaining concerns continue to affect my overall assessment, so I am maintaining my current rating at this stage.
Could you also share complete execution videos of the newly added SPS task? These would help me assess its complexity and how much it broadens the physical validation beyond the USB setup.
W1 & Q1: Rationale for Relative Consequence Feedback
Thank you for this question. We clarify that the auxiliary reference controls the strength of additional consequence feedback, while the demonstrated future remains the absolute optimization target. As derived in Appendix A.2,
∂d1∂LCCR=σ!(Td1−dref+m)>0.
Thus, CCR preserves the direction of reducing the absolute consequence discrepancy. Outperforming the reference attenuates this feedback; it does not establish that the predicted outcome is already satisfactory.
The comparison rule is motivated by the role of CCR as a supplement to action imitation. All three proposals continue to receive direct supervision from demonstrated actions, while CCR emphasizes samples where the refined proposal underperforms an available auxiliary alternative. Moreover, the reference is continually updated as the auxiliary proposals improve. If the reference improves while the refined proposal remains unchanged, the relative gap increases and CCR strengthens its feedback again. This provides an evolving refinement signal rather than a fixed, potentially poor target. Figure 2(a) further shows that both the reference and refined proposal’s absolute consequence discrepancies decrease during training.
The absolute loss comparison in Table 5 supports the practical benefit of this feedback allocation: CCR achieves 99.0% versus 97.9% on LIBERO and 67.5% versus 59.0% on RoboCasa-GR1. Our additional USB ablation also shows 91.5% versus 85.5% (see W2 in our response to Reviewer h4tv).
W2 & Q2: Prediction Accuracy for Policy-generated Actions
Thank you for this suggestion. To directly validate predictions for policy-generated actions, we conduct an execution-based evaluation in LIBERO. We sample 640 states from held-out demonstration trajectories. For each state, we generate the feedforward, coarse, and refined action proposals and execute each proposal from the same restored simulator state. We then compare the world model's predicted DINO features with those extracted from the actual terminal observation produced by that proposal.
The table below reports the mean feature prediction error. We include persistence prediction, which uses the current DINO features as the predicted future, as a reference:
The world model reduces feature prediction MSE by 55.6%-63.0% relative to persistence across the three types of policy-generated action proposals, supporting its ability to predict their actual execution outcomes.
We further examine whether the predicted consequence comparisons agree with actual execution outcomes. For each state, we rank the proposals by their predicted distance to the demonstrated future and compare this ranking with that obtained from their actual terminal observations. The resulting pairwise ranking agreement is 89.4% across 640 states. This directly evaluates whether the world model's comparative feedback reflects the relative quality of executed actions, as required by CCR.
We have added these results to Section 5.4 (Table 6) and the evaluation protocol to Appendix C.3.
W3 & Q3: Broader Real-world Task Coverage
Thank you for this suggestion. We have expanded our real-world evaluation to a complex Set Parts Supply (SPS) sorting task, which requires reliable grasping and placement of diverse industrial parts, including springs, flanges, valves, sealing strips, etc., under randomized object poses and cluttered layouts. This task introduces variation in object geometry and grasping requirements, complementing the contact-sensitive USB manipulation task. We compare ActEffect with π0.5, the strongest baseline in our original real-world evaluation:
ActEffect also achieves better performance than the baseline on this task, extending the evidence for its effectiveness to industrial-part sorting. In addition, our new component ablations on the real-world USB task show that removing consequence feedback reduces success from 91.5% to 78.5% (see W2 in our response to Reviewer h4tv), supporting its contribution to physical task performance.
We have added the SPS results and USB ablations to Tables 3 and 5, respectively, with the SPS task illustrated in Figure 5 and the experimental details provided in Appendix C.2.