acceptodds
Withdrawn from ICLR 2027

Prediction Serves Action: Learning Action Effects in World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) improve action learning by introducing future prediction as auxiliary supervision. Existing WAMs, however, largely retain the complete future observation as the predictive target, even when future prediction is conditioned on robot actions. Such targets contain both scene changes and substantial persistent content, distributing supervision across the complete future observation even though different components might contribute differently to action learning. We therefore introduce Action Effect, a predictive target that focuses on the changes and consequences associated with an action and provides more targeted future supervision. Based on this idea, we propose Effect-WAM, a lightweight WAM that models Action Effects directly within a single DiT action-generation pathway, without a separate future branch. Effect-WAM employs a tightly coupled Propose-Anticipate-Act (PAA) mechanism that forms an intermediate action hypothesis, anticipates the corresponding effects, and uses the resulting Effect representations to refine subsequent action generation. We further construct complementary 2D semantic and robot-centric 3D geometric supervision to learn structured Action Effect representations. Experiments on LIBERO and RoboTwin 2.0, together with real-world evaluations, show strong manipulation performance with a lightweight architecture, supporting Action Effect modeling as an effective alternative to complete future generation. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model, Future Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_knD6OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the limitation of complete-future prediction in World Action Models (WAMs), which spreads supervision over the entire future representation and may be less aligned with action learning, and proposes Effect-WAM, which preserves a single DiT action-generation backbone and flow-matching objective while replacing complete future generation with Action Effect modeling through a Propose–Anticipate–Act mechanism that forms an intermediate action hypothesis, anticipates horizon-specific Effect tokens supervised by 2D DINO semantic changes and robot-centric 3D geometric changes, and feeds these tokens back to refine action generation without test-time future generation. Experiments show that Effect-WAM achieves competitive policy success rates on LIBERO and RoboTwin 2.0 and performs real-world manipulation tasks, while using 80.2% fewer trainable parameters and over 40% less inference memory than FastWAM at a 5.9% latency increase. It demonstrates the potential of action-associated change supervision for efficient world-action modeling.

Critical strengths
  1. A clear and meaningful research question. The paper examines what future information is useful for action learning, motivating Action Effect supervision as an alternative to predicting complete future observations. This is a well-focused question about the role of prediction in robot control.
  2. A coherent integration of prediction and action generation. The PAA mechanism integrates effect representations into a single action-generation pathway; these representations are supervised by complementary 2D feature changes and robot-centric 3D occupancy transitions. Auxiliary prediction heads are removed at inference, while effect representations remain involved in action refinement.
  3. Component ablations support the proposed design. The paper reports improvements of 1.1 and 0.6 percentage points over Action Only with 2D and 3D supervision, respectively. Removing Effect-to-Action feedback reduces average success by 1.5 percentage points, supporting the usefulness of incorporating effect representations into action generation. 【Table 3; §4.4】
  4. Competitive performance with reduced parameter and memory requirements. The method achieves competitive results on LIBERO and RoboTwin 2.0 while substantially reducing trainable parameters and inference memory relative to Fast-WAM. The authors also report real-world rollouts on a dexterous-hand platform, indicating feasibility in the tested settings. 【Tables 1–2; Figures 4–5】
Critical weaknesses
  1. The central predictive-target claim is not directly established through controlled policy evaluation. The diagnostic in Fig. 1(d) and Appendix A.3 compares predictive targets only in a matched 2D DINO setting and evaluates action-probe (R^2), rather than closed-loop policy performance; exact numerical results and uncertainty estimates are also not reported. Therefore, the paper does not directly establish that Action Effect is superior to completefuture or action-conditioned-future targets as a predictive target. A more direct test would compare the closed-loop success rates of these three target types under the same policy architecture and training budget. 【Fig. 1(d); Appendix A.3】
  2. The ablations do not fully separate the benefits of effect supervision from those of the added architecture. Table 3 evaluates 2D/3D supervision and includes a variant without Effect-to-Action feedback, providing useful evidence for these components. However, it lacks a capacity-matched PAA variant without effect supervision. Such a control would help determine how much of the improvement depends on the supervised effect representations, rather than the additional interaction modules alone. 【Table 3; §4.4】
  3. The interpretation of Effect tokens as anticipated action consequences remains only indirectly supported. Removing Effect-to-Action feedback reduces performance, indicating that this pathway is useful. However, the paper does not quantitatively evaluate effect-prediction accuracy or show whether altering the action hypothesis under the same observation produces appropriately different predicted effects. These analyses would help establish whether the tokens capture action-dependent consequences beyond the demonstrated usefulness of the feedback pathway. 【§3.3–3.5; Table 3】
  4. Real-world evaluation establishes feasibility but does not isolate the method’s benefit. The authors report rollouts on two tasks with a single embodiment, including cluttered backgrounds and low-light conditions. However, there is no action-only baseline or another comparable policy. Thus, these results indicate feasibility in the tested settings but do not establish the incremental benefit of Action Effect supervision or comparative robustness. 【§4.3; Appendix A.4】
Questions for authors

Q1: Were complete-future, action-conditioned-future, and Action Effect targets compared under the same policy architecture and training budget in closed-loop evaluation, or is the comparison limited to the action-probe diagnostic in Appendix A.3? Q2: Does the Action Only baseline retain PAA and Effect tokens with supervision disabled, or remove them entirely? This determines whether Table 3 isolates supervision from architectural changes. Q3: Is there quantitative evidence that Effect tokens predict action-dependent consequences—for example, whether different action hypotheses under the same observation yield different predicted effects?

Reviewer_SUASOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to replace the complete future observation, used as the auxiliary target in World Action Models, with the “Action Effect”, in other words, the change between the current and the future observation. Effect-WAM and the PAA mechanism are presented. The Action Effect labels are the token-wise difference of DINOv3 features between o_t and o_{t+h} and occupancy transitions on 3k fixed anchors built from fused agent and wrist depth (3D). PAA takes an intermediate DiT action representation (Propose), lets two horizon-specific Effect queries read it with cross-attention (Anticipate), and feeds them back into the action tokens with a gated cross-attention (Act). At inference, the 2D and 3D heads are removed. The evaluation covers Libero, robotwin 2.0, and two real-world tasks on an xArm7 with a dexterous hand, plus an ablation on Libero and an efficiency comparison with Fast-WAM.

Critical strengths

S1: The question, "Which part of the future is actually useful for action learning?" is good, and the idea is simple. Figures 1 and 3 explain it well. S2: The model is much lighter than Fast-WAM (80.2% fewer trainable parameters and about 40% less inference memory), and it is still on par with the WAM baselines on LIBERO and RoboTwin 2.0.

Critical weaknesses

W1: The main claim, that Action Effect is a better target than the complete future, is never tested inside the policy. The “PAA w/o Effect Act” row has no E2A, so its Effect tokens never go back into the action stream. It is basically Action Only plus auxiliary supervision, and it gains +1.9. This suggests that auxiliary supervision helps, but not that this target helps more than the future target would. There is no row with the same PAA and heads trained on f(o_{t+h}) instead of the difference. Also, the last +1.5 from E2A could come from the extra cross-attention capacity, and there is no PAA row without supervision to check it.

W2: The Fig. 1(d) diagnostic also does not support the claim. Two things are mixed. Future vs Action Effect also changes the conditioning: in Fig. 1(c) the predictor gets the action, in (a) it does not. The clean comparison is Action-conditioned Future vs Action Effect. There, the probe also gets the current representation and the label is f(o_{t+h}) − f(o_t), so the two targets carry the same information. The gap can only come from how well each predictor learns. Predicting the difference may be easier to learn, and that would be fine, but it is a different claim from “more action-relevant information”. Also, it is not clear if a high action R² partly reflects the action leaking through the action-conditioned prediction. The "smaller target area" is also a bit misleading. The 2D label used for training is a dense difference over all tokens, with SmoothL1 on all of them, so the supervised target covers 100% of the map. The smaller area only appears after a threshold that is never given. Fig. 1(d) has no numbers and no error bars.

W3: Is the "effect" an effect on the scene, or just the robot moving? Both labels measure change between two frames. In the 3D label, a large part of that change is the arm and hand occupying and freeing anchors. For 2D, the paper does not say which camera is used. If the wrist view is included, the whole image changes with the camera. The paper never says whether the robot is masked out. Two details make this worse. The horizons (16, 32) match the action chunk (K = 32), so the h = 32 effect is largely where the robot ends up after the chunk. And the Effect tokens are detached from the action loss (A.1), so what they contain is decided only by the labels. If the labels are mostly robot motion, so are the tokens. Then the “Action Effect” is very close to the action itself, and the "anticipated consequences" narrative does not hold. On top of that, the heads are removed at inference, and the paper never reports how well the 2D/3D effects are predicted on held-out data, or shows that the Effect tokens encode anything about the objects. So "anticipates consequences" is claimed, not shown.

W4: The margins are within noise, and some claims go beyond the data. There are no standard deviations and no seeds anywhere.

  • LIBERO: 98.1 vs 97.6 for Fast-WAM, and LingBot-VA is higher (98.5).
  • RoboTwin: 92.2 is a tie with LingBot-VA, which is better on clean, yet the text says Effect-WAM "ranks the best". -Ablation: the margins are 0.6–1.5 points with no variance. The "clear gain" of 1.0 on Object is 5 episodes out of 500. PAA+2D is below Action Only on Object (97.6 vs 98.2). 2D alone gives +1.1 and 3D alone +0.6, but together +3.4. That needs seeds before reading anything into it.
Questions for authors

Basically, to be accepted, the main weaknesses should be fixed. I’ll rephrase what I already pointed out in weaknesses: (W1): Show that the Action Effect target contributes. The bare minimum is the same PAA and heads with target f(o_{t+h}) instead of the difference, and PAA without supervision, with the same budget. (W2): Say whether the diagnostic predictor sees the expert action. Report the actual numbers, including the Current-only reference, with error bars. Either compute Target Area on the label used in training or drop that claim. (W3): Which camera(s) are used for the 2D labels, and is the robot masked from the 2D and 3D labels? Report the 2D/3D prediction quality on held-out data, and show what happens with an object-only effect label. (W4): Add seeds and std for Effect-WAM in Tab. 1-2 and for all rows of Tab. 3. Say whether the baseline numbers are re-run or taken from the papers, and claim only what the data support (fix “ranks the best” on RoboTwin).

Reviewer_PquMOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes action effect, the changes and consequences caused by an action, as the prediction target of World-Action Models instead of pixel recordings of the robot observations widely adopted in existing work.

Critical strengths

The idea of treating only action-induced change as the prediction target is conceptually clean.

Critical weaknesses
  1. Is action effect really an effective alternative to complete future generation (line 53)? We do not see a consistent superiority over existing methods in Tab. 1 and 2. The real-world results (Sec. 4.3 and A.4) are reported without any baseline.
  2. The Fast-WAM comparison is also confusing. Fast-WAM and Effect-WAM adopt different architectures with no direct comparisons on the number of parameters. Additionally, Effect-WAM is also slower at inference, so it is not an efficient choice.
  3. Robot observations are often recorded with noise, e.g., people walking in the background, lighting condition changes, or moving distractors. This pose significant challenges to obtain reliable action effect labels, because differencing features and comparing occupancy cannot separate action-caused change from such exogenous change. Restricting supervision to changed regions may therefore make the model to concentrate more on these nuisance dynamics, making it less "robust" (line 112-113) than it sounds.
Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.