acceptodds
Withdrawn from ICLR 2027

Learning Action–Consequence Representations for Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing future-oriented vision-language-action (VLA) methods typically predict actions and future observations in parallel from the same context, without explicitly modeling how actions lead to subsequent changes in the environment. Consequently, a model may learn to anticipate what will happen next largely from scene context, without learning what will happen as a consequence of a particular action. To address this limitation, we propose a framework for learning action–consequence representations (ACE) from future supervision. ACE models future states as consequences of actions and contrasts demonstrated actions with alternative actions under the same context and observed consequence, enabling future representations to distinguish different actions and their corresponding environmental changes. ACE further emphasizes action-relevant changes by weighting interaction regions and uses task-terminal states to constrain the task relevance of consequence representations. We evaluate different forms of future supervision on LIBERO and RoboCasa under a unified training budget. Under a unified training and evaluation protocol, ACE improves the average success rate on LIBERO by 6.4 percentage points over a matched VLA baseline. On RoboCasa, ACE outperforms VLA and FOCA baselines by 5.0 and 8.0 percentage points, respectively. These results suggest that modeling how actions lead to subsequent environmental changes enables future supervision to translate more effectively into control performance than independent future prediction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Few-Shot Learning, Future Prediction, Counterfactual Learning, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cpBcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes to learn to improve action generation by contrasting action-conditioned consequence representations anchored by future observation semantics. The proposed method is a plug-in module that can be discarded during inference.

Critical strengths

The shortcut problem in multi-conditioned future prediction is interesting and important. The contrastive learning approach is a promising way to force the model to provide trustworthy future representation with respect to the actual action input.

Critical weaknesses
  1. The paper lacks a standard benchmarking results to directly compare ACE with prior works. The paper only experiments on LIBERO and RoboCasa with controlled budgets (Sec. 5.1). Even under the customized setting, ACE provides modest performance improvement with no error bar (Tab. 2).
  2. The exact action representation and coordinate frame are not specified, raising concerns about the factual correctness of the method. Considering the action augmentations (line 237), batch action donors and temporal reversal will lead to state inconsistency for absolute pose actions, and temporal reversal will also lead to state inconsistency on chunk-wise delta actions. Consequently, the model may distinguish these negatives through discontinuities only, rather than consequence understanding.
Questions for authors
  1. When constructing the contrastive learning objective in Eq. 6, this objective merely pushes away augmented actions from the ground truth future, rather than assigning possible future for the actions. For example, for temporal reversal, is it possible to use the history observation as the "future", and is it possible to use zero actions and replicate present observation as the "future"?
Reviewer_xJc8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores VLA models and their limitations that they can predict future observations but may not understand how specific robot actions can cause changes in the environment. To address this, the paper presents a method ACE (Action–Consequence Representation Learning) that allows VLA to learn the relationship between actions and their consequences in the environment. ACE explores three components of action taken, the environment change it produces, and the task outcome the action contributes to. It consists of a action-conditioned sequence predictor, a contrastive objective that helps the model learn to distinguish the actions from their synthetic alternatives (action swapping, reversal, or perturbation), and a soft- interaction-region weighting, and goal-progress supervision to focus on environmental changes and task completion. These components are used together during training and the auxiliary prediction modules are removed during inference. The method is evaluated on the LIBERO and RoboCasa robotic manipulation benchmarks. Overall, the key takeaway of the paper is that it explores how aligning actions with observed future changes can be helpful for robotic manipulation.

Critical strengths
  1. The paper identifies a necessary limitation of existing VLA-based method and explores the three axes of the action taken, the environment change it produces, and the task outcome the action contributes to. Such an understanding is important for robotic control, where different actions can produce different outcomes.

  2. The paper proposes a simple and intuitive approach of comparing the demonstrated action against alternative actions while keeping the current observation, instruction, and observed future fixed. It constructs the alternative actions such that they are not required during training.

Critical weaknesses
  1. The paper has a claim that ACE learns how different robot actions would affect the environment. The contrastive loss mainly teaches the model to distinguish correct actions from alternative actions based on the observed future, however it may not necessarily mean the model can accurately predict what would happen if a different action were executed. Table 13 shows that ACE achieves only 39.3% consequence retrieval accuracy, compared to 25% random chance and 35.3% without contrastive learning. This raises a concern that while ACE improves action discrimination, there is limited evaluation that it learns accurate action–consequence relationships. Does the contrastive objective help to improve consequence prediction or does it primarily learn a separation between action representations?

  2. The paper discusses that ACE helps the model teach the model how actions change the environment. However, the results in Figure 3 show that the model may primarily learn to distinguish actions rather than accurately represent their consequences. The cosine similarity between predicted future features and the actual future features for the model (without contrast) is 0.9963 whereas for ACE is 0.7282. This raises a concern whether ACE learns how actions affect the environment, or is its contrastive objective simply pushing the representations of different actions apart?

Questions for authors

The paper mentions that ACE generates alternative actions by using actions from other demonstrations, reversing, or perturbing actions. However, these actions may be unrealistic, may produce the same outcome as the correct action, or may be too easy to distinguish. Is there any analysis in the paper where it can be shown that the design of such alternative actions is the best way to learn a separation between action representations?

Reviewer_NaufOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ACE, a framework for learning action–consequence representations in VLA models. Instead of predicting future observations independently of actions, ACE uses action-conditioned future prediction and contrasts demonstrated actions with alternative actions under the same context. Interaction-weighted visual targets and terminal-state supervision are further introduced to emphasize task-relevant changes. The framework is evaluated on LIBERO and RoboCasa, together with action-swap and same-state execution analyses.

Critical strengths

Clear motivation and meaningful formulation. The paper identifies an important limitation of future-oriented VLA training: predicting a plausible future does not necessarily require understanding the consequences of a particular action. The within-context action contrast provides a reasonable way to encourage action-sensitive representations.

Useful mechanism analysis. Beyond standard success-rate evaluations, the paper includes action-swap tests and actual execution of alternative actions from matched simulator states. These experiments help examine whether the learned representations are sensitive to actions and whether this sensitivity corresponds to observed outcomes.

Critical weaknesses

Limited and inconsistent improvements over strong baselines. Although ACE improves LIBERO success by 6.4 percentage points over the reported VLA baseline at 50 demonstrations per task, its improvement over the additional matched LoRA baseline is only 1.0 point (46.9% vs. 45.9%). The advantage also varies across demonstration budgets and task suites. On RoboCasa, ACE outperforms the baselines at 20 demonstrations per task but underperforms the original VLA at both 100 and 5 demonstrations. These results make it difficult to establish a consistent advantage of the proposed supervision. Furthermore, matching optimizer updates does not fully control for the additional training computation introduced by ACE.

Insufficient evidence for accurate action–consequence modeling. The contrastive objective substantially improves action sensitivity, but this comes with a large decrease in future-feature agreement. In the same-state execution experiment, ACE achieves 39.3% consequence retrieval accuracy, compared with 35.3% for action conditioning without contrast and 25% chance performance. Meanwhile, its executed-future cosine drops substantially. These results suggest improved relative action discrimination, but provide limited evidence that ACE accurately predicts the actual consequences of different actions. Since the alternative actions used during training do not have ground-truth future outcomes, the connection between the contrastive objective and physically meaningful consequence prediction remains insufficiently established.

Questions for authors

Can the authors further clarify the improvement over the strongest matched LoRA baseline? Would the advantage remain under a matched wall-clock training budget, given the additional computation introduced by ACE?

How do the authors explain the substantial decrease in future-feature cosine similarity after adding action contrast? Can the method be evaluated using more direct physical outcome metrics, such as object displacement, contact states, or articulated-object state changes, rather than mainly relative feature retrieval?

How are alternative actions validated for physical feasibility and outcome differences? Since different actions can produce similar outcomes, could the contrastive objective penalize valid alternative behaviors?

What explains the inconsistent improvements across demonstration budgets and task categories, particularly the weaker results on RoboCasa with 5 and 100 demonstrations per task? Could the authors further analyze when action–consequence supervision is beneficial and when it may interfere with policy learning?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.