acceptodds
Withdrawn from ICLR 2027

EAct: Coupling Structured Effect Space and Action Trajectories for Robot Control

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World-action models guide robot control by anticipating how a scene will change. However, similar visual transitions can arise from trajectories with different motion paths, orientation changes, and gripper timing. Learning to reconstruct future observations therefore does not ensure that the resulting representation captures how the change is carried out. We introduce EAct, which couples a structured effect space with a trajectory-supervised action representation. Its geometry-regularized effect space encourages transitions with similar visual changes to have nearby effect representations. Differences between frozen current and future visual features define the similarity reference, adding relational supervision beyond reconstruction. An effect-conditioned ActionVAE learns trajectory-level execution representations from demonstrations. Through effect-guided policy learning, the policy predicts an effect, decodes it into future visual features, and uses dense transition features to predict an action latent that conditions a flow-matching expert. EAct achieves average success rates of 99.0% across the four LIBERO task suites and 93.04% under RoboTwin clean evaluation, outperforming the compared baselines on both simulation benchmarks. On LIBERO-Object, an effect-only variant achieves comparable success to the original-latent baseline with 80% fewer Stage-2 updates. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Latent Action Representations, Robotic Manipulation, Geometric Regularization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8jFtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

EAct addresses ambiguity in world-action models, where similar visual transitions can correspond to different execution trajectories. It decomposes the transition representation into a geometry-regularized effect latent and an effect-conditioned action latent. The paper reports 99.0% average success on LIBERO and 93.04% on RoboTwin clean settings.

Critical strengths
  • The paper provides detailed construction of the objectives for neighborhood alignment, and the representation analysis shows that the regularization improves the desired alignment property.
  • The learning efficiency and the transfer experiments are interesting and it shows the potential of the method to quickly adapt to new tasks.
Critical weaknesses
  • The geometry regularization is not well justified. The paper doesn't report the ablation of LnbrL_{\text{nbr}}. Section 4.4 only shows that LnbrL_{\text{nbr}} does improves the neighbor alignment, but it can't be considered as an evidence to prove that the geometry regularization does improve the performance of the model in the downstream tasks.
  • The author uses "the map from transition to execution is one-to-many" as the motivation of the action latent. But flow-matching is able to learn one-to-many mapping, and the paper doesn't provide any explanation why the flow-matching is not enough to solve the problem.
  • EAct only reports a 0.4% gain on LIBERO and 0.4% on RoboTwin compared to LaWAM. Since laWAM drops from 98.6% to 98.2% after 25k more steps of fine-tuning, it is unclear whether the gain is due to the proposed method or just a result of random seed noise. Additionally, the lack of real-world experiments makes it difficult to evaluate the generalization to real-world scenarios.
  • Some ablation results are odd. For example, the method is supposed to perform equivalently well as LaWAM after removing the action latent, but it is actually worse than LaWAM. Also it is unclear why removing the whole action latent is better than only removing the residual conditioning.
Questions for authors

see above.

Reviewer_m7tFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EAct, a robot-control framework that combines a geometry-regularized effect representation with a trajectory-supervised action representation. Building on LaWAM, it learns effect latents from visual transitions and trains an effect-conditioned ActionVAE to encode demonstrated trajectories. A downstream policy predicts both representations and uses anticipated visual features and the predicted action latent to condition a flow-matching action expert. Evaluation is conducted on LIBERO and RoboTwin, supplemented by component ablations, representation probes, and limited studies of learning efficiency and action-teacher transfer.

Critical strengths

The paper studies a reasonable distinction between observable scene changes and the execution details required to produce them. It also goes beyond aggregate success rates by examining neighborhood preservation, trajectory-information recovery, and action-latent conditioning. These analyses provide some evidence that the additional action representation captures execution information useful to the policy.

Critical weaknesses
  1. The contribution is insufficiently differentiated from closely related work. Separating desired effects from the motions that realize them has a direct precedent in CAVIN, which learns distinct effect and motion latents and conditions trajectory generation on both. LaWAM already provides the main framework of latent-transition prediction, future-feature decoding, and future-conditioned action generation. EDAR also combines executable trajectory structure with visual consequences when learning action representations. Against this background, EAct’s contribution appears to lie primarily in combining neighborhood regularization with an additional trajectory-supervised latent and its policy-conditioning mechanism. The paper does not convincingly establish why this particular formulation constitutes a substantial advance over these existing approaches. In particular, the benefit of explicitly separating the representations remains insufficiently demonstrated relative to a jointly learned representation or a simpler trajectory-latent baseline.

  2. The empirical evidence is insufficient to establish the value of the added structure and training complexity. The improvements over LaWAM are small: approximately 0.35 percentage points on LIBERO, 0.40 on RoboTwin clean, and 0.28 on RoboTwin randomized. Without independent training repetitions or uncertainty estimates, their consistency remains unclear. Although a continued-training baseline is included for LIBERO, a corresponding matched-budget control is missing for RoboTwin. More importantly, the experiments do not directly establish the contribution of neighborhood regularization to full-policy performance or the advantage of an effect-conditioned ActionVAE over an otherwise comparable trajectory-only model. All control evaluations are simulated, leaving the practical benefit of the proposed representations unvalidated on physical robots. The efficiency study provides limited additional support: it evaluates an effect-only variant on LIBERO-Object and excludes representation-training and preparation costs. Taken together, these results do not provide sufficiently convincing evidence for the proposed method’s methodological or practical significance.

Questions for authors
  1. How does the proposed effect–action factorization differ conceptually from CAVIN’s effect–motion decomposition? Beyond its integration into LaWAM, which aspect of this factorization provides a distinct contribution, and what experimental evidence supports that distinction?

  2. What is the variation in the reported improvements across independent policy-training runs? Does the RoboTwin improvement remain when LaWAM receives the same additional training budget and comparable optimization conditions?

  3. Under otherwise matched settings, how does full-policy performance change when neighborhood regularization is removed? Similarly, how does the effect-conditioned ActionVAE compare with a trajectory-only ActionVAE? These comparisons would help determine whether the proposed structure is necessary for the observed gains.

Reviewer_KpFTOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EAct, a world-action modeling framework that explicitly separates representations of visual effects and action trajectories for robot manipulation. The authors argue that similar initial and final visual observations can correspond to different execution trajectories, making future visual prediction alone insufficient for capturing how an action is performed. To address this limitation, EAct introduces two complementary representations. First, a geometry-regularized effect representation captures observable scene changes by aligning latent-space neighborhoods with similarities derived from frozen visual-feature differences. Second, an effect-conditioned ActionVAE learns trajectory-level representations from demonstrated action sequences. During policy learning, the model predicts an effect representation from the current observation and language instruction, generates future visual features, and predicts an action representation conditioned on the anticipated transition. These representations subsequently condition a flow-matching action expert. The authors evaluate EAct on LIBERO and RoboTwin 2.0, reporting average success rates of 99.0% and 93.04% on LIBERO and RoboTwin Clean, respectively. Additional representation analyses suggest that the learned effect latent better preserves visual-transition neighborhoods, while the action latent captures more execution-related information than previous transition representations.

Critical strengths
  1. A meaningful distinction between visual effects and execution trajectories. A major strength of this paper is its motivation to distinguish the intended visual outcome of manipulation from the trajectory used to achieve that outcome. Existing feature-space world-action models primarily learn representations of future visual transitions, which may not uniquely determine the corresponding execution trajectories. EAct addresses this ambiguity by introducing an effect-conditioned ActionVAE that explicitly encodes demonstrated trajectories. The representation analysis provides supporting evidence that the learned action latent contains information about intermediate motion and gripper behavior that is less accessible from conventional visual-transition representations. This distinction is conceptually meaningful and could benefit manipulation tasks where trajectory details matter despite similar initial and final observations.
  2. Extensive simulation evaluation and useful representation analyses. The paper evaluates EAct on LIBERO and RoboTwin 2.0, covering diverse manipulation tasks and both single-arm and bimanual settings. The authors also provide component ablations investigating the action representation, predictor architecture, future-feature conditioning, and action-latent injection. The representation-level experiments are particularly informative. Neighborhood regularization improves the alignment between effect representations and visual-transition similarities, while execution probes show that action latents encode more trajectory information than the original LaWAM representations. These analyses provide useful evidence that the proposed representations capture different aspects of manipulation.
Critical weaknesses
  1. Marginal performance improvements without sufficient evidence of statistical reliability. Although EAct achieves strong absolute success rates on LIBERO and RoboTwin, the improvements over its primary baseline, LaWAM, are relatively small. On LIBERO, the average success rate increases from 98.6% to 99.0%. On RoboTwin, the improvements are only 0.40 percentage points under the clean setting and 0.28 percentage points under randomized evaluation. These marginal gains raise questions about the practical significance of introducing additional representation-learning stages, a geometry-regularized effect encoder, an ActionVAE, and an action-latent prediction branch. Furthermore, the main policy results are reported without variation across independent training runs. While the evaluation includes many rollout trials, this does not establish the consistency of the improvements across training seeds. Given the high baseline performance and the relatively small differences, multi-seed evaluations are important for determining whether the reported gains are robust. Without stronger evidence of consistent performance improvements, it is difficult to assess whether the additional complexity of EAct is justified over the existing LaWAM architecture.

  2. Limited generalization and practical validation. All manipulation experiments are conducted in simulation, without real-world robotic validation. Although the paper provides a cross-dataset transfer experiment, the transferred ActionVAE still requires training a new downstream policy on the target dataset, along with embodiment and temporal adapters. Therefore, the experiment does not demonstrate zero-shot transfer across robot embodiments. Additionally, the execution representation probes are evaluated on data from the Stage-2 training distribution rather than on unseen manipulation tasks. As a result, the paper provides limited evidence that the learned effect and action representations generalize to new tasks, environments, or robot embodiments. The reported training-efficiency improvement is also limited to an effect-only variant on LIBERO-Object and excludes representation pretraining and target preparation costs. These limitations make it difficult to establish the broader generalizability and practical efficiency of the proposed framework beyond the evaluated simulation settings.

  3. Insufficient ablation to establish the contribution of the structured effect representation to downstream control. One of the main contributions of EAct is the geometry-regularized effect representation, which encourages similar visual transitions to have neighboring latent representations. The paper demonstrates that this regularization improves neighborhood preservation metrics (Table 3, Figure 4). However, for the full EAct model, it does not provide a controlled downstream policy ablation comparing training with and without the neighborhood regularization objective under otherwise identical conditions. The closest evidence is the learning-efficiency study in Section 4.5, where an effect-only variant is compared against a “Stage-1-finetuned” baseline on LIBERO-Object. Yet the paper does not clearly define this baseline, and even if it corresponds to a matched-budget effect encoder trained without the neighborhood loss, the comparison is restricted to a single suite, three Stage-2 checkpoints, and a variant without the action branch. Similarly, the paper motivates the action representation through the one-to-many relationship between visual effects and execution trajectories. However, the experiments do not directly evaluate whether the proposed representation can distinguish or generate multiple valid execution trajectories corresponding to the same visual outcome. At inference time, the model uses the predicted mean of the action-latent distribution rather than explicitly sampling different execution representations.

Questions for authors

Q1. Are the performance improvements over LaWAM statistically significant and consistent across training seeds? The reported improvements over LaWAM are relatively small, particularly on RoboTwin, where EAct improves performance by only 0.40 and 0.28 percentage points under clean and randomized evaluation, respectively. Could the authors provide results across multiple independent training seeds, including variance or confidence intervals? Additionally, could the authors quantify the additional training and inference costs introduced by EAct relative to LaWAM? Q2. Does geometry regularization improve downstream manipulation performance? The paper demonstrates that geometry regularization improves neighborhood preservation in the effect representation. However, could the authors provide a controlled ablation comparing full EAct with and without the neighborhood regularization objective? In particular, it would be useful to compare both downstream manipulation success and execution-representation quality under identical training conditions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.