acceptodds
Withdrawn from ICLR 2027

LEON: Operator-Structured Transitions for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Latent world models for embodied agents model how system states evolve under action. In robot manipulation, World Action Models (WAMs) couple such predictions to policy learning as supervision or future conditions for action generation. Yet their latent transitions typically rely on general-purpose predictors, leaving dynamical structure to be learned implicitly from data. We derive a latent-transition formulation from controlled mechanical evolution. A common finite-dimensional projection of observable dynamics yields a shared operator family affine in physical control and an explicit projection residual. Guided by these results, we introduce the Latent Evolution Operator Network (LEON), which adapts pretrained visual features into learned observables, learns latent control coefficients from the WAM context, and combines factorized operator evolution with complementary additive updates. We instantiate LEON in prediction-supervised policy learning and future-conditioned action generation while preserving the original prediction targets and policy interfaces. Across LIBERO, LIBERO-Plus, and RoboTwin 2.0, LEON improves average success rates over the corresponding WAM baselines and matched transition controls. Ablations support the joint representation and transition design, and controlled-system diagnostics show improved state-range extrapolation. These results establish dynamics-derived latent-transition structure as an effective inductive principle for latent WAMs.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Latent World Models, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3BXJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Koopman-inspired transition module for WAMs. It jointly learns control coefficients and a shared operator family, combining operator-based evolution with a nonlinear additive correction. It replaces the classic Transformer transitions in VLA-JEPA and LaWAM while retaining their prediction targets and policy interfaces. Evaluation includes manipulation benchmarks, ablations, and controlled physical systems, reported in policy success and state-range extrapolation.

Critical strengths
  1. I find the joint learning of observables and structured transitions well motivated. Demonstrating the same formulation in both prediction-supervised policy learning and future-conditioned action generation makes its applicability more convincing than that within a single WAM architecture.

  2. LEON outperforms matched Transformer and residual MLP baselines by 5.1 and 5.4 on LIBERO-Plus, supporting the value of its structured transition beyond predictor reinitialization or generic nonlinear capacity.

Critical weaknesses
  1. Learned observables and Koopman-based state prediction already appear in Efficient Dynamics Modeling in Interactive Environments with Koopman Theory (2024) and Learning Bilinear Models of Actuated Koopman Generators from Partially Observed Trajectories (2024). In LEON, the matched WAM comparisons establish an advantage over generic predictors and internal variants, rather than over an existing learned Koopman transition adapted to the same interface. This leaves unclear how much additional methodological value LEON provides beyond incorporating established operator-learning ideas into modern WAMs.

  2. Parameter sharing is explicit in the architecture; its existence is not in question. What remains less clear is what consistent dynamical behavior the shared operators capture when their coefficients depend nonlinearly on the current representation and the correction is itself an MLP. The component ablations and controlled-system diagnostics provide only indirect evidence about the learned decomposition. I do not require recovery of true physical laws; an operator-level analysis would help explain what the physics-motivated structure contributes beyond an effective conditional predictor.

  3. Separately, the policy results use one training run per variant. Repeated rollout evaluations do not measure training variability, which limits confidence in the smaller gains, particularly the 0.53-percentage-point average improvement on RoboTwin 2.0.

Questions for authors
  1. Which specific aspect of LEON constitutes the main advance over earlier learned Koopman or bilinear transition models? I appreciate a comparison with one such model under the same WAM interface and comparable training budget, with training-run variability reported.
Reviewer_5Y9cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LEON, an operator-structured transition module for latent World Action Models. Motivated by control-affine observable dynamics, LEON combines learned observable representations, input-dependent modulation of shared operator directions, and a nonlinear complementary update. It replaces the transition modules in VLA-JEPA and LaWAM while preserving their prediction targets and policy interfaces. Evaluation covers LIBERO, LIBERO-Plus, RoboTwin 2.0, and controlled dynamical systems. The results show improved policy success rates and extrapolation performance, with matched predictor comparisons and component ablations supporting the proposed design. The reported validation is entirely in simulation.

Critical strengths

I find the central idea interesting and worth pursuing. Introducing structured transitions into latent predictive models is a promising direction, and the paper provides encouraging evidence of its usefulness.

  1. A motivated architectural contribution with applicability across two policy interfaces. LEON introduces an explicit operator structure into latent transitions, combining shared evolution directions with conditional modulation and nonlinear correction. Its integration into both VLA-JEPA and LaWAM demonstrates applicability to prediction-supervised policy learning and future-conditioned action generation. Preserving the existing prediction targets and policy interfaces also makes the approach practical to incorporate into established models.
  2. Useful empirical evidence, particularly under distribution shift. The paper evaluates multiple simulation benchmarks and includes matched Transformer and Residual MLP controls. On LIBERO-Plus, LEON achieves 80.6% average success, compared with 75.5% and 75.2% for these controls. These results provide meaningful evidence beyond the nearly saturated standard LIBERO benchmark. The controlled-system diagnostics further help investigate the motivation behind the structured predictor.
Critical weaknesses

Despite the promising idea, the current evidence leaves uncertainty about the reliability of the improvements and the specific contribution of the learned representation and operator structure.

  1. Training variability and some comparison conditions remain insufficiently controlled. The robot-policy experiments generally report one training run per model or variant. Multiple rollout seeds characterize evaluation variability for fixed weights, but do not establish robustness to training randomness. Although the matched VLA-JEPA [1] study is valuable, the LaWAM [2] comparisons rely on released checkpoints or published results rather than equally controlled retraining. This makes the smaller gains, particularly on RoboTwin, harder to attribute confidently to LEON. Independent training runs and a matched LaWAM training baseline would materially strengthen the empirical conclusions.
  2. The representation-level explanation is only partially isolated by the experiments. The matched MLP comparison supports the complete architecture, but the observable-adaptation ablation simultaneously removes representation mappings and changes the evolution width from 768 to 1024. Consequently, it does not cleanly isolate the benefit of learning an observable space suited to structured dynamics. The controlled-system experiments offer supporting evidence, but use settings substantially simpler than the visual robot models. More direct measurements in the robot feature space would strengthen the connection between the motivating dynamics argument and the observed policy improvements.

References

[1] Jingwen Sun et al. VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model. arXiv preprint arXiv:2602.10098, 2026.

[2] Jialei Chen et al. LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies. arXiv preprint arXiv:2606.15768, 2026.

Questions for authors
  1. Training reliability and matched comparisons: Could you report independent training runs for LEON and the main matched predictor controls? For the LaWAM [2] integration, could you additionally compare against the original predictor trained from the corresponding initialization with the same data, update budget, and evaluation protocol?
  2. Observable adaptation: Could you provide an ablation that keeps the evolution width fixed and approximately matches parameter counts while varying the observable mapping? Reporting both future-feature prediction error and policy success would help isolate the contribution of the learned observable representation.
  3. Prediction and generalization: Can the robot-model experiments establish whether LEON’s policy gains accompany improved prediction against the same frozen feature targets, particularly under distribution shift or at longer prediction horizons? This would help assess how directly the structured transition improves predictive modeling in the deployed framework.
  4. Figure 1 and the implemented information flow: Please identify the cameras and timestamps associated with the multi-view inputs and make the visual input to the VLM explicit. Please also distinguish latent action tokens from executable actions and indicate which connections operate during training and inference in VLA-JEPA [1] and LaWAM [2]. This would resolve ambiguities in how the diagram represents the implementations described in the text.
  5. Real-robot experiments: A small set of real-robot experiments would be valuable for assessing whether the benefits persist under real sensing noise, contact variation, and execution errors.
  6. Broader applicability: Testing LEON within another JEPA-style world-model framework, such as LeWorldModel [3], could also clarify its broader applicability, provided the training resources and control interfaces are appropriately aligned.

References

[1] Jingwen Sun et al. VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model. arXiv preprint arXiv:2602.10098, 2026.

[2] Jialei Chen et al. LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies. arXiv preprint arXiv:2606.15768, 2026.

[3] Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels. arXiv preprint arXiv:2603.19312, 2026.

Reviewer_BJ69OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes LEON, a potential transformation module for the world action model in robot manipulation, based on theoretical deductions such as Koopman operator. It adapts pre trained visual features into observables through token by token observable representation mapping; Modulation of a family of shared operators using implicit control with a bounded range of values, represented by observed quantities and model actions for joint prediction; And use a complementary additive branch to absorb changes outside the operator family; After multi-layer updates, it is read back and mapped back to the original prediction space. LEON can directly replace the Transformer transfer modules of VLA-JEPA and LaWAM, while retaining the original prediction targets, feature space, and policy interfaces. LEON has improved the success rate of LIBERO, LIBERO Plus, and RoboTwin 2.0 simulation robot tasks.

Critical strengths

LEON is a standalone latent transition module. The same set of operator structured transfer can directly replace the Transformer transfer blocks of two types of WAMs with different prediction policy interfaces - VLA-JEPA (predicted future only for training supervision) and LaWAM (predicted future directly conditional action generation), while retaining their respective prediction spaces, prediction targets, and policy interfaces; Two instances use the same transfer formula, and the action fusion mapping is adapted according to their respective WAM action representations. Due to significant differences in the two interfaces and their available action representations, this indicates that the module can be directly replaced between different WAM designs without the need for interface redesign.

Critical weaknesses
  1. The paper uses an extensive formal derivation to justify its transition design. But several symbols the argument depends on are never defined, or are reused with different meanings, so I can neither follow nor verify the chain. For example: (1) the differential operator DD in DΦ(s)D\Phi(s) and DgDg is never defined, and the same letter DD later denotes the learned diagonal base in the operator parameterization; (2) PΦP_\Phi is introduced only as "a fixed linear projection", which projection it is, on which function space it acts, and what II means in (I−PΦ)(I-P_\Phi) are settled only in the appendix, although the value of GjG_j depends on that choice; (3) the measure μ\mu, the space L2(μ)L^2(\mu) and the linear-independence assumption that make GjG_j well defined appear only in the appendix, while Section 3 already relies on them; (4) ⊙\odot, LN, Mean and the expectation notation are used without definition; (5) Φ\Phi (the theoretical observable map) and φ\varphi (the learned map) are nearly identical symbols for two different objects; (6) the basis G~k{\tilde G_k} and the map TT are introduced with no convention for their shapes and no statement that TT depends on the choice of basis. This matters to my assessment beyond readability: the paper's central claim is that the architecture realizes the derived structure, and with these symbols left unpinned a reader cannot determine which derived object a given implementation choice corresponds to.
  2. The evaluation is confined to a narrow set of simulated manipulation benchmarks; a harder benchmark or real-robot validation would be needed to support the generality that is claimed. All policy results come from LIBERO, LIBERO-Plus and RoboTwin 2.0, and the only evidence outside these three is a set of three low-dimensional controlled systems; no real-robot or sim-to-real evaluation is reported. The derivation itself rests on assumptions, a smooth interaction regime, control-affine dynamics, and observables that are functions of the full physical state. And it is exactly these assumptions that a harder, more diverse benchmark or a physical robot would stress, since real contact-rich manipulation is where a smooth-regime assumption is most likely to fail. The picture is further weakened by the fact that one of the three benchmarks is close to saturation while the improvement on the randomized condition of another is marginal, so the positive evidence comes mainly from settings of the same style. This is consequential because the abstract claims that the results "establish dynamics-derived latent-transition structure as an effective inductive principle", which is a statement of generality that the current evidence, three simulated benchmarks of one manipulation family does not yet support.
Questions for authors
  1. Several symbols the derivation depends on are either undefined or reused with different meanings. So Could the authors provide a complete symbol table and spell these definitions out in the main text, together with the derivation steps that are currently only asserted.
  2. The paper claims that the results "establish dynamics-derived latent-transition structure as an effective inductive principle", but the evidence only comes from three simulation benchmarks of the same type of operational task. Can you supplement the results on stronger or more diverse benchmarks, such as RoboDojo or real machine results? The rich contact operation of real machines is precisely the scenario where the assumptions of "smooth interaction interval" and "control affine" relied upon for derivation are most likely to fail, and such results can explain how far the structure can be transferred.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.