acceptodds
Under review as a conference paper at ICLR 2027

Latent evolving World Action Model

Abstract

World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). In VDM-based WAMs, observations are first encoded by a VAE, and the resulting compressed latents are then processed by large video diffusion backbones to extract effective features for action generation. However, this paradigm ties WAM performance and training cost to large-scale video generation pretraining, limiting WAM efficiency and scalability. In this paper, we theoretically and empirically investigate how visual representations affect action generation in WAMs. Our results show that predictive embeddings from Joint-Embedding Predictive Architecture (JEPA) encoders better support action generation than compressed VAE latents, with I-JEPA performing best in our encoder comparison. Based on these findings, we propose LeWAM, which conditions action generation on JEPA embeddings and models environment evolution by predicting future embeddings in the same space, without relying on a video diffusion backbone. We further find that imitation learning matches demonstrated actions but does not distinguish better actions from worse ones, even though small action deviations can greatly affect task success. To address this limitation without additional environment interaction or the human oversight required for resets and safety, we introduce Demonstration-Guided DPO (DemoDPO), an offline preference refinement stage that derives preference supervision directly from demonstrations. With only 0.4B trainable parameters, LeWAM achieves an average success rate of 92.28% on RoboTwin 2.0, comparable to that of state-of-the-art VLAs and WAMs, and maintains practical effectiveness on real-world manipulation tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation, World Action Models, Visuomotor Policy Learning, Predictive Representations, Direct Preference Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LFAHOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates visual representation selection and action generation in World Action Models (WAMs). The authors first theoretically analyze and empirically compare different pretrained visual encoders for action generation, finding that I-JEPA performs best among the evaluated encoders. Based on this finding, they propose LeWAM, which uses frozen I-JEPA features as visual inputs and future-state prediction targets, jointly learning action generation and future-feature prediction with a shared Transformer. They also introduce AdaFuse to fuse visual features from different encoder layers. In addition, the authors propose Demonstration-Guided DPO (DemoDPO), which constructs offline preference pairs based on the mean squared error (MSE) between generated actions and demonstrated actions to further refine the action policy. The evaluation covers 50 tasks on RoboTwin 2.0, three real-world robotic manipulation tasks, visual representation analysis, and inference efficiency. LeWAM achieves an average success rate of 92.28% on RoboTwin 2.0, while also demonstrating lower inference latency and GPU memory consumption.

Critical strengths

A major strength of this paper is its systematic investigation of visual representation selection, together with an efficient WAM architecture. The authors compare multiple frozen visual encoders under a consistent downstream action-generation setting and find that I-JEPA representations achieve strong manipulation performance. LeWAM further avoids relying on a large video diffusion backbone for feature extraction and uses future-feature prediction to provide additional training supervision. The evaluation includes 50 simulation tasks, real-world robotic manipulation, and inference efficiency measurements, providing evidence beyond a single offline metric.

Critical weaknesses

The independent contribution of the core architecture is insufficiently validated. A key design feature of LeWAM is the use of a shared Transformer to jointly learn action generation and future-feature prediction. However, the existing ablations do not directly compare the shared predictor with separate predictors under otherwise comparable settings. Consequently, the current experiments do not establish whether the shared architecture itself provides meaningful benefits beyond those of future-feature prediction and feature fusion.

The methodological novelty and validity of DemoDPO remain insufficiently established. The method largely follows existing DPO/Flow-DPO objectives, with its main additional design being the construction of preference pairs based on the MSE between generated actions and demonstrated actions. However, actions closer to demonstrations are not necessarily more likely to succeed at the task. The paper does not sufficiently establish when this distance-based preference signal reliably reflects action quality, which limits the current evidence for DemoDPO's independent contribution.

The evidence for performance superiority over existing methods is limited. LeWAM exceeds the strongest reported baseline by only 0.08 percentage points in average success rate, with a slightly higher score on clean scenes but a slightly lower score on randomized scenes. This small margin is not inherently problematic, but its significance remains unclear. Moreover, some baseline results are taken from prior papers, and the comparability of their training and evaluation protocols is not fully established. Further clarification of the evaluation settings and uncertainty in the reported results would help substantiate the performance claims.

Questions for authors

Core architecture: Could the authors compare the shared Transformer with separate predictors for action generation and future-feature prediction under the same visual encoder, prediction objectives, and training settings? Such a comparison would help determine whether the shared architecture provides independent benefits beyond those of the other components.

DemoDPO: Why do the authors consider the MSE between candidate actions and demonstrated actions a reliable basis for ranking action quality? When multiple distinct action trajectories can achieve the same goal, how might this preference construction affect the resulting policy? Do the current results support the applicability of this assumption to the evaluated tasks?

Benchmark comparisons: Given the small average success-rate difference between LeWAM and the strongest reported baseline, and the fact that some baseline results are taken from prior papers, could the authors clarify the comparability of the methods in terms of training data, evaluation protocols, and randomized scenarios? Are results from multiple independent runs or uncertainty estimates available to establish whether this difference is meaningful?

3 comments
AuthorsOpenReview ↗

Response to Reviewer LFAH (Part 1 / 3)

We deeply appreciate your constructive review and recognition of our visual-encoder comparison, efficient WAM architecture, and simulation and real-world evaluation. We address your concerns through additional experiments and analyses below.


Q1 & W1: Does the shared Transformer provide benefits beyond those of the other components, compared with separate predictors for action generation and future-feature prediction?

To address whether a shared Transformer offers benefits over separate predictors, we design LeWAM-MoT, following the two-expert architecture of π0.5\pi_{0.5}. Specifically, its action-generation and future-feature-prediction branches use independent parameters and interact through joint self-attention under the same attention mask as LeWAM.

We keep the frozen visual encoder, training data, input sequence, prediction targets, loss weights, optimization schedule, and evaluation protocol identical to LeWAM. We choose the MoT branch widths to approximately match LeWAM’s trainable parameter budget while retaining 12 layers per branch, a head dimension of 64, and power-of-two hidden and FFN widths (1024 and 2048).

MethodTransformer branchesLayers per branchTransformer hidden dimensionAttention heads per layerDimension per headFFN intermediate dimensionTrainable parametersCleanRand.Avg.
LeWAM-MoT (w/o DemoDPO)212102418642048399M88.3285.1886.75
LeWAM (w/o DemoDPO)112128020645120402M91.3290.0690.69

As shown in the table, with comparable trainable parameter budgets, LeWAM outperforms LeWAM-MoT in both clean and randomized settings before DemoDPO post-training. This supports a shared Transformer for jointly learning action generation and future-feature prediction.


Q2 & W2: Why is demonstration MSE a reliable basis for ranking action quality, and how does it affect tasks with multiple valid trajectories?

We design DemoDPO’s preference construction as a local greedy strategy based on demonstration MSE, which works well in our experiments. We agree that being closer to the demonstration does not by itself make an action more likely to succeed, and DemoDPO does not depend on such a graded relationship. It compares only the two ends of each candidate group. When the closest and the farthest candidates are separated by a large MSE gap, we treat the closest one as preferred, and we make no judgment about candidates in between or about pairs with a small gap. Our original submission did not analyze this strategy in detail, and we thank you for highlighting this gap. Below, we address your concern through a closed-loop evaluation and a discussion of tasks with multiple valid trajectories.

(1) Closed-loop Evaluation

We conduct a closed-loop evaluation to assess whether demonstration MSE provides a useful criterion for candidate selection during task execution. In this evaluation, all paired rollouts start from the same initial simulator states. At each replanning step, the RoboTwin expert generates a successful continuation from the current state, and its next action chunk defines the local reference. The policy samples GG candidate action chunks and selects the candidate with the smallest, median, or largest distance to that reference. The median rule selects the candidate ranked ⌊G/2⌋\lfloor G/2 \rfloor-th in ascending order of distance. When G=1G=1, all three rules select the sole candidate. After the selected chunk is executed, the expert replans from the resulting observation. We evaluate G∈{1,4,8}G \in \lbrace 1,4,8 \rbrace, with G=4G=4 matching the DemoDPO setting. Because repeated expert replanning makes these rollouts time-consuming, we evaluate the eight tasks from the DemoDPO ablation on the first 10% of the original validation cases for each task, yielding 20 scenes per task.

The resulting task success rates are as follows:

Selection ruleG=1G = 1G=4G = 4G=8G = 8
Smallest–79.3880.63
Median73.1374.3873.75
Largest–70.6368.75

As shown in the table, the smallest-MSE rule consistently outperforms the largest-MSE rule, while median selection remains close to the single-candidate baseline. These results support demonstration MSE as a local greedy criterion for candidate selection on the evaluated tasks.

Moreover, the gain from increasing GG from 4 to 8 remains limited, consistent with the DemoDPO ablation in Figure 5(c). We therefore choose G=4G=4 to balance performance and candidate-sampling cost.

AuthorsOpenReview ↗

Response to Reviewer LFAH (Part 2 / 3)

(2) How does distance-based preference construction affect the retained supervision?

Recall that DemoDPO applies a threshold εpair\varepsilon_{\mathrm{pair}} when constructing preference pairs, which changes the distribution of retained training samples. A sample is labeled grasp or release if a gripper closing or opening motion starts or ends within its 32-step demonstrated chunk. Samples with no such event are labeled non-event.

εpair\varepsilon_{\mathrm{pair}}010−410^{-4}10−310^{-3}10−210^{-2}
Grasp22.4824.7335.80 (1.59×)42.41
Release8.8910.5311.02 (1.24×)14.69
Non-event68.6464.7353.17 (0.77×)42.90
Success rate71.6973.5674.5067.88

As shown in the table, filtering shifts the training distribution toward grasp and release events, giving these critical moments more preference supervision and improving task success.

However, an excessively large threshold reduces coverage of non-event actions, which may explain the decline in performance. Based on this analysis, we choose εpair=10−3\varepsilon_{\mathrm{pair}}=10^{-3} to emphasize critical gripper events while retaining coverage of non-event actions.

(3) How does this preference construction relate to multiple valid trajectories?

We agree that a single demonstration cannot represent every trajectory that may complete the same task. However, in the current setting, the training data covers diverse scene configurations but provides only one demonstrated mode per scene. Preferring candidates close to the demonstration for that scene keeps the policy within the demonstrated data distribution. This is particularly meaningful around contact events, where successful grasping and placement actions require precise spatial and temporal coordination and therefore admit fewer effective alternatives.

The current benchmark evaluates task completion rather than trajectory diversity. Thus, learning one successful mode is sufficient for the criterion used here. Moreover, evaluation uses held-out seeds and unseen scene configurations, and DemoDPO improves success on these scenes, showing that the policy generalizes the demonstrated behavior rather than simply replaying training trajectories.

For tasks where trajectory diversity itself is important, multiple demonstrations for the same state would provide a stronger basis for learning alternative valid modes. We will explore this direction in future work.


In summary, DemoDPO contributes a demonstration-guided preference construction strategy. While standard DPO can use all labeled pairwise comparisons within a candidate group for training, DemoDPO selects only the minimum- and maximum-MSE candidates and retains their pair only when the distance gap meets the threshold. This selective construction reshapes the training distribution toward critical grasp and release events, strengthening their preference supervision.

Our analyses have explained how this change in the training distribution improves task success, highlighting a contribution in preference construction beyond the underlying DPO objective.

AuthorsOpenReview ↗

Response to Reviewer LFAH (Part 3 / 3)

Q3 & W3: How comparable are the benchmark comparisons, and how should the performance difference be interpreted?

Thank you for raising this question. The compared methods use 2,500 clean and 25,000 randomized demonstrations across 50 RoboTwin 2.0 tasks for benchmark-specific training, as described in Fast-WAM, Motus, LingBot-VA, GigaWorld-Policy, and LaWAM. We evaluate 100 scenes per task per setting under the official clean and randomized configurations. The latter varies background textures, table clutter, table height, and lighting.

We use Fast-WAM as our baseline because it does not use embodied pretraining, avoiding the confounding effects of different and potentially unreleased pretraining datasets while reducing training cost. We match its training data, evaluation protocol, and evaluation seeds. To contextualize comparisons with other models, the table below summarizes their differences in embodied pretraining, task conditioning, and visual input processing alongside their performance.

In the table, resolutions are given as H×WH\times W before concatenation. Embodied PT. indicates pretraining on additional robot data. Avg. SR averages the clean and randomized success rates.

MethodEmbodied PT.Task conditioningHeadLeftRightMulti-view concatenationAvg. SR
π0.5\pi_{0.5}✓Language → PaliGemma224×224224\times224224×224224\times224224×224224\times224Per-view encoding → token concatenation79.75%
GigaWorld-Policy✓Language → UMT5-XXL192×320192\times320192×160192\times160192×160192\times160T-shaped image 384×320384\times320 → VAE85.70%
Motus✓Language → UMT5-XXL + Qwen3-VL240×320240\times320120×160120\times160120×160120\times160T-shaped image 360×320360\times320 → pad to 384×320384\times320 → encoders87.84%
LaWAM✓Language → Qwen3-VL256×256256\times256256×256256\times256256×256256\times256Per-view encoding → token concatenation91.22%
LingBot-VA✓Language → UMT5-XXL256×320256\times320128×160128\times160128×160128\times160Per-view VAE → T-shaped latent concatenation92.20%
Fast-WAM✗Language → UMT5-XXL256×320256\times320128×160128\times160128×160128\times160T-shaped image 384×320384\times320 → VAE91.83%
LeWAM✗Task ID → embedding + AdaLN256×320256\times320128×160128\times160128×160128\times160T-shaped image 384×320384\times320 → resize to 224×224224\times224 → I-JEPA92.28%

T-shaped concatenation arranges the full-width head view above the two wrist views placed side by side. LingBot-VA reverses this vertical order, placing the wrist latents above the head latent.

As shown above, LeWAM achieves competitive performance with LingBot-VA using lower-resolution visual inputs, without additional embodied pretraining or language-derived semantic information.

For a fairer comparison with language-conditioned baselines, we additionally evaluate LeWAM (Text), which incorporates task instructions encoded by UMT5-XXL, reducing the conditioning mismatch with LingBot-VA. We project the text embeddings to the predictor dimension and prepend them to the visual and action tokens, reusing the existing self-attention layers to limit the increase in trainable parameters. Total parameters, including frozen visual and text encoders, are 1.0B for LeWAM and 6.7B for LeWAM (Text).

To quantify scene-sampling uncertainty, we use a stratified bootstrap, adapting the task-stratified approach used in prior robotic manipulation work [1]. Each task–setting combination contains 100 distinct, independently sampled scenes, with one rollout per scene, allowing us to resample outcomes within each combination and recompute aggregate success rates while keeping the checkpoint and task set fixed. Bootstrap standard deviations provide standard errors (SE), and the 2.5th and 97.5th percentiles give 95% confidence intervals (CI).

MethodSettingSuccess rate (%)SE (pp)95% CI (%)
LeWAMClean93.140.34[92.45, 93.82]
LeWAMRand91.420.37[90.67, 92.16]
LeWAMAvg.92.280.25[91.78, 92.77]
LeWAM (Text)Clean93.680.33[93.01, 94.35]
LeWAM (Text)Rand92.960.36[92.25, 93.67]
LeWAM (Text)Avg.93.320.24[92.83, 93.81]

LeWAM (Text) achieves 93.32% overall success, a 1.12-percentage-point improvement over LingBot-VA's reported 92.20%. Its entire 95% CI of [92.83%, 93.81%] lies above that reported score, providing stronger evidence for the performance advantage under language conditioning.

[1] Seo, Younggyo, et al. "Multi-view masked world models for visual robotic manipulation." International Conference on Machine Learning (ICML). PMLR, 2023.

Reviewer_diTTOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LeWAM, which explores building a lightweight world-action model using pretrained visual representations. The method uses frozen I-JEPA features and combines cross-layer feature fusion with a shared predictor to jointly learn action generation and action-conditioned future representation prediction. The future prediction branch is removed during deployment. The authors also propose DemoDPO, an offline policy optimization method that constructs preferences based on the distance between candidate actions and demonstrated actions. The paper evaluates the approach through encoder comparisons, component ablations, simulation and real-world robot experiments, and inference efficiency tests. The results show some practical value, but further evidence is needed to explain why world modeling and preference optimization each improve performance.

Critical strengths
  1. The method has practical value.
    The paper shows a feasible way to achieve competitive manipulation performance without relying on large video diffusion backbones. It also provides measurements of inference latency and memory usage, giving experimental support for its efficiency advantages.

  2. The internal comparisons provide useful empirical evidence.
    The authors compare frozen visual encoders under the same main action learning setup and report component ablations and real-world robot results. These experiments help understand the effects of different representations and training components within the proposed framework.

Critical weaknesses
  1. The contribution of action-conditioned world modeling is not sufficiently isolated.
    The current ablations show that future representation supervision and AdaFuse provide complementary benefits, but they lack comparisons that remove action conditioning or replace future prediction with auxiliary prediction of current representations. Therefore, it remains unclear whether the improvements come from learning action-induced changes in the environment or mainly from additional visual supervision on the shared network. This distinction directly affects the assessment of the paper's main modeling contribution.

  2. The preference criterion and optimization contribution of DemoDPO are not directly verified. I also have concerns about how DPO is applied in this setting.
    Candidate actions are ranked entirely by their mean squared error relative to a single demonstration, but this ranking may not reflect actual task success. Unlike DPO in LLMs or VLMs, action preferences in embodied settings may require environment feedback to determine whether an action is better or even correct. This is especially important when multiple valid trajectories exist or when task success depends on precise contact timing. The existing continued supervised training baseline rules out the explanation of simply adding more training updates, but it does not separate the effect of candidate selection from DPO optimization itself. Therefore, the current evidence is insufficient to show that DemoDPO reliably identifies and reinforces actions that are more likely to succeed.

Questions for authors

Key Questions That May Affect My Evaluation

  1. Does the benefit of future prediction depend on action conditioning?
    Could the authors compare the full method with action-free future prediction and auxiliary prediction of current representations, while keeping the encoder, AdaFuse, and main training settings unchanged? This would help determine whether action-conditioned dynamics modeling is the main source of policy improvement.

  2. Can demonstration distance reliably rank candidate actions by their actual performance?
    Please clarify how action distances are normalized and how different action dimensions are handled. Could the authors execute candidate actions from the same initial simulation states to check whether MSE rankings agree with actual task success? This evaluation would not need to be part of training, so the method would remain offline. My main concern is whether the preferences optimized by DPO actually reflect better actions. A clear explanation or supporting evidence on this point could improve my score.

  3. Does DPO outperform simpler fine-tuning methods under the same candidate supervision?
    Could the authors reuse candidates generated by the same reference policy and compare DemoDPO with supervised fine-tuning on selected actions and supervised fine-tuning weighted by demonstration distance? The existing baseline that continues training on the original demonstrations does not answer this question. This comparison would better isolate the contribution of the preference optimization objective.

B. Minor Clarifications: Experimental Consistency and Hyperparameter Results (Do Not Affect My Current Score)

The following possible reporting issues concern the verifiability of the results. They are not reasons for lowering my current score.

  1. Relationship between Figure 5 and Table 5.
    Appendix C.3 lists the eight tasks used in Figure 5 and states that each task is evaluated on 100 clean and 100 randomized cases. Recalculating the averages from the corresponding tasks in Table 5 on page 26 gives 73.50% and 77.25% before and after DemoDPO, while Figure 5 and the main text report 73.31% and 76.63%. Please clarify whether these results use the same checkpoints and evaluation cases, or come from independent training runs, evaluations, or model versions. These differences do not necessarily indicate calculation errors, but the current paper does not provide enough information to explain the relationship between the two sets of results.

  2. How should the differences between nearby hyperparameter settings be interpreted?
    Figure 5 reports 76.56% and 76.63% for β=4000 and β=5000, respectively. Under the stated 1,600 evaluations per run, this difference corresponds to roughly one additional successful case. The current results are insufficient to determine whether this ranking is stable. If repeated evaluations are available, please report the variation. Otherwise, I suggest describing these settings as having similar performance and clarifying that β=5000 is the setting used in the current experiments, without placing too much emphasis on such a small difference.

7 comments
AuthorsOpenReview ↗

Response to Reviewer diTT (Part 1 / 4)

We deeply appreciate your careful and constructive review, and your recognition of the practical value of LeWAM. Your comments point out several details that our submission did not explain well, and we are very grateful for the controlled experiments you suggested. We hope the experiments and analyses below address your concerns and support a higher assessment of our work.


Q1 & W1: Does the benefit of future prediction depend on action conditioning?

Our original ablation (Table 2) compared LeWAM with and without future prediction, without separating action-conditioned dynamics learning from extra visual supervision. This distinction is central to our world modeling claim. We thank you for suggesting action-free future prediction and auxiliary prediction of current representations.

Recall of LeWAM's future prediction. During training, LeWAM predicts the future at a random horizon. For each sample, we sample δ\delta uniformly from {1,…,32}\lbrace 1, \dots, 32 \rbrace, and the future query predicts a single future state E(Xt+δ)E(X_{t+\delta}). Two inputs determine which state the query predicts. The first is the clean action prefix of length δ\delta. The attention mask hides all later actions (Figure 3c). The second is the temporal coordinate of the query's 3D RoPE, which specifies the horizon δ\delta. Predicting one state per sample is sufficient, because with enough training, uniform sampling covers every horizon.

Experimental design. Starting from LeWAM without DemoDPO post-training, we change only the future prediction branch and build three variants. The encoder (I-JEPA-H/14), AdaFuse, training data and all other training hyperparameters stay the same.

  1. LeWAM-NoAct (action-free future prediction). We remove action conditioning simply by modifying the attention mask, so that the future query cannot attend to the clean action prefix. The query retains the temporal RoPE coordinate specifying δ\delta and is trained to predict E(Xt+δ)E(X_{t+\delta}), allowing the model to distinguish prediction horizons even without action conditioning.
  2. LeWAM-NoAct-Fixed (action-free, fixed interval). Predicting a single state at a random horizon is one of the novel designs of LeWAM. To ensure a fair comparison with our baseline Fast-WAM, which predicts future states at fixed intervals (the 16th and 32nd future steps in our setting), the queries in this variant always predict E(Xt+16)E(X_{t+16}) and E(Xt+32)E(X_{t+32}), also without the action prefix. The loss is averaged over the two targets as in Eq. 6, and the loss weight is unchanged.
  3. LeWAM-Cur (current-representation prediction). Based on LeWAM-NoAct, we change the target to the current representation E(Xt)E(X_t) and set the temporal RoPE coordinate to the current time (δ=0\delta = 0).

We evaluate all variants on all 50 RoboTwin 2.0 tasks, with 100 cases per task in both the clean and randomized settings.

Results and analysis.

MethodPrediction targetAction-conditionedCleanRand.Avg.
Action Only + AdaFuse––88.7685.0086.88
LeWAM-NoActE(Xt+δ)E(X_{t+\delta}), δ∈[1,32]\delta \in [1, 32]✗87.3682.4884.92
LeWAM-NoAct-FixedE(Xt+16)E(X_{t+16}), E(Xt+32)E(X_{t+32})✗88.6485.4887.06
LeWAM-CurE(Xt)E(X_t)✗87.2884.6285.95
LeWAM (w/o DemoDPO)E(Xt+δ)E(X_{t+\delta}), δ∈[1,32]\delta \in [1, 32]✓91.3290.0690.69

The results above lead to the following conclusions:

  • Action conditioning is the source of the gain. LeWAM-NoAct differs from LeWAM only in the attention mask, yet removing the action prefix causes the largest drop and falls even below Action Only + AdaFuse. Without the actions, a future state at a random horizon has many possible outcomes, so the target becomes ambiguous. The random horizon is useful only together with action conditioning, which tells the model what happens after δ\delta steps of the given actions.
  • Action-free future prediction does not explain the gain. LeWAM-NoAct-Fixed follows Fast-WAM’s fixed interval, yet only matches Action Only + AdaFuse and remains well below LeWAM. Fast-WAM prevents video tokens from attending to action tokens, but its pretrained video-generation backbone may make action-free prediction useful for representation learning. Our future predictor operates in latent space without a pretrained backbone, and action-free prediction provides no additional benefit.
  • Visual supervision alone does not explain the gain. LeWAM-Cur stays close to Action Only + AdaFuse. We conjecture that, because the current observation is already part of the input, the model can predict its representation through a shortcut. This target adds little useful supervision and has only a modest effect on action performance.

In summary, under our random-horizon future-prediction design, the benefit depends on action conditioning, rather than on action-free future prediction or visual supervision alone.

AuthorsOpenReview ↗

Response to Reviewer diTT (Part 2 / 4)

Q2 & W2: Can demonstration distance reliably rank candidate actions by their actual performance?

We design DemoDPO to construct preferences using a local, chunk-level greedy rule. Our assumption is not that MSE ranks candidates by success probability. Two candidates whose MSE values are nearly equal may well have the same chance of success, or the slightly farther one may succeed more often. We only assume that, among candidates sampled for the same observation, the one closest to the demonstration is more likely to succeed than the one farthest from it, provided the two are clearly separated. While standard DPO can use all labeled pairwise comparisons within a candidate group for training, we sort candidates sampled from the frozen policy by demonstration MSE and retain only the minimum–maximum MSE pair when its distance gap meets the threshold.


(1) Normalization and Dimension Handling

We compute did_i as the unweighted MSE in the per-dimension z-score normalized action space, which puts joints and grippers on a comparable scale without hand-tuned dimension weights. Each dimension kk is standardized as aknorm=(ak−μk)/σka_k^{\mathrm{norm}} = (a_k - \mu_k)/\sigma_k using its training-set mean μk\mu_k and standard deviation σk\sigma_k, and did_i averages the squared error equally over all Da=14D_a = 14 dimensions (six joints and one gripper per arm). In the RoboTwin data, the gripper values lie in [0,1][0,1], while the joint dimensions span substantially wider raw ranges. Scaling each dimension by its training-set standard deviation places all dimensions on a common standardized scale, so no dimension is weighted solely because it has a larger raw scale.


(2) Closed-loop Evaluation

We thank you for suggesting a direct simulation test to assess MSE-based candidate selection in closed-loop execution. As policy execution changes the visited states, we obtain a new expert reference at each replanning step. We therefore conduct a greedy test-time scaling experiment to evaluate this local selection criterion, as follows:

  1. For each paired comparison, all rollouts start from the same initial simulator states.
  2. From the current observation, the RoboTwin expert replans a successful continuation, and its initial action chunk serves as the reference for selecting the policy’s next action chunk.
  3. After executing the selected action chunk, the resulting observation triggers the next replanning step.
  4. The loop continues until the episode succeeds or reaches the step limit.

Under G∈{1,4,8}G \in \lbrace 1,4,8 \rbrace, we compare three selection rules based on the candidate distances did_i to the reference: greedily selecting the closest candidate, greedily selecting the farthest candidate, and selecting the median-ranked candidate. The median rule selects the candidate ranked ⌊G/2⌋\lfloor G/2 \rfloor-th in ascending order of did_i. G=1G=1 provides no choice and corresponds to the policy without selection, while G=4G=4 matches the setting used in DemoDPO. Due to the computational cost of closed-loop rollouts and the limited rebuttal time, we evaluate the eight tasks from our DemoDPO ablation on the first 10% of the original validation cases for each task, yielding 20 scenes per task. The resulting task success rates are as follows:

Selection ruleG=1G = 1G=4G = 4G=8G = 8
Smallest–79.3880.63
Median73.1374.3873.75
Largest–70.6368.75

Your main concern is whether the preferences optimized by DPO reflect better actions. The Smallest and Largest rows answer this directly, because they execute the chosen and rejected candidates of a DemoDPO pair. At G=4G=4, the chosen side reaches 79.38% and the rejected side 70.63%. At G=8G=8, sampling more candidates pushes the two ends further apart, and the difference widens from 8.75 to 11.88 pp. In contrast, the median-ranked candidate (74.38% and 73.75%) is barely distinguishable from the policy without selection (73.13%). This pattern is consistent with our design choice: MSE is informative when two candidates differ substantially, which is why DemoDPO keeps only the min–max pair and discards pairs below the gap threshold. Under this rule, the preferred action of each pair is more likely to complete the task. Moreover, the gain from increasing GG from 4 to 8 remains limited, consistent with the DemoDPO ablation in Figure 5(c). We therefore choose G=4G=4 to balance performance and candidate-sampling cost.

AuthorsOpenReview ↗

Response to Reviewer diTT (Part 3 / 4)

(3) MSE Sensitivity to Precise Contact Timing

You were concerned that the MSE ranking may be unreliable especially when task success depends on precise contact timing. We indeed observe that many failures of the base policy occur at gripper events. For example, the policy often releases the mug before it hangs on the rack in hanging_mug or closes the gripper before reaching the object in place_object_basket. DemoDPO uses a threshold εpair\varepsilon_{\mathrm{pair}} to filter candidate groups, retaining only those with an MSE gap dil−diw≥εpaird_{i_l}-d_{i_w} \geq \varepsilon_{\mathrm{pair}}, where ili_l and iwi_w index the farthest and closest candidates, respectively. This filtering changes the distribution of training samples, an effect we did not analyze in the original submission. We therefore count how the retained pairs are distributed over gripper events under different εpair\varepsilon_{\mathrm{pair}}. A sample is labeled grasp or release if a gripper closing or opening motion starts or ends within its 32-step demonstrated chunk. Samples with no such event are labeled non-event.

We report the percentage of pairs in each category and the DemoDPO success rate taken from Figure 5(a) under each threshold as follows:

εpair\varepsilon_{\mathrm{pair}}010−410^{-4}10−310^{-3}10−210^{-2}
Grasp22.4824.7335.80 (1.59×)42.41
Release8.8910.5311.02 (1.24×)14.69
Non-event68.6464.7353.17 (0.77×)42.90
Success rate71.6973.5674.5067.88

The table shows how filtering changes both the training distribution and task success:

  • Larger MSE gaps occur more often at gripper events. Filtering raises the share of grasp and release events relative to uniform sampling, giving these critical moments more preference supervision.
  • The improvement involves a trade-off in the training data distribution. Moderate filtering improves task success, whereas an excessively large threshold reduces coverage of non-event actions and degrades performance. This highlights the importance of balancing event coverage.

(4) Discussion of Multiple Valid Trajectories

All reviewers raised the concern that a task may admit several valid trajectories, while a single demonstration cannot cover all of them. We discuss this point from four aspects.

  • One demonstrated mode per scene in the training data. Although the training data covers diverse scene configurations, each scene provides only one expert trajectory, without alternative demonstrations for that same scene. A policy trained on such data rarely produces trajectories far from the demonstrated one at inference. If the policy tries such trajectories, it easily drifts into out-of-distribution states and collapses. Preferring candidates close to the demonstration keeps the policy within the data distribution.
  • Key events have few feasible solutions. As shown in (3), the margin filter raises the share of grasp and release events among the retained pairs. These events allow only limited variation in feasible trajectories. When the gripper is already close to an object, the policy must find a suitable grasp region and close the gripper there. When it places an object on a target region, especially in tasks that require high precision, only a few placements succeed. In such cases, the distance to the demonstration is a meaningful preference.
  • DemoDPO improves success on held-out scene configurations. Evaluation uses held-out seeds and unseen scene configurations, ensuring that the test scenes differ from the training data. DemoDPO still improves the success rate in these scenes, so the policy does not rigidly replay the demonstrations it has seen.
  • The benchmark evaluates task completion. Some states admit several valid trajectories, and our method learns one of them in such cases. The benchmark measures task success rather than trajectory diversity, so producing multiple feasible trajectories is outside the current evaluation criterion.

Under the current task setting and evaluation protocol, preferring the candidate closest to the demonstration works well. Enabling a policy to produce multiple valid trajectories is a meaningful and challenging direction that could benefit from richer embodied data, such as multiple demonstrations collected for the same task. In future work, we will explore this capability in tasks where trajectory diversity is more important, to better assess its value.

In summary, these results support DemoDPO’s design: MSE to the demonstration is a reasonable and effective local greedy criterion for constructing preferences in our setting.

Reviewer_EehrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies visual representations for action generation and proposes LeWAM, a world-action model built on a frozen I-JEPA encoder. AdaFuse combines features from different encoder layers, and a shared predictor jointly learns action generation and action-conditioned future embedding prediction. Future prediction is removed at inference. DemoDPO further refines the policy using preferences based on the distance between sampled actions and demonstrations. The method is evaluated on 50 RoboTwin 2.0 tasks and three real-world manipulation tasks, achieving competitive performance with lower inference latency and memory usage than the evaluated WAM baselines.

Critical strengths
  1. The controlled comparison of frozen visual encoders is useful. The encoders are evaluated with the same action predictor, training data, and schedule, making the results informative for choosing visual representations for robot policies. The depth and segmentation probes provide additional analysis of the information accessible from these representations.
  2. LeWAM achieves an average success rate of 92.28% on RoboTwin 2.0, comparable to the strongest baselines in Table 2, with approximately 0.4B trainable parameters and 1.0B total parameters. Under the same hardware, denoising steps, and action horizon, Table 3 reports 31.90 ms latency compared with 102.38 ms for Fast-WAM. This is a substantial reduction in inference latency while maintaining competitive task success. The component ablations and real-world experiments further support the practical value of the approach.
Critical weaknesses
  1. The benefit of DemoDPO over simpler refinement methods is not fully isolated. Candidates are ranked by their distance to a single demonstration, which may not reflect task success when multiple valid actions exist. The additional supervised-training control is helpful, but it does not establish whether the improvement comes from pairwise preference optimization or from selecting and reinforcing candidates closer to the demonstration. Comparisons using the same candidates and similar computational budgets would help clarify this contribution.
  2. The evaluation leaves uncertainty about the smaller performance gains. No variation across training seeds is reported. The paper also does not clearly distinguish the evaluation cases used for hyperparameter selection from those used for the final results. LeWAM uses task IDs while some external baselines use language conditioning, which also limits how directly the reported scores can be compared.
Questions for authors
  1. How well does demonstration-MSE ranking agree with actual rollout success? Since this ranking determines the preference labels in DemoDPO, evidence that lower-MSE candidates are more likely to succeed would help justify its use.
  2. How does DemoDPO compare with winner-only training using the same sampled candidates and a comparable training budget?
  3. How were simulation checkpoints and DemoDPO hyperparameters selected? Were the selection cases independent of the final evaluation cases?
  4. Averaging the eight tasks listed in Appendix C.3 using Table 5 gives 73.50% for WM + AdaFuse and 77.25% for LeWAM, while Figure 5 reports 73.31% and 76.63%. Were these results obtained from different checkpoints or evaluation seeds?
  5. Future prediction provides a larger improvement when combined with AdaFuse. How does this gain change when the future predictor is trained without the clean action prefix? This would help clarify whether action conditioning is important for the observed improvement.
3 comments
AuthorsOpenReview ↗

Response to Reviewer Eehr (Part 1 / 3)

We sincerely appreciate your careful and constructive review and your recognition of LeWAM’s practical value, including its controlled comparison of frozen visual encoders, competitive performance, and reduced inference latency. Below, we address your concerns through additional experiments and analyses.


Q1 & W1: How well does demonstration MSE ranking agree with actual rollout success?

We do not assume that MSE and task success are correlated at a fine-grained level, e.g., that a candidate with marginally lower MSE is more likely to succeed. What DemoDPO relies on is a coarser property: when the closest and farthest candidates in a group differ substantially in MSE, the closest one is more likely to succeed. Each group therefore contributes at most one min–max pair, and pairs below the gap threshold are discarded. This concern is also raised by Reviewer diTT. We summarize the relevant evidence below, with further analyses provided in our response to Reviewer diTT, Q2.

(1) Closed-loop Evaluation of MSE-Based Candidate Selection

We conduct a closed-loop evaluation of MSE-based candidate selection. Paired rollouts start from identical simulator states. At each replanning step, the RoboTwin expert generates a successful continuation, whose next action chunk serves as the local reference. The policy samples GG candidate action chunks and selects the one with the smallest, median, or largest MSE to this reference. After execution, the expert replans from the resulting observation until success or the step limit. We evaluate G∈{1,4,8}G \in \lbrace 1,4,8 \rbrace with G=4G=4 matching DemoDPO. Because repeated expert replanning is time-consuming and rebuttal time is limited, we use the eight DemoDPO ablation tasks and the first 10% of each task’s original validation cases, yielding 20 scenes per task.

The resulting task success rates are as follows:

Selection ruleG=1G = 1G=4G = 4G=8G = 8
Smallest–79.3880.63
Median73.1374.3873.75
Largest–70.6368.75

To answer your question on agreement with rollout success, we focus on the two rows that correspond to a DemoDPO preference pair. Executing the minimum-MSE candidate succeeds in 79.38% of rollouts at G=4G=4, compared with 70.63% for the maximum-MSE candidate drawn from the same group. With G=8G=8, the two extremes are further apart and the difference increases from 8.75 to 11.88 pp. The median-ranked candidate performs close to the policy without selection (73.13%), so we treat the table as evidence for the min–max contrast only, not for a ranking among all candidates. In our setting, the preferred candidate of a DemoDPO pair succeeds more often than the rejected one.

(2) Discussion of Multiple Valid Trajectories

Our benchmark evaluates task completion rather than trajectory diversity. A task may admit multiple successful trajectories, and DemoDPO uses the available demonstration as a local reference for greedy preference construction. The closed-loop evaluation above supports the usefulness of this criterion for completing the evaluated tasks.

Most of the base-policy failures we observe occur at critical grasp and release events. For example, the policy releases the mug before it hangs on the rack in hanging_mug or closes the gripper before reaching the object in place_object_basket. Although successful trajectories may differ in how they approach an object, the feasible actions at these decisive moments are tightly constrained. The gripper must reach a suitable grasp region before closing, and the mug must be supported by the rack before release. Substantially different actions at these moments can therefore determine whether the task succeeds.

To examine how filtering allocates supervision, we classify retained pairs by whether the demonstrated action chunk contains a grasp event, a release event, or neither. The table reports category shares and policy success rates (%) at each threshold. Parentheses show ratios relative to no filtering.

εpair\varepsilon_{\mathrm{pair}}010−410^{-4}10−310^{-3}10−210^{-2}
Grasp22.4824.7335.80 (1.59×)42.41
Release8.8910.5311.02 (1.24×)14.69
Non-event68.6464.7353.17 (0.77×)42.90
Success rate71.6973.5674.5067.88

As shown in the table, filtering increases the share of grasp and release events in the retained training data, directing more preference supervision toward the decisive moments where failures concentrate. This redistribution explains the improvement in task success. An excessively large threshold reduces coverage of non-event actions and degrades performance, so we choose εpair=10−3\varepsilon_{\mathrm{pair}}=10^{-3} to balance critical-event supervision and coverage of other actions.

Further discussion of critical events and multiple valid trajectories is provided in our response to Reviewer diTT, Q2.

AuthorsOpenReview ↗

Response to Reviewer Eehr (Part 2 / 3)

Q2 & W1: How does DemoDPO compare with winner-only training using the same sampled candidates and a comparable training budget?

This concern is also shared by Reviewer diTT. Apart from the “winner-only” baseline, as you suggested, we also evaluate distance-weighted SFT, as suggested by Reviewer diTT. All comparisons use candidates generated by the same reference policy and comparable training budgets. The “winner-only” baseline corresponds to “SFT on selected actions” in our response to Reviewer diTT, Q3. It fine-tunes the selected candidate actions, while DemoDPO uses the same candidates to construct preference pairs.

MethodCleanRand.Avg.
LeWAM (without post-training)73.8872.7573.31
+ SFT on demonstrations74.1372.3873.25
+ SFT on winner-only actions74.6373.5074.06
+ Distance-weighted SFT70.0068.6369.31
+ DemoDPO77.2576.0076.63

As shown in the table, distance-weighted SFT performs worse than the original LeWAM policy, while winner-only SFT improves over it but remains below DemoDPO.

To further understand this difference, we evaluate “winner-only” SFT and DemoDPO using Pass@k\mathrm{Pass}@k. For each evaluation scene, we sample kk rollouts and count the scene as successful if at least one rollout completes the task. The results are shown below:

MethodPass@1Pass@2Pass@3
+ SFT on winner-only actions74.0676.8178.06
+ DemoDPO76.6377.8878.19
Gap (DemoDPO − winner-only SFT, pp)+2.57+1.07+0.13

The performance gap becomes much smaller as the number of samples increases and is nearly absent at Pass@3. This suggests that “winner-only” SFT can learn useful successful actions, but does not produce them as reliably from a single sample. This behavior arises because DemoDPO’s pairwise objective jointly increases the likelihood of preferred actions and suppresses dispreferred actions, concentrating the policy distribution around preferred actions.


Q3 & Q4 & W2: How were simulation checkpoints and DemoDPO hyperparameters selected, and why do Figure 5 and Table 5 report different results?

Thank you for highlighting this gap. We clarify the procedure below.

To avoid overfitting to the test set, we use the validation set for checkpoint and hyperparameter selection, and a separate test set for final evaluation. The corresponding results in Figure 5 and Table 5 use the same checkpoints, evaluated on these two separate scene sets. In the RoboTwin evaluation script, the first episode uses seed 100000(1+s)100000(1+s), and subsequent episodes increment this seed by one while skipping scenes that the expert fails to solve, until 100 valid cases are collected. The Table 5 test set with s=42s=42 starts from seed 4,300,000, following the evaluation setting of our Fast-WAM baseline, while the Figure 5 validation set with s=0s=0 starts from seed 100,000, and the two sets share no scenes.

For each model, we select a single checkpoint based on the average success across the eight validation tasks and use it for all test tasks, rather than selecting a separate best-performing checkpoint for each test task.


Q5: How does the future prediction gain change when the future predictor is trained without the clean action prefix?

Thank you for raising this question. Recall that, during training, LeWAM samples δ\delta uniformly from {1,…,32}\lbrace1,\dots,32\rbrace and predicts the future representation E(Xt+δ)E(X_{t+\delta}). The target horizon is specified by both the clean action prefix of length δ\delta and the temporal coordinate δ\delta of the query's 3D RoPE. Following your suggestion, we train LeWAM-NoAct, an action-free future predictor, by masking the clean action prefix while keeping the encoder, AdaFuse, training data, and all other hyperparameters unchanged. Both LeWAM-NoAct and LeWAM are evaluated without DemoDPO post-training. We evaluate them on all 50 RoboTwin 2.0 tasks with 100 clean and 100 randomized cases per task.

The results are as follows:

MethodAction conditionedCleanRand.Avg.
Action Only + AdaFuse–88.7685.0086.88
LeWAM-NoActNo87.3682.4884.92
LeWAM (w/o DemoDPO)Yes91.3290.0690.69

As shown in the table, removing the clean action prefix eliminates the improvement over Action Only + AdaFuse. This is because the same observation and prediction horizon can correspond to multiple future states under different action sequences, making the prediction target ambiguous. This result shows that the benefit of random-horizon future prediction depends on action conditioning, rather than on additional visual supervision alone.

Further control experiments on action conditioning and visual supervision are provided in our response to Reviewer diTT, Q1.

AuthorsOpenReview ↗

Response to Reviewer Eehr (Part 3 / 3)

W2: How do evaluation uncertainty and task-ID versus language conditioning affect the interpretation of the small performance gains?

As described in Appendix C.1, we use task IDs instead of language instructions to remove text-representation effects and focus the controlled study on visual embeddings. Your concern highlights a distinction between this controlled analysis and comparison with external baselines, some of which use language conditioning. To reduce this conditioning mismatch, we additionally evaluate LeWAM (Text), which incorporates task instructions encoded by UMT5-XXL, matching the text encoder used by our Fast-WAM baseline. This allows us to examine LeWAM's performance with language conditioning alongside the original task-ID setting.

Given the substantial computational cost of training and the limited rebuttal period, we have not conducted experiments with multiple independent training seeds. However, we have analyzed variability in test performance.

Specifically, each of the 50 tasks is evaluated on 100 distinct, independently sampled scenes in each of the clean and randomized settings, giving 10,000 rollouts per model with one rollout per scene. To estimate how the measured success rate varies with the sampled scenes, we resample the recorded outcomes within each task and setting and recompute the overall success rate. This stratified bootstrap gives the standard errors (SE) and 95% confidence intervals (CI) reported below:

MethodOverall success rate (%)SE (pp)95% CI (%)
LeWAM92.280.25[91.78, 92.77]
LeWAM (Text)93.320.24[92.83, 93.81]

The original LeWAM achieves comparable performance to LingBot-VA, with no established statistically significant advantage. With language conditioning, LeWAM's entire 95% confidence interval lies above LingBot-VA's reported score of 92.20%.

Further comparisons of task conditioning, pretraining, and input settings, together with the detailed uncertainty analysis, are provided in our response to Reviewer LFAH, Q3 & W3.

Comments on the paper

5 comments
Reviewer_diTTOpenReview ↗

Rethinking what future prediction contributes

After rethinking the ablation results, I am lowering my score from 2 to 1. Action-only training already achieves 86.17% on RoboTwin 2.0 with the same frozen visual encoder. Adding future prediction alone raises this to 87.02% (+0.85 points), whereas with AdaFuse the gain is larger: 86.88% to 90.69% (+3.81 points).

I ask the authors to explain what they attribute to future prediction itself and what they attribute to its interaction with AdaFuse, using the existing results. I am not requesting more experiments. I appreciate the rebuttal and find the engineering convincing, but the clearest takeaway is that these components work well together in LeWAM. I do not find the broader research insight sufficient for acceptance at ICLR.

Reviewer_diTTOpenReview ↗

What does world modeling add to an already strong behavior-cloning policy?

My reading is that LeWAM’s core policy is still trained by behavior cloning: it learns to generate demonstrated actions from the current observation. Future prediction provides auxiliary supervision through the shared network, and DemoDPO refines the policy using preferences derived from demonstrations. At deployment, the policy outputs actions directly; it does not compare candidate actions by their predicted consequences.

This distinction matters because action-only behavior cloning already achieves 86.17%. Adding future prediction alone raises this to 87.02%; with AdaFuse, the corresponding gain is larger (86.88% to 90.69%). I therefore read the evidence as showing that future prediction improves a strong imitation policy, particularly when paired with feature fusion. I would appreciate the authors’ interpretation of what these results establish about the role of world modeling beyond this auxiliary training effect.

AuthorsOpenReview ↗

We Argue That Future Prediction Is Essential Because AdaFuse Benefits from It

Thank you for listing these partial results. In the Conclusion section of our paper, we pointed out the role of AdaFuse in integrating complementary features across encoder layers. Our point was that using these cross-layer features helps with future prediction, although we did not state this explicitly.

To clarify, the table below shows all four configurations, including one comparison missing from your comment. This comparison directly supports our conclusion that AdaFuse benefits from Future Prediction.

All configurations use action generation. Success rates are in percent and gains are in percentage points.

Future predictionNo AdaFuseAdaFuseAdaFuse gain
Off86.1786.88+0.71
On87.0290.69+3.67
Prediction gain+0.85+3.81—

We argue that future prediction is essential to our final method because AdaFuse benefits from future prediction. Without future prediction, adding AdaFuse raises the success rate from 86.17% to 86.88%, a gain of only 0.71 points. This is the comparison that was missing from your comment. With future prediction, adding AdaFuse raises the success rate from 87.02% to 90.69%, a gain of 3.67 points. Taken together, these results show that AdaFuse contributes substantially more when paired with future prediction.

Viewed from the other direction, future prediction also benefits from AdaFuse. Without AdaFuse, adding future prediction raises the success rate from 86.17% to 87.02%, a gain of 0.85 points. With AdaFuse, adding future prediction raises the success rate from 86.88% to 90.69%, a gain of 3.81 points.

Taken together, AdaFuse and future prediction thus bring out the best in each other, with each amplifying the gain from the other.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.