acceptodds
Under review as a conference paper at ICLR 2027

MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling

Abstract

World modeling has emerged as an effective co-training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states. However, most WAMs predict future states in the native representation space of pretrained visual backbones, resulting in high-dimensional targets with substantial training cost. We introduce MiniWAM, which instead predicts compact future representations learned from privileged current–future transitions. To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse-dynamics supervision with feature reconstruction to emphasize control-relevant transition information while preserving useful future-state information. With the learned PRISM encoder frozen, MiniWAM is trained to jointly predict the resulting targets and robot actions from current observations. With fewer native future feature tokens, MiniWAM consistently outperforms native future-feature prediction with both DINOv3 and WAN2.1 VAE features, while achieving up to an speedup in world–action training. At 0.25B parameters, MiniWAM is already competitive with substantially larger WAMs on LIBERO, LIBERO-Plus, and Robotwin 2.0 simulation benchmarks. Representation analyses further show that PRISM contributes behavioral structure beyond feature reconstruction alone. These results demonstrate that effective world–action modeling does not require predicting native visual futures, and that compact predictive representations provide a strong and substantially more efficient target for policy learning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world action models, robot learning, visuomotor policy learning, representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_j3sKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MiniWAM, an efficient World Action Model that replaces high-dimensional future visual prediction with compact, control-relevant predictive representations. The key component is PRISM (Predictive Representations via Inverse Spatiotemporal Modeling), which learns a compact representation of current-to-future visual transitions through two complementary objectives: inverse-dynamics prediction and future-feature reconstruction. The proposed framework consists of two training stages. First, PRISM learns a compressed representation from current and future visual features using inverse-dynamics supervision and reconstruction. Second, the PRISM encoder is frozen, and MiniWAM jointly predicts the learned representation and robot actions from current observations using a Mixture-of-Transformers architecture with flow matching. The method is evaluated on LIBERO, LIBERO-Plus, and RoboTwin 2.0 using both DINOv3 and WAN2.1 VAE visual features. Compared with directly predicting native future features, MiniWAM achieves higher manipulation success rates while substantially reducing world-modeling computation. With DINOv3 features, it improves LIBERO success from 93.2% to 97.2% and LIBERO-Plus-2K success from 60.9% to 74.1%.

Critical strengths
  1. Substantial improvements in training efficiency while maintaining competitive manipulation performance. A major strength of this paper is its demonstration that predicting high-dimensional native visual features is not necessary for effective world-action co-training. The results make the proposed approach practically relevant for developing computationally efficient robot learning systems.
  2. Well-structured empirical evaluation with meaningful ablation and representation analyses. The paper provides relatively comprehensive experimental validation across three simulation benchmarks, including robustness evaluations under distribution shifts. Also, the representation analyses, including cross-backbone CKA and behavior-based nearest-neighbor retrieval on unseen tasks, provide additional evidence that PRISM captures behaviorally relevant transition information.
Critical weaknesses
  1. Limited generalization and lack of real-world validation. Although MiniWAM demonstrates strong results on LIBERO, LIBERO-Plus, and RoboTwin 2.0, all experiments are conducted in simulation, with no real-world robotic validation. Furthermore, the experiments primarily evaluate performance on trained task distributions and robustness under controlled perturbations. The paper does not evaluate zero-shot or few-shot generalization to unseen manipulation tasks or transfer across environments and robot embodiments. While the authors analyze PRISM representations on unseen LIBERO-90 tasks, these experiments examine representation similarity rather than downstream policy performance. Additionally, PRISM is trained separately for each downstream setting, leaving its transferability unclear.
  2. Limited scalability of PRISM due to its reliance on action-labeled data. The proposed PRISM framework relies on inverse-dynamics supervision, which requires action-labeled current–future transitions during representation learning. While this is feasible for robot demonstration datasets, it limits the direct applicability of PRISM to large-scale unlabeled video datasets, including egocentric human demonstrations and internet videos. This is particularly relevant because one of the major advantages of world modeling is the ability to leverage diverse visual observations without requiring corresponding robot actions. Furthermore, the inverse-dynamics objective is defined over the action space of the training embodiment, raising questions about how the learned representations can transfer across robots with different action spaces. Although MiniWAM can benefit from pretrained visual backbones, the PRISM representation itself still requires action supervision. This potentially limits the scalability and general applicability of the proposed representation-learning framework.
  3. Insufficient evidence that PRISM provides benefits beyond simple future-feature compression. While the paper demonstrates that PRISM outperforms native future-feature prediction, it remains unclear whether these improvements primarily result from learning control-relevant representations through inverse-dynamics supervision or simply from reducing the dimensionality of the prediction targets. Although the authors provide ablations comparing inverse-dynamics supervision, feature reconstruction, and their combination, they do not compare PRISM against simpler compression strategies, such as spatially and temporally pooled future features, PCA-based compression, or lightweight feature compression with matched token budgets. Such comparisons are particularly important because the main contribution of PRISM is not merely computational efficiency, but the claim that its learned representations provide more effective world-modeling targets. Without these baselines, it is difficult to establish whether the proposed two-stage representation-learning procedure is necessary or whether similar performance and efficiency improvements could be achieved through substantially simpler alternatives.
Questions for authors

Q1. Is PRISM necessary, or would simpler future-feature compression achieve similar results? The paper demonstrates that PRISM outperforms native future-feature prediction and provides ablations on inverse-dynamics supervision and feature reconstruction. However, it remains unclear whether the improvements primarily come from the proposed representation-learning objective or simply from reducing the dimensionality of future prediction targets. Could the authors compare PRISM against simpler compression strategies, such as spatial/temporal pooling or PCA-based compression, under matched token budgets and training conditions? This comparison would help establish whether the proposed two-stage PRISM training is necessary to achieve the reported performance and efficiency improvements.

Q2. How well does PRISM generalize to unseen tasks and environments? The paper evaluates PRISM representations on unseen LIBERO-90 tasks, but these experiments focus on representation similarity rather than downstream policy performance. Furthermore, PRISM is trained separately for each downstream setting. Could the authors evaluate whether a PRISM encoder trained on one task distribution can be reused for unseen tasks without retraining? For example, could a PRISM encoder pretrained on LIBERO-40 be frozen and used for policy training on LIBERO-90, with comparisons against native future-feature prediction? Such experiments would help clarify whether PRISM learns transferable predictive representations or primarily task-specific representations.

5 comments

We thank Reviewer j3sK for the detailed and constructive review. We address each point below.

W3 & Q1. Is PRISM necessary, or would simpler compression suffice?

We thank Reviewer j3sK for this question and agree that simpler compression baselines are important for isolating what PRISM contributes. We show that PRISM is necessary, while simpler future-feature compression doesn't achieve similar results from two experiments that separate the size of the target from what it encodes.

1. PRISM beats pooling to the same size. On RoboTwin 2.0 at a 16-frame horizon, we train the same model with three world targets: native DINOv3 features, DINOv3 features pooled in space and time to the size of PRISM, and PRISM. Pooling barely improves on native features (+0.6 points), whereas PRISM gains 7.7 points.

World targetRoboTwin 2.0 Mean SR (%)
Native DINOv327.7
Pooled DINOv3 (PRISM-sized)28.3
PRISM35.4

2. PRISM beats reconstruction-only compression at every bottleneck size. Table 4 compares PRISM with reconstruction-only bottlenecks of identical size:

BottleneckReconstruction onlyPRISM
4×469.574.1
2×263.367.8
1×157.061.7

LIBERO-Plus-2K success rates (%), from Table 4.

The gain therefore does not come from a smaller target alone: pooled and learned compression of the same size both underperform PRISM. What PRISM encodes brings the accuracy gain, while its compact size brings the efficiency. We have added the pooled baseline to Appendix E.1.

W2. Limited scalability

We thank the reviewer for this question. We first clarify the goal of the paper, then show that MiniWAM can (a) use action-free video to improve performance and (b) transfer PRISM encoders across robots with different action spaces.

1. Goal of the paper. We agree that inverse dynamics requires action labels in Stage 1, and that learning from action-free video is an important advantage of world modeling. Our goal, however, is narrower: a compact target for the world branch of a WAM that makes world–action co-training more efficient and more effective. Like standard WAMs and VLAs, MiniWAM trains on action-labeled robot demonstrations, and Stage 1 uses only those same demonstrations, so PRISM adds no labeling requirement.

2. MiniWAM benefits from action-free video and transfers across robots with different action spaces. Although pre-training on large-scale video or across embodiments is not the focus of this paper, MiniWAM's design lets it scale to video data.

(a) Action-free video. Both of MiniWAM's stages can learn from it:

  • Stage 1: reconstruction needs no actions. The PRISM objective combines inverse dynamics and feature reconstruction; reconstruction alone needs no action labels, so it can train the encoder on video.
  • Stage 2: the world expert is a separate DiT. MiniWAM is a Mixture-of-Transformers with separate world and action DiTs, so the world DiT can learn future prediction from video without the action loss.

To test this, we train on LIBERO-10 with actions for only 10 demonstrations per task (100 of 388) and use the other 288 as action-free video. These videos train the encoder through reconstruction only and the world expert through the world loss only, with action tokens left as pure noise. A third run uses LIBERO-Plus's perturbed replays of the same 288 demos instead. All runs use the same labeled windows per update and the same number of updates; the video windows are added on top.

Action-free videoLIBERO-10 SRLIBERO-Plus SR
None78.639.4
The other 288 demos88.043.6
The other 288 demos from LIBERO-Plus84.651.2

Action-free video raises success by 9.4 points on LIBERO-10, so the inverse-dynamics term does not prevent MiniWAM from learning from video. Perturbed video instead raises LIBERO-Plus success by 11.8 points, suggesting that MiniWAM can absorb the visual diversity that videos would provide, while learning action-relevant information from demonstrations. We have added these experiments to Appendix E.6.

(b) Transfer across robots with different action spaces. Although the inverse-dynamics objective is defined over the training robot's action space, a frozen PRISM encoder trained on RoboTwin 2.0 (bimanual, 14-D absolute joints) and reused on LIBERO-10 (single arm, 7-D delta end-effector) outperforms native prediction (88.4% vs. 79.8%; W1 & Q2). Since the Stage-1 inverse-dynamics model sees only PRISM, without the current observation, instruction, or proprioception, the encoder must capture the motion itself, yielding features that stay useful across embodiments. Unlike WAMs that tile all views into one image (e.g., Fast-WAM), PRISM uses one encoder per camera role (e.g., main or wrist view), so the same encoders can be reused across embodiments.

W1 & Q2. Generalization to unseen tasks, transfer of PRISM, and real-world validation

We thank Reviewer j3sK for this valuable question, which tests whether PRISM learns transferable representations. Our LIBERO-90 analysis characterizes what PRISM encodes; we agree that it is not evidence of downstream policy generalization. Although training separately for each downstream setting is the standard practice for VLAs and WAMs, and MiniWAM adds no additional labels, we show that a frozen PRISM encoder can be reused for unseen tasks as the reviewer requested, and then report real-world experiments.

1. A PRISM encoder trained on one task distribution can be frozen and reused for unseen tasks. Training and evaluating policies on all 90 LIBERO-90 tasks did not fit within the rebuttal period with our hardware budget, so we run two smaller versions of the suggested experiment with the same structure: a PRISM encoder is trained on other data, frozen, and reused for policy training on a held-out suite, against native future-feature prediction. As the held-out suite, we use LIBERO-10, which comes from the same LIBERO-100 task pool as LIBERO-90 and consists of long-horizon, multi-step tasks, making it a demanding transfer target. We test two sources for the frozen encoder:

  • LIBERO-30 (unseen tasks). Stage 1 is trained only on LIBERO-Spatial, -Object, and -Goal (30 tasks). LIBERO-10 is excluded from all Stage-1 data, including the action normalization statistics, so its ten tasks, in kitchen, living-room, and study scenes, are unseen.
  • RoboTwin 2.0 (unseen tasks & embodiment). We reuse the encoders from our RoboTwin 2.0 Stage 1, an even harder transfer: RoboTwin uses a different simulator and a bimanual robot with two wrist cameras and 14-D absolute joint-position actions, whereas LIBERO uses a single arm with one wrist camera and 7-D delta end-effector actions.

Setting. For both sources, we freeze the encoders and train Stage 2 on LIBERO-10 demonstrations alone. The baseline uses the same policy architecture and LIBERO-10 demonstrations but predicts native future DINOv3 features (native co-training in Table 1), with identical Stage-2 budgets (20 epochs, batch size 256, same seed).

World targetLIBERO-10 SR (%)
Native DINOv3 future features79.8
PRISM, LIBERO-30 encoder89.2
PRISM, RoboTwin 2.0 encoder88.4

The LIBERO-30 encoder reaches 89.2% on LIBERO-10, 9.4 points above native future-feature prediction. Although the encoder never saw the target tasks or scenes, it provides a much more effective world-modeling target than native features from the target domain itself. The RoboTwin 2.0 encoder also outperforms native prediction. PRISM therefore learns predictive representations transferable to unseen tasks, scenes, and even embodiments without Stage-1 retraining. We have added this experiment to Appendix E.5.

2. Real-world validation. We conduct real-world experiments on a bimanual Franka Emika setup: the left arm moves a black box to the center, and the right arm then places a cup or a tape into the box. We collect 75 demonstrations and compare MiniWAM predicting native DINOv3 features and PRISM, with the same training protocol, over 20 episodes each:

MethodInference time (ms) ↓TapeCupOverall SR ↑
Native co-training2844/104/108/20 (40%)
MiniWAM (ours)1246/107/1013/20 (65%)

Inference time: raw observations (three cameras and robot state) to an action chunk, 10 denoising steps.

MiniWAM succeeds in 13 of 20 episodes against 8 of 20 for native co-training, with 2.3× faster inference. Figure S4 of our anonymous page shows paired rollouts, together with MiniWAM's robustness to unseen lighting disturbances and its recovery from a failed grasp. We have added this experiment to Section 4.6 and Appendix E.8.

We hope these responses address the reviewer's concerns and are happy to clarify any further questions.

Dear Reviewer,

Thank you for your efforts in reviewing this submission. As the discussion period is approaching its end, I would like to kindly remind you to read the authors' rebuttal and engage in the discussion. Your input is important for reaching a well-informed final assessment.

Thank you for your time and cooperation.

Reviewer_rRP9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an efficient world–action modeling method that first learns a compact future re

Critical strengths
  1. The paper proposes predicting compact future representations to replace dense future features, substantially accelerating model training. MiniWAM is lightweight, with only 0.25B parameters.

  2. Experimental results demonstrate that the method improves task success rates while reducing computational costs and improving training efficiency.

Critical weaknesses
  1. The paper lacks real-world robot deployment experiments and an evaluation of end-to-end deployment efficiency.
  2. The paper lacks case studies or visualizations of policy execution in simulation.
  3. The paper lacks comparisons of inference efficiency in simulation and does not report the inference latency, making it difficult to assess its suitability for real-time robotic control.
Questions for authors
  1. Since PRISM requires separate training for different downstream settings, how can MiniWAM’s generalizability and its ability to model the physical world be validated?
  2. ImageWAM[1] also explores ways to reduce the need to model dense future video features. Could the authors provide a comparison with ImageWAM using the same training demonstrations and evaluation protocol?

[1]: Zhang Y, Zhang W, Qi Z, et al. ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?[J]. arXiv preprint arXiv:2606.19531, 2026.

3 comments

We thank Reviewer rRP9 for the review, and for recognizing that compact future targets substantially accelerate training and that the 0.25B MiniWAM improves success while reducing computational cost. We address each point below.

W1 & Q1. Real-world experiments, clarification on downstream training, and generalization

We thank the reviewer for this question on real-world validation. We show the real-world experiments, clarify the scope of MiniWAM, and then illustrate its generalization.

1. Real-world experiments. We use a bimanual Franka Emika setup, with inference on an NVIDIA RTX PRO 6000 GPU. In the task, the left arm moves a black box to the center, and the right arm then places a cup or a tape into the box. We collect 75 demonstrations and compare MiniWAM predicting native DINOv3 features and PRISM, with the same training protocol, over 20 episodes each. Visualizations are in Figure S4 of our anonymous page, where we also show MiniWAM's robustness to lighting disturbances and its recovery from a failed grasp. We have added it to Section 4.6 and Appendix E.8. The results are:

MethodInference time (ms) ↓TapeCupOverall SR ↑
Native co-training2844/104/108/20 (40%)
MiniWAM (ours)1246/107/1013/20 (65%)

2. Clarification on downstream training. MiniWAM proposes a new compact representation for the world branch of a WAM to predict. Training separately for each downstream setting is the standard practice for VLAs and WAMs: VLAs such as π0.5 and WAMs such as Fast-WAM are trained or fine-tuned on the demonstrations of each benchmark. MiniWAM follows the same practice; its Stage 1 uses only those demonstrations, with no extra data, labels, or simulator states, and its cost is included in the total training speedups of Table 2 (up to 8.46× on RoboTwin 2.0).

3. MiniWAM's evaluated generalization. We have validated generalization by evaluating each trained model under conditions it has not seen:

  • Unseen perturbations (LIBERO-Plus). PRISM and the policy are trained only on standard LIBERO demonstrations, so all seven LIBERO-Plus perturbation types, from camera viewpoints to sensor noise, are unseen. PRISM improves over native co-training by 6.3 (Wan-VAE) and 13.2 (DINOv3) points on LIBERO-Plus-2K (Table 1), and MiniWAM reaches 73.6% on the full LIBERO-Plus (Table 3).
  • Unseen scenes (RoboTwin 2.0). The policy is trained on clean demonstrations only and also evaluated on randomized scenes with unseen clutter, distractors, and textures. Its 46.1% mean success exceeds Fast-WAM (39.9%) and AHA-WAM (33.8%) in the same clean-only track.
  • Representation analysis (LIBERO-90). On the unseen LIBERO-90 tasks, PRISM neighbors are closer in action than native ones (wrist-view action RMSE 0.370→0.272, Table 6; visualized in Figure S2 of our anonymous page).

Q2. Comparison with ImageWAM

We thank the reviewer for pointing to this relevant work. We first clarify the difference between the two methods and then compare the results reported in both papers. Both methods avoid generating future videos, but they change different parts of the WAM: ImageWAM replaces the video-generation backbone with a pretrained image-editing model (FLUX.2 4B) and conditions its action expert on the editing caches, whereas MiniWAM changes what the world branch predicts and uses no pretrained generative backbone. With such different designs, a direct comparison is less controlled. It would still be informative, but retraining ImageWAM on our hardware was not feasible within the rebuttal period: it fine-tunes the 4B FLUX.2 editing transformer together with a 642M action expert (about 4.6B trainable parameters), which we estimate would take more than 20 days on RoboTwin 2.0 with four RTX 3090 GPUs, compared with 2.8 days for MiniWAM.

However, ImageWAM reports results on all three benchmarks with the same training demonstrations as ours, although its training protocol is not strictly the same:

MethodTrainable paramsLIBEROLIBERO-PlusRoboTwin cleanRoboTwin randomized
ImageWAM (FLUX.2 4B)~4.6B98.483.184.418.3
MiniWAM (ours)0.25B97.273.673.418.8

Success rates (%). ImageWAM results are from its Tables 2, 3, and 6.

MiniWAM achieves comparable performance on LIBERO and randomized RoboTwin with about 18× fewer trainable parameters and substantially shorter training, while ImageWAM is stronger on LIBERO-Plus and clean RoboTwin, likely benefiting from a larger backbone.

We have added ImageWAM to Table 3.

W2. Visualizations of policy execution

We thank Reviewer rRP9 for this suggestion. We provide case studies of policy execution, the learned representation, and future predictions, with videos, on an anonymous webpage, and have added them to Appendix G of the revised manuscript, with the predictions and real-world rollouts in Appendices E.2 and E.8:

  • Policy rollouts (Figure S1). Closed-loop episodes of the 0.25B MiniWAM policy on LIBERO-Plus under a shifted camera viewpoint, altered lighting, and a new background texture, and on six RoboTwin 2.0 bimanual tasks, three from the clean split and three from the randomized split. In the randomized stacking task (Figure S1d), for example, one arm first moves the red block to the middle of the table and the other arm then stacks the green block on it, in a cluttered scene unlike the clean training demonstrations.
  • Nearest-neighbor retrieval (Figure S2 and Table S1). For query transitions from unseen LIBERO-90 tasks, we show the top-1 neighbor from a different task under native DINOv3 features and under PRISM, with their scenes. Native neighbors mostly share the query's scene, whereas PRISM neighbors more often come from other scenes in which the robot performs a similar motion. For a query in which the robot carries tomato sauce to a basket (example (c)), native DINOv3 retrieves a same-scene transition with the gripper still open, whereas PRISM retrieves one from another living room in which the robot likewise carries a grasped object.
  • Native Wan-VAE future predictions (Figure S3). Future videos predicted by the native Wan-VAE world branch, next to the ground truth. The predictions are visually faithful, yet this policy is clearly weaker than PRISM on LIBERO-Plus-2K (48.5% vs. 58.1%), so native co-training is not limited by failing to predict the future (Appendix E.2).
  • Real-world experiments (Figure S4). Paired episodes of MiniWAM and native co-training on the same scenes of the bimanual Franka task (W1), together with MiniWAM under lighting disturbances unseen during training and its recovery from a failed grasp (Section 4.6 and Appendix E.8).

W3. Inference efficiency and deployment feasibility

We thank the reviewer for this suggestion and agree that inference latency is important for real-time control. Although MiniWAM primarily targets training efficiency, we additionally measure its inference latency and GPU memory against representative WAMs and VLAs.

We benchmark on a single NVIDIA L40S GPU (BF16) with 200 LIBERO observations. Latency covers the full path from raw observations to an executable action chunk, including visual encoding, denoising, and action post-processing; instruction embeddings are cached per episode. B1 and B16 denote batch sizes 1 and 16.

MethodDenoising stepsLatency B1 (ms) ↓Max control freq. B1 (Hz) ↑Latency B16 (ms) ↓GPU memory (GiB) ↓
MiniWAM (ours)20831205470.88–1.25
MiniWAM (ours)10531893180.89–1.28
Fast-WAM-Joint10329303,67012.7–14.5
DreamWAM10351284,20512.7–15.0
ImageWAM10971031,10810.2–28.8
π010731375546.7–7.6
π0.510771306066.9–8.0

All PyTorch models use torch.compile with CUDA graphs; π0 and π0.5 use JAX/XLA. Control frequency assumes 10 executed actions per chunk, as on LIBERO. Memory ranges span B1 to B16. Our evaluations use 20 steps; the 10-step row matches the baselines.

At batch size 1, MiniWAM takes 83 ms per action chunk with 20 denoising steps, twice as many as the baselines, supporting control at up to about 120 Hz. Reducing to 10 steps leaves the LIBERO success rate unchanged and cuts latency to 53 ms, the lowest in the table. MiniWAM uses about 1 GiB of GPU memory, against 6.7–28.8 GiB for the baselines. At batch size 16, it is on par with the VLAs (547 ms vs. 554–606 ms) and 2–8× faster than the WAMs, so compact world targets also make parallel sampling affordable.

On the real robot (W1), MiniWAM takes 124 ms per action chunk from three cameras and robot state, against 284 ms for native co-training.

We have added this analysis to Appendix E.7.

We hope these responses address the reviewer's concerns and are happy to clarify any further questions.

Dear Reviewer,

Thank you for your efforts in reviewing this submission. As the discussion period is approaching its end, I would like to kindly remind you to read the authors' rebuttal and engage in the discussion. Your input is important for reaching a well-informed final assessment.

Thank you for your time and cooperation.

Reviewer_mP9fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MiniWAM, an efficient world-action model that predicts compact future representations instead of dense visual features. The authors introduce PRISM, which learns these representations through inverse dynamics and feature reconstruction. Experiments on LIBERO, LIBERO-Plus, and RoboTwin 2.0 show competitive policy performance with a 0.25B model and substantially lower training costs.

Critical strengths
  1. Clear motivation and practical value.

The paper presents an interesting direction for efficient world-action modeling. Replacing dense future prediction with compact representations is intuitive and reduces training costs.

  1. Promising experimental results.

MiniWAM achieves competitive performance against much larger models. The experiments across DINOv3 and Wan VAE features provide useful evidence for the effectiveness of PRISM.

Critical weaknesses
  1. Limited methodological novelty.

The method mainly combines existing ideas, including compact future representations, inverse dynamics, and feature reconstruction. Similar directions have been explored in WoG, FLARE, and other latent world-modeling methods. The novelty of PRISM over these approaches is not fully convincing.

  1. Limited evaluation of inference-time efficiency.

The paper provides extensive analysis of training efficiency and representation quality. However, it does not report end-to-end inference latency, GPU memory usage, or actual control frequency. Given the emphasis on lightweight world-action modeling, real-world deployment or inference benchmarks would further strengthen its practical significance.

Questions for authors

Q1. Methodological Novelty

The method appears to combine existing ideas, including compact future representations, inverse dynamics, and reconstruction. Although the paper discusses related methods, I am still unclear about the key methodological novelty over WoG, FLARE, and Being-H0.7.

Q2. Inference Efficiency

Since efficiency is a major motivation, could the authors report the actual inference latency, GPU memory usage, and control frequency of MiniWAM? Real-robot deployment would make the work more convincing.

Q3. Small Model Performance

The performance of the 0.25B model is impressive. I would like to better understand whether the gains mainly come from removing redundant visual information, learning better action-related features, or simply reducing the difficulty of future prediction.

Q4. Generalization

PRISM shows larger gains on LIBERO-Plus than on standard LIBERO. Could the authors provide a breakdown across different perturbation types? Does PRISM actually suppress task-irrelevant visual information?

Q5. Model Scaling

Would PRISM provide similar benefits with larger WAMs, such as 1B or 6B? Does the advantage decrease as model capacity increases?

8 comments

We thank Reviewer mP9f for the detailed review, and for noting both our analysis of training efficiency and representation quality and the strong performance of the 0.25B model. We address each point below.

W1 & Q1. Novelty over prior latent world-modeling methods

We agree that compact future representations, inverse dynamics, and reconstruction each have precedent. Our contribution lies in (i) how the compact representation is learned and (ii) how it is used: as the prediction target of an explicit world–action model (WAM), which lets us answer through controlled comparisons what the world branch of a WAM should predict.

Prior work follows two directions:

  • Future-informed VLAs (FLARE, WoG, Being-H0.7) use future information to shape the policy's own representations: FLARE aligns policy-internal tokens with embeddings of future observations, while WoG and Being-H0.7 use compact future latents as internal states or conditioning signals for action generation and rely on large backbones (7B and 3B, respectively), making training costly.
  • WAMs (DreamZero, Cosmos Policy, Fast-WAM) explicitly predict the future jointly with actions, but in native visual feature spaces such as video-VAE latents.

Unlike future-informed VLAs, MiniWAM keeps the WAM formulation, an explicit generative model of future dynamics, and changes what it predicts:

  1. A compact target learned from the transition. Stage 1 learns the PRISM encoder from privileged current–future transitions with inverse dynamics and reconstruction. The inverse-dynamics model sees only the compact representation, without the current observation, instruction, or proprioception, so PRISM must encode the whole transition rather than complement the current observation, while reconstruction keeps information about the future state. Together, the two objectives outperform either alone at every bottleneck size (Table 4). Stage 2 then freezes the encoder and uses its posterior mean as the world-prediction target.
  2. Independent of the feature family. The same recipe applies to generative (Wan-VAE) and discriminative (DINOv3) features.
  3. Most importantly, a controlled comparison. These designs let PRISM replace the native world target of a conventional WAM with everything else unchanged, isolating the effect of what the world branch predicts from other design factors.

With the same 0.25B model and Stage-2 protocol, changing only the world target from native features to PRISM gives (Tables 1–2):

FeaturesWorld targetLIBEROLIBERO-Plus-2KStage-2 speedup
Wan-VAENative92.850.21.0×
Wan-VAEPRISM94.856.54.6–10.5×
DINOv3Native93.260.91.0×
DINOv3PRISM97.274.14.0–9.5×

Removing co-training altogether lowers LIBERO success to 62.3% (Wan-VAE) and 89.0% (DINOv3), so explicit future prediction remains essential. PRISM makes it both cheaper and more effective. Different from most WAMs and future-informed VLAs, MiniWAM allows a tiny model to achieve competitive performance and makes WAMs more accessible to the community.

We have clarified these distinctions in the related work (Section 2) of the revised manuscript.

W2 & Q2. Inference efficiency and real-robot deployment

1. Inference efficiency. We thank Reviewer mP9f for this suggestion. Although MiniWAM primarily targets training efficiency, we measure its inference latency and GPU memory against representative WAMs and VLAs. At equal denoising steps, MiniWAM is the fastest method and, at batch size 1, uses ~8–14× less GPU memory.

We benchmark on an NVIDIA L40S GPU (BF16) with 200 LIBERO observations. Latency covers the end-to-end path from raw observations to an executable action chunk. B1 and B16 denote batch sizes 1 and 16.

MethodDenoising stepsLatency B1 (ms) ↓Max control freq. B1 (Hz) ↑Latency B16 (ms) ↓GPU memory (GiB) ↓
MiniWAM (ours)20831205470.88–1.25
MiniWAM (ours)10531893180.89–1.28
Fast-WAM-Joint10329303,67012.7–14.5
DreamWAM10351284,20512.7–15.0
ImageWAM10971031,10810.2–28.8
π010731375546.7–7.6
π0.510771306066.9–8.0

All PyTorch models use torch.compile with CUDA graphs; π0 and π0.5 use JAX/XLA. Control frequency assumes 10 executed actions per chunk, as on LIBERO. Memory ranges span B1 to B16.

At batch size 1, MiniWAM takes 83 ms per action chunk with 20 denoising steps, twice as many as the baselines, supporting control at up to about 120 Hz. Reducing to 10 steps leaves the LIBERO success rate unchanged and cuts latency to 53 ms. At batch size 16, it is on par with the VLAs (547 ms vs. 554–606 ms) and 2–8× faster than the WAMs, so compact targets also make parallel sampling affordable. On the real robot (below), MiniWAM takes 124 ms per action chunk from three cameras (284 ms for native co-training).

We have added this analysis to Appendix E.7.

2. Real-robot deployment. We also deployed MiniWAM on a bimanual Franka Emika setup (Section 4.6 and Appendix E.8; visualized in Figure S4). Trained on 75 demonstrations, it succeeds in 13/20 episodes vs. 8/20 for native DINOv3 co-training.

Q3. Where do the gains of the 0.25B model come from?

We attribute them to two complementary factors: the world branch predicts a compact yet useful future, without attending to fine-grained visual detail, and the PRISM encoder is trained to keep action-relevant information. We take the reviewer's three hypotheses in turn.

1. Reducing the difficulty of future prediction alone does not explain the gains.

(i) Accurate futures are not what native co-training lacks. We visualize the native Wan-VAE world branch's predicted future videos next to the ground truth in Figure S3 of our anonymous page. The predictions are visually faithful, yet this policy is clearly weaker than PRISM on LIBERO-Plus-2K (48.5% vs. 58.1%, averaged over two seeds). Native co-training is therefore not limited by failing to predict the future.

(ii) Pooling to the same size underperforms PRISM. On RoboTwin 2.0 at a 16-frame horizon, we train the same model with three world targets: native DINOv3 features, DINOv3 features pooled in space and time to the size of PRISM, and PRISM. Pooling makes prediction as cheap as PRISM but barely improves on native features (+0.6 points), whereas PRISM gains 7.7 points.

World targetRoboTwin 2.0 Mean SR (%)
Native DINOv327.7
Pooled DINOv3 (PRISM-sized)28.3
PRISM35.4

(iii) PRISM beats compression in different bottlenecks. Table 4 compares PRISM with reconstruction-only bottlenecks of identical size. PRISM is better at every capacity, so its targets benefit action learning beyond compression:

BottleneckReconstruction onlyPRISM
4×469.574.1
2×263.367.8
1×157.061.7

LIBERO-Plus-2K success rates (%), from Table 4.

2. PRISM learns action-relevant features. PRISM's inverse-dynamics term adds 4.6 points on average over reconstruction alone (Table 4). Because PRISM is trained to extract action-relevant information from general visual features, its neighborhoods are closer in action across tasks (Table 6; wrist-view action RMSE 0.370→0.272), as visualized in Figure S2 and Table S1 of our anonymous page.

3. PRISM learns to remove task-irrelevant visual information. Robustness gains under visual perturbations are large, e.g., background textures (+26.3) and sensor noise (+15.1), indicating that PRISM learns action-centric features rather than redundant visual information, detailed in Q4.

Overall, PRISM is designed to combine both factors, letting a 0.25B model spend its limited capacity on what matters for control. We have added this analysis to Appendices E.1–E.3.

Q4. Robustness breakdown on LIBERO-Plus

We thank Reviewer mP9f for this question. A breakdown by perturbation type shows where MiniWAM's gains come from. We evaluate PRISM and native DINOv3 co-training on the stratified LIBERO-Plus-2K subset with three seeds (41, 42, 43), and have added this to Appendix E.3:

Perturbation (episodes)PRISM (%)Native (%)Paired gain (pp)
Background textures (217)82.3 ± 3.256.1 ± 3.7+26.3 ± 5.5
Robot initial states (309)59.2 ± 2.937.6 ± 2.8+21.5 ± 0.2
Sensor noise (318)84.6 ± 5.569.5 ± 3.0+15.1 ± 7.9
Language instructions (306)69.0 ± 2.055.3 ± 4.9+13.6 ± 3.0
Light conditions (228)92.7 ± 4.785.2 ± 5.7+7.5 ± 7.4
Object layouts (304)76.8 ± 2.871.2 ± 1.5+5.6 ± 3.8
Camera viewpoints (318)48.4 ± 2.549.6 ± 4.8−1.1 ± 6.6
Overall (2,000)72.0 ± 2.759.8 ± 1.4+12.2 ± 4.1

Mean ± standard deviation over three seeds; gains use paired per-seed differences.

Both closed-loop results and the learned representations suggest that PRISM does suppress task-irrelevant visual information:

  • All three appearance-level perturbations, which change the image but not the task, improve: background textures (+26.3, the largest gain overall), sensor noise (+15.1), and lighting (+7.5). This is consistent with the Stage-1 bottleneck, which forces the PRISM encoder to keep action-relevant information rather than appearance details.
  • In the leave-task-out nearest-neighbor retrieval examples (Table S1 of our anonymous page), 7 of 8 native neighbors come from the query's own scene, against 3 of 8 PRISM neighbors, which instead come from other scenes with similar motion. This indicates that PRISM learns to discard action-irrelevant scene details, such as the background.

Q5. Scaling to larger WAMs

We thank the reviewer for this question. A fully controlled 0.5B PRISM-versus-native comparison would be valuable, but is computationally prohibitive within the rebuttal period: on our available hardware, training the 0.5B native-prediction baseline alone is estimated to require more than 10 days. Nevertheless, we have scaled MiniWAM from 0.25B to 0.5B:

Visual featuresBenchmark0.25B0.5B
Wan-VAELIBERO94.895.6
Wan-VAELIBERO-Plus-2K56.560.7
DINOv3LIBERO97.297.7
DINOv3LIBERO-Plus-2K74.175.8

These results suggest that MiniWAM continues to benefit from increased capacity while keeping its compact target. We also expect the efficiency advantage to hold at larger scales, since the world expert predicts 48 tokens per camera instead of 784 regardless of model size. Nor do we expect the accuracy advantage to stem from limited capacity at 0.25B: as shown in Q3 (i), the native world branch at this scale already predicts visually faithful futures, yet its policy remains weaker than PRISM. Native co-training is thus limited by what its target encodes, not by the expressiveness of the world branch that larger models would add. Confirming this at the 1B–6B scale requires matched comparisons, which we consider an important direction for future work. We have added these results to Appendix E.4.

We hope these responses address the reviewer's concerns and are happy to clarify any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to the AC and all reviewers

We thank all reviewers for their constructive feedback, and the AC for encouraging active discussion. Reviewers recognized that MiniWAM substantially improves training efficiency while remaining competitive (j3sK, rRP9), that it shows predicting dense native visual features is unnecessary for effective world–action co-training (j3sK), the strong performance of the 0.25B model (mP9f, rRP9), and the extensive experiments and representation analyses (j3sK, mP9f). Below we summarize the main concerns and how we addressed them. All new results are in the revised manuscript, with revised main-text passages in blue.

1. Real-world validation (rRP9 W1; j3sK W1; mP9f W2 & Q2). On a bimanual Franka Emika task with 75 demonstrations, MiniWAM succeeds in 13/20 episodes vs. 8/20 for native DINOv3 co-training, with 2.3× faster inference (124 vs. 284 ms) (Section 4.6, Appendix E.8; videos in Figure S4 of our anonymous page).

2. Inference efficiency (mP9f W2 & Q2; rRP9 W3). End to end, MiniWAM is the fastest method at equal denoising steps (53 ms per action chunk, up to 189 Hz) and uses about 1 GiB of GPU memory, against 6.7–28.8 GiB for representative WAMs and VLAs (Appendix E.7).

3. PRISM is more than compression (j3sK W3 & Q1; mP9f Q3 & Q4). At the same target size, pooled features barely improve on native prediction (28.3% vs. 27.7% on RoboTwin 2.0), whereas PRISM reaches 35.4% (Appendix E.1), and PRISM beats reconstruction-only bottlenecks at every size (Table 4). The native world branch already predicts visually faithful futures (Figure S3), yet its policy is weaker (Appendix E.2), so the gain comes from what PRISM encodes. A per-perturbation breakdown shows the largest gains under appearance changes, e.g., background textures (+26.3 points), indicating that PRISM suppresses task-irrelevant visual information (Appendix E.3).

4. Generalization and transfer (j3sK W1 & Q2; rRP9 Q1). Training separately for each benchmark is standard practice for VLAs and WAMs, and PRISM uses no extra data or labels. Beyond this, a frozen PRISM encoder trained on LIBERO-30 reaches 89.2% on the unseen LIBERO-10 tasks vs. 79.8% for native prediction, and one trained on RoboTwin 2.0, a different simulator and bimanual robot with a different action space, reaches 88.4% (Appendix E.5).

5. Learning from action-free video (j3sK W2). With actions for only 100 of 388 LIBERO-10 demonstrations, adding the other 288 as action-free video raises success by 9.4 points, so the inverse-dynamics objective does not prevent MiniWAM from learning from video (Appendix E.6).

6. Model scaling (mP9f Q5). With matched native baselines at 0.1B, 0.25B, and 0.5B, PRISM leads at every size, by 13.2, 13.2, and 11.6 points on LIBERO-Plus-2K, and the 0.1B MiniWAM outperforms the 0.5B native model (68.1% vs. 64.2%) (Appendix E.4).

7. Novelty and related methods (mP9f W1 & Q1; rRP9 Q2). Unlike future-informed VLAs (FLARE, WoG, Being-H0.7), which use future information to shape the policy's own representations, MiniWAM keeps an explicit world-prediction branch that learns future dynamics and changes only what it predicts, enabling controlled comparisons of world targets (Section 2). It also avoids the large backbones of WoG and Being-H0.7, which make training costly. We added ImageWAM to Table 3: MiniWAM is comparable on LIBERO (97.2% vs. 98.4%) and on randomized RoboTwin 2.0 (18.8% vs. 18.3%; Appendices C and G) with about 18× fewer trainable parameters.

8. Visualizations and presentation (rRP9 W2; mP9f follow-up). We added policy rollouts (Figure S1), nearest-neighbor retrievals (Figure S2), future predictions (Figure S3), and real-robot videos (Figure S4) to our anonymous page and Appendices E.2, E.8, and G, revised Figures 1 and 2 for clarity, and disclosed the AI-generated icons in the AI use statement.

Summary of revisions. Section 2 (related work), Section 4.6 and Table 7 (real robot), Table 3 (ImageWAM), Figures 1 and 2, Appendices E.1–E.8 (new experiments), and Appendix G (visualizations).

We thank the reviewers and the AC again for their time and constructive feedback, and we are happy to address any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.