- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MiniWAM, an efficient World Action Model that replaces high-dimensional future visual prediction with compact, control-relevant predictive representations. The key component is PRISM (Predictive Representations via Inverse Spatiotemporal Modeling), which learns a compact representation of current-to-future visual transitions through two complementary objectives: inverse-dynamics prediction and future-feature reconstruction. The proposed framework consists of two training stages. First, PRISM learns a compressed representation from current and future visual features using inverse-dynamics supervision and reconstruction. Second, the PRISM encoder is frozen, and MiniWAM jointly predicts the learned representation and robot actions from current observations using a Mixture-of-Transformers architecture with flow matching. The method is evaluated on LIBERO, LIBERO-Plus, and RoboTwin 2.0 using both DINOv3 and WAN2.1 VAE visual features. Compared with directly predicting native future features, MiniWAM achieves higher manipulation success rates while substantially reducing world-modeling computation. With DINOv3 features, it improves LIBERO success from 93.2% to 97.2% and LIBERO-Plus-2K success from 60.9% to 74.1%.
Critical strengths
- Substantial improvements in training efficiency while maintaining competitive manipulation performance. A major strength of this paper is its demonstration that predicting high-dimensional native visual features is not necessary for effective world-action co-training. The results make the proposed approach practically relevant for developing computationally efficient robot learning systems.
- Well-structured empirical evaluation with meaningful ablation and representation analyses. The paper provides relatively comprehensive experimental validation across three simulation benchmarks, including robustness evaluations under distribution shifts. Also, the representation analyses, including cross-backbone CKA and behavior-based nearest-neighbor retrieval on unseen tasks, provide additional evidence that PRISM captures behaviorally relevant transition information.
Critical weaknesses
- Limited generalization and lack of real-world validation. Although MiniWAM demonstrates strong results on LIBERO, LIBERO-Plus, and RoboTwin 2.0, all experiments are conducted in simulation, with no real-world robotic validation. Furthermore, the experiments primarily evaluate performance on trained task distributions and robustness under controlled perturbations. The paper does not evaluate zero-shot or few-shot generalization to unseen manipulation tasks or transfer across environments and robot embodiments. While the authors analyze PRISM representations on unseen LIBERO-90 tasks, these experiments examine representation similarity rather than downstream policy performance. Additionally, PRISM is trained separately for each downstream setting, leaving its transferability unclear.
- Limited scalability of PRISM due to its reliance on action-labeled data. The proposed PRISM framework relies on inverse-dynamics supervision, which requires action-labeled current–future transitions during representation learning. While this is feasible for robot demonstration datasets, it limits the direct applicability of PRISM to large-scale unlabeled video datasets, including egocentric human demonstrations and internet videos. This is particularly relevant because one of the major advantages of world modeling is the ability to leverage diverse visual observations without requiring corresponding robot actions. Furthermore, the inverse-dynamics objective is defined over the action space of the training embodiment, raising questions about how the learned representations can transfer across robots with different action spaces. Although MiniWAM can benefit from pretrained visual backbones, the PRISM representation itself still requires action supervision. This potentially limits the scalability and general applicability of the proposed representation-learning framework.
- Insufficient evidence that PRISM provides benefits beyond simple future-feature compression. While the paper demonstrates that PRISM outperforms native future-feature prediction, it remains unclear whether these improvements primarily result from learning control-relevant representations through inverse-dynamics supervision or simply from reducing the dimensionality of the prediction targets. Although the authors provide ablations comparing inverse-dynamics supervision, feature reconstruction, and their combination, they do not compare PRISM against simpler compression strategies, such as spatially and temporally pooled future features, PCA-based compression, or lightweight feature compression with matched token budgets. Such comparisons are particularly important because the main contribution of PRISM is not merely computational efficiency, but the claim that its learned representations provide more effective world-modeling targets. Without these baselines, it is difficult to establish whether the proposed two-stage representation-learning procedure is necessary or whether similar performance and efficiency improvements could be achieved through substantially simpler alternatives.
Questions for authors
Q1. Is PRISM necessary, or would simpler future-feature compression achieve similar results? The paper demonstrates that PRISM outperforms native future-feature prediction and provides ablations on inverse-dynamics supervision and feature reconstruction. However, it remains unclear whether the improvements primarily come from the proposed representation-learning objective or simply from reducing the dimensionality of future prediction targets. Could the authors compare PRISM against simpler compression strategies, such as spatial/temporal pooling or PCA-based compression, under matched token budgets and training conditions? This comparison would help establish whether the proposed two-stage PRISM training is necessary to achieve the reported performance and efficiency improvements.
Q2. How well does PRISM generalize to unseen tasks and environments? The paper evaluates PRISM representations on unseen LIBERO-90 tasks, but these experiments focus on representation similarity rather than downstream policy performance. Furthermore, PRISM is trained separately for each downstream setting. Could the authors evaluate whether a PRISM encoder trained on one task distribution can be reused for unseen tasks without retraining? For example, could a PRISM encoder pretrained on LIBERO-40 be frozen and used for policy training on LIBERO-90, with comparisons against native future-feature prediction? Such experiments would help clarify whether PRISM learns transferable predictive representations or primarily task-specific representations.
5 comments
W1 & Q2. Generalization to unseen tasks, transfer of PRISM, and real-world validation
We thank Reviewer j3sK for this valuable question, which tests whether PRISM learns transferable representations. Our LIBERO-90 analysis characterizes what PRISM encodes; we agree that it is not evidence of downstream policy generalization. Although training separately for each downstream setting is the standard practice for VLAs and WAMs, and MiniWAM adds no additional labels, we show that a frozen PRISM encoder can be reused for unseen tasks as the reviewer requested, and then report real-world experiments.
1. A PRISM encoder trained on one task distribution can be frozen and reused for unseen tasks. Training and evaluating policies on all 90 LIBERO-90 tasks did not fit within the rebuttal period with our hardware budget, so we run two smaller versions of the suggested experiment with the same structure: a PRISM encoder is trained on other data, frozen, and reused for policy training on a held-out suite, against native future-feature prediction. As the held-out suite, we use LIBERO-10, which comes from the same LIBERO-100 task pool as LIBERO-90 and consists of long-horizon, multi-step tasks, making it a demanding transfer target. We test two sources for the frozen encoder:
- LIBERO-30 (unseen tasks). Stage 1 is trained only on LIBERO-Spatial, -Object, and -Goal (30 tasks). LIBERO-10 is excluded from all Stage-1 data, including the action normalization statistics, so its ten tasks, in kitchen, living-room, and study scenes, are unseen.
- RoboTwin 2.0 (unseen tasks & embodiment). We reuse the encoders from our RoboTwin 2.0 Stage 1, an even harder transfer: RoboTwin uses a different simulator and a bimanual robot with two wrist cameras and 14-D absolute joint-position actions, whereas LIBERO uses a single arm with one wrist camera and 7-D delta end-effector actions.
Setting. For both sources, we freeze the encoders and train Stage 2 on LIBERO-10 demonstrations alone. The baseline uses the same policy architecture and LIBERO-10 demonstrations but predicts native future DINOv3 features (native co-training in Table 1), with identical Stage-2 budgets (20 epochs, batch size 256, same seed).
| World target | LIBERO-10 SR (%) |
|---|---|
| Native DINOv3 future features | 79.8 |
| PRISM, LIBERO-30 encoder | 89.2 |
| PRISM, RoboTwin 2.0 encoder | 88.4 |
The LIBERO-30 encoder reaches 89.2% on LIBERO-10, 9.4 points above native future-feature prediction. Although the encoder never saw the target tasks or scenes, it provides a much more effective world-modeling target than native features from the target domain itself. The RoboTwin 2.0 encoder also outperforms native prediction. PRISM therefore learns predictive representations transferable to unseen tasks, scenes, and even embodiments without Stage-1 retraining. We have added this experiment to Appendix E.5.
2. Real-world validation. We conduct real-world experiments on a bimanual Franka Emika setup: the left arm moves a black box to the center, and the right arm then places a cup or a tape into the box. We collect 75 demonstrations and compare MiniWAM predicting native DINOv3 features and PRISM, with the same training protocol, over 20 episodes each:
| Method | Inference time (ms) ↓ | Tape | Cup | Overall SR ↑ |
|---|---|---|---|---|
| Native co-training | 284 | 4/10 | 4/10 | 8/20 (40%) |
| MiniWAM (ours) | 124 | 6/10 | 7/10 | 13/20 (65%) |
Inference time: raw observations (three cameras and robot state) to an action chunk, 10 denoising steps.
MiniWAM succeeds in 13 of 20 episodes against 8 of 20 for native co-training, with 2.3× faster inference. Figure S4 of our anonymous page shows paired rollouts, together with MiniWAM's robustness to unseen lighting disturbances and its recovery from a failed grasp. We have added this experiment to Section 4.6 and Appendix E.8.
We hope these responses address the reviewer's concerns and are happy to clarify any further questions.
Dear Reviewer,
Thank you for your efforts in reviewing this submission. As the discussion period is approaching its end, I would like to kindly remind you to read the authors' rebuttal and engage in the discussion. Your input is important for reaching a well-informed final assessment.
Thank you for your time and cooperation.
We thank Reviewer j3sK for the detailed and constructive review. We address each point below.
W3 & Q1. Is PRISM necessary, or would simpler compression suffice?
We thank Reviewer j3sK for this question and agree that simpler compression baselines are important for isolating what PRISM contributes. We show that PRISM is necessary, while simpler future-feature compression doesn't achieve similar results from two experiments that separate the size of the target from what it encodes.
1. PRISM beats pooling to the same size. On RoboTwin 2.0 at a 16-frame horizon, we train the same model with three world targets: native DINOv3 features, DINOv3 features pooled in space and time to the size of PRISM, and PRISM. Pooling barely improves on native features (+0.6 points), whereas PRISM gains 7.7 points.
2. PRISM beats reconstruction-only compression at every bottleneck size. Table 4 compares PRISM with reconstruction-only bottlenecks of identical size:
LIBERO-Plus-2K success rates (%), from Table 4.
The gain therefore does not come from a smaller target alone: pooled and learned compression of the same size both underperform PRISM. What PRISM encodes brings the accuracy gain, while its compact size brings the efficiency. We have added the pooled baseline to Appendix E.1.
W2. Limited scalability
We thank the reviewer for this question. We first clarify the goal of the paper, then show that MiniWAM can (a) use action-free video to improve performance and (b) transfer PRISM encoders across robots with different action spaces.
1. Goal of the paper. We agree that inverse dynamics requires action labels in Stage 1, and that learning from action-free video is an important advantage of world modeling. Our goal, however, is narrower: a compact target for the world branch of a WAM that makes world–action co-training more efficient and more effective. Like standard WAMs and VLAs, MiniWAM trains on action-labeled robot demonstrations, and Stage 1 uses only those same demonstrations, so PRISM adds no labeling requirement.
2. MiniWAM benefits from action-free video and transfers across robots with different action spaces. Although pre-training on large-scale video or across embodiments is not the focus of this paper, MiniWAM's design lets it scale to video data.
(a) Action-free video. Both of MiniWAM's stages can learn from it:
To test this, we train on LIBERO-10 with actions for only 10 demonstrations per task (100 of 388) and use the other 288 as action-free video. These videos train the encoder through reconstruction only and the world expert through the world loss only, with action tokens left as pure noise. A third run uses LIBERO-Plus's perturbed replays of the same 288 demos instead. All runs use the same labeled windows per update and the same number of updates; the video windows are added on top.
Action-free video raises success by 9.4 points on LIBERO-10, so the inverse-dynamics term does not prevent MiniWAM from learning from video. Perturbed video instead raises LIBERO-Plus success by 11.8 points, suggesting that MiniWAM can absorb the visual diversity that videos would provide, while learning action-relevant information from demonstrations. We have added these experiments to Appendix E.6.
(b) Transfer across robots with different action spaces. Although the inverse-dynamics objective is defined over the training robot's action space, a frozen PRISM encoder trained on RoboTwin 2.0 (bimanual, 14-D absolute joints) and reused on LIBERO-10 (single arm, 7-D delta end-effector) outperforms native prediction (88.4% vs. 79.8%; W1 & Q2). Since the Stage-1 inverse-dynamics model sees only PRISM, without the current observation, instruction, or proprioception, the encoder must capture the motion itself, yielding features that stay useful across embodiments. Unlike WAMs that tile all views into one image (e.g., Fast-WAM), PRISM uses one encoder per camera role (e.g., main or wrist view), so the same encoders can be reused across embodiments.