- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors argue that pretrained world models already encode scene dynamics and the space of possible futures. Therefore, the robot need not relearn these dynamics, but rather select the future that accomplishes the task and decode the corresponding actions. To this end, they propose RoboActualizer, a lightweight module with only 60M parameters built on top of a frozen world model encoder. Remarkably, the model can be trained on a single GPU (32 GB peak memory) while achieving decent performance.
Critical strengths
- The paper is well-organized and the writing is clear and logical. For example, the formalization of Eq. 3 cleanly decomposes the problem into three interpretable factors. This decomposition provides a clear conceptual framework and justifies why a tiny downstream model suffices.
- RoboActualizer achieves competitive results across multiple benchmarks while using few trainable parameters.
- RoboActualizer can be trained on a single GPU with 32 GB peak memory and achieves an economic inference latency, significantly faster than all baselines. This makes the approach genuinely practical for real-time robot control.
- The method is also evaluated on two real robot platforms with multiple manipulation tasks.
Critical weaknesses
The approach's performance is inherently bounded by the quality and coverage of the frozen world model's prior. While Table 7 shows that larger world models yield better results, the paper does not discuss the saturation point or what happens when the world model's pretraining distribution significantly mismatches the deployment domain. A discussion of this limitation and potential remedies (e.g., light fine-tuning of the encoder) maybe valuable.
Questions for authors
please see weaknesses.
3 comments
Response to authors rebuttal
OK,I have been positive about this work from the start. I also take a quick look at the other reviews. I'm sticking with my rating.
Thank you very much for your support, and for spending time reviewing our paper and reading other reviews. Please feel free to let us know if you have any further concerns.
Response to Reviewer hfhE
We appreciate your efforts in reviewing our paper. We try to address your concerns as follows:
W1: The saturation point of world model scaling and what happens when the pretraining distribution mismatches.
Our results support the first half of the reviewer’s premise: a stronger world model encoder yields a stronger policy. We trained the same actualizer on frozen V-JEPA 2.1 encoders of three sizes. Moving from 80M to 300M improves LIBERO-Plus from 50.7% to 60.0% (Table 7), far larger than widening the actualizer to DiT-B. Moving from 300M to 2B, however, gives no noticeable further gain. We believe this reflects a ceiling of the encoders rather than our method: the 300M V-JEPA 2.1 model is distilled from the 2B model, and the two are close on the original authors’ own temporal benchmarks (DAVIS label propagation: 67.0 / 68.7 / 69.0 for 80M / 300M / 2B). In other words, the 2B encoder is not substantially stronger than the 300M one by V-JEPA’s own measure, so there is little additional signal for the actualizer to read out. Our results track this curve closely, which is consistent with our claim rather than contradicting it. Locating the true saturation point would require encoders that are stronger than the current model, and pretraining such a model is beyond our compute budget. We list it as future work. As stronger open video encoders appear, our method can adopt them.
For the distribution mismatches, V-JEPA 2.1 is pretrained on real-world internet video, while all our evaluations are on simulated robot scenes (LIBERO, LIBERO-Plus, RoboTwin) and two real robots, so the reported results already include a non-trivial pretraining–deployment gap. LIBERO-Plus in particular applies visual perturbations unseen in both pretraining and fine-tuning, and the frozen encoder transfers well there. To test whether closing this gap helps, we fine-tuned the 300M encoder on LIBERO with LoRA using the V-JEPA 2.1 objective and retrained the actualizer on top. LIBERO is unchanged (98.0%) and LIBERO-Plus drops to 60.9%. We read this as slight overfitting of the encoder to the LIBERO distribution, which argues for keeping the general-purpose frozen encoder: it generalizes better under perturbation and costs nothing on robot data.
Thanks again for your time and effort! For any other questions, please feel free to let us know during the rebuttal window.