- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes OnlineWM, an online post-training framework for action-conditioned video world models, instantiated on HY-World 1.5 (8B, initialized from the WorldCompass checkpoint) with 32 parallel Minecraft simulators. It has two components: Active Online Learning (AOL) and Causality-Aware Fine-Tuning (CFT).
Critical strengths
The problem is relevant. Poor action following for long-tail action–scene combinations is a real failure mode of current interactive world models.
The engineering is substantial and well documented: closed-loop training of an 8B model with 32 parallel simulators.
The paper’s best idea is the coupling between its two components.
Critical weaknesses
The main tables are ablations, not comparisons. No alternative training strategy is evaluated, no other action condition world model compared.
All results use a self-curated test set. Make it hare to the performance relative to existing Minecraft world models.
The paper characterizes selected samples as “novel yet learnable,” but the score is the current loss only; nothing measures learnability or progress.
Questions for authors
Why did the authors choose not to evaluate with the public protocols already established for Minecraft world models, such as the IDM-based action-following metrics of MineWorld or GameWorld Score from Matrix-Game?
How well does Depth Anything 3 recover camera poses on this Minecraft data?
How does OnlineWM relate to RLIR (https://arxiv.org/abs/2509.23958), which also post-trains video world models for action-following?
8 comments
Response to Reviewer K7yr (Part 2/2)
W3: Basis for describing selected samples as novel and learnable
W3. The paper characterizes selected samples as “novel yet learnable,” but the score is the current loss only; nothing measures learnability or progress.
A3. We thank the reviewer for the comment. The training loss, evaluated at a fixed timestep of the flow-matching objective, can serve as an efficient and stable proxy for identifying samples that are challenging but not impossible to learn [5, 6]: it requires only one forward pass, and its stability is supported by Table 8. In OnlineWM, we further decompose this loss into action- and scene-level difficulty to guide the learning curriculum of the generative world model, an approach whose effectiveness is also demonstrated empirically in Table 7.
We will include this discussion and the additional references in the final version.
We hope these clarifications and additional experiments address the reviewer's concerns, and we would appreciate the reviewer's consideration of these points in the final evaluation.
References
[1] Zhang, Yifan, et al. "Matrix-game: Interactive world foundation model." arXiv, 2025.
[2] Wang, Zile, et al. "Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory." arXiv, 2026.
[3] Ye, Yang, et al. "Reinforcement learning with inverse rewards for world model post-training." arXiv, 2025.
[4] Wang, Zehan, et al. "Worldcompass: Reinforcement learning for long-horizon world models." ICML, 2026.
[5] Mindermann, Sören, et al. "Prioritized training on points that are learnable, worth learning, and not yet learnt." ICML, 2022.
[6] Kumar, M., Benjamin Packer, and Daphne Koller. "Self-paced learning for latent variable models." NeurIPS, 2010.
Dear Reviewer K7yr,
Thank you once again for your valuable feedback on our submission. As the discussion period is approaching its end, we would be grateful to know whether our rebuttal has adequately addressed your concerns. Should any questions remain or any points require further clarification, please do not hesitate to let us know, and we will respond promptly.
If you find that our responses have resolved the main concerns, we would greatly appreciate it if you could take the rebuttal and additional evidence into account when updating your assessment.
We sincerely look forward to your feedback.
Best regards,
Authors of Paper 7472
Response to Reviewer K7yr (Part 1/2)
Thank you for recognizing the relevance of the problem, the documented engineering effort, and the coupling between AOL and CFT. Your concerns and questions are organized below.
W1 & W2 & Q3: External comparisons and the relationship to RLIR
A1. We thank the reviewer for the comments and for pointing out relevant work.
The main goal of this paper is to improve action controllability through Active Online Learning (AOL) and Causality-Aware Fine-Tuning (CFT). In Tables 2 and 7, we compare HY-World 1.5 8B, initialized from the WorldCompass [4] checkpoint, under Random online SFT and other variants under fair comparison settings. Relative to Random online SFT, AOL reduces RPErot and RPEtrans by 21.27% and 16.58%, respectively; adding CFT to AOL yields further reductions of 19.09% and 4.22%. These results confirm the effectiveness of our OnlineWM approach.
Following your suggestion, we add comparisons with two action-conditioned world model baselines, Matrix-Game [1] and Matrix-Game 3.0 [2]:
These results show that OnlineWM outperforms all selected baselines.
Relation to other post-training methods for action following. RLIR [3] and WorldCompass [4] optimize world models through reward-based reinforcement learning on model-generated rollouts. Their performance depends heavily on rollout quality and reward-model reliability. Generating these rollouts and obtaining reliable rewards introduce substantial design complexity and computational cost. OnlineWM directly aligns the world model with ground-truth simulator outputs through active selection and causality-aware optimization, providing a more efficient, accurate, and controllable training approach. We will expand the related-work discussion to cite [3, 4] and clarify the relationship to these post-training methods.
Q1 & Q2: Public Minecraft evaluation protocols and comparability
A2. We thank the reviewer for the comment, and we would like to make the following clarifications.
Evaluation protocols. IDM-based metrics typically rely on discrete bins to assess action correctness rather than directly estimating camera poses. Our evaluation compares action following under both discrete action and continuous camera-pose conditioning to assess the effectiveness of OnlineWM training. Estimating camera poses directly from video is therefore more reliable and straightforward than recovering them from discrete inverse-dynamics predictions. For a more comprehensive evaluation, we also use VBench and established video-quality metrics (SSIM and LPIPS) to compare our method with the baselines.
Depth Anything 3 on Minecraft data. Depth Anything 3 is trained on a large-scale dataset and can accurately recover actions directly in Minecraft (>95% accuracy for both view and movement actions). For camera-pose error evaluation, we apply closed-form Sim(3) Umeyama alignment to the camera centers to account for the estimator’s per-scene scale and any global coordinate-frame offset.