- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces WEAVER, a multi-view world model for robotic manipulation that combines flow matching, diffusion forcing, memory, and latent reward prediction. The model achieves improved generation quality and inference efficiency compared with existing world models. Real-robot experiments demonstrate its effectiveness in policy evaluation, synthetic-data-based policy improvement, and test-time planning.
Critical strengths
WEAVER achieves substantially better generation quality and faster inference than existing world models, making it more practical for robotic manipulation and planning. The paper goes beyond video generation metrics and demonstrates improvements in policy evaluation, policy fine-tuning, and test-time planning on real hardware.
Critical weaknesses
Although the paper evaluates several design choices, the contributions of sparse memory, diffusion forcing, and joint multi-view prediction are not sufficiently isolated. This makes it difficult to determine which components drive the improvements in long-horizon consistency. The reported end-to-end planning latency appears difficult to reconcile with the claimed control frequency. More details about how planning is integrated with real-robot execution are needed to assess its practical real-time performance.
Questions for authors
Section 5.2.3 uses four candidate action chunks with a horizon of 12 steps, while Appendix A5.4 reports approximately 1.45 seconds of planning latency. How is the claimed 10.3 Hz control frequency achieved? Is planning performed asynchronously with robot execution? Could the authors provide ablations removing sparse memory and diffusion forcing separately? This would help clarify their respective contributions to long-horizon prediction quality. For test-time planning, could the authors compare WEAVER-based action selection with a direct action-value baseline trained using the same reward supervision? This would help isolate the benefit of world-model imagination.
3 comments
Response to Reviewer mxzQ (Part 2)
Q4. For test-time planning, could the authors compare WEAVER-based action selection with a direct action-value baseline trained using the same reward supervision? This would help isolate the benefit of world-model imagination.
Thank you for this suggestion. We add a direct action verifier trained using the same RoboMeter reward labels as WEAVER’s reward model. The verifier takes the current observation, language input and a 15-step candidate action chunk as inputs and directly predicts a scalar reward score, without generating future observations or performing world-model rollouts.
At test time, we sample four candidate action chunks from the same base policy, score them with the verifier, and execute the highest-scoring candidate. Both methods use the same candidate count, action horizon, and execution/replanning protocol. This comparison tests whether explicit world-model imagination provides additional benefits over direct action-conditioned reward prediction under matched reward supervision and candidate budgets.
Table VI. Real-world task success rates (%) for the base policy, WEAVER-FT planning, and the direct action verifier.
| Task | Base policy (π₀.₅) | WEAVER-FT planning | Direct action verifier |
|---|---|---|---|
| PnP Marker | 30 | 50 | 30 |
| Pour Beans | 30 | 50 | 35 |
| PnP Towel | 40 | 55 | 45 |
| Stack Bowls | 60 | 65 | 65 |
| PnP Bag | 60 | 70 | 60 |
| Average | 44 | 58 | 47 |
WEAVER-FT planning achieves 58% mean success, outperforming the direct action verifier (47%) by 11 percentage points. Direct verification improves over the base policy by 3 percentage points, while WEAVER-FT planning provides a 14-point improvement, outperforming the verifier on four tasks and matching it on Stack Bowls. These results support the value of explicitly imagining future action consequences for candidate selection, beyond directly predicting reward scores under the same RoboMeter supervision and candidate budget.
We hope our response clarified the concerns and we are happy to answer any further questions.
Best Regards,
The Authors
Reminder to Reviewer mxzQ
Dear Reviewer,
Thank you for your positive assessment of our work and your thoughtful feedback. As the discussion period ends today, we would be grateful if you could review our responses and let us know if any questions or concerns remain. We sincerely appreciate your recognition of our contributions and the time you have devoted to reviewing our paper.
Best regards,
The Authors
Response to Reviewer mxzQ (Part 1)
We thank the reviewer for the valuable feedback. The reviewer highlights the generation quality and faster inference than existing world models, and demonstrated performance gains on downstream applications. We address the concerns below:
Q1 How is the claimed 10.3 Hz control frequency achieved?
Thank you for identifying this inconsistency. Our test-time planning uses four candidate action chunks with a 15-step horizon; the reference to 12 steps is a typographical error that we will correct. The reported 10.3 Hz refers to world-model generation throughput, calculated by dividing the 15-step prediction horizon by the 1.4547-second world-model inference time. Including policy inference and reward/critic scoring, a complete planning call takes approximately 1.65 seconds, corresponding to 9.1 action steps per second of planning throughput. Neither figure represents the frequency of separate replanning calls or the robot’s closed-loop control frequency. We will correct the terminology in the manuscript to distinguish computational throughput from execution and replanning frequency.
Q2. Is planning performed asynchronously with robot execution?
No. Our deployment uses synchronous planning and execution. We evaluate the four candidate chunks, execute the selected 15-step chunk, and then replan using updated real observations. Planning and execution do not overlap, so the interval between replanning calls includes both the approximately 1.65-second planning computation and the selected chunk’s execution time. All reported real-robot improvements were obtained with this synchronous implementation. At the same four-candidate, 15-step setting, WEAVER reduces world-model inference time from 29.42 seconds to approximately 1.45 seconds compared with Ctrl-World, achieving a roughly 20-fold speedup.
Q3: Could the authors provide ablations removing sparse memory and diffusion forcing separately?
We have added more discussion on the design choice below:
We thank the reviewer for pointing this out. We provide the ablations below:
Memory and Diffusion Forcing: In the table below and in table 8 of the updated paper, we observe that WEAVER outperforms the variants that do not use (top row) sparse memory and history and (middle row) Diffusion Forcing. Note that for the model trained without Diffusion Forcing, we used the same noise schedule for each future state during inference. Our experiment presents that WEAVER benefits from both memory and Diffusion Forcing.
Table IV Ablation Analysis of WEAVER on Droid & OOD Dataset
DROID
Task Data (OOD)
SPRINT. We additionally evaluate WEAVER without SPRINT on BridgeV2 in Table 11 in the updated paper and table below. In this work, we incorporated SPRINT to reduce the computational cost (training FLOPS) of world-model training. Surprisingly, similar to the SPRINT paper on image generation, we observe that SPRINT improves the model performance for world models (table of experiment on BridgeV2 dataset is below).
Table V Ablations of WEAVER with SRPINT on Bridgev2 Dataset