- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduce a Spatial Memory Intelligence (SMI) framework for controlling the long-horizon video context for video generation model. SMI is composed of MI four coordinated atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering.
Critical strengths
- This paper is clear.
- The approach are intuitive, for example the coordinated atomic operations: spatial clustering, within-cluster sparsifica- tion, action-aware retrieval, and reliability-aware filtering.
- Demonstrated across two models on VBench showcase the effectiveness of SMI.
Critical weaknesses
- The claim of first framework to systematically employ an understanding model for spatial-memory management in long-video world models is too strong, there are many related works.
- The approach is very heuristic.
- The models tested are very old.
- Very limited benchmarks are evaluated.
Questions for authors
- How to generalize your approach to industrial scenarios?
3 comments
Dear Reviewer FTHa,
Thank you again for your comments. Our additional experiments demonstrate the scalability and advantages of our method, as well as its effectiveness in long-horizon generation. We hope that these results and the accompanying clarifications address your concerns, and that our work offers the community a new perspective on spatial-memory management.
We sincerely appreciate your time and effort in reviewing our paper and look forward to your further feedback.
Thank the author for their response. But as illustrated in my weakness 1. "many recent related work" are still missing for comparison.
Thank you for your comments.
W1 Novelty Claim
Our novelty claim focuses on three aspects: (1) Task scope: Spatial-memory management in long-horizon action-conditioned video world models. (2) Comprehensiveness: A unified framework systematically addressing spatial consistency, memory sparsification, and generation stability. (3) Trainable paradigm: Dedicated training datasets and a trained understanding model for memory management, with potential for efficient and scalable deployment.
W2 Data Driven Memory Management and Scalability
Our method learns memory-management decisions from data, providing a scalable framework for memory management. In contrast, heuristic retrieval strategies rely primarily on predefined rules, such as FoV-based retrieval in Context as Memory and WorldMem, and 3D geometric retrieval in VMem.
To validate the scalability of our method, We additionally use GPT6-Astra to systematically perform all four operations for spatial memory management. We evaluate it on 20 scenes. The quantitative comparisons with Base and SMI(Qwen) are shown below, and qualitative comparisons have been added to the updated Figure 8 in the main paper. We also provide qualitative visualizations via this anonymous link: https://anonymous.4open.science/w/memory-video-comparisons/ .
SMI(Qwen) uses the trained Qwen model, whereas SMI(GPT6-Astra) uses Astra without task-specific training.
The results show that Astra’s stronger spatial reasoning capabilities enable substantial improvements. We will also release the implementation details of SMI(GPT6-Astra) to support reproducibility.
However, Astra incurs higher inference latency, approximately 3.17 times that of Base, and a high token cost, totaling several billion tokens. Using a smaller model for memory management therefore also offers clear advantages for on-device deployment.
We also use GPT-6.1-sol to evaluate retrieval accuracy across all cases for geometry-only retrieval, geometry combined with semantics, SMI(Qwen), and SMI(GPT6-Astra), as shown below.
These results demonstrate the framework's flexibility across understanding models and the benefits of stronger spatial reasoning for memory retrieval.
W3 Choice and Release Dates of the Evaluated Models
The HY1.5 and Wan2.2 models in WorldPlay-1.5 were open-sourced in January 2026. We began this research in February 2026, when very few open-source world models were available, and even fewer supported memory. We therefore consider these backbones a reasonable choice for evaluating our method.
W4 Evaluation Benchmarks
We combine reconstruction metrics (PSNR and LPIPS), as used in WorldPlay, Context as Memory , and WorldMem, with VBench, GPT-based assessment, and a user study. These four complementary evaluation protocols assess spatial consistency and visual quality beyond reconstruction alone. We believe this provides a comprehensive evaluation that captures the key dimensions relevant to long-horizon world models.
Q1 Generalization to Industrial Scenarios
We believe that integrating understanding and generation within a unified architecture is a promising direction for practical world models. SMI represents a step toward this goal by introducing a trainable understanding-driven memory manager. However, industrial deployment still requires further improvements in latency, efficiency, and integration with generative models. Exploring end-to-end optimization of understanding and generation is therefore an important direction for future work.