- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents GROME, a training-free memory mechanism for long -horizon generation with action-conditioned video diffusion models. GROME separates the memory into static and dynamic components, both of which are represented by point clouds. The static and dynamic tokens are retrieved and injected in different ways in the final generation pipeline. GROME is implemented on LingBotV2. It is compared with other SOTA methods on the WorldScene Benchmark. Experiments show that GROME achieves the best revisit and quality scores.
Critical strengths
- The paper introduces a point cloud-based memory mechanism that decouples static and dynamic components. Although previous works have explored point cloud-based memory, detecting dynamic objects with segmentation and tracking and processing them in a separate branch is a novel approach.
- The paper's presentation is clear and technically sound. The ablation study verifies all design choices. Using only static or dynamic tokens leads to a drop in revisit consistency. The comparison shows that GROME outperforms other SOTA long-horizon generation methods on the WorldScene Benchmark.
Considering sufficient technical contribution and solid experimental results, I recommend accepting this paper.
Critical weaknesses
- The additional computations for point cloud processing (depth estimation) and dynamic object detection may introduce overhead and affect the performance for real-time applications. The runtime comparison with other methods could be provided.
- The method cannot overcome the limitations of point cloud representation. The consistency of the generated video may degrade when the scene cannot be well represented by point clouds. For example, in the waterfall example on the supplementary webpage, the identity of the waterfall changes after revisiting. Because this is a common weakness of all point cloud-based methods, it does not affect the paper's technical contribution.
Questions for authors
- Can the authors discuss the runtime of GROME compared to other SOTA methods?
- Can the authors discuss the limitations of point cloud representation and how it affects the performance of GROME in different scenes?
2 comments
Thanks for the authors' reply. For Q1, I understand that all point-cloud-based methods incur additional computational overhead. However, GROME has no advantages in inference speed compared to other point-cloud-based methods (like Gen3C). Compared to the video backbone LingbotV2, the inference is at least 3$\times$ slower. Therefore, the efficiency concern remains. For Q2, the authors find a similar limitation in representing Fluid with point clouds.
Overall, my concerns about the weakness of this work still exist. Since these limitations are tightly related to point cloud representation itself, I maintain my rating as Weak Acceptance.
Response to Reviewer xFuR (1/1)
We thank the reviewer for their encouraging comments and constructive suggestions. Below are our point-by-point responses:
W1&Q1. Can the authors discuss the runtime of GROME compared to other SOTA methods?
We thank the reviewer for this insightful suggestion. To provide a clearer understanding of GROME's computational cost, we have conducted end-to-end runtime comparisons with representative world models, including LingBotV2, WorldPlay, VMem, Gen3C, WAH, and AlayaWorld. The results and detailed runtime analysis are also provided in the revised Appendix D.
Runtime Comparison with SOTA Models
As shown in the table, point-cloud-based explicit memory methods (such as VMem and Gen3C) inherently incur inference costs, indicating that computational overhead is a shared challenge across explicit 3D memory mechanisms. For GROME, the extra runtime compared to the original LingBotV2 backbone primarily stems from online geometric estimation, representing a natural trade-off between computational overhead and long-horizon structural consistency.
Nevertheless, reducing the geometric estimation window from 93 to 45 decreases the full rollout time from 2,197.1 s to 1,110.5 s. As further supported by our analysis in Appendix D, DA3 inference accounts for the majority of GROME's extra computational cost, making depth estimation acceleration a key priority.
In future work, we plan to explore chunk-delayed updates and lightweight streaming 3D reconstruction models as alternatives to the current depth estimation pipeline, aiming to reduce geometric reconstruction latency while preserving memory fidelity.
W2&Q2. Can the authors discuss the limitations of point cloud representation and how it affects the performance of GROME in different scenes?
We thank the reviewer for raising this important question regarding the limitations of point-cloud representations and their impact on GROME across different scene types. To investigate this, we separately evaluate the Fluid and Deformable subsets of our test set and compare their revisit consistency against the average performance on static scenes. The quantitative results are reported in Appendix F, together with additional qualitative video examples in the updated supplementary material.
Quantitative Results on Fluid and Deformable Scenes
The results show that revisit consistency on fluid and deformable scenes is lower than the average performance on static scenes, with fluid scenes showing a more pronounced degradation. This highlights the challenge of maintaining reliable static memory as scene geometry evolves over time. When scene geometry remains stable, historical local pointmaps from multiple timesteps provide consistent spatial references. However, for fluid and deformable scenes, continuous geometric evolution may introduce conflicts when historical local pointmaps are consolidated, causing the resulting spatial references to deviate from the current scene geometry.
As illustrated in the supplementary videos, although the pretrained video world model can generate plausible motion based on its learned generative priors and recent context, conflicting historical geometric references may still affect revisit consistency. This is particularly evident in waterfall scenes, where the generated water continues to flow, but waterfall geometric changed after revisiting. Based on these obersevations above, future work could explore improved retrieval for non-rigid content in static memory or explicitly model its temporal evolution in dynamic memory to condition video generation.