- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a video world model (Honeycomb) that stores scene information in six fixed-size feature planes called HexMemory. The model updates the memory with each new video chunk and reads it to guide the next chunk. The model reduces feature storage and memory update costs while achieving good results in novel-view synthesis and revisit consistency.
Critical strengths
-
HexMemory requires limited feature storage, which remains constant as the video length increases. The paper also shows that further reducing the memory size leads to only limited performance degradation. And the model achieves strong results in novel-view synthesis and revisit consistency, indicating that this compressed memory can retain useful scene information.
-
Memory updates are efficient. Each update processes only the newly generated chunk, without reprocessing the entire history or requiring per-scene optimization.
Critical weaknesses
-
The effective temporal resolution of HexMemory decreases as the temporal range expands, which may affect long memory accuracy.
-
HexMemory appears to provide only static scene memory and cannot handle dynamic object state updates. In this case, is it necessary to retain temporal information? For example, comparing the three spatial planes (XY, XZ, and YZ) with the proposed six-plane representation would help assess the improvements brought by the spatiotemporal planes.
Questions for authors
How does memory accuracy change as the spatial and temporal ranges expand? Could the authors evaluate longer sequences and larger scenes to assess the effect of decreasing effective resolution?
3 comments
Q1: How does memory accuracy change as the spatial and temporal ranges expand? Could the authors evaluate longer sequences and larger scenes to assess the effect of decreasing effective resolution?
We thank the reviewer for the insightful feedback. We evaluate memory retention on 100 RE10K clips by sequentially writing video observations while keeping the plane dimensions fixed and allowing spatial and temporal bounds to expand. After each update, we retrieve the same first-chunk features and measure reconstruction MSE. Error increases from 0.0598 at 33 frames to 0.2006 at 129 frames, indicating reduced recall accuracy as history accumulates. These measurements reflect the combined effects of range expansion and repeated memory updates.
For longer sequences and larger scenes, we additionally evaluate closed-loop generation on WorldScore scenes using 2, 4, and 6 chunks, following the same protocol as Spatia. Each trajectory moves outward for half the chunks before returning. Keeping the per-frame camera motion and memory-plane dimensions fixed, longer trajectories explore farther and revisit earlier content after longer histories (i.e., both spatial and temporal ranges expand). Revisit quality decreases as the trajectory length increases, demonstrating the practical coverage–accuracy trade-off under a fixed feature-memory budget. Note that these generation results also include accumulated autoregressive errors (which may be improved by self-forcing approaches in future work), complementing the direct memory-retention measurements above.
Each cell reports PSNR ↑ / SSIM ↑ / LPIPS ↓, as follows:
| Plane resolution | HexMemory (MB) | 2 chunks | 4 chunks | 6 chunks |
|---|---|---|---|---|
| PSNR ↑ / SSIM ↑ / LPIPS ↓ | PSNR ↑ / SSIM ↑ / LPIPS ↓ | PSNR ↑ / SSIM ↑ / LPIPS ↓ | ||
| 512 | 73.9 | 17.22 / 0.504 / 0.311 | 15.61 / 0.453 / 0.471 | 13.68 / 0.400 / 0.590 |
| 384 | 42.6 | 17.14 / 0.503 / 0.313 | 15.53 / 0.449 / 0.477 | 13.44 / 0.393 / 0.602 |
| 256 | 19.8 | 17.10 / 0.500 / 0.319 | 15.39 / 0.444 / 0.490 | 13.53 / 0.393 / 0.616 |
| 128 | 5.8 | 16.77 / 0.490 / 0.338 | 15.09 / 0.433 / 0.519 | 13.28 / 0.384 / 0.643 |
We will now explicitly discuss this in the updated manuscript.
We sincerely thank the reviewers for the helpful feedback, and hope these address your concerns.
Summary of Response - Sincerely appreciate the feedback and looking forward to further engagement!
We thank the reviewer again for the constructive feedback. As the discussion period is coming to an end, we would like to follow up on our rebuttal and highlight the additional experiments addressing the effects of expanding coverage and the role of spatiotemporal planes. We summarize the key findings below and indicate where they are discussed in the revised manuscript.
| Feedback raised | Our response to feedback | Location |
|---|---|---|
| W1 & Q1. How does HexMemory accuracy change as spatial and temporal coverage expands under a fixed memory budget? | We evaluated 2-, 4-, and 6-chunk WorldScore closed-loop trajectories at plane resolutions of 128–512, keeping feature storage fixed at each resolution. Reducing the resolution from 512 to 256 saves approximately 73% of feature storage (73.9 MB to 19.8 MB), with only a 0.12–0.22 dB PSNR decrease across trajectory lengths. Revisit quality declines over longer trajectories, reflecting both expanding coverage and accumulated autoregressive errors. | Section 4.5, Table 4 |
| W2. Are the spatiotemporal planes useful, and can the model preserve dynamic motion? | We compared spatial-only tri-plane memory against the six-plane representation. The six-plane model improves RE10K NVS PSNR from 18.02 to 18.45 dB and WorldScore closed-loop PSNR from 16.03 to 17.22 dB, with improved SSIM and LPIPS. Our additional retention experiment also shows approximately 6.2× lower first-chunk feature MSE after 97 frames (0.1275 vs. 0.7848). We further added a qualitative example of coherent dynamic subject across generated chunks, while acknowledging that rigorous dynamic-state tracking remains future work. | Section 4.5, Table 5; Section 4.4, Figure 7 |
We hope these additional experiments and clarifications address your concerns. We would greatly appreciate any further feedback. Thank you again for your time and consideration!
Author's Response - Thank you for your comments!
We greatly appreciate the reviewer's valuable comments, which help to improve the rigor of the paper. Following your feedback, we have added new experiments and results. We present our responses and findings below.
We thank the reviewer for the comment. We agree that expanding the temporal range under fixed plane dimensions reduces effective temporal resolution and may affect memory accuracy. We have conducted memory-retention and longer-horizon generation experiments to characterize this trade-off. Please refer to our response to Q1 below for the results and discussion.
We thank the reviewer for this valuable suggestion. The spatiotemporal planes form part of our low-rank feature factorization and can indeed contribute spatial detail even in static scenes, which makes retaining the temporal information important too. We have conducted an ablation comparing the three spatial planes with the current six-plane representation on the full RE10K NVS and WorldScore closed-loop evaluations.
On full RE10K NVS, the six-plane representation achieves better mean performance, as follows:
On full WorldScore closed-loop evaluation, the advantage is more pronounced, as follows:
We also evaluate memory retention on 100 real RE10K clips. After writing multiple autoregressive chunks, we retrieve the same first-chunk features and measure their reconstruction MSE. The six-plane representation achieves lower reconstruction error at every tested history length, indicating more accurate recall of earlier content as observations accumulate. After three chunks, its error is approximately 6.2× lower than tri-plane, as follows:
These results show that the additional spatiotemporal planes contribute to static-scene memory accuracy and generation consistency in our trained model.
Regarding dynamic object state updates, we have now also included a new figure (anonymous link here: https://github.com/Anonymous-Submission-20/Honeycomb/blob/main/rebuttal_figures/rebuttal_figureB.pdf), which provides a qualitative example of ducks moving coherently across successive generated chunks as the camera pans back. Compared to Spatia, the ducks incoherently flew backwards as the camera pans back. Nonetheless, dynamic fidelity depends on the quality of underlying video foundation model. While this example illustrates cross-chunk dynamic motion coherence, rigorous evaluation of dynamic-state tracking and the memory’s contribution remains as potential future work.