- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates a limitation of camera-conditioned video world models based on Gated DeltaNet (GDN). When the camera returns to a previously visited region, the model often fails to maintain consistent scene geometry. The authors attribute this problem to long-range forgetting. As new video chunks are processed, repeated state updates gradually weaken information from earlier chunks, even when the new chunks show unrelated regions. To address this issue, the authors propose HLA-WM, a training-free method that caches an affine summary of each chunk's GDN state transition. Using camera poses, the method identifies relevant historical chunks based on camera frustum overlap and combines their cached summaries in chronological order to reconstruct the recurrent state. Experiments on SANA-WM-Bench and MBench-A show improvements in revisit consistency and camera control.
Critical strengths
-
The paper clearly identifies and analyzes a limitation of recurrent linear attention. The chunk-level formulation of GDN and the retention analysis provide an intuitive explanation of why relevant information from earlier chunks is gradually lost.
-
The proposed method makes effective use of camera poses that are already available as model inputs. It requires no additional training and introduces minimal computational overhead.
-
The experimental results consistently show improved revisit consistency across two benchmarks, two trajectory splits, and three inference modes.
Critical weaknesses
-
The method relies on camera poses and a scene-level median depth provided in the trajectory metadata. While these inputs are available in the evaluated setting, it is unclear how robust the method would be to inaccurate pose or depth estimates, or how it could be extended to action-conditioned models without explicit camera poses.
-
The comparison with existing memory methods is limited. Although HLA-WM uses 12 times less memory than full KV caching, the paper does not compare their revisit consistency. It also lacks comparisons with retrieval-based memory methods such as WorldMem and Context-as-Memory, which use similar geometric criteria to select historical information. Without these comparisons, it is difficult to assess the trade-off between memory efficiency and revisit consistency relative to alternative approaches.
Questions for authors
-
How robust is the retrieval to errors in camera poses or scene-level depth estimates? Can the method work in interactive settings where camera poses must be estimated rather than provided?
-
How does HLA-WM compare with full-history KV access or geometry-retrieved KV memory under comparable conditions? The KV-only ablation in Table 4 does not provide such a comparison. This would help clarify the trade-off between memory efficiency and revisit consistency.
-
How much does geometric ranking improve retrieval over alternative selection strategies under the same memory budget? Table 7 shows only a small gain over uniform selection (9.91 vs. 10.00 PSNR), while the retrieval budgets are not strictly matched.
2 comments
Response to reviewer 1Qme(2/2)
Q3. How much does geometric ranking help under the same historical-memory budget?
A3. Geometry-guided retrieval provides meaningful benefits over uniform selection, although its advantage depends on the evaluation horizon and scene characteristics.
-
Matched-budget experiments demonstrate the effectiveness of geometric ranking. As shown in the result table in our response to Reviewer cE3k (Q3), under the default K=1 setting, Geometry-1 improves PSNR by 0.45 dB over Uniform-1 (10.11 vs. 9.66) and 0.53 dB over Random-1, outperforming both across all six metrics. These results demonstrate the benefits of geometric ranking beyond simply retaining additional history. We have also incorporated this comparison into the Experiments section of the revised manuscript.
-
The limited evaluation horizon and scene complexity can reduce the advantage of geometric retrieval. The 60-second rollouts contain only 40 chunks, allowing uniform selection with six chunks to cover much of the history. Occlusions and restricted viewpoints in indoor scenes, together with dynamic objects and repetitive structures in outdoor city scenes, further reduce retrieval effectiveness. We discuss these limitations in the conclusion.
Response to reviewer 1Qme(1/2)
Q1. How robust is retrieval to inaccurate camera poses or depth estimates, and can it support action-conditioned models without explicit poses?
A1. HLA-WM demonstrates robust historical retrieval under inaccurate camera poses and depth estimates. We conduct additional experiments on the complete Hard-Trajectory split, perturbing only the geometric metadata used for retrieval while keeping the generation-conditioning camera trajectories unchanged. We evaluate translation noise, rotation noise, and depth-scale perturbations.
HLA-WM maintains stable revisit consistency across all tested perturbations, with PSNR ranging from 10.03 to 10.11 dB compared with 10.11 dB without perturbation. These results demonstrate that geometry-guided retrieval remains effective under the evaluated pose and depth inaccuracies. For action-conditioned world models, motion-related actions can similarly provide spatial cues for retrieving relevant historical information. We have included the robustness analysis in Appendix A.9 of the revised manuscript.
Q2. Comparisons with full-history KV access, geometry-retrieved KV memory, WorldMem, and Context-as-Memory are missing.
A2. HLA-WM addresses long-range forgetting in fixed-size GDN states through selective historical-state recomposition, a limitation not addressed by existing memory retrieval methods. We have clarified this distinction in Sec. 2 and added comparisons in Appendix A.10.
Additional comparisons demonstrate the effectiveness and memory efficiency of GDN-state recomposition. We evaluate alternative memory mechanisms on the same SANA-WM backbone. The KV-based baselines retain historical softmax KV while keeping the native GDN recurrent states unchanged.
GDN-only recomposition improves all three revisit metrics with minimal additional memory. Full HLA-WM further achieves the best revisit consistency with only 0.88 GB more peak memory than the KV-based alternatives, demonstrating a favorable memory–consistency trade-off.
Existing retrieval methods do not address recurrent GDN-state forgetting. WorldMem retrieves historical visual memory rather than recomposing GDN states. Our adapted KV-retrieval baseline achieves 9.80 dB PSNR compared with 10.11 dB for HLA-WM. Context-as-Memory's official implementation has not been publicly released, preventing a reliable quantitative comparison. We therefore discuss it qualitatively.
HLA-WM preserves substantial memory efficiency while achieving the best revisit consistency. At 60 seconds, GDN summaries require only 2.15 GB compared with 25.82 GB for full-history KV in the corresponding layers. Including softmax KV, HLA-WM requires 10.76 GB of historical cache, 3.20× less than the all-layer full-history KV reference (34.43 GB). It also maintains comparable throughput to native SANA-WM (22.053 vs. 22.403 FPS). Complete memory accounting is provided in Appendix A.11.