- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SceneQuery constructs scene memory from RGB observations and uses queries derived from task instructions and output positions to retrieve information for video generation. The queries remain fixed throughout denoising. The method is evaluated on object grounding and scene navigation on ScanNet++, with improvements in navigation reconstruction and grounding geometry, but no improvement in the primary joint grounding-success rate.
Critical strengths
The query-source comparison in Table 3(b) keeps the memory organization fixed and disables the task adapter. It provides useful evidence that task-derived queries outperform noisy-state queries. The navigation results also show a meaningful reconstruction improvement over the matched Video4Spatial baseline, suggesting that the proposed design has practical value.
Critical weaknesses
- The code is not open sourced. Figure 3 refers to supplementary videos, but I could not find any supplementary files or code in the submission. Were these materials omitted? Do the authors plan to release the implementation and evaluation scripts?
- The evaluation needs clarification. The paper explicitly states that object-grounding evaluation coverage remains unresolved, but does not report valid sample counts or how evaluation failures are handled. With 169 context frames, Table 2 reports an SD of 0.2421, whereas Tables 3 and 4 report 0.1764 for the selected full model. Which checkpoints or settings account for this difference? These unresolved issues undermine confidence in the reported comparisons.
- Comparison with related memory methods is insufficient. The main contribution concerns how scene information is retrieved, yet the experiments omit relevant methods such as MemLearner [1], which learns to query context memory, and representative retrieval approaches such as Context-as-Memory [2] and VMem [3]. Table 3(b) supports task-derived queries over noisy-state queries, but does not establish an advantage over these existing approaches. Comparisons with their retrieval strategies under matched conditions, or a clear explanation of why such comparisons are unsuitable, would help establish the contribution.
[1] Yu et al. MemLearner: Learning to Query Context Memory for Video World Models. ECCV, 2026.
[2] Yu et al. Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval. SIGGRAPH Asia, 2025.
[3] Li et al. VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory. ICCV, 2025.
updates in 2026.11.17
The authors have uploaded the videos for visualization this time, making the work more convincing. The replys address my concerns mostly. Therefore, I'm willing to improve the score from 2 to 3. Good luck.
Questions for authors
- Please clarify the valid sample counts and failure handling for object grounding. Are all methods evaluated on the same samples with the same denominator?
- What checkpoints or settings explain the SD difference between Table 2 (0.2421) and Tables 3/4 (0.1764) at 169 context frames?
- Are there existing comparisons with alternative memory-retrieval strategies beyond noisy-state queries? If so, how were their inputs and training budgets matched?
We sincerely appreciate your comments and valuable suggestions. We provide the following responses to address your concerns and answer your questions point by point.
The appendix documents memory size, training hyperparameters, inference defaults, and evaluation settings. We also provide a supplementary package with visual examples, and we will release the code upon acceptance.
Both issues have different causes: the evaluation set and denominator are shared across methods, while the SD difference comes from the registration anchor. The remaining OG coverage issue is metric coverage: strict IF combines target presence with a centering requirement, so we report presence separately.
For coverage, every method uses the same 128 task instances from 20 unseen ScanNet++ scenes, the same seeds, evaluator, and failure-handling rules. There is no method-specific exclusion, so every rate uses denominator 128; all current runs produce 128 valid outputs, and any invalid evaluator output is counted as a failure. Following Video4Spatial, IF requires the target to be present and its predicted box center to be within a relative tolerance γ=0.6 of the box width/height; IF$_{0.2}$ additionally requires SD$<0.2$. Many strict-IF failures are cases where the object is present but slightly off-center:
For SceneQuery, the target is present in 106/128 cases (82.81%); the lower strict-IF rate mainly reflects the centering requirement rather than missing targets.
For SD, we apologize for putting a local diagnostic value into Table 2. During development, we used context-anchored SD to monitor whether generated scenes stayed consistent with the observed context. The benchmark evaluation follows the Video4Spatial protocol, which measures alignment to 40 fixed target frames. Thus, at 169 context frames, 0.2421 and 0.1764 differ because of the registration anchor, not the checkpoint. The context-length results, previously misplaced in the main text, are now:
The comparison must keep the RGB-only input assumption fixed. Context-as-Memory and VMem require explicit camera poses or 3D indexing, so they are not directly comparable here. MemLearner is the closest compatible strategy, so we added a matched comparison using the same backbone, memory budget, samples, and evaluator:
Under the same RGB-only setting, SceneQuery has higher target presence, better reconstruction, and lower rotation and translation errors than the MemLearner-style query.
Thank you again for your comments. We revised the paper and supplementary visualizations accordingly. We look forward to further comments!