- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GeoMem3D, a framework for 3D spatial reasoning that reconstructs multi-view observations into a query-independent, object-centric structured 3D world memory. It introduces two reasoning paradigms: GeoMem-Holistic, which reasons over the complete memory, and GeoMem-Agentic, which adaptively retrieves spatial evidence. Both are optimized through SFT followed by GRPO. Experiments on VSI-Bench-Tiny and MindCube-Tiny show improvements over the Qwen3-VL-8B backbone.
Critical strengths
- The paper aims to address an important challenge in 3D spatial reasoning: maintaining coherent geometric and semantic relationships across viewpoint-dependent observations.
- Organizing reconstructed geometry into explicit object entities and spatial relations provides an interpretable basis for reasoning.
Critical weaknesses
W1. The central design largely combines established components: 3D reconstruction, object-centric scene graphs, memory retrieval, and SFT–GRPO training. Related approaches, including ConceptGraphs, MSSR (ICLR 2026), and Scene Graph Thinking (ICML 2026), have already explored structured spatial representations or adaptive evidence-based reasoning. The paper does not sufficiently distinguish its methodological contributions from these approaches.
W2. Unclear training-data provenance and confounded performance gains. The authors construct over 5K SFT trajectories using Qwen3-VL-32B, with ground-truth answers provided during trajectory generation. However, the precise training sources, scene-level partitions, and potential overlap with evaluation data are insufficiently documented, raising concerns about answer-conditioned supervision and possible data leakage.
W3. Serious inconsistencies in experimental reporting. Figure 4 reports approximately >80% for GeoMem-Agentic on VSI-Bench-Tiny, whereas Table 1 reports only 59.6%. Similar discrepancies occur for other models and MindCube-Tiny. Furthermore, Figure 4 is labeled with Accuracy (%) despite appearing to reuse the results presented as Explainability Analysis in Figure 1(b). These inconsistencies obscure the reported experimental conclusions.
W4. Insufficient benchmarks and core-component ablations. Evaluation is restricted to VSI-Bench-Tiny and MindCube-Tiny, while Table 3 reports ablations only on the aggregate VSI-Bench-Tiny score. This is inadequate for a method whose principal motivation is improving cross-view consistency and structured spatial reasoning. Without task-specific ablations on relative direction, spatial relations, and route planning, or direct cross-view consistency evaluation, the results do not establish whether the proposed memory addresses the specific limitations identified in the introduction.
Questions for authors
Q: What are the exact sources and scene-level partitions of the 5K SFT trajectories? How is evaluation-data independence ensured when the teacher receives ground-truth answers?
We sincerely thank the reviewer for the positive evaluation and recognition of the importance of cross-view spatial consistency and the interpretability of our structured 3D world memory.
Q1: GeoMem3D's Core Contributions
Existing studies can be broadly categorized into three directions:
(1) 3D Scene Modeling and Reconstruction. ConceptGraphs (ICRA 2024), HOV-SG (RSS 2024), and LEXI-SG (2026) primarily focus on reconstructing and organizing 3D scenes into open-vocabulary or hierarchical representations for geometric and semantic scene mapping.
(2) Scene Representation and LLM Integration. 3DGraphLLM (ICCV 2025) investigates encoding object-level semantic and relational structures into learnable representations for LLMs. Scene Graph Thinking (ICML 2026) further employs graph-aligned SFT and reinforcement fine-tuning on scene graphs derived from image-text data to teach MLLMs structured visual reasoning.
(3) Inference-time Spatial Evidence Acquisition. MSSR (ICLR 2026) adopts a dual-agent framework to iteratively acquire and prune spatial information into a minimal sufficient set for each query.
In contrast, GeoMem3D presents a unified framework that integrates 3D reconstruction, scene perception and understanding. Rather than treating 3D geometry merely as an input representation or an external source of tool-based evidence, or implicitly encoding 3D information into the model itself, we combine explicit 3D world memory with the original 2D visual context. Through a lightweight post-training strategy combining supervised fine-tuning and reinforcement learning (SFT + GRPO), we train MLLMs to adaptively select, integrate, and reason over these complementary information sources, enabling more effective spatial reasoning. Our approach aims not only to improve answer accuracy but also to enhance the transparency and interpretability of the reasoning process by grounding predictions in explicit spatial evidence, as illustrated by the results in paper's Figs. 3 and 4.
Q2. Clarification of Training Data Provenance and Evaluation Separation
We thank the reviewer for raising this concern. We clarify that the over 5K SFT trajectories were generated by Qwen3-VL-32B using the VSI-Bench training data, explicitly excluding all scenes and videos in VSI-Bench-Tiny. The training data sources, scene-level partitioning, and evaluation split follow the official VSI-Bench protocol, ensuring no scene-level overlap between training and evaluation data. Regarding ground-truth answers, they are provided to the teacher as auxiliary guidance to improve trajectory generation and facilitate quality verification, rather than to derive or constrain the intermediate reasoning steps.
Q3. Clarification of Answer Accuracy and Reasoning Interpretability Metrics
We would like to clarify that the results in Table 1/2 and Figure 4 correspond to two different evaluations. Tables 1 and 2 report answer accuracy over all benchmark questions, whereas Figure 4 evaluates reasoning interpretability. Specifically, for the interpretability evaluation, we retain only correctly answered instances and employ GPT-4o as an anonymized evaluator to assess the generated reasoning along three dimensions: logical coherence, consistency between reasoning and final answers, and relevance and sufficiency of referenced spatial evidence. Therefore, the scores above 80% in Figure 4 represent reasoning interpretability scores conditioned on correct predictions, rather than benchmark answer accuracy.
Q4: Ablations and Robustness to Viewpoint Variations
(1) Ablation Study. We have reported the ablation study in the revised manuscript by introducing memory-free SFT and GRPO baselines to distinguish the benefits of structured memory from general post-training. Detailed ablation results across all spatial reasoning tasks are provided in Table 7 of the appendix.
(2) Robustness to Viewpoint Variations. We further evaluate spatial reasoning under two different viewpoint-based frame samplings from the same videos, while keeping the questions and ground-truth answers unchanged.
Table 2: Task-specific Spatial Reasoning Performance under Viewpoint Variations on VSI-Bench-Tiny.
GeoMem3D-Holistic consistently outperforms the video-only baseline in both tasks across different viewpoints, suggesting that structured 3D world memory improves the robustness of spatial reasoning to viewpoint variations. We have also incorporated these experiments into the revised manuscript, as presented in Table 6.