- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SpaceMind++, a video MLLM that explicitly organizes multi-view visual and geometric features into a shared voxelized 3D cognitive map for spatial reasoning. It introduces Coordinate-Guided Deep Iterative Fusion (CDIF), which performs reasoning over the 3D map and transfers the resulting spatial information back to the original visual tokens using coordinate embeddings, 3D RoPE, and cross-attention. The model is trained on a 900K spatial instruction corpus and evaluated on VSI-Bench, SPBench, SITE-Bench, and SPAR-Bench, showing improved performance over prior spatial MLLMs.
Critical strengths
-
The paper introduces a clear inductive bias for multi-view spatial reasoning, which explicitly organizes observations from different views according to their predicted 3D locations and performs reasoning in a shared 3D space. This is conceptually appealing to me, for example, when the camera moves around a table, the same cup may appear in frame 1 from the left side and in frame 20 from the right side. In an implicit representation, these correspond to two distant visual tokens, and the model must learn by itself that they refer to the same or nearby physical location. In an explicit 3D map, the VGGT-predicted coordinates directly project them into the same or neighboring voxels. Cross-view association is imposed by the representation itself.
-
Another strength is that the representation is world-centered. As the camera moves, the 3D location of an object in the map should remain relatively stable. This makes questions such as “which side of the table is the cup on?” more naturally represented in this world-centered coordinate system.
Critical weaknesses
- The main weakness to me is the proposed method itself. It introduces a strong structural prior, but at the cost of reduced flexibility and strong dependence on upstream (VGGT) geometry quality. Errors from VGGT would be inherited into the 3D map. Once 3D coordinates are inaccurate, incorrect cross-view correspondences may be inherited. Voxelization is also inherently lossy, as geometrically different surfaces or small structures may be merged into the same voxel and averaged then.
Relevantly, the paper lacks a control experiment of testing a model without voxelized map construction but with the same 3D coordinate encoding, 3D RoPE, and a comparable attention capacity applied to the original visual tokens. It is hard to say whether the gain comes from the explicit 3D map itself or mainly from stronger coordinate-guided fusion.
From the computational cost perspective, the full method improves over SpaceMind by about 1-1.5 points under matched-data settings, while adding voxelization, coordinate MLPs, 3D RoPE, map self-attention, cross-attention, and gated fusion. The performance-cost trade-off remain a concern to me.
- The novelty claim seems somewhat overstated to me. The paper frames its main conceptual contribution as moving from treating geometry as an auxiliary feature stream to explicitly constructing a cognitive map and reasoning within it. However, existing work such as Cog3DMap already builds compact 3D cognitive maps. Map2Thought uses metric cognitive maps, 3DLLM-Mem maintains 3D spatiotemporal memory. The authors position the main distinction of this paper from them as, using the map as an intermediate neural workspace and then writing the reasoning result back to the original visual-token interface. As a result, the novel part seems narrower to me, which is, the integration of “3D map -> map-level attention -> cross-attention back to 2D visual tokens.” The novelty is less on the idea of explicit cognitive maps for MLLMs. Considering that LLaVA-3D already performs voxel-based aggregation and that 2D->3D->2D feature propagation has long been explored in 3D vision, the actual conceptual novelty appears weaker than suggested by the paper’s framing.
Questions for authors
Please see weaknesses.