- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SpatialMem constructs object-level 3D memory from monocular videos using predicted geometry, segmentation masks, and identity refinement. The memory supports program-based spatial reasoning and learned VLM readout. It achieves 65.3% on ReVSI and reports exploratory gains on SQA3D and ScanQA.
Critical strengths
-
The method addresses merged objects and fragmented identities, with substantial counting gains supporting the refinement pipeline.
-
Competitive ReVSI results are accompanied by useful diagnostics connecting memory-construction errors to downstream counting and geometric-query failures.
Critical weaknesses
-
Observation and computation budgets are not matched: SpatialMem constructs memory from up to 300 frames, while many baselines use 32–64. Controlled comparisons are needed to isolate the benefit of identity refinement from additional observations and processing.
-
Calibration data overlap with evaluation scenes: Appendix B.3 reports 11 overlaps with SQA3D test and 18 with ScanQA validation. Although disclosed as exploratory, the learned-readout results require training-only calibration to establish held-out generalization. This concern does not directly invalidate the ReVSI results.
Questions for authors
-
Does the ReVSI advantage persist at a matched frame budget, such as 64 frames? What are the construction cost, VLM-call counts, and amortized query latency?
-
How do the learned-readout results change after retraining with calibration scenes strictly disjoint from evaluation scenes?
-
How often do queries use instance-level operations versus category-level fallbacks? Can direct instance-quality metrics or a matched alternative-association baseline substantiate the identity-refinement contribution?