- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that MLLMs’ limited ability in reference-centric spatial reasoning is due to a camera-centric shortcut and proposes OrientSAM to mitigate it. The authors categorize samples as consistent or conflicting based on whether the camera-centric and reference-centric answers are the same, and show that accuracy is higher for consistent samples than for conflicting samples. OrientSAM uses external expert models to construct offline training supervision and to supply depth and orientation cues at inference, and integrates RGB, depth, and Fourier-encoded orientation features through structured multimodal tokens. The paper shows that OrientSAM achieves the highest accuracy on Spatial-MM non-camera-view samples among tested models, surpassing the existing APC-vis method by 0.74 percentage points while maintaining competitive camera-view performance
Critical strengths
-
The paper emphasizes MLLMs’ limitations in reference-centric spatial reasoning and proposes that these limitations arise from models’ tendency to select camera-centric shortcut answers, which makes it difficult for them to produce reference-centric answers. This unified diagnostic framework and its findings are useful for disentangling the remaining failures across VLM benchmarks.
-
OrientSAM achieves a meaningful improvement on ViewSpatial P-Rel, reaching 85.04% accuracy and outperforming APC-vis by 12.47 percentage points. This provides evidence of the proposed method’s effectiveness on the person-view relative direction task.
Critical weaknesses
-
The paper claims that the authors identify reference-object orientation as a key factor governing shortcut strength. However, variation in camera-centric answer selection across orientation intervals could also reflect difficulties in orientation recognition or reference-frame transformation. It is therefore unclear how this conclusion follows from the observation that the rate at which the camera-centric answer is selected varies across orientation intervals.
-
The evidence for consistent improvement on the paper’s central objective is limited. For example, the substantial gain on ViewSpatial P-Rel is meaningful, but P-Obj performance is lower than APC-Vis. The improvement on Spatial-MM Non-Cam is also modest at 0.74 percentage points, considering that the evaluation consists of 134 non-camera-view pairs.
-
\citet is used throughout where \citep is required, making it unclear whether a citation is part of the sentence or a parenthetical reference.
Questions for authors
In Figure 2C, how can camera-centric and reference-centric left/right answers differ when the reference object is back-facing, where the two answers would generally be expected to be the same? Could you provide the number of samples in each angle interval, actual examples, and the data used to produce the plot?