- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces StereoVLM, a geometry-aware binocular vision-language model for metric 3D spatial reasoning, which learns transferable 3D representations from calibrated binocular observations. Initialized from Qwen3-VL-Instruct, the model undergoes mixed-task instruction tuning with hierarchical supervision spanning cross-view correspondence, disparity estimation, calibration-conditioned metric depth perception, oriented 3D grounding, and task-target grounding. The authors also introduce Stereo3D-Bench, which comprises eight tasks across three levels: point-level geometry, object-level geometry, and object-level spatial reasoning. StereoVLM-8B achieves a point-level depth score of 95.12% and an average score of 66.98 across the five object-level tasks. Additional experiments with StereoVLM-2B demonstrate transfer gains on external spatial-reasoning benchmarks and downstream RoboCasa manipulation tasks.
Critical strengths
-
Well-motivated geometry-consistent supervision. Geometry-Consistent Stereo Supervision treats cross-view correspondence, disparity, and metric depth as complementary supervision targets derived from the same stereo geometry. Table 1 shows that depth recovered from binocular disparity achieves a score of 74.92% on unseen scenes, substantially outperforming direct binocular depth supervision at 45.89%.
-
Broad evaluation with informative ablations and diagnostic controls. Stereo3D-Bench extends evaluation from point-level geometry to language-referred objects and task-target grounding. The authors further conduct controlled right-view interventions, including single-left, zero-right, duplicated-left, and shuffled-right inputs, while keeping the checkpoint, question, calibration, and decoding procedure fixed.
Critical weaknesses
- Limited comparison with modular stereo-based approaches. The main comparisons focus on general-purpose VLMs and monocular depth VLMs, without evaluating a modular pipeline combining a specialized stereo matching model with a VLM. This comparison would help clarify StereoVLM’s advantages over an explicit geometry-based pipeline.
Questions for authors
I am not very familiar with this research area, and the following questions would help me better understand the paper.
-
Benefits of joint learning. Could StereoVLM be compared with a modular pipeline that uses a stereo estimator, such as FoundationStereo [1], to provide geometry to a VLM?
-
Generalization. The held-out in-house benchmark evaluates unseen-scene generalization, while the calibration perturbation experiment assesses robustness to residual extrinsic errors. Could the authors briefly clarify whether camera parameters, such as focal length and stereo baseline, differ between the training and test data, and whether any such variation is covered by the existing evaluation?
[1] Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, and Stan Birchfield. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
3 comments
I appreciate the authors' efforts for the rebuttal phase. Your responses have addressed my concerns, so I will maintain my positive score.
Thank you for your positive feedback and for maintaining your supportive assessment of our work. We are glad that our responses have addressed your concerns, and we sincerely appreciate your time and thoughtful consideration.
We thank Reviewer #PmFi for the constructive feedback and recognition of our supervision design and evaluation. We address your concerns below.
Comparison with modular stereo-vlm pipelines
Table 17. Accuracy and time efficiency of modular stereo–VLM pipelines and StereoVLM-2B. Both modular pipelines use FoundationStereo (FS) and frozen Qwen3-VL-2B. Latency includes FoundationStereo inference.
We thank the reviewer for this suggestion. We have added comparisons with two modular pipelines combining FoundationStereo and a frozen Qwen3-VL-2B-Instruct (Appendix D.7, Table 17):
StereoVLM-2B outperforms both pipelines on all five semantic tasks in Stereo3D-Bench, achieving an average score of 64.43, compared with 35.81 and 5.07 for the localization-based and direct-input pipelines, respectively. Its mean end-to-end latency is 0.643 s, versus 2.212 s and 0.920 s, corresponding to 3.44× and 1.43× speedups. Both modular latency measurements include FoundationStereo inference.
Compared with these explicit geometry pipelines, StereoVLM directly predicts language-referred object geometry and task targets from stereo images and the task prompt in one VLM call, without a separate stereo estimator or intermediate geometry extraction and encoding. These results demonstrate the practical advantages of the integrated approach in both semantic accuracy and inference efficiency.
Generalization Across Camera Configurations
We clarify the camera configurations used in training and evaluation in the table below.
Table: Camera configurations used in training and evaluation.
To further examine how predictions respond to the supplied baseline, we have added a sensitivity experiment on 400 stereo pairs: 200 from DROID and 200 from our in-house robot dataset (Appendix D.4, Figure 8). Keeping the images, query pixels, focal length, and model checkpoint fixed, we vary only the baseline in the prompt from 0.6 to 1.5 times its nominal value. The median predicted depth ratios broadly follow ideal scaling in both domains, suggesting that the model uses the supplied baseline for metric depth prediction.