- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces SpaceDG, a 3DGS-based dataset and benchmark for evaluating MLLM spatial reasoning under nine realistic visual degradations. Experiments on 25 models reveal significant robustness gaps, while degradation-aware fine-tuning substantially improves performance on both clean and degraded images.
Critical strengths
- The paper addresses an important yet underexplored problem: the robustness of MLLM spatial reasoning under realistic visual degradations, bridging the gap between existing clean-image benchmarks and real-world deployment scenarios.
- The proposed 3DGS-based degradation synthesis pipeline generates diverse visual degradations while preserving geometric consistency and spatial ground truth, enabling large-scale construction of reliable spatial QA data.
- The authors evaluate 25 MLLMs across nine degradation types and 11 spatial reasoning tasks, revealing substantial robustness gaps and identifying fine-grained object-centric reasoning as particularly vulnerable to visual degradations.
- Supervised fine-tuning on SpaceDG substantially improves spatial reasoning performance under both clean and degraded conditions, demonstrating the practical value of the constructed dataset.
Critical weaknesses
- Although the paper reports substantial gains from SpaceDG-SFT, the clean-only SFT baseline already achieves comparable performance. Specifically, degradation-aware SFT improves average degraded performance by only 1.3 percentage points (64.8% → 66.1%), raising questions about the unique contribution of degradation augmentation.
- The dataset is primarily constructed from indoor ScanNet++ scenes with synthetically generated degradations. The lack of evaluations on real-world degraded images leaves uncertainty about how well the benchmark and training improvements transfer to practical deployment scenarios.
- Insufficient experimental evidence for some conclusions. The finding that degradation-guided CoT harms spatial reasoning is supported by experiments on only one model, making it difficult to generalize this conclusion to other MLLMs or reasoning strategies.
Questions for authors
- Limited improvement attributable to degradation-aware training According to Table 6, degradation-aware SFT only improves performance by 1.3 percentage points over clean-only SFT (64.8 vs. 66.1) on degraded images. Could the authors further clarify the unique contribution of degradation augmentation and provide additional experiments to demonstrate whether this improvement is statistically significant?
- Limited real-world generalizability Since SpaceDG is primarily constructed from indoor scenes with synthetically generated degradations, how well do the proposed benchmark and degradation-aware training generalize to real-world degraded images? Could the authors provide additional evaluations on real-world degradation datasets or outdoor environments to validate the practical applicability of their approach?
- Insufficient evidence for the CoT-related conclusion The conclusion that degradation-guided CoT negatively affects spatial reasoning is based on experiments with only Gemini-3.1-Flash-Lite. Could the authors extend this evaluation to additional MLLMs, particularly stronger reasoning models, and investigate whether alternative degradation-aware prompting strategies could improve performance rather than impair it?
6 comments
Responses to Reviewer M5Rb's W2&Q2
W2&Q2. Generalization to degraded images from real-world outdoor scenes
Thank you for highlighting the importance of evaluating real-world outdoor scenes under visual degradation. We evaluate both models on a stratified sample of 500 questions from Ego3D-Bench [1], covering real-world driving scenes from Waymo, nuScenes, and Argoverse. Some of the original images already exhibit naturally occurring degradation, including low light, motion blur, sun glare with clipped highlights, and localized blur in rainy conditions. We evaluate the same questions using the original images and versions with each of the nine SpaceDG degradations applied separately. We report multiple-choice accuracy and average the degraded results across the nine conditions.
| Model | Input Images | Degradation Types | Performance (%) ↑ |
|---|---|---|---|
| Qwen3-VL-8B | Original | Low light, motion blur, sun glare with clipped highlights, and localized blur in rain | 45.97 |
| Qwen3-VL-8B | Degraded | Nine SpaceDG types | 42.42 |
| SpaceDG-SFT-Qwen3-VL-8B | Original | Low light, motion blur, sun glare with clipped highlights, and localized blur in rain | 57.02 |
| SpaceDG-SFT-Qwen3-VL-8B | Degraded | Nine SpaceDG types | 56.16 |
As shown in the table, SpaceDG-SFT improves accuracy under the added degradations by 13.74 percentage points over the base model and retains nearly all of its accuracy on the original images. Notably, SpaceDG-SFT is trained only on degraded indoor scenes from ScanNet++, so these results provide evidence that its gains transfer to outdoor driving scenes. In addition, its accuracy decreases by only 0.86 percentage points from original to degraded conditions, compared with 3.55 percentage points for the base model. These results demonstrate that SpaceDG-SFT transfers both its spatial reasoning capability and degradation robustness from indoor training scenes to a substantially different outdoor domain.
[1] Mohsen Gholami et al. Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes. ICLR 2026.
Responses to Reviewer M5Rb's W3&Q3
W3&Q3. Degradation-guided CoT with additional MLLMs
| Model | Method | Performance (%) ↑ |
|---|---|---|
| GPT-6-Astra | w/o CoT | 56.18 |
| GPT-6-Astra | with CoT | 54.71 |
| Qwen3.8-Flash-Next | w/o CoT | 54.09 |
| Qwen3.8-Flash-Next | with CoT | 53.32 |
We extend the two-stage CoT method to GPT-6-Astra and Qwen3.8-Flash-Next on SpaceDG-Bench-900, which comprises 100 questions evaluated under each of nine degradations. For each model, we hold the images, questions, and reasoning-effort setting fixed. The CoT prompt asks for a degradation label and a brief spatial explanation before the final answer, as described in Section 5.1. As shown in the table, the overall changes are −1.47 and −0.77 percentage points, respectively, providing further evidence that degradation-guided CoT can impair model performance under visual degradation.
As illustrated in Figures 11–19, our analysis suggests that visual degradation can lead models to misinterpret image content. CoT reasoning then proceeds from incorrect premises, propagating these perceptual errors and ultimately producing incorrect conclusions.
Responses to Reviewer M5Rb's W1&Q1
Dear reviewer M5Rb, we sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. In the following, we provide a detailed response to address your concerns.
W1&Q1. Incremental gain from degradation-aware SFT
Thank you for raising this concern. To examine the benefits of degradation-aware SFT, we extend SFT into a second epoch. As shown in the table, clean-only SFT and SpaceDG-SFT achieve average scores of 67.3 and 69.3, respectively, across the nine SpaceDG-Bench degradations. Relative to epoch 1, SpaceDG-SFT gains 3.2 points, compared with 2.5 points for clean-only SFT, widening its advantage from 1.3 to 2.0 points. This widening suggests that degradation-aware supervision can deliver increasing benefits with continued training. It further motivates the conclusion that, compared with clean-only SFT, degradation-aware SFT can use additional training exposure and computation more efficiently to improve spatial reasoning robustness.
We also conduct paired bootstrap analyses with 10,000 replicates at both the scene and question levels using 1,102 questions from 320 scenes, keeping both models' predictions and all nine degradation variants together within each sampled unit. The second-epoch gain is +1.97 points before rounding, corresponding to +2.0 in the table above. Both 95% percentile confidence intervals exclude zero, supporting a positive average advantage of SpaceDG-SFT on degraded images.