- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a new multi-select benchmark on spatial understanding and evaluates multiple VLMs.
Critical strengths
The key idea that VLMs' spatial understanding of a referent needs more comprehensive evaluation is interesting. The paper is also easy to follow.
Critical weaknesses
The task's distinctiveness needs further justification. Besides, the analysis of the evaluation results, e.g., error types and mechanisms, remains limited.
Questions for authors
- The paper lacks a clear characterization of description diversity in the benchmark. One of the central distinctions of CoSpatial-Bench from existing spatial QA benchmarks is its emphasis on multiple valid descriptions that differ in spatial relations, reference objects (anchors), reference frames, or levels of specificity. The authors should clarify the construction criteria and provide a quantitative breakdown of the resulting dataset.
- Section 5.1 mainly summarizes model rankings and metric differences. Could the authors provide more detailed analyses of model performance and failure modes? For example, Doubao Seed 2.0 Pro and Qwen3.8-Max achieve nearly identical EFC scores but substantially different false selection rates. It would be useful to understand whether these differences arise from conservative selection strategies, relation types, or something else.
- What is the motivation for introducing GRPO training, and what does the improvement actually demonstrate? The connection to the benchmark's central objective is not sufficiently explained.
I think the above points need further clarification. I am happy to raise my score if the authors address them.