- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces SpaReCo-Bench, a benchmark connecting spatial recognition to compositional use through shared object views and linked questions. It organizes 15 task identifiers into state, identity, relation, and composition families, reported as 25 evaluation entries, and combines controlled object-view questions with retained tasks from existing benchmarks. The evaluation includes 11 general-purpose or spatially tuned VLM checkpoints, targeted fine-tuning of Qwen2.5-VL-3B, linked condition-following analyses, and external evaluation. The main reported finding is that source-task recognition gains transfer unevenly to related selection and transformation tasks, whereas direct supervision improves some attribute-composition tasks. The intended contribution is a controlled diagnostic evaluation framework rather than a new model.
Critical strengths
It departs from the isolated scoring paradigm adopted by conventional spatial benchmarks, and couples basic recognition tasks with compositional decision tasks through shared object views. This design enables precise pinpointing of whether model errors stem from inadequate spatial recognition or insufficient compositional reasoning.In particular, its hierarchical task gradient and paired experimental design clearly delineate three distinct levels of compositional ability—attribute recombination, explicit transformation application, and analogical rule transfer—achieving substantially finer diagnostic granularity than standard benchmarks.
Critical weaknesses
Dense task identifiers, experimental groupings, and transfer matrices fragment the narrative. Frequent cross-referencing obscures the main argument and makes the experimental evidence difficult to follow.
Despite 11 checkpoints in Table 2, general-purpose baselines are limited to 2B-7B models, and the key transfer results rely on Qwen2.5-VL-3B alone. Larger models, newer open models such as Qwen3.5 and InternVL-3.5, and proprietary references such as Gemini 3.8 Flash and GPT-6 are absent. This limits the generality of the reported compositional deficits.
The benchmark primarily combines established tasks in viewpoint recognition, cross-view matching, attribute conjunction, rotation, and visual analogy. Explicit task links provide useful organization, but their distinct diagnostic value over existing evaluations is not established. Table 1 demonstrates coverage rather than a new measurement capability.
The evidence does not clearly separate recognition errors from failures to use correctly recognized information. Low compositional accuracy alone cannot establish a distinct reasoning deficit, limiting the strength of the central claim.
Questions for authors
What diagnostic insight is unique to the linked evaluation, and what evidence isolates its benefit? Are unseen attribute combinations or transformation patterns held out alongside object identities?
Does low C2/C3 accuracy persist when all required components are correctly recognized under matched input contexts? Do ground-truth controls or human results rule out presentation issues and task ambiguity?
Do the key deficits and transfer patterns persist across model scales, newer models, and a second backbone?