acceptodds
Under review as a conference paper at ICLR 2027

SpaEx: Target-Aligned Spatial Evidence for Multimodal Agents

Abstract

For spatial reasoning, multimodal agents built on vision language models (VLMs) must interpret measurements relative to the objects and reference frames specified by a goal. Aligned states still leave candidate comparisons to the model. We introduce SpaEx, a target-aligned interface coupling object-centered perception with explicit spatial execution and fusing candidate evidence with native VLM scores in the original answer space. With the VLM and perception tools fixed, the reader fits only four scalar parameters. Across SpatialMQA, SPAR-Bench single-view reasoning and CV-Bench 3D, SpaEx improves two backbones by 5.3–18.9 percentage points over matched task-adapted VLMs and attains the highest accuracy and semantic Macro-F1 among compared methods in every benchmark–backbone setting. Under shared observations, readers supplied with executed comparisons outperform those given aligned states even with longer training and compact inputs. Candidate-level fusion also recovers correct answers ranked first by neither branch. Although execution and fusion contribute differently across tasks, these results support explicit computation as an effective interface between spatial perception and native visual judgment.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision–Language Models, Spatial Reasoning, Object-Centric Perception, Structured Execution, Evidence Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LAL6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SpaEx, a target-aligned spatial reasoning framework that combines object-centric perception, explicit geometric execution, and candidate-level fusion with native VLM scores. It supports viewpoint-dependent relations and metric comparisons while keeping the perception models and VLMs fixed during reader calibration. Experiments on SpatialMQA, SPAR-Bench, and CV-Bench 3D demonstrate consistent improvements over task-adapted baselines across two VLM backbones. Ablations further investigate the benefits of executed evidence over aligned spatial states and the complementary roles of geometric execution and native VLM predictions.

Critical strengths
  1. SpaEx consistently improves performance across three spatial reasoning benchmarks and two VLM backbones. The matched comparisons, multiple random seeds, and additional-training controls provide convincing empirical support.

  2. The ablations distinguish geometric execution from native-score fusion and demonstrate that explicitly computed candidate evidence can be more effective than presenting aligned spatial states directly to VLMs. The analysis of complementary predictions is particularly valuable.

Critical weaknesses
  1. SpaEx primarily supports predefined spatial operations and benchmark-specific interfaces. Only 380 of 1,076 SpatialMQA test questions admit execution, while SPAR-Bench and CV-Bench exploit public task specifications or visual markers. Generalization to unrestricted spatial reasoning remains unclear.

  2. The framework largely integrates existing perception, geometric operations, and score fusion. The aligned-state comparisons do not fully isolate execution from differences in representation and available information. Moreover, adaptive fusion provides little improvement over constant fusion and no additional accuracy on CV-Bench, weakening the justification for the proposed adaptive reader.

Questions for authors
  1. Can the authors provide a more tightly controlled comparison between aligned-state readout and executed evidence, matching native-score access, input representation, and training budget? How would SpaEx compare against a complete APC implementation?

  2. How well does SpaEx generalize to spatial questions without benchmark-specific markers or task categories? Can the authors quantify the impact of extending execution coverage beyond the currently supported SpatialMQA questions?

1 comment
AuthorsOpenReview ↗

Response to Reviewer LAL6

Thank you for the positive assessment and the precise questions. Results below are new unless a table of the paper is cited; details will be in the revised appendix. Two numbers are Qwen / LLaVA; SPAR is the task macro.

Q1 (W2). A tighter state-versus-execution comparison, and complete APC.

Tighter comparison. Two controls give both readers the same information and vary only whether the candidate comparisons are executed (three seeds, full test sets):

ReaderState only+ executed comparisons
Decoder-internal, SpatialMQA74.38 / 72.7479.24 / 77.32
Decoder-internal, SPAR65.53 / 62.1079.38 / 79.45
Decoder-internal, CV-Bench 3D88.56 / 76.5693.86 / 91.14
Text reader, SPAR72.03 / 70.4179.45 / 80.52

The internal readers share operands, candidate binding, capacity, examples, updates and loss. Both text readers receive the aligned state, the executor's conventions, native scores and entropy, and are trained on the same 502 examples. Execution wins all 18 internal-reader cells (+3.3 to +19.3 points) and all six text-reader seed pairs (+7.4 / +10.1 on average).

Complete APC. We ran the official APC visual pipeline. It reproduces its published 3DSRBench numbers (70.5 / 65.9 against 72.78 / 66.47). Under our protocol (original questions and candidates, accuracy in %):

SpatialMQACV-Bench 3DSPAR
APC as released (Qwen2.5-VL-7B)27.757.321.8
APC pipeline, our backbones and matched adapters50.0 / 40.662.8 / 65.740.2 / 39.4
Matched LoRA73.6 / 71.887.3 / 77.066.0 / 62.2
SpaEx79.1 / 77.194.2 / 92.582.2 / 81.1

With the same backbones and adapters, APC is 11–31 points below the matched LoRA alone and 27–42 points below SpaEx. The paper's APC-style control stays labelled as a shared-observation control.

Q2 (W1). Generalization without markers or task categories; quantifying a coverage extension.

  • Markers and categories. SpatialMQA uses neither: a text-only planner reads the question, and open-vocabulary detection localizes the objects. For CV-Bench and SPAR we replaced the category-based parsers with the same question-only planner. It writes the same programs as the templates on all 1,200 CV-Bench questions and on 91.7% of SPAR questions. SPAR accuracy is 78.6 / 77.2 (82.2 / 81.1 with categories), still 12.6 / 15.1 points above the matched LoRA.
  • Coverage extension. Three admission rules send more SpatialMQA questions to the unchanged executor and the frozen readers: camera-viewpoint questions and questions that name no viewpoint are executed in the camera frame, and the person-overlap threshold is lowered. The rules were fixed on train and dev, and the test set was evaluated once. Coverage rises from 35.3% to 66.8% (380 → 719 of 1,076 questions). Accuracy goes from 79.12 / 77.08 to 79.74 / 77.85 (+0.62 [−0.22, 1.42] / +0.77 [−0.19, 1.71]). So the 35% is not a ceiling of the method: coverage nearly doubles without loss. The gain is small because the VLM already answers the 339 added questions at 85.3 / 82.5; fusion still ends above that (87.3 / 85.0). The remaining 357 questions need new operators or better binding.

W2. Positioning and adaptive fusion.

  • Positioning. SpaEx adds no new perception model or geometric primitive. Its contribution is the interface: fixed typed operators compute evidence for each of the VLM's original candidates, and this evidence is added to the VLM's own candidate scores, with exact fallback. Prior systems replace the VLM's answer with a program output or with the VLM's reading of supplied states. The controls in Q1 show that this choice matters.
  • Adaptive fusion. We agree that it is not a contribution. Adaptive weights add −0.16 to +0.51 points over constant fusion (Table 2). The revision presents the two-parameter constant reader as the minimal SpaEx and removes the claim. On CV-Bench, execution and fusion predict identically; Section 5.4 reports this as a fusion boundary.
Reviewer_aDjaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SpaEx, a target-aligned spatial reasoning framework that integrates object-centered perception, explicit geometric execution, and native vision-language model (VLM) predictions. The central motivation is that providing VLMs with geometric representations does not necessarily enable them to perform the spatial computations required by a given question.This paper presents \textbf{SpaEx}, a target-aligned spatial reasoning framework that integrates object-centered perception, explicit geometric execution, and native vision-language model (VLM) predictions. The central motivation is that providing VLMs with geometric representations does not necessarily enable them to perform the spatial computations required by a given question.

Specifically, SpaEx first extracts task-relevant object states through pretrained perception tools, including object localization, segmentation, monocular geometry estimation, and pose estimation. It then identifies the target objects, reference objects, and required coordinate frames based on the question. A set of predefined geometric operators performs reference-frame projection, spatial relation evaluation, or metric distance comparison to produce evidence for individual answer candidates. Finally, a lightweight reader combines these executed candidate-level scores with the native VLM predictions. The default adaptive reader introduces four trainable scalar parameters while keeping the task-adapted VLM and perception tools frozen during calibration.

The authors evaluate SpaEx on SpatialMQA, SPAR-Bench, and CV-Bench 3D using Qwen3-VL-4B and LLaVA-OneVision-1.5-4B. The results show improvements of approximately 5.3--18.9 percentage points over matched task-adapted VLM baselines. Additional analyses examine the contributions of geometric execution, aligned-state readout, constant versus adaptive fusion, and the interaction between geometric evidence and native VLM scores.

Critical strengths
  1. A clear and interpretable spatial reasoning framework.

    SpaEx presents a well-structured framework that separates spatial perception, explicit geometric computation, and candidate-level answer selection. By identifying question-relevant objects and performing reference-frame projections or distance comparisons, the method provides a relatively transparent alternative to relying entirely on implicit geometric reasoning within VLMs. The explicit numerical operations make the intermediate reasoning process easier to inspect and verify.

  2. Consistent empirical improvements across multiple benchmarks and backbones.

    The proposed method achieves substantial improvements over matched task-adapted VLM baselines on SpatialMQA, SPAR-Bench, and CV-Bench 3D, with reported gains of 5.3–18.9 percentage points. These evaluations cover different spatial reasoning requirements, including observer-relative directional judgments, imagined reference frames, compound spatial relations, and metric distance comparisons. The improvements across two VLM backbones provide useful empirical evidence for incorporating explicit geometric information into spatial question answering.

  3. Interesting evidence of complementarity between geometric execution and native VLM predictions.

    The paper investigates how explicit geometric evidence interacts with native VLM predictions. In particular, candidate-level fusion can recover correct answers even when neither the geometric executor nor the original VLM ranks them first. This observation suggests that the two sources can provide complementary information rather than merely replacing one another. The separation of execution-only and fusion-based results also helps clarify their different contributions across spatial reasoning tasks.

  4. Informative ablation studies and practical fallback design.

    The authors provide several relevant ablations, including execution-only predictions, constant-weight fusion, aligned-state readout, alternative evidence representations, and additional-training controls. These comparisons help identify the benefits and limitations of the proposed components. Moreover, SpaEx preserves native VLM predictions when geometric execution is unsupported, preventing the system from forcing potentially invalid geometric decisions. The paper also reports execution coverage and acknowledges cases where adaptive fusion offers little or no additional benefit.

Critical weaknesses
  1. Insufficient differentiation from existing tool-augmented spatial reasoning frameworks.

    My primary concern is that the proposed method provides a relatively limited methodological contribution beyond existing tool-augmented spatial reasoning approaches. Much of the SpaEx pipeline relies on pretrained perception tools and standard geometric operations, including object localization, 3D position estimation, coordinate transformations, vector projections, and distance comparisons.

    Prior work has already explored combining specialized perception tools with explicit geometric computation. For example, SpatialAgent in SpatialScore (Wu et al., CVPR 2026) integrates multiple spatial perception tools within Plan-Execute and ReAct frameworks. SpaEx primarily differs by converting geometric measurements into candidate-level compatibility scores and combining them with native VLM scores. Although this interface is reasonable, the underlying geometric operations and weighted score fusion are relatively straightforward.

    More importantly, the main experimental baselines are primarily conventional VLM approaches, including LoRA, CoT, SoM, and VCD, rather than competitive tool-augmented spatial reasoning systems. Consequently, the reported improvements do not sufficiently establish whether SpaEx offers a substantial advantage over the most relevant prior methods. These limitations make the novelty and significance of the proposed contribution insufficiently convincing.

  2. The central comparison between aligned spatial states and executed evidence contains important confounding factors.

    A central claim of the paper is that providing executed candidate-level geometric comparisons is more effective than supplying aligned spatial states and allowing the VLM to perform the necessary computations. However, the experiments do not fully isolate the effect of explicit computation.

    In Table 3, execution with fallback substantially outperforms the trained aligned-state reader on SPAR-Bench. Nevertheless, Appendix C.6 explicitly acknowledges that the two approaches differ in more than whether geometric computation has been performed. The aligned-state interface does not provide certain task-specific compatibility conventions, including relation tolerances, compatibility penalties, and normalization rules. It also omits native VLM scores and entropy available to the numerical fusion reader.

    These differences are important because the geometric executor incorporates task-specific rules for mapping continuous measurements to discrete answer candidates. The aligned-state reader must instead infer or learn parts of these rules from limited supervision. Therefore, the performance gap may reflect differences in available information, representation format, and readout mechanisms rather than the isolated benefit of external geometric computation.

    Although the facts-versus-vectors experiments on SpatialMQA provide a more closely controlled comparison, differences in input representation, sequence length, and computational requirements remain. Thus, the experiments demonstrate the practical usefulness of executed evidence under the evaluated configurations, but do not fully establish the stronger conclusion that explicit execution is inherently superior to aligned-state reasoning.

  3. The necessity and independent contribution of the adaptive fusion mechanism are not convincingly established.

    The proposed adaptive reader combines geometric evidence with native VLM scores using weights conditioned on prediction entropy and geometric dispersion. However, the ablation results suggest that most of the reported improvements can be achieved using simpler alternatives.

    In Table 2, constant-weight fusion achieves 79.28% accuracy on SpatialMQA with Qwen, slightly exceeding the adaptive reader's 79.12%. Both methods achieve 77.08% with LLaVA. On CV-Bench 3D, execution with fallback, constant fusion, and adaptive fusion produce identical predictions for both backbones. Even on SPAR-Bench, where adaptive fusion performs better, its additional gains over constant fusion are modest compared with the improvements introduced by geometric execution.

    Furthermore, Appendix E.3 reports that an MLP reader using executed evidence achieves 82.78% and 82.27% accuracy on SPAR-Bench, exceeding the primary SpaEx reader's 82.18% and 81.10%, respectively. Although the MLP has greater trainable capacity, this result suggests that the proposed four-parameter formulation is not necessarily the most effective way to utilize executed evidence.

    Overall, these observations indicate that the dominant performance gains arise from the availability of geometric evidence rather than the proposed adaptive weighting mechanism. While the lightweight reader is computationally simple, its necessity and independent methodological contribution remain insufficiently justified.

  4. Limited generality due to predefined geometric operators, task-specific assumptions, and restricted execution coverage.

    Although SpaEx is presented as a general spatial reasoning interface, its implementation depends heavily on predefined geometric operations, benchmark-specific decision rules, and reference-frame conventions.

    For example, SpatialMQA uses estimated human head axes to construct observer-relative coordinate frames and represents objects using estimated 3D positions. Candidate relations are evaluated through projections of normalized displacement vectors. Although mathematically reasonable, this formulation assumes that natural-language spatial relations can be adequately represented through such geometric comparisons. In practice, an object can simultaneously satisfy multiple directional relations, and a single representative point may not capture the spatial extent or complex geometry of an object.

    Similarly, SPAR-Bench relies on predefined execution templates and relation-specific tolerances. These task-dependent rules raise questions about whether SpaEx provides a generalizable spatial reasoning mechanism or primarily implements specialized geometric solvers for known benchmark tasks.

    This limitation is particularly evident on SpatialMQA, where only 380 of 1,076 test questions (35.3%) support geometric execution. The remaining questions rely entirely on native VLM fallback. Although retaining all test questions is appropriate, such limited execution coverage restricts the scope of the demonstrated geometric reasoning capability.

    Moreover, the complete pipeline depends on multiple external perception tools whose computational costs and estimation errors may affect the final results. The paper does not fully characterize uncached end-to-end inference latency or robustness to inaccurate geometric measurements.

    Consequently, the current evaluations mainly demonstrate effectiveness on predefined spatial comparisons, while providing limited evidence for generalization to unseen spatial relations, more complex compositions, or broader multimodal agent settings.

Questions for authors
  1. Distinction from existing tool-augmented spatial reasoning frameworks.

    How does SpaEx fundamentally differ from existing tool-augmented spatial reasoning approaches, particularly SpatialAgent in SpatialScore (CVPR 2026) and Geometrically-Constrained Agent (CVPR 2026)? These methods have already explored the integration of specialized spatial perception tools and explicit geometric computation. Beyond converting geometric measurements into candidate-level compatibility scores and applying weighted fusion, what constitutes the primary methodological novelty of SpaEx? Could the authors provide a controlled comparison against a representative tool-augmented spatial reasoning framework under comparable perception inputs, answer-scoring protocols, and computational budgets?

  2. Fairness and validity of the aligned-state versus executed-evidence comparison.

    The paper argues that providing executed geometric evidence is more effective than supplying aligned spatial states. However, Appendix C.6 indicates that these approaches differ not only in explicit computation but also in the task conventions and information provided to the reader. In particular, the aligned-state reader lacks certain compatibility rules, normalization conventions, and native VLM scores. Could the authors provide a more strictly controlled comparison in which both approaches receive identical geometric observations, reference-frame definitions, relation tolerances, and native VLM scores, differing only in whether candidate comparisons are explicitly executed? This would help clarify the actual contribution of external geometric computation.

  3. Necessity and independent contribution of adaptive fusion.

    Table 2 shows that constant-weight fusion matches or slightly exceeds adaptive fusion on SpatialMQA, while execution-only and fusion-based methods produce identical predictions on CV-Bench 3D. Furthermore, Appendix E.3 reports that an MLP reader using executed evidence achieves higher accuracy than the primary four-parameter reader on SPAR-Bench. Given these observations, what is the essential contribution of the proposed entropy- and dispersion-dependent weighting mechanism? How do the authors distinguish improvements arising from explicit geometric execution, candidate-level score fusion, and the specific adaptive weighting formulation?

  4. Generality of the predefined geometric operators and spatial relation assumptions.

    To what extent does SpaEx provide a generalizable spatial reasoning mechanism rather than a collection of benchmark-specific geometric operators? For example, SpatialMQA represents objects using estimated 3D positions and evaluates directional relations through projections onto observer-relative axes. However, spatial relations may be ambiguous when an object simultaneously satisfies multiple directional predicates, or when object extent and observer orientation affect the interpretation. Similarly, SPAR-Bench relies on predefined execution templates and relation tolerances. How do the authors justify these geometric assumptions, and what are the main limitations when extending the framework to unseen spatial relations or more complex compositional tasks?

  5. Execution coverage, geometric reliability, and practical computational efficiency.

    On SpatialMQA, geometric execution is supported for only 380 of 1,076 test questions (35.3%), while the remaining questions rely on native VLM fallback. What are the primary factors limiting execution coverage, and how should this restricted applicability affect the interpretation of the framework's generality? Moreover, the admission checks establish the availability and numerical validity of geometric measurements but do not guarantee their perceptual correctness. How does the proposed method account for potentially inaccurate object positions, depth estimates, or reference-frame orientations when combining geometric evidence with native VLM predictions? Finally, how should the reported lightweight four-parameter reader be interpreted in terms of the computational cost of the complete, uncached perception-and-reasoning pipeline?

If the author can answer my questions and explain the weaknesses, I'm willing to raise my score.

2 comments
AuthorsOpenReview ↗

Response to Reviewer aDja (Part 1/2)

Thank you for the detailed questions and for your willingness to reconsider. Results below are new unless a table of the paper is cited; details will be in the revised appendix. Two numbers are Qwen / LLaVA; SPAR is the task macro.

Q1 (W1). SpatialAgent and the Geometrically-Constrained Agent (GCA). In both agents, generated reasoning or code produces the answer and replaces the VLM's own. SpaEx computes evidence for each original candidate with fixed typed operators and adds it to the VLM's own candidate scores, with exact fallback where execution is unsupported. We ran both agents under our protocol (original questions and candidates; accuracy in %):

SpatialMQACV-Bench 3DSPAR
SpatialAgent, Qwen3-VL-4B core, as released38.187.642.2
Matched LoRA + SpatialAgent's working58.4 / 54.684.3 / 81.361.1 / 56.7
Matched LoRA73.6 / 71.887.3 / 77.066.0 / 62.2
SpaEx79.1 / 77.194.2 / 92.582.2 / 81.1
Every 8th test question (135 / 150 / 173)
GCA, Qwen3-VL-4B controller (adapted)20.064.019.6
Matched LoRA + GCA's working57.5 / 57.888.2 / 80.962.4 / 60.1
Matched LoRA, same questions71.1 / 70.688.2 / 76.766.6 / 63.2
SpaEx, same questions74.8 / 73.693.1 / 93.681.6 / 82.2
  • SpatialAgent uses the same Qwen3-VL-4B as our Qwen backbone. SpaEx is 41.0 / 39.0, 6.7 / 4.9 and 40.0 / 38.9 points above it.
  • "+ working" holds backbone, adapter and scoring fixed: the agent's steps and tool results, without its final answer, are appended to the question, and our matched adapters score the original candidates. SpaEx is 9.9–24.4 points above SpatialAgent's working and 4.9–22.1 points above GCA's.
  • GCA setting. Its release serves a 235B controller, which we cannot run. We use the released agent with Qwen3-VL-4B as controller, a 900 s limit per question and replies capped at 2,048 tokens. This is an adapted setting, not a reproduction: the 4B controller finishes 45% / 87% / 33% of questions, so the own-answer row says little about GCA as published (90.8 on CV-Bench 3D). One question takes 280–644 s, so the subset is set by the time available. We will run more questions if you wish.
  • A tool agent on our adapted backbones with SpaEx's own tools (Grounding DINO, SAM, MoGe-2, Python; Visual Sketchpad protocol) scores 54.3 / 50.1 on SpatialMQA and 83.9 / 70.6 on CV-Bench 3D, below the matched LoRA alone.

Q2 (W2). A strictly controlled state-versus-execution comparison. We agree that Table 3 confounds computation with conventions and access to native scores. Three new controls remove this (three seeds, full test sets):

ReaderState only+ executed comparisons
Decoder-internal, SpatialMQA74.38 / 72.7479.24 / 77.32
Decoder-internal, SPAR65.53 / 62.1079.38 / 79.45
Decoder-internal, CV-Bench 3D88.56 / 76.5693.86 / 91.14
Text reader, SPAR72.03 / 70.4179.45 / 80.52
  • The internal readers share operands, candidate binding, capacity, examples, updates and loss. Execution wins all 18 model–benchmark–seed cells (+3.3 to +19.3 points).
  • Both text readers receive the aligned state, the executor's conventions, native scores and entropy in the same template, and are trained on the same 502 examples. The executed comparisons are one extra line. They add 7.4 / 10.1 points, in all six seed pairs.
  • A numerical MLP on SpatialMQA does equally well from state and from executed evidence (difference +0.12 [−0.81, 1.05] / +0.03 [−0.84, 0.99]). The advantage is therefore specific to VLM readers, and we will state the claim that way: for VLM readers, executed candidate comparisons work better than aligned states under identical observations.

(Q3–Q5 follow in Part 2/2.)

AuthorsOpenReview ↗

Response to Reviewer aDja (Part 2/2)

(Continued from Part 1/2. Two numbers are Qwen / LLaVA; SPAR is the task macro.)

Q3 (W3). Execution, fusion and adaptive weighting. Table 2 separates them (cumulative gain over the matched LoRA, in points):

+ execution with fallback+ constant fusion+ adaptive weights
SpatialMQA+0.21 / +0.34+5.46 / +4.93−0.16 / 0.00
SPAR+12.87 / +16.43+2.79 / +2.33+0.51 / +0.17
CV-Bench 3D+6.92 / +15.560 / 00 / 0

Execution drives SPAR and CV-Bench. Fusion with the VLM's scores drives SpatialMQA, where execution covers 35% of the questions. Adaptive weighting adds almost nothing, and we agree that it is not a contribution: the revision presents the two-parameter constant reader as the minimal SpaEx. A 4,545-parameter MLP on the same SpatialMQA evidence is 0.77 / 0.46 points below the four-parameter reader, so the evidence matters, not the form of the weighting.

Q4 (W4). Generality of the operators and geometric assumptions. Four primitives are shared across the benchmarks: relative displacement, frame construction and projection, distance comparison, and interval compatibility. Parsing and candidate semantics are benchmark-specific, and they can be shared as well: one question-only planner writes the same programs as the CV-Bench templates on all 1,200 questions and on 91.7% of SPAR questions (SPAR 78.6 / 77.2, still 12.6 / 15.1 points above the matched LoRA). SPAR candidates are compound relations and are scored jointly (Eq. 5). We will state two limitations: objects are represented by points, and a new relation needs a new operator. Questions outside the operator set fall back to the VLM and gain nothing.

Q5 (W4). Coverage, reliability and cost.

  • Coverage. Of the 696 unsupported SpatialMQA questions, 489 lack an actor-view plan and 82 fail the person-overlap check (App. B.1), so the limit lies mostly in admission, not in the operators. New: three admission rules, fixed on train and dev, raise coverage from 35.3% to 66.8% with frozen readers. Accuracy is 79.7 / 77.9 against 79.1 / 77.1: no loss, and a small gain, because the VLM already answers the added questions at 85.3 / 82.5.
  • Reliability. Admission checks validity only. Execution alone is 71.32% accurate on supported questions. Fusion repairs 221 / 218 errors of the matched LoRA and damages 43 / 48 of its correct answers (three seeds), because geometry overrides the VLM only when the VLM's margin is small (Eq. 7).
  • Robustness to geometric error (new). Log-normal distance noise with σ = 0.1, about the error of current monocular depth estimators, costs at most 0.9 points on CV-Bench. Rotating every SpatialMQA displacement by 60° changes 56% of the executed answers, and accuracy stays above the matched LoRA (76.2 / 74.0 against 73.6 / 71.8). Only at σ = 0.5 does the frozen Qwen reader fall below the matched LoRA (by 4.2 points); refitting its scalars on equally noisy data restores that level.
  • Cost (new). Median end-to-end time per question at batch size 1 on one A100: 4.2 s on SpatialMQA, 0.26 / 0.34 s on CV-Bench and 1.24 / 1.73 s on SPAR, with peak memory below 15 GiB. The tool agent from Q1 needs 9–17 s per question in 11 of its 12 runs.
Reviewer_GqbtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SpaEx, a target-aligned spatial reasoning framework that integrates object-centered visual perception, explicit geometric execution, and native vision-language model (VLM) predictions. Given an image, question, and candidate answers, SpaEx extracts question-relevant object states, performs predefined spatial operations such as coordinate projection and distance comparison, and combines the resulting candidate-level evidence with native VLM scores through a lightweight fusion mechanism.

The authors evaluate SpaEx on three spatial reasoning benchmarks—SpatialMQA, SPAR-Bench, and CV-Bench 3D—using Qwen3-VL-4B and LLaVA-OneVision-1.5-4B. Experimental results demonstrate consistent improvements of approximately 5.3–18.9 percentage points over matched task-adapted VLM baselines. Additional ablations investigate the contributions of explicit execution, aligned-state representations, and native prediction fusion, suggesting that executed spatial evidence can be more effective than directly supplying geometric states to VLMs.

Critical strengths
  1. Consistent empirical improvements across diverse spatial reasoning tasks.

The proposed method achieves substantial improvements across three benchmarks covering observer-relative spatial relations, imagined viewpoints, and metric distance comparisons. The improvements are consistent across both evaluated VLM backbones, demonstrating the practical effectiveness of integrating explicit geometric computation with VLM predictions. Importantly, the authors also compare against additional-training controls and alternative geometric evidence representations, providing stronger empirical support than comparisons against standard VLM baselines alone.

  1. Meaningful investigation of explicit computation and complementary visual evidence.

Beyond improving benchmark accuracy, the paper investigates whether explicitly executed spatial comparisons provide more useful information than aligned geometric states. The controlled comparisons between executed facts and aligned vectors offer interesting evidence that VLMs may struggle to reliably perform numerical spatial computations even when the necessary measurements are available. The analysis of execution-only predictions and score-level fusion also demonstrates that geometric computation and native visual judgments can provide complementary information, including cases where fusion recovers correct answers not ranked first by either individual branch.

Critical weaknesses
  1. Limited methodological novelty compared with existing spatial reasoning frameworks.

The proposed SpaEx framework primarily integrates existing pretrained perception modules, predefined geometric operations, and a lightweight score-level fusion mechanism to enhance VLM spatial reasoning. Although the experimental improvements are encouraging, the overall methodology appears largely incremental and does not introduce a fundamentally new spatial representation, reasoning mechanism, or learning paradigm.

Several prior works have already explored closely related ideas. For example, SpatialPIN (Ma et al., NeurIPS 2024) incorporates external 3D priors into VLM reasoning; APC (Lee et al., ICCV 2025) explicitly reconstructs and transforms spatial representations into target viewpoints; and VADAR (Marsili et al., CVPR 2025) integrates visual perception with executable programs for 3D spatial reasoning. SpatialReasoner (Ma et al., NeurIPS 2025) also explicitly connects 3D perception, computation, and reasoning.

Compared with these approaches, SpaEx's main distinction appears to be converting predefined geometric computations into candidate-level scores and combining them with native VLM predictions. However, the proposed adaptive fusion mechanism offers only marginal or inconsistent improvements over simple constant-weight fusion, further raising questions about its methodological contribution.

Overall, the paper does not sufficiently establish what fundamentally new spatial reasoning capability or mechanism is introduced beyond the integration of existing components.

  1. Insufficient comparison with directly relevant spatial reasoning methods.

Despite reporting consistent improvements across three benchmarks, the main experimental comparisons focus on Vanilla VLMs, LoRA, CoT, Set-of-Mark, and VCD. These baselines primarily represent general-purpose VLM inference, adaptation, prompting, or prediction modification, rather than competitive spatial reasoning systems that explicitly leverage external 3D information or executable geometric reasoning.

In particular, closely related methods such as SpatialPIN, APC, VADAR, SpatialReasoner, and SpaceTools are discussed in the related work but are not included as complete-method baselines in the main evaluation. Although the appendix provides an APC-inspired aligned-state reader, the authors acknowledge that this control does not reproduce the full APC framework and cannot establish superiority over it.

This omission is particularly important given the limited methodological novelty discussed above. Without comparisons against existing spatial reasoning frameworks under comparable evaluation settings, it remains unclear whether the reported improvements arise from SpaEx's particular design or simply from introducing additional spatial perception and geometric computation capabilities that are already available in prior methods.

Consequently, the current experiments demonstrate that explicit geometric evidence can improve standard VLM predictions, but do not adequately establish that SpaEx offers a meaningful advantage over existing spatial reasoning approaches.

Questions for authors
  1. What are the key methodological differences between SpaEx and existing spatial reasoning frameworks such as SpatialPIN, APC, and VADAR?

  2. Why are these closely related methods not included in the main experiments? Could the authors provide comparisons on compatible benchmarks?

  3. How much of the improvement comes from additional 3D perception tools rather than the proposed execution and fusion mechanisms?

  4. Given its marginal improvements over constant-weight fusion, what justifies the additional adaptive fusion mechanism?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Gqbt

Thank you for the review. Results are new unless a paper table is cited; details go in the revised appendix. Two numbers are Qwen / LLaVA; SPAR is the task macro.

W1 / Q1. Differences from SpatialPIN, APC, VADAR and SpatialReasoner.

  • SpatialPIN and APC give the VLM 3D object states or a view transformed into the reference frame. The VLM must still derive the comparison for each candidate.
  • VADAR returns the output of a generated program.
  • SpatialReasoner fine-tunes the VLM and uses no tools at inference; SpaceTools trains a tool-calling policy that emits the answer.

SpaEx differs in the interface. Fixed typed operators compute evidence for each of the VLM's original answer candidates, and this evidence is added to the VLM's own candidate scores, with exact fallback where execution is unsupported. None of these systems keeps the base VLM's candidate scores. Keeping them lets fusion recover answers that neither branch ranks first (40 per backbone on SPAR; Fig. 3). With SpaEx's observations held fixed, a text reader given the APC-style aligned state reaches 72.03 / 70.41 on SPAR, and the same reader given the executed comparisons 79.45 / 80.52 (new matched control). We agree the operators are standard geometry; the claim concerns the interface.

W2 / Q2. Comparisons with complete methods. None of them reports SpatialMQA or single-view SPAR results, so we ran them under our protocol (original questions and candidates; accuracy, %):

SystemSpatialMQACV-Bench 3DSPAR
APC as released (Qwen2.5-VL-7B)27.757.321.8
APC pipeline, our backbones and matched adapters50.0 / 40.662.8 / 65.740.2 / 39.4
SpatialReasoner-7B, released checkpoint34.279.821.9
SpaceTools-3B checkpoint, substitute open tools34.074.637.3
— its tool trace, read by our adapters73.9 / 71.888.9 / 78.465.1 / 58.8
3D-prior prompt (reduced SpatialPIN-style), Qwen3-VL-4B39.590.543.9
— its 3D object states, read by our adapters68.0 / 66.687.0 / 76.161.2 / 57.8
VADAR code, Qwen3-VL-4B for GPT-4o, every 4th question12.66.38.4
— its program trace, read by our adapters75.1 / 72.087.0 / 73.963.6 / 57.8
Matched LoRA73.6 / 71.887.3 / 77.066.0 / 62.2
SpaEx79.1 / 77.194.2 / 92.582.2 / 81.1
  • SpaEx is above every row (on VADAR's subset: 77.7 / 74.2, 94.1 / 93.8, 80.8 / 80.0).
  • Fidelity. Official APC reproduces its 3DSRBench numbers (70.5 / 65.9 against 72.78 / 66.47 reported), and the SpatialReasoner checkpoint its CV-Bench numbers (Depth 87.0 against 87.3).
  • Matched comparisons. APC as released and SpatialReasoner are zero-shot. With our backbones and matched adapters, APC is 11–31 points below the matched LoRA alone. "Read by our adapters" appends a system's tool calls, states or program trace, without its final answer, to the question and scores the original candidates with the matched LoRA adapters that SpaEx uses. SpaEx is 2.2–23.3 points above these read-outs.
  • Adapted settings. SpaceTools expects its own tool servers; VADAR and SpatialPIN call GPT-4o / GPT-4V and a 3D-reconstruction service that is no longer available. We therefore use substitute open tools for SpaceTools, Qwen3-VL-4B in VADAR's code, and a reduced SpatialPIN-style prompt without mesh reconstruction. These are not reproductions. With a 4B model writing VADAR's programs, only 14–42% of questions end with an answer, so its own-answer row says little about VADAR with GPT-4o; the read-out row is the informative one. VADAR takes 73–213 s per question, hence the subset.

Q3. Do the gains come from the tools, or from execution and fusion? We ran a tool agent (Visual Sketchpad protocol) with SpaEx's tools (Grounding DINO, SAM, MoGe-2 depth, Python), the same adapted backbones and candidate scoring, but no target-aligned execution. It scores 54.3 / 50.1 on SpatialMQA and 83.9 / 70.6 on CV-Bench 3D. That is 19.3 / 21.7 and 3.4 / 6.4 points below the matched LoRA, and 24.8 / 27.0 and 10.4 / 22.0 below SpaEx; all eight intervals exclude zero. On SPAR (one seed; a run takes 34–53 GPU-hours) it scores 55.2 / 49.6: 11.1 / 12.8 points below the matched LoRA and 26.7 / 31.5 below SpaEx on that seed. Table 9 points the same way: with the same tools, camera axes in place of the measured observer frame give 73.17 / 71.84 on SpatialMQA, and image-plane cues in place of 3D measurements 87.50 / 78.08 on CV-Bench. The tools are necessary but not sufficient.

Q4. Adaptive fusion. We agree: it is not a contribution. Adaptive weights add −0.16 to +0.51 points over constant fusion (Table 2). The gains come from execution (SPAR, CV-Bench 3D) and from constant fusion with the VLM's scores (SpatialMQA +5.46 / +4.93). The revision presents the two-parameter constant reader as the minimal SpaEx and removes the claim.

Soundness. We would gladly address any concern beyond novelty and baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.