acceptodds
Withdrawn from ICLR 2027

Disentangling 3D Modeling from Spatial Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

17 views

Abstract

In this work, we explore an alternative paradigm for spatial reasoning by explicitly disentangling 3D perception from reasoning, rather than jointly acquiring implicit 3D perception and reasoning through large-scale training. Our key observation is that modern perception models excel at estimating continuous 3D geometry, whereas large language models (LLMs) are particularly effective at compositional and symbolic reasoning. Motivated by these complementary strengths, we propose the Disentangled Spatial Reasoner (DiSR), a simple yet effective framework that reconstructs the physical world into structured 3D evidence using off-the-shelf expert perception models and fine-tunes an LLM with LoRA to perform reasoning solely over this explicit geometric evidence. Without large-scale 3D VQA training or complex tool-use policies, DiSR achieves competitive performance on popular spatial reasoning benchmarks. Beyond its strong performance, DiSR offers improved interpretability, modularity, and computational efficiency, demonstrating that explicit separation of perception and reasoning is a scalable and effective alternative paradigm to end-to-end modeling for spatial intelligence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Spatial Reasoning, Vision-Language Models, Multimodal Large Language Models, 3D Scene Understanding, Geometric Reasoning, Spatial Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_fHELOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Disentangled Spatial Reasoner (DiSR), a framework that separates 3D perception from spatial reasoning. It uses pretrained perception models to construct structured 3D geometric evidence and fine-tunes an LLM with LoRA to reason over this evidence. The approach aims to reduce the need for large-scale 3D VQA training while improving interpretability and training efficiency. Experiments on 3DSRBench, SPAR-Bench, and CV-Bench-3D demonstrate competitive spatial reasoning performance, with improvements on 3DSRBench and SPAR-Bench. The paper also provides diagnostic analyses of different pipeline components and evaluates the framework with different MLLM backbones.

Critical strengths
  1. Effective and training-efficient framework. The proposed approach achieves strong performance on multiple spatial reasoning benchmarks, particularly 3DSRBench and the metric estimation tasks in SPAR-Bench, while requiring substantially less downstream training data and reported training GPU hours than several specialized baselines. These results demonstrate the practical potential of combining pretrained 3D perception models with lightweight LLM adaptation.

  2. Interpretable and modular design. By explicitly constructing structured 3D evidence, DiSR enables a clearer separation between perception and reasoning. The ground-truth replacement experiments provide useful insights into the sources of errors, while the cross-backbone evaluation offers initial evidence of the framework's extensibility.

Critical weaknesses
  1. Insufficient ablation studies to isolate the key contributions. The paper lacks critical comparisons, such as using structured 3D evidence without LoRA fine-tuning and fine-tuning the original MLLM under comparable training conditions without the proposed evidence representation. Consequently, it is difficult to determine how much of the reported improvement comes from explicit 3D evidence, task-specific fine-tuning, or their combination.

  2. Limited methodological novelty and insufficient validation of the central claim. The proposed framework primarily integrates existing 3D perception models with an LoRA-adapted LLM through a structured textual interface. While the empirical results are promising, the experiments mainly demonstrate the effectiveness of the complete pipeline rather than establishing why disentanglement itself is preferable to alternative approaches. This weakens the evidence supporting the paper's central methodological contribution.

Questions for authors
  1. Contribution of structured 3D evidence versus LoRA fine-tuning. The paper reports substantial improvements over the original Qwen3-VL-8B, but it remains unclear how much of the gain comes from the explicit 3D evidence and how much comes from the 330K training samples. Could the authors provide controlled comparisons using (i) structured 3D evidence with the frozen, non-fine-tuned LLM and (ii) RGB-based fine-tuning under comparable training conditions? These results would help establish whether the improvement primarily comes from disentanglement or task-specific supervision.

  2. Necessity and effectiveness of the proposed disentangled reasoning framework. Since DiSR integrates existing 3D perception models with an LLM through a structured textual interface, could the authors clarify what distinguishes the proposed method from simpler modular baselines? In particular, for depth and distance estimation tasks, how does the LoRA-adapted LLM compare with directly applying deterministic geometric computations to the same reconstructed 3D evidence? Such comparisons would help clarify the specific contribution and necessity of the learned reasoning module.

  3. Training data construction and efficiency comparison. Could the authors provide more details about the composition and generation of the 330K training samples, including the proportions of question-parsing, spatial-reasoning, and synthetic data? Additionally, does the reported computational efficiency account for the cost of generating 3D evidence using the pretrained perception models? Clarifying these aspects would help assess the reproducibility and fairness of the efficiency claims.

2 comments
AuthorsOpenReview ↗

Response to Reviewer fHEL

Q1. Contribution of structured 3D evidence versus LoRA fine-tuning: can the authors compare a frozen LLM with structured evidence and RGB-based fine-tuning under comparable conditions?

Reply: Thank you for this suggestion. We compare RGB and evidence-language adaptation using the same 1.2K real-image question–answer instances. Both start from the untuned Qwen3-VL-8B and train for ten epochs with rank-128 QLoRA, learning rate 2×10−52\times10^{-5} and effective batch size 16. The photographs, questions, options and labels are identical.

The evidence branch trains language adapters with question-parsing and numerical reasoning/answer supervision. The RGB branch trains vision-encoder/merger and language adapters with only the final option letter as the target. Table R1 reports their results on the full 3DSRBench test set.

Table R1. Full 3DSRBench accuracy and training time for the 1.2K-QA comparison, with the RGB backbone as a baseline. The best accuracy is bold and the second best is underlined.

MethodAccuracy (%)Training GPU-hours
Qwen3-VL-8B (RGB)53.70—
Untuned Qwen3-VL-8B + evidence58.70‾\underline{58.70}—
RGB adaptation (1.2K QA)53.583.48
DiSR-8B-LoRA (1.2K QA)59.165.34

Evidence-language adaptation exceeds RGB adaptation by 5.58 percentage points. The untuned evidence baseline is already strong, with adaptation adding 0.47 points. Thus, the comparison supports the value of explicit 3D evidence in this small-data setting. The supervision and updated modules of the two training schemes are specified above.

Q2. What distinguishes DiSR from simpler modular baselines, and how does the learned reasoner compare with deterministic geometric computation on the same reconstructed evidence?

Reply: DiSR learns to map natural-language questions to object slots and to answer from a shared geometric representation. A deterministic solver requires a program to specify the operation and object roles. Table A9 in Appendix F of the revised manuscript compares these answering methods on all test questions in eight directly computable relations: the solver achieves 69.13%, versus 60.06% for the untuned evidence-conditioned LLM and 69.36% for DiSR-8B-LoRA. Across all twelve relations, the solver achieves 66.41%.

For the reviewer’s depth/distance examples, the CV-Bench-3D solver achieves 91.17%/90.50%, compared with 92.83%/91.67% for the submitted DiSR model. On questions with complete object centers, their predictions agree exactly. We agree that these tasks can be solved effectively by geometric programs. DiSR’s contribution is the learned question-to-evidence interface and a shared answering model across spatial tasks.

Q3. Please clarify the construction and composition of the 330K training samples, and whether the efficiency comparison includes the frozen experts’ evidence-generation cost.

Reply: Thank you. Data construction consists of real-image evidence collection, geometric expansion and two-stage training. The real-image stage produces 1.2K image–question–answer instances using the frozen perception pipeline. Each instance is converted into a question-parsing target (Q,P)(Q,P) and an evidence-conditioned reasoning target (Q,ET,A)(Q,E_T,A).

The expansion stage samples object centers, extents, sizes and orientations over ordinary, long-tail and near-boundary configurations. Geometric rules determine the answer for each sampled configuration and produce its numerical reasoning target. The sampled geometry is serialized into the same evidence format used at inference. This expands coverage without running the perception experts for every synthetic example.

The reported 0.33M corpus contains 328,157 supervision rows: 161,100 question-parsing rows and 167,057 reasoning rows. Training uses two stages of 112,157 and 216,000 rows, with one epoch per stage. The 1.2K-QA experiments in Table R1 use ten epochs for the RGB/evidence comparison.

The reported 58.75 GPU-hours cover only the two optimization stages. For the real-image collection, summed per-image processing time is 3.33 hours for the 1,350 images from which the 1.2K instances are selected.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer fHEL,

Thank you again for your constructive review.

We have responded to all the questions and concerns in above comments. As the discussion closes in a few days, we would be grateful to know whether these results resolve your concerns, or whether any specific issue still requires clarification. If the new evidence changes your assessment, we would sincerely appreciate your reflecting that in your final score. Thank you again for the constructive discussion.

Best,

Authors

Reviewer_57U1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes the Disentangled Spatial Reasoner (DiSR), an alternative framework for 3D spatial reasoning in multimodal foundation models. Rather than relying on end-to-end implicit 3D acquisition through massive 3D VQA pre-training, the authors argue for an explicit separation of perception and reasoning:

  1. Explicit 3D Perception: Off-the-shelf expert perception models reconstruct 2D observations into structured 3D physical evidence (such as 3D bounding boxes, metric depths, or point maps).

  2. Symbolic / Compositional Reasoning: An LLM is fine-tuned with LoRA to perform multi-step spatial and geometric reasoning conditioned strictly on this explicit 3D representation.

Critical strengths
  1. Decoupling perception from reasoning directly addresses a known failure mode in end-to-end MLLMs: while vision models excel at continuous geometry estimation, LLMs excel at symbolic rule-following and multi-hop logic. Combining both explicitly avoids forcing the LLM backbone to implicitly resolve both simultaneously.

  2. DiSR bypasses expensive end-to-end pre-training on millions of spatial QA pairs. The system is easily upgradable as new, stronger 3D perception foundation models emerge, without requiring retraining the entire reasoning pipeline from scratch.

Critical weaknesses

Hallucination citation in the author list:

  • Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL http://papers.nips.cc/paper_files/paper/2023/hash/ 413885e70482b95dcbeeddc1daf39177-Abstract-Conference.html.

  • "Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso de Melo, Jianwen Xie, and Alan L. Yuille. Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning. **In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzyeh Ghassemi, Nancy Chen, Ivan Vladimir Meza Ru ´ ´ız, and Arturo Loaiza-Bonilla (eds.), ** Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025b. URL http://papers.nips.cc/paper_files/paper/2025/hash/ cdd63c6c813a6ec826051b6a49f11b5e-Abstract-Conference.html."

  • Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 of Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. URL https://proceedings.mlr. press/v267/wang25er.html.

  • Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gul Varol (eds.), ¨ Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XXV, volume 15083 of Lecture Notes in Computer Science, pp. 131–147. Springer, 2024. doi: 10.1007/ 978-3-031-72698-9\ 8. URL https://doi.org/10.1007/978-3-031-72698-9_8.

Questions for authors
  1. Ablation on Perception Degradation: How robust is DiSR to noisy or missing upstream 3D predictions or when visual predicrtion make mistake? Did you evaluate DiSR under controlled synthetic noise (e.g., perturbing bounding box coordinates or depth values) to measure how well the LLM reasons under imperfect perception?

  2. Comparison Against End-to-End Specialist Baselines: How does DiSR compare in parameter count and inference latency against recently proposed end-to-end spatial specialists (e.g., SpatialVLM, Spatial-SSRL, SenseNova-SI) when evaluated on the same benchmark subsets?

  3. The main contribution is author can use off-the shief visual model to parse the scene, which is not a new toplic as agentic spatial reasoning such as [1]

[1] Marsili, Damiano, et al. "Visual agentic ai for spatial reasoning with a dynamic api." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 57U1 (1/2)

Clarification of the reference concern. The four references identified in the review are genuine publications, and their paper-author lists in our manuscript are correct. We checked each against its official proceedings page:

  • 3D-LLM (NeurIPS 2023): Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. [Official proceedings]
  • SpatialReasoner (NeurIPS 2025): Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso de Melo, Jianwen Xie, and Alan L. Yuille. [Official proceedings]
  • Orient Anything (ICML 2025): Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. [Official proceedings]
  • PointLLM (ECCV 2024): Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. [Official proceedings]

The names following “In ... (eds.)” are proceedings editors, separately identified from the paper authors by the bibliography style. These entries are not hallucinated citations or fabricated author lists. We respectfully ask that the reference concern and the associated ethics flag be reconsidered in light of these official records.

Q1. How robust is DiSR to noisy or missing upstream 3D predictions? Can controlled perception perturbations quantify its behavior under imperfect evidence?

Reply: We added a controlled experiment on all 1,200 CV-Bench-3D questions (600 Depth and 600 Distance). Before rerunning the frozen perception pipeline, we perturb GT box widths and heights as w′=w(1+σzw)w'=w(1+\sigma z_w) and h′=h(1+σzh)h'=h(1+\sigma z_h), with independent standard Gaussian draws, fixed centers, and symmetric size constraints at image boundaries. Each noise level uses three seeds. Parsing plans, expert prompts, evidence format, the original DiSR-8B-LoRA model and decoding are fixed. All questions remain in the denominator, including perception failures.

Table R1. Accuracy (%) under controlled box-size perturbations. Noise rows report mean ± sample standard deviation over three seeds. The best and second-best mean accuracies in each column are bold and underlined.

Box conditionDepthDistanceOverall
GT boxes, no perturbation93.6794.5094.08
GT boxes, σ=0.05\sigma=0.0592.78 ± 0.1093.72‾\underline{93.72} ± 0.5993.25 ± 0.25
GT boxes, σ=0.10\sigma=0.1093.17 ± 0.6093.72‾\underline{93.72} ± 0.1993.44‾\underline{93.44} ± 0.34
GT boxes, σ=0.20\sigma=0.2093.39‾\underline{93.39} ± 0.5193.22 ± 0.5493.31 ± 0.13
GT boxes, σ=0.30\sigma=0.3091.94 ± 0.3590.00 ± 0.4490.97 ± 0.19

Overall accuracy stays within 0.83 percentage points of the clean GT-box reference at σ=0.05\sigma=0.05–0.200.20. At σ=0.30\sigma=0.30, it falls by 3.11 points to 90.97%, with Distance more affected than Depth. Thus, moderate box-size errors have a limited effect in this setting, while stronger corruption degrades the reconstructed evidence and final answers. The experiment measures box-size errors propagated through perception; direct depth corruption and missing-field interventions are outside its scope.

The clean reference in Table R1 is rerun under this fixed-parsing protocol. The corresponding correction to the stale GT-grounding row in submission Table 6 is 93.33% Depth and 94.33% Distance for the original evaluation protocol, consistent with the final archived results and Appendix Table A5’s Depth breakdown.

AuthorsOpenReview ↗

Response to Reviewer 57U1 (2/2)

Q2. How does DiSR compare in parameter count and inference latency with end-to-end spatial specialists such as SpatialVLM, Spatial-SSRL and SenseNova-SI on the same benchmark items?

Reply: We evaluated the released Spatial-SSRL-7B and SenseNova-SI-1.3-InternVL3-8B on all 11,686 3DSRBench questions. Table R2 compares them with both DiSR variants. DiSR-7B-LoRA and DiSR-8B-LoRA achieve 65.52% and 67.62% accuracy, respectively. DiSR-8B-LoRA exceeds Spatial-SSRL by 15.26 percentage points and SenseNova-SI by 5.45 points.

Table R2. Performance and training cost on 3DSRBench. Learnable parameters count those optimized during adaptation. Training time is measured in GPU-hours.

ModelLearnable params (B)Training time (GPU-hours)Training hardwareAccuracy (%)Latency (s)
Spatial-SSRL-7B8.292NRNR52.362.48
SenseNova-SI-1.3-InternVL3-8B7.640‡7.640^{\ddagger}≈9,216†\approx9,216^{\dagger}128× GPUs$^{\dagger}$62.170.36
DiSR-7B-LoRA0.38153.561× RTX 4080 SUPER65.52‾\underline{65.52}11.79
DiSR-8B-LoRA0.34958.751× RTX 4080 SUPER67.6212.36

NR: not reported or unavailable. Learnable counts are computed from the released training configurations: all parameters in Spatial-SSRL’s GRPO stage; language model and connector in SenseNova-SI’s InternVL3 recipe. ‡\ddagger The latter is the public 8M-data recipe; the 1.3 release does not separately specify its training configuration. †\dagger SenseNova-SI’s paper reports 128 GPUs for about three days on its 8M-data setup (128×72≈9,216128\times72\approx9,216 GPU-hours); GPU type and the evaluated 14M-data release’s cost are not reported.

DiSR trades additional inference computation for low training cost. Its explicit parsing, grounding and 3D reconstruction produce higher accuracy with 0.33M supervision rows. The 7B and 8B variants optimize only 0.381B and 0.349B adapter parameters, requiring 53.56 and 58.75 GPU-hours, respectively, on a single RTX 4080 SUPER; the backbone and perception experts remain frozen. The adapters are merged into the backbone at inference. Real-image evidence preparation takes a further summed 3.33 processing hours, and subsequent geometric expansion requires no additional expert inference.

The Spatial-SSRL paper reports 81,053 QA pairs and SFT followed by GRPO, without reporting training GPU-hours. The SenseNova-SI paper reports an 8M-data training setup using 128 GPUs for approximately three days. Its official release summary lists 14M spatial training examples for the evaluated 1.3-InternVL3-8B checkpoint. DiSR achieves higher accuracy with adaptation on one consumer GPU and substantially less spatial supervision than SenseNova-SI.

Q3. How does the contribution differ from existing agentic spatial-reasoning systems such as VADAR?

Reply: We thank the reviewer for highlighting this related work. Both VADAR and DiSR leverage visual specialists, but their central contributions differ. VADAR focuses on agentic program synthesis: LLM agents dynamically extend a Python API and compose specialist calls to solve spatial queries. DiSR focuses on explicitly disentangling 3D evidence construction from reasoning over that evidence.

Specifically, DiSR first uses frozen perception experts to construct object-level geometric evidence, including centers, extents, and orientation axes in a shared 3D coordinate frame. This evidence is then serialized and provided to a LoRA-adapted LLM, which answers the question solely from the explicit geometry, without accessing images or generating executable reasoning programs. Thus, perception determines the geometric evidence, while the reasoner learns how to interpret and combine it.

Our contribution lies in this explicit separation and its empirical validation. The intermediate evidence provides an inspectable boundary between perception and reasoning, enabling controlled evaluation of their respective errors through ground-truth substitution (Table 6 in the revised manuscript). DiSR therefore investigates the effectiveness of learning spatial reasoning over explicit 3D evidence, complementing VADAR’s emphasis on dynamic tool composition.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer 57U1,

Thank you again for your constructive review.

We have responded to all the questions and concerns in above comments. As the discussion closes in a few days, we would be grateful to know whether these results resolve your concerns, or whether any specific issue still requires clarification. If the new evidence changes your assessment, we would sincerely appreciate your reflecting that in your final score. Thank you again for the constructive discussion.

Best,

Authors

Reviewer_v7ixOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DiSR separates single-image spatial perception from reasoning. A question parser and grounding model identify relevant objects; frozen experts construct 3D geometric records; and a LoRA-adapted LLM answers from their textual serialization without the image (Fig. 2). Evaluation covers spatial accuracy, retained general capabilities, and diagnosis through ground-truth intermediate replacements.

Critical strengths
  1. Explicit geometry is a useful design choice for combining object relations. Figure 2 and Eq. 5 show object centres, extents, and orientation axes in a shared frame. Table 1 reports 67.62 overall on 3DSRBench, including 63.30 on multi-object relations versus 51.20 for HiSpatial and 51.80 for SpatialReasoner.

  2. Lightweight adaptation is practically attractive. The reported training uses 0.33M instances and approximately 59 hours on one RTX 4080 (Table 1; Sec. 4.1). Table 4 shows retained general VQA performance, and Table 7 demonstrates the design with two backbones.

Critical weaknesses
  1. The central interface is not specified well enough to interpret what the reasoner learns. Eq. 5 and Fig. 2 identify the geometric fields, but Sec. 3.3 defines serialization only abstractly. The shared gravity-aligned frame is stated, but a numerical example, complete axis/unit conventions, and definitions of the two size descriptors are missing. Sec. 3.4 identifies parsing and reasoning supervision, yet leaves the construction of evidence/answer pairs and synthetic-data coverage unclear.

  2. The system-level evidence does not isolate the stronger reasoning interpretation. Section 3.3 (lines 256–258) motivates learning reasoning patterns over explicit geometry, but Tables 1–3 change both the available evidence and the answering system. Table 2 reports strong metric estimation but 26.37 on ObjRel-OO versus 61.26 for the image-based backbone. The authors acknowledge limited training coverage (p.7, lines 364–367); this is plausible, but the result alone cannot distinguish coverage, geometric error, or interface mismatch. Table 6 establishes reliable use of accurate evidence on two CV-Bench tasks, not broad compositional transfer. Separation itself also has precedents: NS-VQA (Yi et al., NeurIPS 2018) reasons over an explicit scene representation using program execution.

  3. The headline evaluation scope needs clarification. Table 2 covers six selected single-view tasks, whereas SPAR-Bench contains twenty and reports a SPAR-mix-trained reference model (Sec. 3.3; Table 4). Different training regimes can justify separate comparisons, but the scope of the state-of-the-art claim should be explicit.

Questions for authors
  1. Could you walk through one existing training example from numerical geometry to the exact serialized input and answer target, including how that evidence and target were produced?

  2. How should we understand the learned reasoner’s contribution within the useful overall system? Please clarify the scope of the compositional-reasoning claim in light of the acknowledged SPAR task-coverage gap and existing diagnostics.

  3. Could you clarify the evaluation regime for the starred model, including the adaptation examples' source, separation from evaluation items, and counting convention, and distinguish that regime from the original model when stating the SPAR result?

3 comments
AuthorsOpenReview ↗

Response to Reviewer v7ix (1/2)

Q1. Could you walk through one existing training example from numerical geometry to the exact serialized input and answer target, including how that evidence and target were produced?

Reply: From image to training target. Figure 3 in the revised manuscript shows how training samples are constructed from images. For the real-image seeds, COCO category annotations supply object names, and Qwen predicts their grounding boxes before the frozen geometry pipeline. A task template is filled with the selected objects; geometric rules then produce the answer and reasoning target. For the illustrated sofa–TV pair, the center difference is Δ=[1.841,0.778,−1.010]\Delta=[1.841,0.778,-1.010], and its projection onto the sofa’s left axis is −1.876-1.876. The negative projection yields “right” and option B. Each QA provides a question-to-object-slots parsing target and a question-plus-evidence-to-reasoning-and-answer target. The 1.2K real-image QA seeds are then extended with sampled geometry, using the same evidence fields and target-generation rules, to form the combined 0.33M training corpus.

3D evidence fields. Centers are means of the filtered object point clouds; extent_xyz gives their axis-aligned spans. Lengths are in meters, scalarsize=max⁡(ex,ey,ez)\mathrm{scalar_size}=\max(e_x,e_y,e_z), and lateralradius=12ex2+ez2\mathrm{lateral_radius}=\frac{1}{2}\sqrt{e_x^2+e_z^2}. Front and left axes are unit vectors in the same gravity-aligned frame, with positive YY upward. Before gravity alignment, positive XX points image-left and positive ZZ points forward. Real-image training records are translated to a camera-centered frame before target generation; this translation leaves relative center differences and the illustrated left/right projection unchanged.

An existing training example. The record below is from the frozen training file reasoner_sft_camera_depth.jsonl (index 922). Its geometry is sampled: the two camera ranges are 2.386796 and 2.439845 m, with centers [−0.606541,0.388042,2.275593][-0.606541,0.388042,2.275593] and [−2.287573,0.366421,0.765237][-2.287573,0.366421,0.765237]. The generator serializes each object’s center, extent, size and orientation, and computes the answer and reasoning target from that same geometry. The smaller Euclidean range selects “blue chair.” Target distances are computed before the evidence is rounded to three decimals. This training record uses a camera-centered frame.

The exact user message and assistant target are reproduced below; only display wrapping is added. The construction metadata and task-family label are excluded from the model input.

Exact user message

Question: Between blue chair and the lamp, which is nearer to the camera?

Evidence document:
World frame:
- camera: origin of the 3D coordinate frame
- global up: positive Y axis
Target objects:
- q1 (blue chair): center_3d=[-0.607, 0.388, 2.276]; extent_xyz=[0.337, 2.729, 3.015]; scalar_size=3.015; lateral_radius=1.517; front_axis=[0.000, 0.000, 1.000]; left_axis=[1.000, 0.000, 0.000]
- q2 (the lamp): center_3d=[-2.288, 0.366, 0.765]; extent_xyz=[1.046, 0.792, 0.380]; scalar_size=1.046; lateral_radius=0.556; front_axis=[0.000, 0.000, 1.000]; left_axis=[1.000, 0.000, 0.000]

Exact assistant training target

<thinking>This is a camera-depth comparison. The evidence gives objects.q1.geometry.center_3d = [-0.607, 0.388, 2.276] and objects.q2.geometry.center_3d = [-2.288, 0.366, 0.765], while world.camera_center_3d is [0.000, 0.000, 0.000]. I compute the camera distances as ||center(q1)-camera|| = 2.387 and ||center(q2)-camera|| = 2.440. Because smaller distance means closer to the camera, I select blue chair. The final answer is blue chair.</thinking><answer>blue chair</answer>

Synthetic coverage. Starting from the real-image QA seeds, geometric sampling expands the training supervision. It varies object mentions, positions, extents, orientations, question wording and option order, including small relation margins and boundary cases. Evidence and targets are generated together from the sampled scene.

AuthorsOpenReview ↗

Response to Reviewer v7ix (2/2)

Q2. How should we understand the learned reasoner’s contribution, particularly the ObjRel-OO gap and the scope of the compositional-reasoning claim?

Reply: We paired all 364 ObjRel-OO outputs from the original and adapted models. Accuracy rises from 26.37% (96 correct) to 68.96% (251 correct): adaptation corrects 181 previously wrong answers and changes 26 previously correct answers to wrong ones.

The largest observed change is in choosing the option that combines the two objects’ left/right, above/below, and closer/farther relations as seen by the observer. Agreement between the selected option and the label increases from 50.00% to 86.26% for left/right, from 48.63% to 79.67% for above/below, and from 49.45% to 79.67% for farther/closer. Object-slot counts are correct on all 364 questions in both runs; complete-center availability changes only from 362 to 363 questions. In 172 of the 181 corrected cases, the ordering of the two centers along all three coordinate axes is preserved across the runs.

Figure A7 in Appendix H of the revised manuscript shows the benchmark question spar_six_001858, comparing the toilet (red box) with the bar (blue box). In this example, both runs place the toilet at X≈0.353X\approx0.353 and the bar at X≈−0.174X\approx-0.174 in the frame where positive X points left. The original model selects “right, below, closer”; the adapted model selects the correct “left, below, closer”. The targeted training supplies explicit observer-frame comparisons and teaches how to match the combined horizontal, vertical and depth cues to an option. This addresses a gap in task coverage beyond the individual geometric operations in the original supervision.

These results support improved relational selection after task adaptation. They compare complete pipeline runs, whose numerical evidence can differ. Tables R1–R2 provide the complementary untuned-LLM and deterministic-solver comparisons. Here, “compositional reasoning” means combining object-level geometric quantities within the evaluated tasks, such as a center difference followed by a local-axis projection in Figure 3 in the revised manuscript. The supported scope is composition of these quantities on the studied tasks.

Relation to NS-VQA. NS-VQA is an important precedent for separating perception and reasoning, but the evaluated settings differ in two respects. Task scope. DiSR works on open-world images and covers metric distances, object orientations, and camera- or object-dependent reference frames, including multi-object relations not represented in CLEVR’s evaluated setup. Reasoning interface. NS-VQA maps each question to an explicit symbolic program. DiSR instead supplies continuous 3D evidence from frozen experts and trains a language model to combine quantities such as center differences, distances, and local-axis projections. The same learned interface can compose these quantities across task types without a separate hand-written rule for every question. Table R2 separates what direct geometric execution solves from the additional cases handled by the adapted reasoner.

Q3. Please clarify the starred model’s adaptation data, separation from evaluation, counting convention and relationship to the original SPAR result.

Reply: The starred model starts from the frozen Epoch-2 DiSR-8B-LoRA checkpoint and trains an additional LoRA adapter on 2,816 supervision rows for ten epochs. Half cover Depth-OC and ObjRel-OO; the other half replay the original spatial tasks. The set contains 2,560 reasoning rows and 256 parsing rows.

Task-focused adaptation. The added supervision follows the ObjRel-OO question format, with varied geometric configurations targeting relational selection. The training corpus contains the original 0.33M examples plus 2,816 adaptation rows. These rows are presented for ten epochs (28,160 presentations); Table 1’s “0.36M” reflects the training count, not the number of distinct examples.

The unstarred model reports the original training regime, and the starred model reports the additional task-adaptation regime. Table 2 compares six selected single-view SPAR tasks, rather than all 20 SPAR-Bench tasks. Our performance claim is restricted to this six-task setting, with the two training regimes labeled separately.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer v7ix,

Thank you again for your constructive review.

We have responded to all the questions and concerns in above comments. As the discussion closes in a few days, we would be grateful to know whether these results resolve your concerns, or whether any specific issue still requires clarification. If the new evidence changes your assessment, we would sincerely appreciate your reflecting that in your final score. Thank you again for the constructive discussion.

Best,

Authors

Reviewer_SDc4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a framework, DiSR, which decouples 3D perception from spatial reasoning. A VLM parses the question into object slots and grounds referenced objects; frozen experts lift those regions into a shared gravity-aligned frame and produce per-object evidence (center, extent, size descriptors, front/left axes). A LLM with LoRA performs reasoning using the evidence to generate answer. Since it needs no images while training, it achieves higher training efficiency. Experiments were conducted on three datasets (3DSRBench, SPAR-Bench, CV-Bench-3D), with corresponding failure analyses and improvement measures provided.

Critical strengths
  1. Competitive results with modest additional training. With approximately 0.33M samples and 59 RTX 4080 SUPER GPU-hours, DiSR achieves 67.62% on 3DSRBench. This supports the practical value of reusing frozen perception experts and training an evidence-based reasoning model.

  2. The explicit evidence interface enables useful error diagnosis. Ground-truth object boxes improve performance, while ground-truth 3D evidence yields 100% on both CV-Bench-3D tasks (Table 6). These interventions help identify perception bottlenecks.

Critical weaknesses
  1. Missing controls with identical evidence leave the source of improvement unresolved. The paper does not compare an untuned LLM, the LoRA reasoner, and a deterministic geometric solver where applicable using the same predicted 3D evidence. Consequently, the contributions of perception experts, evidence representation, and reasoning training remain unclear, weakening the central methodological claim.
Questions for authors
  1. With parsing, grounding, and predicted 3D evidence fixed, how much does LoRA improve over the untuned LLM? How does a deterministic solver perform on directly computable tasks? This would establish whether reasoning training provides an independent benefit.
  2. Do the reported 59 GPU-hours include training-evidence generation (frozen experts' inference time)?
2 comments
AuthorsOpenReview ↗

Response to Reviewer SDc4

W1. With parsing, grounding, and predicted 3D evidence fixed, how much does LoRA improve over the untuned LLM? How does a deterministic solver perform on directly computable tasks?

Reply: Thank you for suggesting this comparison. On the full 3DSRBench test set, DiSR-8B-LoRA achieves 67.62% accuracy, compared with 58.70% for the untuned Qwen3-VL-8B supplied with structured 3D evidence: an improvement of 8.92 percentage points. The LoRA-trained model performs substantially better than the untuned evidence-conditioned LLM, supporting the benefit of fine-tuning for reasoning over 3D evidence.

Table R1. Accuracy (%) on the full 3DSRBench test set. The RGB and DiSR results are from submission Table 1; the untuned LLM is evaluated with structured 3D evidence.

MethodOverallHeightLocationOrientationMulti-Object
Qwen3-VL-8B (RGB)53.7048.0071.00‾\underline{71.00}40.5046.70
Untuned Qwen3-VL-8B + evidence58.70‾\underline{58.70}63.3368.2354.15‾\underline{54.15}50.12‾\underline{50.12}
DiSR-8B-LoRA67.6262.75‾\underline{62.75}72.8569.4063.30

We compare all twelve 3DSRBench relations in Table R2, separating the eight relations directly handled by geometric operators from four qualitative relations that require additional decision rules. All evidence-conditioned methods use the same types of serialized centers, extents and orientation axes.

Table R2. Accuracy (%) on all 11,686 3DSRBench questions.

CategoryRelationQwen3-VL-8B (RGB)Qwen3-VL-8B (Untuned + Evidence)Geometry Solver + EvidenceDiSR-8B-LoRA
(a) Direct geometric relations
HeightHigher48.0063.3362.3262.75‾\underline{62.75}
LocationCloser to camera86.4080.3881.12‾\underline{81.12}80.68
OrientationIn front of57.8074.71‾\underline{74.71}75.2975.29
On the left29.5073.9375.07‾\underline{75.07}77.08
Viewpoint34.4013.4157.1455.69‾\underline{55.69}
Multi-ObjectCloser to61.4074.8675.43‾\underline{75.43}77.14
Facing65.0061.8567.92‾\underline{67.92}70.81
Viewpoint toward object13.1014.2953.6450.73‾\underline{50.73}
Overall8 relations52.9260.0669.13‾\underline{69.13}69.36
(b) Qualitative relations
LocationAbove61.70‾\underline{61.70}56.79—70.38
Next to59.0067.26—62.24‾\underline{62.24}
Multi-ObjectParallel46.00‾\underline{46.00}45.13—57.82
Same direction47.4053.78‾\underline{53.78}—59.59
Overall4 relations55.1955.95‾\underline{55.95}—64.12
Full test12 relations53.7058.70‾\underline{58.70}—67.62

On the eight directly computable relations, DiSR performs comparably to the geometric solver (69.36% vs. 69.13%). DiSR also answers the four qualitative relations through the same evidence interface, achieving 64.12% accuracy compared with 55.95% for the untuned evidence-conditioned LLM. It improves Above, Parallel and Same direction, while the untuned LLM remains stronger on Next to. These results show that the learned reasoner extends coverage beyond direct geometric operators without requiring a separate hand-written decision rule for each qualitative relation.

W2. Do the reported 59 GPU-hours include training-evidence generation (frozen experts’ inference time)?

Reply: The reported 59 GPU-hours refer to LoRA optimization: 19.50 hours for Stage 1 and 39.25 hours for Stage 2, totaling 58.75 hours. Frozen-expert inference is excluded.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer SDc4,

Thank you again for your constructive review.

We have responded to all the questions and concerns in above comments. As the discussion closes in a few days, we would be grateful to know whether these results resolve your concerns, or whether any specific issue still requires clarification. If the new evidence changes your assessment, we would sincerely appreciate your reflecting that in your final score. Thank you again for the constructive discussion.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.