acceptodds
Under review as a conference paper at ICLR 2027

Reconstruction as Spatial Evaluation: Diagnosing Cross-View Spatial Intelligence in Large Vision-Language Models

Abstract

Spatial intelligence fundamentally requires a model to combine local visual observations into a geometrically consistent representation of the physical world. Many static benchmarks assess isolated spatial judgments, while interactive embodied evaluations combine perception with exploration, planning, and control. To bridge this gap, we introduce RECONSPATIAL, a benchmark centered on multi-view 3D scene reconstruction across indoor, tabletop, and outdoor environments. Given real multi-view images and target-object queries, large vision- language models (LVLMs) jointly predict oriented 3D bounding boxes in a shared, camera-anchored metric coordinate frame. Reconstructions are evaluated along two complementary axes: Structure for relative layout and Metric for physical scale and camera-anchored placement. Nine atomic spatial QA tasks, grouped into Geometric Perception and Spatial Relations, probe spatial skills involved in joint reconstruction. Offline navigation extends evaluation to goal-directed planning and provides a setting to examine reconstructed maps as intermediate reasoning representations. Evaluation of 18 frontier LVLMs reveals a consistent gap between relative scene structure and metric reconstruction; even GPT-6-Astra scores 81.4% on Structure but only 58.7% on Metric. Diagnostic analyses link atomic spatial skills to reconstruction quality, while showing that relational consistency alone does not ensure metric accuracy. Explicit 3D maps provide an intermediate representation that separates scene construction from path generation, revealing how geometric errors limit navigation success. Finally, spatial fine-tuning yields broad transfer across both spatial and general vision-language benchmarks.

Primary area:
datasets and benchmarks
Keywords:
Large Vision-Language Models, Spatial Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KjroOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a benchmark for vision-language models that evaluates their ability to reconstruct 3D scenes from 2D images. In the base form, the model is provided with a series of images of a single scene, taken from multiple viewpoints, from which it has to reconstruct the scene as a set of 3D bounding boxes. To further evaluate the spatial understanding the authors introduce a series of question answering tasks to isolate constituent capabilities needed to fully reconstruct the scene (such as camera pose estimation or relative depth prediction). Finally, a navigation task evaluates spatial planning capabilities. The authors provide baseline evaluation on a vast collection of state of the art models, both open and proprietary, and also show a simple fine-tuning baseline.

Critical strengths
  • the problem of 3D scene reconstruction from (multiple) 2D views is an important issue. Until recently it was mostly addressed with dedicated method, but is currently in many cases solvable with general purpose MLLM/VLM models with high accuracy. Thus creating VLM specific benchmarks for this task is indeed needed.
  • the proposed benchmark (or rather set of benchmarks) is designed to provide a thorough and extensive evaluation of VLM capabilities for 3D scene reconstruction and understanding, from the base image to bounding box prediction task, through question answering to offline navigation scenarios. This allows not only to test if the reconstruction itself can be done by the model, but also evaluates wider understanding of the spatial information extracted from the images.
  • the "structure" and "metric" metrics separate the relative layout quality of the reconstruction from physical scale prediction, which is a useful distinction to better understand the VLM predictions.
  • provided evaluation is extensive, main results provide a vast collection of baseline methods, while additional experiments and qualitative examples allow to better understand the capabilities of the current models. The results are consistent with overall expectations.
Critical weaknesses
  • the main claim of scene reconstruction is largely overstated - what the benchmark actually evaluates is only high-level layout estimation, with only a very coarse bounding box representation as the output, and even object identities are provided to the model as input. There is no object recognition, no surface geometry, texture or other visual appearance evaluation. I would rather describe the task as key object-level 3D layout reconstruction than the standard 3D scene reconstruction, which in the wider embodied AI/robotics community usually means 2D images -> 3D point cloud/mesh/NeRF/GS task. This is also true for the navigation task, which once again is a simplified 2D map navigation scenario. This overstatement needs to be addressed before the paper is accepted.
  • at the time of the review, the benchmark data and code is not available - I am not able to ascertain the soundness of the evaluation at the time. At the same time Appendix C mentions a rather implementation-specific details like json file format, which is rather odd for a no-code submission (LLM text writing artefacts?).
  • the benchmark doesn't seem to isolate the impact of the multi-view information - the presented tasks, at least according to the qualitative samples in the paper can largely be solved with only a single view. It's not clear if the multi-view information is actually combined by the LLM effectively.
  • fine-tuning is limited to only SFT, while spatial tasks often benefit from reasoning capabilities of the model, that require RL-based fine-tuning such as GRPO to train.
  • limited number of test cases per category, most notably the offline navigation is only 346 cases, which is quite low for a static benchmark.
  • no human performance baseline, or human annotation verification protocol mentioned.
Questions for authors
  • What is the number of unique physical scenes used to create the benchmark (not test cases or images, the physical rooms/scenes)?
  • Was the full benchmark suite or a subset verified by a human? Can you provide a human baseline for at least the questions answering part as a reference point?
  • Can you provide an experiment showing if the multi-view information is genuinely used by the models (e.g. single-view vs multi-view or number of images/performance plot, with the most relevant image as the main single view)
  • Can you provide the benchmark data and code for review?
7 comments
AuthorsOpenReview ↗

Response to the Reviewer Kjro - Part 1

W1. Object-level reconstruction and offline navigation

We apologize for the overly broad task description. We agree that our terminology did not clearly convey the object-level scope. To clarify the scope, ReconSpatial evaluates joint object-level 3D layout reconstruction and offline, open-loop 2D ground-plane navigation. Oriented boxes encode object centers, extents, and orientations. Object identities are supplied by design to decouple object recognition from 3D spatial reasoning, allowing direct evaluation of joint geometric consistency and camera-anchored placement.

W5 / Q1. Scene diversity and evaluation size

ReconSpatial includes 2,574 evaluation instances across 493 source scenes/captures:

Evaluation groupTask instancesScenes/captures
Geometric Perception739336
Spatial Relations649153
Reconstruction840192
Offline Navigation34651
Union2,574493

Reconstruction spans 71 ScanNet scenes, 98 CA-1M captures, 19 tabletop scenes/sequences, and four nuScenes driving sequences covering 126 outdoor keyframes. Navigation covers 51 ScanNet scenes, with 85.8% of episodes requiring obstacle detours and reference routes of 1.20–11.02 m. Together, these evaluations cover indoor, tabletop, and outdoor environments at different physical scales, alongside varied obstacle-avoidance demands.

W6 / Q2. Human verification and human reference

Human verification. The full benchmark suite underwent manual quality review: each question was checked against its corresponding ground truth to assess the correctness and consistency of the task and reference answer.

Human reference. On a balanced sample of 400 MCQs across eight atomic tasks, excluding metric-depth estimation because precisely estimating numerical depths in metres from images is difficult for humans, human accuracy reaches 87.75% overall. We compare GP/SR scores with the top two models on the overall leaderboard:

ReferenceGP (MCQs)SR
Human87.33%88.00%
GPT-6-Astra82.58%62.44%
Qwen3.8-Max71.16%58.80%

Model values are full-benchmark averages over the corresponding MCQ task types. The absolute accuracy gaps are 4.75–16.17% on GP and 25.56–29.20% on SR in favor of the human reference.

W3 / Q3. Cross-view coverage and single-view control

We thank the reviewer for this insightful suggestion!

Cross-view target coverage. To assess whether one view can cover the full target set, we audited all reconstruction instances. At least 81.2% (682/840) require multiple views for complete coverage under the construction criteria, as queried objects are distributed across frames (Figures 5 and 10). Tabletop scenes occupy a compact spatial extent, so one view can cover all targets; other viewpoints provide complementary depth and geometric cues.

Additional geometric information. To isolate the geometric benefit of extra views at fixed target coverage, we compared the best single view against all provided views across indoor, tabletop, and outdoor domains, evaluating only the object subset already visible in the selected single view, with identical queries and reference coordinates. Multi-view minus single-view gains are:

ModelΔStructure (%)ΔMetric (%)
GPT-6-Astra+0.82+3.38
GPT-5.6-Sol+0.97+0.02
Gemini-3.8-Flash+0.57+2.63

Even for targets already visible in a single view, additional viewpoints improve both Structure and Metric scores across three representative models (+0.02 to +3.38%). Together, these analyses confirm the need for multi-view target coverage in most instances and the geometric benefit of additional viewpoints.

W2 / Q4. Review access to data and code

We thank the reviewer for raising this concern. Public release of the benchmark data and evaluation code is undergoing institutional approval. For review, Appendix B.5 provides 12 complete task examples covering all nine atomic QA tasks, reconstruction, and navigation, with visual inputs, exact prompts, and reference answers; Appendix I provides prediction-versus-GT comparisons. Appendix C documents the output interface and scoring rules to support reproducibility ahead of code release. We are committed to releasing the benchmark data and evaluation code upon approval.

AuthorsOpenReview ↗

Response to the Reviewer Kjro - Part 2

W4. SFT and RL-based fine-tuning

We appreciate this helpful suggestion!

To evaluate RL-based fine-tuning, we extended SFT-50K with Group Sequence Policy Optimization (GSPO) [1], a group-relative RL method related to GRPO, using 2,000 examples (one epoch, 64 queries per batch, eight responses per query). Rewards use binary correctness for MCQs, mean relative accuracy for numerical QA, and R=(Structure+Metric)/2∈[0,1]R=(\mathrm{Structure}+\mathrm{Metric})/2 \in [0,1] for reconstruction, with reward RR if R>0.5R>0.5 and zero otherwise.

ModelSpatial Avg.General Avg.GPSRReconstructionNav.
Base60.9872.0938.4420.0331.008.10
SFT-50K65.1374.5854.2243.3740.6710.40
SFT-50K + GSPO-2K67.1975.7855.5346.2144.3312.14

Across the same external evaluations, GSPO improves all 10 spatial benchmarks, 7/8 general benchmarks, and all four ReconSpatial task groups over SFT, extending the transfer study with an RL-based baseline. The training setup and results are included in Appendix H.1 (Table 18).

[1] Group Sequence Policy Optimization, arXiv 2025

I thank the authors for providing most of the requested information. The scene diversity clarification, human verification and human baseline, cross-view coverage and cross-view control, as well as GSPO baseline are all important results, supporting the claims. On the negative side, the access to data is pending "institutional approval", therefore I still cannot inspect the data myself. I am leaning towards increasing my score by one, I'll await other reviewers’ responses and make my final decision then. Once again, I thank the authors fot their efforts.

Reviewer_zeecOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ReconSpatial, a benchmark for evaluating how well vision-language models reconstruct 3D scenes from multiple images. Given images from different viewpoints and queries identifying specific objects, models predict the objects’ 3D positions, sizes, and orientations in a shared coordinate frame. The benchmark includes spatial questions, scene reconstruction across indoor, tabletop, and outdoor settings, and offline navigation. It evaluates relative scene structure separately from physical scale and placement. The authors test 18 models and use additional experiments to investigate reconstruction errors, the usefulness of 3D maps for navigation, and the benefits of spatial fine-tuning.

Critical strengths
  1. I find the motivation for evaluating joint scene reconstruction convincing. Answering individual spatial questions correctly does not necessarily mean that a model can recover a consistent arrangement of several objects. The proposed task makes this distinction directly observable, while providing object regions reduces the influence of detection errors. Covering indoor, tabletop, and outdoor scenes also makes the evaluation more informative than a study restricted to one physical setting.
  2. The paper makes a substantial effort to understand the observed failures. Beyond comparing models, the authors examine geometric information, additional views, size anchors, and the use of reconstructed maps for navigation. I particularly appreciate the paired comparisons and scene-cluster bootstrap intervals, which make the empirical findings easier to assess. The large navigation gains from ground-truth maps provide useful evidence that accurate scene information matters, even though I have reservations about the stronger interpretation discussed below.
Critical weaknesses
  1. I am concerned that the reconstruction scores do not fully reflect the stated objective of recovering all queried objects. Appendix C states that, once at least three objects are matched, the geometric components are computed on the matched subset, while coverage does not enter either headline score. This means that a prediction omitting difficult objects is not directly penalized for those omissions and could potentially score better than a more complete reconstruction. I cannot determine from the reported results whether this materially affects the model comparisons, so I would like to see coverage statistics and a sensitivity analysis that explicitly penalizes missing targets. Since joint reconstruction is the central contribution, this is my most consequential concern.
  2. Some of the conclusions drawn from the information interventions seem stronger than the experiments establish. Appendix G notes that correct facts are described as measured and true, whereas shuffled facts are described as potentially unreliable, so the comparison changes both the information and the model's instructions for how to trust it. Furthermore, numerical depth supplies object-specific coordinates, which is a stronger intervention than providing a global scale reference. Similarly, the navigation map comparison changes geometry and its stated reliability, and may also change completeness and target association. These results support the usefulness of accurate numerical scene information, but I do not think they isolate metric scale as the core bottleneck or establish that navigation failures arise primarily from perception rather than planning.
  3. I found some discrepancies between the claimed uniform evaluation and the conditions documented in the appendix. The main text describes identical prompts and execution settings, but Appendix C reports historical prompt differences and different API integrations, while Appendix F acknowledges varying historical inference configurations across models. Parsing also substantially affects some navigation results, including 59 unparseable responses for Gemini-3.8-Flash. I appreciate that these issues are disclosed, but they make it harder to separate spatial performance from prompt and output-format effects. A common-prompt comparison on the affected conditions, together with a uniformly applied syntax-normalization sensitivity analysis, would be more convincing.
Questions for authors
  1. What is the reconstruction coverage for each model, and do the main findings remain unchanged when missing queried objects are explicitly penalized?
  2. Do the geometric intervention gains persist when correct and shuffled facts receive identical reliability descriptions? Can the map-substitution comparison use the same provenance wording as well?
  3. Can you separate global scale errors from translation, relative configuration, extent, and orientation errors to support the claimed bottleneck more directly?
6 comments
AuthorsOpenReview ↗

Response to the Reviewer zeec - Part 1

W1 / Q1. Object coverage and omission sensitivity

We thank the reviewer for highlighting this important concern.

Mean object coverage is 95.98–100% across model–domain groups, including zero coverage for empty or unparseable outputs. Multiplying each instance's Structure and Metric by its matched/queried object fraction leaves all rankings unchanged, with Structure > Metric in all 54 model–domain pairs. The largest reconstruction-score decrease is 0.17% (Gemini-3.5-Flash: 41.01→40.84); GPT-6-Astra remains at 70.03, and Qwen3.8-Max changes from 52.60 to 52.59. Omissions therefore do not materially drive the current rankings or gap. We will add coverage and sensitivity tables, plus a discussion of the completeness limitation of matched-subset scoring, to the appendix.

W2 / Q2. Identical descriptions for correct and shuffled information

We appreciate this insightful suggestion.

Following Sec. 5.1 (Fig. 4), we repeated the correct-versus-shuffled information interventions, changing only their reliability descriptions to identical neutral wording.

ModelDepth: ΔMetric (%), 95% CIRelations: ΔMetric (%), 95% CI
GPT-5.6-Sol+16.89 [14.44, 19.46]+9.02 [6.44, 11.90]
GPT-6-Astra+20.95 [18.07, 23.99]+6.12 [4.33, 8.21]
Qwen3.8-Max+15.63 [13.15, 18.24]+7.09 [3.75, 10.26]
Gemini-3.8-Flash+15.65 [12.29, 18.88]+6.58 [4.00, 9.19]

Both cues improve Metric; the absolute difference between their gains is 7.87–14.84%, favoring depth across all four models. Depth's advantage therefore persists under identical reliability descriptions.

We also repeated map substitution from Sec. 5.2 on the full navigation evaluation set, matching the wording and target-entry identification task for Self-map and GT-map. GT-map improves success by 47.69–70.52% across all four models; explicit target ID adds smaller gains. The reconstruction and navigation controls are included in Appendices G.1 and F.1 (Tables 16 and 13).

W2 / Q3. Global scale and other geometric errors

We thank the reviewer for this helpful question.

We apologize for the unclear wording. The original sentence, “Controlled interventions isolate metric scale as the core bottleneck,” can suggest global scaling alone. We intended to highlight limitations in metric reconstruction accuracy, including object placement. We propose the following clearer wording: “Object-level metric depth yields larger Metric gains than relational cues under the evaluated conditions.”

To distinguish global scaling from object-placement improvements, we compare no-injection and depth-injected predictions (left → right), uniformly rescaling each to the GT spatial spread. Global scale is the absolute log-ratio of centered layout spreads; Translation measures remaining centroid displacement, Configuration the center RMSE after removing this displacement, Extent the relative box-size error, and Orientation the box-axis angular error. No rotation alignment is applied.

ModelGlobal scale (abs. log)Translation (m)Configuration (m)Extent (%)Orientation (°)
GPT-5.6-Sol0.157 → 0.1190.464 → 0.3200.411 → 0.22834.78 → 36.3528.60 → 27.16
GPT-6-Astra0.104 → 0.0850.352 → 0.2610.238 → 0.16129.69 → 29.7514.25 → 16.03
Qwen3.8-Max0.140 → 0.1240.547 → 0.3620.422 → 0.24233.12 → 37.4429.80 → 26.77
Gemini-3.8-Flash0.179 → 0.1630.893 → 0.3850.421 → 0.25339.78 → 39.2830.46 → 32.03

If depth injection only corrected global scaling, matching both predictions to the GT spread would remove its placement advantage. Instead, Translation decreases by 25.8–56.9% and Configuration by 32.4–44.6% across all four models after this normalization. These reductions show improved object placement and relative layout beyond overall scene scale; Extent and Orientation do not improve consistently.

AuthorsOpenReview ↗

Response to the Reviewer zeec - Part 2

W3. Common prompts and output-format sensitivity

We appreciate the reviewer’s suggestion.

Prompt consistency. We use common task prompts and scoring rules, with each model accessed through its supported API. During tabletop evaluation, some Gemini responses failed to preserve the supplied object IDs, so we added an explicit ID-copy reminder. To evaluate the effect of this prompt revision, we reran the affected tabletop evaluations with the unified prompt:

ModelΔReconstruction (%)
Claude-Opus-4.7+3.48
Claude-Opus-4.8−1.19
Gemini-3.5-Flash−3.05
Qwen3.7-Plus−0.66
Gemini-3.1-Pro+1.93

The Structure–Metric gap persists under the unified prompt.

Output-format sensitivity. Some Gemini responses deviate from the required navigation XML schema by placing coordinates in tag bodies or omitting the fixed track ID. To evaluate how output-format deviations affect the reported scores, we applied the same deterministic syntax normalization to all 18 models, preserving numerical waypoints and scoring criteria. Gemini-3.8-Flash improves from 12.14% to 14.74% (+2.60%), and Gemini-3.7-Flash from 16.18% to 16.47% (+0.29%); the other 16 models are unchanged. Thus, format normalization improves these Gemini scores, while navigation success remains low.

AuthorsOpenReview ↗

Kind Request to Reviewer zeec for Feedback on Our Rebuttal

Dear Reviewer,

We hope this message finds you well. As the discussion period is nearing its end, we would be grateful if you could let us know whether our rebuttal has addressed your concerns, or if any further clarification is needed. Your feedback means a great deal to us.

Thank you again for your time and careful evaluation of our paper.

Best regards, The Authors

Reviewer_W4iGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RECONSPATIAL, a benchmark for evaluating cross-view spatial understanding through joint 3D scene reconstruction. Given multiple images and object queries, models predict oriented 3D boxes in a shared camera-anchored frame, with separate scores for relative structure and metric accuracy. The benchmark also includes nine atomic spatial tasks and offline navigation to connect local spatial judgments with reconstruction and planning. Across 18 LVLMs, relative layouts are recovered more successfully than metric geometry. Further experiments examine how geometric information affects reconstruction, whether reconstructed maps support navigation, and how spatial fine-tuning transfers to other tasks.

Critical strengths
  1. RECONSPATIAL provides a useful benchmark for evaluating whether LVLMs can integrate real-world observations across views into a coherent 3D scene. It covers indoor, tabletop, and outdoor environments and evaluates object positions, sizes, and orientations in a shared coordinate frame. The separate Structure and Metric scores help distinguish plausible relative layouts from accurate metric geometry, providing a more detailed assessment than isolated spatial questions alone.

  2. The empirical evaluation is extensive, covering 18 LVLMs, including advanced models such as GPT-6-Astra and Claude-Opus-5, across indoor, tabletop, and outdoor scenes. The analysis goes beyond reporting aggregate scores: scorer controls, geometric information interventions, and qualitative examples help explain the observed differences between relative layout and metric reconstruction.

Critical weaknesses
  1. The contribution relative to prior reconstruction-based evaluations needs sharper justification. The related-work section distinguishes RECONSPATIAL through real multi-view inputs, full 3D object boxes, and separate Structure and Metric scores. However, it does not clearly explain which evaluation limitations remain in IR3D-Bench, Blueprint-Bench, and Video2Layout, or how the proposed formulation resolves them. Section 4.3 links atomic skills to reconstruction through correlations across models, but this does not establish whether joint reconstruction exposes failures that local tests miss on comparable inputs. The benchmark expansion is useful, while the introduction’s claim of a new evaluation paradigm needs more specific support.

  2. The main diagnostic claims are stronger than the intervention results support. Figure 4 shows that numerical depth for each queried object improves Metric more than ordinal facts, but this does not distinguish errors in global scale from errors in individual object placement. Identifying metric scale as the core bottleneck therefore requires a more specific analysis of where the gains arise. Likewise, Table 2 shows substantial navigation gains in the GT map condition and includes a comparison with GT + target ID. However, the GT map and Self-map conditions also differ in how map accuracy is described and how the target is associated with a map entry. These results show that accurate scene information helps navigation, but they do not isolate perceptual geometry as the primary cause of failure.

Questions for authors
  1. Which limitations of IR3D-Bench, Blueprint-Bench, and Video2Layout does RECONSPATIAL resolve? Please make the comparison explicit in terms of input assumptions, reconstructed geometry, and evaluation targets.

  2. Do correct local spatial judgments coexist with failed joint reconstruction on the same observations? To complement the correlations across models in Section 4.3, compare results on the relevant atomic tasks with reconstruction outcomes on a small shared set of scenes and explain any discrepancies.

  3. In Figure 4, which Metric components account for the gains from numerical depth versus ordinal facts? Please report each model’s changes in Scale, Position, and View IoU relative to the corresponding shuffled controls.

  4. Table 2 already compares the GT map and GT + target ID conditions. Does the difference in navigation success between GT map and Self-map persist when map accuracy is described identically and the target is associated with a map entry in the same way? A paired comparison on a representative subset would help. Please report the success rates and their paired difference with a confidence interval, and briefly explain the subset selection.

4 comments
AuthorsOpenReview ↗

Response to the Reviewer W4iG

W1 / Q1. Comparison with prior reconstruction evaluations

Compared with IR3D-Bench’s single-view synthetic CLEVR scenes of cubes, spheres, and cylinders, ReconSpatial uses real multi-view observations and directly scores 3D boxes without Blender scene instantiation or rendering. Blueprint-Bench evaluates room-level floor plans and leaves individual objects’ 3D placement outside its scope. Video2Layout’s 2D BEV boxes omit vertical positions and extents. ReconSpatial evaluates structured-text predictions across real tabletop, indoor, and outdoor scenes, jointly measuring relative structure and camera-anchored metric placement.

WorkInput assumptionsReconstructed geometryEvaluation targets
IR3D-BenchSingle synthetic CLEVR image; supplied fixed camera parametersRenderable 3D primitivesProjected localization, relations, masks, appearance
Blueprint-BenchReal apartment photographs2D room floor plansConnectivity, counts, size-rank room matching
Video2LayoutAI2THOR/ScanNet framesMetric 2D BEV object boxesArea, relative distances, layout angles, spatial QA
ReconSpatialReal indoor/tabletop/outdoor views; object queries; no supplied intrinsics/posesCamera-anchored oriented 3D object boxesStructure: adjacency/shape/order; Metric: position/view IoU/scale

W1 / Q2. Correct local judgments and inaccurate joint reconstruction

We thank the reviewer for this helpful question!

Using the same ordered source views, we observe correct local answers alongside joint reconstruction errors (Appendix I.1, Fig. 14). Qwen3.8-Max correctly orders four objects by depth, yet its six-object reconstruction reverses the lamp/drawers order. Gemini-3.8-Flash correctly ranks object-pair distances but reconstructs the clothes/basket pair in the wrong front-to-back order. In another scene, it correctly judges the black bag to be left of the green bottle, while its reconstructed bin and towel—objects outside that local QA query—have horizontal position errors of 0.94 and 1.27 m. Correct local judgments therefore do not guarantee jointly consistent geometry across the complete queried scene.

W2 / Q3. Metric components and the bottleneck claim

We appreciate this insightful suggestion!

For the submitted Figure 4 experiment, we reaggregated component gains from correct versus shuffled information.

ModelInformationΔScale (%)ΔPosition (%)ΔView IoU (%)ΔMetric (%)
GPT-5.6-SolNumerical depth+10.87+24.80+11.06+15.58
GPT-5.6-SolRelations−7.56+3.01−1.18−1.91
GPT-6-AstraNumerical depth+6.86+13.12+6.28+8.75
GPT-6-AstraRelations−6.62+3.50+0.83−0.77
Qwen3.8-MaxNumerical depth+9.36+19.68+8.90+12.65
Qwen3.8-MaxRelations+0.79+5.37+1.02+2.39
Gemini-3.8-FlashNumerical depth+6.06+20.01+8.35+11.47
Gemini-3.8-FlashRelations+4.68+7.36+1.65+4.56

Depth improves all three components, most strongly Position. We propose the following narrower wording for the “core bottleneck” claim: “Object-level metric depth yields larger Metric gains than relational cues under the evaluated conditions.”

With identical neutral descriptions for correct and shuffled information, depth still yields larger Metric gains (+15.63–20.95%) than relations (+6.12–9.02%) across all four models. These matched-description controls are included in Appendix G.1 (Table 16).

W2 / Q4. Navigation with matched wording and target association

We thank the reviewer for this suggestion.

On the full navigation evaluation set, we used identical neutral map descriptions. Self-map and GT-map both require target-entry identification; GT-map + target ID supplies that entry explicitly.

ModelSelf-map (%)GT-map (%)GT-map + target ID (%)ΔSuccess (GT − Self, %), 95% CI
GPT-5.6-Sol14.4584.9788.15+70.52 [64.24, 76.38]
GPT-6-Astra35.5595.9596.82+60.40 [52.20, 67.58]
Qwen3.8-Max12.4360.1266.76+47.69 [38.94, 57.68]
Gemini-3.8-Flash17.4472.9775.00+55.52 [47.89, 62.84]

GT-map improves success for every model under matched wording and target-association instructions; target ID adds smaller gains. Map substitution separates scene construction from planning and shows that scene representation quality substantially affects success. The control settings and results are included in Appendix F.1 (Table 13).

AuthorsOpenReview ↗

Kind Request to Reviewer W4iG for Feedback on Our Rebuttal

Dear Reviewer,

We hope this message finds you well. As the discussion period is nearing its end, we would be grateful if you could let us know whether our rebuttal has addressed your concerns, or if any further clarification is needed. Your feedback means a great deal to us.

Thank you again for your time and careful evaluation of our paper.

Best regards, The Authors

AuthorsOpenReview ↗

Follow-up Reminder: Discussion Period Closing within 24 Hours

Dear Reviewer,

We hope this message finds you well. With fewer than 24 hours remaining in the discussion period, we kindly ask if you could take a moment to review our rebuttal and the revised manuscript.

Could you please let us know if our responses have adequately addressed your concerns, or if any issues remain unclear? If we have successfully resolved your questions, we would be deeply grateful if you would consider reflecting this in your final evaluation.

We remain available to clarify any remaining questions before the deadline. Thank you again for your time and careful consideration.

Best regards, The Authors

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Additional Results

Dear Reviewer W4iG,

Thank you again for your careful review and constructive suggestions. As the discussion period draws to a close, we would appreciate hearing whether our rebuttal and revised manuscript have addressed your concerns.

In particular, we provided matched-observation examples of correct local QA alongside reconstruction errors, component-level analyses of the depth interventions, and navigation controls with identical map descriptions and target-association instructions.

If these additions have resolved your concerns, we would be grateful if you could take them into account in your final evaluation. We remain available for any clarification.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.