acceptodds
Under review as a conference paper at ICLR 2027

Reason the Scene, Reconstruct the Space, Render the Unseen: TinkerCube for Spatial Intelligence

Abstract

Spatial reasoning, 3D reconstruction, and novel-view synthesis describe complementary aspects of the physical world, yet are typically isolated, limiting exchange between semantic and geometric evidence. We present TinkerCube, a coordinated multimodal model connecting spatial understanding, target-view geometry prediction, and controllable visual generation. TinkerCube augments a Mixture-of-Transformer-Experts backbone with a multi-view geometry pathway and role-specific interfaces that align 2D appearance, camera states, and cross-view structure in a shared context. Geometry-aware reasoning supplies explicit spatial evidence to the language expert; language-conditioned geometry converts motion and content instructions into target cameras, depths, and point maps; render-guided generation projects a reference-only point cloud into the target view to constrain completion. Three-stage training progressively aligns spatial tokens with language, learns motion-conditioned target geometry, and adapts coverage-aware view synthesis while preserving earlier capabilities. Across spatial question answering, depth and camera estimation, point-map reconstruction, and novel-view synthesis, TinkerCube delivers consistent performance, with leading results on major spatial-reasoning benchmarks and strong view-synthesis performance on DL3DV and RealEstate10K. These results show that integrating semantic reasoning, 3D geometry, and visual generation through joint training and interaction effectively strengthens spatial intelligence.

Primary area:
generative models
Keywords:
Spatial Intelligence, Multimodal Reasoning, 3D Reconstruction, Novel-View Synthesis.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1qNJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TinkerCube combines language, multi-view geometry, and image generation for spatial QA, geometry prediction, and novel-view synthesis.

Critical strengths
  1. The spatial QA gains are convincing, and the matched-data controls help rule out extra training data as the main explanation.
  2. The writing is clear and the logic is easy to follow.
  3. The component ablations support the proposed language–geometry interfaces.
Critical weaknesses
  1. Unclear whether the geometry baselines receive the same target-view information. Without this, the comparison is hard to judge.

  2. The paper does not show that generated views improve reasoning or reconstruction. View-synthesis gains over MetaView are small and mixed.

  3. The overall framework is overcomplicated and which part is initialized with pretrained model, how much is benifited from this pretraining?

Questions for authors
  1. What inputs does each geometry baseline receive? Can you compare them under the same input setting?
  2. Does feeding generated views back into the model can improve QA or even reconstruction?

3.Did the generated images really help with understanding? Creating seemingly reasonable images does not mean that the true hidden space has been restored; the paper does not directly prove that after filling in the generated images, the question-answering or reconstruction performance improves.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1qNJ (Part 1)

Thank you for recognizing our gains in spatial QA, matched data controls, clear presentation, and language and geometry interface ablations. In response to your key questions, we add analyses of input fairness, generation feedback, and pretraining benefits. We appreciate the opportunity to learn from and discuss your feedback.

W1 and Q1: Target view information and fairness of geometry comparisons

Thank you for asking about the fairness of the geometry comparisons. Tables 3, 4, and 5 follow the official RGB input setting of the geometry baselines, and every method receives the same shared RGB observations. To support multimodal interaction and the integration of three dimensional geometric features, TinkerCube can also receive a target view condition. During training, we apply dropout to this condition so that the model can predict geometry when the target condition is empty. Therefore, in the baseline comparisons, we set the target view condition to empty and use the same input information as the baselines. Appendix G now states this evaluation setting explicitly.

To further analyze the role of the target view condition, we add target view information to the same RGB observations. ETH3D Accuracy and Completion decrease by 0.022 and 0.039 m, while Sintel AUC@3 and AUC@30 increase by 3.9 and 3.6 points:

InputETH3D Acc.↓ETH3D Comp.↓Sintel AUC@3↑Sintel AUC@30↑
SharedRGB0.1480.14236.575.4
RGB+TargetView0.1260.10340.479.0

These results show that TinkerCube already provides competitive geometry prediction under shared inputs and can further improve prediction accuracy and completeness by using richer target conditions.

W2, Q2, and Q3: Feedback from generated views to reasoning and reconstruction

Thank you for asking whether generated images genuinely improve understanding and reconstruction. Our original experiments mainly examine how three dimensional geometry supports spatial reasoning and NVS. To further test feedback in the reverse direction, we add experiments on spatial reasoning and three dimensional reconstruction with generated views.

Spatial reasoning with generated views. We use generated images as alternative visual expressions of the available observations and learned structural priors, rather than as true recovery of hidden space. More suitable viewpoints can reduce viewpoint conversion, present cross image object relations within one view, and provide more complete appearance cues for partially visible objects. Specifically, we select five intermediate viewpoints that balance scene coverage and joint visibility of separated regions, generate their images, and combine them with the original inputs to answer the same questions.

InputViewSpatialViewSpatial PersonSSRDViewSpatial Camera RelationSPARScanQAEM
Original56.245.162.765.631.5
WithGeneratedViews57.848.665.566.532.6
Gain(pp)1.63.52.80.91.1

Generated feedback improves ViewSpatial by 1.6 points, including gains of 3.5 and 2.8 points on Person SSRD and Camera Relation, and improves SPAR and ScanQA by 0.9 and 1.1 points. These results show that generated views help the model use existing spatial relation evidence more effectively. Effective feedback also depends on suitable camera selection. Appendix I.5.1 provides the complete selection protocol and visual cases showing how intermediate views reduce occlusion, integrate cross image relations, and simplify reference frame conversion.

Three dimensional reconstruction with generated views. Generated feedback has two roles in reconstruction: supplying transition observations between reference views to refine local geometry, and covering insufficiently observed regions to expand the scene point cloud. We sample five real reference images from a scene window, select three intermediate viewpoints according to missing projected coverage, generate their images, and evaluate reconstruction after adding each view. Every setting is scored against the same real geometry from the scene window. R and G denote real and generated images, while C and A denote Completion and Accuracy in meters, both lower is better.

Input7ScenesC7ScenesAETH3DCETH3DA
5R0.03700.03840.20930.0512
5R+1G0.03560.03730.12690.0482
5R+2G0.03530.03720.11970.0465
5R+3G0.03420.03690.04970.0458
AuthorsOpenReview ↗

Response to Reviewer 1qNJ (Part 2)

(Continuing from the previous part)

Accuracy and Completion improve continuously on both datasets as generated views are added. The additional views improve local point cloud quality and cover space that is insufficiently observed by the original references. Appendix I.5.2 provides the complete protocol and quantitative results, while Appendix A.2, Figures 7 and 8, shows progressive point cloud expansion. These experiments directly demonstrate that generated views support spatial reasoning and improve both the accuracy and completeness of three dimensional reconstruction.

W2: Comparison with MetaView

Thank you for asking about this comparison. MetaView is a recent state of the art NVS specialist at ECCV 2026 with about 28B parameters, whereas the TinkerCube expert responsible for image generation uses only 7B. With this more compact generation component, TinkerCube achieves comparable NVS performance and higher PSNR on DL3DV and RealEstate10K, with 16.218 versus 16.190 and 19.419 versus 19.339, respectively. It also achieves higher SSIM on RealEstate10K, with 0.682 versus 0.677.

Our primary focus is the unified modeling of spatial reasoning, three dimensional reconstruction, and NVS, together with the interactions among these capabilities. We include MetaView to present the generation ability of our unified model against a strong specialist reference. The results show that TinkerCube supports all three spatial tasks while its 7B generation expert retains synthesis quality comparable to a leading specialist model. They also support the value of coordinating geometry and semantic information for NVS.

W3: Architecture complexity, initialization, and benefits of our design

Thank you for asking about the architecture and the benefits of pretraining. Although the complete framework may appear complex, the three experts have clear and complementary roles. BAGEL initializes the understanding and generation experts with basic multimodal understanding and image generation capabilities. VGGT Ω initializes the geometry expert with basic three dimensional geometry capabilities. These pretrained weights provide a reliable general starting point and a stable optimization platform for subsequent spatial interaction and capability expansion.

On this foundation, we introduce interaction structures designed for spatial tasks. GAR brings camera and cross view geometry into spatial reasoning. LCG uses motion and content routing and target view modulation to convert semantic instructions into camera and geometry predictions. RGG constrains image generation through geometric rendering, coverage masks, and camera rays. Together with progressive training across three stages, these designs extend the basic pretrained abilities into coordinated spatial perception, modeling, and generation.

The matched results separate the general foundation supplied by pretraining from the gains produced by our design. The table reports actual scores, with gains over the corresponding baseline in parentheses:

SettingSPARViewSpatialVSI
BAGEL39.342.026.7
Full TinkerCube65.6 (+26.3)56.2 (+14.2)61.4 (+34.7)
TinkerCube S1 with GAR versus BAGEL plus S160.4 (+4.6)52.7 (+3.8)55.2 (+3.6)

The BAGEL scores show that pretraining provides a reusable vision and language foundation, while leaving substantial room for specialized spatial improvement. The GAR comparison uses identical initialization, S1 data, and training budget, while the fusion comparison uses the same pretrained components, data, and training budget across all three stages. On geometry, TinkerCube reduces Sintel camera ATE by 24.6% relative to VGGT Ω and improves AUC@30 by 2.4 and 3.8 points on Sintel and 7 Scenes. These results show that pretraining mainly supplies general abilities and training efficiency, while the substantial spatial gains arise from our interaction interfaces, spatial adaptation, and progressive coordinated optimization.

We hope these additions address your questions and would be happy to discuss any remaining concerns. If these clarifications strengthen your confidence in our work, we would be grateful if you could consider whether a slightly higher score might be appropriate. Thank you again for your time, careful consideration, and constructive suggestions, which have helped us improve the paper.

AuthorsOpenReview ↗

Follow Up to Reviewer 1qNJ

Dear Reviewer 1qNJ,

We apologize for following up again during this busy period. Thank you sincerely for your careful review and thoughtful suggestions. Your feedback has given us valuable insights and helped us improve our work, and we have learned a great deal from this exchange.

When you have a chance, we would be grateful to hear whether our responses and additional experiments have addressed your concerns. We greatly value the opportunity to learn from your perspective and will make every effort to address any remaining questions thoroughly. We would be happy to provide further explanations and continue an open and constructive discussion.

If you feel that our additional experiments and clarifications have addressed your concerns or strengthened your confidence in the paper, we would also be very grateful if you could consider whether a slightly higher score might be appropriate.

It has been a pleasure to exchange ideas with you. We look forward to hearing from you whenever convenient and would sincerely appreciate any further feedback. Thank you again for your patience, time, and care in helping us improve our work.

With sincere regards,
The Authors

AuthorsOpenReview ↗

Follow Up on Our Rebuttal

Dear Reviewer 1qNJ,

We apologize for following up once more. As the discussion period draws to a close, we would appreciate a brief update on whether our responses and additional experiments have addressed your concerns. We would be happy to clarify any remaining questions.

We understand that scores may still be updated after the author reviewer discussion period ends. If our additions have resolved your concerns or strengthened your confidence in the paper, we would be very grateful if, when convenient, you could consider whether an upward score adjustment would be warranted.

Thank you sincerely for your time and consideration.

With sincere regards,
The Authors

Reviewer_MzYEOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This work introduces TinkerCube, a unified architecture built upon a Mixture-of-Transformer-Experts backbone with three interconnected modules: the Geometry-Aware Reasoner, Language-Conditioned Geometry, and Render-Guided Generator (RGG). The architecture supports spatial-language alignment, target-view geometry learning, and coverage-aware novel-view synthesis. The experiments cover spatial reasoning, open-ended 3D question answering, depth prediction, camera estimation, point-map reconstruction, and novel-view synthesis. Overall, the results suggest that explicit geometric representations can benefit both spatial reasoning and image generation, while language conditions can be used to control target cameras and target-view geometry.

Critical strengths

1. The research problem is important, and the overall design is appealing. Spatial reasoning, 3D reconstruction, and novel-view synthesis are typically studied independently. This work establishes clear information pathways through explicit camera parameters, depth, point maps, and point-cloud projection, allowing the different capabilities to interact through reusable geometric intermediates.

2. The evaluation is comprehensive and demonstrates the effectiveness of the unified architecture. The paper evaluates single-image, multi-image, video-based, and open-ended spatial reasoning, as well as camera estimation, depth prediction, point-map reconstruction, and novel-view synthesis. The model achieves strong results on SPAR-Bench, ViewSpatial-Bench, VSI-Bench, ScanQA, SQA3D, Sintel, ETH3D, DL3DV, and RealEstate10K, demonstrating a broad range of spatial capabilities.

3. The ablation studies and matched comparisons are thorough and demonstrate mutually beneficial interactions among the modules. Table 14 separates the contribution of spatial instruction tuning from that of geometry tokens. Tables 9–10 analyze the key components of GAR and RGG. Tables 15, 17, and 18 further compare frozen composition, one-shot fusion, and the complete interfaces. Together, these experiments support the claim that the proposed modules are not merely assembled side by side.

Critical weaknesses

1. I am concerned about the model’s performance on out-of-distribution understanding tasks.

TinkerCube performs well on SPAR-Bench, ViewSpatial-Bench, and VSI-Bench. However, samples in these benchmarks are largely template-based. It is unclear whether the training data contain samples with matching task formulations or templates (for example, through direct use of the corresponding training corpora, such as SPAR-7M or VSI-Train, or through synthetic data generated using similar rules and templates). I wonder whether the reported results represent genuine out-of-distribution generalization.

In contrast, MMSI-Bench and MMSI-Video-Bench are fully human-annotated benchmarks whose samples are more difficult to reproduce through template-based data generation. They could therefore provide a stronger evaluation of the model’s out-of-distribution capabilities. I would be interested in seeing TinkerCube’s performance on these benchmarks.

Questions for authors

Regarding Weakness 1, could the authors:

  1. Clarify whether the training data contain samples with the same task types or templates as those used in VSI-Bench, ViewSpatial-Bench, and SPAR-Bench, and whether the reported results should be considered out-of-distribution generalization?

  2. Report evaluation results on MMSI and MMSI-Video?

3 comments
AuthorsOpenReview ↗

Response to Reviewer MzYE

Thank you for recognizing the importance of the problem, the value of the unified architecture, and the completeness of our experiments and ablations. In response to your constructive suggestions on generalization, we have added data and template separation details and results on MMSI Bench and MMSI Video Bench. We welcome further discussion and the opportunity to learn from your feedback.

W1 and Q1: Training tasks, template relations, and out of distribution generalization

Thank you for asking about training data and generalization. We use no VSI Train data in S1, S2, or S3, no training samples matching the formats or templates of ViewSpatial Bench, and no synthetic data constructed by imitating these benchmarks. For SPAR, we use only the official SPAR 7M RGBD training data under a strict split that excludes evaluation data from training. Our training whitelist restricts eligible scenes, original images, and source videos at data loading time, excluding evaluation content and all derived data from training. We first collect the evaluation scene and source data identifiers from each benchmark and remove them from the whitelist. SPAR Bench scenes are excluded from the SPAR, ScanNet, and ScanNet++ training whitelists. ScanNet evaluation scenes in ViewSpatial Bench are excluded by scene identifier. VSI Bench scenes are excluded from ScanNet, ScanNet++, and ARKitScenes according to their source and scene identifiers; repeated ARKitScenes captures of the same location are also excluded. This whitelist excludes data associated with evaluation scenes across all three training stages. TinkerCube encounters none of the evaluation formats or question templates of BLINK, ViewSpatial Bench, and VSI Bench during training. ScanQA and SQA3D additionally require free form answer generation rather than multiple choice responses. Competitive results across these distinct question and answer formats demonstrate generalization across evaluation formats, question templates, and output forms. We also evaluate MMSI Bench and MMSI Video Bench, which are entirely designed and annotated by humans, to further test this capability. Together, strict data separation and evaluation across formats show that TinkerCube learns transferable spatial representations rather than relying on specific samples or templates. Appendix C.1 details the data sources, whitelist construction, and separation rules for each benchmark.

W1 and Q2: MMSI Bench and MMSI Video Bench

Thank you for this suggestion. To further evaluate out of distribution generalization, we test TinkerCube on MMSI Bench [1] and MMSI Video Bench [2], both of which are entirely designed and annotated by humans.

On MMSI Bench, TinkerCube achieves an average score of 36.20%, exceeding the specialized spatial perception models SpaceR (27.4%), ViLaSR (30.2%), VST 7B (32.5%), and Cambrian S (27.0%). The subcategories cover positional relationships across views, geometric attributes, camera and object motion, and multistep spatial reasoning:

ModelC–CO–OR–RC–OO–RC–RAttribute Meas.Attribute Appr.Motion Cam.Motion Obj.MSRAvg.
TinkerCube31.1835.1133.3348.8443.5334.9440.6325.7644.5934.2131.8236.20

On MMSI Video Bench, TinkerCube achieves an average score of 33.20%, exceeding SenseNova SI 1.1 BAGEL (23.8%), VST 7B (24.9%), SpaceR SFT 7B (24.7%), and ViLaSR (28.3%). The model shows consistent ability across spatial construction, motion understanding, planning, prediction, and cross video reasoning:

ModelSpatial ConstructionMotion UnderstandingPlanningPredictionCross-Video ReasoningAvg.
TinkerCube33.8429.5530.6531.7138.9533.20

Together, the two benchmarks extend the generalization evidence from cross view relationships, geometric attributes, and motion perception to video spatial construction, planning, prediction, and cross video reasoning. Combined with the preceding separation of training data and question templates, these results further show that TinkerCube learns transferable joint semantic and geometry representations that generalize to unseen data, human designed questions, and different forms of spatial and temporal reasoning.

We will include the overall scores and category results for both benchmarks in the revised manuscript.

We hope these additions address your concerns. We welcome further questions and discussion. If these explanations strengthen your confidence in our work, we would be grateful for a more positive assessment. Thank you for your time and suggestions.

References

[1] Sihan Yang et al. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence. ICLR, 2026.

[2] Jingli Lin et al. MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence. arXiv preprint arXiv:2512.10863, 2025.

I appreciate the authors’ response and the additional experiments, which have addressed most of my questions. After reviewing the rebuttal, I remain positive about the paper and will retain my original rating.

AuthorsOpenReview ↗

Appreciation to Reviewer MzYE

Thank you very much for carefully reviewing our rebuttal and additional experiments. We sincerely appreciate the time and effort you devoted to evaluating our work, as well as your thoughtful feedback and positive assessment.

Reviewer_q1i5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TinkerCube, a coordinated multimodal model that combines spatial reasoning, target-view geometry prediction, and controllable novel-view synthesis. TinkerCube builds on a Mixture-of-Transformer-Experts backbone and introduces three interacting components: a Geometry-Aware Reasoner that injects scene, camera, and cross-view register tokens into multimodal reasoning; a Language-Conditioned Geometry module that converts motion and content instructions into target cameras, depth maps, and point maps; and a Render-Guided Generator that conditions novel-view synthesis on projected reference geometry, coverage, camera rays, appearance, and semantic context. The model is trained progressively in three stages for spatial-language alignment, target-view geometry learning, and coverage-aware generation. The paper evaluates TinkerCube across spatial reasoning, open-ended 3D question answering, depth and camera estimation, point-map reconstruction, and novel-view synthesis. It reports strong results across these task families, including leading performance on several spatial-reasoning benchmarks, camera estimation, and ETH3D reconstruction, as well as competitive or leading novel-view synthesis performance. Component and stage-wise ablations are provided to analyze the proposed interfaces and training pipeline.

Critical strengths

1.A consequential integration of reasoning, geometry, and generation. TinkerCube does more than attach multiple output heads to a shared backbone. GAR exposes camera and cross-view structure to the language model, LCG converts semantic motion instructions into explicit target geometry, and RGG uses projected geometry and coverage as structured generation conditions. These reusable geometric intermediates make the proposed integration technically meaningful.

2.Broad and unusually consistent empirical validation. The paper evaluates reasoning, open-ended 3D QA, depth, camera pose, point-map reconstruction, and novel-view synthesis across many benchmarks. TinkerCube performs strongly across all of these task families rather than obtaining a large gain on only one selected task. Tables 7-10 further provide useful stage-wise and retrained component ablations, showing that the gains are associated with the proposed spatial tokens, geometry pathway, target rendering, and conditioning interfaces.

Critical weaknesses

1.Training-data attribution and evaluation separation are insufficiently clear in the main paper. Stage 1 uses approximately 11.02M instruction records, while Stages 2-3 use datasets that are also closely related to several evaluation benchmarks, including SPAR, ScanNet-derived tasks, DL3DV, and RealEstate10K. The main paper does not explicitly state the scene-level split rules, cross-dataset deduplication procedure, or whether evaluation questions and target trajectories are guaranteed to be absent from training. This is consequential because some of the strongest claims rely on benchmark improvements after very large-scale, heterogeneous supervision. I do not see evidence of leakage, but a precise confirmation is needed to interpret the results confidently.

2.The evidence supports a coordinated pipeline more strongly than fully mutual capability improvement. The paper argues that reasoning, reconstruction, and generation support one another, and Figure 1 states that generated views can enrich reasoning and iterative reconstruction. The ablations convincingly show benefits from spatial alignment to reasoning and from semantic/geometric conditioning to reconstruction and generation. However, Stage 3 does not improve the reported reasoning or geometry metrics in Table 7, and the main experiments do not directly evaluate whether generated views improve subsequent reasoning or reconstruction. The paper should either provide this closed-loop evaluation or narrow the claim from mutual cooperation to a primarily directed reasoning-to-geometry-to-generation pipeline.

Questions for authors

1.Can the authors confirm that all evaluation samples are strictly disjoint from the 11.02M Stage-1 records and the Stage-2/3 training data at the scene, frame, question, and trajectory levels? This is particularly important for SPAR/SPAR-7M-RGBD, ScanQA/SQA3D and the ScanNet-derived training sources, as well as DL3DV and RealEstate10K. Please describe any deduplication or contamination checks. A clear confirmation would materially increase my confidence in the empirical results.

2.Figure 1 states that generated views enrich spatial reasoning and iterative reconstruction. Is there a quantitative experiment in which RGG-generated views are fed back into GAR or LCG and improve reasoning or reconstruction? If not, should the central claim be revised to describe a directed interaction from reasoning to geometry to generation rather than mutual improvement among all three capabilities?

3.What are the total parameter count and inference/training costs of TinkerCube relative to the principal baselines? Given the large training corpus and the combination of understanding, geometry, and generation experts, this information would help determine whether the gains arise from the proposed interfaces or primarily from increased model and data scale.

4 comments
AuthorsOpenReview ↗

Response to Reviewer q1i5 (Part 1)

Thank you for recognizing the value of our unified design for reasoning, geometry, and generation, as well as the breadth of our experiments. In response to your key questions, we add analyses of data separation, feedback from generated views, and computational cost. We appreciate the opportunity to learn from and discuss your feedback.

W1 and Q1: Training data construction and separation from evaluation

Thank you for raising this important question. We confirm that the S1, S2, and S3 training data are strictly separated from the final evaluation data at the levels of scenes, source images and videos, frames, questions, and trajectories.

We build a shared JSON training whitelist for all three stages. For each benchmark, we collect its evaluation scene or source video identifiers and remove them from the whitelist. Once a scene is excluded, its images, depth, point clouds, descriptions, QA pairs, target frames, and trajectories are also excluded. Specifically, SPAR uses only official training QA from eligible scenes. ScanQA and SQA3D validation and test scenes are excluded from all ScanNet derived training data. All 140 DL3DV Benchmark scenes are removed by hash from DL3DV 10K and RealCam Vid. RealEstate10K follows the official training and test split and source video identifiers, and RealCam Vid clips from test videos are also removed. This rule operates at the underlying scene and video levels and covers all three training stages. Appendix C.1 provides the complete protocol.

The matched data ablations discussed in Q3 further show that the gains arise from the model design. In summary, all three stages follow strict training and evaluation separation, while geometry representations and coordinated interfaces still provide consistent gains when the training data are fixed.

W2 and Q2: Feedback from generated views to reasoning and reconstruction

Thank you for asking how generated views feed back into reasoning and reconstruction. Our original experiments mainly examine how three dimensional geometry supports spatial reasoning and NVS. Table 7 shows no further improvement in reasoning or geometry after S3 because this stage trains only the generation expert, while the understanding and geometry experts remain frozen. To keep the evaluation inputs identical across stages, the reasoning and geometry evaluations invoke neither the generation expert nor generated views. Table 7 therefore measures retention of the preceding capabilities rather than feedback gains. To directly test this reverse interaction, we add experiments on spatial reasoning and three dimensional reconstruction with generated views.

Spatial reasoning with generated views. Based on the reference geometry, we compute five intermediate viewpoints that balance scene coverage and joint visibility of separated regions, generate their images, and use them together with the original images for question answering. We do not claim that the generated views constitute real observations. Instead, they transform existing cross image relations, camera information, and structural priors into potentially more accessible visual representations, for example by observing objects from more suitable directions, presenting relations from multiple images within a single view, and completing the appearance of partially visible objects. The results are as follows:

InputViewSpatialPersonSSRDCameraRelationSPARScanQAEM
Original56.245.162.765.631.5
WithGeneratedViews57.848.665.566.532.6
Gain(pp)1.63.52.80.91.1

Generated views improve ViewSpatial by 1.6 points, including gains of 3.5 and 2.8 points on Person SSRD and Camera Relation, and improve SPAR and ScanQA by 0.9 and 1.1 points. The larger gains on relational questions indicate that intermediate views reduce the difficulty of cross image correspondence and viewpoint conversion. Appendix I.5.1 provides the complete protocol and representative cases.

Three dimensional reconstruction with generated views. We sample five real images from one scene window to construct the initial point cloud. We then select three intermediate viewpoints according to missing projected coverage, generate their images, and evaluate reconstruction after adding each view. Every setting is scored against the same real scene geometry. R and G denote real and generated images, while C and A denote Completion and Accuracy in meters, both lower is better.

Input7ScenesC7ScenesAETH3DCETH3DA
5R0.03700.03840.20930.0512
5R+1G0.03560.03730.12690.0482
5R+2G0.03530.03720.11970.0465
5R+3G0.03420.03690.04970.0458
AuthorsOpenReview ↗

Response to Reviewer q1i5 (Part 2)

(Continuing from the previous part)

Accuracy and Completion improve continuously on both datasets. Generated views supply transition observations that refine local geometry and cover regions that are insufficiently observed in the original inputs, progressively expanding the reconstructed space. Together, the two experiments show that generated views support reasoning by reorganizing existing spatial evidence and improve reconstruction by adding useful intermediate views. Appendix I.5 provides the complete protocol, and Appendix A.2, Figures 7 and 8, shows continuous scene exploration and point cloud expansion.

Q3: Parameter count, training and inference costs, and attribution of gains

Thank you for asking about model scale and computational cost. Most existing baselines address only one spatial task, whereas TinkerCube unifies three capabilities. We therefore compare both specialist and unified models:

ModelParametersSupported capability
VLM 3R7BSpatial reasoning
SpaceR7BSpatial reasoning
π³About 1BThree dimensional modeling
MetaViewAbout 28BNVS
BAGEL14BGeneral understanding and generation
TinkerCubeAbout 15BReasoning, modeling, NVS

TinkerCube consists of a 7B understanding expert, a 7B generation expert, and a 1B geometry expert. Its total scale remains compact relative to combining multiple specialist models.

TinkerCube uses an expert architecture that activates only the paths required by the current task, keeping inference cost controlled. Inference rate is reported as the number of seconds required per iteration:

Task settingTinkerCube inference rateBAGEL inference rateTinkerCube peak memoryRelative cost
Spatial reasoning with 4 images0.386s/it0.365s/it35.7GB5.8% increase
NVS with 50 steps3.7s/it3.4s/it53.6GB8.8% increase

Training uses 16 A100 GPUs for 20k, 25k, and 30k steps in S1, S2, and S3, respectively, and updates only the experts required by each stage. Thus, TinkerCube supports all three capabilities while the practical cost of each task remains close to BAGEL.

Matched controls further separate architectural gains from scale:

ControlMatched settingResult
Table 14: TinkerCube versus BAGELSame S1 dataAbout 1B geometry expert and interfaces improve every benchmark
Table 15: Joint versus frozen pathsSame S2 dataJoint semantic and geometry optimization outperforms freezing either path

These controls attribute the gains to geometry representations, cross module interfaces, and coordinated training rather than to additional data or parameter count.

We hope these additions address your questions. We welcome further discussion and would be grateful for a more positive assessment if these clarifications strengthen your confidence in our work. Thank you for your time and constructive suggestions.

Thank you for the additional experiments and detailed clarifications. They address my main questions. After considering the response and the overall evidence, I will maintain my original score.

AuthorsOpenReview ↗

Response to Reviewer q1i5

Thank you for reviewing our response and additional experiments. We’re glad they addressed your main questions and sincerely appreciate your thoughtful feedback.

We were encouraged by your original comment that the reusable geometric intermediates “constitute the main reason I favor acceptance. ( https://openreview.net/revisions?id=pCqyXbc1EG ) ” We noticed that this phrase is no longer included in the revision of your review, while the positive technical assessment remains. In light of your original support and your confirmation that our response addressed your main questions, we would be grateful if you could reconsider whether the additional experiments and evidence warrant a slightly higher score. We would be very happy to discuss any remaining concerns and provide further clarification.

Thank you again for your time and constructive suggestions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Reviewer Feedback (Part 1)

We sincerely thank the reviewers for their careful assessment of our work. We have addressed the questions raised by each reviewer in the individual responses and provided the corresponding clarifications and additional experiments. To facilitate an overall assessment, we summarize the main issues and key evidence below and further clarify the core contributions and advantages of the paper. Full protocols and analyses are provided in the individual responses and appendices.

1. Generated views improve both reasoning and reconstruction

To directly test feedback from generation to the other capabilities, we add two experiments. For reasoning, five intermediate viewpoints are selected using the reference geometry to balance scene coverage and joint visibility. Their generated images are combined with the original observations to answer the same questions. The intermediate views make existing cross image relations, camera information, and structural priors easier to use by reducing viewpoint conversion, bringing separated relations into a common view, and providing more complete appearance cues.

Reasoning benchmarkOriginalWith generated views
ViewSpatial56.257.8
Person SSRD45.148.6
Camera Relation62.765.5
SPAR65.666.5
ScanQA EM31.532.6

For reconstruction, we start from five real reference images and progressively add three generated intermediate views selected according to missing projected coverage. Every setting is evaluated against the same real scene geometry, and both Completion and Accuracy are lower when better.

Dataset and metric5 real views5 real + 3 generated views
7 Scenes Completion0.03700.0342
7 Scenes Accuracy0.03840.0369
ETH3D Completion0.20930.0497
ETH3D Accuracy0.05120.0458

The larger gains on relational reasoning, together with the progressive reconstruction improvements, demonstrate a bidirectional interaction: geometry supports generation, while generated intermediate views reorganize spatial evidence for reasoning and supply useful transition observations for reconstruction. This directly validates a central advantage of TinkerCube: reasoning, geometry, and generation form a mutually reinforcing system rather than three merely colocated capabilities. Appendix I.5 provides the protocols and quantitative results, and Appendix A.2, Figures 7 and 8, presents representative cases and progressive point cloud expansion.

2. Training separation and out of distribution generalization

The S1, S2, and S3 training data are strictly separated from all final evaluation data at the levels of scenes, source images and videos, frames, questions, and trajectories. A shared training whitelist removes every evaluation scene or source video identifier before data loading; once a scene is excluded, all associated images, depth, point clouds, descriptions, question answer pairs, target frames, and trajectories are also excluded. In particular, ScanQA and SQA3D validation and test scenes are removed from all ScanNet derived training data; all 140 DL3DV Benchmark scenes are removed by hash from DL3DV 10K and RealCam Vid; and RealEstate10K follows its official training and test split, with clips from test source videos also removed from RealCam Vid. We use no VSI Train data, no samples matching the formats or templates of ViewSpatial Bench, and no synthetic data constructed by imitating these benchmarks. Appendix C.1 documents the complete protocol.

The evaluation also covers unseen benchmark formats, question templates, and answer forms. BLINK, ViewSpatial Bench, and VSI Bench formats are absent from training, while ScanQA and SQA3D require free form answers rather than multiple choice responses. We further test TinkerCube on additional fully human designed image and video spatial intelligence benchmarks, where it outperforms the relevant comparison methods. Together with strict data and template separation, these results provide further evidence that the unified model learns transferable joint semantic and geometry representations rather than relying on evaluation samples or benchmark specific templates. This breadth across unseen data, question forms, and spatial and temporal tasks is a key advantage of our approach. Detailed settings and results are provided in the corresponding reviewer discussion.

AuthorsOpenReview ↗

Summary of Reviewer Feedback (Part 2)

3. Performance gains arise from the proposed design with controlled cost

TinkerCube contains a 7B understanding expert, a 7B generation expert, and a 1B geometry expert. BAGEL and VGGT Ω provide general understanding, generation, and geometry initialization, while our GAR, LCG, and RGG interfaces and progressive training establish task specific spatial interaction. The comparison with the original BAGEL shows the gains obtained by the complete spatial framework:

SettingSPARViewSpatialVSI
BAGEL39.342.026.7
Full TinkerCube65.6 (+26.3)56.2 (+14.2)61.4 (+34.7)

Path freezing ablations further show that joint optimization outperforms freezing either the understanding or geometry path. On geometry, TinkerCube reduces Sintel camera ATE by 24.6% relative to VGGT Ω and raises AUC@30 by 2.4 and 3.8 points on Sintel and 7 Scenes. These results show that pretraining supplies a reusable foundation, while the substantial spatial gains come from our geometry representation, interaction interfaces, spatial adaptation, and coordinated optimization.

The expert architecture activates only the paths required by each task. Relative to BAGEL, four image reasoning increases time per iteration by only 5.8%, and 50 step NVS by only 8.8%:

TaskTinkerCubeBAGELTinkerCube peak memory
Four image reasoning0.386 s/it0.365 s/it35.7 GB
50 step NVS3.7 s/it3.4 s/it53.6 GB

Training uses 16 A100 GPUs for 20k, 25k, and 30k steps in S1, S2, and S3, respectively, with only the experts required by each stage updated. Thus, TinkerCube unifies spatial reasoning, three dimensional reconstruction, and NVS with a modest per task overhead, while the ablations support the contribution of the proposed architecture and training strategy. The combination of broad task coverage, substantial performance gains, and inference cost close to BAGEL demonstrates the practical advantage of the expert design.

Together, these clarifications establish strict separation between training and evaluation, provide direct evidence for cross task feedback, and distinguish the contribution of our design from pretraining, scale, and data. They further demonstrate that TinkerCube combines transferable spatial intelligence, effective interaction among three capabilities, and controlled computational cost within one unified framework. We will incorporate the new protocols, results, and clarifications into the revised manuscript. We sincerely thank the reviewers for the time, care, and constructive effort devoted to evaluating and improving our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.