acceptodds
Withdrawn from ICLR 2027

SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks, yet their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images – referred to as spatial intelligence – remains limited. Existing approaches attempt to address this gap by using real-scene spatial question answering datasets that require dense geometric annotations. However, constructing such labels is costly, time-consuming, and often noisy due to reliance on external perception modules. In this work, we propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks. We introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset further incorporates controlled color modulation as visual cues to encourage anchor-based reasoning in visually complex conditions. Experiments demonstrate that LVLMs trained on our dataset through either direct answering or reasoning-based prediction significantly outperform baselines and generalize to real-world spatial tasks, despite the dataset’s synthetic and compact nature.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Spatial Intelligence, Large Vision-Language Models, Synthetic Dataset

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BJTjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper's key strength is its construction paradigm, using abstract block structures rendered in simulation to produce both training data and evaluation items whose ground truth is exact by construction, avoiding the noisy meta data that make real-scene datasets costly and annotation-heavy. It instantiates this paradigm with 15k rendered items across three task families plus a 600-question held-out SB-Bench while GPT-5 scores only 43.8 and Gemini-2.5-flash 36.4, showing the rendered items are diagnostic rather than trivial. The rendered proxy also carries transferable signal rather than renderer trivia, since a recipe-matched control and an equal-size Synthetic-Real control do not transfer while training on the block tasks lifts MindCube by +6.2 to +25.1 across all four backbones.

Critical strengths
  • The paper's key strength is using abstract block structures rendered in simulation by three types (3D-to-2D projection, viewpoint transformation and structural combination) to produce both training data and evaluation items that probe spatial skills, with ground truth exact by construction rather than estimated by noisy perception modules.
Critical weaknesses
  • The gains are highly inconsistent across domains, in-domain SB-Bench and the conceptually nearest benchmark MindCube, yet MMSI-Bench gains marginally with all models stuck in the near-chance band. The improvements on MMSI, SPBench, MMMU may not support the motivation that foundational spatial skills could be improved by the proposed structured synthetic tasks.
  • How do the authors address the perceptual ambiguity inherent in their visual stimuli? In Figure 2(c), the leftmost render can be read either as two disjoint components or as a single, already-connected structure, since isometric renders of face-touching blocks carry no explicit cue distinguishing contact between two objects from connectivity within one object. Does such ambiguity interfere with the model's spatial understanding and reasoning, and was any analysis or control performed to rule out that models exploit these ambiguous cases?
Questions for authors

The perspective imagination of real-world objects is related to much more elements, such as materials, colors, designs. Could the proposed structured blocks reflect these information correctly?

Reviewer_z6fgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SpatialBlock introduces 15K synthetic block problems covering projection, viewpoint transformation, and structural combination, with color cues for depth and correspondence. Models use direct-answer training or LoRA initialization followed by GRPO. Evaluation spans four backbones, a synthetic benchmark, and real-scene spatial tasks.

Critical strengths
  1. A small, inexpensive synthetic dataset produces substantial transfer to real scenes across model families. For example, direct training raises Qwen2.5-VL-7B from 31.3 to 48.8 on MindCube. This is the paper’s most consequential result.
  2. The data-design controls are useful: matched sample exposure with SpatialLadder, monochrome variants, individual task types, and Synthetic-Real. They provide evidence that the choice of synthetic tasks matters.
Critical weaknesses
  1. Structural separation between training, validation, and SB-Bench is unspecified. The paper does not explain whether configurations, rotated equivalents, or reused components cross splits, and the checkpoint-selection validation set is unclear. These details matter when synthetic test accuracy approaches saturation. The supplementary angle test reuses configurations and does not settle this issue.
  2. The broad reasoning claims are stronger than the results. Transfer is most convincing on MindCube, while some other gains are small and absolute-distance performance declines. The initialization comparison also changes both adaptation and teacher-reasoning supervision, so it does not isolate the claim that LoRA preserves reasoning ability.
Questions for authors
  1. How are underlying structures and components separated across training, checkpoint validation, and SB-Bench?
  2. What does LoRA initialization alone achieve under the same reasoning prompt and inference budget, before GRPO?
  3. Are the smaller real-scene improvements stable across runs?
Reviewer_khtgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the lack of spatial intelligence in large vision-language models: the ability to understand and reason about three-dimensional structures from two-dimensional images. Current solutions rely on expensive and noisy real-world datasets with dense geometric annotations. To address this, the authors introduce a synthetic dataset of 15,000 block-stacking problems inspired by human cognitive development. The dataset covers three core tasks, including three-dimensional to two-dimensional projection, viewpoint transformation, and structural combination, enhanced with color cues to mimic real world anchor based reasoning. The authors train models using either direct answer prediction or a reasoning-based approach with reinforcement learning. Experiments show that these models significantly outperform existing baselines and generalize well to real world spatial tasks, proving that targeted synthetic training can effectively build spatial intelligence.

Critical strengths

The motivation part was strongest to me. Inspiration from human cognitive development provides a strong, intuitive foundation for the chosen block-stacking tasks.

Critical weaknesses

In my view, the critical weaknesses are:

First, the risk of artificial shortcuts over true understanding. The model is trained on a very simple, synthetic environment using color cues to represent depth or structure. There is a strong concern that the model is just memorizing these specific color patterns and block arrangements, rather than learning genuine three dimensional geometry. This makes its claimed success on complex real-world scenes highly questionable.

Second, the evaluation setup is too limited and potentially unfair. By testing exclusively on multiple-choice questions, the model can simply guess or eliminate wrong answers, which hides whether it can actually generate or understand spatial relationships from scratch. Additionally, the comparison with top proprietary models may be unfair if those models were not given specialized, task specific prompts to do their best.

Third, the training methodology shows signs of fragility. The dataset is quite small, which raises the risk of overfitting. Furthermore, the reinforcement learning reward system is overly simplistic, checking only for basic formatting and length rather than verifying if the step by step reasoning is actually logical. This is supported by unexplained performance drops in some of the larger reasoning models, suggesting the training process might be unstable or reliant on specific tricks rather than being a robust solution.

Questions for authors

My questions are as follows:

  1. What specific evidence or ablation can you provide to show that the model is using genuine three-dimensional geometry rather than relying on color shortcuts when evaluated on real-world images without color cues?

  2. Could you include an evaluation on open-ended spatial reasoning tasks or coordinate prediction to prove the model can generate spatial understanding without the crutch of multiple-choice options? How can you modify the reward function to include step-by-step logical verification, ensuring the chain of thought is actually correct and not just formatted correctly?

  3. Can you provide a deeper analysis or visualization of the model outputs to explain exactly why full fine-tuning degrades reasoning ability compared to the lightweight alternative? What metrics or analyses can you share to prove the model is not simply memorizing the synthetic samples, such as a learning curve or a test on entirely new block configurations?

  4. Did you evaluate the proprietary baseline models using advanced task-specific prompting, and if not, could you provide those results to ensure a fair comparison? Can you run an ablation where the color cues are systematically removed or scrambled in the real world evaluation to prove the model does not over rely on this artificial shortcut? What additional benchmarks or qualitative analyses can you provide to demonstrate that the learned representation is truly holistic and not just overfitting to the specific biases of the chosen out of domain datasets?

  5. Can you provide a detailed error analysis of the larger reasoning model on the in-domain benchmark to explain why the reinforcement learning phase causes a drop in accuracy?

  6. The related works are not well-explored. There are a lot of work done in this area, in 2026; yet only 3 works are mentioned. Can you please discuss all relevant works, maybe with a table, and position your work accordingly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.