- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper's key strength is its construction paradigm, using abstract block structures rendered in simulation to produce both training data and evaluation items whose ground truth is exact by construction, avoiding the noisy meta data that make real-scene datasets costly and annotation-heavy. It instantiates this paradigm with 15k rendered items across three task families plus a 600-question held-out SB-Bench while GPT-5 scores only 43.8 and Gemini-2.5-flash 36.4, showing the rendered items are diagnostic rather than trivial. The rendered proxy also carries transferable signal rather than renderer trivia, since a recipe-matched control and an equal-size Synthetic-Real control do not transfer while training on the block tasks lifts MindCube by +6.2 to +25.1 across all four backbones.
Critical strengths
- The paper's key strength is using abstract block structures rendered in simulation by three types (3D-to-2D projection, viewpoint transformation and structural combination) to produce both training data and evaluation items that probe spatial skills, with ground truth exact by construction rather than estimated by noisy perception modules.
Critical weaknesses
- The gains are highly inconsistent across domains, in-domain SB-Bench and the conceptually nearest benchmark MindCube, yet MMSI-Bench gains marginally with all models stuck in the near-chance band. The improvements on MMSI, SPBench, MMMU may not support the motivation that foundational spatial skills could be improved by the proposed structured synthetic tasks.
- How do the authors address the perceptual ambiguity inherent in their visual stimuli? In Figure 2(c), the leftmost render can be read either as two disjoint components or as a single, already-connected structure, since isometric renders of face-touching blocks carry no explicit cue distinguishing contact between two objects from connectivity within one object. Does such ambiguity interfere with the model's spatial understanding and reasoning, and was any analysis or control performed to rule out that models exploit these ambiguous cases?
Questions for authors
The perspective imagination of real-world objects is related to much more elements, such as materials, colors, designs. Could the proposed structured blocks reflect these information correctly?