acceptodds
Under review as a conference paper at ICLR 2027

MaRR: Map-Based Rotation Reasoning for Vision–Language Models

Abstract

Forward mental rotation requires predicting an object’s unseen pose after a specified rotation, rather than recognizing a visible configuration. Our controlled diagnostics show that frozen vision– language model features contain decodable rotation information and support a supplied transformation, while inferring and reusing transformations across objects remains unreliable. Building on this insight, we introduce MaRR (Map-based Rotation Reasoning), a teacher-guided self-distillation framework that combines object- adaptive representations with a shared, object-independent transformation procedure. Using teacher traces, MaRR teaches VLMs to map object-specific features, rotate their spatial relations, and match the predicted configuration to candidate images. Fine- tuning Qwen3.5-4B with this supervision improves held-out CAD accuracy, with gains extending to shifted CAD renderings and block objects, although transfer to public benchmarks remains selective. A matched comparison shows that trace design introduces tradeoffs: natural teacher responses yield stronger OOD transfer and physical-choice consistency, while standardized MaRR traces achieve higher accuracy under candidate permutation. Together, these findings suggest that structured supervision can help VLMs organize and use their latent spatial knowledge, while the way the procedure is expressed in teacher traces affects transfer and robustness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, spatial reasoning, mental rotation, 3D understanding, self-distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2ySEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies forward mental rotation in vision-language models (VLMs). The task requires predicting an object’s unseen pose after a specified rotation. The paper first conducts diagnostic experiments on frozen visual features and learned transformation operators. Then, a map–rotate–match supervision framework (MaRR) is proposed, which tracks object-specific geometric cues through a shared transformation procedure. In order to train VLMs, a training data generation process is developed in the CAD field that includes a strong teacher model. Experiments are conducted on the constructed evaluation set and open-source dataset.

Critical strengths
  1. The task is clearly defined. Fixing the camera and world frame helps distinguish commanded object rotation from viewpoint changes and object recognition. The object-disjoint splits and same-object distractors provide useful controls against simple identity matching.
  2. The paper conducts diagnostic experiments to reveal task characteristics, which are used to guide the construction of training data.
Critical weaknesses
  1. The paper studies the forward mental rotation task, but does not analyze the connection and differences between this task and related tasks in existing spatial understanding. Meanwhile, the paper mainly construct a dataset in the CAD field, but does not specify the significance of this task for the field.
  2. The paper lacks innovation and only constructs an SFT dataset, and then trains the Base model (Qwen3.5-4B).
  3. Severe degradation on a public rotation benchmark. Table 2 shows that MaRR-SFT drops from 52.3 to 29.4 on SpinBench Forward, and from 23.5 to 20.0 on SRBench Hard.
  4. The Related Work section mainly introduces spatial understanding methods, without involving research on forward object rotation.
  5. Writing and formatting issues reduce clarity.
Questions for authors

See Weaknesses.

Reviewer_C93xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Map-Based Rotation Reasoning (MaRR) to improve VLMs' forward mental rotation ability. It first conducts controlled diagnostic experiments to investigate the limitations of existing VLMs in spatial reasoning, followed by a teacher-guided supervision pipeline. Extensive experiments demonstrate improvements on the constructed datasets, although generalisation to public benchmarks remains inconsistent.

Critical strengths
  1. The paper addresses a well-motivated problem. The authors conduct controlled diagnostic experiments on VLMs' spatial reasoning, and separate the readable rotation information, supplied transformation action, and inferred cross-object reuse. Through the experiments, they show interesting results for understanding the limitation of 3D reasoning for VLM.

  2. The proposed teacher-supervised training pipeline achieves promising results. With only 2.1k training samples, the Qwen3.5-4B model achieves an 30% improvement on held-out CAD objects, with additional gains on OOD CAD and Block Objects.

Critical weaknesses
  1. The paper primarily introduces a structured SFT approach rather than a novel 3D representation or neural architecture, which somewhat limits its methodological novelty. More importantly, the ablation results show that Natural Direct Teacher Reasoning outperforms Explicit MaRR in both accuracy and OOD generalisation. This raises questions about the specific benefits of enforcing the Map–Rotate–Match structure over conventional teacher-generated reasoning traces.

  2. The evaluation relies primarily on multiple-choice accuracy, which may not adequately reflect genuine 3D spatial understanding. For example, the model might learn correlations between 2D visual cues and rotation labels rather than performing actual spatial transformations.

  3. Generalisation across benchmarks remains limited. While MaRR achieves significant improvements on held-out CAD objects, its performance on public benchmarks is inconsistent and sometimes deteriorates, particularly on SpinBench Forward Rotation and SRBench Hard.

Questions for authors
  1. Given that Natural Direct outperforms Explicit MaRR on several benchmarks, could the authors clarify the unique contribution of structured Map–Rotate–Match supervision? An answer-only SFT baseline and a length-controlled CoT comparison would help isolate its benefits.

  2. Could the authors provide a deeper analysis of why the learned reasoning does not generalise consistently across public rotation benchmarks?

Reviewer_t7QGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MaRR teaches forward mental rotation through a map–rotate–match reasoning procedure. It constructs teacher traces using staged visual reasoning, program-checked rotations, and stronger-model supervision, then fine-tunes Qwen3.5-4B. Experiments show substantial gains on held-out CAD objects, with selective transfer to other rotation benchmarks.

Critical strengths
  1. Controlled diagnostics distinguish decodable visual information, supplied transformations, and inferred cross-object transformation reuse.
  2. Object-disjoint evaluation and the matched comparison between standardized and natural teacher traces provide useful experimental controls.
  3. The method improves CAD accuracy while substantially shortening generated responses.
Critical weaknesses
  1. The independent benefit of the proposed method is unclear. Under matched training conditions, natural teacher responses achieve higher accuracy than standardized MaRR traces on CAD, OOD CAD, and block objects, including 47.3% versus 38.7% on OOD CAD (Table 5). Although MaRR achieves higher accuracy after candidate permutation, natural supervision yields better physical-choice consistency. These results weaken the case for the elaborate trace-construction pipeline and leave unclear what it contributes beyond straightforward teacher-response distillation.

  2. External baseline comparisons are difficult to interpret. MaRR receives task-specific fine-tuning, whereas external baselines are evaluated zero-shot. Several baselines also suffer substantial truncation or format failures; for example, 270/300 DeepSeek CAD responses are truncated. Although these issues are documented, they limit conclusions about relative reasoning ability. A matched answer-only SFT baseline and more reliable baseline response completion would help separate procedural learning from task adaptation and output-format compliance.

  3. Generalization and prediction stability remain limited. MaRR reduces SpinBench forward-rotation accuracy from 52.3% to 29.4% and also underperforms its base model on SRBench Hard. On the held-out CAD permutation test, only 58.0% of predictions select the same physical candidate across orders despite similar aggregate accuracy. These findings limit the evidence for reliable, transferable rotation reasoning.

  4. Insufficient technical depth and literature positioning. The diagnostics motivate procedural supervision but do not directly establish why the particular object-adaptive map–rotate–match design is necessary. The related-work discussion provides limited methodological comparison with prior visual program distillation and structured spatial reasoning approaches. Moreover, evaluated methods such as 3DThinker and SpatialStack lack corresponding bibliographic references. The paper needs a more substantive explanation of its technical distinctions and stronger evidence validating those distinctions.

Questions for authors
  1. Which components of the MaRR pipeline provide benefits beyond natural teacher-response distillation, particularly when trace length and content are controlled?
  2. Do the gains persist against matched answer-only SFT and under evaluation settings that reduce baseline truncation and format failures?
  3. What drives the public-benchmark regressions, and do the candidate-order findings hold across multiple independent permutations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.