- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
Authors propose to improve the spatial reasoning of MLLMs by applying representational alignment from VGGT to the hidden vision features of an MLLM. They first apply SFT and then do GRPO and observe substantial improvements. The core insight is to apply a 3D transformation to the vision features before alignment, which authors argue drives the bulk of their improvements.
Critical strengths
The experiments are relatively solid. With the exeption of the baseline highlighted in "Questions," I feel that the experiments are relatively solid and show the utility of the method.
Also, the paper is somewhat easy to follow. While I believe some improvements to clarify could be made, there is a clear flow and it can be understood with relatively little "effort".
Critical weaknesses
Incremental novelty and lack of discussion about related works:
First, this work feels relatively incremental. I feel it is somewhat obvious (given that REPA and similar methods have been around for 2 years now), that if we improve the visual representation within a language model for specific tasks (i.e. 3D), we will get better results on related tasks.
A core work that authors appear not to have cited or discussed is Yoon et al, 2025. This work introduces REPA for multimodal LLMs (using the original DINOv2) and shows solid increases in perceptual tasks. I feel that it is conceptually similar to the core insight of this paper, and should be adequately discussed before publication.
Visual Representation Alignment for Multimodal Large Language Models. Heeji Yoon et al. arXiv 2025.
Questions for authors
-
In my opinion a core reason that MLLMs are interesting is they are used for **general purpose vision tasks. ** However, the approach presented in the paper basically goes in the opposite direction, to finetune an MLLM to be useful for primarily spatial tasks while disregarding many other important visual tasks. Does the SE(3) alignment come at a significant cost for other vision tasks (i.e. chart understanding, pointing, etc)? In this case, it may simply be improving the task-specific representation while degrading general-purpose representations.
-
Authors first do an SFT stage and then a GRPO stage. If we do a FLOP-for-FLOP analysis, how much (if at all) does the GRPO outperform a continued SFT?
-
In my opinion, the fair comparison is not comparing to the base model, but comparing to the base model after SFT and GRPO, but without the REPA. I could not find this comparison in the paper. If I missed it, please point me to it and highlight it more prominently in the paper, or provide results in dsicussion.
Response to Reviewer CPiQ
Thanks for your careful reading and positive feedback on our experiments and presentation! We address your concerns below.
CPiQ-W1
REPA is a good foundation for our work. Its success motivates geometric supervision, but does not tell us how alignment should account for camera motion.
Camera motion changes the view of a scene while its geometry remains unchanged. We use SE(3) to bring camera rotation and translation into the alignment process, so that geometric supervision explicitly accounts for viewpoint changes. The gains over naive alignment across three spatial tasks also support this pose-aware design (original Tabs. 3 and 7).
CPiQ-W2
Thanks for pointing it out! We acknowledge that VIRAL is closely related to our work and should have been discussed.
VIRAL addresses the loss of useful visual information under text supervision. Our work shares this motivation, and we have added the citation and discussion to §2. However, we focus more specifically on spatial tasks, using SE(3) to account for camera motion in alignment. We also discuss the role of spatial understanding in general-purpose MLLMs in Q1.
CPiQ-Q1
We agree that general-purpose vision is important. Our focus on spatial understanding is motivated by applications such as embodied navigation and robotic inspection, which require reasoning about locations, distances, and routes. In these settings, smaller models with stronger spatial reasoning can be more useful than larger general-purpose models.
We also evaluate general visual abilities, including reading text in images, visual perception, and multimodal reasoning. Tab. 6 (original) shows some degradation after spatial training, but the declines are modest. Our SFT includes general video data from LLaVA-Hound for retaining visual capabilities.
CPiQ-Q2
Good point. We compare the two training routes on VSI-Bench using Geo-REPA-3B. For a practical comparison, we use training time on the same hardware as the budget. Both start from the same SFT checkpoint and run for 34 additional hours. Continued SFT uses the same QA data as GRPO.
We choose GRPO to improve reasoning with task QA data that lacks CoT annotations. Continued SFT on these data supervises the answers, whereas GRPO lets the model generate its own reasoning and learn from answer rewards, as in many prior works including Spatial-MLLM.
We have added this comparison to §4.4.
CPiQ-Q3
Thanks for pointing this out! The Baseline in Tab. 3 (original) and No alignment in Tab. 7 (original) both follow the same SFT, cold-start, and GRPO pipeline as Geo-REPA, without REPA at any stage. We summarize the 3B results below.
Thank you again for helping us explain our motivation and training choices more clearly. We have incorporated the discussion and revisions into the paper.