- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes SAGE, a reinforcement learning framework to improve spatial reasoning in vision-language models.
Critical strengths
The idea of self-evolving post-training is interesting.
Critical weaknesses
-
The paper is quite hard to read and follow. The method section is especially hard to follow, with terminologies such as "original–dual pairs", "canonical answer space", "extraction failure", "bijective answer transformation", "context–proposal examples (h,y)" are introduce but not clearly defined or explained. Due to the low readability of the method section, it's hard to provide constructive feedback on the methodology. The method figure (figure 2) is especially hard to read and to understand the method. There is too much information and mathematical equations in very small fonts. It will be more readable if it (1) presents the overall workflow without every mathematical detail, and (2) present the most novel and important designs of the method compare to prior methods.
-
It is unclear whether the experiments sufficiently support the main claims. The authors also combine literature-reported results with their own evaluations. Some closely related methods, such as SVQA-R1 and SpatialEvo, are discussed but not directly compared with SAGE. Without adapting existing methods to the same data and training conditions, it is difficult to assess SAGE's improvements over prior approaches. Additionally, the evaluation metrics are not clearly defined in the main paper. For example, it is hard to interpret what "VSI-Bench (+1.6–6.7)" means without consulting the appendix. Overall, the low readability of the experimental section makes it difficult to provide feedback.
Overall, in its current form, the paper has substantial issues with readability and experimental validation that need to be addressed before it is ready for publication.
Questions for authors
Additional questions related to W1:
- Can the authors provide clear definitions and intuitive examples for the above notions, rather than relying primarily on mathematical equations?
- What is the motivation behind each major method component? How does each component address limitations of prior methods?
Additional questions related to W2:
- Table 3 compares SAGE with fixed transformations and without online discovery. Can the authors provide a comparison between fixed and self-evolving transformations? Is adaptation the only difference between these settings?
- How does the choice of VLM for proposing duality operations affect SAGE's performance? What happens when using different VLMs as proposers?
9 comments
Response to Reviewer osu4 Part 2
W2. Direct comparison and evaluation conditions
We appreciate the reviewer’s concern about combining literature-reported scores with our own evaluations. We initially relied on some reported results because reproducing the full evaluation required substantial implementation and computational effort. Following this suggestion, we have rerun those evaluations under our protocol and replaced the reported scores with our own results.
We also added direct comparisons with the two suggested methods. We reproduced SVQA-R1 on Qwen2.5-VL-7B and evaluated the released SpatialEvo-7B checkpoint using the same evaluation protocol as SAGE.
| Method | VSI-Bench | ViewSpatial | VideoMMMU | MMVU (MC) | MVBench | TempCompass | VideoMME | CV-Bench | SPAR | MMSI | MindCube | OST | OmniSpatial | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-7B (base) | 32.5 | 37.4 | 46.1 | 59.2 | 59.1 | 70.4 | 50.6 | 74.2 | 21.8 | 26.4 | 29.5 | 38.1 | 23.7 | 43.8 |
| SVQA-R1-7B | 32.0 | 37.1 | 48.9 | 59.0 | 60.4 | 70.8 | 52.4 | 74.0 | 22.6 | 26.2 | 30.7 | 38.9 | 24.2 | 44.4 |
| SpatialEvo-7B | 41.1 | 42.8 | 48.2 | 58.2 | 59.3 | 68.2 | 52.4 | 75.3 | 31.7 | 29.2 | 33.3 | 38.9 | 30.7 | 46.9 |
| SAGE-7B | 37.9 | 44.1 | 53.4 | 65.8 | 68.1 | 78.4 | 66.5 | 77.2 | 34.4 | 32.3 | 44.5 | 46.3 | 38.3 | 52.9 |
SAGE achieves an average of 52.9, compared with 44.4 for SVQA-R1 and 46.9 for SpatialEvo. It outperforms SVQA-R1 on all 13 benchmarks and SpatialEvo on 12, with SpatialEvo scoring higher on VSI-Bench. The revised related work explains how SAGE's evolving operation pool and correctness-gated consistency reward differ from these approaches. We clarify the source and evaluation settings of each result in the appendix.
W2. Meaning of the reported metrics
The reported VSI-Bench range summarizes absolute gains across three backbones. These are +5.4 for Qwen2.5-VL-7B, +1.6 for Qwen3-VL-4B, and +6.7 for Qwen3-VL-8B.
VSI-Bench equally averages multiple-choice accuracy and numerical mean relative accuracy. The latter averages correctness across relative-error thresholds. Our ablation average weights all 13 benchmarks equally. We clarified the gains in the main results discussion and VSI/MMVU scoring in the table caption. The appendix gives full metric definitions.
W2 Q1. Fixed versus self-evolving transformations
Table 3 tests operation selection in Panel C and online pool evolution in Panel D. Panel C changes which available operations are used, while Panel D changes whether the pool can evolve.
| Setting | Operation use | Average score |
|---|---|---|
| Fixed-all | Apply every applicable operation | 49.3 |
| Random- | Sample up to operations randomly | 47.0 |
| Adaptive- | Balance usage across operations | 49.8 |
| Fixed initial pool | Disable discovery and retirement/replacement | 44.6 |
| SAGE | Select by probe feedback and evolve the pool | 52.9 |
The 44.6 setting fixes the initial operation pool by disabling discovery and retirement/replacement. It retains the same backbone, initialization, training data, and reward. It tests pool evolution as a whole, rather than discovery alone. We clarified its label and protocol. Removing only retirement gives 48.6, and removing only reactivation gives 50.2.
Random- and Adaptive- share SAGE's active-capacity limit and solver rollout budget. Fixed-all uses more operations, costing 42.8 GPU-hours versus SAGE's 26.8.
W2 Q2. Different VLMs as proposers
We evaluated Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B as the shared proposer and solver. The following table summarizes the main results across all 13 benchmarks.
| Shared proposer and solver | Base avg. | SAGE avg. | Gain |
|---|---|---|---|
| Qwen2.5-VL-7B | 43.8 | 52.9 | +9.1 |
| Qwen3-VL-4B | 46.0 | 55.7 | +9.7 |
| Qwen3-VL-8B | 48.4 | 58.9 | +10.5 |
SAGE improves every benchmark for each of the three VLMs. These results show that the shared proposer–solver design works across the tested model families and sizes.
Using separate proposer and solver models achieves the same average score as sharing one model, but requires more computation and memory. The relevant results from Table 3 in the main paper are shown below.
| Configuration | Average score | GPU-hours | Peak memory (GiB/GPU) |
|---|---|---|---|
| Separate proposer and solver | 52.9 | 39.0 | 79.8 |
| Shared proposer and solver | 52.9 | 26.8 | 76.2 |
Sharing provides a simpler architecture with one trainable model, reducing training cost by 31.3% in GPU-hours and peak memory by 3.6 GiB per GPU, while retaining the same average performance.
We especially appreciate the suggestions that helped clarify the manuscript and its comparisons, and welcome further discussion.
Dear Reviewer osu4,
Thank you for your insightful comments and suggestions. We have carefully addressed the raised concerns in our rebuttal and provided additional clarifications and experimental results where applicable. We would greatly appreciate it if you could take some time to review our responses.
Thank you again for your time and consideration.
Best regards,
Authors
Response to Reviewer osu4 Part 1
We sincerely thank the reviewer for the careful reading and detailed suggestions. Your feedback has guided our revisions to the method presentation and experimental comparisons. Below, we clarify the key definitions and design choices, present the added comparison, and address each question.
W1 Q1. Definitions, examples, and the method figure
The revised Figure 2 highlights the workflow, evolving pool, and correctness gate. We enlarged key labels and simplified the connections between probing, operation evolution, and consistency training. Detailed optimization equations now appear in the text and appendix rather than the figure.
Consider a cup to the left of a plate in the camera view. Horizontal reflection places it on the right. This example illustrates the terms below.
Applicability conditions specify when a mapping is valid, here for left/right in the camera view. Answering “left” on both images is correct on the original but inconsistent across the pair. The proposer outputs an operation y, while the solver answers the task.
W1 Q2. Component motivation and distinction from prior work
SAGE adapts its transformation pool to the solver and uses paired predictions to guide learning. Probing finds inconsistently handled relations, guiding operation selection. Discovery adds operations, while retirement and rechecking keep the pool responsive as the solver changes. The correctness gate prevents rewarding consistency between incorrect answers.
The submitted paper already compares methods across 13 benchmarks, including published work such as Video-R1, VideoRFT, and SpatialLadder. Although ICLR's guidelines do not require comparisons to work available solely on arXiv [3], we appreciate the relevance of the suggested works and have expanded our comparison.
In their arXiv versions, SVQA-R1 [1] uses view-consistent rewards with prescribed spatial perturbations, while SpatialEvo [2] evolves question generation and solving using geometric environments for supervision. SAGE proposes executable transformations of existing labeled examples, with explicit answer mappings and applicability conditions, and uses solver feedback to manage them. The revised Stage 2 explains each component's role before its optimization details.
References
[1] Wang P, Ling H. SVQA-R1: Reinforcing spatial reasoning in MLLMs via view-consistent reward optimization[J]. arXiv preprint arXiv:2506.01371, 2025.
[2] Li D, Zhao Y, Cheng X, et al. SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments[J]. arXiv preprint arXiv:2604.14144, 2026.
[3] ICLR 2027. Reviewer Guidelines. FAQ on comparisons with recent and non-peer-reviewed work.