- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Spatial-OPSD for improving spatial reasoning in vision-language models through privileged-context on-policy self-distillation. A teacher initialized from the same checkpoint as the student receives question-specific geometric information, while the student sees only the original visual question. Training minimizes token-level reverse KL on student-generated responses without using task-answer labels in the objective. The teacher is frozen within each round and refreshed from the student between rounds. Experiments evaluate one-round training across four 4B model families and three-round training on SenseNova-SI-Qwen3-VL-8B, using five spatial benchmarks. Additional analyses examine teacher-refresh schedules, distillation objectives, and numerical noise in the spatial priors.
Critical strengths
-
A useful application of privileged spatial information. Using native or estimated geometry to supervise a model without directly optimizing against task-answer labels is a practical direction. The resulting student requires no geometric tools or privileged inputs at inference.
-
Improvements in some settings. Table 1 reports average gains of approximately 3.97 and 2.94 percentage points for Qwen3-VL-4B and InternVL3.5-4B, respectively. The matched three-round comparison in Table 3 also provides evidence that refreshing the teacher helps beyond simply extending fixed-teacher training on Qwen3-VL-4B/SPAR-Bench.
Critical weaknesses
-
The contribution beyond existing OPSD is not sufficiently established. The method largely follows privileged-context OPSD, with question-specific spatial information replacing privileged solutions and the teacher refreshed between rounds. This is a reasonable adaptation, but the experiments do not clearly establish the value of the proposed training procedure over simpler ways of using the same geometric information. In particular, a relevant missing baseline is self-SFT: the same model receives the spatial priors, generates responses without access to GT answers, and those responses are used to fine-tune the student on the original inputs. The answer-supervised SFT and GRPO results in Table 4 do not resolve this question because they use different supervision. These omissions make it difficult to assess how much methodological value the proposed recipe adds.
-
The evidence for broadly effective, recursive improvement is incomplete. In Table 2, three rounds improve SenseNova-SI-Qwen3-VL-8B from 53.54 to 54.78 on average. Approximately 81% of the net gain comes from SPAR-Bench; the other four benchmarks improve by only 0.30 percentage points on average. In Table 1, Gemma and LLaVA gain only about 0.42 points on average, with ViewSpatial regressions of 7.16 and 1.95 points. More importantly, the experiments do not directly validate the proposed mechanism of repeatedly restoring an informative teacher–student advantage. Table 2 omits intermediate-round results for the 8B model, and Figure 4b provides training-set diagnostics rather than tracking teacher and student performance across rounds. The teacher-refresh comparison is restricted to Qwen3-VL-4B/SPAR-Bench. Thus, the results support the usefulness of the recipe in certain settings, but do not establish whether successive rounds reliably produce additional gains or why the supervision remains effective as the model improves. Uncertainty estimates are also needed to interpret the small changes.
-
Limited methodological novelty relative to prior visual self-distillation. Vision-OPD [1] already uses a privileged visual input—an evidence-centered crop—to construct a teacher from the same model and supervise a full-image student through token-level distillation on student-generated trajectories, without task-answer labels in the distillation objective. It also investigates teacher-update instability and compares fixed, unregularized, trust-region, and EMA teacher strategies (Section 4.3.1). Relative to this prior framework, Spatial-OPSD primarily changes the privileged input to question-specific geometry and adopts round-wise teacher refresh. These differences may be useful, but the paper neither discusses this closely related work nor sufficiently establishes a substantive methodological advance over it, particularly without a comparison against an EMA teacher using the same spatial priors.
[1]:Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation, https://arxiv.org/abs/2605.18740
Questions for authors
-
Controlled comparison: How does Spatial-OPSD compare with self-SFT using responses generated by the same model with the same spatial priors, without GT answers? Matching the initialization, training data, and training budget would help determine the added value of on-policy distillation.
-
Clarification and mechanism validation: Could you report the five-benchmark results after each round for SenseNova-SI-Qwen3-VL-8B, with uncertainty estimates, and show whether the privileged teacher retains an accuracy advantage at each round on held-out examples? This would clarify whether the final improvement reflects repeated progress and support the proposed explanation.
-
Additional generalization evidence: Are the improvements retained on an additional spatial or embodied reasoning evaluation, such as EmbSpatial, the spatial subsets of BLINK, or ERQA? Given the concentration of gains and the observed regressions, such evidence would help assess how broadly the method improves spatial ability.
4 comments
4. Round-wise improvements, held-out teacher advantage, and three training seeds
We thank the reviewer for the insightful questions about continued teacher improvement. We extended the Qwen3-VL-4B round-wise analysis across all five benchmarks:
| Strategy / round | SPAR | MindCube | MMSI | ViewSpatial | VSI | Avg. |
|---|---|---|---|---|---|---|
| Shared R1 | 44.788 | 31.810 | 29.000 | 40.700 | 55.730 | 40.406 |
| Round-wise R2 | 47.464 | 31.143 | 29.500 | 41.820 | 55.943 | 41.174 |
| Round-wise R3 | 47.636 | 32.858 | 29.500 | 41.750 | 56.187 | 41.586 |
| Fixed Teacher R2 | 44.170 | 33.620 | 28.200 | 40.472 | 53.268 | 39.946 |
| Fixed Teacher R3 | 43.168 | 37.048 | 28.500 | 40.350 | 50.004 | 39.814 |
The round-wise five-benchmark average increases from 40.406 → 41.174 → 41.586, exceeding fixed-teacher R3 by 1.772 points.
On held-out spatial questions, we compared the same model without geometry (student condition) and with privileged geometry (teacher condition):
| Diagnostic round | Teacher − Student | 95% Bootstrap CI |
|---|---|---|
| R0 | +22.865 | [18.764, 26.826] |
| R1 | +14.244 | [10.848, 17.634] |
| R2 | +8.713 | [5.670, 11.821] |
| R3 | +8.050 | [5.162, 10.914] |
The privileged teacher retains a positive advantage across all rounds and remains 8.050 points ahead of the student at R3. Thus, geometric evidence continues to provide informative supervision as the student improves.
We also report results across three independent training seeds on all five benchmarks:
| Seed | SPAR | MindCube | MMSI | ViewSpatial | VSI | Avg. |
|---|---|---|---|---|---|---|
| 42 | 44.788 | 31.810 | 29.000 | 40.700 | 55.730 | 40.406 |
| 43 | 42.372 | 34.286 | 28.700 | 40.577 | 56.629 | 40.513 |
| 44 | 43.912 | 31.524 | 29.100 | 40.087 | 55.974 | 40.119 |
| Mean ± SD | 43.691 ± 1.223 | 32.540 ± 1.519 | 28.933 ± 0.208 | 40.455 ± 0.324 | 56.111 ± 0.465 | 40.346 ± 0.203 |
The five-benchmark average is 40.346 ± 0.203 across the three seeds, demonstrating stable performance.
5. Benefits beyond SPAR training data
To assess dependence on SPAR training examples, we also trained No-SPAR OPSD with SPAR examples excluded and evaluated all five benchmarks:
| Configuration | SPAR | MindCube | MMSI | ViewSpatial | VSI | Avg. |
|---|---|---|---|---|---|---|
| Base | 35.148 | 24.570 | 28.000 | 39.010 | 55.430 | 36.432 |
| No-SPAR OPSD | 37.264 | 26.380 | 28.347 | 39.582 | 55.864 | 37.487 |
| Spatial-OPSD | 44.788 | 31.810 | 29.000 | 40.700 | 55.730 | 40.406 |
With SPAR training data removed, No-SPAR OPSD still improves on Base across all five benchmarks, raising the average from 36.432 to 37.487 (+1.06 points). This supports applicability beyond a particular training corpus.
6. Additional spatial benchmarks: BLINK and EmbSpatial
We thank the reviewer for suggesting evaluations beyond the original spatial benchmarks. We additionally evaluated Qwen3-VL-4B on BLINK and EmbSpatial:
| Benchmark | Base | Spatial-OPSD | Improvement (points) |
|---|---|---|---|
| BLINK | 71.00% | 71.50% | +0.50 |
| EmbSpatial | 75.577% | 76.346% | +0.769 |
Spatial-OPSD improves on both BLINK and EmbSpatial. These additional results further demonstrate the transfer of spatial reasoning capabilities beyond the original five benchmarks.
We thank the reviewer again for identifying the most important questions for sharpening this work. In the revision, we will cite Vision-OPD explicitly, distinguish its crop-conditioned visual learning from our multi-view geometric supervision, and clarify the differences from the solution-conditioned original OPSD. The added teacher-argmax self-SFT, EMA, round-wise, held-out, No-SPAR, BLINK, and EmbSpatial results provide further evidence of the method's effectiveness, recursive training mechanism, and applicability to additional spatial evaluations.
Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts, and the new results provide useful evidence for the effectiveness of the proposed approach.
However, my primary concern regarding methodological novelty remains unresolved. At its core, Spatial-OPSD appears to be a multi-round extension of existing privileged-context OPSD, where the privileged information is replaced with spatial geometry and the teacher is periodically refreshed using the updated student. While these are reasonable design choices, I do not see a fundamentally new learning mechanism beyond repeatedly applying the established OPSD framework.
The comparison with EMA demonstrates that freezing the teacher within each round is empirically beneficial, but this primarily validates a training schedule rather than establishing a substantial methodological contribution. Similarly, the distinction from Vision-OPD appears to lie mainly in the type of privileged information and the application domain.
I acknowledge the additional empirical evidence, including the self-SFT comparison and multi-round improvements. Nevertheless, the improvements over self-SFT and on additional benchmarks remain modest, and the requested intermediate-round results for the stronger 8B model are still missing.
Overall, while I recognize the practical value of applying privileged-context OPSD to spatial reasoning, I remain unconvinced that the proposed method offers sufficient novelty beyond a multi-round adaptation of existing techniques. Therefore, I will maintain my original rating of 2.
We sincerely thank the reviewer for the thoughtful follow-up and for recognizing the usefulness of our additional experiments. We appreciate your continued concern regarding methodological novelty and the request for intermediate-round results on the stronger 8B model.
Following your suggestion, we now provide the complete round-wise results on SenseNova-SI-Qwen3-VL-8B, an already spatially specialized model.
| Round | SPAR | MindCube | MMSI | ViewSpatial | VSI | Avg. |
|---|---|---|---|---|---|---|
| R0 (Base) | 40.8 | 73.7 | 37.7 | 51.2 | 64.3 | 53.54 |
| R1 | 44.3 | 73.8 | 37.8 | 51.3 | 64.2 | 54.28 |
| R2 | 45.5 | 74.1 | 38.0 | 51.4 | 64.1 | 54.62 |
| R3 | 45.8 | 74.4 | 38.1 | 51.4 | 64.2 | 54.78 |
The five-benchmark average improves consistently across rounds (53.54 → 54.28 → 54.62 → 54.78), with a total gain of 1.24 percentage points. These results extend our round-wise evidence to a stronger, spatially specialized model.
Regarding methodological novelty, we appreciate the reviewer's perspective and agree that Spatial-OPSD has a relatively simple formulation. This simplicity is intentional. We believe that the simplicity of a method should be distinguished from the novelty of the research problem it addresses and the contribution it makes. Our goal is not to introduce a more complicated distillation objective, but to establish a simple and effective framework for geometry-driven model self-evolution. Rather than relying on verified task-specific answers or a more capable external teacher, Spatial-OPSD transforms reusable scene geometry into privileged supervision and renews the teacher using the improved student between rounds. In this way, the teacher's advantage arises from information asymmetry rather than model capability asymmetry.
The value of this design is supported by our experiments. As reported in Point 4 of our previous rebuttal, round-wise teacher refresh achieves a five-benchmark average of 41.586 on Qwen3-VL-4B, compared with 39.814 for the fixed-teacher strategy, while privileged geometry still provides an 8.050-point held-out teacher advantage at R3. Together with the new 8B round-wise results above, these findings support the effectiveness of our framework for recursive spatial self-improvement. We believe that achieving this capability through a simple and effective design is a strength of Spatial-OPSD. We hope that its contribution can be considered in terms of the research problem it addresses and the learning capability it enables, rather than solely the complexity of its constituent operations.
We sincerely hope that these additional results and clarifications address your remaining concerns. If there are any additional questions and feedback you'd like us to consider, please let us know. Your insights are very valuable to us, and we would be happy to discuss any remaining questions.
We sincerely thank the reviewer for the careful and insightful assessment and constructive suggestions. We especially appreciate your recognition of the practical value of privileged spatial supervision without inference-time tools, as well as the encouraging results on Qwen3-VL, InternVL, and teacher refresh. Your questions about methodological positioning, Vision-OPD, and the persistence of the teacher–student advantage are valuable and motivated several additional controlled experiments.
1. Contributions beyond the original OPSD
We agree that the paradigm of student on-policy trajectories and dense token-level self-distillation originates from OPSD. Our contributions address two challenges specific to spatial reasoning.
First, the original Self-Distilled Reasoner / OPSD studies mathematical reasoning with ground-truth solutions or verified reasoning traces as privileged teacher context. Spatial-OPSD instead uses question-relevant geometry—depth, instance regions, reconstructed 3D relations, and camera poses—obtained through a perception/reconstruction pipeline and deterministically formatted as spatial context. This transfers reusable scene structure without using task-specific GT answers or answer-derived rewards in the distillation objective.
Second, we introduce round-wise recursive self-improvement: the privileged teacher remains frozen within each round, while the improved student initializes the next round's teacher and student. Reintroducing spatial evidence creates a new teacher–student information asymmetry, supporting repeated improvements. These contributions establish geometry-driven, answer-label-free spatial supervision coupled with recursive model evolution.
2. Relation to Vision-OPD: distinct research challenges and recursive teacher refresh
We thank the reviewer for pointing us to the important Vision-OPD work. We recognize its contribution to label-free visual on-policy distillation, and we will explicitly cite and discuss it in the revision.
The two methods address different starting challenges. Vision-OPD focuses on fine-grained visual perception, enhancing evidence through targeted image crops. Spatial-OPSD addresses more demanding geometric understanding across single images, multiple images and viewpoints, camera motion, and 3D relations. Since these spatial facts can be difficult to recover from visual input alone, our framework integrates a richer tool chain spanning depth, segmentation, 3D reconstruction, geometric relation extraction, and question-relevant spatial subgraphs.
Moreover, we develop recursive self-improvement through a frozen teacher within each round and teacher refresh from the improved student between rounds, rather than applying privileged evidence for a single round.
Following the reviewer's suggestion, we also compared per-step EMA teacher updates (α = 0.05) against within-round frozen teachers using matched geometry, data, and training budgets:
The within-round frozen teacher outperforms per-step EMA by 3.512 points on the five-benchmark average. This controlled comparison supports stable teacher supervision within a round and recursive updates at round boundaries.
3. Teacher-argmax self-SFT baseline
We appreciate the reviewer's suggestion to include an answer-label-free self-SFT control. The privileged teacher greedily generates pseudo-answers with spatial context, and the student learns from those answers through ordinary SFT. Spatial-OPSD instead distills the privileged teacher's token distribution on the student's own on-policy trajectories.
Spatial-OPSD achieves a five-benchmark average of 40.406, above greedy self-SFT at 39.292. This directly addresses the suggested pseudo-answer SFT alternative.