acceptodds
Under review as a conference paper at ICLR 2027

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

Abstract

On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored. Recent approaches use privileged visual information, such as image crops corresponding to a question, to improve fine-grained perception, but their gains are confined to tasks that benefit from such visual zooming and require either human-annotated grounding data or external teacher models. We introduce a different form of on-policy self-distillation for MLLMs that provides the teacher with textual, spatially grounded guidance identifying the visual elements relevant to a query. We use procedurally generated scenes with automatically available object identities and spatial coordinates, enabling scalable and annotation-free post-training. The teacher uses this spatial guidance to locate and integrate evidence from multiple relevant image regions, while the student learns to reproduce the resulting behavior from the image and question alone. Our approach consistently improves performance on counting, document and chart understanding benchmarks across multiple models. Importantly, although post-training uses only synthetic scenes, the resulting improvements transfer to real-world perception benchmarks, yielding a 3.23-point gain in average performance across CVBench, V*, ZoomBench, BLINK, HR-Bench, and MME-RealWorld. These results show that spatially grounded privileged information can induce broader perceptual capabilities through on-policy self-distillation, enabling substantial synthetic-to-real transfer beyond the task and data distribution used for post-training. Code and models will be made public.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal large language models, on-policy self-distillation, structured visual perception

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_USJHOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors present WHERE-OPD, an On-Policy Self-Distillation (OPSD) framework for Multimodal Large Lange Models (MLLMs) that leverage spatially grounded guidance as privileged information for the teacher model. The authors train Qwen models on synthetic data generated procedurally without human annotation, and evaluate on various visual reasoning benchmarks. The authors claim that WHERE-OPD outperforms existing distillation methods for MLLMs.

Critical strengths
  1. The writing is clear and easy to follow.
  2. The authors evaluate models on a comprehensive suite of visual reasoning benchmarks.
  3. The synthetic-to-real transfer demonstrated by WHERE-OPD has far-reaching implications on the usefulness of the method and scalability of the training data.
Critical weaknesses
  1. The central hypothesis that spatially grounded guidance can improve the quality of privileged information requires more mechanistic explanation. See question 1 below.

  2. Base models are limited to the Qwen model family. Testing more base model architectures would support the generalizability of the method.

Questions for authors
  1. Can the authors explain why counting was selected as privileged information amongst many other visual capabilities of MLLMs? Why does counting lead to generalization to other visual skills required by the various benchmarks?
  2. Why are WHERE-OPD’s improvements on high-resolution benchmarks more modest compared to general benchmarks (Tab.1)?
Reviewer_HFuGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Where-OPD, a spatially guided on-policy self-distillation approach for multimodal large language models. A frozen teacher and a trainable student are initialized from the same checkpoint. Both receive the image and question, while the teacher additionally receives textual hints specifying the locations of target objects and their count. The student learns by matching the teacher’s next-token distributions along student-generated trajectories. Training uses 3,000 procedurally generated scenes involving a single 2D counting task, with privileged hints obtained automatically from scene metadata. Experiments on three Qwen-family models across 15 benchmarks report improvements extending beyond synthetic counting to chart understanding and other real-world visual tasks. Additional experiments compare supervision signals, hint formats, and scene-generation settings.

Critical strengths
  1. The transfer from simple synthetic counting scenes to diverse visual tasks is a useful empirical finding. Despite using only procedural counting scenes for post-training, the method improves the 15-benchmark average by 4.45, 1.76, and 1.01 points on Qwen3.5-4B, Qwen3.5-9B, and Qwen3-VL-4B, respectively (Table 1). The improvements extend to tasks such as chart understanding, hallucination evaluation, and general visual question answering. The Qwen3.5-4B results are particularly encouraging, and the main results for Qwen3.5-4B and Qwen3.5-9B are averaged over three training runs. These findings suggest that a small amount of carefully structured synthetic supervision can improve capabilities beyond the task and visual distribution used during post-training.

  2. The approach provides a simple and practical way to construct privileged supervision. Procedural scene generation makes object identities, attributes, and coordinates available by construction, allowing the complete post-training examples and hints to be generated without manual annotation or a separate model for producing the hints. The method also preserves the student architecture and requires no hints at inference time. This makes the approach practically appealing and provides a controlled setting for studying privileged information in multimodal distillation. The same-data comparisons against alternative training signals and hint formats in Table 2 are useful steps toward understanding the recipe.

Critical weaknesses
  1. The hint ablations do not fully isolate the contribution of correct spatial correspondence. The coordinate-based hints outperform answer-only hints, supporting the usefulness of the richer hint format. However, removing the final count still leaves the answer recoverable from the number of listed target instances. The existing comparisons therefore do not fully distinguish the contribution of correct image locations from that of structured target enumeration and associated differences in hint format. The additional gains could reflect benefits of enumeration as well as the use of correct spatial correspondence. The cropped-image hint baseline is also insufficiently specified, and its substantial degradation on CountQA remains unexplained. Relative to the base model, the average decline of this variant in Table 2 is driven by CountQA; its average over the other six benchmarks actually improves. Since counting requires access to all matching instances, details such as crop selection, target coverage, and whether the full image remains available are important for interpreting this result. Without these details or diagnostics of the teacher’s performance under the different hint conditions, it is difficult to determine what the comparison establishes about the relative benefits of spatial text and localized visual inputs.

  2. Missing proxy-task performance measurements limit interpretation of the training comparisons. The paper reports attention statistics and qualitative counting examples on held-out synthetic scenes (Table 9 and Figure 10), but does not report aggregate counting accuracy on the synthetic distribution. Consequently, the limited average downstream gains of GRPO in Table 2 do not distinguish insufficient learning of the counting task from successful task learning with limited transfer. The relationship between improvements on the synthetic proxy task and improvements on real-image benchmarks also remains unexamined. These missing diagnostics limit the interpretation of the training-signal comparison and our understanding of the observed transfer.

Questions for authors
  1. Interpretation and construction of the hint ablations. Could the authors compare correct coordinate hints with incorrect coordinates and nonspatial enumeration, while keeping the number of entries, answer information, and text format as closely matched as possible? This would help distinguish the additional contribution of correct spatial correspondence from that of the enumeration format. For the cropped-image hint baseline, how are crops selected and presented to the teacher? Are the full image and all target instances retained, and is the answer information comparable to that in the textual hints? Reporting the teacher’s accuracy or correct-answer probability on synthetic scenes under the different hint conditions would help assess whether the crop construction weakens the teacher’s supervision and inform the interpretation of the substantial downstream CountQA degradation.

  2. Optional comparison with real-image training. As an optional comparison, could the authors evaluate synthetic Where-OPD against the same method trained on a similarly sized subset of PixMo-Count, using comparable hint formats and training budgets? Accounting for differences in annotation quality, target-count distribution, and counting difficulty would make this comparison more informative about the practical trade-offs of the synthetic-data choice. As a further optional extension, have the authors investigated whether synthetic Where-OPD provides a useful initialization for subsequent OPSD or RL training on real-image tasks? It would be interesting to examine whether such continuation produces further improvements while preserving the broader gains, and how it compares with applying real-image training directly to the base model under a comparable total training budget.

  3. Synthetic-task performance and its relationship to transfer. Could the authors report counting accuracy on training and representative held-out synthetic samples for the base model, SFT, GRPO, and Where-OPD, evaluated without privileged hints and under comparable inference and scoring settings, alongside their downstream scores? This would help distinguish whether GRPO’s limited average downstream gains reflect insufficient optimization of the counting task or successful task learning with limited transfer. If available, paired synthetic-task and downstream results at a few training checkpoints would also help assess whether better performance on the proxy task is associated with stronger downstream transfer.

Reviewer_yFRqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Where-OPD, an on-policy self-distillation method for improving multimodal perception. A frozen copy of the initial model receives the same image and question as the student, plus a textual hint listing question-relevant objects' coordinates and their count. The student is trained on its own sampled responses using token-level Jensen-Shannon divergence and does not receive hints at inference. Training uses 3,000 procedurally generated counting scenes. Evaluation covers 15 benchmarks and three Qwen backbones, with additional studies of supervision type, hint content, scene generation, data size, teacher updates, and thinking mode.

Critical strengths
  1. The synthetic-to-real transfer is useful and empirically interesting. Training uses simple geometric counting scenes, yet improvements extend to chart understanding, hallucination-related benchmarks, and other real-image tasks.

  2. The paper includes meaningful controls and a substantial appendix. The ten-page appendix provides generator and training details, category-level results, scene/data-size/EMA studies, thinking-enabled evaluation, and attention-layer selection. This substantially improves documentation and presentation.

Critical weaknesses
  1. ViCuR already studies textual, question-relevant visual cues as teacher privilege. The contribution should be distinguished primarily through automatic spatial-hint generation and demonstrated transfer, rather than the general idea of textual visual privilege. There is no direct comparison with this closely related alternative.

  2. The evidence does not isolate the contribution of spatially correct coordinates from answer-bearing textual structure. In Table 2, the "no total" hint still enumerates exactly one entry per target object. Its list length therefore reveals the count even without the concluding answer. The comparison with an answer-only hint changes both spatial information and the form/length of answer-bearing supervision. The 72.35 versus 71.27 averages do not, by themselves, establish that correct coordinate values cause the additional benefit. A matched control retaining the target category, number of entries, template, and answer while replacing locations with unrelated coordinates would directly test this claim. Table 9's synthetic attention enrichment and the qualitative real-image examples are supportive observations, but do not replace such a control or establish the proposed transfer mechanism.

  3. Only the main Where-OPD results for Qwen3.5-4B and 9B are averaged over three training runs. All ablations and the Qwen3-VL-4B results use single runs. The main average advantages over the strongest listed alternatives are just 0.13 points for 9B and 0.16 for Qwen3-VL-4B. Without uncertainty on these comparisons, "highest average" should not imply a robust advantage across backbones. This does not negate the much larger 4B improvement.

  4. The supplied PDF promises a future code/model release but provides no accessible repository. Independent reproducibility is currently limited. Appendix A.3 also leaves the fallback LLM judge's identity, prompt, and usage rate unspecified, and does not sufficiently identify benchmark splits/subsets and image-preprocessing settings. An identical pipeline across methods is valuable, but is not a reproducible specification.

Questions for authors

see Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.