- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors present WHERE-OPD, an On-Policy Self-Distillation (OPSD) framework for Multimodal Large Lange Models (MLLMs) that leverage spatially grounded guidance as privileged information for the teacher model. The authors train Qwen models on synthetic data generated procedurally without human annotation, and evaluate on various visual reasoning benchmarks. The authors claim that WHERE-OPD outperforms existing distillation methods for MLLMs.
Critical strengths
- The writing is clear and easy to follow.
- The authors evaluate models on a comprehensive suite of visual reasoning benchmarks.
- The synthetic-to-real transfer demonstrated by WHERE-OPD has far-reaching implications on the usefulness of the method and scalability of the training data.
Critical weaknesses
-
The central hypothesis that spatially grounded guidance can improve the quality of privileged information requires more mechanistic explanation. See question 1 below.
-
Base models are limited to the Qwen model family. Testing more base model architectures would support the generalizability of the method.
Questions for authors
- Can the authors explain why counting was selected as privileged information amongst many other visual capabilities of MLLMs? Why does counting lead to generalization to other visual skills required by the various benchmarks?
- Why are WHERE-OPD’s improvements on high-resolution benchmarks more modest compared to general benchmarks (Tab.1)?