- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates where on-policy distillation of VLMs should concentrate token-level supervision. Given that image-induced changes are distributed more broadly, this paper proposes RGB-OPD for multimodal on-policy distillation. RGB-OPD uses teacher representation sensitivity to maintain distillation in the output space, outperforming standard OPD by 0.91% for 32B to 8B run.
Critical strengths
-
Teacher representation sensitivity is used to only rank positions, and the distillation target stays in the output space. This avoids the width and layer matching problems of hidden-state alignment.
-
This paper gives honest reporting of the scores, like outperforming standard OPD by 0.91% for 32B to 8B run, together with +2.54 on MathVerse, +0.96 on VisOnlyQA, and +0.92 on WeMath.
Critical weaknesses
- The effect of representation-guided budgeting is not separated from the other changes, and the gains are small.
Relative to Standard OPD, RGB-OPD changes three things at once: the objective (sampled-token surrogate to coarse top-K reverse KL), the H-guided budget, and the termination hinge. Table 1 contains neither "Standard OPD + termination correction" nor a uniformly weighted coarse-KL baseline, so the +0.80/+0.91 gains cannot be attributed to H.
The only direct test of H is in Figure 4: seed 42, two of five benchmarks, 8B→2B only. RGB-OPD outperforms "w/o H-Budget" by 0.63 on MathVerse (about 5 of 788 items) and by 0.40 on MMStar (about 6 of 1,500). It outperforms Random by 0.89 and 0.74 points (7 and 11 items). Table 5 gives seed-to-seed SDs of the paired differences of 1.30 (MathVerse) and 1.11 (MMStar), so every one of these gaps is below 0.8 SD. This 42 can be a favorable seed, and the evidence is still within noise.
- The representation-output mismatch is measured against a narrow output-side probe, so the case for representations over the full output distribution is not made.
O uses only the sampled token's log-probability, while H aggregates relative ℓ2 changes over all layers and hidden dimensions. Appendix A.1 ("multi-token observability") argues that a broader output support exposes more of the displacement. The like-for-like comparison is a distribution-level output signal, for example, the JPEG-corrected KL between the teacher's next-token distributions under I and I_shuf, possibly on the same top-K support. That signal is not analyzed in Figure 2, and is also not tried as a selector.
RGB-OPD up-weights an output-space target at high-H positions. If the teacher's full distribution is image-sensitive there, a distribution-level output selector should find those positions without hidden states. If it is not, the up-weighted target carries little visual information, and it is unclear how matching it transfers visual reasoning.
It is important because the mismatch is the paper's first stated contribution and the justification for the cost of the probe.
Questions for authors
- Could you show the effect of representation-guided budgeting with clear separation from other changes?
- How could you justify that these small gains show the effectiveness of this proposed method?
- How does a distribution-level output-sensitivity selector compare with H, such as both in the Figure 2 overlap analysis and as a training selector with the same α and β?
- Is H correlated with the coarse discrepancy D_t or with student entropy at the same positions?
- How is OPRD-Bridge configured, e.g., which layers are aligned, how are teacher and student widths mapped, and what loss weight is used, does it include the termination correction? Because it nearly matches RGB-OPD on MathVerse at 8B→2B (25.76 vs. 25.89).