- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a localization and observation tradeoff in visual on policy self distillation. A teacher conditioned on a relevant image crop can improve fine detail recognition while weakening localization in a student that receives the complete image. The authors connect teacher and student token probability ratios to coefficients in divergence gradients and analyze localization and observation spans in verified and model generated reasoning trajectories. Their proposed method, PA OPSD, attenuates strongly negative crop teacher log ratios through sigmoid gating and introduces a second teacher conditioned on the complete image with an annotated evidence box. The two signals define a fused target distribution for student distillation. Experiments use Qwen3.5 models with four and nine billion parameters on visual reasoning and grounding benchmarks. Component ablations, alternative teacher designs, annotation audits, and diagnostic analyses accompany the evaluation. The reported results suggest that the combined intervention preserves observation benefits while recovering localization performance without visual operations during inference.
Critical strengths
-
The paper identifies a concrete failure mode of crop based distillation and evaluates it on dedicated grounding benchmarks, rather than relying solely on aggregate visual question answering accuracy.
-
The experimental analysis includes component removal, alternative calibration rules, alternative localization teachers, and a human annotation audit. These provide useful evidence that both proposed components contribute within the tested setup.
Critical weaknesses
-
Omitted context is confounded with other crop effects. Equation (5)'s mutual information interpretation lacks an established joint distribution linking the two input conditions. Cropping also changes resolution, tokenization, and framing. Appendix C checks parameter discrepancy under identical crops, not these effects. Without controlled observation changes, the analysis does not isolate omitted context as the cause of localization degradation.
-
Negative gradient coefficients do not prove behavioral suppression. Appendix B derives a valid but nonunique scalar coefficient representation. Actual probability updates depend on the summed gradients, normalization, and interactions across tokens and contexts. Sections 4.3 and 6.4 infer suppression and correction from log ratios without measuring these updates. Their mechanistic interpretation therefore exceeds the derivation.
-
Negative tail ratios do not identify unreliable supervision. Comparing tail probabilities for LOC and OBS omits category prevalence, OTHER tokens, and correctness. Verified references show some valid localization receives negative signals, but not that training tail tokens are mostly correct. Appendix D.1 also uses greedy decoding rather than policy sampling. Negative magnitude is therefore not established as a reliable reason to weaken correction.
-
Calibration may preserve confident errors as well as valid localization. Equation (6) drives extremely negative corrections toward zero, and fusion normalization can alter individual signals. Section 6.4 usefully shows positive mean observation ratios on verified correct trajectories, but does not test adequate correction of genuinely incorrect tokens or preservation of individual valid tokens. Selective protection requires correctness conditioned evaluation within generated negative tails.
-
A simpler box teacher alternative is missing. Tables 1 and 2 never evaluate the box teacher alone, so they cannot establish that retaining crop supervision is necessary. Table 3 supports calibration within the fused design, but not whether replacing the crop teacher avoids the need for repair. Comparisons with Section 2.1's selective distillation approaches are also absent. The full combination works, but its necessity relative to simpler privileged grounding remains unresolved.
Questions for authors
See Weaknesses.