- Rating
- 2
- Confidence
- 4
- Informativeness
- 1(−1)
- Presentation
- 1(−1)
- Soundness
- 1
Read review
Paper summary
The authors propose CA-OPD, an interleaved on-policy distillation variant tailored for small VLMs that generate structured outputs such as GUI coordinates and OCR text. During student rollouts, any token assigned a probability below a threshold by the teacher is overwritten with the teacher's argmax prediction, and generation continues from the corrected prefix. This same accept-or-replace gate also governs the per-token training objective, applying cross-entropy loss at replaced positions and top- forward KL divergence at retained ones. Compared to Speculative Knowledge Distillation, the proposed method introduces three main modifications: an absolute-probability acceptance criterion instead of a rank-based one, a gating-dependent loss assignment, and an annealed threshold schedule.The authors evaluate CA-OPD by distilling two domain-specific Qwen3.5-9B teachers into an SFT-initialized Qwen3.5-0.8B student, alongside an additional Qwen3-VL-8B to 2B transfer setting. In the main experiments, CA-OPD outperforms OPD and SKD, but its gains over Offline KD are modest. The largest gain is on ScreenSpot-Pro, while OCR improvements are smaller. Results against Offline KD are mixed in the additional Qwen3-VL setting. Ablations support selective prefix correction. Threshold annealing improves ScreenSpot-Pro, but adds little on some OCR metrics.
Critical strengths
-
The paper shows why the method works and I think is quiet soundness. In Table 2, random replacement and gating without prefix writeback both stay at the OPD level. The gain appears only when confidence-based placement is combined with writeback. Figure 2(b) shows that most rejected tokens are still in the teacher's top-25, so SKD would accept them.
-
The comparisons are fair and well reported. All methods share the same initialization, data, and step budget. The stratified analysis defines its buckets before training and uses held-out rollouts to avoid selection bias. Because of this, it is also easy to see where the gains are small, especially compared with Offline KD.
Critical weaknesses
-
Improvements over Offline KD are modest, especially on OCR tasks. In the Qwen3-VL experiments, CA-OPD also trails Offline KD on several benchmarks.
-
Building upon SKD, CA-OPD introduces a probability-based gating mechanism, a gate-dependent loss function, and a threshold annealing strategy. Ablation studies demonstrate that this approach enables selective prefix correction, though the additional gains provided by the annealing strategy vary by task. Compared to existing interleaved distillation methods, its overall contribution is relatively limited.
-
Experiments cover only 0.8B and 2B students with short training schedules. Generalization to larger students and longer training remains unclear. Since the gate uses absolute teacher probabilities, its robustness to differences in teacher calibration also needs further evaluation.
Questions for authors
-
Robustness against stronger baselines: Given that plain Offline KD closely matches CA-OPD and even outperforms it on several Qwen3-VL benchmarks, can the authors demonstrate consistent gains over stronger offline objectives (e.g., reverse KL distillation [1] or LLaVA-KD [2]) and mixed-data online methods without confidence gating (e.g., GKD with [3] or DistiLLM [4])?
-
Necessity of the annealing schedule: Scheduled sampling [5] already achieves a gradual transition from teacher to student prefixes and is cited in the paper without empirical comparison. Does annealing provide a consistent advantage over a tuned fixed threshold across tasks and runs?
-
SKD comparison: Could the authors include the stronger () SKD configuration from Table 10 in the comprehensive benchmark comparisons? How much of an advantage does CA-OPD still maintain when compared against this baseline?
References
[1] Y. Gu, L. Dong, F. Wei, M. Huang. MiniLLM: Knowledge Distillation of Large Language Models. ICLR 2024.
[2] Y. Cai et al. LLaVA-KD: A Framework of Distilling Multimodal Large Language Models. ICCV 2025.
[3] R. Agarwal et al. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR 2024.
[4] J. Ko, S. Kim, T. Chen, S.-Y. Yun. DistiLLM: Towards Streamlined Distillation for Large Language Models. ICML 2024.
[5] S. Bengio, O. Vinyals, N. Jaitly, N. Shazeer. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks. NeurIPS 2015.
10 comments
Response to Weakness 2
Dear Reviewer:
Thank you for your careful reading and constructive feedback.
Contribution beyond interleaved generation. CA-OPD connects two decisions that arise together in structured visual prediction: which token should enter the prefix, and how the student should learn at that position. Once a coordinate digit or character is committed, subsequent predictions depend on it. Our design uses the same teacher-support decision to govern both prefix construction and token-level supervision.
SKD uses teacher rank to accept or replace proposals and applies a common divergence objective across positions (SKD, Alg. 1). CA-OPD instead uses absolute teacher probability: rejected proposals receive teacher-argmax writeback and CE toward the committed target, while retained proposals receive distributional KL. Thus, verification also determines the form of supervision.
The ablation in Table 2 directly tests this additional role. Keeping the probability gate and argmax writeback unchanged, replacing rejected-position CE with KL reduces ScreenSpot-Pro from 45.92 to 45.26 and OCRBench-v2 English from 52.47 to 51.75:
| Rejected-position supervision | SS-Pro | OCRv2-EN |
|---|---|---|
| CE (CA-OPD) | 45.92 | 52.47 |
| KL | 45.26 | 51.75 |
This supports the value of gate-dependent supervision beyond trajectory modification alone.
Our fixed-prefix diagnostic further examines the local consequences of probability-versus-rank disagreement. On SFT-init rollouts from 128 randomly sampled training inputs per domain, we select the first actual-change position per sequence in each disagreement group. From the same preceding prefix, we either retain the student proposal or write back the teacher's argmax, then continue greedily without further intervention. At α≈0.25:
| Gate disagreement | Domain | Pairs | Mean replacement benefit [95% CI] |
|---|---|---|---|
| CA rejects, K=25 accepts | Grounding | 125 | +68.70 [27.60, 115.04] |
| CA rejects, K=25 accepts | OCR | 80 | +0.0590 [0.0068, 0.1202] |
| K=1 rejects, CA accepts | Grounding | 42 | +6.44 [−26.90, 40.63] |
| K=1 rejects, CA accepts | OCR | 44 | −0.0043 [−0.0332, 0.0303] |
Table note. Benefit is distance reduction in 0–1000 coordinates (grounding) or increased 1−NED against the training reference (OCR), intervals are descriptive 95% paired-bootstrap CIs. Both arms share a 1024-token cap.
At proposals admitted by K=25 but rejected by CA, replacement improves mean downstream outcomes. At supported alternatives retained by CA but rejected by K=1, additional replacement shows no clear average benefit. These local results illustrate why the verification criterion matters, while the CE-to-KL ablation tests the additional use of verification to assign supervision.
Annealing varies this joint decision over training, changing both which proposals are retained and where CE versus KL is applied. We present the fixed-threshold and schedule comparisons in our response to Question 2.
Response to Weakness 3
Dear Reviewer:
Thank you for your careful reading and constructive feedback.
Scope and teacher calibration. Our experiments target distillation into compact students, covering Qwen3.5-0.8B and Qwen3-VL-2B with their respective teachers. The additional Qwen3-VL setting extends the evaluation beyond the main model configuration: CA-OPD outperforms both OPD and SKD on five of the six GUI/OCR benchmarks. Our empirical conclusions concern these student sizes and the evaluated training budgets, larger students and substantially longer CA-OPD training remain outside the present evaluation.
For teacher confidence, we assess whether the support signal leads to useful interventions through downstream outcomes. In our fixed-prefix diagnostic at α≈0.25, teacher-argmax replacement improves grounding click success by 13.60 percentage points [6.40, 20.80] and OCR 1−NED by 0.0744 [0.0210, 0.1345], with descriptive 95% paired-bootstrap intervals. Both branches use the same preceding prefix and continuation budget.
These results provide direct evidence of intervention utility under the evaluated teachers. We interpret the probability floor as a teacher-support threshold rather than a calibrated probability of task correctness. Absolute-probability gating can nevertheless be affected by changes in teacher calibration, Appendix I discusses the potential need to retune the floor under systematic teacher–student mismatch.
Response to Weakness 1
Dear Reviewer:
Thank you for your careful reading and constructive feedback.
Gains over Offline KD. Under the same initialization, training data, and optimizer budget, CA-OPD achieves higher mean scores than Offline KD on all six GUI/OCR benchmarks in the main setting (Table 1). The improvements are +0.91 on OCRBench-v2 English, +0.31 on OCRBench-v2 Chinese, and +0.33 on CC-OCR, alongside +0.45 on ScreenSpot-v2 and +1.69 on ScreenSpot-Pro.
The aggregate OCR scores also encompass different capabilities. In Appendix D.1, CA-OPD achieves higher mean scores in all eight OCRBench-v2 capability categories, with gains ranging from +0.09 to +2.29 points. The largest improvements occur in Spotting (+2.29) and Referring (+1.39). On ScreenSpot-Pro, improvements span all six application categories, ranging from +1.03 to +2.17 points. The tables below reproduce these results, averaged over three training runs.
OCRBench-v2 by capability category (Table 7).
Referring, Spotting, and Calculation are English-only.
ScreenSpot-Pro by application category (Table 6).
These results show that the main-setting gains extend across multiple capabilities and applications. On OmniDocBench, CA-OPD preserves performance close to the SFT initialization: 70.43 versus 70.49, compared with 70.40 for Offline KD.
In the additional Qwen3-VL setting (Table 16), CA-OPD scores below Offline KD on ScreenSpot-Pro, CC-OCR, and OCRBench-v2 Chinese by 0.28, 0.29, and 0.46 points, respectively. Meanwhile, it exceeds both OPD and SKD on five of the six GUI/OCR benchmarks, including gains of +1.96 and +1.78 on ScreenSpot-Pro. Thus, the evidence supports improvements over Offline KD across the evaluated categories in the main setting, while the additional setting demonstrates improvements over on-policy baselines with task-dependent performance relative to Offline KD.