- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Contrastive On Policy Distillation (COPD), a method designed to reduce reasoning length while maintaining or improving task performance. Unlike standard on policy distillation (OPD), which aligns the student model with a single teacher distribution, COPD uses two contrasting prompts that encourage light thinking (LT) and heavy thinking (HT). The teacher evaluates each token generated by the student under both prompts, and the difference between their log probabilities serves as a token level advantage signal to update the student policy. The method aims to encourage concise reasoning by increasing the probability of tokens preferred under LT and suppressing those preferred under HT, without requiring explicit correctness rewards or length penalties. Experiments across multiple model configurations and nine multimodal benchmarks show that COPD reduces output length while improving average accuracy compared to existing OPD methods.
Critical strengths
The method can reduce the generation length effectively and intuitively.
Critical weaknesses
The paper demonstrates that the proposed contrastive advantage improves evaluation accuracy, but it is unclear why the log probability ratio between LT and HT should provide supervision signals correlated with answer correctness. The authors hypothesize that the method suppresses redundant reasoning, but do not systematically analyze whether the contrastive advantage actually identifies unnecessary reasoning steps or explain how this improves correctness despite the absence of explicit correctness supervision.
The comparison between LT only and LT minus HT in Table 6 is conducted using only one teacher and student pair, Qwen3 VL 8B and Qwen3 VL 2B. Although the overall method is evaluated across multiple model configurations, it remains unclear whether the benefits of this specific contrastive formulation consistently transfer to other models. Additional ablations using alternative reference prompts or reasoning effort levels would also help establish the importance of the proposed formulation.
The paper claims that COPD encourages more efficient reasoning by suppressing redundant steps. However, the evaluation primarily focuses on answer accuracy and output length, without systematically examining which reasoning steps are removed. It remains unclear whether the method learns to selectively eliminate unnecessary reasoning or simply develops a stronger preference for shorter outputs.
Although COPD improves average accuracy, certain benchmarks exhibit accuracy degradation after training. The paper provides limited analysis of why aggressive compression hurts these tasks or under what conditions additional reasoning and verification remain necessary.
I believe there are existing baselines specifically designed to reduce generation length. It would be important to include comparisons with these methods as well.
Questions for authors
See weaknesses above.