- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Evidence-Aligned Distillation, a framework designed to improve fine-grained visual understanding. It employs a crop-conditioned EMA teacher to guide a full-image student using prefixes generated by the student itself. Evidence is estimated from differences in the student's predictions between original and evidence-masked images, and distillation weights are assigned based on the positive directional agreement between teacher corrections and this evidence signal. Experiments with 4B and 9B models demonstrate improvements in fine-grained evaluation performance, despite retaining only about 6% of the original unweighted loss mass along the same trajectories.
Critical strengths
- The approach distinguishes the privileged supervision target from the evidence-based signal used to determine its weight. Ablation studies on model states, visual contrast, normalization, and clipping provide further support for these design choices.
- EAD outperforms Vision-OPD on fine-grained tasks by a large margin.
Critical weaknesses
-
Equation 8 modifies the selection of supervision signals and the overall magnitude of the distillation loss. Without controls using uniformly scaled dense losses and randomly shuffled weights, it is difficult to determine whether the improvements specifically arise from evidence alignment.
-
EAD achieves relatively modest average improvements over VAD. However, the absence of results across independent training seeds and paired uncertainty estimates makes it unclear whether these improvements over the strongest baseline are consistent and statistically reliable.
Questions for authors
-
Does EAD outperform globally downweighted dense distillation and shuffled EAD weights? These controls would separate selection from loss attenuation.
-
How stable is the EAD–VAD difference across training seeds? Paired evaluation intervals would also clarify the strength of the ranking.
-
Does the evidence reference remain useful with another masking fill or an irrelevant-region mask? Mean-color occlusion inside the ground-truth box may introduce effects beyond evidence removal.
-
What are the additional training time, memory, and annotation requirements? The 6% loss-mass figure does not measure computational savings.