PIDScore: A Student-Centric Adaptive Fusion Method for Knowledge Distillation

Knowledge distillation aims to train lightweight student models by balancing soft supervision from a teacher model with hard supervision from ground-truth labels, making it a key challenge to determine an appropriate fusion weight. Existing methods that use fixed fusion weights lack adaptability, while another class of strategies that rely on dynamic adjustments based on teacher signals fail to fully account for the learning state of the student model itself. To overcome these limitations, we propose PIDScore, a student-centric adaptive fusion method inspired by Proportional-Integral-Derivative (PID) controller theory. PIDScore relies solely on the student's own state and the ground-truth labels, dynamically adjusting the fusion weight at the sample level and operating completely independent of the teacher model. Specifically, it employs a PID branch that relies solely on the student model, integrating student’s current prediction error (P), historical performance (I) and learning trend (D) to offer precise, fine-grained optimization guidance. As a teacher-agnostic, plug-and-play module, PIDScore demonstrates strong effectiveness and excellent generalization, consistently improving the performance of various state-of-the-art distillation methods on CIFAR-100, MS-COCO and ImageNet across extensive experiments. The code of this work will be released upon acceptance.