Score Bridge Distillation: Student-Adaptive Targets for One-Step Diffusion Models

Distribution matching distillation accelerates diffusion model sampling by training a few-step or one-step student with the score difference between the teacher and student distributions, without requiring sample-wise correspondence between their generated outputs. However, the score differences evaluated at different noise levels can induce conflicting update directions for the student, causing them to partially cancel and thereby slowing convergence and limiting the final generation quality. Existing progressive distillation methods rely on a sequence of interme- diate distributions fixed before distillation; as the student evolves, this predefined path can become increasingly misaligned with its current distribution. To address this limitation, we propose Score Bridge Distillation (SBD), which periodically reconstructs teacher-guided bridge targets from the current student score estimate, thereby recalibrating the intermediate target as the student evolves while contin- uously guiding it toward the teacher distribution. SBD alternates between two stages: Bridge Score Adaptation and Bridge Distribution Matching. Bridge Score Adaptation constructs a bridge by adapting the current student score toward the teacher. Bridge Distribution Matching then trains the student to approximate this distribution. As the student distribution evolves, SBD repeatedly reconstructs the bridge distribution so that the training target at each stage remains adapted to the student’s current state. This progressive matching process reduces gradient conflicts across noise levels, enabling the student to approach the teacher distribution more effectively. Extensive experiments show that SBD achieves faster convergence and better generation quality than existing distillation methods.