One Step, Provably: Distributional Guarantees for Score Distillation

Diffusion models generate high-quality samples but require many sequential network evaluations. Score distillation enables one-step generation from a pretrained teacher diffusion model, yet theoretical guarantees for its distributional accuracy remain limited. We analyze one-step score distillation using an equivalent reverse-time diffusion whose marginals match the student's forward-noised distributions. Applying this framework to Score Identity Distillation (SiD), we establish an explicit population-level total-variation bound between the forward-noised student and data distributions at a positive noise level corresponding to early stopping in the reverse-time analysis. The bound accounts for the attained SiD objective, teacher-score and score-surrogate errors, and terminal distribution mismatches, with no sampling discretization term. For bounded generators, we further construct a single ReLU score-surrogate network with explicit size bounds for any prescribed mean-square approximation accuracy, uniformly over the analyzed noise range. Finally, a Gaussian example shows that SiD can mitigate the effect of teacher-score bias and achieve smaller distributional error than exact probability-flow sampling driven by the same teacher.