Timestep Weighting in Frozen Diffusion Classifiers: Global Partitions and Pointwise Reachability
Abstract
Diffusion classifiers aggregate class-conditional denoising losses across timesteps, often using weights selected for accuracy. Yet evaluating particular weighting rules leaves open which decisions any shared weighting can express. We characterise this decision family for frozen scorers with nonnegative weights. Linear and mixed-integer programs compare a label-aware pointwise ceiling with the best shared weighting under the same strict-margin criterion, separating individually feasible decisions from jointly feasible ones. A positive gap occurs in all six audited loss tables, spanning two model families and two datasets. On Stable Diffusion 1.5, the gaps are exactly 11 and 10 images on the original Pet and CIFAR-10 tables, respectively, and 2–3 images on higher-accuracy CIFAR-10 noise blocks with Pillow (PIL) preprocessing; each table contains 100 images. Rank-preserving controls implicate cross-timestep class-order agreement in restricting reachability. A controlled timestep-support intervention changes pointwise reachability on Stable Diffusion. In an idealised Gaussian model with correctly specified means and class-shared covariances, we characterise Bayes-preserving metrics, prove spectral and cone obstructions, and derive an exact distance to the preserving set. The finite-table audit separates fitting shortfalls from shared-family incompatibility; the Gaussian results separately characterise population partition preservation under explicit assumptions.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.