HARMONY: Adaptive Self-Guidance through Hierarchical Subnetwork Probing
Abstract
Guidance improves diffusion generation, but its quality and cost depend on the auxiliary prediction used to construct the correction. We introduce HARMONY, which obtains complementary guidance signals from a diffusion transformer's own hierarchy and adapts their influence across time and space. An early readout of the residual stream supplies a weak prediction without repeating transformer blocks, while an optional block-dropped pass provides a second direction. Agreement between these probes controls when and where guidance is applied. A conditional error model relates population agreement to optimal shrinkage and explains the effects of unequal probe quality and correlated errors. Across the reported ImageNet, text-to-image, and video evaluations, HARMONY improves generation quality while its Lite variant requires only two additional readout-head evaluations. On DiT-XL/2, the full method reduces FID from 2.27 to 1.93 with increased recall. The framework connects the choice of a weak prediction with adaptive guidance strength through information already available during inference.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.