How Does Denoising Guidance Encourage Creativity in Diffusion Models?

Diffusion models can generate images that go beyond the examples seen during training, but where does this creativity come from? We show that an important part of it can emerge in the *tail of denoising* with denoising guidance. Classifier-free guidance after substantial object structure has formed can preserve pose and layout while guiding the semantic content of generated images. We analyze the tail of deterministic DDIM sampling and show that its local geometry is governed by a *conditional PCA* whose dominant directions depend jointly on the intermediate denoising state and the guidance class. Iterative denoising amplifies these directions through a power-iteration mechanism. We test these predictions using only post-training analysis of a pretrained DiT model trained on ImageNet, and observe the predicted class- and image-dependent geometry. Our results identify a simple geometric mechanism through which guidance can encourage compositional creativity in diffusion models.