Field-theoretic diffusion models and sampling as spatial pattern formation

Although images can be viewed as large vectors, such a representation fails to exploit nontrivial spatial structure: for example, nearby pixel locations tend to have similar colors and intensities. Similarly, theoretical analyses of image-based diffusion models tend to portray the input of denoisers and score functions as a grid of numbers rather than as a continuous object. In this work, we argue that the sampling dynamics of image-based diffusion models (and related concepts, like locality and translation equivariance) are usefully understood in terms of an idealization in which space is treated as continuous, which makes them a kind of spatial pattern formation process. To make this perspective concrete, we introduce and formalize field-theoretic diffusion models, whose central objects are fields: mappings from a real-valued pixel space to a real-valued color space. We consider field-theoretic diffusion models interesting both because they model the structure of architectures with strong spatial inductive biases, like convolutional neural networks, and because they provide a bridge between diffusion model theory and the theory of spatial pattern formation. We present theoretical results concerning sampling, including exact solutions for linear model sampling dynamics and brief results on the sampling dynamics of specific nonlinear models. The continuum description yields a novel picture of sampling in terms of reaction-diffusion-like dynamics, where different 'chemicals' drift, diffuse, and interact. We illustrate this intuition using a variety of examples. Our work demonstrates the theoretical benefits of reasoning about sampling via a continuous-space description.