Hierarchical Normalizing Flows for Interpretable Scale-wise Extrapolation

Generative models of multiscale physical systems usually lack the inductive bias needed to generalize to larger systems than they were trained on. We study scale-wise extrapolation with a hierarchical generative model whose levels are the maps between adjacent scales. Because each level adds one scale, a model of a larger system can be constructed by adding a new level, often a copy of an existing one, rather than by retraining. This extrapolation is possible when the way the target distribution changes across scales has low-dimensional structure. The simplest case is self-similarity, which the renormalization group (RG) formalizes as a fixed point of a coarse-graining map. Our proposed model is a hierarchical normalizing flow whose levels are local, symmetry-equivariant continuous normalizing flows, trained with regularizers and a normalization that fix the reparameterization freedom, making the levels identifiable and interpretable. We characterize this freedom and give conditions under which the regularizers leave at most one optimal solution. We then test the method on two-dimensional lattice models, which are exactly self-similar at a phase transition and can be checked against exact results and Monte Carlo simulations. For the 2D Ising model, independently trained models learn the same levels, and across temperatures the learned levels trace a low-dimensional manifold reflecting the RG flow. At the phase transitions of the Ising and XY models, we show that a model trained at one lattice size extends to larger lattices by repeating one level, without new training. In both cases, the extrapolated models match models trained directly on larger systems, whereas a parameter-matched flow-matching model extrapolates poorly.