- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The authors put forward a hierarchical generative model which is claimed to learn almost unique (up to local monotonous stretching) latent representations. For simple self-similar data from physics, it shows a better ability to extrapolate to larger scales.
Their pipeline is as follows: They have continuous degrees of freedom on sites. They first use translation invariance of the data and apply a local/site-level non-linear transformation to make site-marginals Gaussian. In the manuscript, they commit to 2x2 site blocks. They then transformer each configuration using a two-step coarse-graining (each one realized by an MLP modelling the velocity of a normalizing flow): One shifted block transformation, producing "external magnetization" fields coming from nearby blocks, and another transformation, acting within the blocks, which again outputs 4 variables -- one computed as the mean over the block is interpreted as the collective degree of freedom to be pushed to later steps, and 3 other variables (z's) which describe the strictly local information at that scale. Their generative model assumes the z's are decoupled Gaussians. Their final loss, further encourages Gaussian behaviour of each site value of . Matching the log-likelihood, together with the latter loss, encourages the transformations to push $ x$'s (conditioned on the previous layer) and $ z$'s into their appropriate Gaussian behaviour. Finally, a Wasserstein-2 penalty based on the velocity field is added to the loss to support uniqueness.
The authors further claim to prove that for realizable distribution, uniqueness can be proven up to the aforementioned stretching.
On the numerical side the authors demonstrate that their approach beats flow matching using a U-Net (with suitable U-Net extrapolation), and looks stable across O(10) extrapolation (at least when tying the layers, namely in the repeated mode). Direct comparisons between between MC and their approach show some discrepancies but not major ones, and clearly much better than the baseline. Finally, tracking internal representations in PCA space, they find similar structures to RG flows.
Critical strengths
The paper studies an interesting qualitative problem (generative models for self-similar physics that reflect RG) and offers a valuable addition to prior work, promoting uniqueness via regulators and establishing some related mathematical results.
The method shows promise for studying critical systems and might help extract RG flows of more complex theories.
Critical weaknesses
I found the work quite hard to read and decipher. For instance, the paragraph "Hierarchical block flow" is extremely dense and often uses Jargon and references when it can simply state what is being done. It anthropomorphises objects (e.g. a decimator, which decides what each cell passes to the next scale), instead of indicating how they are defined and what pushes them to the right "decision" (it the loss objective and that's not stated clearly). It uses dramatization and narrative suspense (e.g. "Objective. This architecture realizes a density in many ways, since an invertible map applied at each site of a hidden field (hℓ or xℓ, the interfaces between layers) and undone by the next layer leaves qθ unchanged"-- how does this amount to an objective? what selects "among them" later on).
For a physics readership, perhaps the key readership of such a work, a more detailed study of the critical properties would be nice -- how far are critical exponents computed from this ansatz? How well can you estimate the critical temperature?
For an ML readership, the applications and benefits to real-world data (image, language) are not obvious. Also, this problem lacks clear benchmarks, and extrapolating the U-Net might not be the cutting-edge technology here. Images most likely contain a mix of different patch sizes as far as coarse-graining goes, and one committed patch size may cut off important features.
Questions for authors
Can you use the Sandik method (https://cpl.iphy.ac.cn/EN/abstract/article_105571.shtml) to estimate several critical exponents from your ansatz?
Readability. Both paragraphs are rewritten. "Hierarchical block flow" now defines the disentangler and the decimator as block maps applied to every cell, and states that neither is prescribed: both are learned, and the objective determines them, including which function of the four cell values the decimator keeps. "Objective" now opens with the loss, then says why the likelihood alone does not determine the block maps and what each regularizer selects.
Exponents and β_c. Our method is not designed to replace finite-size scaling or Monte Carlo for measuring exponents. Its purpose is to use the structure of the RG to interpret the learned block maps and to extrapolate without training data at the larger size, when low-dimensional structure such as self-similarity exists.
However, we can use the exponents as a further check of the extrapolation, with power-law fits at β_c over the extended models at L=128 to 1024 (Table B.11). Again, this is done without additional data at larger system sizes (no data or training beyond L=64).
The models are trained at the exact β_c, so the extrapolation assumes β_c and does not estimate it. Estimating it requires sweeping a fine grid of βs. Extrapolating generative models, like ours, might be a good way to accelerate MCMC sampling to generate the data required for this kind of estimate, but this is outside of the scope of this work.
Real-world data. Our method is targeted at multiscale physical systems, where the ideas from the renormalization group give a reason to expect low-dimensional structure across scales. As a first test on more complex data, Appendix B.8 applies the model to a passive scalar in forced 2D turbulence (TURB-Scalar, Calascibetta et al. 2025), whose scaling is intermittent and multifractal. The model successfully identifies the low-dimensional structure: at fine scales the learned decimators are nearly self-similar and evolve slowly with scale (Figure B.11). An L=128 model extended to L=256 without training there outperforms naive extrapolation of a flow-matching model (Table B.13, Figure B.12).
Errors are mean absolute log ratios to held-out data for the power spectrum and the second- and fourth-order structure functions. FM upsampled is the flow-matching model's L=128 samples interpolated to L=256.
In this experiment, the larger non-universal scales are captured less well, and the self-similar extension does not fully account for the intermittency. We do, however, see evidence of imperfect but non-trivial extrapolation (Table B.13, Figure B.12) and similarity/low-dimensional structure between scales (Figure B.11). Future work should consider combining our interpretable hierarchical model with more flexible generative models for non-universal behavior, and study extrapolation in the presence of intermittency.
Benchmarks. Because we are in the unusual problem setting of attempting to do interpretable extrapolation, not many other approaches exist. Super-resolution methods, for example, require data at the higher resolution to successfully generalize to larger sizes. If we assume there are no longer-range correlations beyond the current system size (finite correlation length), our flow-matching baseline is a simple and natural choice when trying to extrapolate convolutional architectures without additional high-resolution data. This, of course, breaks down at or near non-trivial fixed points (like the critical point), which is where our method has an advantage.
Patch size. The locality built into our model is both an advantage for interpretability and a limitation in expressivity, especially when there are highly non-local interactions. We are, in principle, able to progressively increase the receptive field by adding more disentangler layers, which our theory allows for, so this may be one way to improve expressivity in a controlled way in more complex systems.