- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies training as a Wasserstein gradient flow in a controlled, analytically tractable setting (Gaussian likelihood + Gaussian prior), where the posterior is available in closed form. The authors build a measurement pipeline (energy, geometry, dynamics) and validate it against exact references. They report three main empirical findings: (1) calibration error is bounded by and tracks the transport distance to the posterior; (2) the convergence rate of this calibration is set by the smallest Hessian eigenvalue; and (3) the generative-vs-discriminative crossover occurs only under misspecification. The paper positions itself as a foundational first step before scaling to real networks.
Critical strengths
The controlled setting is a sensible choice for validating measurement tools, and the use of closed-form posteriors as ground truth is methodologically clean.
Critical weaknesses
Line 032: "Modern studies of training artificial systems tend to look at one thing at a time." This is not correct. Modern DNNs can be trained with multiple combined losses. Line 037: " These are usually studied with separate tools, and their interactions, where the interactions of energy, geometry, and dynamics have not been addressed." This is not correct either.
The theories are all well-established. The paper largely empirically confirms known results in a Gaussian toy setting.
All results are in the Gaussian/quadratic regime.
The paper's claimed contributions are overstated, and the motivation is not clear.
Questions for authors
Fig. 1: Is this landscape a real landscape of a network, or from toy data, or AI-generated?
We thank the reviewer for their time and for acknowledging that using closed-form posteriors in a controlled setting is a methodologically clean way to validate measurement tools.
On Motivation and "One Aspect at a Time" (Lines 032–037): The main goal of our research program is to study learning dynamics through the simultaneous interaction of dynamics, geometry, and energy—for example, to ask how landscape geometry and free-energy dissipation interact during early memorization or generalization loss. The Line 032 "Modern studies of training artificial systems tend to look at one thing at a time," points to this general aim of the whole project and the lack of such a coherent framework of study. What we meant was that the energy (loss surface), the dynamics (optimizer trajectory), and the geometry (metric/preconditioner) are typically analyzed with separate theoretical tools, whereas the Wasserstein gradient flow / JKO framework couples all three into a single variational object. Though it is true that different studies have examined training modern DNNs with multiple loss terms, it is not completely compatible with the main goal of our study. Similarly, our restriction to the Gaussian/quadratic regime in this paper is an intentional first step to verify the measurements against exact analytical ground truth before moving to non-convex networks where no ground truth exists.
On the Ethics Flag and AI Use Statement: We strongly but respectfully reject the characterization of our work as "unprepared AI slop."