- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
A multimodal extension of pullback geometry is introduced. It replaces the unimodal Gaussian latent distribution, which was used in prior work, by a Gaussian mixture model and uses the a responsibility-weighted expected component precision to define a Riemannian metric. The resulting geometry is positive definite, and experiments indicate that is can better represent multimodal datasets than unimodal constructions applied in prior work. The construction is supported by theoretical analysis.
Critical strengths
-
A important limitation of pullback geometries is addressed. A single Gaussian latent space is not enough for data that consists of multiple manifold-like structures. The proposed step from unimodal Gaussians to mixtures is natural and allow for an interpretation as going from a single chart to multiple local charts.
-
The proposed Riemannian metric using P^2 is elegant, has desired properties (smoothness, positive definiteness) and agrees with the existing Gaussian metric from Diepeveen et al. (2025) for K=1. Experiments demonstrate befits from the proposed construction compared to prior work.
Critical weaknesses
-
The exact score-based metric is derived but the introduced metric discardes one of the terms, the covariant term. It is a reasonable construction and produces a valid metric. However, this is a design choice and the metric is not uniquely derived from first prinicples. This means that alternative choices of a metric are possible but an exploration of alternative design choices is missing. While the paper demonstrates that the proposed metric works, it is not clear how much of the observed improvement comes from learning a Gaussian mixture, and how much comes specifically from choosing the metric to be P^2. For example, would using P^a or (P-Cov)^2 yield similar (or worse or better) results?
-
While the Cov term cannot be guaranteed to be positive definite, it is not clear to me whether or not the Cov term in applications is actually harmful. Even though positive definteness cannot be established it still may be of benefit in applications. I am missing some experiments including Cov term and shed light on these questions.
-
The theoretical results are established under strong assumptions that I think will not hold in applications. Therefore, it merely serves as a motivation for the approach than establishing guarantees.
Questions for authors
The concavity results are proved for certain star-shaped mixtures. To what extent do you expect the guarantees provided by the theory hold for the mixtures learned in the experiments?
2 comments
Thank you for the detailed responses and the willingness to communicate the identified weaknesses more clearly in the paper. Overall, my “weak accept” rating remains unchanged.
We clarify the motivation for P2, the effect of component-score covariance, and the scope of the theoretical guarantees for learned mixtures.
Thank you for the constructive review.
Response to Weakness 1: Metric choice and alternative constructions
We agree that P2 is not uniquely forced by the exact score-Hessian construction. It is a deliberate design choice motivated by several properties: it is smooth and strictly positive definite for every nondegenerate GMM, explicit and tractable to evaluate, and exactly recovers the original SBPRG metric when K=1. Within the family Pa, exact recovery of the single-Gaussian metric requires a=2. This exponent also yields the algebraic cancellation used to derive the explicit GLC condition in our structured theoretical analysis; we will add the detailed derivation and a comparison with general a to the appendix.
These properties justify our choice but do not show that P2 is empirically optimal. Other constructions, including regularized versions of the score-Hessian metric, are possible. The current submission does not systematically compare all such choices and therefore does not completely separate the benefit of learning a GMM from the benefit of the specific P2 metric. We will state this limitation explicitly.
Response to Weakness 2: Effect of the component-score covariance
Let sk(z)=∇logN(z;μk,Ak) denote the score of component k, and define its responsibility-weighted covariance as Cs(z)=Covγ(z)[sk(z)].
This is the covariance of the component scores, not the covariance Ak of an individual Gaussian. With the expected precision P(z)=∑kγk(z)Ak−1, the exact mixture log-density Hessian is H(z)=−P(z)+Cs(z).
Although Cs(z) is positive semidefinite, it is added to the negative-definite term −P(z). It can therefore partially or completely cancel expected precision in some directions. If a nonzero vector v satisfies H(z)v=(−P(z)+Cs(z))v=0, then v⊤H(z)⊤H(z)v=0.
Thus, the direct score-Hessian tensor H(z)⊤H(z) is only positive semidefinite at that point and does not define a Riemannian metric. Near cancellation can likewise cause poor conditioning. This is why the exact score-Hessian construction is theoretically unsafe for the role required in this work. We will add a detailed derivation of this cancellation and an explicit two-component example to the appendix.
We agree, however, that this argument does not establish that the covariance term is empirically harmful whenever H(z) remains nonsingular. It may contain useful information in such regions. The submitted experiments do not include an ablation retaining this term, so we do not claim to have isolated its empirical effect. A systematic comparison of P(z)2 and H(z)⊤H(z), including conditioning and path quality, remains important future work.
Response to Weakness 3 and Question 1: Scope of the theoretical guarantees
We agree that the GLC and curvature results require strong structure. The shared-mean and common-eigenbasis assumptions are used to derive explicit conditions. The general guarantee is instead that P2 defines a smooth, strictly positive-definite metric for every nondegenerate GMM.
The appendix partially extends GLC to mixtures with arbitrary separated means and shared covariance, but we do not have corresponding guarantees for mixtures with both arbitrary means and arbitrary covariances. Moreover, global GLC cannot generally hold for sufficiently separated modes because every continuous path connecting separated high-density regions must pass through lower density.
The exact star guarantees do not apply directly to all mixtures learned in the experiments. Wavy-X and BlueWhale use a common component mean. For MNIST, we relax the shared-mean condition by allowing component means to vary within a dataset-specific radius. The star theory therefore applies most directly to the common-mean settings and motivates structured post-training diagnostics, while path behaviour for the relaxed MNIST mixture is assessed empirically.
The Gaussian-star class was chosen because it provides greater flexibility than a single Gaussian while remaining sufficiently structured for explicit analysis. This balance is relevant because our flow is constrained by isometry regularization and cannot be assumed to transform arbitrary data into any desired latent configuration.
For learned mixtures with relaxed assumptions, the theory provides a structured reference for post-training diagnostics, but not a theorem-level guarantee. For mixtures with relaxed means, the reported star–Ricci quantity should therefore be interpreted as a structured post-training diagnostic rather than the exact Ricci curvature of the general learned metric.
We will distinguish clearly between general metric validity, exact star guarantees, and theory-informed diagnostics and empirical validation under relaxed assumptions.