acceptodds
Under review as a conference paper at ICLR 2027

Gaussian Mixture Variational Inference with Wasserstein–Fisher–Rao Gradient Flows

11 views

Abstract

The choice of geometry affects the efficiency of gradient-based algorithms for variational inference (VI). We study Wasserstein–Fisher–Rao (WFR) gradient flows over Gaussians and their mixtures. For Gaussian VI with smooth, strongly log-concave targets, we prove exponential convergence of a WFR splitting scheme to the optimal Gaussian approximation. For general targets, we establish stationarity guarantees for Gaussian and mixture VI. The splitting scheme evaluates target statistics twice per iteration, once for each geometry. For efficiency, we instead develop a switched algorithm that chooses one suitable geometry per update. For Gaussian targets, we show that Wasserstein warm-up followed by Fisher–Rao refinement avoids the logarithmic initialization delay of Fisher–Rao and the slow convergence of Wasserstein on ill-conditioned targets. We extend this approach to Gaussian mixtures and propose adaptive rules for choosing each component’s switching time. We prove descent and stationarity guarantees for the resulting algorithm. Across benchmarks in machine learning and PDE-based Bayesian inverse problems, the switched method is competitive with or better than competing methods and is substantially less sensitive to initialization.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
sampling, variational inference, Gaussian mixture models, Wasserstein-Fisher-Rao gradient flows

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fTa7OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores whether switching between Wasserstein and Fisher–Rao geometry is useful for Gaussian and Gaussian-mixture-based variational inference with gradient flows. The paper's main contribution is an algorithm for inference with Gaussian mixtures. The scheme selects the per-component geometry adaptively. The approach is mainly analytical, giving exponential convergence rates for their splitting scheme. The method is also validated empirically on a high-dimensional mixture, density approximation for a normalizing flow, and inverse problems, showing superior performance in terms of ELBO and total variation over three baselines.

Critical strengths

The derivation of a switching rule from Wasserstein to Fisher–Rao in the Gaussian case is well motivated by the analysis in Theorem 3.1, which covers the regimes where the mean is not recovered and the covariance is incorrectly scaled. Carrying this to the mixture case, they show that switching still minimizes the KL even with component-wise geometries, which was surprising. The analysis is mostly convincing (see weaknesses), and the empirical results support that their method is competitive in terms of ELBO and total variation. Altogether, I lean towards acceptance.

Critical weaknesses

Theorem 3.1 does not cover the non-scaled regime for the covariance, that is, C0≠ϵC∗C_0 \neq \epsilon C_* for any ϵ>0\epsilon > 0. Without this case, the analysis of Theorem 3.1 seems partial (see question below), because we rarely know the covariance a priori. With theory covering this regime, or a convincing argument for why it is not relevant, I would move this paper from a weak accept to a clear accept.

Questions for authors
  • How is the higher MC variance accounted for when using Stein's identity to avoid Hessian computations?
  • Does the covariance recovery (Theorem 3.1.1) hold for mixture components? If not, can you say anything about the covariance?
  • The practical switching rule for Gaussian mixtures is intuitive in the Gaussian setting, but the mixture entropy couples the components. Is there a stronger motivation for why a one-time switch is a good choice? Or do you expect other schemes could be better (either in theory or in practice)?
  • Can you say anything about recovery in the regime where C0≠ϵC∗C_0 \neq \epsilon C_* for any ϵ>0\epsilon > 0, in either the Gaussian or the Gaussian mixture case? If not, is there an argument for why this regime can be safely ignored?
3 comments
AuthorsOpenReview ↗

Response to Reviewer fTa7 — Part 1/2

We thank the reviewer for the positive assessment, for identifying general initial covariances as an important theoretical gap, and for the constructive questions. The revision addresses this gap with new results, as described below.

General initial covariances. The restriction C0=εC⋆C_0=\varepsilon C_\star in the original Theorem 3.1 was made to isolate the two phases cleanly. The revised manuscript removes it: Theorem B.1 and the new Proposition B.2 allow arbitrary C0≻0C_0\succ0. For the Gaussian target N(m⋆,C⋆)\mathcal N(m_\star,C_\star), let μ=μmin⁡(C⋆)\mu=\mu_{\min}(C_\star) and c0=μmin⁡(C0)<μ/2c_0=\mu_{\min}(C_0)<\mu/2. Then

(C_t^{\mathrm{FR}})^{-1}=(1-e^{-t}) C_\star^{-1}+e^{-t} C_0^{-1}.$$ Consequently, the first times to reach the covariance floor $\mu I/2$ satisfy $$T_{\mathrm W}\le\tfrac12\log2,\qquad T_{\mathrm{FR}}\ge\log\Bigl(\frac{\mu}{2c_0}\Bigr).$$ The Wasserstein time is thus independent of the initial eigenvectors and of the smallest initial eigenvalue, whereas Fisher–Rao incurs a delay logarithmic in $1/c_0$. Proposition B.3 proves the discrete analogue for general smooth, log-concave targets: one Wasserstein forward–backward step with step size $h$ gives $C_1\succeq hI$, while pure Fisher–Rao needs logarithmically many fixed-size steps as $\beta c_0\to0$. These results cover the initial-covariance regime identified by the reviewer. We also have new sharp, affine-invariant convergence results for the Fisher–Rao gradient flow in Gaussian VI with general, non-Gaussian targets, which are being prepared as a separate paper. If the reviewer finds it valuable, we would be happy to include these techniques in this paper to extend the analysis beyond Gaussian targets. **Monte Carlo variance of the Stein estimator.** Estimators based on Stein's identity can indeed have higher Monte Carlo variance; this is the price of avoiding Hessian evaluations. To reduce unnecessary noise, we subtract the sample mean of the scores, which removes constant-score contributions while preserving unbiasedness (Appendix E). In our experiments, the method remains stable with moderate per-component sample sizes, which suggests that it is robust to Monte Carlo noise at the reported sampling budgets.
AuthorsOpenReview ↗

Response to Reviewer fTa7 — Part 2/2

Covariance recovery for mixture components. We thank the reviewer for this question, which led us to extend Section 3 to Gaussian mixtures (Proposition 4.5 in the revision). For every component kk taking a Wasserstein step of size ηn,k\eta_{n,k}, the projected forward–backward covariance satisfies μmin⁡(C^n,kW)≥min⁡{μ+,max⁡{μ−,ηn,k}},\mu_{\min}\bigl(\widehat C_{n,k}^{\mathrm W}\bigr)\ge\min\bigl\lbrace\mu_+,\max\lbrace\mu_-,\eta_{n,k}\rbrace\bigr\rbrace, regardless of the overlap among components, where μ−≤μ+\mu_-\le\mu_+ are the covariance bounds of the projection. For a Fisher–Rao step, the effect of the mixture entropy is controlled by χn,k=Eqn,k[−log⁡rn,k(X)]≤log⁡(1/wn,k)\chi_{n,k}=\mathbb E_{q_{n,k}}[-\log r_{n,k}(X)]\le\log(1/w_{n,k}), where rn,kr_{n,k} is the component responsibility defined in Section 4. If τn,k[log⁡(1/wn,k)+βμ+]<1\tau_{n,k}[\log(1/w_{n,k})+\beta\mu_+]<1, then μmin⁡(C^n,kFR)≤1+τn,k1−τn,k[log⁡(1/wn,k)+βμ+]μmin⁡(Cn,k).\mu_{\min}\bigl(\widehat C_{n,k}^{\mathrm{FR}}\bigr)\le\frac{1+\tau_{n,k}}{1-\tau_{n,k}\bigl[\log(1/w_{n,k})+\beta\mu_+\bigr]} \mu_{\min}(C_{n,k}). Therefore, Wasserstein provides an immediate covariance floor, whereas Fisher–Rao cannot enlarge an arbitrarily small eigenvalue by more than a bounded factor per step when the weight is bounded away from zero. This result requires only mild additional assumptions, and further guarantees beyond Theorem 4.3 are an interesting direction for future work.

Motivation for a one-way switch. In the setting of Theorem 4.4 (single-Gaussian VI with a smooth, strongly log-concave, radial target), the one-way structure is a consequence of the analysis rather than an extra restriction. For mixtures, we do not prove this optimality; Proposition 4.5 shows that the reason to start with Wasserstein persists. Once the covariance is reasonably scaled, Fisher–Rao benefits from affine invariance, which has also been observed empirically, so we design the algorithm to stay with Fisher–Rao after switching. Other adaptive schemes may help in some cases, and Theorem 4.3 guarantees descent and stationarity for any admissible scheme.

We hope these responses address the reviewer's concerns. If so, we would be grateful if the reviewer would consider updating the score, and we would be glad to clarify anything further. Thank you again for your time and thoughtful feedback.

Reviewer_fTa7OpenReview ↗

Response to rebutal

I thank the authors for their clarifications. I appreciate the addition of Proposition 4.5, which strengthens the case for switching by showing that the Wasserstein update recovers from collapse in a single step, whereas the Fisher–Rao update may require multiple steps. However, I share the concern raised by Reviewer y81C that the paper neither establishes the optimality of the switching rule nor compares it empirically against alternative approaches. I will therefore maintain my current score.

Reviewer_u6jiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines Wasserstein gradient flow and Fisher-Rao gradient flow, and aims to achieve better performance on Variational Inference tasks spanning from low dimensional 2d Gaussion and medium dimensional 196 MNIST. The authors compare with two baseline methods and show improved performance. I am sorry that I don't have enough time to check all the proof and the equations.

Critical strengths
  • switch between two gradient flow is simple but effective. Results also showed improved performance in the evaluated task.
Critical weaknesses
  • my first concern is missing related work of [1], I am wondering what the difference is between this method and the proposed method in [1], I think switching the gradient flow as the main contribution is not enough given prior work has already considered combining them to learn GMM mixtures. I would suggest that the authors distinguish the work from prior work. Better positioning of the work and a direct comparison of the work would help assess the novelty and the contribution of the proposed work.

[1] Yan Y, Wang K, Rigollet P. Learning Gaussian mixtures using the Wasserstein–Fisher–Rao gradient flow[J]. The Annals of Statistics, 2024, 52(4): 1774-1795.

  • my second question would be how is the switching criteria defined? and how would be a fixed step switch compared with proposed method? I would suggest the authors to add an ablation here.

  • My third concern is component-wise switching for Gaussian mixture. In the work [2], the authors have already decompose the GMM to update each Gaussian individually. I am wondering what the difference is here?

[2] Arenz O, Zhong M, Neumann G. Trust-region variational inference with Gaussian mixture models[J]. Journal of Machine Learning Research, 2020, 21(163): 1-60.

  • Only ELBO and qualitative results are reported; how is the wall-clock time compared between different methods?

  • In the MoG40 experiment, the initial covariance is set very small. How about setting the covariances differently, will the improved performance still hold? Is this a design choice that is selected to favor the proposed method?

  • In the evaluated setting, the authors disable the adaptive mode-adding functionality in the baseline [3], if we allow [3] to dynamically add and delete Gaussian component, will the result still hold?

[3] Oleg Arenz, Philipp Dahlinger, Zihan Ye, Michael Volpp, and Gerhard Neumann. A unified perspective on natural gradient variational inference with Gaussian mixture models. Transactions on Machine Learning Research,

Questions for authors

see weakness. I am happy to discuss with the authors.

2 comments
AuthorsOpenReview ↗

Response to Reviewer u6ji — Part 1/2

We thank the reviewer for the constructive questions and the opportunity to clarify our switching criterion and its relation to earlier mixture methods.

Relation to Yan et al. (2024). We have added this work to the revised discussion of related work, which concerns nonparametric maximum-likelihood estimation of a Gaussian location mixture with a fixed kernel covariance, using Wasserstein movement of the particle locations and Fisher–Rao updates of the particle weights only. Our setting is reverse-KL variational inference for a general target π∝e−V\pi\propto e^{-V} over full-covariance Gaussian mixtures. In particular, we switch the geometry used for each component's mean and covariance, while updating the mixture weights separately. Thus, although both works share the Wasserstein–Fisher–Rao motivation, their optimization problems, state variables, and update rules differ.

Definition and justification of the switching rule. The rule is stated in revised Section 4.3 and detailed in Appendix E. For component kk at iteration nn, let Λn,k=Eqn,k[∇2V]\Lambda_{n,k}=\mathbb E_{q_{n,k}}[\nabla^2V] and, when Λn,k≻0\Lambda_{n,k}\succ0, define the underdispersion score Tn,k(C)=[−log⁡λmin⁡(Λn,k1/2CΛn,k1/2)]+.T_{n,k}(C)= \left[-\log\lambda_{\min} \left(\Lambda_{n,k}^{1/2}C \Lambda_{n,k}^{1/2} \right)\right]_+ . Using the same target-statistic evaluations, we compute the Wasserstein and Fisher–Rao candidate covariances. A component that has not yet switched moves permanently to Fisher–Rao when its current score is zero, when the Fisher–Rao candidate has no larger score than the Wasserstein candidate, or when the prescribed cap on Wasserstein steps is reached. If the curvature estimate is not positive definite, the curvature test is skipped. Theorem 4.4 proves that this rule is optimal for single-Gaussian VI with strongly log-concave, smooth, and radial targets. We do not claim optimality for general Gaussian mixtures, but we expect the insight to carry over in practice.

Following the reviewer's suggestion, Appendix K.2 compares componentwise switching with a fixed switching schedule. Componentwise switching performs better on both MoG40 and the breast-cancer posterior, supporting the design of the rule.

Relation to Arenz et al. (2020). We agree that Arenz et al. already use responsibility-based decompositions, from the perspective of the expectation–maximization algorithm, to update each component of a Gaussian mixture separately; we build on this established idea rather than claim it as new. Our question is different: which geometry should be used to update each mean and full covariance, and when should it change? The separable upper bound (Proposition 4.1) handles the coupling created by the mixture entropy, so different components can take Wasserstein or Fisher–Rao forward–backward steps in the same iteration. Theorem 4.3 then proves monotone descent and an O(N−1)O(N^{-1}) projected-stationarity residual for any admissible componentwise choice. The two phases are motivated quantitatively. Wasserstein gives a covariance floor independent of the smallest initial eigenvalue (Propositions B.3 and 4.5), whereas after covariance recovery, the affine invariance of Fisher–Rao avoids the worst-case linear condition-number dependence of Wasserstein dynamics for Gaussian targets (Theorem 3.1 and Corollary 3.2). The empirical advantage of Fisher–Rao geometry has also been observed in prior work on Gaussian mixture VI.

AuthorsOpenReview ↗

Response to Reviewer u6ji — Part 2/2

Wall-clock comparisons. Appendix K.1 now compares computational efficiency. For MoG40, we compare all methods in wall-clock time. For NICE digits and the Darcy-flow inverse problem, we measure cost by the number of target value–gradient evaluations, since these dominate the computational cost in these higher-dimensional problems. These comparisons show that WFR remains its efficiency when performance is measured against computational cost.

Other covariance initializations. Appendix F evaluates MoG40 under several initializations, with the same initial mixtures shared across methods in each setting. The performance gap narrows when the initial covariances are well scaled, but our method remains competitive. Small initial covariances are the regime that our method is designed for, and these results show that its advantage there does not come at a cost when the initialization is favorable.

Adaptive component addition and deletion in the baseline. Appendix K.3 now includes SAMTRUX, which dynamically adds and deletes Gaussian components. Compared with SEMTRUX, SAMTRUX improves the KL divergence and target-evaluation efficiency on MoG40. However, its component updates still follow Fisher–Rao geometry, and its sensitivity to small-covariance initialization persists. On NICE digits, SAMTRUX achieves a lower final KL divergence but is less computationally efficient because it uses more components. Overall, these results suggest that adapting the number of components does little to alleviate the difficulties caused by small-covariance initialization, which may happen in high dimensional inference as scale-mismatch is common.

We hope these responses address the reviewer's concerns. If so, we would be grateful if the reviewer would consider updating the score, and we would be glad to clarify anything further. Thank you again for your time and thoughtful feedback.

Reviewer_y81COpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes using Wasserstein updates to escape very small initial covariances, then switching each Gaussian component to Fisher–Rao updates for refinement. Its strongest convergence results concern single-Gaussian approximation; its practical contribution is an adaptive algorithm for Gaussian mixtures. The experiments support faster optimization from severely underscaled initial covariances, with more mixed evidence about final approximation quality.

Critical strengths

The theoretical analysis is sound and relatively complete for the Gaussian VI part though it follows standard techniques. They also provide an interesting stationarity guarantee for their Gaussian mixture VI algorithm.

Critical weaknesses
  1. It seems that the algorithm proposed for Gaussian mixture VI does not strictly or even approximately follow WFR flow as suggested in the title. My understanding is that the splitting discretization per iteration can roughly recover WFR as noted in Gallouët–Monsaingeon paper. But the algorithm actually used in the paper switches permanently from Wasserstein to FR. Moreover, the mixture updates use component-level geometry. For example, their FR block naturally corresponds to the Fisher metric of the labelled distribution P(k,x)=wkqk(x)P(k,x)=w_kq_k(x), which generally differs from the Fisher metric of the marginal mixture q(x)q(x). Although this does not mean the algorithm is invalid, it is very misleading indeed because their title is Gaussian Mixture Variational Inference with Wasserstein--Fisher--Rao Gradient Flows

  2. The authors failed to recognize a large number of closely related literature. Although I haven't found exactly same approach proposed in earlier work, it inevitably leads to significant issues regarding novelty discussion and the claimed empirical advantage over alternatives.

    a. Yan, Wang, and Rigollet. Learning Gaussian Mixtures Using the Wasserstein–Fisher–Rao Gradient Flow. AoS, 2024. This work already combines Wasserstein movement of Gaussian component locations with Fisher–Rao weight updates, providing a direct precedent for the submission’s WFR mixture framework, although it studies likelihood-based estimation with fixed covariance.

    b. Liu, Ghosal, Balasubramanian, and Pillai. Towards Understanding the Dynamics of Gaussian–Stein Variational Gradient Descent. NeurIPS, 2023. Gaussian–SVGD develops Gaussian VI through Stein gradient flows, establishes convergence under strong log-concavity, and identifies a kernel whose Gaussian geometry coincides with Bures–Wasserstein geometry, directly connecting to the submission’s single-Gaussian analysis.

    c. Galy-Fajou, Perrone, and Opper. Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation. Entropy, 2021. Gaussian Particle Flow uses affine flows and particle estimates to optimize a Gaussian variational approximation, including convergence analysis for Gaussian targets, making it relevant to the submission’s Gaussian flow and covariance dynamics.

    d. Zhang, Li, Du, and Qian. DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework. IJCAI, 2022. DPVI combines particle transport with adaptive weights through a WFR population flow, establishing a direct precedent for transport-and-reweighting approaches to VI.

    e. Rønning, Nalisnick, Ley, Smyth, and Hamelryck. ELBOing Stein: Variational Bayes with Stein Mixture Inference. ICLR, 2025. Stein Mixture Inference uses interacting particles to parameterize component distributions and optimize an entropy-augmented mixture ELBO, providing a close mixture-VI precedent and a relevant Gaussian-guide comparator.

  3. The strongest empirical conclusion I would accept is “a Wasserstein warm-up can substantially improve Fisher–Rao mixture VI from difficult initializations.” but this alone is incremental contribution. The stronger claim that their curvature-based, componentwise switching is an important advance remains insufficiently demonstrated.

    The algorithm feels ad hoc. Several design choices can change together. Covariance bounds, branch step sizes, and switching all contribute. The authors themselves credit covariance clipping with stabilizing expansion, but do not isolate its contribution. The descent theorem permits arbitrary geometry choices, so it cannot establish that their particular selector is effective.

Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer y81C — Part 1/2

We thank the reviewer for the careful reading, and for finding our Gaussian VI analysis sound and the stationarity guarantee for the mixture algorithm interesting. Below we clarify the geometry of our algorithm and its relation to prior work, and we describe new theoretical support for the switching rule.

WFR terminology and the geometry of the mixture. We agree with the reviewer that the algorithm used in our experiments is not a direct time discretization of the WFR gradient flow of the marginal mixture density, and we apologize for the confusion caused by the original title. The title did not distinguish the motivating WFR flow and its splitting discretization (Section 2) from the practical algorithm, which selects a Wasserstein or a Fisher–Rao update for each component (Section 4). We have therefore changed the title to Switched Wasserstein and Fisher–Rao updates for Gaussian mixture variational inference. The revision also distinguishes the Fisher–Rao metric of the labeled distribution P(k,x)=wkqk(x)P(k,x)=w_kq_k(x) from that of the marginal q(x)=∑kwkqk(x)q(x)=\sum_k w_kq_k(x). The componentwise metric is a deliberate choice that lets each component update independently, as in the trust-region method of Arenz et al. (2020).

Positioning with respect to prior work. We thank the reviewer for these references. We agree that our discussion of related work was incomplete, as it focused primarily on Gaussian VI, and we have added all five works to the revised manuscript. We do not claim to be the first to combine transport with reweighting or to update mixture components separately.

The closest work in its use of Wasserstein–Fisher–Rao geometry for Gaussian mixtures is Yan, Wang, and Rigollet (2024), who study maximum-likelihood estimation of Gaussian location mixtures with a fixed kernel covariance, using Wasserstein updates for the component locations and Fisher–Rao updates for the mixture weights. In contrast, we consider reverse-KL VI for a given target and study which geometry to use for each component's mean and full covariance. Liu et al. (2023) and Galy-Fajou et al. (2021) study Gaussian VI through Gaussian–SVGD and particle flows, while Zhang et al. (2022) and Rønning et al. (2025) develop particle-based VI with dynamic reweighting and Stein mixtures, respectively. These are important related approaches, but none of them studies the adaptive choice between Wasserstein and Fisher–Rao updates for the covariances of Gaussian components.

Our contribution is therefore the geometry-selection mechanism for full-covariance Gaussian mixture VI, together with theoretical results characterizing the complementary regimes of the two geometries and the resulting switched algorithm.

AuthorsOpenReview ↗

Response to Reviewer y81C — Part 2/2

Theoretical support for the switching rule. We agree that the stationarity theorem (Theorem 4.3) alone cannot show that our selector is preferable, since it proves descent and first-order stationarity for any admissible componentwise choice of geometry. The revision therefore adds two results specific to the proposed rule.

First, Theorem 4.4 proves that the rule is optimal in a model setting. For single-Gaussian VI with αI⪯∇2V⪯βI\alpha I\preceq\nabla^2V\preceq\beta I and radial VV, a centered isotropic initialization q0=N(0,c0I)q_0=\mathcal N(0,c_0I) with 0<c0<c⋆0<c_0<c_\star, and fixed step sizes η≤(2β)−1\eta\le(2\beta)^{-1} and τ≤α/β\tau\le\alpha/\beta, the switching rule satisfies F(qNsw)=min⁡G∈{W,FR}NF(qNG),N≥1.\mathcal F(q_N^{\mathrm{sw}})=\min_{G\in\{\mathrm W,\mathrm{FR}\}^N}\mathcal F(q_N^G),\qquad N\ge1. In this setting, the one-way switch is optimal among all sequences of geometry choices, rather than merely heuristic.

Second, Proposition 4.5 extends the covariance-recovery mechanism behind the switch to Gaussian mixtures: a Wasserstein step immediately restores a covariance floor, while a Fisher–Rao step expands a small eigenvalue by at most a bounded factor when the responsibility term is bounded.

Empirically, componentwise switching also outperforms a fixed switching schedule on MoG40 and the breast-cancer posterior (Appendix K.2). Together, these results support the proposed rule beyond the descent guarantee.

We hope these responses address the reviewer's concerns. If so, we would be grateful if the reviewer would consider updating the score, and we would be glad to clarify anything further. Thank you again for your time and thoughtful feedback.

I appreciate the efforts in addressing my concerns. I think the first concern about the misalignment between title and algorithm is well addressed.

For related work, my point was not just about those work being relevant to the current paper. If I understand correctly, the algorithm proposed in this article is somewhat technically incremental and the strongest point on significance is about the improvement of empirical performance based on a simple theoretical observation. Thus, I believe a more thorough comparison with previous approaches is needed.

Prop 4.5 is a very good step towards addressing my concern on explaining theoretical intuition behind the proposed switched algorithm. But still, it does not establish optimality of such approach or useful comparison with other approaches in literature that aim for solving the same problem.

Based on these points, I am not convinced about the significance of the paper's contribution to this field, especially considering ICLR's high standard and the increasing number of submission in relevant fields, and am inclined to maintain the current rating.

As a side note, I also find the misleading initial submission a bit concerning. It felt like an attempt to promote the significance of the work by leaving gaps between what was achieved and what was claimed. Such overclaiming can lead to even more issues in the reviewing process nowadays. As disclosed, I prompted AI for the initial screening and summarizing but it did not flag this misalignment at all.

Thank you for the follow-up and for confirming that the concern about the title has been addressed.

Significance. We understand the reviewer’s view and recognize that assessments of significance can differ. Our aim is to understand which geometry to use in Gaussian mixture VI, and when. We show that Wasserstein and Fisher–Rao updates are effective in complementary regimes, turn this into a switching rule with theoretical support, and find that the resulting method is substantially less sensitive to initialization while remaining competitive with established methods, including the trust-region and natural-gradient methods of Arenz et al. with adaptive components. We appreciate that the reviewer found Proposition 4.5 a good step in this direction, and we agree that optimality for general mixtures remains open.

The original submission. In their comment, the reviewer wrote:

“As a side note, I also find the misleading initial submission a bit concerning. It felt like an attempt to promote the significance of the work by leaving gaps between what was achieved and what was claimed. Such overclaiming can lead to even more issues in the reviewing process nowadays. As disclosed, I prompted AI for the initial screening and summarizing but it did not flag this misalignment at all.”

We appreciate the reviewer raising this concern. We agree that the original title was less precise than it should have been, but we respectfully disagree that the submission was an attempt to promote its significance by leaving gaps between what was achieved and what was claimed.

As the original review itself noted, the issue concerns naming, not the validity of the algorithm. The paper is built on the standard Wasserstein–Fisher–Rao geometry: Section 2 proposes the discrete splitting scheme of WFR for Gaussian VI and proves its exponential convergence, and the switched algorithm in Section 4 builds on Section 2 and the comparison between Wasserstein and Fisher–Rao geometries in Section 3, applying each geometry at the component level where it is most effective while halving the target evaluations per iteration. The original abstract stated this explicitly: “For efficiency, we instead develop a switched algorithm that chooses one suitable geometry per update.” What was achieved and what was claimed were thus stated together from the start. We changed the title only to make this distinction more explicit.

We thank the reviewer again for the engagement and respect their final judgment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.