acceptodds
Under review as a conference paper at ICLR 2027

Finite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows

Abstract

Wasserstein gradient flow extends gradient descent to probability measures. Its Hessian-guided perturbed variant (PWGF) adds Gaussian perturbations to escape saddle points in nonconvex problems. We investigate when its approximation by finitely many interacting particles remains accurate over growing time horizons. Our analysis retains the curvature accumulated along the population-driven reference path: negative curvature can amplify approximation errors, while subsequent positive curvature can damp their influence. This captures favorable scenarios in which temporary instability is compatible with accurate tracking over growing horizons. Under regularity assumptions and a prescribed common perturbation schedule, we prove particle and objective-value tracking bounds on a high-probability event for reference paths satisfying explicit conditions on accumulated curvature. To handle state-dependent Gaussian jumps, we construct a population-first coupling that preserves the reference particles' conditional independence and reduces jump errors to covariance comparison. We verify the conditions in a variance-plus-cosine model, where curvature recovery yields a growing-horizon tracking guarantee. We also establish local attraction, transverse descent, and positive second variation in two regions of a regularized matrix-factorization model, motivating a positive–negative–positive curvature pattern.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Wasserstein gradient flows, sampling, optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LWEUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies the problem of nonconvex optimization over Wasserstein space. In particular, it develops a notion of perturbation based on Hessian information to prevent gradient-flow-based methods from getting stuck at saddle points. This mirrors the developments in perturbation methods for nonconvex optimization in Euclidean space. The work is purely theoretical. The authors establish a finite-sample theorem for tracking the curvature of the objective, along with a result on a matrix factorization model showing, in more detail, the curvature locally around a stationary point.

Critical strengths

The work tackles an important problem of nonconvex optimization over the space of measures. Translating ideas from Euclidean optimization to optimization over measures is natural and can lead to many interesting applications. The theory developed naturally considers the geometry of Wasserstein space.

The substantial theory may contribute an important foundation to the literature. In particular, the study and application of Wasserstein Hessians is important.

Critical weaknesses

The paper is almost impossible for a human to read and understand. The main theorems don't seem to say anything directly about the actual method of interest - Hessian Perturbed Wasserstein Gradient Flow - in a direct way, and instead talk about tracking finite-sample bounds and local geometry in a population setting. But there is no signposting or roadmap that I can grab onto. No clear algorithm that can be referenced and explained. Furthermore, at a full 128 pages of material, it is impossible to check these results for a conference.

While I agree that optimization over measures is important, it is unclear how practical the results of this paper could be. Especially in the context of ICLR, I think that motivating practical applications is important. Does the developed theory change at all how we would approach any of the mentioned nonconvex optimization problems over Wasserstein space? Do we observe advantages?

Questions for authors

What applications is this work important for?

Can you comment on the main ideas of the proof?

Can you give a better, more human readable overview of the algorithm, theoretical framework, and important results in the paper?

3 comments
AuthorsOpenReview ↗

To Reviewer LWEU

We thank the reviewer for the comments and questions, which led us to reorganize the main text.

W1. We are sorry that the paper was long and hard to follow. In the revision, the assumptions are stated before Theorem 3.1, and the proof overview comes right after the theorem and its example. The Introduction now ends with an outline of the paper. The method is written out as Algorithm 1 in Appendix C.1.

The method in Theorem 3.1 is PWGF run with NN particles (Algorithm 1), and it uses only the particles. Under Assumptions 3.1–3.3 and a jump schedule fixed in advance, the theorem bounds the distance rtr_t between these particles and reference particles drawn from the population PWGF of Yamamoto et al. (2025), and the gap in objective value ∣F(ρXt)−F(μt)∣|F(\rho_{X_t})-F(\mu_t)|. Section 5 studies the population-level geometry of a matrix-factorization model. It motivates the curvature scenario of Figure 1 and is separate from the theorem.

W2 and Q1. Nonconvex optimization over probability measures appears in several learning problems. Kim & Suzuki (2024) study in-context learning with an attention layer and a three-layer teacher–student network. Yamamoto et al. (2025) give matrix decomposition and a three-layer network as examples, and our Section 5 studies a product-measure version of the matrix problem. Yamamoto et al. (2025) proposed PWGF to escape saddle points in such problems and proved guarantees for the flow of measures. A run uses finitely many particles, and our paper bounds the gap between the particle run and the flow of measures when the jump schedule is fixed in advance.

Advantages over other methods were observed by Yamamoto et al. (2025). In their experiments on the in-context learning objective of Kim & Suzuki (2024) and on matrix decomposition, run with their Algorithm 2, PWGF reduced the loss faster than gradient flow without noise, and as fast as or faster than isotropic noise. Our experiments (Appendix C) measure how closely a particle run follows the population flow. With a common prescribed schedule (Appendix C.4), the median of rtr_t over trials has a negative log–log slope in NN at every recorded time.

Our bound weights sampling errors by the curvature accumulated along the path. This suggests that a run needs more particles when the path spends a long time in negative curvature, and that earlier errors matter less after the curvature turns positive.

Q2. We compare the particles XiX^i with reference particles YiY^i. The YiY^i start from the same points and move with the population drift and the population jumps. Given the population fields, they are i.i.d. samples of μt\mu_t, so concentration inequalities apply to them. At each jump the two systems are driven by one common Gaussian vector (Lemma A.14), and the two jumps then differ only through their covariance factors.

Between jumps, linearizing the two ODEs gives, to leading order, r˙t≤−κtrt+Γt\dot r_t\le-\kappa_t r_t+\Gamma_t. Here κt\kappa_t is the smallest eigenvalue of the Hessian sampled along the reference particles, and Γt\Gamma_t is a sampling error of order N−1/2N^{-1/2}. By this inequality, an error created at time ss arrives at time tt multiplied by exp⁡(−∫stκu,du)\exp(-\int_s^t\kappa_u \\,du). It grows while κu<0\kappa_u<0 and shrinks while κu>0\kappa_u>0. We keep this weight in the bound, and its integral over ss is the factor CtY\mathfrak C_t^Y of Theorem 3.1.

At a jump, the past error is multiplied by a bounded factor and a new error is added. The new error is bounded by the square root of a covariance mismatch of order N−1/2N^{-1/2} (Lemma A.13), which gives the N−1/4N^{-1/4} term. Jump errors are carried forward with the same weight, and their sum is the factor JtY\mathfrak J_t^Y.

Q3. The algorithm alternates two steps. Between jump times, each of the NN particles follows the gradient flow of FF evaluated at the particles' empirical measure. At each jump time, one Gaussian field is drawn, with covariance given by the squared Hessian of FF at the empirical measure, and each particle is shifted by ηℓ\eta_\ell times the value of the field at its position.

We compare this particle system with the population PWGF μt\mu_t through rtr_t, the root-mean-square distance between the particles and coupled samples of μt\mu_t, and through the gap in objective value.

Theorem 3.1 bounds both quantities by N−1/2(1+CtY)+N−1/4JtYN^{-1/2}(1+\mathfrak C_t^Y)+N^{-1/4}\mathfrak J_t^Y, up to small powers of NN. The first term comes from errors made between jumps and the second from errors made at jumps, each weighted by the curvature along the path as in Q2. The theorem holds under Assumptions 3.1–3.3, on an event of probability at least 1−p1-p, for T≤N1/4−υ−ζT\le N^{1/4-\upsilon-\zeta}, and for reference paths on which the two factors satisfy the stated bounds. Example 3.1 checks all conditions in a variance-plus-cosine model. There they hold with probability at least 1−N−1/161-N^{-1/16}, and the error is at most N−3/16+εN^{-3/16+\varepsilon} up to time N1/8N^{1/8}.

I think that the readability issue persists - for example you state that Theorem 3.1 discusses Algorithm 1, but Algorithm 1 is referenced nowhere in text. Instead, the reader is left to parse the assumptions and earlier discussion to try to parse what the theorem is about and can tell us. More discussion of the result itself would also help, especially at a higher/pedagogical level.

Highlighting your response to Q2 in text would be particularly useful.

Your response to Q3 is helpful, where you give a simple and intuitive explanation of the algorithm. But again I fail to see how the text clearly explains and motivates this.

AuthorsOpenReview ↗

Response to the follow-up comment by Reviewer LWEU: second revision

Thank you for reading the revision and for the suggestions. Based on them, we rewrote the explanations in Sections 2 and 3 and uploaded a second revision. The assumptions, the theorem and the proofs are the same as before. In the new PDF, the text added this time is in green, and the changes of the first revision are still in blue. The changes are listed below.

Section 2 (the method). We now explain what the method does before the formal definitions.

  • The section now starts with a short description of the particle method. It is similar to our response to Q3.
  • Algorithm 1 is now in Section 2.2 (it was in Appendix C.1). We refer to it at the start of Section 2, in Section 2.2 and before Theorem 3.1.
  • The subsections have new titles. Each subsection starts with one sentence that says what it defines. Section 2.3 explains why we use reference particles.

Section 3 (the theorem). We now give the main idea of the bound first, and explain each formal statement briefly.

  • The start of the section explains why the bound uses the curvature along the path and not a Lipschitz constant.
  • Section 3.1: we added a short explanation after each assumption.
  • Section 3.2: we added an explanation of the two factors after Definition 3.1. It is based on our response to Q2. It explains how the error changes between jumps and at a jump.
  • Section 3.3: the paragraph before Theorem 3.1 explains what the theorem shows. It is based on our response to Q3. After the theorem, we discuss the bound in two simple cases and explain what it suggests in practice.
Reviewer_zDryOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Hessian-guided perturbed WGF (Yamamoto et al., 2025) escapes saddles by adding Gaussian-field jumps whose covariance is the squared integral Hessian. Its guarantees cover only the population flow. This paper bounds the error of the NN-particle implementation. The main result (Theorem 3.1):The setting is moment-structured objectives with a prescribed common jump schedule (not adaptive). The comparison uses conditionally i.i.d.\ reference particles, and errors are propagated by the sampled curvature exp⁡(−∫κuY,N du)\exp(-\int \kappa_u^{Y,N}\,du). With high probability, along paths satisfying a controlled-curvature condition and with T≤N1/4−υ−ζT \le N^{1/4-\upsilon-\zeta},$ r_t \vee |F(\rho_{X_t}) - F(\mu_t)| \lesssim_\varepsilon \Xi^{\varepsilon}\Big[(N^{-1/2} + T/N)(1 + C_t^Y) + (N^{-1/4} + \sqrt{T/N}),J_t^Y\Big], $ where CtYC_t^Y and JtYJ_t^Y accumulate continuous and jump errors. Later positive curvature damps earlier errors, avoiding worst-case Gronwall blow-up. Jumps contribute N−1/4N^{-1/4} because the jump error is the square root of a covariance mismatch. A population-first Gaussian coupling drives both jumps with one standard Gaussian. It preserves both marginal laws and the reference particles' conditional i.i.d. structure, which reduces jump error to covariance comparison.

Critical strengths

Conditioning that makes concentration valid despite random, state-dependent jumps. The main technical obstacle is real. The perturbation covariance KμK_\mu depends on the current measure, the population path is itself random, and the particles interact, so standard i.i.d. sampling bounds do not apply directly. The paper's population-first coupling (Lemma A.14) seems to solve this cleanly.

Error propagation that keeps the curvature history, so the bounds can extend beyond fixed time. Classical propagation-of-chaos arguments close with a worst-case Lipschitz constant, giving eLTe^{LT} growth. That is hopeless for saddle-escape dynamics, which pass through negative curvature by design. Here, errors are instead propagated by the sampled curvature,$G_\kappa^Y(t,s) = \exp\Big(-\int_s^t \kappa_u^{Y,N}, du\Big). $ Errors created during an unstable phase can therefore be forgotten once positive curvature returns, and continuous and jump errors get separate response factors CtYC_t^Y and JtYJ_t^Y. The variance-plus-cosine example shows the mechanism working end to end. The particle system escapes a saddle, the transient amplification is paid for, and tracking and the objective gap then reach O(N−1/2+ε)O(N^{-1/2+\varepsilon}) over a growing horizon. For a method whose purpose is to traverse negative curvature, this is the right structural form of a guarantee. It is also a conceptual advance over fixed-horizon Gronwall bounds, even if the general conditions remain to be verified in richer models.

Critical weaknesses

Two weaknesses make the strongest case for rejection.

  1. The main theorem assumes away the hard part, so its content may be vacuous. The guarantee holds only on events defined by conditions placed on the very quantities being controlled:

*Non-spikiness is a condition on the coupling error X − Y itself.

*Uniform moment and Lipschitz bounds are imposed on the empirical system X over a growing horizon.

*The controlled-curvature condition is imposed on the realized reference path.

The paper explicitly declines to lower-bound the probability of these events, so in general the theorem may hold only on a negligible or empty set. The sole model where everything is verified, the 1-D variance-plus-cosine example, is close to degenerate:

*All particle errors are identical, so non-spikiness holds trivially.

*The dynamics reduce to a scalar mean ODE.

*The jump field is spatially constant.

That example exercises none of the difficulties the general machinery is built to handle: spiky errors, genuinely spatial jump fields, or high-dimensional curvature. The paper’s own limitations section concedes that deriving these conditions from model primitives remains open. That is the central question of a finite-particle approximation result.

  1. The paper does not answer the question it poses for the method it studies, and its experiments point the other way. The paper asks when finitely many particles accurately approximate population PWGF. Yet PWGF as defined by Yamamoto et al. triggers jumps adaptively. Here, analytical tractability is bought by imposing a deterministic schedule shared by both systems, which a practical implementation cannot use, since the empirical system has no access to the population’s jump times.

The experiments then run the real adaptive method. Schedules desynchronize, and paired errors stay around 0.6 even at N=32,768N = 32,768, without decreasing in NN. No experiment tests the setting the theorem actually covers, or checks the predicted $N^{-1/2} $ and N−1/4N^{-1/4} rates. The $N^{-1/4} $ jump rate may itself be loose: in the example the true mismatch appears closer to N−1/2N^{-1/2}.

The empirical evidence therefore suggests that finite-particle PWGF as practitioners would run it does not track the population flow. The theory is silent there, so the paper’s headline claim of accurate tracking over growing horizons is largely unsupported for the method in question.

Questions for authors
  1. Do the theorem’s conditions hold with non-negligible probability in any non-degenerate model?

  2. Why should non-spikiness hold after Gaussian jumps?

  3. Under a prescribed common schedule, do experiments show the predicted rates, and is the N−1/4N^{-1/4} jump rate sharp?

  4. What do the adaptive experiments imply for the paper’s central question?

3 comments
AuthorsOpenReview ↗

To Reviewer zDry

We thank the reviewer for the careful reading and the accurate summary of our analysis.

W1. We agree that deriving the path conditions from model primitives is the central question, and the paper answers it in Example 3.1 only. We also agree with the three points about that example. All paired errors are identical, the dynamics reduce to a scalar ODE for the mean, and the jump field is constant in space. In the revision, a sentence after Theorem 3.1 separates what is proved from what is assumed. The event Ωp\Omega_p has probability at least 1−p1-p, and Assumption 3.2 and the path conditions are hypotheses on the realized paths. Under these hypotheses, the theorem shows that finite-particle errors, including those made at jumps, are carried along the reference path with a weight determined by its curvature.

To look beyond the example, the revision adds experiments with a jump schedule fixed in advance and shared by both systems (Appendix C.4). The objective is F(ν)=14∣AνAν⊤−M∗∣F2F(\nu)=\frac14\\|A_\nu A_\nu^\top-M_*\\|_F^2 with Aν=∫tanh⁡(W)ν(dW)A_\nu=\int\tanh(W)\nu(dW). The particles are matrices W∈R32×2W\in\mathbb R^{32\times2}, the jump fields depend on the position, and there are four jumps. Over ten trials, the median of rtr_t has a negative log–log slope in NN at every recorded time (Figures 10 and 11). In most trials the later jumps leave rtr_t almost unchanged. In a few trials (one of ten at N=32768N=32768), rtr_t grows to about 2 after the jump at t=10t=10.

W2. A schedule fixed in advance needs no access to the population. Its times and amplitudes are chosen before the run, and in Appendix C.4 the particle systems run without the population. The Scope paragraph of Section 1 now says that Theorem 3.1 is restricted to such schedules and gives no guarantee for the adaptive rule.

As the reviewer notes, the submission had no prescribed-schedule experiment. The revision adds these experiments (Appendix C.4), and Q3 reports the rates. The value 0.6 in the adaptive runs is the size of one jump. It appears while only one of the two systems has jumped, and Appendix C.5 reproduces it by shifting one jump time in a common schedule (Q4).

Q1. The paper proves this for Example 3.1 only, where the conditions hold with probability at least 1−N−1/161-N^{-1/16}. For the matrix-factorization model of Appendix C we have numerical evidence. It consists of the tracking results under W1 and the recorded ratio of Assumption 3.2(iii), whose largest value in the adaptive runs of Appendix C.2 is 1.017. A proof in such a model is future work.

Q2. The paper assumes non-spikiness and proves it in Example 3.1 only. The assumption is used in one place, the bound on the cubic Taylor remainder. For the jumps, the coupling gives the following. Both systems are driven by the same Gaussian vector (now noted in Assumption 3.2(iii)), so at a jump Xi−YiX^i-Y^i changes by ηℓ\eta_\ell times the difference of the two jump fields at particle ii. In Appendix C the jump is one Gaussian field evaluated at all particle positions, ξν(W)=JWLAνSν1/2z\xi_\nu(W)=J_WL_{A_\nu}S_\nu^{1/2}z with a single zz. In the runs shown there, the recorded ratio stays close to one across the jumps (Figures 4 and 8).

Q3. With the prescribed schedule of Appendix C.4, the log–log slope of the median of rtr_t against NN is about −1-1 just after the first jump and between −0.4-0.4 and −0.5-0.5 afterwards (Figure 10). The later slopes are close to the exponent −1/2-1/2 of the first term in Theorem 3.1. The N−1/4N^{-1/4} jump term is not visible. The mismatch between the population jump and the NN-particle jump from the same positions has slope close to −1/2-1/2 at the later jumps and −1-1 at the first. The experiments give no evidence that N−1/4N^{-1/4} is sharp, and Appendix C.4 records this as a limitation of our analysis. The exponent comes from Lemma A.13, which bounds the jump mismatch by the square root of a covariance mismatch of order N−1/2N^{-1/2}. Under a nondegeneracy condition on the features that generate the jump covariance, the matrix square root is Lipschitz, and we expect N−1/2N^{-1/2}.

Q4. They show that in the adaptive runs the same-time distance rtr_t is dominated by differences in jump timing. While only one of the two systems has jumped, they are apart by the size of that jump, even if they follow the same path. In Appendix C.5 we shifted one particle jump time in a common schedule. At t=10,20,30t=10,20,30 this raises rtr_t to about 0.6 at N=32768N=32768, and it is back at its previous level about five time units after both have jumped (Figure 12). This is why the maximum of rtr_t in the adaptive runs stays near 0.6 for N≥8192N\ge8192. Appendix C.6 compares the particle and population paths without regard to timing, through the discrete Fréchet distance. The two paths get closer as NN grows for N≥8192N\ge8192, and the median distance at N=32768N=32768 is below 0.1 (Figure 17, top). For N≤4096N\le4096 most particle systems follow a different path. A guarantee for the adaptive rule needs control of these shifts in jump times.

Thank you to the authors for their rebuttal, which clarifies the scope of the results. I am satisfied with the clarifications, which address my concerns and strengthen the manuscript. I appreciate the authors' efforts and will raise my rating from 2 to 3.

We sincerely thank the reviewer for carefully reading our response and for raising the rating. Your comments have helped us improve the paper.

Reviewer_5FV7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies finite-particle approximation of Hessian-guided perturbed Wasserstein gradient flows under a prescribed common jump schedule. It develops curvature-dependent particle and objective-value tracking bounds and a population-first Gaussian coupling that preserves the reference particles' conditional independence. A variance-plus-cosine example verifies a growing-horizon tracking guarantee, while a regularized matrix-factorization model provides local geometric motivation for the curvature pattern.

Critical strengths

The curvature-dependent tracking analysis and population-first Gaussian coupling provide a useful framework for distinguishing temporary error amplification from subsequent curvature-driven damping.

Critical weaknesses

W1. The applicability of Theorem 3.1 depends on strong pathwise assumptions whose verification beyond the illustrative example remains unclear. In particular, Assumption 5.2(iii) imposes non-spikiness directly on the coupled particle errors, and the theorem separately requires bounds on the sampled curvature-response factors. These are properties of the evolving coupled system rather than readily checkable conditions on the objective and initialization. The variance-plus-cosine example is favorable because all paired errors coincide. Additional sufficient conditions derived from model primitives, or a less symmetric verified example, would clarify the theorem's practical scope.

W2. The prescribed deterministic jump schedule limits the connection to the adaptive PWGF algorithm motivating the paper. Both jump times and amplitudes are fixed, and the number of jumps is uniformly bounded in the main theorem. Consequently, the result does not account for disagreement between population and empirical perturbation triggers, or for the resulting difference in their jump counts. Although the paper acknowledges this limitation, it should clarify which guarantees for the original adaptive algorithm can actually be inferred and explain the additional obstacle to synchronizing its perturbations.

W3. The matrix-factorization analysis does not yet establish applicability of the tracking theorem to that model. Proposition 4.1 describes local attraction, transverse descent, and positive second variation in different regions, but the paper explicitly does not construct a connecting PWGF trajectory, transfer these variations to the sampled curvature, or verify the controlled-curvature path condition. Figure 2 is therefore geometric motivation rather than a verified trajectory exhibiting the proposed curvature pattern. Establishing even one nontrivial trajectory satisfying the tracking assumptions in this model would substantially strengthen the connection between Sections 3 and 4.

Questions for authors

Q1. Is the N−1/4N^{-1/4} jump-error scale intrinsic to the class considered, or a consequence of the covariance-factor comparison used in the proof? Lines 449--454 explain how it arises from a square-root estimate, but do not establish its necessity. Could additional covariance nondegeneracy improve this rate, or could an example demonstrate its sharpness?

3 comments
AuthorsOpenReview ↗

To Reviewer 5FV7

We thank the reviewer for the careful reading, and for the assessment of the population-first coupling and the curvature-dependent analysis. The revised PDF adds experiments under a prescribed common schedule (Appendix C.4–C.6).

W1. We agree that both conditions are properties of the coupled system. In the revision, Assumption 3.2, with non-spikiness as item (iii), is stated before Theorem 3.1, and the bounds on the curvature-response factors appear in the theorem as conditions on the realized reference path. A sentence after the theorem says that the probability bound covers the event Ωp\Omega_p, and that Assumption 3.2 and the path conditions are hypotheses on the realized paths. Example 3.1 verifies them with probability at least 1−N−1/161-N^{-1/16}, and it is favorable in the way the reviewer describes, since all paired errors coincide.

For a less symmetric model we have a numerical check. Appendix C records the ratio of Assumption 3.2(iii) along runs of a matrix-factorization model with particles in R32×2\mathbb R^{32\times2}. Its largest recorded value in the adaptive runs of Appendix C.2 is 1.017, and Figure 8 shows it for the prescribed schedule. These are values at recorded times in these runs. Sufficient conditions in terms of the objective and the initialization are future work, as the Conclusion says.

W2. From Theorem 3.1 alone, no guarantee for the adaptive algorithm can be inferred. The Scope paragraph of Section 1 now says that the theorem is restricted to schedules fixed in advance. For the adaptive rule the paper proves one statement (Lemma A.36, Remark A.4). On paths coupled under a common schedule, the trigger decisions of the two systems agree at a given time under a margin condition.

This statement is at a fixed time, and the adaptive rule jumps when a statistic crosses a threshold. To bound the difference between the crossing times of the two systems, one needs a lower bound on the speed at which the statistic crosses the threshold, and the paper has no such bound. In addition, rtr_t is discontinuous in the jump time. While only one system has jumped, rtr_t has the size of one jump, whatever NN is. The Scope paragraph now mentions this second point.

We measured this effect in the model of Appendix C, with the smooth objective F(ν)=14∣AνAν⊤−M∗∣F2F(\nu)=\frac14\\|A_\nu A_\nu^\top-M_*\\|_F^2 with Aν=∫tanh⁡(W)ν(dW)A_\nu=\int\tanh(W)\nu(dW) and a rank-two target M∗M_*. In Appendix C.5 we shifted the particle jump time by offsets between −1-1 and 0.50.5 and kept the rest of the schedule common. At t=10,20,30t=10,20,30, rtr_t rises to between 0.5 and 0.8 while one system has jumped, for each tested NN, and it returns to its previous level within about five time units after both have jumped (Figure 12). Delaying the first jump at t=0t=0 has a smaller effect that lasts longer. Outside these intervals the adaptive and prescribed runs are at the same level for N≥8192N\ge8192 (Figure 15). Appendix C.6 compares the adaptive particle and population paths without regard to timing, through the discrete Fréchet distance. This distance decreases as NN grows for N≥8192N\ge8192 (Figure 17, top). For N≤4096N\le4096 the particle system often skips the first jump, and most systems follow a different path.

W3. We agree with this reading. Section 5 is the motivation for the curvature scenario, and the paper applies Theorem 3.1 in Example 3.1 only. In the revision, Proposition 5.1 is arranged in three groups that correspond to the valley, the saddle and the basin of Figure 1. The text after the proposition lists the three steps that separate these local statements from the theorem. They are a PWGF trajectory joining the three regions, the transfer of the second variations to the sampled curvature, and the verification of the path condition. As numerical evidence in the related model of Appendix C, under a common prescribed schedule the median of rtr_t over trials has a negative log–log slope in NN at every recorded time (Appendix C.4, Figure 10). The verified trajectory that the reviewer suggests would be the natural next step for this model.

Q1. It is a consequence of the covariance-factor comparison in the proof. The paper has no lower bound and no example that attains the rate, and Section 3 now refers to it as our bound on the jump mismatch. In the new experiments of Appendix C.4 we computed the mismatch between the population jump and the NN-particle jump from the same positions. Its log–log slope in NN is close to −1/2-1/2 at the later jumps and −1-1 at the first jump (Figure 10, bottom row), so the N−1/4N^{-1/4} scale is not visible in these runs. Appendix C.4 records this as a limitation of our analysis.

We expect that nondegeneracy improves the rate. The loss comes from taking the square root of a covariance mismatch of order N−1/2N^{-1/2}. Under a nondegeneracy condition on the features that generate the jump covariance, the matrix square root is Lipschitz, which should give N−1/2N^{-1/2}. A proof of this refinement is future work.

Thank you for the clarification. I will increase my score by one point.

We sincerely thank the reviewer for carefully reading our response and for raising the score. Your comments have helped us improve the paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Comment about the revision

We thank the reviewers for their comments. The revised PDF has the following changes. New and rewritten text is in blue. Moved text and redrawn figures are left unmarked.

  1. Order of the main text. The submission stated the theorem in Section 3, treated matrix factorization in Section 4, and gave the assumptions and the proof overview in Section 5. The revision moves the assumptions to the start of Section 3, before the theorem. The proof overview is now Section 4, right after the theorem and its example, and matrix factorization is Section 5. With these moves, Assumptions 5.1–5.3 became Assumptions 3.1–3.3. In the matrix-factorization section, Assumption 4.1 and Proposition 4.1 became Assumption 5.1 and Proposition 5.1. Definition 3.1, Theorem 3.1, Example 3.1, Figures 1 and 2 and the equation numbers are unchanged.
  2. Figures 1 and 2. Figure 1 is redrawn and moved to the Introduction. It shows a population path and nearby runs passing a saddle on a toy two-dimensional landscape, with the curvature along the path and the two curvature-response factors. Figure 2 (matrix factorization) is redrawn in the same style, and its labels follow the items of Proposition 5.1.
  3. New experiments (Appendix C.4–C.6). Appendix C.4 uses a jump schedule fixed in advance and shared by both systems. Appendix C.5 shifts one jump time and adds adaptive runs. Appendix C.6 compares paths without regard to timing.
  4. Clarifications. The Introduction says that Theorem 3.1 is restricted to prescribed schedules, names the obstacle to synchronizing adaptive jump times, and ends with an outline of the paper. Appendix C.1 states the particle method as Algorithm 1. Section 3 says which parts of the theorem are proved and which are assumed, notes that both systems share the Gaussian vector at a jump, and refers to the N−1/4N^{-1/4} jump term as a bound from our proof. The text after Proposition 5.1 relates its three groups to Figure 1 and lists the steps between the proposition and the theorem. Proposition 5.1 is regrouped in the order of Figure 1, which exchanges its items (i) and (ii).

The responses use the numbering of the revised PDF.

AuthorsOpenReview ↗

General Comment about the second revision

After Reviewer LWEU's follow-up comment, we rewrote the explanations in Sections 2 and 3 and uploaded a second revision. The assumptions, the theorem, the proofs and the numbering are the same as before. In the new PDF, the text added this time is in green, and the changes of the first revision are still in blue.

  • Section 2: the section now starts with a short description of the particle method. Algorithm 1 is now in Section 2.2 (it was in Appendix C.1). The subsections have new titles.
  • Section 3: the start of the section explains why the bound uses the curvature along the path. We added short explanations after each assumption, after Definition 3.1, and before and after Theorem 3.1.

Two places are different from what our earlier responses say. Algorithm 1 is in Section 2.2, not in Appendix C.1. The sentence that both systems share the Gaussian vector at a jump (response to Reviewer zDry, Q2) is now in the paragraph after Assumption 3.2, not inside the assumption.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.