acceptodds
Under review as a conference paper at ICLR 2027

Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals

13 views

Abstract

Simulation-free training of latent SDEs relies on a variational posterior process whose one-time marginals are tractable, typically Gaussian. Such marginals, however, do not determine the underlying dynamics: many processes share the same marginals while differing in their temporal structure, and existing parameterizations fix this structure implicitly, which restricts the posterior family and biases the learned model. We introduce Gaussian flow dynamics, which construct stochastic processes directly from smoothly evolving Gaussian marginals while making the marginal-preserving, or gauge, degrees of freedom explicit and parameterizable. The construction admits state-dependent diffusion coefficients and recovers every linear SDE with additive noise and a non-degenerate Gaussian initial distribution. Building on it, we propose Gauge Matching, a simulation-free method for latent SDE learning that combines Gaussian flow dynamics with the SDE Matching objective. Gauge Matching costs at most quadratically in the latent dimension per step, like SDE Matching, but learns the temporal structure of the posterior beyond its one-time marginals. It comes within a nat of Helmholtz-SDE, which computes the gauge from the prior Jacobian at cubic cost, on the linear benchmark where the exact posterior is known, matches it on nonlinear systems, and applies where Helmholtz-SDE does not, to state-dependent noise.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
stochastic differential equations, Gaussian processes, latent sde, diffusion

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6LR7OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary
  • Summary

This paper investigates the limitations of simulation-free variational inference for latent stochastic differential equations (SDEs), focusing on the fact that one-time marginal distributions do not uniquely determine the temporal structure of a stochastic process. The authors introduce Gaussian flow dynamics, which parameterize Gaussian marginal distributions using time-dependent means and covariance factors while explicitly representing the marginal-preserving degrees of freedom through a skew-symmetric gauge generator. Building on this construction, they propose Gauge Matching, which extends SDE Matching by learning the gauge jointly with the variational posterior rather than fixing it implicitly. The theoretical contributions include a representation theorem for linear SDEs with additive noise, a characterization of affine marginal-preserving vector fields, and a strictly convex optimization formulation for selecting the optimal gauge. The authors also establish that the variational family can approximate the exact posterior process in the linear-Gaussian conjugate setting. Experiments on linear and nonlinear stochastic systems show that learning the gauge substantially improves posterior approximation compared with fixed-gauge methods, achieves performance comparable to Helmholtz-SDE at lower computational cost, and extends to state-dependent diffusion coefficients.

Critical strengths
  • Strengths
  1. Clear and well-motivated problem formulation. The paper identifies an important limitation of existing simulation-free latent SDE inference methods: matching one-time marginals does not determine the temporal dependence structure of a stochastic process. Making the associated gauge freedom explicit is a meaningful direction for improving variational posterior expressiveness.

  2. Elegant mathematical formulation. The decomposition of the covariance square root into a Cholesky factor and an orthogonal transformation provides a clean separation between marginal distributions and marginal-preserving dynamics. Parameterizing the gauge through a skew-symmetric generator avoids explicitly constructing the orthogonal transformation while preserving the simulation-free property.

Critical weaknesses
  • Weaknesses
  1. The conceptual novelty relative to Helmholtz-SDE is somewhat limited. The importance of gauge freedom in simulation-free latent SDE inference and the limitations of fixed-gauge posterior parameterizations have already been established by Smith et al. (2026). The current paper builds directly on this observation and proposes to learn the skew-symmetric gauge generator instead of computing a correction based on the prior drift. This is a useful methodological development, particularly in terms of computational efficiency and compatibility with state-dependent diffusion. However, the fundamental idea of exploiting marginal-preserving dynamics is not entirely new. The paper also acknowledges that Smith et al. briefly discussed learning the gauge in their appendix. I would encourage the authors to clarify more precisely which aspects of the proposed approach go beyond this earlier suggestion. In particular, the contribution appears to lie primarily in the efficient parameterization, integration with simulation-free optimization, and systematic empirical validation, rather than in introducing gauge-based posterior inference itself.

  2. The expressiveness of the learned gauge remains restricted to affine marginal-preserving dynamics. The theoretical completeness results are established for linear SDEs with additive noise and non-degenerate Gaussian initial distributions. Proposition 2 characterizes the affine vector fields that preserve Gaussian marginals, but the learned gauge does not represent general nonlinear marginal-preserving vector fields. For nonlinear prior processes, the optimal posterior correction may contain nonlinear components that cannot be captured by the proposed skew-symmetric linear parameterization. Consequently, optimizing the gauge can improve the posterior within the chosen family without eliminating the underlying expressiveness limitation. Although this restriction is acknowledged in the conclusion, its practical significance is not extensively evaluated. In particular, it remains unclear how the approximation quality degrades when the nonlinear marginal-preserving component becomes substantial. A more explicit discussion of the relationship between the affine gauge family and the unrestricted optimal posterior process would strengthen the paper.

  3. The computational complexity claims require careful qualification. The paper emphasizes that Gauge Matching has quadratic computational complexity in the latent dimension, compared with the cubic operations required by Helmholtz-SDE. This is an important advantage. However, the stated complexity relies on efficient evaluation of the diffusion covariance operations and its divergence. For general dense, state-dependent diffusion coefficients, these operations may be substantially more expensive. Moreover, the gauge parameterization introduces (d(d-1)/2) additional outputs from the inference network. Thus, even if the associated linear algebra scales quadratically, the overall training cost also depends on the architecture used to predict these parameters and their time derivatives. The reported timing experiments show favorable performance up to moderate latent dimensions, but the practical scaling advantages would benefit from a more detailed separation of network evaluation, automatic differentiation, gauge parameterization, and matrix-operation costs.

Questions for authors
  • Questions for the Authors
  1. Expressiveness beyond affine gauges: Can the authors characterize or empirically investigate the approximation error caused by restricting the marginal-preserving correction to affine vector fields? Are there nonlinear SDE examples where the optimal gauge within the proposed family remains substantially suboptimal relative to a more expressive posterior process?

  2. Novelty relative to Helmholtz-SDE: Since Smith et al. (2026) already discuss learning the gauge generator as a possible extension, could the authors clarify the precise conceptual and technical differences between that suggestion and the proposed Gauge Matching formulation?

  3. State-dependent diffusion: The ability to handle state-dependent noise is an important contribution. Could the authors provide a more detailed analysis of the computational and statistical trade-offs for different diffusion structures, including dense state-dependent covariance matrices?

4 comments

Part 1 of 2

We thank the reviewer for the thorough review and for finding the problem formulation clear and well motivated and the mathematical formulation elegant. We address each weakness together with the corresponding question; table, figure and equation numbers refer to the submitted version.

Weaknesses:

  1. (Question 2.) The general response, points 1–3, states what goes beyond the suggestion in Appendix A.5 of [1]. In short: [1] compute the gauge from the prior drift at every evaluation; we make it a learned coordinate of the variational family within the SDE Matching framework, through the factorization S=LOS = LO, which retains the closed form that [1] saw as sacrificed, and we prove that this loses nothing (Theorem 1, Propositions 2 and 3, Corollary 4), with the order-one correction identified as the optimal gauge of the linearized prior. No prior Jacobian, no cubic solve and state-dependent diffusion follow; the last is the limitation that [1] name "learning the divergence-free correction" would lift. In the camera-ready version, if accepted, we will rewrite the paragraphs "Relation to Helmholtz-SDE" (Section 4) and "Simulation-Free Methods and the Gauge" (Section 6) along these lines.

  2. (Question 1.) We agree, and the relation is as follows. For state-independent Σ\Sigma, Theorem 1 and Proposition 2 say that the affine gauge family is exactly the set of Gaussian processes with the given marginals. The order-one correction of Helmholtz-SDE lies in the same family; beyond it lie the corrections of order ℓ≥2\ell \ge 2, for which [1] observed no notable improvement, and the state-dependent generator of Appendix B.1, which contains all of them. The distance of the affine optimum from the unrestricted optimum can be bounded: because the qq-weighted Helmholtz decomposition of an affine field is affine, the linearization of hh about the posterior mean projects identically onto both families, and at the same marginals the two minimal path-space KL terms differ by at most half the posterior mean square of the Taylor remainder h(Z)−h(m)−∇h(m)(Z−m)h(Z) - h(m) - \nabla h(m)(Z - m) in the Σ−1\Sigma^{-1}-weighted norm, per unit time, which to leading order scales with the square of the posterior variance and the squared second derivatives of the drift, relative to the process noise. The affine restriction is therefore immaterial for posteriors that are narrow on the scale over which the prior Jacobian varies (the Lotka–Volterra system in log coordinates, where the order-one correction is already optimal, Table 5) and can matter for wide posteriors under strongly nonlinear drifts. The Stuart–Landau experiment (Table 4) is that regime: the order-one correction misses the optimal affine gauge by 8κ2tr(P)det⁡(P)/s2≈0.138 \kappa^2 \mathrm{tr}(P) \det(P) / s^2 \approx 0.13 nats per unit time (PP the posterior covariance, κ\kappa the shear, ss the noise scale), the learned gauge by a tenth of that, and Gauge Matching is better by 1.6 ± 0.3 nats. Prompted by the question, we have also computed the remaining distance on this system: the residual is cubic, so the unrestricted optimum lies in the six-dimensional space of degree-two generators of Appendix B.1 for d=2d = 2, and a cross-fitted least-squares fit on the Monte Carlo samples of the gauge-excess estimator gives it at the fitted marginals (5 seeds). The learned gauge is 0.38 ± 0.01 nats per unit time above the unrestricted optimum (3.8 nats per trajectory, 15% of the path-space KL term), of which 0.37 is the affine restriction and 0.01 the optimization; the order-one correction is 0.43 above it and the fixed gauges 0.73–0.76, so the learned affine gauge recovers about half of what the fixed gauges miss in this stress case. Doubling and quadrupling the posterior standard deviation raises the restriction to 12 and 610 nats per unit time, the steep growth the Taylor argument predicts for a cubic drift; examples where the affine optimum is substantially suboptimal are therefore wide posteriors under strongly nonlinear drifts, and the state-dependent generator of Appendix B.1, trained the same way, is the construction for them. We will add the bound with its proof to Appendix C.2, these numbers to Appendix D.2, and the discussion to Section 4.

(Our response continues in the comment below.)

Part 2 of 2

  1. (Question 3.) We agree and will qualify the statement. The O(d2)\mathcal{O}(d^2) is the linear algebra of (15) at one (t,z)(t, z) beyond the network evaluations (caption of Table 1): with Zt=m+LεZ_t = m + L\varepsilon the whitened state is η=ε\eta = \varepsilon, so the drift needs L˙ε\dot L\varepsilon, LΩεL\Omega\varepsilon, one triangular solve and ΣL−⊤ε\Sigma L^{-\top}\varepsilon. The gauge head is one linear layer with d(d−1)/2d(d-1)/2 outputs (2016 outputs and 0.2M parameters at d=64d = 64); the full Cholesky factor adds d(d+1)/2d(d+1)/2 outputs in the same way for every full-covariance posterior, including Helmholtz-SDE; only m˙\dot m and L˙\dot L need time derivatives, from one forward-mode pass regardless of dd, and Ωϕ\Omega_\phi needs none. Helmholtz-SDE instead pays dd forward-mode passes for the prior Jacobian and an O(d3)\mathcal{O}(d^3) eigendecomposition and Sylvester solve per evaluation, with gradients through both. Four posterior processes separate the components, each of the first three adding one to the previous and Helmholtz-SDE replacing the gauge head by the Jacobian and solve (ms per training iteration, batch of 256, forward and backward pass and optimizer step; GPU RTX 3090 as in Figure 5C, CPU with 16 threads; Appendix D.1):

    ddSDE Matching+ full LϕL_\phi (Cholesky gauge)+ gauge head (Gauge Matching)Cholesky gauge + Jacobian and solve (Helmholtz-SDE)
    8 (GPU)10.212.012.716.0
    32 (GPU)12.114.115.119.9
    128 (GPU)10.214.215.333.1
    64 (CPU)24.935.242.1181.1

    On the CPU, where no launch overhead hides the arithmetic, the full LϕL_\phi adds 10 ms, the gauge head 7 ms, and the Jacobian with the solve 146 ms over the Cholesky gauge at d=64d = 64; total training time scales the same way (general response, point 4). For a dense state-dependent Σ\Sigma, the product Σw\Sigma w stays O(d2)\mathcal{O}(d^2), but the exact divergence costs dd Jacobian–vector products, or one with a Hutchinson-type estimate, which keeps a valid, looser bound on the negative log-likelihood that tightens with more probes; and the whitening of the residual by gg, which every method pays, becomes a d×dd \times d solve unless gg is triangular. The divergence belongs to the ODE-to-SDE conversion (Equation 7), is present for every gauge and vanishes for state-independent Σ\Sigma, so it is not a cost of the gauge. On the statistical side, the optimal gauge is the unique minimizer of a strictly convex quadratic for any Σ\Sigma (Proposition 3), so the learned gauge has a well-defined target for every structure; for the diagonal Σ\Sigma of latent SDE practice (Σii\Sigma_{ii} a function of ziz_i [2, 3]) the estimator is that of the state-independent case with an exact divergence, which is the setting of Tables 5–6; we have not run dense state-dependent diffusion coefficients. We will state the quadratic cost for the diagonal structure and the cost of the dense case explicitly in Section 3.

We hope that these clarifications, in particular the account of what goes beyond the extension sketched in [1], the bound relating the affine family to the unrestricted posterior process, and the component-wise cost separation, will strengthen your support for the acceptance of our paper!

References:

[1] Smith et al. "Closing the approximation gap in simulation-free latent SDEs."

[2] Li et al. "Scalable gradients for stochastic differential equations."

[3] Bartosh et al. "SDE Matching: Scalable and simulation-free training of latent stochastic differential equations."

Thanks for the authors' response, which have resolved most of my concerns. Thus, I have updated my score.

We are glad that our response resolved the reviewer's concerns, and we thank the reviewer for updating the score. Thank you very much for your time and effort!

Reviewer_2QzrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes gauge matching as an improvement on simulation-free training of neural SDEs by decomposing the time dependent diffusion coefficient in a Gaussian flow until a lower triangular and orthogonal form, using this to simplify some of the maths for the drift term in an equivalent SDE (in law).

Critical strengths

Simulation-free training of neural SDEs is an interesting topic. For strengths:

  1. Detailed discussion of related work and good choice of comparisons in the experimental setting,
  2. Considered non-linear SDEs
Critical weaknesses
  1. Whilst mentioned in prose in §1, it remains unclear from the methods section in §3 and §4 showing how this work compares to prior work [1], like a table, figure, or set of equations showing "theirs" vs "ours". For the reader this makes it harder to graps what the contribution is.
  2. Eq (11) and subsequent forms used in the paper seem quite expensive to evaluate (as noted at the bottom of page 4) with Eq (15). Forward-mode autodiff is used to obtain m˙\dot m and L˙\dot L and lots of VJPs or JVPs are need to compute ⟨∇∣Σ(t,x)⟩\langle \nabla \mid \Sigma(t, x)\rangle as noted at O(d2)\mathcal O(d^2). For larger systems this seems cost prohibitive which is why CNFs (neural ODEs) trained with MLE used Hutchinson's trick to make the divergence more computationally tractable (but approximate).
  3. In §2-4 we spent most of the paper looking at SDEs with state dependent diffusions. But page 4 focuses mostly on the setting of linear SDEs with additive noise, namely (12). How does this extend to multiplicative-noise?
  4. On a related note the multiplicative noise results are relegated to the appendix and I don't believe the main paper currently stands on it's own. The experimental results section is quite hard to follow and it is not clear to me that the claimed performance benefits are empirically verified.

References

[1] Henry D Smith, Brian L Trippe, and Scott W Linderman. Closing the approximation gap in simulation-free latent SDEs. arXiv preprint arXiv:2606.16138, 2026

Questions for authors
  1. Why does [1] have a higher cost of O(d3)\mathcal O(d^3) compared to Gauge matching O(d2)\mathcal O(d^2)? I don't see the Jacobian calculation in (17) just the gradient of hh which is cheaper to compute with autograd.
  2. From Table 2 it seems that the prior work of Helmholtz-SDE is generally superior in terms of performance with better metrics and lower variance as well compared to both instances of Gauge matching. Additionally, I don't see a comparison on wallclock or other measures of computational cost so I am left unsure of what the benefit of Gauge matching, right now I would be inclined to use Helmholtz-SDEs. Thus why should I turn to use Gauge matching to solve my SDE modelling problem as a practitioner?

I believe most of my concerns can be addressed during the rebuttal period and would be inclined to update my assessment, my review is for the manuscript as submitted.

4 comments

Part 1 of 2

We thank the reviewer for the careful reading and for finding the topic interesting, the related work detailed and the comparisons well chosen. We also appreciate the reviewer's openness to updating the assessment. Below we address the weaknesses and questions; table, figure and equation numbers refer to the submitted version. For the overall comparison with Helmholtz-SDE and our recommendation, please see the general response.

One point of the summary first, because it bears on the comparison: the factorization S=LOS = LO (Equation 13) is of the square root of the posterior covariance, Zt=m+SεZ_t = m + S\varepsilon, not of the diffusion coefficient gg, which is shared with the prior. LL carries the marginals; OO leaves them unchanged and enters the dynamics only through Ω=O˙O⊤\Omega = \dot O O^\top. This generator is the gauge, which fixed-gauge methods set implicitly, Helmholtz-SDE computes from the prior drift, and Gauge Matching learns. We will state this in the first paragraph of Section 3.

Weaknesses:

  1. All posterior processes in the paper share the marginals N(m,LL⊤)\mathcal{N}(m, LL^\top) and the drift (15), in which the gauge is the single term LΩηL\Omega\eta; they differ only in where Ω\Omega comes from. In the notation of (17), with G=L⊤Σ−1LG = L^\top\Sigma^{-1}L,

    Ωfixed=F(L,L˙,Σ),G ΩH+ΩH G=L⊤(Σ−1Jˉm−Jˉm⊤Σ−1)L,    Jˉm=∇zh(t,m)−L˙L−1,ΩGM=Ωϕ(t,X).\Omega_{\mathrm{fixed}} = F(L, \dot L, \Sigma), \qquad G\,\Omega_{\mathrm{H}} + \Omega_{\mathrm{H}}\,G = L^\top\bigl(\Sigma^{-1}\bar J_m - \bar J_m^\top\Sigma^{-1}\bigr)L, \;\; \bar J_m = \nabla_z h(t, m) - \dot L L^{-1}, \qquad \Omega_{\mathrm{GM}} = \Omega_\phi(t, X).

    The fixed gauges (SDE Matching, the Cholesky gauge, the symmetric gauge of [4]) prescribe Ω\Omega as a function of LL, L˙\dot L and Σ\Sigma (Appendix B). Helmholtz-SDE at order one recomputes Ω\Omega at every evaluation from the d×dd \times d Jacobian of the prior drift at the mean and an O(d3)\mathcal{O}(d^3) solve, for state-independent Σ\Sigma (Appendix C.2). Gauge Matching reads Ωϕ\Omega_\phi off the posterior network, trained by the same ELBO as mϕm_\phi and LϕL_\phi, for any Σ(t,z)\Sigma(t, z). Proposition 3 ties them together: for state-independent Σ\Sigma the path-space KL term is a strictly convex quadratic in Ω\Omega minimized by (17) with the expected Jacobian; a fixed gauge attains it only under the symmetry conditions of Appendix C.2; ΩH\Omega_{\mathrm{H}} is the minimizer for the linearized prior; Gauge Matching minimizes the exact objective over the same family. In the camera-ready version, if accepted, we will add this display and enumeration to Section 4 above Table 1 and extend Table 1 with the source of Ω\Omega.

  2. Equation 15 is cheaper than its form suggests. m˙\dot m and L˙\dot L come from one forward-mode pass in tt, as in SDE Matching; Ωϕ\Omega_\phi needs no derivative. With Zt=m+LεZ_t = m + L\varepsilon the whitened state is η=ε\eta = \varepsilon, so the drift needs the products L˙ε\dot L\varepsilon and LΩεL\Omega\varepsilon and one triangular solve: O(d2)\mathcal{O}(d^2), of which the gauge is the single product LΩεL\Omega\varepsilon. The terms in Σ\Sigma belong to the ODE-to-SDE conversion (Equation 7) and are present for every gauge; the divergence vanishes for state-independent Σ\Sigma (most of our experiments), and for the diagonal gg with gi(zi)g_i(z_i) used in latent SDE models [2, 3] and in Tables 5–6 it is the vector of dd derivatives ∂iΣii\partial_i\Sigma_{ii}, one pass, with no Jacobian–vector products in zz anywhere. Only a dense state-dependent Σ\Sigma would need dd Jacobian–vector products or, as the reviewer says, a Hutchinson-type estimate, which still gives a valid, looser bound on the negative log-likelihood that tightens with more probes (Appendix B.1 uses the same device for the state-dependent generator). Measured per iteration (RTX 3090, batch of 256, forward and backward pass): 9–12 ms for SDE Matching, 11–15 ms for Gauge Matching and 14–33 ms for Helmholtz-SDE for dd from 2 to 128; on a 16-thread CPU at d=64d = 64, 25, 42 and 181 ms (Appendix D.1). The cost that grows with dd is Helmholtz-SDE's Jacobian and solve (Question 1). We will add the dense-Σ\Sigma qualification to Section 3.

(Our response continues in the comment below.)

Part 2 of 2

  1. Sections 3 and 4 hold for a general diffusion coefficient: Equations 11 and 15 hold for any Σ(t,z)\Sigma(t, z), Proposition 2 does not involve Σ\Sigma, and the strict convexity in Proposition 3 holds for any uniformly positive definite Σ(t,z)\Sigma(t, z). The restriction to additive noise on page 4 is only for Theorem 1, a completeness statement (the family contains every such SDE), not a restriction of the family; we will say so there. With a state-dependent Σ\Sigma the posterior process keeps its Gaussian marginals but is no longer a Gaussian process, and the optimal gauge is no longer the Sylvester solution (17) but a d(d−1)/2d(d-1)/2-dimensional linear system whose coefficients are posterior expectations with Σ(Z)−1\Sigma(Z)^{-1} inside (Appendix C.2), to be estimated at every evaluation. This is where the Helmholtz-SDE correction stops, since [1] requires a state-independent Σ\Sigma, and the learned gauge continues: Gauge Matching never forms the system. Empirically: Table 5 (Lotka–Volterra with multiplicative noise in the original coordinates: period error 22% to 13%, better on all three metrics on all 10 seeds) and Table 6 (d=8d = 8, multiplicative noise: negative ELBO lower by 6.6 ± 0.8 nats with the generative model known and 4.8 ± 0.3 with it learned).

  2. We agree that these results belong in the main text and that Section 5 asked a lot of the reader. In the camera-ready version, if accepted, Table 5 and a per-iteration cost table move into Section 5.2, and each paragraph of Section 5 will open with the question it answers. The claimed benefits are verified as follows: accuracy, within a nat of the exact gauge in the conjugate setting (Figure 2) and equal to Helmholtz-SDE on the Lorenz system across nine observation protocols (Tables 2–3); cost, 1.17x against 1.43x the training time of SDE Matching on the Lorenz system and the per-iteration numbers above (Figure 5C); scope, Table 4 (known cubic drift, 1.6 ± 0.3 nats better on every seed) and Tables 5–6 (state-dependent noise).

Questions:

  1. h:Rd→Rdh : \mathbb{R}^d \to \mathbb{R}^d is vector-valued, so ∇zh\nabla_z h in (17) (entries ∂hi/∂zj\partial h_i / \partial z_j, Proposition 3) is its d×dd \times d Jacobian, not a gradient. Automatic differentiation gives one row or column per pass, so it costs dd passes through the prior drift network, as [1] also state (their Appendix B.3: O(Kℓ+1Cr)\mathcal{O}(K^{\ell+1} C_r), with KK their latent dimension and CrC_r one residual evaluation). Equation 17 is then a Sylvester equation in the d(d−1)/2d(d-1)/2 entries of Ω\Omega, which [1] solve after an eigendecomposition at cubic cost ("The computation scales cubically in the latent dimension", their Section 4.3); our reimplementation does the same for d≥8d \ge 8 (Appendix D.1). Gauge Matching replaces both steps by a network output, hence the timings in Weakness 2. We will make the Jacobian and its cost explicit next to (17).

  2. On Table 2: Gauge Matching (53.2 ± 0.7) and Helmholtz-SDE (53.0 ± 0.6) agree within two standard errors, as do the latent log-likelihood (−0.06 ± 0.01 for both) and the autocorrelation error (0.056 ± 0.009 against 0.052 ± 0.006), with error bars of comparable size in all rows. The gauge excess (0.001 against 0.044 nats per unit time) is lower for Helmholtz-SDE by construction: the learned Lorenz drift is nearly affine over the posterior support (Appendix D.2), so the order-one correction is the optimal gauge there, and the learned gauge reaches the same optimum to within 0.2 nats over the trajectory, the price of learning it. On wall-clock: relative to SDE Matching, training took 1.17x for Gauge Matching and 1.43x for Helmholtz-SDE on this system, for the same number of iterations, which both methods need to converge (Appendix D.2 and the training curves); we will add this column to Table 2. On why Gauge Matching: please see Weakness 4 above and the general response (points 4 and 6, and the recommendation); we add that the diagonal-LϕL_\phi row of Table 2 is SDE Matching plus one linear head and recovers 22.5 of the 24.7 nats.

We hope that these clarifications, in particular the equations comparing the methods, the cost of (15) and (17) with measured timings, and the planned restructuring of Section 5, address your concerns and will strengthen your support for our paper!

References:

[1] Smith et al. "Closing the approximation gap in simulation-free latent SDEs."

[2] Li et al. "Scalable gradients for stochastic differential equations."

[3] Bartosh et al. "SDE Matching: Scalable and simulation-free training of latent stochastic differential equations."

[4] Course and Nair. "Amortized reparametrization: Efficient and scalable variational inference for latent SDEs."

Looks good, I have updated my score in the good faith assumption that the authors will update their prose to improve the clarity of the paper.

We thank the reviewer for the response and for updating the assessment; the camera-ready version will contain the changes described above. Thank you very much for your time and effort!

Reviewer_5EZHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper draws on inspiration from Smith et al. (2026) to solve latent stochastic differential equations via variational inference, rather than simulation, efficiently and accurately. Their solution is aimed at solving the equivalence between one-time marginal distributions of SDEs, which can differ in their drift terms by q-divergence free vector fields, leading to very different joint distributions over time. Smith et al. (2026) note that in the case of affine SDEs with Gaussian marginals, the paths are characterized by a square-root factor of the covariance at time t and a skew-symmetric matrix called the gauge. The authors of this paper explicitly parameterize the square-root factor and gauge as neural networks and train latent SDEs using this strategy.

They apply their method to several datasets, demonstrating that gauge matching outperforms methods with fixed gauges and matches Smith et al. 2026 in the majority of cases. In cases where the true gauge is known, gauge matching recovers it. Overall, this paper introduces a promising technique for efficiently learning latent SDEs without simulation.

Critical strengths

The main strengths of this paper are the theoretical framing and the set of experiments. The authors provide a compelling set of theoretical results building on Helmholtz-SDE to demonstrate that in the commonly-used linear SDEs, optimizing over square-root factors of the Gaussian flow covariance and the gauge is sufficient to minimize the path-space KL term of the latent SDE ELBO. They then show that in practice, when the true gauge is known, their fitting procedure recovers that gauge, and outperforms methods that specify the gauge prior to fitting. It's a nice combination of solid theoretical results and experiments.

Critical weaknesses

The main weakness of this paper is that the primary advantages of this method are not in the main paper. As stated, the advantages of this method over Smith et al. (2026) are 1. its computational efficiency and 2. its applicability to SDEs with state-dependent diffusion coefficients. However, the cases where gauge matching outperforms Helmholtz-SDE are only briefly mentioned in the main text and are mostly left to e appendix. Given that the utility of this method depends on its comparison to potentially more computationally expensive methods, it's important to highlight those results. Without these comparisons, it's a neat approach to fitting latent SDEs, but the advantage of this approach over other methods isn't entirely clear, and it feels more like an alternative training technique for a prior piece of work.

Questions for authors

Fig. 5: It seems like the Helmholtz-SDE takes longer per iteration than gauge matching at all latent dimensions, but in the main text it's stated that gauge matching costs 1.3-3.2 times more per GPU training iteration at d ≤\leq 128, the dimensions reported in fig. 5. What's the reason for the discrepancy between these results?

Fig. 6: why does SDE matching with a learned observation model outperform Helmholtz-SDE here?

Are there any common applications where ∇zh(t,m)\nabla_z h(t,m) is not affine in zz?

3 comments

We thank the reviewer for the careful reading and for finding the theory compelling and its combination with the experiments nice. Below we address the weakness and the questions; table, figure and equation numbers refer to the submitted version.

Weaknesses:

We agree that the two practical properties that distinguish Gauge Matching from Helmholtz-SDE, its cost and its applicability to state-dependent diffusion coefficients, deserve more room in the main text than Appendix D and the summary sentences of Section 5.2. In the camera-ready version, if accepted, we will use the additional page to move the Lotka–Volterra table (Table 5) and a per-iteration cost table into Section 5.2, add a training-time column to Table 2 (relative to SDE Matching: 1.17x for Gauge Matching, 1.43x for Helmholtz-SDE), and state the comparison in one sentence at the end of Section 4: the same accuracy where Helmholtz-SDE applies and its linearization is accurate, at lower cost; higher accuracy where the linearization is poor (Table 4); the only one of the two for state-dependent noise (Tables 5–6). On the relation to Smith et al., please see the general response, points 1–3: the gauge becomes a learned coordinate of the variational family rather than a correction computed from the prior at every evaluation, and the completeness and optimality results show that this loses nothing.

Questions:

  1. Both ratios in that sentence are relative to SDE Matching: per GPU iteration, Gauge Matching costs 1.05–1.5x SDE Matching for d≤128d \le 128 and Helmholtz-SDE 1.3–3.2x, as Figure 5C shows. In absolute terms (RTX 3090, batch of 256 trials, forward and backward pass), 9–12 ms for SDE Matching, 11–15 ms for Gauge Matching and 14–33 ms for Helmholtz-SDE for dd from 2 to 128; on a CPU, where the O(d3)\mathcal{O}(d^3) solve is not hidden behind kernel launches, Helmholtz-SDE takes 6.4x and 7.3x as long as SDE Matching at d=32d = 32 and 64, against 1.4x and 1.7x for Gauge Matching (Appendix D.1). We will reword the sentence.

  2. With a learned observation matrix CC, every full-covariance posterior (Cholesky gauge, Helmholtz-SDE and Gauge Matching alike) converges to a rank-deficient CC: some latent directions are unobserved and revert to the prior, which raises the reconstruction term, while the diagonal posterior of SDE Matching does not (Appendix D.1, Remarks). This is a local optimum of the joint optimization with a full LϕL_\phi, not an effect of the gauge: within the full-covariance family the ordering is unchanged (the Cholesky gauge stays 7.1 and 1.1 nats above Helmholtz-SDE at d=16d = 16 and 32), and with CC fixed (Figure 6A–B) the fixed gauges trail Helmholtz-SDE on every seed at d=8d = 8, while Gauge Matching with a diagonal LϕL_\phi, which is SDE Matching plus the gauge head, is the best model at d=16d = 16 at the longer budget. At d=32d = 32 the diagonal LϕL_\phi is fitted faster than a full one, so SDE Matching overtakes the full-covariance models there despite a ten times larger gauge excess (Appendix D.1). For larger dd the configuration to try first is therefore a diagonal LϕL_\phi with a learned gauge, which we have not run with a learned CC; Section 5.2 will state the recommendation and this limitation.

  3. Yes, whenever the drift is known or constrained and is not a polynomial of degree at most two: population models in log coordinates (exponential terms; Table 5), oscillators with cubic terms (Table 4), FitzHugh–Nagumo or conductance-based neuron models, kinetics with saturating rates, mechanical systems with trigonometric terms. Two caveats, both in the paper: with a drift network learned jointly with the posterior, the fitted drift is nearly affine over the posterior support in all our experiments (Appendix D.2), so, as Proposition 3 predicts, the two methods then agree in accuracy and differ in cost; and the difference requires a posterior that is wide relative to the curvature of the drift (in log coordinates the Lotka–Volterra posterior is narrow and the two agree, Table 5). The broader practical case is state-dependent noise, where the Helmholtz-SDE correction is not available at all (Tables 5–6).

We hope that these clarifications, the cost comparison in absolute numbers and the planned changes to Sections 4 and 5 will strengthen your support for the acceptance of our paper!

I appreciate the authors' thoughtful responses to my questions and those of the other reviewers. I think their proposed changes sharpen the description of their method and will allow readers to assess its quality more clearly. I would like the AC to note that although my rating has not changed, my support for this paper has increased and is between a weak and strong accept (~3.5).

We thank the reviewer for the response and for the increased support of our paper. Thank you very much for your time and effort!

Comments on the paper

2 comments
AuthorsOpenReview ↗

Gauge Matching and Helmholtz-SDE: what is new, and when to use which

All three reviews ask how Gauge Matching relates to Helmholtz-SDE [1] and when to prefer it; we answer once here and refer to it below.

  1. Parallel developments of SDE Matching. Both address the gauge freedom identified in [1]. Helmholtz-SDE adds to a fixed-gauge posterior a correction computed from the prior drift at every evaluation; we develop a parallel approach within the SDE Matching framework and make the gauge a learned coordinate of the variational family. Appendix A.5 of [1] mentions learning the skew term in passing, calls it "sacrificing the closed-form structure" of SDE Matching, and leaves it undeveloped. A passing suggestion is not a method: the parameterization, learning algorithm, theory and evaluation that make it work are what this paper provides.

  2. The factorization S=LOS = LO. Writing the square root of the posterior covariance as a Cholesky factor times an orthogonal matrix separates the marginals (LL) from the gauge (OO). The dynamics depend on OO only through the skew-symmetric generator Ω=O˙O⊤\Omega = \dot O O^\top (the gauge field of Equation 14), so OO is never formed and Ω\Omega is one more head of the posterior network, trained by the same ELBO as mm and LL; or OO is a Householder flow with an explicit flow map (Appendix B). Marginal sampling and drift evaluation stay in closed form, which resolves the objection of [1]; the affine correction of Helmholtz-SDE is one point of a family that SDE Matching itself represents and learns.

  3. The theory establishes more than convenience. Theorem 1: every linear SDE with additive noise and a non-degenerate initial law is in the family. Proposition 2: the affine marginal-preserving fields are exactly the gauge fields LΩL−1(z−m)L\Omega L^{-1}(z - m). Proposition 3: the path-space KL term is a strictly convex quadratic in Ω\Omega for any positive definite Σ(t,z)\Sigma(t, z), with the closed-form minimizer (17) for state-independent Σ\Sigma; the order-one correction of Helmholtz-SDE is this minimizer for the linearized prior, so the two coincide exactly when the expected Jacobian of the prior drift equals its Jacobian at the mean. Corollary 4: exactness in the conjugate setting for the learned family.

  4. Practical consequences. Gauge Matching needs no Jacobian of the prior drift (dd forward-mode passes per evaluation for Helmholtz-SDE) and no cubic Sylvester solve: per training iteration it costs 1.05–1.5x SDE Matching on a GPU for d≤128d \le 128 against 1.3–3.2x for Helmholtz-SDE, and 1.7x against 7.3x on a CPU at d=64d = 64 (Figure 5C, Appendix D.1). The ratios carry over to total training time: all methods run the same number of iterations, and in the training curves Gauge Matching and Helmholtz-SDE converge in the same number of them (Lorenz: 1.17x against 1.43x of SDE Matching's time; Appendix D.2). It applies to state-dependent diffusion coefficients, where the Helmholtz-SDE correction is unavailable (Appendix C.2): exactly and cheaply for the diagonal structures of latent SDE practice (Tables 5–6), and for a dense Σ\Sigma with a single-probe estimate of ∇⋅Σ\nabla \cdot \Sigma that still gives a valid, looser bound on the negative log-likelihood, tightening with more probes. Its gauge is a bounded network output; the Sylvester solve is ill-conditioned for near-singular covariances (Appendix D.2).

  5. Beyond affine gauges. A state-dependent generator (Appendix B.1) spans every marginal-preserving field, including the corrections of all orders of [1], and stays simulation-free. At fixed marginals the affine optimum is within half the posterior mean square of the Taylor remainder of the prior drift of the unrestricted one: immaterial for posteriors narrow on the scale of the prior Jacobian's variation, and 0.37 nats per unit time on the Stuart–Landau stress case built against it (reply to Reviewer 6LR7, Weakness 2).

  6. Demonstrated. Within a nat of the exact gauge in the conjugate setting and 89–97% of the fixed gauges' gap removed up to d=64d = 64 (Figures 2, 5A); equal to Helmholtz-SDE on the Lorenz system at lower cost (Tables 2–3); better by 1.6 ± 0.3 nats for a known cubic drift (Table 4); the only one of the two for multiplicative noise (Tables 5–6).

Recommendation. Gauge Matching is our recommended default. Helmholtz-SDE is mainly worth considering for low-dimensional problems with state-independent diffusion, particularly when the prior drift is close to affine or quadratic over the posterior support, where its Helmholtz-SDE correction is accurate and inexpensive.

In the camera-ready version, if accepted, Section 4 will state points 1–3 with a display of how each method obtains Ω\Omega and an extended Table 1, move the Lotka–Volterra table and a cost table into Section 5.2, and add the recommendation; the revised PDF adds only the AC's two requests, Appendix B.1 and one sentence in Section 5.2

[1] Smith et al. "Closing the approximation gap in simulation-free latent SDEs."

AuthorsOpenReview ↗

Summary of the discussion

We thank the reviewers and the Area Chair for their time and feedback. We are grateful for the reviewers' recognition of the problem formulation as "clear and well-motivated", the mathematical formulation as "elegant", the theory as "a compelling set of theoretical results" that makes "a nice combination of solid theoretical results and experiments", and the related-work discussion and the choice of comparisons as strengths.

During the discussion we clarified the relation to Helmholtz-SDE and when to prefer each method (general response above), the cost of the drift evaluation against the Jacobian and Sylvester solve of Helmholtz-SDE, with measured per-iteration and end-to-end training times, how the construction covers state-dependent noise, and the readings of Table 2 and Figure 6; we also computed the distance of the affine gauge family from the unrestricted optimum on the Stuart–Landau system. The revised PDF includes Appendix B.1 on state-dependent gauge generators. For the camera-ready version we have committed to the display of how each method obtains the gauge with the extended Table 1 in Section 4, the Lotka–Volterra and cost tables and the training-time column in Section 5.2, opening each experiment with its question, the clarified Section 3, the bound and the computed gap in Appendices C.2 and D.2, and the recommendation.

Following the discussion, Reviewer 2Qzr raised their score to 3 and Reviewer 6LR7 to 4, and Reviewer 5EZH stated that their support increased to between weak and strong acceptance. We trust that the discussion will bolster your support for acceptance!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.