- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
- Summary
This paper investigates the limitations of simulation-free variational inference for latent stochastic differential equations (SDEs), focusing on the fact that one-time marginal distributions do not uniquely determine the temporal structure of a stochastic process. The authors introduce Gaussian flow dynamics, which parameterize Gaussian marginal distributions using time-dependent means and covariance factors while explicitly representing the marginal-preserving degrees of freedom through a skew-symmetric gauge generator. Building on this construction, they propose Gauge Matching, which extends SDE Matching by learning the gauge jointly with the variational posterior rather than fixing it implicitly. The theoretical contributions include a representation theorem for linear SDEs with additive noise, a characterization of affine marginal-preserving vector fields, and a strictly convex optimization formulation for selecting the optimal gauge. The authors also establish that the variational family can approximate the exact posterior process in the linear-Gaussian conjugate setting. Experiments on linear and nonlinear stochastic systems show that learning the gauge substantially improves posterior approximation compared with fixed-gauge methods, achieves performance comparable to Helmholtz-SDE at lower computational cost, and extends to state-dependent diffusion coefficients.
Critical strengths
- Strengths
-
Clear and well-motivated problem formulation. The paper identifies an important limitation of existing simulation-free latent SDE inference methods: matching one-time marginals does not determine the temporal dependence structure of a stochastic process. Making the associated gauge freedom explicit is a meaningful direction for improving variational posterior expressiveness.
-
Elegant mathematical formulation. The decomposition of the covariance square root into a Cholesky factor and an orthogonal transformation provides a clean separation between marginal distributions and marginal-preserving dynamics. Parameterizing the gauge through a skew-symmetric generator avoids explicitly constructing the orthogonal transformation while preserving the simulation-free property.
Critical weaknesses
- Weaknesses
-
The conceptual novelty relative to Helmholtz-SDE is somewhat limited. The importance of gauge freedom in simulation-free latent SDE inference and the limitations of fixed-gauge posterior parameterizations have already been established by Smith et al. (2026). The current paper builds directly on this observation and proposes to learn the skew-symmetric gauge generator instead of computing a correction based on the prior drift. This is a useful methodological development, particularly in terms of computational efficiency and compatibility with state-dependent diffusion. However, the fundamental idea of exploiting marginal-preserving dynamics is not entirely new. The paper also acknowledges that Smith et al. briefly discussed learning the gauge in their appendix. I would encourage the authors to clarify more precisely which aspects of the proposed approach go beyond this earlier suggestion. In particular, the contribution appears to lie primarily in the efficient parameterization, integration with simulation-free optimization, and systematic empirical validation, rather than in introducing gauge-based posterior inference itself.
-
The expressiveness of the learned gauge remains restricted to affine marginal-preserving dynamics. The theoretical completeness results are established for linear SDEs with additive noise and non-degenerate Gaussian initial distributions. Proposition 2 characterizes the affine vector fields that preserve Gaussian marginals, but the learned gauge does not represent general nonlinear marginal-preserving vector fields. For nonlinear prior processes, the optimal posterior correction may contain nonlinear components that cannot be captured by the proposed skew-symmetric linear parameterization. Consequently, optimizing the gauge can improve the posterior within the chosen family without eliminating the underlying expressiveness limitation. Although this restriction is acknowledged in the conclusion, its practical significance is not extensively evaluated. In particular, it remains unclear how the approximation quality degrades when the nonlinear marginal-preserving component becomes substantial. A more explicit discussion of the relationship between the affine gauge family and the unrestricted optimal posterior process would strengthen the paper.
-
The computational complexity claims require careful qualification. The paper emphasizes that Gauge Matching has quadratic computational complexity in the latent dimension, compared with the cubic operations required by Helmholtz-SDE. This is an important advantage. However, the stated complexity relies on efficient evaluation of the diffusion covariance operations and its divergence. For general dense, state-dependent diffusion coefficients, these operations may be substantially more expensive. Moreover, the gauge parameterization introduces (d(d-1)/2) additional outputs from the inference network. Thus, even if the associated linear algebra scales quadratically, the overall training cost also depends on the architecture used to predict these parameters and their time derivatives. The reported timing experiments show favorable performance up to moderate latent dimensions, but the practical scaling advantages would benefit from a more detailed separation of network evaluation, automatic differentiation, gauge parameterization, and matrix-operation costs.
Questions for authors
- Questions for the Authors
-
Expressiveness beyond affine gauges: Can the authors characterize or empirically investigate the approximation error caused by restricting the marginal-preserving correction to affine vector fields? Are there nonlinear SDE examples where the optimal gauge within the proposed family remains substantially suboptimal relative to a more expressive posterior process?
-
Novelty relative to Helmholtz-SDE: Since Smith et al. (2026) already discuss learning the gauge generator as a possible extension, could the authors clarify the precise conceptual and technical differences between that suggestion and the proposed Gauge Matching formulation?
-
State-dependent diffusion: The ability to handle state-dependent noise is an important contribution. Could the authors provide a more detailed analysis of the computational and statistical trade-offs for different diffusion structures, including dense state-dependent covariance matrices?
4 comments
Part 2 of 2
-
(Question 3.) We agree and will qualify the statement. The is the linear algebra of (15) at one beyond the network evaluations (caption of Table 1): with the whitened state is , so the drift needs , , one triangular solve and . The gauge head is one linear layer with outputs (2016 outputs and 0.2M parameters at ); the full Cholesky factor adds outputs in the same way for every full-covariance posterior, including Helmholtz-SDE; only and need time derivatives, from one forward-mode pass regardless of , and needs none. Helmholtz-SDE instead pays forward-mode passes for the prior Jacobian and an eigendecomposition and Sylvester solve per evaluation, with gradients through both. Four posterior processes separate the components, each of the first three adding one to the previous and Helmholtz-SDE replacing the gauge head by the Jacobian and solve (ms per training iteration, batch of 256, forward and backward pass and optimizer step; GPU RTX 3090 as in Figure 5C, CPU with 16 threads; Appendix D.1):
SDE Matching + full (Cholesky gauge) + gauge head (Gauge Matching) Cholesky gauge + Jacobian and solve (Helmholtz-SDE) 8 (GPU) 10.2 12.0 12.7 16.0 32 (GPU) 12.1 14.1 15.1 19.9 128 (GPU) 10.2 14.2 15.3 33.1 64 (CPU) 24.9 35.2 42.1 181.1 On the CPU, where no launch overhead hides the arithmetic, the full adds 10 ms, the gauge head 7 ms, and the Jacobian with the solve 146 ms over the Cholesky gauge at ; total training time scales the same way (general response, point 4). For a dense state-dependent , the product stays , but the exact divergence costs Jacobian–vector products, or one with a Hutchinson-type estimate, which keeps a valid, looser bound on the negative log-likelihood that tightens with more probes; and the whitening of the residual by , which every method pays, becomes a solve unless is triangular. The divergence belongs to the ODE-to-SDE conversion (Equation 7), is present for every gauge and vanishes for state-independent , so it is not a cost of the gauge. On the statistical side, the optimal gauge is the unique minimizer of a strictly convex quadratic for any (Proposition 3), so the learned gauge has a well-defined target for every structure; for the diagonal of latent SDE practice ( a function of [2, 3]) the estimator is that of the state-independent case with an exact divergence, which is the setting of Tables 5–6; we have not run dense state-dependent diffusion coefficients. We will state the quadratic cost for the diagonal structure and the cost of the dense case explicitly in Section 3.
We hope that these clarifications, in particular the account of what goes beyond the extension sketched in [1], the bound relating the affine family to the unrestricted posterior process, and the component-wise cost separation, will strengthen your support for the acceptance of our paper!
References:
[1] Smith et al. "Closing the approximation gap in simulation-free latent SDEs."
[2] Li et al. "Scalable gradients for stochastic differential equations."
[3] Bartosh et al. "SDE Matching: Scalable and simulation-free training of latent stochastic differential equations."
Thanks for the authors' response, which have resolved most of my concerns. Thus, I have updated my score.
We are glad that our response resolved the reviewer's concerns, and we thank the reviewer for updating the score. Thank you very much for your time and effort!
Part 1 of 2
We thank the reviewer for the thorough review and for finding the problem formulation clear and well motivated and the mathematical formulation elegant. We address each weakness together with the corresponding question; table, figure and equation numbers refer to the submitted version.
Weaknesses:
(Question 2.) The general response, points 1–3, states what goes beyond the suggestion in Appendix A.5 of [1]. In short: [1] compute the gauge from the prior drift at every evaluation; we make it a learned coordinate of the variational family within the SDE Matching framework, through the factorization S=LO, which retains the closed form that [1] saw as sacrificed, and we prove that this loses nothing (Theorem 1, Propositions 2 and 3, Corollary 4), with the order-one correction identified as the optimal gauge of the linearized prior. No prior Jacobian, no cubic solve and state-dependent diffusion follow; the last is the limitation that [1] name "learning the divergence-free correction" would lift. In the camera-ready version, if accepted, we will rewrite the paragraphs "Relation to Helmholtz-SDE" (Section 4) and "Simulation-Free Methods and the Gauge" (Section 6) along these lines.
(Question 1.) We agree, and the relation is as follows. For state-independent Σ, Theorem 1 and Proposition 2 say that the affine gauge family is exactly the set of Gaussian processes with the given marginals. The order-one correction of Helmholtz-SDE lies in the same family; beyond it lie the corrections of order ℓ≥2, for which [1] observed no notable improvement, and the state-dependent generator of Appendix B.1, which contains all of them. The distance of the affine optimum from the unrestricted optimum can be bounded: because the q-weighted Helmholtz decomposition of an affine field is affine, the linearization of h about the posterior mean projects identically onto both families, and at the same marginals the two minimal path-space KL terms differ by at most half the posterior mean square of the Taylor remainder h(Z)−h(m)−∇h(m)(Z−m) in the Σ−1-weighted norm, per unit time, which to leading order scales with the square of the posterior variance and the squared second derivatives of the drift, relative to the process noise. The affine restriction is therefore immaterial for posteriors that are narrow on the scale over which the prior Jacobian varies (the Lotka–Volterra system in log coordinates, where the order-one correction is already optimal, Table 5) and can matter for wide posteriors under strongly nonlinear drifts. The Stuart–Landau experiment (Table 4) is that regime: the order-one correction misses the optimal affine gauge by 8κ2tr(P)det(P)/s2≈0.13 nats per unit time (P the posterior covariance, κ the shear, s the noise scale), the learned gauge by a tenth of that, and Gauge Matching is better by 1.6 ± 0.3 nats. Prompted by the question, we have also computed the remaining distance on this system: the residual is cubic, so the unrestricted optimum lies in the six-dimensional space of degree-two generators of Appendix B.1 for d=2, and a cross-fitted least-squares fit on the Monte Carlo samples of the gauge-excess estimator gives it at the fitted marginals (5 seeds). The learned gauge is 0.38 ± 0.01 nats per unit time above the unrestricted optimum (3.8 nats per trajectory, 15% of the path-space KL term), of which 0.37 is the affine restriction and 0.01 the optimization; the order-one correction is 0.43 above it and the fixed gauges 0.73–0.76, so the learned affine gauge recovers about half of what the fixed gauges miss in this stress case. Doubling and quadrupling the posterior standard deviation raises the restriction to 12 and 610 nats per unit time, the steep growth the Taylor argument predicts for a cubic drift; examples where the affine optimum is substantially suboptimal are therefore wide posteriors under strongly nonlinear drifts, and the state-dependent generator of Appendix B.1, trained the same way, is the construction for them. We will add the bound with its proof to Appendix C.2, these numbers to Appendix D.2, and the discussion to Section 4.
(Our response continues in the comment below.)