acceptodds
Under review as a conference paper at ICLR 2027

VEST: Variational Drift Estimation from Noisy Observations

Abstract

Nonparametric drift estimation for stochastic differential equations from multiple independent trajectories with additive observation noise remains challenging in statistics and machine learning. However, direct regression on noisy increments amplifies measurement error, whereas regression on smoothed trajectories treated as exact discards uncertainty and dependence between consecutive latent states. Thus, we propose Variational Drift Estimation from Noisy Observations (VEST), a method that alternates between Gaussian Markov smoothing and neural drift learning. An iterated extended Kalman smoother proposes updates to the Gaussian approximation, while the drift is updated by regressing on conditional targets derived from its adjacent state moments. An acceptance rule based on a shared variational objective determines whether each proposed update is accepted. Moreover, we give theoretical guarantees on the discretization error of the variational objective and the convergence of the algorithm. Experiments show that VEST outperforms existing related methods.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Nonparametric drift estimation, Noisy observations, Variational inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_haqCOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the problem of nonparametric drift estimation for stochastic differential equations (SDEs) from multiple independent trajectories corrupted by additive observation noise. VEST alternates between Gaussian Markov smoothing (E-step) and neural drift learning (M-step), using conditional posterior-increment targets and deterministic cubature integration to incorporate posterior uncertainty into drift regression. Both updates are safeguarded by an Armijo-type acceptance criterion on a shared variational objective. The core motivation is to simultaneously address the issues of "observation noise amplification" and "discarding latent-state uncertainty".

Critical strengths
  • Good ablation experiment design. The "Without safeguards" ablation clearly demonstrates the critical role of the safeguard mechanism for numerical stability.
  • The paper provides a well-structured and thorough theoretical analysis in Section 4 and Appendix E. Table 1 shows that VEST achieves great drift estimation accuracy, often by a substantial margin.
  • The paper correctly identifies a critical flaw in naive increment regression (Section 2): the Δ−1\Delta^{-1}-scaled conditional noise term that biases the estimate when regressing noisy increments. By formulating a marginal-likelihood objective and optimizing an ELBO, VEST addresses this issue from principles.
Critical weaknesses
  • Theorem 2 assumes exact gradients of the variational objective for its convergence guarantees. However, the actual M-step uses gradients of the cubature-approximated loss (Eq. 10), which is only an approximation of the true objective. Whether the theoretical guarantees remain valid for the practically?
  • VEST's computational cost per outer iteration is likely higher than SING and SDE Matching, since its outer iterations each require running IEKS on every trajectory, multiple neural inner steps, and Armijo backtracking for both updates. However, the paper does not report runtime, parameter counts, or wall-clock time for convergence.
Questions for authors

Please refer to 'Critical Weaknesses'.

1 comment
AuthorsOpenReview ↗

Convergence guarantees for the cubature objective and comparisons of runtime, parameter counts, and time to target accuracy

Thank you for your careful reading and constructive feedback! We respond to your concerns below.

W1. FΔ\mathcal F_\Delta denotes the discrete ELBO defined using exact Gaussian expectations, whereas F~Δ\widetilde{\mathcal F}_\Delta denotes the deterministic objective obtained by replacing the expected transition terms with finite weighted sums over cubature points. The loss in Eq. (10) indeed uses a cubature approximation. Our convergence analysis is formulated for the resulting cubature objective F~Δ\widetilde{\mathcal F}_\Delta.

Assumption (A4) requires exact gradients of the cubature objectives—F~Δ,n\widetilde{\mathcal F}_{\Delta,n} for the Gaussian update and F~Δ(ϕk+1,⋅)\widetilde{\mathcal F}_\Delta(\phi_{k+1},\cdot) for the neural update. With ϕk+1\phi_{k+1} fixed, the M-step loss in Eq. (10) satisfies −∇θL^k(θ)=∇θF~Δ(ϕk+1,θ).-\nabla_\theta\widehat{\mathcal L}_k(\theta) = \nabla_\theta\widetilde{\mathcal F}_\Delta(\phi_{k+1},\theta). Here, the exact-gradient requirement in Theorem 2 applies to the cubature objective corresponding to the loss in Eq. (10). We apologize that this distinction was not sufficiently clear in the original presentation.

W2. We first compare VEST and SING on the same NVIDIA L40 across seven systems in Table 1, using five seeds per system. Taking the one-dimensional OU system as an example, a complete run takes 8.34 s for VEST and 42.68 s for SING (Table F.9). To measure the cost of each step, we run a separate timing experiment on the same OU system. Both methods run for 16 outer iterations, with a budget of 10 E step iterations and 10 M-step updates per round. Initial JAX compilation takes 5.77 s for VEST and 31.67 s for SING. Over rounds 2–16, the E step takes 0.0080 s per round for VEST and 0.0342 s for SING. This difference in E-step runtime can be partly explained by the update mechanisms: VEST linearizes the dynamics and computes posterior moments directly by Kalman smoothing, whereas SING's update requires gradient evaluations for both the likelihood and transition terms. The M step takes 0.0013 s per round for SING and 0.0030 s for VEST, including approximately 0.0023 s for Armijo backtracking (Table F.11).

Moreover, we compare VEST and SDE Matching on the same GPU across all ten Table 1 settings, using five seeds per setting. Under Table 1 configurations, mean runtime ranges from 8.33 to 25.01 s for VEST and from 40.72 to 42.15 s for SDE Matching with 4,000 training updates (Table F.9). On the 1D OU system, VEST and SING each use a drift network with 24 hidden units and 73 trainable parameters. SDE Matching uses 100 hidden units and 301 trainable parameters in its drift network, with a further 51,402 parameters in its posterior network (Table F.11).

Finally, we compare the time needed to reach a drift RMSE of 0.1 or lower on the 1D OU system. VEST and SDE Matching reach this target on all five seeds, taking 8.09 s and 9.81 s on average, respectively. SING reaches it on four of five seeds within 400 rounds, averaging 48.56 s over those four.

Reviewer_ETSCOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a variational EM algorithm for drift estimation of non-linear SDEs with known diffusion.

Critical strengths
  • Very relevant research question. Learning stochastic dynamics from data is still a hard problem, especially when one wants to find the true drift that governs the system.
Critical weaknesses
  • The work assumes known diffusion. This is a critical weakness, since recovering the dynamics from noisy observations is particularly difficult where the diffusion is unknown.
Questions for authors
  • Your work assumes known diffusion. Please indicate very clearly in the manuscript which of the compared methods from previous work also assume known diffusion. Is the comparison fair, i.e., if another method learns the diffusion as well, have you adapted this method such that it also uses the known diffusion?
  • I would very much like to see more real-world data comparisons, particularly comparing your method to the three most recent related works (Ganguly 2025, Bartosh 2025, Hu 2025), if that is feasible on such short notice.

minor: in your equations there are hyperlinks to the appendix. As far as I understand this is not intentional, if it is, I do not understand what it means. E.g., eq. (11), the '1' in 1/N is a hyperlink to lines 1440 in the appendix, in my pdf.

3 comments
AuthorsOpenReview ↗

Fair comparisons with matched, fixed diffusion and additional baseline results on real data

Thank you for your thoughtful questions and suggestions! We address each point below.

Q1. To ensure a fair comparison, VEST and all baselines that use diffusion are supplied with the same diffusion coefficient, which is held fixed during fitting. We have clarified these settings in Subsection 5.1 and Appendix F.5.

The table below summarizes how each baseline handles diffusion and how we configure it in our experiments.

MethodDiffusion in the original methodDiffusion in our experiments
Naive increment regressionNo known diffusion required.No diffusion input.
Penalized LSNo known diffusion required.Penalty scale estimated from observed increments.
EM-SMC-RKHSAssume known diffusion.Same true diffusion as VEST.
SINGAssume known constant diffusion.Same true diffusion as VEST.
SDE MatchingJointly learned with drift.Same true diffusion as VEST.

For the real-data experiments, diffusion is calibrated from training data as follows.

One-dimensional optical-trap experiment. We estimate the diffusion constant and observation-noise variance from the variance and lag-one covariance of position increments, using only training trajectories. Methods requiring these inputs use the same estimates, held fixed during fitting.

Two-dimensional harmonic-trap experiment. We calibrate a common isotropic diffusion from paired detector recordings in the training data, averaging the estimated transition-noise covariances across the three experimental conditions. This diffusion is held fixed during VEST training.

Q2. We compare VEST with EM-SMC-RKHS (Ganguly et al., 2025), SDE Matching (Bartosh et al., 2025), and SING (Hu et al., 2025) on the Eb10 optical trap data. All four methods use the same training trajectories, with the measurement noise standard deviation and the diffusion coefficient fixed. The estimated barriers are 8.71,kBT8.71\\,k_BT for VEST, 8.36,kBT8.36\\,k_BT for EM-SMC-RKHS, and 7.89,kBT7.89\\,k_BT for SING. SDE Matching could not recover two wells over the evaluation interval and VEST gives the closest estimate to the nominal barrier of 10,kBT10\\,k_BT. We have added these comparisons to Table 2 in Section 5.4.

Minor comment. We apologize for the confusion. We have recompiled the manuscript and checked Eq. (11). In our copy of the updated PDF, the numerator appears without a hyperlink. We hope this resolves the issue on your end as well.

Thank you for the clarification. I retain my score

Many thanks for your time and feedback!

Reviewer_3kLMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies nonparametric drift estimation for stochastic differential equations from multiple independent trajectories observed with additive Gaussian noise, assuming that the (possibly state-dependent) diffusion and the noise covariance are known. The authors argue that regressing on noisy increments amplifies measurement error, whereas regressing on smoothed trajectories treated as exact discards uncertainty and the dependence between consecutive latent states. The proposed method, VEST, alternates between Gaussian Markov smoothing and neural drift learning: an iterated extended Kalman smoother proposes updates to the Gaussian approximation of the latent path, the drift network is regressed on conditional targets built from adjacent-state moments (including the lag-one covariance), and an acceptance rule based on a shared variational objective decides whether each update is accepted. Theoretical results relate the discrete objective to an Euler path model and establish conditional stationarity of the safeguarded alternating scheme. Experiments on seven synthetic systems (up to ten dimensions, with constant and state-dependent diffusion) and a real optical-trap dataset show lower drift error than naive regression, penalized least squares, EM-SMC-RKHS, SING, and SDE Matching in most settings.

Critical strengths
  1. The paper is well-written and easy to follow, and the motivation for why both naive increment regression and smoothed-mean regression fail is very clear.
  2. Using both the marginal and the lag-one covariance in the conditional regression target is principled and technically correct.
  3. Treating the smoother as a proposal mechanism and safeguarding both the Gaussian and neural update against a shared objective is a sensible design, and the convergence analysis accounts for it.
  4. The experiments are broad: constant and state-dependent diffusion, up to ten dimensions, noise and sampling-interval sweeps, trajectory-count sweeps, paired ablations, and a real dataset. VEST has the lowest mean error in nine of the ten main settings.
  5. The baselines are relevant and are applied within their supported scope
Critical weaknesses
  1. The method largely combines established components: Kalman smoothing with lag-one covariances, variational/EM learning of dynamics, line-search-safeguarded smoothers, and neural drift regression. The paper should state more precisely what is new relative to these works and SING [1].
  2. The abstract promises guarantees on the discretization error of the variational objective, but Theorem 1 only relates the discrete ELBO to an Euler path model and bounds the Wasserstein distance between prior laws; it does not bound the gap between the implemented (cubature) objective and the true SDE objective.
  3. The theory assumes bounded, globally Lipschitz diffusion, yet all state-dependent examples have quadratically growing diffusion. Theorem 2 assumes exact gradients and uncapped backtracking, unlike the implementation (eigenvalue floors, a 25-trial cap, early stopping).
  4. The computational cost is not reported: there is no runtime or memory comparison with SING [1], SDE Matching.
  5. Known diffusion and noise covariance is a strong assumption, and in the real-data experiment the calibrated noise falls to a numerical floor (~0.025 nm vs. ~2.5 nm increments), making it nearly noise-free.
  6. The ablations change both E- and M-steps or remove both safeguards at once, and the 32-feature linear variant beats neural VEST on the double-well and Van der Pol systems, so the benefit of a neural drift is unclear.
  7. The results use a single network initialization, the 5D evaluation uses training-trajectory states, and small gaps (e.g., 0.198 vs. 0.202) lack paired statistics.

References

  • [1] A. Hu, H. D. Smith, and S. W. Linderman. SING: SDE Inference via Natural Gradients. NeurIPS, 2025.
Questions for authors
  1. How sensitive is VEST to a misspecified diffusion coefficient or noise covariance?
  2. Can authors include runtime and memory comparisons with the baselines, and discuss how the cost scales with the state dimension and the number of trajectories?
  3. Can authors separate the E-step and M-step safeguards, and compare against a simple damped or small-step update without safeguards?
  4. Can authors report results over different initializations, and paired seed-wise differences?
  5. How sensitive are the real-data barrier estimates to the noise floor and to the estimated diffusion coefficient?
  6. When should one prefer the neural drift over the linear-in-features variant?

Any discussion around these will help improve the paper over its current state.

3 comments
AuthorsOpenReview ↗

Thank you for your helpful and valuable comments! Responses follow the questions, with related weaknesses cross-referenced and remaining weaknesses addressed separately. All synthetic experiments use five data seeds per setting, and runtime and GPU memory are measured on the same NVIDIA L40. Abbreviations: App. = Appendix; Tab. = Table.

Q1/W5. VEST is relatively stable under moderate covariance misspecification in most settings. Changing observation noise covariance RR by ±20% changes mean drift RMSE by less than 4% in every setting; the same changes to diffusion covariance a(x)a(x) change it by at most 20% in 8 settings. Larger perturbations (0.5 or 2 times either covariance) tend to increase mean drift RMSE (App. F.3).

Q2/W4. VEST is faster on average than SING and SDE Matching: mean runtimes are 8.33–25.01 s, 41.53–54.49 s (where applicable), and 40.72–42.15 s, respectively (App. F.9). On 1D OU, complete VEST/SING runs take 8.34/42.68 s. In a separate timing test (16 rounds, E/M budgets 10/10), initial JAX compilation takes 5.77/31.67 s. Over rounds 2–16, E steps take 0.0080/0.0342 s per round: VEST linearizes dynamics and computes posterior moments directly by Kalman smoothing, whereas SING requires likelihood and transition gradients. M steps take 0.0030/0.0013 s per round; VEST's safeguard takes about 0.0023 s. Sampled peak GPU memory, averaged over seeds, is 562–802 MiB for VEST, 534–670 for SING where applicable, 562–632 for SDE Matching, and 490–514 for EM-SMC-RKHS (Tab. F.10). On 10D Kuramoto, VEST smooths all 15 trajectories in parallel; computing covariance gradients 1 trajectory at a time uses 581 MiB/25.01 s. On OU with 40 trajectories, dimensions 1/5/10 take 19.65/45.52/59.07 s and 562/565/662 MiB; in dimensions 5 and 10, covariance gradients are computed 1 trajectory at a time. In 1D, 20/40/80 trajectories take 9.85/19.65/13.82 s and 534/562/582 MiB.

Q3/W6. Removing only the E-step safeguard changes mean drift RMSE by at most 2.1% in every setting. Removing only the M-step safeguard causes 44/50 runs to terminate with numerical errors; the remaining 6 have RMSE at least 102010^{20}. Replacing both safeguards with E-step damping 0.1 and M-step learning rate 0.01 gives finite estimates in all 50 runs, but increases mean RMSE by 0.122–0.362 in 8 settings (App. F.8).

Q4/W7. We test 5 network initializations per data seed. In 8 of 10 settings, the relative standard deviation of drift RMSE across initializations, averaged over 5 data seeds, is below 10% (Tab. F.3). Tab. F.1–F.2 report paired differences for every baseline comparison in Tab. 1. VEST has lower drift RMSE in 139/155 constant-diffusion and 48/55 state-dependent comparisons. On the 5D bistable ring, mean drift RMSEs on 400 states from independent trajectories are 1.167/1.173 for VEST/SDE Matching at the reference setting (Tab. F.4).

Q5/W5. We adapt Vestergaard et al.'s covariance method [1] to short-time increments of Eb10 training trajectories, with a noise standard deviation floor of 0.0249 nm. At calibrated DD, noise standard deviations of 0.0249/0.0788/0.249/2.54 nm give barriers of 8.71/8.68/8.58/8.07 kBTk_BT. For a bead of diameter 1.5,μm1.5\\,\mu\mathrm{m} in room-temperature water, the Stokes–Einstein reference range is approximately 0.29–0.33 μm2/s\mu\mathrm{m}^2/\mathrm{s}, including our calibrated value of 0.311. At noise standard deviation 0.0249 nm, D=0.30/0.311/0.32D=0.30/0.311/0.32 μm2/s\mu\mathrm{m}^2/\mathrm{s} gives barriers of 9.20/8.71/8.48 kBTk_BT.

Q6/W6. The choice depends on how well the bases suit the system. When domain knowledge or modeling experience suggests suitable bases, VEST-LR offers a closed-form M-step (App. C.2) and good estimation accuracy. When suitable bases are hard to specify, neural VEST offers a flexible alternative. Neural VEST has lower mean drift RMSE in 6 of 10 tested settings, while VEST-LR performs better in the other 4 (Tab. F.7).

W1. Our contribution couples Gaussian smoothing and drift learning with joint stationarity analysis. We formulate drift learning as diffusion-weighted regression on conditional posterior-increment targets, evaluated by deterministic cubature. IEKS generates Gaussian proposals, while direction checks and Armijo tests safeguard both Gaussian and neural updates against the same objective. Under stated assumptions, this coupling yields joint stationarity guarantees for Gaussian and drift parameters. Relative to SING, the distinction lies in the regression formulation, safeguards for both Gaussian and drift updates, and the joint stationarity analysis. The regression also accommodates known state-dependent diffusion. We clarified these contributions in the Introduction.

W2. We revise the abstract to clarify that Theorem 1 equates the discrete ELBO with that of a continuous Euler model and bounds the Wasserstein distance between the Euler and SDE path distributions.

W3. These are sufficient conditions for the theoretical guarantees. The 7 constant-diffusion settings satisfy the diffusion conditions; the 3 state-dependent settings test performance beyond them. In practice, covariance floors aid numerical stability; the 25-trial cap and early stopping limit computation. Direction, feasibility, and objective acceptance checks remain.

[1] Vestergaard et al., Phys. Rev. E 89, 022726 (2014).

Dear Authors,

Thank you for the rebuttal. I will keep my score.

Thank you for your time and feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.