acceptodds
Under review as a conference paper at ICLR 2027

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

Abstract

Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs). Existing methods either fit an explicit density by an ELBO (DSVI, IPVI, DDVI, DBVI) or sample by MCMC (SGHMC). We instead frame DGP inference as posterior transport: learn a deterministic sampler that maps a tractable reference measure to posterior-relevant inducing variables, regularised by a path prior derived from the Doob-bridged reference diffusion. Our realisation, OM-Path (formally FBVI-bridge-Path), applies Song's probability-flow ODE to DBVI's Doob-bridged forward SDE; the reference drift is closed-form from the bridge marginal coefficients (no score matching) and the path regulariser is the Onsager–Machlup action. At the finite used in training, the objective is a data-fit term plus the classical Onsager–Machlup small-tube path prior; Theorem 1 identifies its small-noise limit with the conditional MAP path of the corresponding tempered Doob-bridge path posterior. Two strict path-space ELBO variants on the same backbone are derived as ablations. Under a matched-seed paired Wilcoxon test against DBVI on seven UCI regression benchmarks, OM-Path wins significantly on the two largest datasets (power , NLL against the DSVI baseline of ; protein , RMSE vs. ), ties on yacht / qsar, and concedes the small- noisy cells to DBVI. It also outperforms the strict-ELBO variants on UCI cells, and our gradient-variance measurements show a substantial Hutchinson penalty on power, though the size of that effect is dataset-dependent.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
deep Gaussian processes, variational inference, posterior transport, probability flow ODE, Onsager-Machlup action, large deviations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fkCROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an alternative to Variational Inference in Deep Gaussian Processes (DGPs). They frame DGP inference as posterior transport, and learn a deterministic sampler that maps to inducing variables. As a theoretical contribution, they identify the small-noise limit with the MAP path of the corresponding path posterior. The paper performs very well empirically on UCI datasets.

Critical strengths
  • The method is quite an interesting alternative to Variational Inference and novel to the best of my knowledge.
  • The paper performs very well empirically.
Critical weaknesses

I think there is an issue with Proposition 4 and its proof in Appendix D.7. It claims that the Euler discretization error can be bounded using only the spatial and temporal Lipschitz constants of the velocity field and the variance of the initial distribution. However, these quantities alone should not be sufficient to control the error.

The issue is that a Lipschitz constant controls how quickly the velocity field changes between different points, but not how large the velocity itself can be. For example, consider the simple one-dimensional ODE

U˙t=Ut+b,U0∼N(0,κ).\dot U_t=U_t+b,\qquad U_0\sim\mathcal N(0,\kappa).

The spatial Lipschitz constant is always 1, and the temporal Lipschitz constant is 0, regardless of the value of (b). Proposition 4 would therefore give the same error bound for any (b). However, the actual squared Euler error is

[e−(1+1/N)N]2(κ+b2).\left[e-(1+1/N)^N\right]^2(\kappa+b^2).

This can become arbitrarily large as (b) increases, even though all the quantities appearing in the proposed bound remain unchanged.

The problem seems to occur in the final step of Appendix D.7, where the authors attempt to control the expected squared velocity using the variance of the centered initial state. Centering removes the mean of the initial distribution, but it does not remove a constant component of the velocity field. In the example above, the initial distribution is already centered, yet the discretization error still depends on (b).

Questions for authors

Is it possible to clarify or correct the result of Proposition 4?

Reviewer_SVpLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles inference over inducing variables in Deep GPs. The authors frame DGP inference as posterior transport and leverage Song's probability-flow ODE in a Doob's h-transform setting. The paper is interesting in how it combines several more or less established ideas for the Deep GP setting. The authors show that the approach is useful in established but rather simple experiments.

The paper has merits, but my initial assessment is that it does not quite meet the bar for acceptance due to a combination of several points: The convoluted presentation, combination of existing methodology and ideas, and lack of proper motivation and motivating examples.

Critical strengths

The paper has merits in how it presents the problem and tackles it with considerable technical depth. The approach itself is interesting for people who have attempted to solve issues related to inference in Deep GP models. The framing of the inference problem as posterior transport and realising the simplifying connection to Doob's h-transform are interesting.

Critical weaknesses

This paper has three apparent key weaknesses:

  1. Presentation. Even if the technical ideas appear sound, the paper is written in a convoluted and overly dense manner. The delivery of the content is formatted almost as a bullet list (without bullets, though) and not as a coherent whole that would read well. The idea itself is not that complicated and thus the presentation does not merit the interesting idea.

  2. Methodological contributions. Even if the idea is interesting, it appears like a direct combination of rather standard tooling or recently proposed tooling in other applications in the community, applied to this specific problem. Thus, motivating acceptance based on the methodological contributions alone does not appear feasible.

  3. Motivation and experiments. The experiments offer a wide variety of setups, but were kind of a disappointment. The paper lacks proper real-world motivating example studies. The included examples are what one would expect for validation and toy-like insights, and even if the results are promising (especially in the larger-data examples), it remains hard to draw definitive conclusions about how the proposed methodology help in practice.

Overall, the paper appears ok, but I find it hard to see that the community would embrace this work with a lot of excitement.

Questions for authors

I would be happy to hear comments on the weaknesses, primarily focusing on the concerns related to methdological contributions and simplistic experiments.

Reviewer_GAdsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OM-Path, a deterministic posterior transport method for approximate inference in Deep Gaussian Processes (DGPs). The method combines Doob-bridged diffusion with a probability-flow ODE and introduces the Onsager–Machlup action as a path regularizer, avoiding score matching and divergence estimation. The paper also establishes a theoretical connection to MAP path estimation in the small-noise limit. Experimental results demonstrate competitive performance across multiple benchmarks, particularly on larger regression datasets, although the method remains less effective on some smaller datasets.

Critical strengths
  1. The paper proposes a computationally efficient approach to posterior inference in DGPs. The method combines Doob bridging with a probability-flow ODE and introduces Onsager–Machlup path regularization to achieve deterministic posterior transport. By utilizing a closed-form reference drift, it avoids score matching and Hutchinson divergence estimation, reducing computational overhead compared with some diffusion-based inference methods. Furthermore, the paper introduces strict-ELBO variants to compare the performance of different optimization objectives.

  2. The experimental evaluation is relatively comprehensive. The paper conducts experiments on UCI regression, large-scale regression, classification, and Bayesian optimization tasks, comparing the proposed method with representative approaches such as DSVI, DDVI, and DBVI. Statistically significant performance improvements over DBVI are reported on the power and protein datasets, supported by corresponding statistical analyses. In addition, the paper includes ablation studies that provide further evidence of the method's effectiveness and discusses its limitations on certain datasets.

Critical weaknesses
  1. There is a discrepancy between the theoretical objective and the practical training objective. Equation (11) computes the path regularization term under the learned trajectory distribution, whereas Equation (12) approximates it using the reference bridge distribution. Although the authors acknowledge this difference, the approximation error and its potential impact have not been sufficiently analyzed.

  2. The evaluation of posterior approximation accuracy remains insufficient. The paper provides predictive performance evaluations, uncertainty analyses, and posterior visualizations, but lacks a direct quantitative comparison with the target posterior. Furthermore, the theoretical analysis primarily concerns the constructed Doob-bridge path posterior, and its relationship to the original DGP inducing-variable posterior requires further clarification.

Questions for authors
  1. Equations (11) and (12) use different sampling distributions to compute the regularization term. Could the authors analyze the impact of this approximation on the optimization results and provide a comparative experiment using the two approaches?

  2. Theorem 1 establishes the small-noise conditional MAP property of the Doob-bridge path posterior. Could the authors further clarify the relationship between this path posterior and the original DGP inducing-variable posterior?

  3. Equation (9) and Appendix D.3 use different initial covariance specifications. Could the authors clarify whether this difference affects the correspondence between Theorem 1 and the practical sampling procedure?

  4. In Appendix P, Table A13 reports a bridge-related improvement of 0.021 on the energy dataset, whereas the accompanying text reports 0.001. Please verify this discrepancy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.