- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an alternative to Variational Inference in Deep Gaussian Processes (DGPs). They frame DGP inference as posterior transport, and learn a deterministic sampler that maps to inducing variables. As a theoretical contribution, they identify the small-noise limit with the MAP path of the corresponding path posterior. The paper performs very well empirically on UCI datasets.
Critical strengths
- The method is quite an interesting alternative to Variational Inference and novel to the best of my knowledge.
- The paper performs very well empirically.
Critical weaknesses
I think there is an issue with Proposition 4 and its proof in Appendix D.7. It claims that the Euler discretization error can be bounded using only the spatial and temporal Lipschitz constants of the velocity field and the variance of the initial distribution. However, these quantities alone should not be sufficient to control the error.
The issue is that a Lipschitz constant controls how quickly the velocity field changes between different points, but not how large the velocity itself can be. For example, consider the simple one-dimensional ODE
The spatial Lipschitz constant is always 1, and the temporal Lipschitz constant is 0, regardless of the value of (b). Proposition 4 would therefore give the same error bound for any (b). However, the actual squared Euler error is
This can become arbitrarily large as (b) increases, even though all the quantities appearing in the proposed bound remain unchanged.
The problem seems to occur in the final step of Appendix D.7, where the authors attempt to control the expected squared velocity using the variance of the centered initial state. Centering removes the mean of the initial distribution, but it does not remove a constant component of the velocity field. In the example above, the initial distribution is already centered, yet the discretization error still depends on (b).
Questions for authors
Is it possible to clarify or correct the result of Proposition 4?