- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper provides the improved theoretical results for Stein Variational Gradient Descent (SVGD) in both the mean-field and the finite-particle settings. The paper established the SOTA convergence rates for Mean-Field SVGD under less restrictive assumptions that cover at least Gaussian mixtures as target distributions and sharp convergence rates for Finite-Particle SVGD under broadly applicable conditions that cover at least Gaussian target distributions. In addition, the paper proposes a new adaptive stepsize strategy that achieves a better rate. The authors also considers the rate of Finite-Particle SVGD under an even weaker set of assumptions.
Critical strengths
-
The paper is well-written and clearly-organized. The paper focused on mean-field SVGD and finite-particle SVGD with in-depth analysis, including detailed discussion on the assumptions and the rate comparisons with previous works. The presentation is easy to follow.
-
The analysis that weaken the relatively restrictive assumptions is basically solid and more applicable to common target distributions. The adaptive stepsize designation that can further improve the dependence is interesting and insightful and could be applied to other mean-field gradient flow analysis.
Critical weaknesses
-
I would suggest moving some of the core novel technical ideas and the comparison with other methods in Subsection C.4 and D.3 to the main body of the paper to help the readers not only get the improved results but also the methodological innovation that could be applied to other related areas.
-
For finite-particle SVGD, when comparing the rates with previous methods in line 403, the paper claims that making a direct comparison in dimension with other methods is not suitable. However, the dependence in dimension is also an important part in SVGD convergence analysis. It would be beneficial to add at least a more informative discussion on this issue.
Questions for authors
-
For finite-particle SVGD, what is the reason that “extending the finite-particle analysis to the weaker upper-smoothness setting appears technically challenging” in line 381? It would be more informative to add a brief discussion on this point in the main body of the paper.
-
Does the adaptive stepsize scheme applicable to other mean-field gradient flow analysis? Could this stepsize scheme be further improved to achieve even better convergence rates?