- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks when the variance of the log density ratio under a Gaussian approximation certifies, rather than merely approximates, the error . Its contributions are:
- Main result. If the target is everywhere at least times as curved as (), then . The constant is optimal, and the result extends the known concave case to nonconcave log ratios.
- Degree-resolved refinement. A matrix bound that splits into Hermite components never exceeds the matrix Fisher (log-Sobolev) certificate and is exact for Gaussian targets. It shows that weak curvature is paid only through the degree-one and degree-two components, which are the stationarity residuals of Gaussian VI. Hence at an exact VI fit of a log-concave target, with no curvature margin.
- Validation. The certificates are made computable via polynomial surrogates with rigorous residual bounds. On Bayesian logistic regression (Spambase, Covertype) they are 0.29-0.67x the matrix Fisher certificate. They certify more fits at tight tolerances under strong priors, and the degree-resolved bound keeps improving with in a synthetic fixed-prior study.
Critical strengths
-
Clean and nice presented theory. The main bound is optimal, and its proof is transparent. The degree-resolved refinement says where weak curvature matters, and the link to VI stationarity residuals explains why converged VI fits certify far better than Laplace fits.
-
Clear contribution with a normalizer-free implementation. The paper goes beyond an inequality to a deterministic procedure that needs neither sampling nor the normalizer, and it reports its numerical-assurance scope honestly.
Critical weaknesses
-
Limited experiments and comparisons. The only systematic baseline is matrix Fisher, which the degree-resolved bound dominates by construction. Other computable error bounds (e.g. Kasprzak et al., 2025; Huggins et al., 2020) are not compared. Useful tolerances are reached only under very strong priors (, where the prior supplies at least 50-80% of the curvature).
-
Underdeveloped motivation. It is not argued convincingly when such certificates are needed in practice or what decisions they would change. The link to log-variance losses for neural samplers is not substantiated, since the theory covers only Gaussian (or strongly log-concave, two-sided) .
Questions for authors
-
The paper notes that stationarity in a restricted Gaussian family does not give the cancellation . In practice VI is often mean-field, where only the diagonal of vanishes, or trained with noisy reparameterization gradients. What does the certificate give in these cases?
-
How do your certificates compare with the computable Laplace bounds of Kasprzak et al. (2025) on the same logistic models, either in TV directly or via Pinsker?
-
The applications are limited to the evidence bracket and event probabilities. Can the certificates be turned into bounds on posterior means, marginal variances or predictive probabilities, for example through the transport conversion in Appendix? How would such bounds compare with those of Huggins et al. (2020) on the same fits?
2 comments
Thanks for the clarification. I will keep my score
Thank you for the constructive review. The revision adds the comparisons you asked for, on identical fits for both methods (Section 6.4; Appendix G.3 to G.5 and G.7).
Q1. Mean-field and noisy-gradient VI (Appendix G.3, Table 8). Theorem 3.2 holds for every Gaussian. Mean-field stationarity removes the mean residual a and the diagonal of the covariance residual B but keeps its off-diagonal entries, so
KL(Q∥P)≤21tr(BW2(K)B)+23∑k≥3Vk,
with a weight W2 that grows only like log(1/κ). On Spambase (n=1024, d=58) the mean-field certificates are 0.00103 nats at gp=1 and 0.0586 at gp=4, against estimated KL of 8.36×10−4 and 0.0419. On Banknote, where features are strongly correlated, the certificate attributes 4.2 of its 4.29 nats to the off-diagonal term and so points to a full-covariance family; the estimated KL is 1.19 nats for the mean-field fit and 0.0176 for full covariance. For noisy-gradient VI we compute a and B from the target, independently of the optimizer: after 2×104 Adam steps on Spambase (gp=1, 4; medians over three seeds) the certificates are 0.185 and 0.257 nats against estimates of 0.161 and 0.179. On Banknote, where κ=0.01, the mean residual dominates (8.5–8.9 nats against 0.79–0.80) and calls for more optimization. Over the 37 unconverged, mean-field and noisy Gaussians of our validation set, the certificate is a median of 1.40 times the estimated KL (Table 23).
Q2. Kasprzak et al. (2025) (Appendix G.4). On the 24 Spambase and Covertype targets of Section 6.4, our total-variation bound for the Laplace Gaussian, via Pinsker, is below one on 15 (0.024–0.046 at gp=1); no evaluated variant of their Theorem 17 is below one on any (Table 10). Their theorem needs a likelihood maximizer, which separated designs lack, but not global strong log-concavity; ours needs a log-concave likelihood and the prior's curvature. On 12 synthetic fixed-prior Laplace fits where their theorem can be evaluated with certified constants (Table 11), their canonical bound is vacuous on all 12 and ours is informative on 9; with Corollary B.6 (Theorem 3.2 with the likelihood's curvature near the fit), ours is informative on all 12 (0.0199–0.0205 at d=8, n=32,768, against 0.200–0.210 for the best variant of theirs).
Q3. Means, variances, predictions; Huggins et al. (2020) (Appendix G.4 and G.5). Applied coordinate-wise, the transport inequality of Appendix F.2 gives, for coordinate j with fitted standard deviation sj and any certificate C,
∣EPθj−μj∣≤δj,∣sdP(θj)−sj∣≤δj,δj=2C(Λ−1)jj,
and for a predictive probability kl(pQ(x)∥pP(x))≤C, which we intersect with a transport interval. On the six full-covariance VI fits at gp=1 (Spambase, Covertype), all 327 coordinates have δj≤0.1sj and 299 of 300 predictive intervals have half-width at most 0.02. On the 24 of 48 fits where their Gaussian-reference bound is finite, the median ratio of their interval width to ours is 0.97 for means, 0.96 for predictions, 0.80 for standard deviations and 0.72 for variances. Ours use no sampling, theirs plug-in Monte Carlo estimates; under weaker priors both are much wider (Table 13).
Baselines and prior strength. The scalar certificate is not ordered with matrix Fisher: in Table 2, at KL≤0.001 it accepts 11 of 48 historical and 6 of 36 prospective fits, against 1 and 0 for Fisher. Useful KL certificates do not require $g_p\in\lbrace 1,4\rbrace $: with a fixed N(0,I) prior the full-covariance VI certificate is 0.0425–0.090 nats on Banknote, 1.97–2.43 times the KL computed by quadrature (Section 6.4, Table 21), 0.0184–0.405 nats on seven of ten standard small data sets (Table 17) and 4.03×10−6 nats on HIGGS at full size (10,999,950 observations, d=29; Table 19). In a sweep over prior scales (Table 22), the largest gp certified at each tolerance equals the largest at which the estimated KL meets it in seven of the eight cells.
Motivation and neural samplers. Certificates are needed when a fitted Gaussian replaces the posterior and its accuracy must be guaranteed without sampling or ZP: to accept a fit at a stated tolerance (Section 1 and Appendix G.5), and to choose between Gaussians (on Banknote 0.0425 nats for full-covariance VI, 0.995 for Laplace, 4.29 for mean-field; Table 8). With the population loss LQ=21VarQ(g), Theorem 3.1 reads KL(Q∥P)≤LQ/κ, so for Gaussian Q the link to log-variance losses is a theorem, and Theorem C.1 extends it to strongly log-concave Q with two-sided curvature.