- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes sequential Bayesian quadrature (BQ) with a Hilbert space GP (HSGP) approximation of the Matérn kernel. HSGP replaces the kernel with m = bᵈ tensor-product Dirichlet-Laplacian sine features. The estimator and an integral-variance-reduction acquisition are computed in this feature space, and a separation constraint keeps designs quasi-uniform.
Critical strengths
- The scope is highly relevant
- The writing is clear and concise
- The proofs are detailed and correct, and the authors acknowledge the dependence on an unknown s.
- The result is a practical method that should work well.
Critical weaknesses
-
Novelty is limited. (a) The estimator is the standard weight-space BQ with Laplacian features, and HSGP-based quadrature already appears in the literature. (b) The acquisition is the standard one-step integral-variance reduction.
-
Sequential design: The entire design can be precomputed offline, so the method is effectively a deterministic design.
-
Complexity: The tensor-product set m = bᵈ grows exponentially in d. I missed the explanation of how this would be handled.
-
Experiments: Error bars on results are missing.
Questions for authors
Does the framework allow a true sequential design?
How replicable are the results (stats on performance metrics)?
HSGP beats exact full-kernel BQ with the same kernel by 10× to 2500×, which is implausible for an approximation of a kernel. What could be the reason for this?
4 comments
I agree with the authors' response and will raise my score accordingly.
Thank you for taking the time to consider our response and for raising your score. We sincerely appreciate your time and thoughtful evaluation of our work.
Thank you again for continued consideration of our revised manuscript and for further raising your score. We are very encouraged by your support.
Novelty, Adaptive Design, Basis Selection, and Reproducibility
Thank you for recognizing the relevance of our research problem. Your questions have prompted us to articulate our novelty more precisely and to clarify the distinction between response-independent and response-adaptive designs.
Comment. Novelty is limited. (a) Estimator is the standard ..., (b) ... one-step integral-variance reduction.
Response. Regarding (a), we agree that there is substantial work on weight-space BQ and HSGP. However, our sequential HSGP-BQ framework makes several distinct contributions. We added a related-work paragraph on Page 2 of the Introduction, beginning with ``Compared with the random Fourier feature ..."
(i) We use deterministic bases that are orthonormal on the bounded domain (−L,L)d, rather than random features sampled from a spectral density on Rd. (ii) Our method approximates both the kernel matrix and integral, which differs from kernel ridge regression (KRR). (iii) We select Laplacian features in ascending order of eigenvalues, rather than tensor-product truncations. (iv) Our theoretical analysis does not require the integrand f to belong to the kernel RKHS or to be a realization of Gaussian process. (v) We answer the question that how many features are sufficient and necessary.
We would also be grateful if you could point us to any existing work specifically addressing HSGP-based sequential Bayesian quadrature. We would be happy to discuss and cite such work in the revised manuscript.
Regarding (b), our sequential sampling rule additionally incorporates a γ-separation constraint, as specified in Equation (10). This geometric constraint is essential for quasi-uniformity guarantees.
Comment. The design can be precomputed ..., a deterministic design.
Response. Our HSGP-BQ estimator and acquisition function are deterministic, so the resulting design is also deterministic for a fixed n. However, the design points are not prescribed in advance. Determining the sampling locations still requires optimization of acquisition sequentially. Based on the suggestions from reviewers, we implemented a sharper response-adaptive variant where the kernel parameters are updated adaptively to the sequentially observed response. Please see the updated Section 4 and our response to your question regarding "true sequential design" below.
Comment. The tensor-product set m=bd grows exponentially in d.
Response. Our new Proposition 2 selects the first m multi-indices in ascending order of Laplacian eigenvalues, ensuring that the number of features is exactly m, without requiring m to be a perfect d-th power. This differs from tensor-product construction.
Comment. Error bars are missing.
Response. In the revised manuscript, we added error bars to the experimental plots. We conduct 100 independent replicates for each sampling method. In Figures 1--3, the shaded bands are constructed as the estimated RMSE plus or minus 1.96 times its delta-method standard error. These confidence bands are more evident in the noisy simulation than in the noiseless simulation or the QM9 experiment.
Comment. Does the framework allow a true sequential design?
Response. We added HSGP-MLE, a response-adaptive variant of our method that re-estimates the kernel lengthscale and nugget by maximizing the marginal likelihood after every 10 additional observations. The resulting lengthscale and nugget can change the next acquisition maximizer. This is a genuinely sequential design where the next sampling location depends on the sequentially observed response.
Comment. How replicable are the results (stats on performance metrics)?
Response. Each sampling method is evaluated over 100 replicates with different random seeds. We report both integration RMSE and the cumulative computational time cost of sequential sampling, see our revised Section 4.
Comment. HSGP beats exact full-kernel BQ ..., the reason for this?
Response. Full-kernel BQ is not an exact method in simulation. Although it retains the exact kernel matrix, it still requires approximation of the kernel mean ∫k(x,x′)dx.
In our initial submission, we used 4096 quasi-Monte Carlo (QMC) points to approximate kernel mean. This resolution was insufficient and contributed to the relatively poor performance. In revised manuscript, we use 214 QMC points, substantially improving its integration accuracy. Nevertheless, HSGP still achieves up to a 5x reduction in RMSE compared with Full-kernel BQ in some settings. Further increasing the number of QMC points may improve Full-kernel BQ's accuracy, but would also increase its time cost. Even without MLE updates, Full-kernel BQ is already 7--20 times more expensive than HSGP in our revised experiments.
Thank you for your thoughtful questions. We hope that our clarifications better distinguish our contributions from existing weight-space BQ approaches and that the new experimental results address your concerns.