acceptodds
Under review as a conference paper at ICLR 2027

Sequential Bayesian Quadrature with Hilbert space Gaussian processes

Abstract

Bayesian quadrature provides a probabilistic approach to numerical integration, but exact sequential Bayesian quadrature becomes increasingly expensive as function evaluations accumulate. We develop a scalable sequential Bayesian quadrature (BQ) method based on Hilbert space Gaussian process (HSGP) approximations of Mat\'ern kernels. Both the quadrature estimator and an integral-targeted variance-reduction acquisition are represented in a finite Laplacian eigenbasis, while a geometry-stabilization constraint guarantees quasi-uniform sequential designs. For integrands in Sobolev spaces, we establish nonasymptotic mean-squared integration-error bounds that allow the target smoothness to differ from that of the Mat\'ern reproducing kernel Hilbert space (RKHS), covering both noiseless and noisy observations. The error bounds explicitly track the feature budget, kernel lengthscale, regularization, sampling resolution, and observation noise, and determine how many features are sufficient to preserve the corresponding full-kernel quadrature rates. They further characterize response-independent parameter sequences that avoid repeated marginal-likelihood evaluation and optimization. Computation is carried out entirely in the finite feature space, substantially reducing the complexity for each integral evaluation relative to full-kernel BQ. Numerical experiments confirm the predicted trade-offs among accuracy, feature dimension, and runtime against full-kernel and approximate Bayesian quadrature baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian quadrature, kernel integral, Hilbert space Gaussian processes, sequential design, convergence theory

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_c9RpOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes sequential Bayesian quadrature (BQ) with a Hilbert space GP (HSGP) approximation of the Matérn kernel. HSGP replaces the kernel with m = bᵈ tensor-product Dirichlet-Laplacian sine features. The estimator and an integral-variance-reduction acquisition are computed in this feature space, and a separation constraint keeps designs quasi-uniform.

Critical strengths
  • The scope is highly relevant
  • The writing is clear and concise
  • The proofs are detailed and correct, and the authors acknowledge the dependence on an unknown s.
  • The result is a practical method that should work well.
Critical weaknesses
  • Novelty is limited. (a) The estimator is the standard weight-space BQ with Laplacian features, and HSGP-based quadrature already appears in the literature. (b) The acquisition is the standard one-step integral-variance reduction.

  • Sequential design: The entire design can be precomputed offline, so the method is effectively a deterministic design.

  • Complexity: The tensor-product set m = bᵈ grows exponentially in d. I missed the explanation of how this would be handled.

  • Experiments: Error bars on results are missing.

Questions for authors

Does the framework allow a true sequential design?

How replicable are the results (stats on performance metrics)?

HSGP beats exact full-kernel BQ with the same kernel by 10× to 2500×, which is implausible for an approximation of a kernel. What could be the reason for this?

4 comments
AuthorsOpenReview ↗

Novelty, Adaptive Design, Basis Selection, and Reproducibility

Thank you for recognizing the relevance of our research problem. Your questions have prompted us to articulate our novelty more precisely and to clarify the distinction between response-independent and response-adaptive designs.

Comment. Novelty is limited. (a) Estimator is the standard ..., (b) ... one-step integral-variance reduction.

Response. Regarding (a), we agree that there is substantial work on weight-space BQ and HSGP. However, our sequential HSGP-BQ framework makes several distinct contributions. We added a related-work paragraph on Page 2 of the Introduction, beginning with ``Compared with the random Fourier feature ..."

(i) We use deterministic bases that are orthonormal on the bounded domain (−L,L)d(-L,L)^d, rather than random features sampled from a spectral density on Rd\mathbb{R}^d. (ii) Our method approximates both the kernel matrix and integral, which differs from kernel ridge regression (KRR). (iii) We select Laplacian features in ascending order of eigenvalues, rather than tensor-product truncations. (iv) Our theoretical analysis does not require the integrand ff to belong to the kernel RKHS or to be a realization of Gaussian process. (v) We answer the question that how many features are sufficient and necessary.

We would also be grateful if you could point us to any existing work specifically addressing HSGP-based sequential Bayesian quadrature. We would be happy to discuss and cite such work in the revised manuscript.

Regarding (b), our sequential sampling rule additionally incorporates a γ\gamma-separation constraint, as specified in Equation (10). This geometric constraint is essential for quasi-uniformity guarantees.

Comment. The design can be precomputed ..., a deterministic design.

Response. Our HSGP-BQ estimator and acquisition function are deterministic, so the resulting design is also deterministic for a fixed nn. However, the design points are not prescribed in advance. Determining the sampling locations still requires optimization of acquisition sequentially. Based on the suggestions from reviewers, we implemented a sharper response-adaptive variant where the kernel parameters are updated adaptively to the sequentially observed response. Please see the updated Section 4 and our response to your question regarding "true sequential design" below.

Comment. The tensor-product set m=bdm=b^d grows exponentially in dd.

Response. Our new Proposition 2 selects the first mm multi-indices in ascending order of Laplacian eigenvalues, ensuring that the number of features is exactly mm, without requiring mm to be a perfect dd-th power. This differs from tensor-product construction.

Comment. Error bars are missing.

Response. In the revised manuscript, we added error bars to the experimental plots. We conduct 100 independent replicates for each sampling method. In Figures 1--3, the shaded bands are constructed as the estimated RMSE plus or minus 1.96 times its delta-method standard error. These confidence bands are more evident in the noisy simulation than in the noiseless simulation or the QM9 experiment.

Comment. Does the framework allow a true sequential design?

Response. We added HSGP-MLE, a response-adaptive variant of our method that re-estimates the kernel lengthscale and nugget by maximizing the marginal likelihood after every 10 additional observations. The resulting lengthscale and nugget can change the next acquisition maximizer. This is a genuinely sequential design where the next sampling location depends on the sequentially observed response.

Comment. How replicable are the results (stats on performance metrics)?

Response. Each sampling method is evaluated over 100 replicates with different random seeds. We report both integration RMSE and the cumulative computational time cost of sequential sampling, see our revised Section 4.

Comment. HSGP beats exact full-kernel BQ ..., the reason for this?

Response. Full-kernel BQ is not an exact method in simulation. Although it retains the exact kernel matrix, it still requires approximation of the kernel mean ∫k(x,x′)dx\int k(x,x')dx.

In our initial submission, we used 4096 quasi-Monte Carlo (QMC) points to approximate kernel mean. This resolution was insufficient and contributed to the relatively poor performance. In revised manuscript, we use 2142^{14} QMC points, substantially improving its integration accuracy. Nevertheless, HSGP still achieves up to a 5x reduction in RMSE compared with Full-kernel BQ in some settings. Further increasing the number of QMC points may improve Full-kernel BQ's accuracy, but would also increase its time cost. Even without MLE updates, Full-kernel BQ is already 7--20 times more expensive than HSGP in our revised experiments.

Thank you for your thoughtful questions. We hope that our clarifications better distinguish our contributions from existing weight-space BQ approaches and that the new experimental results address your concerns.

I agree with the authors' response and will raise my score accordingly.

Thank you for taking the time to consider our response and for raising your score. We sincerely appreciate your time and thoughtful evaluation of our work.

Thank you again for continued consideration of our revised manuscript and for further raising your score. We are very encouraged by your support.

Reviewer_RyRbOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This article studies BQ for integral approximation while utilizing kernel approximation. The key contribution of the article is the theoretical results controlling the approximation errors by stating an effective "maximal"-rate for the sample size nn in terms of the approximation parameters. This way, as long as the feature budget (number of approximation features in the quadrature) and kernel parameters evolve accordingly to the sample size, the integration error can be controlled---this is, one can study how kernel approximation affects integral approximation, with explicit rates relating the two.

Critical strengths
  1. Integral approximation is an important problem in ML, and the authors managed to develop a fast and accurate method for tackling this problem.
  2. The very good empirical results likely follow from the derived theoretical results, allowing to control both the integral approximation error and the kernel approximation error dynamically and simultaneously.
  3. While there might be concerns about the writing of the proofs section, no major glaring errors were found.
Critical weaknesses
  1. Writing could be improved, and it can be at times a little rough. (i) The paper does not have any paragraph containing the paper structure. Also, sections do not contain any indicators or introduction to the structure to guide the reader through them. (ii) Proof's sections are unstructured; typically one would expect one singular section/subsection per-proof, but this article spreads the proofs through sections and multiple subsections, sometimes stopping to state and prove additional lemmas within the same section. This unorganized structure of the appendix makes it difficult to parse and digest the proofs. (iii) A slight sentence stating its connection to related work is missing in the introduction. Something simple as "as far as our understanding goes, we are the first to develop [a,b,c]" or "We improve over other BQ methods by also doing [d,e], which has not been done in the past". (iv) Lastly, writing is at times imprecise; for example: (p. 192) "Here we combine it with the integration-specific acquisition equation 6 for BQ.", where? ("In this section we combine...", "in the following we combine...", "in Section X we combine...").

  2. There are some concerns on the significance of the results; perhaps not due to any glaringly wrong steps, but rather due to some steps not having sufficient detail. For example (i; p. 641) "the posterior covariance matrix of θm\theta_m is..." leads to a formula without citing/referencing where it is draw from, (ii; 704) the error formula is introduced without referencing the equation in the main text where it comes from, (iii; 1259-1269) the steps dealing with the inequalities are a little obscure, requiring some back and forth between the main part and back to understand where some of the inequalities come from (it can be solved with some referencing on top of the inequalities). Note that this a just some examples caught on an initial parse of the proofs, as authors tend to not reference/cite where the expressions used in their proofs come from.

Questions for authors

The manuscript is past the acceptance threshold. However, some edges might be left in order to improve the scoring further.

Q1. Could the authors consider addressing weakness 1.(i) and 1.(iii)?

Q2. The article employs 4 assumptions in order to derived their good empirical performance. Typically, one expects a short discussion regarding how common/easy to fulfill this assumptions are, but this is missing. How easy are this assumptions to fulfill in practice?

3 comments
AuthorsOpenReview ↗

Writing, Presentation, Proof Organization, and Assumptions

Thank you very much for your confirmation of our work. Your constructive suggestions and examples are particularly helpful in guiding our revisions and improving the clarity of our writing.

Comment. (i) The paper does not have any paragraph containing the paper structure..., to guide the reader through them.

Response. In the revised manuscript, we added a paragraph at the end of the Introduction outlining the paper's organization, beginning with "The remainder of this paper is organized as follows...". In addition, Sections 2--4 now begin with brief overviews of their subsections to help readers navigate the manuscript and understand its structure.

Comment. (ii) Proof's sections are unstructured ..., makes it difficult to parse and digest the proofs.

Response. We reorganized the proof of Theorem 1 in Appendix B to improve its clarity and logical flow. Section B.1 presents the error decomposition, Section B.2 provides a high-level proof roadmap explaining how the auxiliary lemmas are combined to establish the error bounds, and Section B.3 contains the proofs of these lemmas. The introduction to Appendix B also clarifies the organization and dependencies of these components.

Comment. (iii) A slight sentence stating its connection to related work is missing in the introduction.

Response. We added a related-work paragraph on Page2 of the Introduction, beginning with ``Compared with the random Fourier feature constructions of ...", to clarify the distinctions between our HSGP method and existing approaches.

To the best of our knowledge, our work is the first to establish sharp theoretical results characterizing the sufficient and necessary numbers of Laplacian features required to preserve the full-kernel quadrature convergence rates under both oversmoothing and undersmoothing settings, with and without observation noise.

Comment. (iv) Lastly, writing is at times imprecise.

Response. We replaced ambiguous expressions with explicit references to equations and results. For example, the original statement "Here we combine it with the integration-specific acquisition equation 6 for BQ" (p. 192) has been revised to "The sequential rule [10] combines this constraint [9] with the integration-specific acquisition [8]."

We also added explicit references to relevant definitions and equations in the statements of theorems and propositions. For instance, Theorem 1 now specifies ``Consider a rank-mm approximation defined in [3]--[5]."

We addressed the remaining comments as follows: (i) for p. 641, we reorganized the proof in Appendix A to follow a general-to-specific structure; (ii) for p. 704, we added an explicit reference to Equation (11) in the main text; and (iii) for pp. 1259--1269, we added a reference to the integration risk formula (34) when deriving the parameter-tuning rules, together with additional computational details.

Comment. The article employs 4 assumptions ..., How easy are this assumptions to fulfill in practice?

Response. We added a brief discussion on Page 5, immediately before Section 3.1, to clarify the practicality of our assumptions. These assumptions are relatively standard in the kernel literature, although the main practical challenge is that the smoothness ss of the unknown integrand ff is generally unavailable.

Specifically, (1) common domains, such as boxes and balls, with bounded integration densities satisfy the domain and measure conditions; (2) sufficiently smooth integrands satisfy the Sobolev regularity condition s>d/2s>d/2, allowing smoothness levels both below and above the kernel RKHS order τ\tau; (3) our framework accommodates both noiseless observations and independent, mean-zero observation errors with bounded variance, including non-Gaussian noise; and (4) quasi-uniformity can be achieved using regular grids or guaranteed by the geometry-stabilized rule in Lemma 1.

Thank you again for your detailed and constructive feedback. We sincerely appreciate your time and consideration, and hope that our revisions and responses address your concerns.

I am satisfied with the author's comments---I will update my scores accordingly.

Thank you very much for your positive feedback and for updating the scores. We appreciate your detailed suggestions, which helped us improve the clarity and presentation of the manuscript.

Reviewer_6uMXOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper develops a scalable sequential Bayesian quadrature (BQ) method using Hilbert space Gaussian process (HSGP) approximations of Matérn kernels. Both the quadrature estimator and an integral-targeted acquisition are represented in a finite Laplacian eigen basis, and a geometry-stabilization constraint enforces quasi-uniform sequential designs.

The method avoids repeated marginal-likelihood optimization, and experiments compare it with full-kernel BQ, sparse-spectrum BQ, and randomly pivoted Cholesky kernel quadrature on synthetic 2D problems and a 5D QM9 finite-population mean.

Critical strengths
  1. Impressive theory. Most finite-feature BQ papers bound kernel or posterior approximation error; this paper bounds the integration error directly, with explicit dependence on mm, ℓ\ell, λ\lambda, hnh_n, ss, τ\tau, and σϵ2\sigma_\epsilon^2. The separation of the deterministic quadrature rate from the n−1n^{-1} noise floor is clean.

  2. The sufficient budgets in Corollary 1 ( m≍nm \asymp n for s≤τs \le \tau, m≳nτ/sm \gtrsim n^{\tau/s} for s>τs > \tau, and m≳nd/(2s)m \gtrsim n^{d/(2s)} under fixed noise) are complemented by a lower bound (Proposition 3). The necessity argument is the most interesting part of the paper.

Critical weaknesses

1.Although this paper has impressive mathematical theory, I think this paper is better suited to some statistical journals rather than artificial intelligence conference. Because this paper is about computing an integral with numerical methods. The authors do not present an example of how the Bayesian quadrature can be used in machine learning and AI models.

  1. The baselines are not tuned comparably. Full-kernel BQ uses the same fixed ℓ=0.5\ell = 0.5, σ2=1\sigma^2 = 1, and γ\gamma-greedy constraint; SSBQ uses MLE for ℓn\ell_n and λn\lambda_n. The proposed method uses a hand-chosen schedule (Cm,cλ)(C_m, c_\lambda) and, in the noisy case, a large cλ=103c_\lambda = 10^3. A full-kernel BQ with marginal-likelihood-tuned hyperparameters would be a much stronger baseline; without it, the runtime/accuracy comparison may reflect hyperparameter choices rather than the HSGP representation. The paper should add (i) a tuned full-kernel BQ, (ii) a Nyström BQ baseline (Chatalic et al., 2022), and (iii) a sensitivity analysis over (Cm,cλ)(C_m, c_\lambda).

  2. The method requires the unknown smoothness ss. Algorithm 1 takes ss as an input to set α\alpha (Eq. 11); the experiments use the true ss (or a value just below it). The discussion mentions estimating ss by quadratic variation, but no such estimator is implemented or evaluated. As submitted, the method is an oracle-schedule method; the “scalable” claim should be qualified, and a data-driven ss (or a sensitivity analysis over misspecified ss) is needed.

  3. The geometry guarantee relies on an approximation to hnh_n for d≥2d \ge 2. Lemma 1 assumes the true fill distance hnh_n; the implementation approximates it with a Sobol point set (Section 3.2). The γ\gamma-feasible region (Eq. 7) is defined using this approximation, so the theoretical quasi-uniformity is not guaranteed for the implemented algorithm. The paper should bound the approximation error or compute hnh_n more accurately, and show empirically that hn/qnh_n/q_n stays bounded.

Questions for authors

see weaknesses

3 comments
AuthorsOpenReview ↗

Additional Experiments, Baseline Comparisons, Exact Fill Distance, and Smoothness Sensitivity

We appreciate your positive assessment of our theoretical contributions and your constructive suggestions regarding the numerical experiments.

Comment. This paper is better suited to statistical journals..., but lacks an example of ... machine learning or AI.

Response. We added a related-work paragraph in the Introduction. We believe Bayesian quadrature is an important problem in machine learning, as it addresses computationally challenging tasks where evaluating the integrand is expensive. There is an extensive body of literature on this topic, as discussed in our Introduction. Representative examples include Chatalic et al. (2022), Epperly and Moreno (2023), and Warren and Ramos (2024), with related work published at leading machine learning conferences.

In addition, our Section 4.2 demonstrates the practical application of sequential Bayesian quadrature by estimating the mean molecular polarizability using the real-world QM9 dataset.

Comment. The paper should add (i) a tuned full-kernel, (ii) a Nystrom, and (iii) a sensitivity analysis over (Cm,cλ)(C_m, c_\lambda).

Response. (i) We added two additional baselines, HSGP-MLE and Full-kernel-MLE, both re-estimate the lengthscale and nugget by maximizing the GP marginal likelihood every 10 additional observations. Figures 1--3 present an end-to-end comparison of the methods, accounting for their different parameter settings and sequential sampling procedures. The results show that MLE improves integration accuracy in some experiments.

(ii) We also included the Nystrom BQ baseline (Chatalic et al., 2022) in both the simulation and real-world QM9 experiments. Since Nystrom samples landmarks independently of the observed responses, there is no acquisition for Nystrom. The Nystrom method uses an unregularized Moore--Penrose pseudoinverse to solve the kernel system, which may partly explain its larger integration errors in the noisy setting.

(iii) We added Tables 1 and 2 to report the sensitivity of HSGP to (Cm,cλ,γ)(C_m, c_\lambda, \gamma) in the noiseless and noisy settings, respectively. We conducted a grid search over these parameters for both HSGP and HSGP-MLE and select the configurations achieving the lowest RMSE for the subsequent experiments.

Comment. The unknown smoothness (s)..., the “scalable” claim should be qualified, and a data-driven (s) (or a sensitivity analysis over misspecified) is needed.

Response. We conducted new experiments (Figure 1--3) that do not require ss is known, where a conservative but sufficient choice is to set m=Cmnm=C_m n for HSGP method. In this case, ``scalability" means our method remains computationally advantageous by replacing the intractable kernel-mean integral with analytically HSGP feature means.

In addition, we added an experiment in Appendix I.4 to investigate the sensitivity to misspecification of ss. We hold all other parameters fixed and vary the exponent α\alpha in mn≍nαm_n\asymp n^\alpha. For τ=3.5\tau=3.5 and an integrand with smoothness approaching s=4s=4, Corollary 1 suggests a critical exponent of α=τ/s≈0.875\alpha=\tau/s\approx0.875. We consider four settings: α=1.0\alpha=1.0 (conservative), α=0.875\alpha=0.875 (critical), α=0.8\alpha=0.8 (moderately optimistic), and α=0.6\alpha=0.6 (strongly optimistic). Figure 6 shows that HSGP tolerates conservative (1.0) and moderately optimistic (0.8) smoothness choices in this experiment, while a more optimistic choice (0.6) can allocate too few features to sustain the same error decay. HSGP-MLE remains more robust in this experiment.

Our Appendix I.3 further examines robustness to kernel smoothness mismatch in the oversmoothing regime s<τs<\tau.

Section 3.2 discusses higher-order quadratic variation as a possible approach to estimating ss from observations (Loh et al., 2021; Loh & Sun, 2023). However, the accuracy of this estimation often relies on a large sample size when ss is large. Developing reliable data-driven smoothness estimation remains an open question, as in the Discussion section.

Practical guidance: If prior knowledge suggests that ff belongs to the kernel RKHS, i.e., f∈Hτf\in H^\tau, then setting s=τs=\tau is sufficient, as guaranteed by our Theorem 1. This prior knowledge is also commonly adopted in the kernel quadrature literature (e.g., Chatalic et al., 2022).

Comment. The geometry guarantee relies on an approximation to hnh_n..., and show empirically that hn/qnh_n/q_n stays bounded.

Response. In the revised experiments, we compute hnh_n exactly using a clipped Voronoi diagram (Aurenhammer & Klein, 2000), implemented with scipy.spatial.Voronoi. The computational procedure is described in Section 3.2 and Appendix I.2. We verify the quasi-uniformity of the HSGP sequential designs empirically. As shown by Figure 4 (Appendix I.2), the mesh ratio hn/qnh_n/q_n remains below a constant, consistent with the theoretical bound in Lemma 1.

We hope our responses can address your concerns. Thank you again for your time and thoughtful consideration.

Thank you so much for the response. Honestly speaking, as a Bayesian ML researcher, I feel hard to read this theoretical paper. Therefore, I used AI to review this paper. My general feeling is that the paper does not fit an AI venue like ICLR! This paper is a complete math or stats paper with fancy math theory, stochastic methods. Even I am very familiar with Bayesian computation, this paper is hard to understand. Also you have none ML/AI examples to illustrate the application of your theory/method with neural networks... Even if this paper gets accepted by ICLR 2027, not much readers really care about it... I expect ICLR to accept papers that are about artificial intelligence/neural networks, so I am not impressed by theoretical math papers. LLMs can produce many fancy math theories as well...Anyhow, I keep my rating unchanged, and let the AC to decide whether to accept your paper...

AuthorsOpenReview ↗

Relevance to the Probabilistic Methods Track

Thank you for following up and for sharing your perspective on the relevance of our work to ICLR.

Our work was submitted under the "Probabilistic Methods (Bayesian methods, variational inference, sampling, UQ, etc.)" track, as specified in the ICLR 2027 Call for Papers. Our work develops a computationally efficient sequential Bayesian quadrature method using Hilbert space Gaussian process approximations, with theoretical guarantees for preserving full-kernel quadrature convergence rates. We demonstrate the practical performance of our method using both simulations and a machine learning application to the real-world QM9 dataset. We believe our methodological and theoretical contributions fit this track.

We would also like to point out that theory-driven kernel methods, probabilistic numerical methods, and Gaussian processes have been studied at previous ICLR conferences in the same track. Some examples are:

  • "Symmetry-Aware Bayesian Optimization via Max Kernels" (ICLR 2026), which develops a theoretically grounded kernel construction for Gaussian process-based Bayesian optimization;
  • "Revisiting Nonstationary Kernel Design for Multi-Output Gaussian Processes" (ICLR 2026), which develops a generalized spectral kernel duality and a computationally efficient kernel construction;
  • "Identifiability for Gaussian Processes with Holomorphic Kernels" (ICLR 2025), which establishes a theoretical framework for kernel parameter identifiability.

While these papers address different problems and include different applications, they illustrate that theoretical kernel methods and probabilistic inference are important research topics within the ICLR community.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank the area chairs and reviewers for their time and constructive feedback. We appreciate the reviewers' recognition of our theoretical contributions [RyRb, c9Rp, 6uMX], as well as the relevance and practical value of our method [RyRb, c9Rp].

Our paper develops a scalable sequential Bayesian quadrature (BQ) method based on Hilbert space Gaussian processes (HSGP). Our main contributions are summarized as follows:

  • End-to-end parameter-explicit theory. We establish nonasymptotic integration error bounds that explicitly quantify the effects of feature approximation, kernel parameters, regularization, and observation noise. Our theory allows the smoothness of the integrand to differ from that of the kernel RKHS, covering both oversmoothing and undersmoothing regimes, with and without observation noise.

  • Sufficient and necessary feature budgets. We derive sufficient and necessary conditions on the number of Laplacian features needed to preserve the convergence rates of full-kernel BQ. In particular, our results identify the regimes where sublinear feature budgets are sufficient and provide explicit theoretical guarantees on the trade-off between computational efficiency and statistical accuracy. We show that the derived lower bounds on feature budgets are tight in sequential designs.

  • Sequential BQ without likelihood optimization. We develop an integral-targeted acquisition function that can be evaluated entirely in the finite HSGP feature space, together with a geometry-stabilization constraint that guarantees quasi-uniform sequential designs. Our theory also provides response-independent parameter schedules that avoid repeated marginal-likelihood estimation and optimization.

We have addressed the reviewers' comments point by point in our individual responses. The main revisions are summarized below:

  • We added response-adaptive baselines, which update kernel parameters using marginal likelihood. We also added a Nyström BQ baseline and increased the QMC resolution for the full-kernel baseline. Results are reported in Figures 1–3. [6uMX, c9Rp]

  • We added experiments examining sensitivity to smoothness misspecification in Appendix I.4 and conducted sensitivity analysis over HSGP parameters, reported in Tables 1–2. We also replaced the approximate fill-distance with an exact computation based on clipped Voronoi diagrams. Appendix I.2 and Figure 4 examine the empirical mesh ratio hn/qnh_n/q_n. [6uMX]

  • We added confidence bands over 100 seeds for evaluation metrics RMSE and time cost. [c9Rp]

  • We added Proposition 2 to explain our feature selection rule based on ascending eigenvalues, which differs from tensor-product construction. [c9Rp]

  • We added a detailed related-work paragraph to Introduction to clarify how our method differs from existing approaches to kernel integration and kernel approximation. [RyRb, c9Rp]

  • We added section-level roadmaps, improved equation references, clarified the practicality of our assumptions, and reorganized the proofs in Appendix A-E. [RyRb]

We hope that these additional theoretical clarifications, numerical experiments, and presentation improvements address the major concerns and provide a clearer picture of our contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.