acceptodds
Withdrawn from ICLR 2027

CONTRAST-LEVEL POSTERIOR REGRET CERTIFICATES FOR QUADRATIC COMPOSITES WITH UNCERTAIN GP HYPERPARAMETERS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning Gaussian-process hyperparameters changes uncertainty in nonlinear com- posite objectives. We study posterior regret certificates for finite-design squared- norm losses over continuous hyperparameter families. A common posterior cou- pling yields event-transfer bounds from signed-quadratic contrast coefficients without assuming sub-Gaussian composite tails. Under a correctly specified model, predictable observations and valid retained-family coverage, the bounds control joint false certification at a fixed terminal time. Structured two-design posteriors separate intrinsic linear ambiguity from the logarithmic penalty of a specified optimized concentration formula. An explicit event-specific remainder then gives a numerically validated finite separation: at the same posterior and parameter box, our certificate is below a fixed risk threshold while an optimally tilted, sign- conditioned information bound remains inconclusive. Informative observations reduce disagreement across a fixed family, with a sharp boundary rate, eventual almost-sure narrowing and a failure case for an unobserved relevant direction. Direct robust probability remains tighter in the finite example, and the information bound wins in a high-information case. The results characterize uncertainty and certificate conservatism; they do not establish faster Bayesian optimization

Primary area:
optimization
Keywords:
Bayesian optimization, Hyperparameter uncertainty, Composite objectives

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_yHbpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a relatively theory-oriented paper that starts from a practical problem. Considering that model uncertainty may be overlooked when estimating GP hyperparameters, this paper uses contrast coefficients under the RBF kernel to derive posterior regret bounds over continuous hyperparameter regions. Instead of computing the regret probability for every hyperparameter setting individually, it uses loss contrasts between different designs to control how the probability changes with hyperparameters. Subsequently, the paper analyzes the distinction between predictive uncertainty arising from the hyperparameters themselves and the conservatism of the theoretical bounds, and proves that, under specific conditions, its event-specific bound can provide a tighter certification result than existing information-theoretic bounds.

Critical strengths
  1. Clearly defined and systematic theoretical analysis

Rather than broadly studying uncertainty in GPs, the authors focus on how changes in hyperparameters affect regret events, demonstrating a reasonable and concrete perspective compared with existing studies. Moreover, the authors not only derive a probability bound but also analyze its conservatism, failure conditions, and the scope of its statistical guarantees, demonstrating a careful and systematic theoretical investigation.

Critical weaknesses
  1. The advantages of the core theoretical method over existing studies remain unclear

Theorem 1 establishes a posterior regret probability bound over a continuous hyperparameter set. However, existing probability sensitivity methods can also obtain uniform probability bounds over continuous hyperparameter regions by controlling the derivatives of event probabilities with respect to hyperparameters. Although this paper provides a specific theoretical bound for quadratic composites, it does not sufficiently clarify what substantial advantages it offers over existing methods (e.g., in terms of bound tightness, computational cost, or scope of applicability).

  1. Lack of convincing experimental validation

The paper mainly focuses on theoretical analysis, with experiments limited to a carefully constructed numerical example and simulations of random observation histories, without validation on real Bayesian Optimization tasks. In particular, the authors' most convincing certification advantage comes from a specifically constructed example, while the random simulations do not show that their method outperforms existing information-theoretic methods. Therefore, the current experiments are insufficient to demonstrate that the proposed method offers consistent advantages in more general practical problems, nor do they adequately demonstrate its practical value.

Questions for authors
  1. Please further explain the advantages of the proposed theoretical method over existing probability sensitivity bounds.

  2. Please include additional experiments on real-world Bayesian Optimization tasks.

  3. Please include direct experimental comparisons with existing probability sensitivity bounds.

Reviewer_8E8cOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper claims to derive posterior regret certificates for Bayesian optimization of squared-norm objectives that hold uniformly over a range of GP hyperparameters. It studies how the regret probability changes across a hyperparameter cell to obtain a fixed-budget false-certification guarantee.

Critical strengths
  • The paper's core idea of bounding regret-event transfer across hyperparameters through the signed quadratic contrast between losses under a common posterior coupling, rather than bounding each loss separately, is a reasonable approach.
Critical weaknesses
  • The current version falls well below the bar for an academic publication. The structure, message, and contributions are unclear, and the technical writing is of unacceptable quality.

Just to list a few examples of poor writing:

  • h(i)h(i) (Section 3) : never defined

  • "feature-plus-noise Hilbert space": what does this mean?

  • kk (Sec. 6): v±=a2(1±k)v_\pm = a^2(1\pm k): where is kk defined?

  • HpH_p (Theorem 3) “bounds the directional Hessian": bounds in what sense

  • BconcB_{\text{conc}} (Sec. 5) “the preceding one-reference signed-quadratic transfer formula ... with its optimal failure budget,”: where is the signed-quadratic transfer formula?

  • fc(ε)f_c(\varepsilon), “contrast density” (Sec. 5): How is this object defined?

  • ua,uℓu_a, u_\ell (Sec. 7): Where were these defined?

  • “the previous conservative Hessian certificate” (Sec. 7): no formula or reference.

Questions for authors

The manuscript needs substantial revision for clarity before it can be properly evaluated: notation should be defined before use and used consistently, key terms should be introduced once in plain language, and the main results should be stated in the main text with their constants rather than deferred to the appendices.

Reviewer_ngNbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper appears to address how to certify recommending a point when the hyperparameters of a RBF GP are unknown. A typical approach is one that is based one posterior, instead of just choosing the highest value observed so far. This is complicated as the hyperparameters of the posterior need to be estimated/averaged. The authors appear to derive a posterior probability bound on the simple regret under unknown hypers.

Critical strengths
  • The primary contribution here seems to be Theorem 1, which bounds the posterior regret from choosing the wrong hypers.
  • Theorem 2 extends the certification to adaptively chosen points, which is a meaningful extension for BO.
Critical weaknesses

This paper is unintelligible. Even after reading the introduction, I was barely able to identify what the problem the authors are trying to solve. Starting from Section 3 a rain of variable definitions are thrown at you without any explanation. The poor presentation made it very difficult to track anything, interpret the guarantees, or even understand how the results support the paper’s claims.

  • The problem setup is severely lacking
  • Section 2 lists several peripheral works and attempts to distinguish the contributions, but even the unlabelled table provides little help to show what the actual novelty or contribution is here.
  • “Transfer” here does not appear to be defined. What is being transferred? Why is it useful? How should we interpret it?
  • Several terms and variables are defined without meaning
  • The problem needs a much clearer introduction.
  • There seems to be only one experiment in Table 1, seemingly highlighting that the worst-case regret decreases with more observations? The purpose of this and its relation to the results needs to be clearer.

This work is severely underprepared and falls significantly below the bar for an ICLR paper. I thus recommend clear rejection.

Questions for authors

Can you run a real-world experiment (e.g. an actual BO loop) to demonstrate how this work would materially affect the decision?

Reviewer_i1iyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on Gaussian process settings with high uncertainty in hyperparameters and investigates posterior regret certification for squared-norm objectives. Specifically, using a common pathwise coupling of the candidate posteriors, it bounds changes in regret-event probabilities across hyperparameter cells through the coefficients of signed quadratic loss contrasts, without assuming sub-Gaussian tails for the composite loss. These bounds are integrated with hyperparameter-family retention and posterior simulation to get a fixed-terminal joint false-certification guarantee. In addition, the paper distinguishes intrinsic hyperparameter ambiguity from certificate conservatism, establishes observation-dependent narrowing rates, and shows a finite example where an event-specific certificate succeeds while an optimized information bound remains inconclusive.

Critical strengths

S1. The core ideas and concepts are well motivated, and the formulation is well grounded.

S2. The explicit treatment of nonlinear composite uncertainty through squared-norm losses seems interesting, without assuming sub-Gaussian distributions for the composite loss, bypassing inappropriate simplification of the composite objective.

Critical weaknesses

W1. In my mind, the main contribution beyond existing methods in the literature remains quite unclear. The paper itself acknowledges that Gaussian conditioning, quadratic concentration and probability sensitivity are well-established tools. The claimed novelty is the contrast-coefficient transfer bound over continuous hyperparameter cells and its integration into a terminal certification contract. However, while the comparison with Wilson (2024) and Capone et al. (2022) clarifies the differences in statistical guarantees, it does not sufficiently establish when the proposed guarantee provides a meaningful advantage. Also, the general results (Theorems 1–2) apply to finitely many designs, but the results on sharpness, observation-dependent narrowing and finite separation (Theorems 3–4, Proposition 5) are derived only for a two-design, single-output RBF model. So it remains to me unclear what the broader impact and practical relevance are relative to existing certification methods.

W2. I believe the practical relevance and merits of the proposed method are not clearly demonstrated. The proposed bound outperforms the information-bound comparator in a carefully constructed example with zero-valued sample means and a narrow hyperparameter box. Nonetheless, in that same example, direct robust probability is both simpler and tighter, as the authors acknowledge. In the random-history experiments, the proposed bound never certifies a history that the comparator fails to certify. It passes in 0/100 histories at n=1 versus 14/100 for the comparator, and they tie at 74/100 at n=32. The paper therefore shows no realistic setting where the proposed certificate changes a decision relative to existing alternatives.

W3. The full process in the paper is not evaluated end-to-end: While the numerical tests utilize two designs and a fixed hyperparameter box known to contain the truth, the paper does not assess the full procedure end-to-end for retaining plausible hyperparameters, covering that set, and computing certificates across multiple designs; thus the practical computational cost and utility in larger problems remain quite unclear.

Questions for authors

Please refer to the weaknesses above. In addition:

  • Could the paper describe a setting where direct robust probability is unavailable or too expensive, so that the proposed certificate is actually needed?
  • How sensitive are the transfer bounds to the posterior coupling and coordinate origin?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.