acceptodds
Under review as a conference paper at ICLR 2027

Robust Human-AI Collaborative Bayesian Optimization under Spatially Varying Expert Reliability

Abstract

Human knowledge can accelerate Bayesian optimization (BO), but such knowledge is typically qualitative, noisy, biased, and, more importantly, spatially varying in reliability. Although this spatial heterogeneity better reflects practical expertise, it is largely overlooked by existing human-augmented BO methods, which control human influence globally or according to a prespecified schedule. To address this limitation, we propose a novel trust-aware human-AI collaborative BO framework that adaptively modulates the contribution of human knowledge across the search space. Specifically, input-dependent trust is estimated from local human-objective ranking agreement using localized conformal calibration and then incorporated into the multi-fidelity kernel. Furthermore, we establish two theoretical guarantees: a calibration guarantee that controls the long-run frequency of assigning high trust to human guidance inconsistent with objective observations, and a sublinear cumulative regret bound for the resulting trust-aware BO procedure. Experiments with spatially varying emulated experts, as well as two real-world experiments, consistently demonstrate that the proposed method outperforms standard and human-augmented BO baselines under spatially varying expert reliability while remaining robust to misleading guidance.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian optimization, human-AI collaboration, conformal calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WZ9AOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a human-AI collaborative Bayesian optimization (BO) framework that incorporates expert knowledge through a spatially varying trust mechanism. Specifically, the method learns a latent utility function from human feedback and employs conformal calibration to estimate input-dependent trust weights. These weights are then incorporated into a trust-aware kernel, allowing the BO algorithm to exploit expert knowledge while accounting for its potentially heterogeneous reliability across the search space.

Critical strengths
  1. Clear presentation. The paper is generally well written, and the overall methodological description is relatively clear and logically organized, making the proposed framework easy to follow.

  2. Research significance and potential novelty. Incorporating expert knowledge into BO is an important research direction, and the proposed approach appears to offer an interesting perspective. Similar to Incorporating Expert Prior in Bayesian Optimisation via Space Warping, this work integrates expert knowledge into the kernel structure to influence the BO process. However, this paper adopts a potentially more interesting mechanism by using conformal calibration to estimate spatially varying trust weights before incorporating expert information into the kernel.

Critical weaknesses
  1. Insufficient experimental evaluation. Although the methodological presentation is relatively clear, the experimental section is considerably weaker. The evaluation covers only a limited number of tasks and compares the proposed method against a relatively small set of baselines. Furthermore, several important experimental details are insufficiently explained, making it difficult to fully assess the effectiveness and robustness of the proposed approach.

  2. Poor figure readability and presentation errors. The font sizes of the axis labels and tick labels in the experimental figures are too small, making the results difficult to read. The authors should improve the figure layouts and increase the font sizes. There are also apparent textual inconsistencies that require careful proofreading. For example, on page 8, the phrase "all three functions" appears inconsistent with the corresponding figure, which presents results on four benchmark functions.

Questions for authors
  1. How is the latent utility function obtained in practice, and how are the synthetic reliability parameters determined?

    The method models the latent expert utility using a posterior distribution of the form π(x)∣DtP≈N(μπ,t(x),σπ,t2(x))\pi(x)\mid D_t^P \approx \mathcal{N}(\mu_{\pi,t}(x),\sigma_{\pi,t}^2(x)). However, in the synthetic experiments, the authors construct a simulated human latent utility function as fhuman(x)=a(x)fˉ(x)+ϵf_{\mathrm{human}}(x)=a(x)\bar f(x)+\epsilon, where ϵ∼N(0,0.012)\epsilon\sim\mathcal{N}(0,0.012) and a(x)a(x) is a spatially varying coefficient designed to control expert reliability.

    My main concern is how the proposed method obtains a meaningful latent utility function from an available human-feedback dataset DLFD^{LF} in realistic applications, where the true objective function and the underlying expert utility are generally unknown. The paper should clarify the assumptions required for learning this utility function and explain how its quality can be assessed without access to the ground-truth expert utility.

    In addition, Appendix B.2 specifies aweak=0.08a_{\mathrm{weak}}=0.08, agood=0.35a_{\mathrm{good}}=0.35, and abad=−0.10a_{\mathrm{bad}}=-0.10, together with manually designed spatial reliability regions. How were these values and regions selected? Are the experimental conclusions robust to alternative choices? More broadly, the combination of the synthetic utility construction, conformal rank prediction, and online calibration introduces considerable methodological complexity. The authors should better justify these design choices and demonstrate that the resulting framework is practical beyond the specifically constructed synthetic settings.

  2. Why is the additive kernel construction appropriate, and what justifies its effectiveness?

    The proposed kernel is kt(x,x′)=k0(x,x′)+ωt(x)ωt(x′)hπ,t(x)hπ,t(x′)k_t(x,x')=k_0(x,x')+\omega_t(x)\omega_t(x')h_{\pi,t}(x)h_{\pi,t}(x'). This construction incorporates expert information through an additive kernel component modulated by the estimated trust weights. However, the paper should provide a clearer justification for why this particular additive construction is appropriate and explain how it enables reliable expert information to improve BO while limiting the influence of unreliable feedback. Additional theoretical discussion or ablation studies comparing alternative kernel constructions would strengthen this design choice.

  3. How are the expert priors for the baselines constructed, and how should the unexpected comparative results be interpreted?

    The experimental section does not sufficiently explain how expert prior information is constructed for π\piBO and CoExBO, or how the corresponding hyperparameters are selected. These details are essential for ensuring a fair and reproducible comparison.

    Moreover, in Figures 2 and 3, vanilla GP-UCB appears to outperform π\piBO and CoExBO in some settings. This raises questions about whether the expert information provided to these baselines is appropriately specified and whether the baseline implementations and hyperparameter settings are comparable. The authors should explain these results and provide sufficient details to rule out potential implementation or experimental-design issues.

    Similarly, in the appendix experiments involving adversarial expert information, the proposed method appears to outperform vanilla GP-UCB. While such an outcome is not necessarily impossible, it is unexpected and requires further investigation. The authors should explain how the proposed trust mechanism can yield this improvement when the expert feedback is adversarial, and provide additional evidence—such as ablation studies, trust-weight visualizations, or repeated experiments—to support the interpretation.

    Overall, the paper would benefit substantially from a more thorough and transparent experimental evaluation. At present, the methodological development is considerably more extensive than the empirical validation, and the limited experimental coverage and insufficiently explained results make it difficult to determine how reliably the proposed method performs across different settings.

Reviewer_d8dGOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors tackle the case of black-box optimization, a problem ubiquitous in science where one aims to maximize an objective function using only function calls and no other information about the function: no analytical form, derivatives, convexity information, etc. Due to the complexity of the problem, a significant line of work attempts to integrate as much as possible auxiliary information to accelerate convergence, a typical example being the addition of human expert feedback. However, by nature, this feedback is noisy, biased, and has nonuniform reliability over the input space: an expert is only knowledgeable in his area of expertise...

To tackle this issue, the authors propose a human-AI collaborative Bayesian Optimisation framework where an input-dependent level of expert reliability is estimated based on human feedback, like pairwise feedback. This level is estimated using conformal calibration and then plugged back into a multifidelity kernel, where it can therefore inform the next design selection. After describing their method, the authors establish a robustness guarantee with respect to human guidance inconsistent with the objective, and then demonstrate the strength of their approach on synthetic and real-world experiments.

Critical strengths
  • The method proposed by the authors is well-motivated, sound, simple (which is a good thing), and seems to perform well.
Critical weaknesses
  • I found that too many details were either obfuscated or defined later. This gives a strong impression of an unpolished paper: the idea of the method is there, but everything around it (careful description, experiments) is not. A non-exhaustive list:
  1. In Section 2.1 background, the posterior predictive equations of GPs are given, but the notion of kernel is not introduced, i.e., the function kk is not introduced in this section.
  2. Equation (3) mentions that π\pi is obtained as an MVN distribution using approximate inference. Which method in particular? LA, VI, EP, MCMC??? This step is actually fundamental when learning preferences from pairwise feedback, as the results greatly differ from one scheme to another [1], Here, this is not mentioned anywhere in the manuscript. Moreover, as the authors compare against π\piBO, are we to assume that the same π\pi prior is used for π\piBO? Then again, it would be nice to know precisely how π\pi is computed.
  3. λ\lambda appears in Equation 9 but is only defined in the next subsection, Theorem 4.1.
  • The experiments section is very short (one page), limited to a descriptive account of the results that anyone could read from the figures, there is no insight gained from it. Why is the method working? What are its edge cases? Are there some method hyperparameters that require careful tuning, or whose value can be set using relevant heuristics (λ\lambda? η1\eta_1?) This information would help anyone wanting to use the method itself.

  • The related work could mention one related approach: [2] also estimates a nonuniform level of expert trust, but the setting differs. It is not multi fidelity BO, but Preferential BO where this first phase provides an estimate of expert uncertainty. [3] is another, where expert feedback plays the role of lower fidelity (so again, the setting differs a bit, as one can query the target, then the expert and so on...), and the algorithm alllows leveraging this feedback even when it is unreliable.

[1] Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes, ICML, 2023

[2] Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization, Proceedings of The 1st Symposium on Probabilistic Machine Learning, 2026

[3] Multi-Fidelity Bayesian Optimization with Unreliable Information Sources, AISTATS, 2023

Questions for authors
  • For the 2D synthetic experiments, I would suggest providing in the appendix a plot of both the objective and the expert trust landscape, which allows clearly seeing whether high-valued regions of the objective align with regions where the expert is confident. Currently, the appendix only provides a parametric description of the expert reliability landscapes; a visual intuition could be more helpful.
  • While the appendix provides an account of how BO regret varies as a function of expert reliability, it would actually be helpful to understand how important is the estimation of this expert reliability. In the real-world experiments, the expert provides 100 pairs to estimate the reliability, how sensitive are the results to this number? What about 20, 10 pairs? This of course depends on the expert reliability landscape itself... Also note that for synthetic experiments, the number of pairs used to estimate expert reliability is not indicated. It is one thing to design a method that enables leveraging expert information, it's another thing to show that it actually works in realistic settings. Similarly, the notion of "cost" is never discussed anywhere. I suppose that the setting considered by the authors is such that one can acquire initial expert feedback for a fraction of the price of actual black-box objective evaluations. Still, this should be mentioned.
  • See weaknesses for more questions.

I will say again that I really liked the method, but that at this stage, many clarifications are required, mostly regarding experiments.

Reviewer_xsGoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

They propose a BO framework that allows for human inputs in a reliable way. In particular, they use ideas from conformal calibration, that guarantees human inputs does not harm the process with high probability for the very least. They show some theoretical and empirical findings backing up their framework.

Critical strengths
  • The problem of collaborative BO is very interesting.
  • I particularly like the idea of using conformal calibration methods, to provide meaningful and practical guarantees about the quality of human inputs on the final performance of the BO.
Critical weaknesses
  • It is good to write the section 2.1 for a little bit of a broader spectrum of readers (beyond BO experts). Like what is x? what is the high level of BO, what are the inputs, what are the outputs, what is the goal?

  • The related work section is heavily centered around BO works, whereas human-AI works, and the idea of enforcing no-harm, from either party, human or AI, is studied outside of BO literature. It is good to cover those works too. In particular, there are two works [1] and its followup [2], that are particularly very relevant. They essentially solve the other direction of your problem, which is how to make sure the AI feedback/contribution does not harm the human outputs. And they do use similar flavored ideas from conformal inference. This is just an example, but the authors should discuss these kind of works beyond BO literature.

  • The authors discuss (theoretically and empirically) the potential of a harm by human feedback, and provide algorithmic ways to enforce no-harm. What can we say about the usefulness. What if the algorithm enforces no harm, at the cost of significantly reducing the potential usefulness on some instances. Can we make any guarantee on that too? How about empirical evidence?

[1]: Human-AI Collaborative Uncertainty Quantification, ICML 2026

[2]: Multi-Round Human-AI Collaboration with User-Specified Requirements, ICML 2026

Questions for authors

none.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.