acceptodds
Under review as a conference paper at ICLR 2027

Elicitation-Augmented Bayesian Optimization

Abstract

Human-in-the-loop Bayesian optimization (HITL BO) methods utilize human expertise to improve the cost-efficiency of BO. Most of the HITL BO methods assume that an expert can explicitly quantify their domain knowledge, for instance by pinpointing query locations or specifying their prior beliefs about the location of the maximum as a probability distribution. However, domain knowledge is often tacit and can be expressed only implicitly. We take this into account by eliciting the expert's domain knowledge through pairwise comparisons of designs. We interpret the expert's pairwise judgements as noisy evidence about their mental model, and couple this model with a surrogate model of the objective function via a learned correlation. We develop a principled cost-aware method for combining pairwise comparisons with direct observations. The proposed method can adapt to expert bias by devaluing incorrect information: when pairwise queries are cheap and reliable, it substantially improves sample-efficiency over observation-only BO, and when queries are costly or unreliable, it devalues them.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian Optimization, Human-in-the-loop Bayesian Optimization, Active learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sRjNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper combines expensive experimental evaluations with pairwise comparisons from a subject expert with the goal of optimizing a known utility function over an unknown objective ff, where the structure of UU is known and assumed to be linear. Authors propose a mechanism for efficiently combining the two sources of information, such that there maybe misalignment in the projected preference model and the actual objective ff. Authors provide experimental analysis across a range of tasks and compare with a range of baselines as well to demonstrate empirical merit of approach.

Critical strengths
  1. As opposed to naive preference optimization works, authors make a distinction between the emergent model from pairwise data and the actual function to be optimized. This is potentially very useful in settings where there is a lot of cheap pairwise data, such as self driving applications, where user data maybe available in the form of feedback, and actual tests are extremely costly.
  2. Authors have provided an extensive literature review, that captures several existing domains that ask related questions, and the modeling framework also leverages inspiration in the right way from these fields, such as multi-fidelity BO and preferential optimization, resulting in a very intuitive mathematical framework.
Critical weaknesses
  1. A strong limitation of this work is lack of actual expert in experiments for generating reliable pairwise data. Authors rely on synthetic framework for pairwise generation, while it is understandable that human studies may not be possible, it is fairly straightforward to use an LLM as a proxy source of cheap pairwise data, or leverage existing human datasets. In the absence of human studies, LLM proxy or use of existing datasets/benchmarks to construct pairwise comparisons, it is difficult to verify the practical merit of this work.
  2. It seems like an inefficient operation to use a global ρ\rho in modeling the correlation because in regions where the data is locally aligned, good pairwise demos will remain less trustworthy as a consequence of low global ρ\rho, similarly high overall correlation does not imply locally sound correlation, disturbing the accuracy of model. Please see questions for more clarification needed on this and other topics.
Questions for authors
  1. Can authors provide successful replication using real world pairwise data to demonstrate the robustness of framework? If not, what remains the key limitation in doing so, and what can be said about real world applicability from synthetic data?
  2. Does a global ρ\rho suffice in modeling the misalignment between pairwise and real data? What motivates a global ρ\rho instead of a spatially informed one?
  3. In the experiment results in Figure 1 and 2, UCB and logEI outperform the approach as they remain uncontaminated by the misaligned pairwise data, this raises questions about whether the pairwise data is being used effectively at all or not. Can authors clarify if this is due to the global ρ\rho used in modeling, or is there another reason?
4 comments

Thank you for your review. We have uploaded the revision in which we address both of the critical weaknesses you pointed out: (1) To verify that EA-BO works also on real preferential data, have conducted additional experiments where the comparisons are provided either by an human expert or by an LLM expert (Section 6.1., Figure 6, Appendix E.2), as you suggested. (2) To enable modelling the expert reliability ρ\rho locally, we have extended EA-BO by adding a spatially varying correlation parameter, and verified experimentally that this extended version can learn and utilize a simple spatially varying correlation structure (Appendix F.2).

Weakness 1 & Question 1: Replication using real-world pairwise data

The concern of the Reviewer was that the lack of actual expert in experiments for generating reliable pairwise data. The Reviewer suggested either using an LLM expert or existing human datasets to generate pairwise comparisons.

We tried both of these suggestions by adding two tasks: (1) Optimizing the solubility of drug-like molecules (Boobier et al., 2017). The pairwise comparisons are based on the estimates of the two human experts collected by Boobier et al. (2017). The average reliabilities of these experts were 8383\\% and 6262\\%. (2) Optimizing the catalytic activity for Buchwald-Hartwig reaction (Ahneman et al., 2018). The pairwise comparisons are provided either by an LLM that is briefed by 80 measured reactions from a previous related experiment, or by an unbriefed LLM, following the approach by McCoy et al. (2026) (see Section 6.1 and Appendix E.2. for additional details).

See the table below of the summary of the results. The full results can be found in Figure 6 and Table 5. On the solubility task, there was not a significant difference between EA-BO and the evaluation-only methods. EA-BO outperformed the comparison-only methods for the less reliable expert (ρ=0.62\rho = 0.62).

The results on the Buchwald-Hartwig data set suggest that an LLM can be used instead of the human expert to improve the cost-efficiency of BO: when the LLM was briefed, both EA-BO and the comparison-only methods outperformed the evaluation-only methods. This means that the LLM expert alone outperformed the experiments-only methods. When the LLM was not briefed, it acted basically as an unreliable expert, and thus the comparison-only methods failed. However, as with the synthetic data sets, EA-BO was able to identify the unreliability of the expert, and, consequently, to reach the similar final utility as the evaluation-only methods.

Mean AUC / Final utility on realistic-feedback settings. Solubility percentages denote expert comparison accuracy.

SettingEA-BO (AUC / final)CoExBO (AUC / final)Best eval.-only (AUC / final)Best comp.-only (AUC / final)
Solubility (83%)0.855 / 0.9360.841 / 0.9360.859 / 0.9360.890 / 0.924
Solubility (62%)0.848 / 0.9360.802 / 0.9070.859 / 0.9360.747 / 0.736
B-H, briefed0.754 / 0.9140.668 / 0.9050.715 / 0.8610.761 / 0.861
B-H, unbriefed0.601 / 0.8570.656 / 0.8860.715 / 0.8610.367 / 0.288

Weakness 1 & Question 2: Modelling the spatially variable correlation

The Reviewer suggests that using a global correlation parameter ρ\rho is inefficient, and asked why we chose it instead of a spatially informed one. To address this issue, we extended EA-BO so that it can model a spatially varying correlation; see Appendix F.2 (for output-specific correlations, see Appendix F.1 and our response to the Reviewer aYBp).

Specifically, we model the expert reliability by the spatially varying correlation parameter ρ(x)=Φ(η(x))\rho(x)=\Phi(\eta(x)), where η\eta has a GP prior with a linear kernel and Φ\Phi is a probit function. We tested the extended model on the Hartmann6 data set with a linear ρ(x)\rho(x) with true reliability ρ(x)=0.9\rho(x) = 0.9 at the neighborhood of the global optimum and ρ(x)=0.1\rho(x) = 0.1 at the neighborhood of the local optimum (and the reverse setting).

Including the spatially varying correlation in the model slightly improved the optimization performance (see table below for mean AUCs), suggesting the utility of enabling the correlation to depend on the input when an expert that is reliable in one region of the design space, but unreliable in the other.

Mean AUC for EA-BO wt. G and L denote the global and local optima.

ProblemTrue reliabilityEA-BO (global ρ\rho)EA-BO (local ρ\rho)
Hartmann6(ρG,ρL)=(0.9,0.1)(\rho_G,\rho_L)=(0.9,0.1)0.7770.791
Hartmann6(ρG,ρL)=(0.1,0.9)(\rho_G,\rho_L)=(0.1,0.9)0.6100.652

The reason that we originally modelled the correlation parameter ρ\rho by a global constant is that the objective in BO is optimization, not parameter estimation. Instead of spending a too large proportion of the budget towards learning a complicated correlation structure, the EA-BO with the global ρ\rho just quickly learns whether the expert is in general reliable or not, and allocates the rest of the budget accordingly.

However, the above results demonstrate that—as the Reviewer suggested—estimating the spatially varying correlation structure can improve the optimization performance, at least when the correlation structure is simple enough so that it can be learned within a limited budget. Thus, we have included the extended version of the EA-BO that can model the spatially varying ρ\rho into the revised article.

Question 3: The performance of EA-BO when the expert is misaligned

The Reviewer asks why the evaluation-only UCB and logEI outperform EA-BO in the misaligned results of Fig. 1 and Fig. 5, and, specifically, if this is due to pairwise data being used inefficiently or due to EA-BO using a global correlation parameter ρ\rho.

When the expert is aligned (ρ=1\rho = 1), EA-BO outperforms the evaluation-only methods UCB and logEI, since its mixed (both comparisons and evaluations) strategy can utilize cheaper and informative comparisons. However, when the expert is misaligned (ρ=0\rho = 0), the evaluation-only strategy is optimal, and thus the performance of the UCB and logEI is the upper limit: it is not possible to use pairwise queries efficiently, since they are completely uninformative. Further, having a spatially varying ρ\rho in the model is not useful, since the true ρ\rho is globally zero.

In the case of the misaligned expert, EA-BO is able to quickly learn that the expert is not reliable, and thus switch from the mixed strategy to the optimal evaluation-only strategy: the small performance difference between the evaluation-only methods and EA-BO is equal to the proportion of the budget that EA-BO has to spend for detecting an unreliable expert. EA-BO outperforms the earlier HITL BO method CoExBO (Adachi et al., 2024), which uses uncertainty weighting and scheduled decay to handle unreliable experts instead of estimating reliability.

References

Ahneman, Derek T., et al. "Predicting reaction performance in C–N cross-coupling using machine learning." Science 360.6385 (2018): 186-190.

Boobier, Samuel, Anne Osbourn, and John BO Mitchell. "Can human experts predict solubility better than computers?." Journal of cheminformatics 9.1 (2017): 63.

McCoy, Stephen A., et al. "Large language models as proxies for human experts in synthesis optimization." Digital Discovery 5.10 (2026): 4058-4062.

Thanks for answering the questions and weaknesses. I appreciate the authors looking into a spatially variable correlation. While I appreciate the inclusion of human and LLM expert results, they amplify my concerns pertaining to performance of EA-BO compared to evaluation-only baselines. Overall, the question of how to incorporate an expert's pairwise judgement is difficult to answer universally precisely because the degree of alignment can vary considerably across tasks.

In the newly added results, the Buchwald–Hartwig: briefed LLM result seems to be the only one where EA-BO clearly outperforms evaluation-only baselines (please correct me if I am wrong). To fundamentally resolve this issue, we either need (1) better modeling paradigms (for eg the spatially variable correlation and potentially more) to combat the sensitivity of the approach to an unaligned expert, or, (2) we need a theoretical explanation supporting that EA-BO is indeed able to quickly learn when an expert is not reliable, without wasting too many queries. The present scope of the paper offers neither, and the spatially variable correlation is included as an additional ablation in the paper, whereas in my opinion, it should be a standard choice, and authors must explore the issue of robustness to weakly aligned experts in greater detail. I agree that the objective in BO is optimization and not parameter estimation, but in the absence of good quality data, parameter estimation can be essential in informing an accurate cost landscape to optimize over, because currently the expert data maybe corrupting the optimization globally. In the absence of either strong modeling choices, empirical results supporting robustness to expert data, or theoretical explanation (for eg regret analysis explaining the fallback to evaluation-only paradigm for weakly aligned experts), I would keep my original score. I am open to increasing the score if the authors believe the paper addresses one of these issues clearly and can point me to specific parts in the paper for that.

Thank you for your thorough response. We have revised the article to address two out of three main concerns mentioned in your response (Providing additional theoretical results was unfortunately not feasible within a remaining day of the author response period.). In particular, we have: (1) strengthened modelling choices by making the spatially variable correlation model the standard model (See Section 3 for the updated model definition, Figure 1–3 for the updated main results, and Appendix F.2 for the ablation comparing the original and updated model.); (2) provided empirical results supporting robustness to data from weakly aligned experts (See Appendix E.6.).

Thank you for insisting on these revisions. They make the article stronger. In particular, modelling the spatially variable correlation improves the optimization performance when the true reliability is spatially variable (Figure 20, Table 20), while causing no observable performance penalty when it is global (Figure 19).

Update: the model with spatially varying correlations is now the standard choice

As you suggested in your response, we have now revised the article so that modelling a spatially variable expert reliability parameter ρ(x)\rho(x) is the default choice (See Section 3, where we define the model.). We have re-ran the main experiments of the article using the spatially variable version (Figures 1 and 3) and added a comparison where also the ground truth reliability is spatially variable (Figure 2).

See also the updated Appendix F.2., where we compare the performance of the original model and the updated model. When the true reliability ρ\rho is globally constant, there is no observable performance penalty from using a more complicated spatially variable model (Figure 19). However, when the true reliability is spatially variable, the updated model is able to leverage it to outperform the original model, in particular on the Hartmann6 data set (Figure 20 and Table 20).

Empirical results supporting robustness to data from weakly align experts

We provide additional empirical evidence that EA-BO is able to quickly learn when an expert is not reliable, without wasting too many queries in the regime where the expert is weakly aligned (see Appendix E.6).

We have reproduced the main results in the table below. Entries are mean comparisons performed by EA-BO (the updated version with spatially variable correlation parameter) in successive intervals of 20 budget units. When the expert is misaligned, EA-BO buys significantly fewer comparisons already in the second interval. By the third interval, i.e. 2/52/5 into the total budget, only few unreliable comparisons are bought. Thus the detection and identification happens while there is substantial optimization budget left.

Further, Figures 7–10 indicate that the drop in the amount of queries performed coincides with the drop of the posterior mean of the reliability parameter ρ\rho, verifying that the algorithm works in a principled fashion.

ProblemExpert / true reliability0–2020–4040–6060–80
Ackley5Constant, ρ=1\rho=110.016.412.711.5
Ackley5Constant, ρ=0\rho=010.00.00.00.0
Hartmann6Constant, ρ=1\rho=110.020.014.47.7
Hartmann6Constant, ρ=0\rho=010.08.21.70.2
B-HBriefed LLM10.018.018.014.4
B-HUnbriefed LLM10.012.83.51.1

Could previously collected comparisons continue to harm optimization?

In Figures 14-16 we examine the effect of removing the comparisons from EA-BO’s dataset and computing the exact posterior using only the evaluations. This allows us to measure the incurred harm from allowing misspecified data into the dataset.

We see that in all of the misaligned cases the incurred harm is close to 00, suggesting that the only negative effects from querying a misspecified expert are the opportunity cost of not spending the budget spent on queries on evaluations instead and the locations of bought evaluations. Thus, there is negligible residual harm from the misalignment to the posterior.

Reviewer_RNxWOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes a cost-aware HITL BO method, EA-BO, which is aligned with practical human-involved experimental settings for expensive black-box latent function ff approximation. Three features of EA-BO make it outstanding in the HITL BO literature: i) pairwise comparison of human experts is used (more robust and natural than quantitative evaluation); ii) the potential mistakes in human judgments are considered; and iii) the BO algorithm is constrained by evaluation cost. There have been some predecessors for each feature alone in the literature. However, it is novel to combine these three in a coherent way and leave a sound and applicable structure in the human-involved optimization and preferential learning field. Their experiments illustrate that the proposed method (EA-BO) can achieve more efficient optimization than the BO methods without human interaction when their judgment is accurate, while maintaining a more stable estimation than those that treat expert information as an oracle. This illustrates the flexibility of EA-BO in real-world applications.

Critical strengths
  1. The research question is well-demonstrated. This paper provides a clear research gap and detailed but concise comments on the related work. Also, the contributions of the paper are fairly stated, with great potential for future applications.
  2. Sound and innovative methodology:
  • Bridge the real latent function ff and the human mental model gg with LMC and map (f,g)(f,g) to the latent GP (u0,u1)(u_0,u_1) sharing the same kernel function.
  • build a cost-aware acquisition function by considering the expected one-step gain for both attempts (evaluation or comparison) and their individual cost.
Critical weaknesses

The experiments are kind of insufficient to demonstrate the full contributions of EA-BO. Parameter ρ\rho, which stands for human reliance, is the key parameter for EA-BO methods. It is estimable according to Section 3.2. However, in the experiments, ρ\rho seems to be fixed externally. For Figures 1 and 2, it is fixed at either 0 (biased) or 1 (aligned). For Figure 3 (middle subplot), it is also used as an external setting with some known value in [0,1][0,1]. The parameter recovery performance and sensitivity of ρ\rho in detecting humans' judgments in simulated data are expected to be shown. The benchmark comparison of the methods when humans' judgment is mildly biased (such as ρ=0.5\rho = 0.5) is also supposed to be displayed.

Questions for authors
  1. The non-monotonic relationship between AUC and ρ\rho is intriguing. I am curious to see the EA-BO performance benchmarking with other methods when ρ∈(0,1)\rho\in (0,1), such as the ρ=0.25\rho = 0.25 in the Ackley dataset.
  2. How is the parameter recovery performance of ρ\rho in a simulated dataset?
  3. In the discussion section, "... exploiting cheap pairwise queries in the early stages of the optimization..., and then using more accurate direct objective function evaluations to find the maximum from these regions". I didn't see any analysis of the proportion of objective function evaluations vs. pairwise comparisons over the total cost. Would the authors show more details on it?
3 comments

Thank you for your positive review. Most importantly, we want to clarify that EA-BO learns the parameter ρ\rho from the data in all of the experiments. We have uploaded the revised article in which we address the other concerns: experiments at the intermediate values of ρ\rho (Appendix E.3, Table 6), parameter recovery (Figure 11, Table 12), and budget allocation (Figure 4, Table 7).

Weakness 1: Fixed ρ\rho in the experiments

The Reviewer states as a weakness that the even though the parameter ρ\rho measuring the expert’s reliability is estimable, its fixed externally in the experiments.

This critique seems to be based on a misunderstanding of our experimental setup. We want to clarify that EA-BO learns the value of the correlation parameter ρ\rho from the data, and we do not give EA-BO the true value of ρ\rho in any of the experiments (except in the ablation of Fig. 17, where we explicitly measure the cost of learning the parameter ρ\rho). EA-BO is able to detect the reliability of the expert automatically, and allocate its budget between pairwise comparisons and direct function evaluations accordingly (see Figure 4, middle panel).

The values of ρ\rho in Figures 1, 5 and 4 refer to the value used to generate the data. We denote by ρ\rho as the posterior mean of this parameter estimated by EA-BO, which probably explains the misunderstanding. We will ensure that this is clear in the text.

Weakness 2 & Question 2: Parameter recovery performance for ρ\rho

The Reviewer requests to see the parameter recovery performance of the parameter ρ\rho in a simulated dataset. The trajectories of the posterior estimate of the parameter ρ\rho are shown in Figure 11 for the aligned (ρ=1\rho = 1) and the misaligned (ρ=0\rho = 0) expert. EA-BO is able to quickly identify whether the expert is reliable or not. Note that in the case of an unreliable expert, the posterior estimate of ρ\rho does not go below value 0.20.2, since at that point EA-BO detects that the comparisons are anyway too unreliable to be cost-effective, and switches to the evaluation-only strategy.

To test the parameter recovery performance also on the intermediate values of ρ\rho, we measured the final posterior means for the true values ρ=0/0.25/0.5/0.75/1\rho = 0/0.25/0.5/0.75/1 (see Table 12). EA-BO is able to identify the expert reliability with sufficient accuracy: the final posterior means (ρ=0.27/0.29/0.68/0.91/1.00\rho = 0.27/0.29/0.68/0.91/1.00) are slightly biased towards alignment due to an informative prior.

Weakness 3 & Question 1: Experiments with intermediate values of ρ\rho

The Reviewer requested to see the experimental results with intermediate values of ρ\rho, such as ρ=0.25\rho = 0.25 and ρ=0.5\rho = 0.5. To address this request, we have conducted additional experiments at the requested values of ρ\rho. See Appendix E.3 for the details and the full results. The table below contains the summary of the results. EA-BO significantly outperforms the earlier HITL BO method CoExBO and all the comparison-based methods also on the intermediate values of ρ\rho on both data sets, and all the evaluation-only methods on the Hartmann6 data set.

Mean AUC at intermediate expert reliability.

ProblemTrue ρ\rhoEA-BOBest evaluation-onlyBest comparison-onlyCoExBO
Ackley0.250.620.680.310.39
Ackley0.50.670.680.470.44
Hartmann60.50.730.6530.380.36

Question 3: Budget allocation

The reviewer requested an analysis of the proportion of objective function evaluations vs. pairwise comparisons over the total cost.

Budget allocation for comparisons at different stages of the optimization can be found at Table 7. The proportion of objective function evaluations vs. pairwise comparisons over the total cost as a functions of the evaluation noise cevalc_{\mathrm{eval}}, the reliability of the expert ρ\rho, and the comparison noise σcomp\sigma_{\mathrm{comp}} can be found in Figure 4 of the revised article. To summarize the results, the proportion of the direct function evaluations increases when (1) the evaluation cost cevalc_{\mathrm{eval}} decreases; (2) the expert reliability ρ\rho decreases; or (3) the comparison noise σcomp\sigma_{\mathrm{comp}} increases.

Reviewer_RNxWOpenReview ↗

Reply to author's comments

Thanks for explaining how ρ\rho works and its performance under more natural simulation settings in Appendix E.2 and Table 8. The results are quite interesting and have persuaded me about the flexibility of EA-BO.

Only one concern remains: in Figure 5, ρ\rho appears to be overestimated in the biased condition (true ρ=0\rho = 0). Do you have results for an intermediate condition (e.g., ρ=0.5\rho = 0.5) that would show whether this bias is systematic? There is no need to run additional experiments if such results are not already available.

Thank you for the follow-up. We are glad to hear that the additional results were useful.

Yes, we have the posterior means for the intermediate values of ρ\rho. The final values at the budget exhaustion (along with the number of comparisons performed) for the Ackley data set can be found in Table 12, and are reproduced below. The trajectories can be found on the middle panel of Figure 11.

The slight overestimation of ρ\rho is systematic. Based on the trajectories of Figure 11, the dominant explanation depends on the true value of ρ\rho:

  1. When the expert is unreliable (ρ≤0.25\rho \leq 0.25), the posterior mean of ρ\rho drops sharply, and then levels off. The levelling off happens when EA-BO detects that the expert is too unreliable for the pairwise comparisons to be cost-effective, and switches to the optimal evaluation-only strategy. Hence, the estimate of ρ\rho is not updated anymore.
  2. When the expert is moderately reliable ρ≥0.5\rho \geq 0.5, the posterior mean of ρ\rho drops slowly, and has not yet reached a true value at the budget exhaustion. This is due to the informative prior towards alignment (see Appendix D.2); we use it to encourage comparisons early, where they are most effective for exploration.

Thanks for suggesting a further analysis of the reasons for overestimation of ρ\rho: we were able to identify that the explanation depends on the reliability regime. This makes the exposition of the article more clear. We have updated the above analysis into the revised article (Appendix E.7).

Final posterior mean of ρ\rho and comparisons bought by EA-BO on Ackley.

True ρ\rhoEstimated ρ\rhoComparisons performed
00.2710.5
0.250.2915.5
0.50.6867.0
0.750.9174.5
11.0079.0
Reviewer_aYBpOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a human-in-the-loop Bayesian optimisation approach that uses pairwise comparisons to incorporate expert knowledge alongside direct function evaluations. The method uses correlated Gaussian processes to model the objective function and expert preferences, with a learned correlation parameter to account for potentially unreliable expert feedback. A cost-aware acquisition function determines whether to seek further expert comparisons or evaluate the objective directly. Experiments on single- and multi-objective benchmark problems show improved optimisation efficiency when expert comparisons are cheap and reliable, while remaining competitive when expert feedback is unreliable.

Critical strengths

The motivation for the problem is grounded. It's profoundly easy for an expert to provide pairwise comparison than to provode exact value. Thus this work is built on to a solid line of work is also solves a practically relevant problem. The aspect of cost-aware optimisaiton and brining that into whether to evaluate or ask the expert is pretty interesting. The technical expositions seem correct and thus the overall presentation is solid. There has also been a decent technical contrinution in terms of extending the derivation of closed form conditional mean from probit likelihood from sible objective to multi-objective scenario.

Critical weaknesses

Simplistic modelling of expert reliability: The use of a single global correlation parameter assumes that expert reliability is consistent across the design space and different objectives. In practice, experts may be reliable in certain regions but not others. This is a critical flaw in my opinion.

Limited evaluation of realistic expert behaviour: The experiments rely on simulated expert preferences, primarily considering perfectly aligned or completely uninformative experts. These settings do not adequately capture systematic biases, inconsistent judgments or locally varying expertise.

Insufficient baseline comparisons: Although the paper compares against several BO approaches, it lacks a simple baseline that would combine a preference-learnt GP, fused with actual evaluation learnt GP and then processed through one of several cost-aware acquisition function. This is a critical gap to understand the true benefit of the proposed algorithm.

Also, the novelty of the method is primarily in combining the exisiting methods and not a ground reaking method. Thus it falls on the middle of the scale based purely on the methodological contribution.

Questions for authors

It would be interesting to understand what would authors think if the expert's feedback is noisy and is non-linear function of the experimental space. What would need to be done to include such a realistic expert model?

Further, I would like the suthor to relfect on the simple baseline comparsion that I mentioned in the weakness section. Would that be a strong contender to the proposed algorithm? Why or why not?

3 comments

Thank you for your review. We have uploaded a revised article where we address all three critical weaknesses: (1) We perform additional experiments where comparisons are made by an human expert or an LLM expert (Section 6.1., Figure 6, Appendix E.2). (2) We extend the proposed method so that it can handle output-specific (Appendix F.1) and spatially variable (Appendix F.2) reliability. (3) We add the requested baseline to the experiments (Appendix E.5).

Weakness 1: Simplistic modelling of expert reliability

The Reviewer criticizes the use of a single global correlation parameter, and requests that it should vary across the design space and different objectives. To address this concern, we extended EA-BO so that it can model both output-specific and locally varying correlations:

(1) We added the output-specific correlation parameters ρj\rho_j for each j=1,…,mj = 1, \dots, m. We tested this extension on the Branin-Currin data set with true reliabilities (ρ1,ρ2)=(0,1)(\rho_1, \rho_2) = (0,1) and (ρ1,ρ2)=(1,0)(\rho_1, \rho_2) = (1,0). The extended EA-BO robustly identified whether or not an expert was reliable for the given output: the estimated posterior means for the correlations (ρ1,ρ2)(\rho_1, \rho_2) were (0.41,0.99)(0.41,0.99) and (0.99,0.33)(0.99,0.33), respectively. However, modelling output-specific correlations did not improve the optimization performance (see table below for mean AUCs).

(2) We added the spatially varying correlation parameter ρ(x)=Φ(η(x))\rho(x)=\Phi(\eta(x)), where η\eta has a GP prior with a linear kernel and Φ\Phi is a probit function. We tested the extended model on the Hartmann6 data set with a true reliability ρ(x)=0.9\rho(x) = 0.9 at the neighborhood of the global optimum and ρ(x)=0.1\rho(x) = 0.1 at the neighborhood of the local optimum, and the reverse setting. The estimates correctly ordered reliability at the two optima, but remained biased towards the value at the global optimum, where sampling is concentrated. Modelling locally varying correlations slightly improved the optimization performance (see table below for mean AUCs).

Mean AUC for global and extended EA-BO. The extension is output-specific on Branin–Currin and spatial on Hartmann6; G and L denote the global and local optima.

ProblemTrue reliabilityEA-BO (global)EA-BO (extended)
Branin–Currin(ρ1,ρ2)=(0,1)(\rho_1,\rho_2)=(0,1)0.8650.856
Branin–Currin(ρ1,ρ2)=(1,0)(\rho_1,\rho_2)=(1,0)0.8470.836
Hartmann6(ρG,ρL)=(0.9,0.1)(\rho_G,\rho_L)=(0.9,0.1)0.7770.791
Hartmann6(ρG,ρL)=(0.1,0.9)(\rho_G,\rho_L)=(0.1,0.9)0.6100.652

Weakness 2: Limited evaluation of realistic expert behaviour

The concern of the Reviewer was that the experiments rely on simulated expert preferences, primarily considering perfectly aligned or completely uninformative experts. We address this concern by conducting additional experiments in which preferences are provided either by an human expert or by an LLM expert. The performance of EA-BO is similar as on the synthetic data sets. When the expert is reliable, it outperforms CoExBO and outperforms or matches the performance of the evaluation-only methods; when the expert is unreliable, it is able to detect the unreliability and maintain a competitive performance, whereas the comparison-only methods fail (see Section 6.1 and Appendix E.2. for additional details).

We added two tasks: (1) Optimizing the solubility of drug-like molecules (Boobier et al., 2017). The pairwise comparisons are based on the estimates of the two human experts collected by Boobier et al. (2017). The average reliabilities of these experts were 8383\\% and 6262\\%. (2) Optimizing the catalytic activity for Buchwald-Hartwig reaction (Ahneman et al., 2018). The pairwise comparisons are provided either by an LLM that is briefed by 80 measured reactions from a previous related experiment, or by an unbriefed LLM, following the approach by McCoy et al. (2026).

See the table below of the summary of the results. The full results can be found in Figure 6 and Table 5. On the solubility task, there was not a significant difference between EA-BO and the evaluation-only methods. EA-BO outperformed the comparison-only methods for the less reliable expert (ρ=0.62\rho = 0.62).

The results on the Buchwald-Hartwig data set suggest that an LLM can be used instead of the human expert to improve the cost-efficiency of BO: when the LLM was briefed, both EA-BO and the comparison-only methods outperformed the evaluation-only methods. This means that the LLM expert alone outperformed the experiments-only methods. When the LLM was not briefed, it acted basically as an unreliable expert, and thus the comparison-only methods failed. However, as with the synthetic data sets, EA-BO was able to identify the unreliability of the expert, and, consequently, to reach similar final utility as the evaluation-only methods.

Mean AUC / Final utility on realistic-feedback settings. Solubility percentages denote expert comparison accuracy.

SettingEA-BO (AUC / final)CoExBO (AUC / final)Best eval.-only (AUC / final)Best comp.-only (AUC / final)
Solubility (83%)0.855 / 0.9360.841 / 0.9360.859 / 0.9360.890 / 0.924
Solubility (62%)0.848 / 0.9360.802 / 0.9070.859 / 0.9360.747 / 0.736
B-H, briefed0.754 / 0.9140.668 / 0.9050.715 / 0.8610.761 / 0.861
B-H, unbriefed0.601 / 0.8570.656 / 0.8860.715 / 0.8610.367 / 0.288

Weakness 3 & Question 2: Insufficient baseline comparison

The Reviewer requested a baseline that models evaluations and comparisons by independent GPs that are combined and does acquisition via a cost-aware acquisition function. We added this baseline (”Fusion”, see Appendix E.5 for details) to the experiments of Fig. 1 (see also Table 9). EA-BO outperforms the baseline both in the case of a reliable (ρ=1\rho=1) and an unreliable expert (ρ=0\rho=0). The baseline is not a strong contender to EA-BO: since the GPs are independent, it cannot detect that the expert advice is incorrect, and, consequently, fails when the expert is unreliable.

Mean AUC for EA-BO and Fusion (Appendix E.5).

ProblemTrue ρ\rhoEA-BOFusion
Ackley00.6230.449
Ackley10.8130.731
Hartmann600.4680.280
Hartmann610.8610.838

Question 1: Realistic expert model

The Reviewer asked what would authors think if the expert’s feedback is noisy and is non-linear function of the experimental space. In our expert model, the expert’s feedback is already noisy, and it is generated from the non-linear function of the experimental space X\mathcal{X}:

(1) The mental model of the expert, denoted by g:X→Rmg:\mathcal{X} \rightarrow \mathbb{R}^m, is a non-linear function of the experimental space in all of the experiments, and we model it by an infinite-dimensional GP surrogate.

(2) All of our experiments include comparison noise. It is defined by the parameter σcomp\sigma_{\mathrm{comp}} that has the value σcomp=0.1\sigma_{\mathrm{comp}} = 0.1 in the main experiments. In addition, we study the effect of varying the value of σcomp\sigma_{\mathrm{comp}} in our ablations (See Section 6.2 and Fig. 4).

We have included additional experiments in which also the utility function U:Rm→RU:\mathbb{R}^m \rightarrow \mathbb{R} (that is a function on the observation space Rm\mathbb{R}^m) is non-linear (weighted Chebyshev utility) to the revised article; See Appendix E.4.

References

Ahneman, Derek T., et al. "Predicting reaction performance in C–N cross-coupling using machine learning." Science 360.6385 (2018): 186-190.

Boobier, Samuel, Anne Osbourn, and John BO Mitchell. "Can human experts predict solubility better than computers?." Journal of cheminformatics 9.1 (2017): 63.

McCoy, Stephen A., et al. "Large language models as proxies for human experts in synthesis optimization." Digital Discovery 5.10 (2026): 4058-4062.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.