acceptodds
Under review as a conference paper at ICLR 2027

IQS-BO: In-Context Query Selection for Bayesian Optimisation

Abstract

Bayesian Optimisation (BO) is a powerful framework for the optimisation of expensive black-box functions, but typically requires refitting a surrogate and maximising an acquisition function at every evaluation step. In-context approaches based on Prior-data Fitted Networks (PFNs) amortise part of this cost by pre-training transformers on functions drawn from synthetic priors. PFNs4BO amortises the surrogate but still relies on a numerically maximised acquisition function, while FIBO performs BO fully in-context by sampling optimiser locations from a learned density, which fixes the decision rule and admits no surrogate. Learned acquisition functions score a finite candidate set with a trained network, but, lacking a label for the query, learn the score by reinforcement learning on previously solved tasks. We propose IQS-BO, a PFN that learns the query decision by supervised learning on synthetic priors. In a single forward pass, IQS-BO predicts the probability that each candidate maximises the objective over the set, and we show that the minimiser of its objective is the posterior probability of this event. The model can be pre-trained without a surrogate for fully in-context BO, or take the predictions of a fixed probabilistic surrogate as additional input, amortising only the decision step. Our method proposes queries at a fraction of the cost of acquisition-based methods, while either matching or outperforming standard BO with Gaussian processes (GPs) and available in-context methods on synthetic and real-world benchmarks. Finally, we propose a mixture prior for pre-training PFNs which combines samples from GPs with functions exhibiting warped inputs, isolated narrow optima, or plateaus that are poorly modeled by stationary kernels common in GP surrogates. We show that pre-training on this prior can lead to improved optimisation performance.

Primary area:
optimization
Keywords:
bayesian optimisation, black-box optimisation, in-context learning, prior-data fitted networks, gaussian processes

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_zdFTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IQS-BO, an in-context Bayesian optimisation method that selects the next evaluation point from a finite candidate set using a pretrained Transformer. The method is trained on synthetic functions to predict which candidate has the highest objective value, using a supervised classification loss. It can operate without a surrogate model or use predictions from a pretrained PFN or a Gaussian process. The paper also introduces a mixture prior for pretraining and evaluates the method on synthetic and real-world problems, including batch optimisation.

Critical strengths
  1. The paper presents an interesting approach to learning query selection through supervised training on synthetic functions, without requiring reinforcement learning.
  2. The experiments include several relevant BO baselines, controlled comparisons using the same candidate sets, and evaluations of batch optimisation. The results show competitive optimisation performance and lower query-selection time in the tested settings.
Critical weaknesses
  1. The probability that a candidate is the best can also be estimated using Monte Carlo sampling from a probabilistic surrogate's joint posterior. The paper does not compare IQS-BO with this approach in terms of optimisation performance and computational cost.
  2. The training objective only identifies the best candidate, without considering improvement magnitude or information gain. It is unclear how this objective balances exploration and exploitation over multiple BO iterations.
  3. The experiments show that performance depends strongly on candidate generation, particularly local perturbations around the best observations. It is unclear whether the results generalise to other candidate sampling strategies.
  4. The experiments only consider three- and four-dimensional problems, with batch experiments restricted to three dimensions. More experiments on higher-dimensional problems are needed to evaluate the scalability of IQS-BO.
  5. In Equation (6), each candidate is assigned a score independently of the other candidates before applying softmax. However, the probability of being the best generally depends on the joint posterior distribution over all candidates. It is unclear whether the proposed architecture can accurately represent this probability.
Questions for authors
  1. How does IQS-BO compare with directly estimating the probability of being the best candidate using Monte Carlo posterior sampling?
  2. How does the proposed training objective balance exploration and exploitation without explicitly considering improvement magnitude or information gain?
  3. How well does IQS-BO perform with other candidate generation strategies, given the sensitivity shown in Table 2 and Appendix F.5?
  4. How would IQS-BO perform on higher-dimensional optimisation problems?
  5. The proposed method can be viewed as a learned candidate-scoring rule. Beyond the comparisons with EI, what advantages does it offer over acquisition functions such as PI, UCB, and KG?
  6. Given that each candidate is scored independently before softmax, can the proposed architecture accurately represent the posterior probability that a candidate is the best among all candidates?
5 comments

We thank the reviewer for the analysis and for the accurate questions, which led to useful additional analyses and experiments. The revised paper now includes a comparison of the probabilities predicted by the network with Monte Carlo estimates (App. I), an analysis of its exploration-exploitation decisions ( App. J), a comparison with five further acquisition functions and more candidate generators (Sec. 4.4, App. G.5), and a high-dimensional study up to 200 dimensions (Sec. 4.3, App. H). Overall, these additions confirm that the network accurately approximates the posterior target, meaningfully adapts its exploration and exploitation to the context, is robust to variations in the candidate set and scales well to high dimensions.

Weakness (W1) and Question (Q1): "The probability that a candidate is the best can also be estimated using Monte Carlo sampling from a probabilistic surrogate's joint posterior. The paper does not compare IQS-BO with this approach in terms of optimisation performance and computational cost. [...] How does IQS-BO compare with directly estimating the probability of being the best candidate using Monte Carlo posterior sampling?"

We added this comparison in the new App. I. In short, IQS-BO estimates the probability that each candidate is the best about as accurately as Monte Carlo under a correctly specified GP (a GP that has the same mean and kernel functions used to generate the data, with only the hyperparameters fitted to the available context). On real problems, IQS-BO selects better queries than Monte Carlo with GP surrogates, and reaches better optimisation performance at a smaller cost.

To come to these conclusions, we investigated the following two routes:

1. Exact posterior. First, we considered a setting where the posterior distribution is available in closed form, and posterior probabilities for each candidate can be accurately estimated by Monte Carlo. We draw functions from a GP with a Matérn-3/2 kernel, 400 for each context size n=10, 20 and 80, integrate out the hyperparameters, and construct the posterior probability distribution of each candidate being the best within a set of 64 candidates generated from the data. We then compare these distributions with those implied by IQS-BO and by Monte Carlo sampling using a GP with the same kernel used to generate the data, but with hyperparameters fitted by MAP. This setting favours Monte Carlo, since its surrogate models the same distribution that generated the data. Without any fine-tuning, the network is about as accurate as Monte Carlo with GP-MAP, and more accurate with few observations, which determine the GP's hyperparameters poorly. Fine-tuned on tasks drawn from the process that generated the data, the network is closer to the exact probabilities than Monte Carlo with GP-MAP at every context size. This indicates that the gap in posterior approximation accuracy is not a limit of the architecture, and can be reduced with additional training.

2. Real optimisation tasks. Second, we compared the quality of the queries on more realistic optimisation tasks, where no posterior is available in closed form and Monte Carlo is limited by how well its surrogate is specified. On 600 BO runs on the 3D benchmarks, we compared the next query proposed by IQS-BO with that of two Monte Carlo variants. The first uses a GP with a fixed kernel, the same surrogate of IQS-BO-GP. The second uses a GP selected at every query among ten with different kernels, to fit each problem as closely as possible, which is expensive. The query of IQS-BO is the true best candidate in 26% of cases, against about 18% for Monte Carlo with the fixed GP. Monte Carlo with the selected GPs reaches the same query quality of IQS-BO only after several thousand draws, at about 50 times its cost per query.

Both experiments show that IQS-BO, trained with the proposed objective and priors, is capable of accurately estimating the probability that each candidate is the best in a single forward pass. It does so at a lower cost than Monte Carlo with well-specified GPs, and without requiring the selection of kernel and mean functions for each optimisation problem.

Weakness (W2) and Question (Q2): "The training objective only identifies the best candidate, without considering improvement magnitude or information gain. It is unclear how this objective balances exploration and exploitation over multiple BO iterations. [...] How does the proposed training objective balance exploration and exploitation without explicitly considering improvement magnitude or information gain?"

While the proposed loss does not explicitly model information gain or use estimates of uncertainty, the network learns to adjust its exploration-exploitation decisions directly from the context. Importantly, these decisions remain accurate on functions outside the pre-training priors (App. K, see also our reply to Reviewer SGpx).

The network is trained to predict the probability that each candidate is the best within a set. With few observations, this probability is spread over unexplored regions, and the network explores. As observations accumulate, it concentrates near the best observations, and the network exploits. To quantify this shift from exploration to exploitation, and to compare it with acquisition functions, we measured the weight that the queries of each method put on uncertainty relative to predicted value (App. J). For IQS-BO, this weight is large with few observations and falls to almost zero as the context grows. Thompson sampling and UCB behave similarly, whereas expected improvement and the knowledge gradient keep a slowly declining weight and explore more later in a run.

In our experiments, including improvement magnitude in the loss did not improve performance. To test this, we pre-trained IQS-BO to predict the improvement of each candidate over the best observation so far. This variant performs similarly to GP-MAP with expected improvement, but IQS-BO with the proposed loss outperforms it in 84 to 87% of paired runs (Table 1). In addition, evaluated using the same GP surrogate and candidates, IQS-BO-GP outperforms expected improvement and the knowledge gradient, as well as max-value entropy search, which considers information gain (Fig. 6).

Weakness (W3) and Question (Q3): "The experiments show that performance depends strongly on candidate generation, particularly local perturbations around the best observations. It is unclear whether the results generalise to other candidate sampling strategies. [...] How well does IQS-BO perform with other candidate generation strategies, given the sensitivity shown in Table 2 and Appendix F.5?"

Our ablations indicate that IQS-BO is robust to both the composition and the size of the candidate set, as long as the set keeps small perturbations of the best observations.

To test robustness to candidate composition, we ran the same IQS-BO model, without retraining, with twelve further candidate generators, each varying one aspect of the default set (App. G.5). To separate the effect of the candidate set from that of the learned selection, we also ran five standard acquisition functions on the same sets. On all sets that keep small perturbations of the best observations, IQS-BO maintains its performance, and has a lower mean final regret than all considered acquisition functions. When queries are selected from a finite set, this dependence on perturbations is not specific to IQS-BO. All acquisition functions degrade in performance without them because space-filling candidates rarely land close enough to the best observations, making exploitation decisions less accurate

Weakness (W4) and Question (Q4): "The experiments only consider three- and four-dimensional problems, with batch experiments restricted to three dimensions. More experiments on higher-dimensional problems are needed to evaluate the scalability of IQS-BO. [...] How would IQS-BO perform on higher-dimensional optimisation problems?"

The revised paper includes a new high-dimensional study (Sec. 4.3 and App. H). We use ten scalable problems at d=20, 50, 100 and 200, with 700 evaluations per run, and compare with state-of-the-art methods for high-dimensional BO: GIT-BO and TabPFN-EI, which rely on a tabular foundation model, and the trust-region method TuRBO. IQS-BO keeps its architecture and is pre-trained once for each dimension. We also evaluate IQS-BO-TabPFN, which reads a frozen TabPFN as surrogate.

IQS-BO scales well to high dimensions. The two IQS-BO models achieve the two best mean ranks at every dimension, with a lower regret than GP-MAP, TuRBO and GIT-BO and a regret similar to TabPFN-EI. IQS-BO is also faster, proposing a round in under 0.1 s, one to two orders of magnitude less than all surrogate-based baselines.

The same holds in batch optimisation at d=50 (Fig. 5). At every considered batch size (q=10, 20,50), IQS-BO has the best GAP on more problems than any baseline, and a mean GAP above that of GP-MAP, TuRBO and GIT-BO. Only TabPFN-EI reaches a slightly higher GAP, but takes 80 to 90 times longer to propose its queries.

Question (Q5): "The proposed method can be viewed as a learned candidate-scoring rule. Beyond the comparisons with EI, what advantages does it offer over acquisition functions such as PI, UCB, and KG?"

We have extended the controlled comparison of the paper from EI to PI, UCB, Thompson sampling, the knowledge gradient (KG) and max-value entropy search (Sec. 4.4). Each acquisition function uses the same GP surrogate as IQS-BO-GP and is evaluated on the same candidate sets, which isolates the contribution of the learned selection rule.

The results indicate three main advantages. First, the learned rule selects better queries: IQS-BO-GP reaches a lower final regret than every acquisition function except UCB in 61 to 79% of paired runs, at every context size. Against UCB, it wins 59 to 60% of runs with 40 initial observations and ties at small context sizes, for the two strongest values of β\beta, chosen after the runs. Second, IQS-BO has no exploration parameter to tune, since its weight on uncertainty adapts to the context (App. J). Third, IQS-BO does not require a probabilistic surrogate, and selects queries directly from the observations in a single forward pass.

Weakness (W5) and Question (Q6): "In Equation (6), each candidate is assigned a score independently of the other candidates before applying softmax. However, the probability of being the best generally depends on the joint posterior distribution over all candidates. It is unclear whether the proposed architecture can accurately represent this probability. [...] Given that each candidate is scored independently before softmax, can the proposed architecture accurately represent the posterior probability that a candidate is the best among all candidates?"

As correctly highlighted by the reviewer, independent scores cannot represent this posterior probability exactly in general; we have made this point clearer in the paper (see Sec. 3.2 and App. D). In practice, however, the architecture can accurately approximate it. Although each score is computed independently, the probabilities are not, since through the normalisation of the softmax the probability of each candidate depends on the scores of all the others. What the architecture cannot represent is that the odds of one candidate against another, exp⁡(z(D,cj)−z(D,ck))\exp\big(z(\mathcal D,c_j)-z(\mathcal D,c_k)\big), can change with the rest of the set. Adding a near-duplicate of a candidate, for example, splits its probability between the two copies, and so lowers its odds against every other candidate.

Proposition 1 shows that, for any network, the pre-training loss equals the expected entropy of the posterior, which no model can reduce, plus the expected KL divergence from the posterior to the network's output. The better the individual scores, the smaller this divergence. We measured this distance empirically, on tasks on which the exact posterior can be computed (see our reply to your first question (W1-Q1)). As detailed there, fine-tuning the network brings it closer to the exact posterior. This agrees with the decomposition, by which any reduction of the loss is a reduction of the distance to the posterior, and shows that independent scores are not the limit of the accuracy of the approximation.

We also tested training a model with self-attention among the candidates. This variant did not improve BO performance, but gives up two practical properties of independent scores. First, since candidates attend only to the observations, the cost of a forward pass grows linearly with the number of candidates, whereas attention among candidates makes it grow quadratically. Second, since each score depends on the whole set, sets larger than those seen in pre-training fall outside the training distribution, whereas independent scores let the size of the candidate set change between pre-training and deployment.

We hope that these additions address the reviewer's concerns, and we would be glad to answer any further questions.

Reviewer_fh3ROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces IQS-BO (In-Context Query Selection for Bayesian Optimisation), a method that replaces acquisition-function maximization with a learned query-selection policy. Given an observation history D={(xi,yi)}i=1nD=\{(x_i,y_i)\}_{i=1}^n and a finite candidate set C={c1,…,cM}C=\{c_1,\ldots,c_M\}, the method predicts \begin{equation} p_\theta(j\mid D,C)\approx \mathbb P\left(j=\arg\max_{k=1,\ldots,M}f(c_k)\mid D,C\right). \end{equation}

The model is trained on synthetic functions sampled from a pre-training prior. Because the underlying function is available during training, the best candidate can be identified directly, providing an exact supervised classification label. The authors prove that minimizing the population cross-entropy loss recovers the posterior probability that each candidate is the best within the set, assuming a well-specified model. The paper investigates three variants:

  1. IQS-BO: A fully in-context model that directly processes observations and candidates without an explicit surrogate.
  2. IQS-BO-PFN: A model that additionally uses predictions and representations from a frozen PFNs4BO surrogate.
  3. IQS-BO-GP: A model that uses posterior information from a Gaussian process whose hyperparameters are fitted to the observed data. For batch optimization, queries are selected sequentially, with previously selected batch points included as pending tokens. The authors also introduce a mixture pre-training prior combining Gaussian process samples with warped functions, localized Gaussian bumps, and plateau functions. The evaluation includes 15 synthetic and real-world benchmark problems in three and four dimensions, comparisons with GP-based BO, PFNs4BO, FIBO, NAP, and Sobol search, and batch optimization experiments with batch sizes up to 50. The reported results show that IQS-BO generally outperforms competing in-context methods, performs competitively with GP-based BO, and achieves substantial reductions in query-selection time. The controlled comparison against expected improvement on the same candidate set provides additional evidence that the learned selection rule contributes to the observed gains.
Critical strengths
  1. Simple and effective supervised formulation of query selection. The central idea is appealing: since synthetic functions can be evaluated at arbitrary points during pre-training, the authors can obtain exact labels identifying the best candidate and train a query-selection model using ordinary cross-entropy. This avoids the complexity of reinforcement learning or policy-gradient methods used in some previous learned acquisition approaches. It also makes pre-training straightforward and independent of expensive optimization rollouts.
  2. A unified framework supporting different surrogate configurations. The paper provides a flexible framework that works without a surrogate, with a frozen PFN surrogate, or with a conventionally fitted GP. This is useful because it separates the benefits of amortized query selection from those of amortized surrogate modeling. In particular, the IQS-BO-GP variant allows a relatively controlled comparison with conventional GP-based BO.
  3. Strong empirical results against relevant baselines. The evaluation includes several important baselines, including GP-MAP, GP-FB, PFNs4BO, FIBO, NAP, and Sobol search. The results in Table 1 and Figure 2 (pages 6–7) indicate that IQS-BO is competitive across different initial context sizes and generally performs particularly well on the three-dimensional benchmarks. Importantly, the authors also introduce EI-CAND, which uses the same GP and candidate set as IQS-BO-GP but selects candidates using expected improvement. This controlled comparison in Figure 3 (page 8) provides more convincing evidence for the value of the learned selection rule than comparisons against standard GP-BO alone.
  4. Particularly promising results for batch Bayesian optimization. The batch optimization experiments are among the strongest aspects of the paper. Figure 4 (page 9) shows that IQS-BO maintains strong optimization performance at batch sizes of 10, 20, and 50, while requiring substantially less query-selection time than GP-based and PFNs4BO baselines. This is practically relevant because the computational overhead of acquisition optimization can become substantial when selecting large batches.
  5. Useful investigation of the pre-training prior. The proposed mixture prior includes stationary GP samples, warped functions, localized bumps, and plateau functions. The experiments indicate that this richer prior can improve not only IQS-BO but also other PFN-based approaches. This suggests that the design of synthetic training distributions is an important factor in the effectiveness of amortized BO.
  6. Reasonable ablations and reproducibility details. The authors investigate candidate-set composition, candidate-set size, and sequential versus simultaneous batch selection. Table 2 (page 9) demonstrates that perturbations around promising observations are particularly important, while the batch-selection ablation supports the use of pending-point conditioning. The appendices also provide implementation details, training configurations, and benchmark definitions.
Critical weaknesses
  1. The supervised objective is not directly aligned with the objective of Bayesian optimization. This is my primary concern. The method learns the probability that a candidate is the best within a finite set: \begin{equation} \mathbb P\left(f(c_j)=\max_{k}f(c_k)\mid D,C\right). \end{equation}

Maximizing this probability is optimal for identifying the best candidate under a 0–1 classification loss. However, BO generally seeks to minimize simple regret or maximize the value of the best observation after a limited number of evaluations. These objectives are not equivalent. For example, consider two candidates. Candidate A has a high probability of being marginally better than the other candidates, whereas candidate B has a lower probability of being the best but a potentially much larger improvement over the incumbent. The proposed classification objective may prefer A even when evaluating B has greater expected improvement or greater long-term value. Furthermore, the objective does not explicitly account for the information gained from an evaluation. It therefore lacks the exploration incentives associated with nonmyopic BO policies. Although the authors correctly establish Bayes optimality under their specified classification loss, the paper should more clearly distinguish this result from optimality for the actual BO objective. 2. The theoretical contribution is relatively limited. Proposition 1 establishes that the population cross-entropy minimizer recovers the conditional distribution of the classification label. This follows from the standard proper-scoring-rule property of cross-entropy and the nonnegativity of KL divergence. The result is correct and provides a useful interpretation, but it is not a particularly novel theoretical insight. More substantive analysis would be valuable, such as characterizing the relationship between posterior-best-candidate probability and expected improvement, or identifying conditions under which the proposed decision rule is effective for minimizing optimization regret. 3. Strong dependence on the candidate-generation strategy. Although the method is presented as learning query selection, it can only select points contained in the candidate set. Appendix C.2 specifies that the candidate set combines 40% scrambled Sobol points, 30% uniformly sampled points, and 30% local perturbations around the three best observed locations. The ablation in Table 2 shows that replacing these local perturbations with space-filling candidates substantially degrades performance. This raises an important concern: a considerable portion of the optimization capability may come from the manually designed candidate sampler rather than the learned decision rule. The EI-CAND comparison helps separate these effects, but the method's dependence on the candidate distribution remains a significant limitation. It is also unclear whether the same sampler would remain effective in higher dimensions, constrained domains, or problems with highly localized optima. 4. Evaluation is restricted to low-dimensional problems. The experiments cover only three- and four-dimensional optimization problems. This is a substantial limitation for a method intended to serve as a general BO framework. Many practical applications, including hyperparameter optimization, scientific experiment design, and engineering optimization, involve substantially larger search spaces. In higher dimensions, finite candidate sets become increasingly sparse, and local perturbations may be less effective. The paper does not establish whether the learned query-selection mechanism can scale beyond the relatively small problems considered. 5. Generalization across dimensions, batch sizes, and candidate distributions is insufficiently studied. The authors train separate models for fixed input dimensions and batch sizes. This introduces additional pre-training requirements when the deployment configuration changes and limits the flexibility of the proposed approach. The paper notes that candidate-set size can vary because scores are computed independently for each candidate. However, this does not necessarily establish robustness to changes in candidate-set distributions or problem geometry. A stronger evaluation would examine whether a single trained model can transfer across dimensions, batch sizes, or candidate-generation schemes. 6. The batch-selection training objective needs stronger justification. The batch extension is practically interesting but raises a conceptual question. As described in Appendix B.3, the training loss for subsequent batch positions is applied only when the best candidate from the sampled training set has not already been selected. This encourages diversification, but it is not equivalent to directly optimizing the expected best function value across the entire batch. Moreover, the training procedure uses a fixed candidate set within a task, while deployment generates fresh candidate sets for successive batch positions. The paper does not establish whether this training procedure approximates a well-defined Bayes-optimal batch decision rule. Given that large-batch performance is one of the paper's strongest empirical claims, a more careful analysis of the batch objective would strengthen the contribution. 7. Computational comparisons do not fully establish end-to-end efficiency. The paper reports substantial savings in query-selection time, which is a meaningful advantage. However, these comparisons primarily measure the computational cost of proposing queries. They do not fully account for the one-time pre-training cost, the need for separate models across dimensions and batch sizes, or the overall cost of achieving a target optimization quality. The authors report pre-training times of approximately 2.7–6.1 hours on a single H100 GPU. This may be reasonable when the model is reused across many tasks, but the economic advantage depends on the deployment setting. A comparison of total computational cost across different numbers of optimization tasks would help quantify when amortization becomes worthwhile. 8. The robustness of the method under prior misspecification remains unclear. The paper shows that a richer synthetic prior improves performance on several benchmarks. However, the model's query-selection policy is learned directly from this prior. Unlike GP-based BO, where the surrogate and acquisition function can sometimes be adjusted separately, IQS-BO implicitly encodes both the statistical assumptions and the decision rule in the trained network. This raises questions about robustness when the target function differs substantially from the pre-training distribution. Although the experiments include real-world benchmarks, a more systematic study of prior misspecification would be useful.

Questions for authors

Q1. Why is posterior-best-candidate probability the appropriate objective for Bayesian optimization? The proposed method maximizes the probability that a candidate is the best within the candidate set, which is Bayes-optimal under a 0–1 classification loss. However, BO is generally evaluated using simple regret, expected improvement, or final optimization performance. Could the authors explain under what assumptions maximizing the posterior-best-candidate probability is preferable to maximizing expected improvement or expected function value? Are there theoretical conditions under which this decision rule minimizes one-step simple regret? Q2. Can the proposed supervised learning framework accommodate alternative optimization objectives? Since the synthetic training functions are fully known, it should also be possible to construct labels or soft targets based on expected improvement, regret, or other utility functions. Have the authors considered learning a decision rule that directly minimizes expected simple regret rather than predicting the identity of the best candidate? A comparison between classification-based query selection and regret-oriented supervised objectives would help clarify whether the proposed target is essential to the observed performance. Q3. How much of the performance improvement comes from the candidate-generation strategy? The candidate sampler includes local perturbations around the three best observations, and the ablation study indicates that these perturbations are critical. Could the authors compare IQS-BO against stronger candidate-based acquisition optimization methods using exactly the same candidate sets, including alternative acquisition functions such as UCB or Thompson sampling? It would also be useful to evaluate performance when candidate sets are generated independently of the observed function values. Q4. How does IQS-BO scale to higher-dimensional optimization problems? The evaluation is restricted to three and four dimensions. Could the authors provide results on problems with 10, 20, or more dimensions? In particular, how do performance and computational cost change as the dimension increases while the candidate-set size remains fixed?

3 comments

We thank the reviewer for the positive assessment of our work and for the useful remarks and questions. The revised paper now includes a comparison with five further acquisition functions using the same GP surrogate and candidate sets(Fig. 6), an ablation with more candidate generators (App. G.5), a high-dimensional study up to 200 dimensions (Sec. 4.3, App. H), a comparison of the probabilities predicted by the network with Monte Carlo estimates (new App. I), an analysis of its exploration-exploitation decisions (Sec. 4.4, new App. J), and an evaluation on functions outside the pre-training priors (new App. K).

Weakness (W1) and Questions (Q1, Q2): "The supervised objective is not directly aligned with the objective of Bayesian optimization. [...] Are there theoretical conditions under which this decision rule minimizes one-step simple regret? [...] Can the proposed supervised learning framework accommodate alternative optimization objectives?"

We agree that the proposed loss does not correspond to a standard BO objective, as it does not explicitly model the size of an improvement or the value of information. In practice, however, the network makes accurate exploration and exploitation decisions in-context, and learns to trade uncertainty against predicted value similarly to acquisition functions such as UCB and Thompson sampling (App. J, see also our reply to Reviewer zdFT, W2/Q2). In addition, when given the same GP surrogate and candidates, IQS-BO-GP outperforms acquisition functions that model these quantities explicitly, such as the knowledge gradient and max-value entropy search (Fig. 6).

Regarding the theoretical conditions, the network targets the Bayes-optimal decision for identifying the best candidate, whereas expected improvement is the Bayes-optimal query for one-step simple regret. A sufficient condition for the two rules to coincide is that the posteriors at the candidates are independent and differ only in location, in which case both select the candidate with the highest posterior mean. They can differ when the uncertainty varies between candidates, since expected improvement also weighs the size of a possible improvement.

Finally, the proposed framework can accommodate alternative objectives, since the sampled function is known at every candidate during pre-training and any quantity computed from its values can serve as a training target. We tested two alternatives that include the values of the candidates in the loss. In the first, the network predicts the improvement of each candidate, learning to approximate expected improvement in-context. In the second, it is trained with soft labels, which weight each candidate by its value. In both cases, the network trained with the proposed loss performs better (Sec. 4.4, Table 1).

Weakness (W2): "The theoretical contribution is relatively limited. [...] More substantive analysis would be valuable, such as characterizing the relationship between posterior-best-candidate probability and expected improvement, or identifying conditions under which the proposed decision rule is effective for minimizing optimization regret."

We agree with this remark. Proposition 1 follows from the cross-entropy being a proper scoring rule, and its role is only to state precisely what the network learns. The revised paper complements it in three ways.

First, it states more clearly the limits of the result, as independent scores cannot represent the posterior exactly in general (App. D). Second, it validates the accuracy of the approximations of the network on functions drawn from a GP, where the posterior probabilities can be computed almost exactly. Although Monte Carlo has access to the correct GP, the network is as accurate without any fine-tuning, and more accurate at small context sizes once fine-tuned on functions from the same GP (App. I.1, see also our replies to Reviewer zdFT, W1/Q1). Third, it shows how the proposition extends to batch optimisation.

Regarding the two suggested additional analyses, the relationship with expected improvement was already discussed in our reply to W1. For optimisation regret, we do not have a theoretical guarantee, and identifying conditions under which the decision rule minimises it remains an open question. Our evidence on this point is only empirical, since IQS-BO reaches a regret lower than or similar to that of the considered baselines across a variety of dimensions, batch sizes and optimisation problems.

Weakness (W3) and Question (Q3): "Could the authors compare IQS-BO against stronger candidate-based acquisition optimization methods using exactly the same candidate sets, including alternative acquisition functions such as UCB or Thompson sampling? It would also be useful to evaluate performance when candidate sets are generated independently of the observed function values."

As requested, we added a comparison of IQS-BO-GP with PI, UCB, Thompson sampling, the knowledge gradient and max-value entropy search, all using the same GP surrogate and evaluated on the same candidate sets (Fig. 6). IQS-BO-GP outperforms all of them, with ties only against UCB at small context sizes (see also our reply to Reviewer zdFT, Q5).

We also evaluated candidate sets generated independently of the observed values, with Sobol and uniform points only. In this case, IQS-BO-GP and every acquisition function lose performance, since space-filling candidates rarely fall close enough to the best observations. The need for perturbations of the best observations when proposing queries from a finite set is shared by all selection rules, which cannot otherwise propose accurate exploitative queries. Finally, we validated that IQS-BO-GP is robust to how the candidates are generated. We ran the same model, without retraining, with twelve further candidate generators, each varying one aspect of the candidate set used in the paper. On all sets that keep small perturbations of the best observations, its performance remains close to that with the default set (Fig. 13).

Weakness (W4) and Question (Q4): "Could the authors provide results on problems with 10, 20, or more dimensions? In particular, how do performance and computational cost change as the dimension increases while the candidate-set size remains fixed?"

The revised paper now includes a high-dimensional study on ten problems at d=20, 50, 100 and 200, against state-of-the-art methods such as GIT-BO, TabPFN-EI and TuRBO (see Sec. 4.3 and App. H for more details). As the dimension increases, we observe that a larger candidate set contributes more to a lower regret, and we use a set of 2,048 candidates in all runs.

Regarding performance, the regret increases with the dimension for every method, as expected. IQS-BO and IQS-BO-TabPFN nevertheless achieve the two best mean ranks at every dimension, with a lower regret than GP-MAP, TuRBO and GIT-BO and a regret similar to that of TabPFN-EI. Their margin over the baselines is largest at d=20 and 50, and narrows at d=200.

Regarding computational cost, the time to propose a round stays almost constant from d=20 to d=200, since a forward pass depends only on the number of observations and candidates considered. Including more candidates has a small effect on cost, because candidates attend only to the observations and the cost of a forward pass grows only linearly with their number. Overall, IQS-BO proposes queries one to two orders of magnitude faster than the surrogate-based baselines. In batch optimisation at d=50, IQS-BO also has the best GAP on more problems than any baseline (Fig. 5), and proposes its queries 80 to 90 times faster than TabPFN-EI.

Weakness (W5): "A stronger evaluation would examine whether a single trained model can transfer across dimensions, batch sizes, or candidate-generation schemes."

We agree that one model per dimension and batch size is the main practical limitation of the approach. A single model transfers well across candidate-generation schemes (see our reply to W3/Q3) and across sizes of the candidate set (Table 1). Across batch sizes, however, it does not. The surrogate-free model pre-trained at q=2 loses much of its performance when used at q=50. IQS-BO-PFN degrades much less, which we attribute to its surrogate being conditioned on the points already selected in the batch, as well as to the large scope of its PFN surrogate (App. G.2). A model is also tied to its dimension, since the coordinates of each token enter its first linear layer. An embedding that does not depend on the dimension, such as one token per input dimension as in TabPFN, could make a single model applicable to any dimension, though at a considerably larger pre-training cost and possibly a lower accuracy.

Our comparison with TabPFN-EI supports this direction. Although TabPFN is a general purpose tabular model pre-trained at a far larger scale and for longer, IQS-BO outperforms TabPFN-EI in 67 to 72% of paired runs in low dimensions and ranks ahead of it in high dimensions, while proposing queries one to two orders of magnitude faster (Sec. 4.3). This indicates that pre-training specific to BO can bring better performance at a lower pre-training cost, which motivates, as future work, a large foundation model specific to black-box optimisation, pre-trained once across dimensions and batch sizes, and capable of fully in-context adaptation to an even larger set of optimisation problems.

Weakness (W6): "The paper does not establish whether this training procedure approximates a well-defined Bayes-optimal batch decision rule."

We thank the reviewer for this remark. The revised paper now explicitly extends Proposition 1 to the batch loss (App. D). For each point of the batch, the network learns the posterior probability that a candidate is the best among those not yet selected. This target does not depend on how the earlier points of the batch were drawn in pre-training, and the training procedure learns a well-defined quantity.

The resulting batch rule is greedy. At each step, as in the single-query case, it makes the Bayes-optimal decision for identifying the best remaining candidate. It does not optimise a joint criterion over all points of the batch, such as choosing the batch with the highest expected best value. In our experiments, this sequential rule performs better than the alternative of taking the q highest-scoring candidates in a single forward pass (Table 1).

Weakness (W7): "A comparison of total computational cost across different numbers of optimization tasks would help quantify when amortization becomes worthwhile."

As for other pre-trained models such as PFNs4BO and TabPFN, pre-training costs a few hours on GPUs. In return, every run is cheaper. Since it avoids both surrogate fitting and the maximisation of an acquisition function, IQS-BO proposes queries one to two orders of magnitude faster than all the considered surrogate-based baselines and, unlike several learned methods, requires no re-training or fine-tuning for different optimisation tasks. Counting proposal time alone, pre-training is repaid after roughly 1,500 to 3,500 runs in low dimensions, and after 100 to 900 runs at d≥20d\ge20, where all models are slower. The pre-training cost is therefore easily amortised in settings where an optimisation task must be repeated many times, or in high-throughput optimisation. The advantage in terms of speed is even larger in BO with large batches.

However, in real-world settings, where evaluations are expensive and data are scarce, the main saving often comes from limiting the number of evaluations and not from proposal time. Since it avoids specifying a kernel and fitting hyperparameters, which can be inaccurate with little data, IQS-BO can reach an optimum in fewer BO iterations and save resources required for evaluation.

Weakness (W8): "The robustness of the method under prior misspecification remains unclear. [...] a more systematic study of prior misspecification would be useful."

We thank the reviewer for the suggestion. The new App. K now provides a study of the out-of-prior performance of the model (see also our replies to Reviewer SGpx). We first measure the smoothness and the number of local optima of the functions implied by the two priors and of every test function, and find that most of our benchmarks lie outside the HEBO prior. We then evaluate both models, without retraining, on the 24 BBOB functions and on twelve standard functions chosen to lie outside both priors. The mixture-prior model keeps its advantage on both sets, and the HEBO-prior model, whose prior contains almost none of these functions, obtains a performance similar to standard BO with GPs. This shows that IQS-BO successfully adapts in-context to functions with characteristics different from those seen in pre-training. The mixture prior improves performance further and helps the network better generalise to a broader range of problems.

We hope that these additions address the reviewer's concerns, and we would be glad to answer any further questions.

Reviewer_SGpxOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a method for Bayesian Optimization using pre-trained neural networks. The idea is to train an attention based neural network model which predicts the index of the best candidate given a set of historical evaluations and a set of candidate points. This is an example of in-context learning where a model learns from examples provided in the context and can make predictions directly without any re-training.

A transformer architecture is used with the historical data attending to each other and the candidates attend only to the historical data. This allows the model to scale to any number of candidates and not be restricted to a fixed size. The final probability of each candidate being the best among the candidate set is the output of a softmax function. The model is trained on synthetic data generated from a variety of priors.

Evaluation on benchmark datasets shows improved performance compared to other baselines.

Critical strengths
  • The proposed method is interesting in the sense that it is a step towards general in-context black box optimizers. The idea of having one general pre-trained network is a promising step towards zero shot generalization similar to other in-context methods such as LLMs themselves.
  • In-context BO is an active area of research, however to the best of my knowledge, this particular approach of predicting the index of the best candidate is a novel paradigm.
  • The method is scalable to any number of candidate datapoints without any re-training. This can be attributed to the specific attention structure in the transformer architecture. The idea is relatively simple yet seems to be effective.
Critical weaknesses

The similarity and improvement over several other missing related work is unclear.

Using transformers for sequential decision making in general is a well studied area and several references are missing. In particular, tabular foundational models can make predictions on general tabular data and can be used to model black box functions without fitting a surrogate model. Such baselines are missing.

Missing related work (LLM summarized):

  • ExPT: Synthetic Pretraining for Few-Shot Experimental Design — Tung Nguyen, Sudhanshu Agrawal, Aditya Grover (NeurIPS 2023). Pretrains a Transformer on diverse synthetic functions and, at test time, conditions on a few observed (x,y)(x,y) examples to directly generate candidate optima for a new black-box function. Probably the closest missing citation conceptually. NeurIPS Proceedings

  • Learning to Learn without Gradient Descent by Gradient Descent — Yutian Chen, Matthew W. Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P. Lillicrap, Matt Botvinick, Nando de Freitas (ICML 2017). Trains an RNN optimizer on synthetic functions so that it can directly choose evaluation points for unseen derivative-free black-box functions, learning an exploration–exploitation strategy through its training objective. Proceedings of Machine Learning Research

  • Towards Learning Universal Hyperparameter Optimizers with Transformers (OptFormer) — Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Richard Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc’Aurelio Ranzato, Sagi Perel, Nando de Freitas (NeurIPS 2022). Trains a Transformer on large collections of historical hyperparameter-optimization trajectories to jointly model the objective function and an optimization policy that proposes new configurations. NeurIPS Papers

  • ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization — Jamison Meindl, Yunsheng Tian, Tony Cui, Veronika Thost, Zhang-Wei Hong, Johannes Dürholt, Jie Chen, Wojciech Matusik, Mina Konaković Luković (2025). Pretrains an optimization policy using offline RL on millions of synthetic GP functions and trajectories generated by multiple BO algorithms, then performs black-box optimization zero-shot on unseen functions. arXiv

  • Reinforced In-Context Black-Box Optimization (RIBBO) — Lei Song, Chen-Xiao Gao, Ke Xue, Chenyang Wu, Dong Li, Jianye Hao, Zongzhang Zhang, Chao Qian (IJCAI 2025). Learns an entire black-box optimization algorithm from offline optimization trajectories, using a Transformer-like in-context policy to directly propose subsequent evaluations from the optimization history. IJCAI

  • In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization — Herilalaina Rakotoarison, Steven Adriaensen, Neeratyoy Mallik, Samir Garibov, Eddie Bergman, Frank Hutter (ICML 2024). Introduces a pretrained PFN surrogate that performs learning-curve inference entirely in context and uses it inside freeze-thaw BO, avoiding online retraining of the surrogate. Proceedings of Machine Learning Research

  • GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models — Rosen Yu, Cyril Picard, Faez Ahmed (ICLR 2026). Uses a pretrained TabPFN foundation model as a zero-shot BO surrogate and exploits gradients of its predictions to construct useful low-dimensional search subspaces before applying an acquisition function. ICLR Proceedings

  • Pre-trained Gaussian Processes for Bayesian Optimization (HyperBO) — Zi Wang, George E. Dahl, Kevin Swersky, Chansoo Lee, Zachary Nado, Justin Gilmer, Jasper Snoek, Zoubin Ghahramani (JMLR 2024; earlier preprint 2021). Learns a GP prior from data collected on related optimization tasks and transfers that pretrained prior to new BO problems, rather than fitting the prior from scratch for each task. Journal of Machine Learning Research

  • Few-Shot Bayesian Optimization with Deep Kernel Surrogates — Martin Wistuba, Josif Grabocka (ICLR 2021). Meta-learns a shared deep-kernel GP surrogate across previous optimization tasks so that it can adapt to a new black-box objective using only a few observations. IBM Research

  • Predicting from Strings: Language Model Embeddings for Bayesian Optimization — Tung Nguyen, Qiuyi Zhang, Bangding Yang, Chansoo Lee, Jörg Bornschein, Yingjie Miao, Sagi Perel, Yutian Chen, Xingyou Song (2024). Uses pretrained language-model embeddings together with in-context regression to build a general-purpose surrogate for BO over heterogeneous/string-described search spaces.

Questions for authors
  • Can you shed more light on the pre-training distribution? Is it pre-trained on distributions with similar smoothness as the evaluation datasets.
  • Can a network trained on inputs sampled with a specific smoothness generalize to inputs with a different smoothness?
2 comments

We thank the reviewer for the useful comments and questions, which we address individually. The revised paper adds the requested baselines, with TabPFN-EI and ZeroShotOpt in the main results (Sec. 4.2), GIT-BO in the new high-dimensional study (Sec. 4.3), and four further learned methods in an additional comparison (App. A.5). In addition, a new appendix measures the smoothness and the number of local optima of the pre-training priors and of the test functions, and evaluates IQS-BO more clearly on functions outside its priors (App. K). Overall, the new experiments strengthen our original results. IQS-BO matches or outperforms every added baseline in up to 200 dimensions, and performs well on functions outside its pre-training priors.

Weakness (W1): "The similarity and improvement over several other missing related work is unclear. Using transformers for sequential decision making in general is a well studied area and several references are missing. In particular, tabular foundational models can make predictions on general tabular data and can be used to model black box functions without fitting a surrogate model. Such baselines are missing."

We agree that the positioning of our method, especially against tabular foundation models and other transformer-based BO methods, was incomplete. We have extended the related work to cover the suggested methods and to clarify how IQS-BO differs from them (App. A).

To our knowledge, IQS-BO is the only method that learns the query decision of BO by supervised learning on a synthetic prior. Amortised surrogates such as PFNs4BO, TabPFN and GIT-BO learn a posterior predictive distribution in-context, but require an acquisition function to be specified and maximised on their own outputs. IQS-BO instead learns the query decision directly, and selects queries in a single forward pass. Other transformer-based optimisers also propose queries directly, but are usually trained with reinforcement learning on solved tasks or on the trajectories of other optimisers. In contrast, IQS-BO requires only a synthetic prior to provide targets during pre-training, and adapts in-context to a variety of optimisation tasks without retraining or fine-tuning.

In our experiments, IQS-BO outperforms BO using TabPFN v2 as surrogate in low dimensions, ranks better in high dimensions, and proposes queries one to two orders of magnitude faster. It also outperforms GIT-BO, with a lower final regret at every dimension from 20 to 200 and a higher GAP in batch optimisation (Sec. 4.3). We also included a comparison with methods that learn to propose queries directly by reinforcement learning (NAP, ZeroShotOpt), backpropagation through synthetic functions (RNN-Opt) or conditional generative modelling (ExPT), or that meta-learn a GP surrogate (FSBO). IQS-BO performs best among them, with the best average rank and the lowest regret in the considered dimensions (App. A.5).

Question (Q1): "Can you shed more light on the pre-training distribution? Is it pre-trained on distributions with similar smoothness as the evaluation datasets?."

As requested, we measured the smoothness and the number of local optima of functions drawn from each prior and of every test function (App. K.1, Fig. 19). The HEBO prior draws GP paths with a single Matérn-3/2 kernel and covers only a narrow band of smoothness, outside which lie almost all of the benchmarks considered in the paper. The mixture prior is instead deliberately broad. Besides stationary GP paths, from rough (Matérn-1/2) to very smooth (squared-exponential), it contains input-warped paths, sums of narrow Gaussian bumps and piecewise-flat functions with sharp steps. These shapes are highly uncommon or even impossible under GP prior sampling. We included them because a model pre-trained on GP draws can only read a plateau or a sharp step as noise or as a very short lengthscale, with the idea that pre-training on a richer set of functions could improve the network's generalisation. As a result, the prior covers a wider range of smoothness, which contains every considered benchmark except Ackley. Our results show that this improves performance on a variety of optimisation tasks, at almost no cost on exact GP draws. Importantly, the performance of IQS-BO does not depend on smoothness coverage, as the same models perform well on functions chosen to be outside both priors, as discussed in the reply to your second question.

Question (Q2): "Can a network trained on inputs sampled with a specific smoothness generalize to inputs with a different smoothness?"

Yes. Beyond our benchmarks, which already lie mostly outside the HEBO prior (see Q1), we evaluated IQS-BO, without retraining, on two further test sets (App. K.2, Fig. 20): the 24 BBOB functions, which range from discontinuous to very smooth, and twelve standard functions that lie outside both priors in terms of smoothness, number of local optima, or both. The mixture-prior model improves over standard BO with a GP surrogate (GP-MAP) in 60 to 74% of runs on both sets, more often than any other pre-trained method, including TabPFN-EI. The HEBO-prior model, whose prior contains almost none of these functions in terms of smoothness, performs similarly to GP-MAP.

These results show that IQS-BO can achieve strong BO performance on functions whose smoothness lies outside its pre-training prior, and is capable of successfully adapting to a broad range of optimisation problems in-context. The mixture prior, which by design covers a larger range of smoothness, improves performance further and helps the network generalise better to a wider range of functions.

We hope that these additions address the reviewer's concerns, and we would be glad to answer any further questions.

Reviewer_Go1kOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper suggests an in-context query selection process for Bayesian optimization over a continuous space by training a transformer instead of the classical way of maximizing an acquisition function at every round and refitting a surrogate model. The paper does this by running a pretrained scoring network on a finite candidate pool. This converts the continuous optimization problem into a finite class classification problem. This network is trained on two synthetic priors: the HEBO prior (from PFNs4BO-HEBO model) and the mixture prior (proposed in the paper). Mixture prior's warped, bump, and plateau families decide which landscapes it can handle. These broader training prior and local candidate proposals make the proposal cost low while giving the desired optimization results.

Critical strengths
  1. The labeling method is simple and avoids finding each training function's global optimizer labels and policy gradient training. This is making the learning amortized across many tasks.
  2. The measured proposal-time advantage is substantial from the reported configurations.
Critical weaknesses
  1. Finding the best label only tells us which candidate has the largest value. It is not clear how it is related to the most valuable candidate from the reducing the optimization regret or informing the future decisions perspective.
  2. The paper has an independent candidate architecture whereas the posterior winner probabilities are generally dependent on competing candidates and their correlations. The paper doesn't show the accuracy of posterior approximation as claimed. In my opinion, a targeted posterior validation experiment will strengthen the contribution.
Questions for authors
  1. The paper mentions that the context selection is being done adaptively but never specifies exactly how it is done. It'll be good to know this detail and the insight behind this choice. Is there distribution shift between those context and self-generated context during the deployment? If so, how is it addressed?
  2. The paper reports the contributions from different factors together. Adding some experiments for separating out the effects of the learned model from the effects of candidate sampler will give a clearer picture.
2 comments

We thank the reviewer for the positive assessment and for the valuable questions and observations, which we address individually below.

Weakness (W1): "Finding the best label only tells us which candidate has the largest value. It is not clear how it is related to the most valuable candidate from the reducing the optimization regret or informing the future decisions perspective."

While the proposed loss does not explicitly model the size of an improvement or the value of information, IQS-BO learns to adjust its exploration-exploitation decisions directly from the context (Sec. 4.4).

The network is trained to predict the probability that each candidate is the best within a finite set. With few observations, this probability is spread over unexplored regions, and the network explores. As observations accumulate, it concentrates near the best observations, and the network exploits. To quantify this shift and compare the behaviour of the network with that of standard acquisition functions, we measured the weight that the queries of each method put on uncertainty relative to predicted value (App. J, see also our reply to Reviewer zdFT, W2/Q2). For IQS-BO, this weight is large with few observations and falls to almost zero as the context grows, similarly to Thompson sampling and UCB.

We also tested whether including improvement magnitude in the loss could improve performance. To do so, we pre-trained the network to predict the improvement of each candidate over the best observation so far, so that it approximates expected improvement. This variant performs similarly to GP-MAP with expected improvement, but IQS-BO with the proposed loss outperforms it in 84 to 87% of paired runs. We didn't test a loss that explicitly includes information gain, though we expect a similar outcome. In fact, when given the same GP surrogate and evaluated on the same candidate sets, IQS-BO-GP outperforms the knowledge gradient and max-value entropy search, which model the size of an improvement and the value of information, respectively (Fig. 6).

Weakness (W2): "The paper has an independent candidate architecture whereas the posterior winner probabilities are generally dependent on competing candidates and their correlations. The paper doesn't show the accuracy of posterior approximation as claimed. In my opinion, a targeted posterior validation experiment will strengthen the contribution."

We thank the reviewer for the suggestion, which we have implemented in the revised paper. As correctly noted, the scores of the candidates are computed independently, but the predicted probabilities are not, since the softmax normalises each of them over the whole candidate set. Although the exact posterior probabilities cannot be represented in general, the network approximates them well, as now validated in our experiments.

The new App. I validates the accuracy of the probabilities predicted by the network in two settings. First, on functions drawn from a GP, where the GP posterior is available in closed form and the posterior probabilities can be approximated almost exactly. We compare the predictions of the network with these probabilities and with Monte Carlo estimates from a GP with the same kernel and mean function as the data-generating process. Then, on real benchmarks, where no exact posterior is available, we compare the queries of IQS-BO with those selected by Monte Carlo with GPs fitted to the data (see also our replies to Reviewer zdFT, W1/Q1 and W5/Q6, for more details).

In both settings, the network accurately approximates the posterior probability that each candidate is the best. In the first, the network places its probability on the same candidates as the exact posterior (Fig. 15). Although only Monte Carlo has access to the correct GP, the network is as accurate without any fine-tuning, and more accurate once fine-tuned on functions from the same GP, especially at small contexts. In the second, the queries of IQS-BO are as good as or better than those selected by Monte Carlo, at a smaller proposal cost, and without the need to specify a kernel or fit hyperparameters to the data.

Question (Q1): "The paper mentions that the context selection is being done adaptively but never specifies exactly how it is done. It'll be good to know this detail and the insight behind this choice. Is there distribution shift between those context and self-generated context during the deployment? If so, how is it addressed?"

We thank the reviewer for this remark. App. C.2 now specifies the procedure. Each pre-training context imitates a BO run on the sampled function. After an initial Sobol design, points are added sequentially. Each new point is either the next Sobol point or, with a probability that rises as the context grows, a Gaussian perturbation of an earlier point xix_i, selected with probability ∝exp⁡(f(xi)/T)\propto\exp(f(x_i)/T) to favour good observations. The temperature TT sets how strongly good observations are favoured, and is drawn at random for each context.

The insight is that, at deployment, the network reads data produced by BO, which typically looks like a space-filling start followed by points that concentrate increasingly near good observations. We train the network on contexts that mix the two types of points in varying proportions, from only Sobol points to almost only perturbations. Empirically, this allows the network to perform well on contexts made of its own queries, and with candidate sets generated differently from those of pre-training (App. G.5, see also our reply to Reviewer zdFT, W3/Q3).

We also tried pre-training the same network on contexts made only of Sobol points. This model is outperformed by the original one in most paired runs, supporting the idea that pre-training on contexts that resemble BO runs helps the network make more accurate decisions during real optimisation tasks.

Question (Q2): "The paper reports the contributions from different factors together. Adding some experiments for separating out the effects of the learned model from the effects of candidate sampler will give a clearer picture."

We thank the reviewer for this suggestion. The revised paper now separates the two contributions with two experiments.

First, we fix the candidate sampler and change only the selection. We use the same GP surrogate as IQS-BO-GP and run BO on the same candidate sets, with one of six standard acquisition functions in place of the network (Fig. 6). IQS-BO-GP outperforms all six, with ties only against UCB at small context sizes. Since the surrogate and the candidate sets are identical, the only difference is how the query is selected, and the improvement can be attributed to the selection learned by the network.

Second, we vary the candidate sampler and repeat the comparison on each of them. We run the same network, without retraining, and five acquisition functions on candidate sets obtained with twelve different generators (Fig. 13). On every candidate set, IQS-BO-GP has the lowest mean final regret of all selection rules. This shows that the advantage of the learned selection does not depend on a specific candidate sampler, and that the accuracy of the network extends to contexts different from those seen at pre-training. Different candidate generators, in turn, affect all selection rules in a similar way, since the performance of IQS-BO-GP and of the five acquisition functions rises or falls together. In particular, all of them rely on perturbations of the best observations to achieve good performance, since space-filling points rarely fall close enough to the best observations and therefore do not allow any selection rule to exploit accurately.

In summary, the choice of candidate sampler affects all selection rules in a similar way, and IQS-BO-GP outperforms standard acquisition functions on all the considered generators. This isolates the performance gain, which can be attributed to the selection rule learned by the network and not to a specific type of candidate set, as long as such set contains small perturbations of the best observations.

We hope that these additions address the reviewer's concerns, and we would be glad to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.