- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes IQS-BO, an in-context Bayesian optimisation method that selects the next evaluation point from a finite candidate set using a pretrained Transformer. The method is trained on synthetic functions to predict which candidate has the highest objective value, using a supervised classification loss. It can operate without a surrogate model or use predictions from a pretrained PFN or a Gaussian process. The paper also introduces a mixture prior for pretraining and evaluates the method on synthetic and real-world problems, including batch optimisation.
Critical strengths
- The paper presents an interesting approach to learning query selection through supervised training on synthetic functions, without requiring reinforcement learning.
- The experiments include several relevant BO baselines, controlled comparisons using the same candidate sets, and evaluations of batch optimisation. The results show competitive optimisation performance and lower query-selection time in the tested settings.
Critical weaknesses
- The probability that a candidate is the best can also be estimated using Monte Carlo sampling from a probabilistic surrogate's joint posterior. The paper does not compare IQS-BO with this approach in terms of optimisation performance and computational cost.
- The training objective only identifies the best candidate, without considering improvement magnitude or information gain. It is unclear how this objective balances exploration and exploitation over multiple BO iterations.
- The experiments show that performance depends strongly on candidate generation, particularly local perturbations around the best observations. It is unclear whether the results generalise to other candidate sampling strategies.
- The experiments only consider three- and four-dimensional problems, with batch experiments restricted to three dimensions. More experiments on higher-dimensional problems are needed to evaluate the scalability of IQS-BO.
- In Equation (6), each candidate is assigned a score independently of the other candidates before applying softmax. However, the probability of being the best generally depends on the joint posterior distribution over all candidates. It is unclear whether the proposed architecture can accurately represent this probability.
Questions for authors
- How does IQS-BO compare with directly estimating the probability of being the best candidate using Monte Carlo posterior sampling?
- How does the proposed training objective balance exploration and exploitation without explicitly considering improvement magnitude or information gain?
- How well does IQS-BO perform with other candidate generation strategies, given the sensitivity shown in Table 2 and Appendix F.5?
- How would IQS-BO perform on higher-dimensional optimisation problems?
- The proposed method can be viewed as a learned candidate-scoring rule. Beyond the comparisons with EI, what advantages does it offer over acquisition functions such as PI, UCB, and KG?
- Given that each candidate is scored independently before softmax, can the proposed architecture accurately represent the posterior probability that a candidate is the best among all candidates?
5 comments
Weakness (W2) and Question (Q2): "The training objective only identifies the best candidate, without considering improvement magnitude or information gain. It is unclear how this objective balances exploration and exploitation over multiple BO iterations. [...] How does the proposed training objective balance exploration and exploitation without explicitly considering improvement magnitude or information gain?"
While the proposed loss does not explicitly model information gain or use estimates of uncertainty, the network learns to adjust its exploration-exploitation decisions directly from the context. Importantly, these decisions remain accurate on functions outside the pre-training priors (App. K, see also our reply to Reviewer SGpx).
The network is trained to predict the probability that each candidate is the best within a set. With few observations, this probability is spread over unexplored regions, and the network explores. As observations accumulate, it concentrates near the best observations, and the network exploits. To quantify this shift from exploration to exploitation, and to compare it with acquisition functions, we measured the weight that the queries of each method put on uncertainty relative to predicted value (App. J). For IQS-BO, this weight is large with few observations and falls to almost zero as the context grows. Thompson sampling and UCB behave similarly, whereas expected improvement and the knowledge gradient keep a slowly declining weight and explore more later in a run.
In our experiments, including improvement magnitude in the loss did not improve performance. To test this, we pre-trained IQS-BO to predict the improvement of each candidate over the best observation so far. This variant performs similarly to GP-MAP with expected improvement, but IQS-BO with the proposed loss outperforms it in 84 to 87% of paired runs (Table 1). In addition, evaluated using the same GP surrogate and candidates, IQS-BO-GP outperforms expected improvement and the knowledge gradient, as well as max-value entropy search, which considers information gain (Fig. 6).
Weakness (W3) and Question (Q3): "The experiments show that performance depends strongly on candidate generation, particularly local perturbations around the best observations. It is unclear whether the results generalise to other candidate sampling strategies. [...] How well does IQS-BO perform with other candidate generation strategies, given the sensitivity shown in Table 2 and Appendix F.5?"
Our ablations indicate that IQS-BO is robust to both the composition and the size of the candidate set, as long as the set keeps small perturbations of the best observations.
To test robustness to candidate composition, we ran the same IQS-BO model, without retraining, with twelve further candidate generators, each varying one aspect of the default set (App. G.5). To separate the effect of the candidate set from that of the learned selection, we also ran five standard acquisition functions on the same sets. On all sets that keep small perturbations of the best observations, IQS-BO maintains its performance, and has a lower mean final regret than all considered acquisition functions. When queries are selected from a finite set, this dependence on perturbations is not specific to IQS-BO. All acquisition functions degrade in performance without them because space-filling candidates rarely land close enough to the best observations, making exploitation decisions less accurate
Weakness (W4) and Question (Q4): "The experiments only consider three- and four-dimensional problems, with batch experiments restricted to three dimensions. More experiments on higher-dimensional problems are needed to evaluate the scalability of IQS-BO. [...] How would IQS-BO perform on higher-dimensional optimisation problems?"
The revised paper includes a new high-dimensional study (Sec. 4.3 and App. H). We use ten scalable problems at d=20, 50, 100 and 200, with 700 evaluations per run, and compare with state-of-the-art methods for high-dimensional BO: GIT-BO and TabPFN-EI, which rely on a tabular foundation model, and the trust-region method TuRBO. IQS-BO keeps its architecture and is pre-trained once for each dimension. We also evaluate IQS-BO-TabPFN, which reads a frozen TabPFN as surrogate.
IQS-BO scales well to high dimensions. The two IQS-BO models achieve the two best mean ranks at every dimension, with a lower regret than GP-MAP, TuRBO and GIT-BO and a regret similar to TabPFN-EI. IQS-BO is also faster, proposing a round in under 0.1 s, one to two orders of magnitude less than all surrogate-based baselines.
The same holds in batch optimisation at d=50 (Fig. 5). At every considered batch size (q=10, 20,50), IQS-BO has the best GAP on more problems than any baseline, and a mean GAP above that of GP-MAP, TuRBO and GIT-BO. Only TabPFN-EI reaches a slightly higher GAP, but takes 80 to 90 times longer to propose its queries.
Question (Q5): "The proposed method can be viewed as a learned candidate-scoring rule. Beyond the comparisons with EI, what advantages does it offer over acquisition functions such as PI, UCB, and KG?"
We have extended the controlled comparison of the paper from EI to PI, UCB, Thompson sampling, the knowledge gradient (KG) and max-value entropy search (Sec. 4.4). Each acquisition function uses the same GP surrogate as IQS-BO-GP and is evaluated on the same candidate sets, which isolates the contribution of the learned selection rule.
The results indicate three main advantages. First, the learned rule selects better queries: IQS-BO-GP reaches a lower final regret than every acquisition function except UCB in 61 to 79% of paired runs, at every context size. Against UCB, it wins 59 to 60% of runs with 40 initial observations and ties at small context sizes, for the two strongest values of , chosen after the runs. Second, IQS-BO has no exploration parameter to tune, since its weight on uncertainty adapts to the context (App. J). Third, IQS-BO does not require a probabilistic surrogate, and selects queries directly from the observations in a single forward pass.
Weakness (W5) and Question (Q6): "In Equation (6), each candidate is assigned a score independently of the other candidates before applying softmax. However, the probability of being the best generally depends on the joint posterior distribution over all candidates. It is unclear whether the proposed architecture can accurately represent this probability. [...] Given that each candidate is scored independently before softmax, can the proposed architecture accurately represent the posterior probability that a candidate is the best among all candidates?"
As correctly highlighted by the reviewer, independent scores cannot represent this posterior probability exactly in general; we have made this point clearer in the paper (see Sec. 3.2 and App. D). In practice, however, the architecture can accurately approximate it. Although each score is computed independently, the probabilities are not, since through the normalisation of the softmax the probability of each candidate depends on the scores of all the others. What the architecture cannot represent is that the odds of one candidate against another, , can change with the rest of the set. Adding a near-duplicate of a candidate, for example, splits its probability between the two copies, and so lowers its odds against every other candidate.
Proposition 1 shows that, for any network, the pre-training loss equals the expected entropy of the posterior, which no model can reduce, plus the expected KL divergence from the posterior to the network's output. The better the individual scores, the smaller this divergence. We measured this distance empirically, on tasks on which the exact posterior can be computed (see our reply to your first question (W1-Q1)). As detailed there, fine-tuning the network brings it closer to the exact posterior. This agrees with the decomposition, by which any reduction of the loss is a reduction of the distance to the posterior, and shows that independent scores are not the limit of the accuracy of the approximation.
We also tested training a model with self-attention among the candidates. This variant did not improve BO performance, but gives up two practical properties of independent scores. First, since candidates attend only to the observations, the cost of a forward pass grows linearly with the number of candidates, whereas attention among candidates makes it grow quadratically. Second, since each score depends on the whole set, sets larger than those seen in pre-training fall outside the training distribution, whereas independent scores let the size of the candidate set change between pre-training and deployment.
We hope that these additions address the reviewer's concerns, and we would be glad to answer any further questions.
We thank the reviewer for the analysis and for the accurate questions, which led to useful additional analyses and experiments. The revised paper now includes a comparison of the probabilities predicted by the network with Monte Carlo estimates (App. I), an analysis of its exploration-exploitation decisions ( App. J), a comparison with five further acquisition functions and more candidate generators (Sec. 4.4, App. G.5), and a high-dimensional study up to 200 dimensions (Sec. 4.3, App. H). Overall, these additions confirm that the network accurately approximates the posterior target, meaningfully adapts its exploration and exploitation to the context, is robust to variations in the candidate set and scales well to high dimensions.
Weakness (W1) and Question (Q1): "The probability that a candidate is the best can also be estimated using Monte Carlo sampling from a probabilistic surrogate's joint posterior. The paper does not compare IQS-BO with this approach in terms of optimisation performance and computational cost. [...] How does IQS-BO compare with directly estimating the probability of being the best candidate using Monte Carlo posterior sampling?"
We added this comparison in the new App. I. In short, IQS-BO estimates the probability that each candidate is the best about as accurately as Monte Carlo under a correctly specified GP (a GP that has the same mean and kernel functions used to generate the data, with only the hyperparameters fitted to the available context). On real problems, IQS-BO selects better queries than Monte Carlo with GP surrogates, and reaches better optimisation performance at a smaller cost.
To come to these conclusions, we investigated the following two routes:
1. Exact posterior. First, we considered a setting where the posterior distribution is available in closed form, and posterior probabilities for each candidate can be accurately estimated by Monte Carlo. We draw functions from a GP with a Matérn-3/2 kernel, 400 for each context size n=10, 20 and 80, integrate out the hyperparameters, and construct the posterior probability distribution of each candidate being the best within a set of 64 candidates generated from the data. We then compare these distributions with those implied by IQS-BO and by Monte Carlo sampling using a GP with the same kernel used to generate the data, but with hyperparameters fitted by MAP. This setting favours Monte Carlo, since its surrogate models the same distribution that generated the data. Without any fine-tuning, the network is about as accurate as Monte Carlo with GP-MAP, and more accurate with few observations, which determine the GP's hyperparameters poorly. Fine-tuned on tasks drawn from the process that generated the data, the network is closer to the exact probabilities than Monte Carlo with GP-MAP at every context size. This indicates that the gap in posterior approximation accuracy is not a limit of the architecture, and can be reduced with additional training.
2. Real optimisation tasks. Second, we compared the quality of the queries on more realistic optimisation tasks, where no posterior is available in closed form and Monte Carlo is limited by how well its surrogate is specified. On 600 BO runs on the 3D benchmarks, we compared the next query proposed by IQS-BO with that of two Monte Carlo variants. The first uses a GP with a fixed kernel, the same surrogate of IQS-BO-GP. The second uses a GP selected at every query among ten with different kernels, to fit each problem as closely as possible, which is expensive. The query of IQS-BO is the true best candidate in 26% of cases, against about 18% for Monte Carlo with the fixed GP. Monte Carlo with the selected GPs reaches the same query quality of IQS-BO only after several thousand draws, at about 50 times its cost per query.
Both experiments show that IQS-BO, trained with the proposed objective and priors, is capable of accurately estimating the probability that each candidate is the best in a single forward pass. It does so at a lower cost than Monte Carlo with well-specified GPs, and without requiring the selection of kernel and mean functions for each optimisation problem.