acceptodds
Under review as a conference paper at ICLR 2027

Outcome-Ranked Proposal Tuning: Training LLMs to Transfer Experience Across Bayesian Optimization Tasks

Abstract

Bayesian optimization (BO) is widely used for expensive black-box optimization. When related optimization problems arise repeatedly, experience from previous tasks can improve optimization on new ones. We aim to train a large language model (LLM) to transfer the knowledge from the prior tasks to new ones through BO initialization. We introduce Outcome-Ranked Proposal Tuning (ORPT), which trains an LLM to propose initialization points based on how they affect the subsequent BO process. ORPT trains the LLM to favor initialization candidates based on their effect on subsequent BO, rather than their standalone objective values. We further mathematically show that these learned preferences are linearly connected to the expected outcome of downstream BO. Experiments on synthetic Branin optimization and antimicrobial peptide design show that ORPT outperforms a broad range of single-task, multi-task, and LLM-based BO baselines, yielding stronger initializations and better downstream optimization. These gains persist through subsequent BO, while ablations confirm that downstream BO outcomes provide a stronger training signal than candidate objective values alone. Overall, our results show that learned initialization is an effective mechanism for transferring experience across related optimization tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Models, Preference Optimization, BO Initialization, Bayesian Optimization.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qQQAOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper aims to use an LLM to transfer knowledge from prior tasks to new ones for initialisation of the new BO task. Their hypothesis is that a roll-out on a down stream task provides a better prior for transfer. The LLM is trained to propose points based on how they affect a downstream BO task. They also prove the relationship between learned objectives and the rollout BO utility. The validation is done on one synthetic function(but creating related tasks by varying task parameter) and for peptide design.

Critical strengths

The idea of improving BO initialisation through roll out on the downstream BO task is interesting.

Critical weaknesses

• One question is the additional cost of 2M*H extra calls to the oracle for the rollout. The comparison between BOLT should be on equal budget – that is rollout evaluations to spend on more more BOLT BO runs. As far as I can see this has not been demonstrated. • Bolt’s results already showed that the best initial point was a good proxy and beat the baselines. To show this extra work is worth it, the gain must be demonstrated – in this case, almost same in synthetic, and better but close in peptide, but only 1 case is demonstrated. Does this hold in general? It is not clear. • The policy learns what helps a particular BO procedure with this rollout. If the BO procedure changes or horizon changes, this will change, and robustness across operators and horizons is not demonstrated. • The results are minimal – demonstrated on one synthetic, and one real world case.

Questions for authors

No specific questions, other than the issues raised above.

3 comments

We thank reviewer qQQA for their constructive reviews and recognizing our work as (1) interesting idea of improving BO initialisation through roll out on the downstream BO task. We have addressed your concerns below.

W1. Additional oracle cost

One question is the additional cost of 2M*H extra calls to the oracle for the rollout. The comparison between BOLT should be on equal budget – that is rollout evaluations to spend on more BOLT BO runs. As far as I can see this has not been demonstrated.

We clarify that ORPT's additional oracle cost is incurred during training, not during optimization of held-out tasks. Our contribution is to learn better BO initializers from completed tasks using rollout-informed preferences, rather than to reduce the total training-time oracle budget.

We additionally report the computational cost for OPRT. For peptide optimization, the oracle costs through 900 training tasks are:

Oracle evaluationsBOLTORPT
Training BO trajectories18.90M18.90M
Preference candidate scoring–7.09M
Preference BO rollouts–34.04M
Held-out evaluation1.05M1.05M
Total19.95M61.08M

ORPT incurs additional cost but achieves better initialization (6.54 vs. 8.57 µM) and final BO performance (5.93 vs. 7.15 µM).

These results demonstrate the benefit of additional training-time supervision for learned BO initialization. We agree that allocating the same additional training budget to BOLT is an important efficiency comparison, but it addresses a different question from the initialization quality studied here. We have clarified this distinction and report the additional training costs explicitly in the revision (Appendix D).

W2/W4. Additional experiments on synthetic and a real world dataset

Bolt’s results already showed that the best initial point was a good proxy and beat the baselines. To show this extra work is worth it, the gain must be demonstrated – in this case, almost same in synthetic, and better but close in peptide, but only 1 case is demonstrated. Does this hold in general? It is not clear. / The results are minimal – demonstrated on one synthetic, and one real world case.

The submission covered Branin and peptide design. To test whether the benefit extends beyond those settings, we have added molecular lead optimization and two synthetic task families in rebuttal. Unlike the submitted Branin family, the optimum in our three-dimensional quadratic family moves with the task parameter. We have also evaluated a mixed 53-dimensional Ackley family with a task-dependent objective as

New settingHeld-out evaluationBOLTORPT
Quadratic: 30 training, 10 test tasks5 initial points + 100 GP-UCB calls; final score ↑20.93121.002
Ackley-53: 50 training, 20 test tasks5 initial points + 50 GP-UCB calls; final score ↑0.2790.335
Molecular: 900 training, 50 test tasks with disjoint target proteins1,000 initial candidates + 5,000 LOL-BO calls; predicted affinity ↑6.6326.674

ORPT improves the mean final outcome in all three added settings. The quadratic result addresses the fixed-optimum limitation of Branin; the molecular result extends the evaluation to a second application domain with target-dependent objectives. We have added the experiment details and results in Table 1, 2 at Section 4.1 of the revision.

W3. Robustness to the rolling horizon H and BO operator

The policy learns what helps a particular BO procedure with this rollout. If the BO procedure changes or horizon changes, this will change, and robustness across operators and horizons is not demonstrated.

Rolling horizon H. Our Branin ablation in our paper (see Table 3) shows that ORPT achieves better final BO performance than the H=0H=0 variant across the positive rollout horizons tested. This suggests that the benefit of rollout supervision is robust to the choice of HH within the tested range. The ablation holds the BO operator fixed, so these results concern sensitivity to HH under the same operator.

BO operator. We agree that the preferences learned by ORPT depend on the BO operator used during training. However, this does not necessarily restrict their usefulness to that operator.

Let O1\mathcal O_1 denote the BO operator used to construct training preferences and O2\mathcal O_2 a different operator used for downstream BO on a held-out task. Following Section 3.5, write Qi(x)=QH,πrefOi(C,x)Q_i(x)=Q_{H,\pi_{\mathrm{ref}}}^{\mathcal O_i}(C,x) for their marginal H-rollout utilities.

Under the population preference model, training with O1\mathcal O_1 favors candidates with larger Q1(x), corresponding to the proposal reweighting

πλ(x∣C)∝πref(x∣C)exp⁡(λQ1(x)).\pi_\lambda(x\mid C)\propto\pi_{\mathrm{ref}}(x\mid C)\exp(\lambda Q_1(x)).

Differentiating at the reference proposal at λ=0\lambda=0 gives Equation (15) in the manuscript.

ddλlog⁡πλ(x∣C)(λ=0)=Q1(x)−Eπref[Q1]\frac{d}{d\lambda}\log\pi_{\lambda}(x\mid C) (\lambda=0)=Q_1(x)-\mathbb E_{\pi_{\mathrm{ref}}}[Q_1]

Under the same population assumptions as in Section 3.5, the first-order change in expected pool utility at λ=0\lambda=0 when downstream BO uses O2\mathcal{O}_2 is

ddλJHO2(πλ;C)(λ=0)=mpref Eπref[Q2(x)(Q1(x)−Eπref[Q1])]=mprefCovπref(Q1,Q2).\frac{d}{d\lambda}J_H^{\mathcal O_2}(\pi_\lambda;C) (\lambda=0)=m_{\mathrm{pref}}\,\mathbb E_{\pi_{\mathrm{ref}}}\left[Q_2(x)\left(Q_1(x)-\mathbb E_{\pi_{\mathrm{ref}}}[Q_1]\right)\right]=m_{\mathrm{pref}}Cov_{\pi_{\mathrm{ref}}}(Q_1,Q_2).

Thus, if the two operators assign positively correlated marginal utilities to initialization candidates, the proposal update favored by O1\mathcal O_1 also improves the expected pool utility under O2\mathcal O_2, to first order.

We expect related GP-based BO operators to benefit from similar initialization candidates, although this condition has not been verified empirically across operators. Our result provides a sufficient condition rather than a guarantee for arbitrary BO operators. We have clarified this viewpoint in the revised manuscript (Appendix E).

I thank the responses to W1-W4. Thank you for clarifying the budget issue. Regarding the real utility with respect to demonstration, I still find the current experiments weak. I retain the original score

Reviewer_e7BfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces ORPT, a method for proposing better initialization points in a multi-task Bayesian optimization setting, extending BOLT (Zeng et al., 2026) by adding direct preference optimization to prefer, between two candidates xi and xj, the one that leads to a better result after a single BO step when added to otherwise identical initial pools. As in BOLT, they first fine-tune the LLM on the best points from completed BO trajectories, so that it proposes candidates with high objective values. However, as the authors claim, this does not guarantee a good optimization trajectory, which is why they then train the LLM to propose candidates that help the subsequent BO. They also show, under idealized assumptions, that this improves the expected quality of the sampled initial pools for small updates. They validate the approach on a synthetic Branin task and on peptide design against BOLT, single-task BO, multi-task GP baselines, OptFormer and LLAMBO, including scaling with the number of training tasks.

Critical strengths

Being able to transfer knowledge from one optimization problem to another is an important issue in multi-task Bayesian optimization. The paper motivates this problem well and offers an interesting idea: a candidate in the initial pool should be judged by how it helps the BO that follows, not only by its own objective value. Comparing matched pools that differ in one candidate keeps the LLM generating one candidate at a time, which is practical.

Critical weaknesses

The central premise is not tested, and the tasks don't test transfer.

The method rests on the claim that a candidate's value as a starting point differs from its standalone value. The paper never shows how often these disagree. The only evidence is the H = 0 vs H ≥ 1 ablation (Table 2), which is run only on Branin. But in Branin, changing t does not move the optimum. For every t < 1 the minimizers are the same three points, so a model that learns one location can solve every held-out task. ORPT's five LLM proposals alone already score −4.75, the same as its final BO result. The construction is attributed to Volpp et al. and Perrone et al., but MetaBO translates and scales Branin (§5, p. 7), and Perrone et al. use quadratics, not Branin (§4.2, p. 6). In both, the optimum moves across tasks. Additionally, in peptides: every task shares the same objective (the same MIC predictor against the same organism), so memorizing what the oracle rewards is a real alternative explanation, and it is never tested. With Branin unable to separate the methods, the evidence rests on a single domain, whose tasks share one objective.

The comparisons don't isolate the contribution.

BOLT is the only like-for-like comparison (same LOLBO, only the starting pool changes). There the results are mixed: on peptides ORPT is better at 252, 378 and 630 tasks but worse at 126 (9.33 vs 8.80) and at 900, the largest checkpoint (5.81 vs 5.68); on Branin the two are indistinguishable. The gain is on the criterion the paper argues against. ORPT's clearest advantage is in the quality of the initial samples before BO (Table 3, Fig. 3b), which is the standalone criterion BOLT already targets. After BO, the criterion ORPT is designed for, the advantage is inconsistent. Since ORPT is BOLT plus an extra DPO stage, part of any gain may simply come from the additional training.

  • Budgets aren't matched. Building the preferences uses extra oracle calls on the training tasks (App. C.3.2), which are not reported.
  • The other baselines don't test the initialization claim. They change both the starting pool (random) and the surrogate. The transfer surrogates do worse than single-task BO on peptides, POGPE and SGPE use at most 20 of the training tasks, and no non-LLM initializer using the same training data is included.
  • The budget doesn't match the motivation. The paper motivates BO by a limited evaluation budget, but 1000 initial points plus 20000 oracle calls is not that regime.
  • No measure of run-to-run noise. The ± values are the spread across held-out tasks, with one run per task and no paired tests.
Questions for authors

Clarifications:

  • The premise. For the retained preference pairs, what fraction of rollout labels disagree with the ordering by standalone objective value? This can be computed from the existing pairs and would show directly whether the method learns something BOLT's signal does not.
  • Budget. How many oracle calls and how much compute does preference construction use, including the acquisition batch size in the rollouts? How do ORPT and BOLT compare at a matched total budget?
  • Diversity. Does ORPT reduce the diversity of the initial pool compared with BOLT (number of unique sequences, pairwise distances)? That would explain better best-of-k samples without better final BO.
  • Peptides. How far is each test seed from its nearest training seed, and does any training task's best peptide already satisfy a test seed's 75% similarity constraint? Testing for memorization could, for example, report how close generated candidates are to training-trajectory sequences and how performance changes with the test seed’s distance to its nearest training seed.
  • Details. Please state K_sft, how pairs are selected (Algorithm 1, line 6), the baselines' "common domain-specific initialization", the Branin domain and t values, the number of runs per task. Code would resolve most of this.

Suggestions:

  • Evaluation. Replace the Branin family with one where the optimum moves across tasks (e.g. MetaBO-style translations), and add a domain where the objective itself changes across tasks (e.g. BOLT’s query-optimization domain, or peptides against a different organism).
  • Compare initializers, not pipelines. Fix LOLBO and vary only the starting pool: random, frozen base LLM, best points from the nearest training tasks, data-driven warm start (FSBO-style), BOLT and ORPT, charging each for all oracle calls.
  • Test the premise on peptides. Run H = 0 vs H ≥ 1 there, and build labels from pools the same size as at test time. If H = 0 matches ORPT, the gain comes from the additional DPO stage rather than from BO-outcome labels; a BOLT baseline with matched extra training would also help separate the two.
  • Report results at small budgets (around 100–200 oracle calls), where initialization should matter most.
3 comments

We thank the Reviewer e7Bf for their constructive feedback, and recognizing our work as (1) transfer knowledge from one optimization problem to another as an important issue (2) offers an interesting idea: a candidate in the initial pool should be judged by how it helps the BO (3) keeps the LLM generating one candidate at a time, which is practical.

W1-1/W2-1/Q1/S3. The premise and the contribution isolation

The method rests on the claim that a candidate's value as a starting point differs from its standalone value. The paper never shows how often these disagree. The only evidence is the H = 0 vs H ≥ 1 ablation (Table 2), which is run only on Branin.

To examine the contribution of rollout-informed preference construction, we've compared H=0 and H=1 on peptide optimization. With H=0, candidates are ranked by standalone MIC. The original (H=1) criterion used only the outcomes of matched initialization pools after one BO acquisition round and yielded limited gains on peptides. We therefore revised H=1 to combine standalone predicted MIC with these rollout outcomes, favoring candidates that perform well individually and lead to strong BO outcomes.

Training tasksH=0 final MIC (µM)H=1 final MIC (µM)
1268.428.79
2527.577.52
3787.017.01
6307.146.31
9006.985.79

As shown, at 900 training tasks, H=1 achieves a final MIC of 5.79 µM, compared with 6.98 µM for H=0. Consistent with the Branin horizon ablation (Table 2 in the submission), this result suggests that the rollout-informed preference construction improves downstream BO at large training scales. See Table 3 (Effect of BO rollout supervision) in section 4 for details including ablations with peptide, and synthetic functions including ackley-53, and quadratic function.

We have also revised the peptide experiments since the original submission, including the preference data construction. The results above are from these updated experiments, and we have reported the complete experimental settings and results in the revision (see section 4.1, appendix C (Table 5, 6)). We also have toned down our original claim and revised the manuscript that rollout outcomes can complement standalone scores in method [introduction: line 59-60, method: line 146-147].

W1-2/S1. Evaluation on varying synthetic tasks

But in Branin, changing it does not move the optimum. For every t < 1 the minimizers are the same three points, so a model that learns one location can solve every held-out task. ORPT's five LLM proposals alone already score −4.75, the same as its final BO result. The construction is attributed to Volpp et al. and Perrone et al., but MetaBO translates and scales Branin (§5, p. 7), and Perrone et al. use quadratics, not Branin (§4.2, p. 6). In both, the optimum moves across tasks.

As Reviewer e7Bf pointed out, the submitted Branin family changes with tt but retains the same minimizer locations. We have therefore added two synthetic experiments that test transfer across more substantial task changes: Quadratic function, and Ackley-53.

Quadratic function: ft(x)=−[1+t2∑i=13xi2+(0.1+9.9t)∑i=13xi+(0.1+9.9t)]f_t(\mathbf x)=-\left[\frac{1+t}{2}\sum_{i=1}^{3}x_i^2 + (0.1+9.9t)\sum_{i=1}^{3}x_i +(0.1+9.9t) \right]

Ackley-53: $f_t(\mathbf{x})=-\left[-20\exp!\left(-0.2\sqrt{\frac{1}{d}\sum_{i=1}^{d}x_i^2}\right) -\exp!\left(\frac{1}{d}\sum_{i=1}^{d}\cos(2\pi x_i)\right)+20+e+0.2\sum_{i=1}^{50}\cos!\left(2\pi\left(t+\frac{i-1}{50}\right)\right)b_i\right]$

New task familyTask variationTraining / held-out tasksEvaluationBOLTORPT
QuadraticThe exact minimizer moves with t30 / 105 initial + 100 BO calls20.931 ± 19.51721.002 ± 19.543
Ackley-53A t-dependent term changes the objective over 50 binary variables50 / 205 initial + 50 BO calls0.279 ± 0.1420.335 ± 0.141

As shown in the table, ORPT improves final BO performance on both the quadratic and Ackley-53 benchmarks, where the optimum varies across tasks. These results demonstrate that its benefits extend beyond Branin’s fixed minimizer setting. We have added new experiment results in the revised manuscript (see Section 4 (Table 1: Synthetic optimization; Ackley-53, Quadratic)).

W2-2/Q2. Budget of oracle calls and compute

How many oracle calls and how much compute does preference construction use, including the acquisition batch size in the rollout?

For peptide design, we used 5000 oracle calls, for synthetic function (Branin) we used 50 oracle calls. For acquisition batch size for rollout, we used 50 for peptide, and 1 synthetic function.

We report the computes required for preference construction at peptide:

Oracle evaluationsBOLT-900ORPT-900 (8 shared base set, H=1)
Training BO trajectories18.90M18.90M
Preference candidate scoring07.09M
Preference BO rollouts034.04M
Held-out initialization and BO1.05M1.05M
Total19.95M61.08M

As shown, preference construction uses 7.09M + 34.04M = 41.13M. We have clarified this distinction and report the additional training costs explicitly in the revision (Appendix D).

W2-3/S2. Compare initializers on Quadratic

The other baselines don't test the initialization claim. They change both the starting pool (random) and the surrogate. The transfer surrogates do worse than single-task BO on peptides, POGPE and SGPE use at most 20 of the training tasks, and no non-LLM initializer using the same training data is included.

We clarify that our method’s contribution is to tackle the multi-task Bayesian Optimization through learning to propose an initialization, rather than tackling the surrogate model for multi-task BO.

To this end, Table 1 in the submission includes the comparison with multi-task BO baselines: methods tackling the surrogate (SGPE, POGPE), method tackling the initializer (BOLT, ORPT). We acknowledge your point that our comparison lacks the initializer itself.

For this, we have added initializer baseline experiments at Quadratic family on [−10,10]3[-10,10]^3, including Random Init, Best points from the nearest training points, and data-driven warmstart (FSBO), BOLT, and ORPT.

Random samples five points uniformly; Training Best selects the five best distinct points from training trajectories; FSBO uses a deep-kernel GP trained on full training tasks (30 tasks, in this setting) and conducts a genetic algorithm over initialization pools that jointly maximize predicted performance across training tasks. Finally, subsequent BO uses the same procedure as the other initializers.

InitializerInitial simple regret ↓Final simple regret ↓
Random35.282 ± 29.0370.204 ± 0.170
Training best8.536 ± 9.9350.211 ± 0.133
FSBO0.286 ± 0.1920.151 ± 0.102
BOLT0.329 ± 0.2610.172 ± 0.126
ORPT0.155 ± 0.1570.101 ± 0.099

With the downstream BO procedure fixed, these results provide direct evidence that ORPT’s learned initialization improves optimization performance relative to the evaluated alternatives, including non-LLM warm-start methods. We have added the detailed experiment setup and the results at revision (see Section 4.6 (Table 4)).

Q3. Diversity of peptides

Does ORPT reduce the diversity of the initial pool compared with BOLT?

First of all, all accepted sequences in the initial pool are unique under the shared deduplication and feasibility rules, so there aren't single, repeated samples in the initial pool.

Furthermore, we measured diversity in the peptide initialization pools of ORPT-900 and BOLT-900. For each task, we averaged pairwise Levenshtein distances normalized by the reference length, then averaged across the 50 held-out tasks.

MeasureBOLTORPT
Mean normalized pairwise distance0.31640.3344
Mean edit distance from the reference3.523.81
Candidates at the feasibility boundary73.0%95.8%

We found that, ORPT’s mean pairwise distance is higher by 0.0180, and is higher than that of BOLT on 42 of 50 tasks. Thus, ORPT does not show the reduction in initial-pool diversity. (Note that ORPT also achieves a lower final MIC after BO.) We have added those results in the revised manuscript (see Table 10 in Appendix C.6.3).

W1-3/Q4. Memorization likelihood in peptides experiment

In peptides: every task shares the same objective (the same MIC predictor against the same organism), so memorizing what the oracle rewards is a real alternative explanation, and it is never tested. With Branin unable to separate the methods, the evidence rests on a single domain, whose tasks share one objective.

We first clarify that ORPT's held-out performance cannot be explained by simply reusing training solutions. For this, we have examined whether peptides from training tasks satisfy the 75% similarity constraint of held-out tasks and whether such peptides could account for ORPT's initialization performance. We have reported this result at Table 11 in Appendix C.6.4.

Each held-out reference is far from every training reference. The nearest training reference is at median similarity 0.667 (median edit distance 6). The closest case across all 50 held-out seeds reaches 0.750, and none exceeds it. Furthermore, only 0.03% of peptides from training trajectories satisfy the constraint of at least one held-out task, covering 27 of the 50 tasks. To assess whether these reusable peptides could explain ORPT's performance, we compared their best MIC against ORPT's initial proposals on those 27 tasks. ORPT achieves a median MIC of 5.7 µM, compared with 72.0 µM for the best reusable training peptides, and outperforms them on all 27 tasks (median paired improvement of 8.8×). Thus, even when training peptides are feasible on held-out tasks, directly reusing them does not explain ORPT's initialization performance.

Q5. Experimental details

Details. Please state K_sft, how pairs are selected (Algorithm 1, line 6), the baselines' "common domain-specific initialization", the Branin domain and t values, the number of runs per task. Code would resolve most of this. We report the detailed experiment setups below.

  1. KsftK_{\mathrm{sft}}. Peptide: 1,000; Branin: 20.

  2. Pair selection (Algorithm 1, line 6). For example, consider peptide experiment, We (1) uniformly sample up to 24 eligible candidates per task, (2) enumerate all candidate pairs (up to 276 to save compute), (3) evaluate each candidate using shared initialization pools (M=8 for peptide), (4) retain pairs with nonzero mean rollout differences and reliability z1, and (5) select up to 30 pairs with the highest reliability scores. In the revised peptide experiment, step (4) additionally requires the rollout-preferred candidate to have a better standalone objective value.

  3. Common domain-specific initialization. (1) Peptide baselines (STBO, MTBO, POGPE, SGPE, OptFormer) share the same pool of 1,000 randomly mutated peptides satisfying the 75% similarity constraint. (2) Branin baselines use five uniformly sampled points from the two-dimensional domain: x1∈[−5,10],x2∈[0,15]x_1 \in [-5, 10], x_2 \in [0,15].

  4. Branin settings. Search domain: ([-5,10]\times[0,15]). Training: (t_i=0.005+0.02i), (i=0,\ldots,49). Evaluation: (t_j=0.05j), (j=0,\ldots,19). Each task uses 5 initial evaluations and 50 additional BO evaluations.

  5. Independent runs. One training run per method and one BO run per held-out task at each checkpoint. Reported standard deviations are across tasks (50 peptide; 20 Branin), not random seeds.

  6. Code. Code will be released upon acceptance.

We have added each baseline’s training hyperparameters (Table 5 in Appendix C.2), initialization procedure (Appendix C.5), and domain-specific implementation details. (Appendix C.1) in the revised manuscript.

Reviewer_Z9RROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ORPT, which trains an LLM to generate initial candidates for Bayesian optimization (BO). The LLM is first fine-tuned on good candidates from previous BO tasks. ORPT then compares two initial candidate pools that differ in only one candidate. It runs BO from each pool and uses the results to decide which candidate is preferred. These preferences are used to further train the LLM with DPO. On a new task, the LLM only provides the initial candidates; BO then continues without further LLM queries. The paper also provides a theoretical explanation for why this approach can locally improve expected BO performance under idealized assumptions. Experiments on Branin functions and AMP optimization report better initialization and better final results at several training-tasks. The AMP results are based on predicted MIC by the AI model APEX.

Critical strengths
  1. A well-motivated training signal for learned initialization. The paper makes a useful distinction: a candidate with a good score is not necessarily the most helpful candidate for subsequent BO. It compares two initial pools that differ in only one candidate, then uses the BO results to decide which candidate is preferred. This allows the model to learn from the performance of the whole pool while still generating candidates one at a time. I consider this the paper’s main methodological strength. The theory also helps explain this approach under certain assumptions. However, it only supports improvement for small changes near the reference model; it does not guarantee that the full training procedure will improve BO performance.
  2. A useful comparison with BOLT at initialization and after BO. BOLT is a useful baseline because it uses the same LLM and the same BO procedure as ORPT. The paper compares both the initialization quality and the final BO results using different numbers of training tasks. Table 3 shows that ORPT achieves better average initialization scores in all tested AMP settings, although its final BO results are not always better. Evaluating both stages helps show where ORPT provides benefits, rather than looking only at the final score.
Critical weaknesses
  1. The contribution of BO feedback and the value of the added complexity remain unclear. Compared with BOLT, ORPT adds preference training. However, the AMP experiments lack a control that uses the same training but ranks candidates by their own scores rather than downstream BO results. The H=0 ablation is reported only for Branin. Thus, it is unclear how much of the AMP improvement comes specifically from BO feedback rather than additional preference training. The paper also lacks a smaller learned proposer using the same historical data, preference objective, candidate budget, and downstream BO. Such a comparison would help assess whether the LLM offers an advantage over simpler generator. The authors should also clarify how their STBO baseline relates to ApexGO, particularly in template adaptation and initialization. The extra costs of preference-data rollouts, fine-tuning, and candidate generation are not clearly reported.
  2. The AMP benchmark needs clearer specification and stronger evidence that the gains are reliable. Section 4.1 specifies a sequence-similarity threshold of 0.75, but the manuscript does not clearly identify the reference-sequence source, peptide-length ranges, allowed amino acids and sequence edits, or the similarity metric. The MIC predictor, checkpoint, and output definition are also unclear, and no representative prompt–candidate example is provided. These details are needed to reproduce the benchmark and check whether all methods use the same constraints. The similarity between training and test reference peptides should also be reported to assess how well the method generalizes across tasks. The reported standard deviations reflect variation across test tasks, not repeated runs or uncertainty in the differences between methods. ORPT’s final results are also not consistently better: BOLT has a lower mean MIC at the 126-task and 900-task checkpoints. Paired comparisons and repeated runs would help establish whether the improvements are reliable. The current results alone do not establish either statistical significance or its absence.
Questions for authors
  1. AMP benchmark specification. Could you provide the exact reference-sequence dataset and split identifiers, reference/candidate length constraints, permitted sequence edits, and similarity calculation? Please also identify the MIC predictor/checkpoint, the pathogen-specific output or aggregation being optimized, and any output transformation. A precise reusable configuration and a representative prompt–output example would help make the benchmark reproducible.
  2. Generalization and robustness. How were the 900 training tasks and 50 held-out tasks constructed, and what is the sequence-similarity relationship between their reference peptides? How many independent training and optimization runs were performed? Please report paired performance differences on the same test tasks, with uncertainty intervals, so that the consistency of the gains can be assessed.
  3. Contribution of BO rollout supervision. Can you compare ORPT with objective-ranked preference tuning (H = 0) on the AMP tasks, using the same model, training data, and optimization settings? This would help isolate the contribution of downstream BO feedback. It would also be useful to compare runs with similar best initial objective values, to assess whether ORPT improves subsequent BO beyond providing a better starting point.
  4. Simpler proposers and additional cost. Have you evaluated a smaller task-conditioned proposal model such as peptide generative model using the same historical data, initialization size, and downstream BO procedure? What are the incremental oracle and computational costs of preference construction, fine-tuning, and generation, separately from the shared historical trajectory costs? This would help me judge whether the performance gains justify the added complexity.
2 comments

We thank Reviewer Z9RR for their constructive feedback, and recognizing our work as: (1) well motivated training signal for BO initialization (2) useful comparison with BOLT at initialization.

We have addressed your concerns below.

Weakness 1

W1-1/Q3. Contribution/Effect of BO feedback

However, the AMP experiments lack a control that uses the same training but ranks candidates by their own scores rather than downstream BO results. The H=0 ablation is reported only for Branin.

To show the effect of BO rollout supervision on real-world data, we conducted a full-scale ablation study on peptide. Here, H=0H=0 ranks candidates by their standalone MIC, whereas H=1H=1 constructs preferences based on the outcomes of one BO acquisition round from matched initialization pools.

Notice that, in the submitted version, H=1H=1 determined preferences solely from BO rollout outcomes. In the revised version, we additionally consider standalone objective quality to encourage proposals that perform well both at initialization and during subsequent BO. Specifically, we've retained only preference pairs whose rollout-based rankings agree with their standalone objective rankings.

Training tasksBOLTORPT (H=0)ORPT (H=1)
1269.568.428.79
2529.387.577.52
3788.157.017.01
6307.547.146.31
9006.556.985.79
( lower is better.)

After being tied at 378, H=1H=1 shows a trend achieving lower final MICs than H=0H=0 on the larger training tasks. The comparison therefore supports a benefit from the H=1H=1 preference-construction procedure at larger training scales. We have added these results to the revised manuscript with the additional ablation result with synthetic functions including ackley-53, quadratic (Table 3), and softened our original claim and revised the manuscript to emphasize that rollout outcomes can complement standalone scores [introduction: line 59-60, method: line 146-147].

W1-2/Q4. Additional computational complexity of ORPT

The extra costs of preference-data rollouts, fine-tuning, and candidate generation are not clearly reported.

We acknowledge your point that ORPT requires additional complexity to learn the preference of downstream BO, while our introduction of our submission suggested an efficiency contribution. We revised the manuscript to state our contribution more precisely: ORPT studies whether preferences based on downstream BO outcomes can train a better initializer for unseen tasks. We have toned down the claim that ORPT reduces total oracle evaluations or computes in introduction. (Introduction: line 52-55)

Oracle evaluations of ORPT. To put ORPT’s additional oracle cost in context, we report its evaluation counts alongside those of BOLT for the peptide experiments through the 900-task checkpoint.

Peptide experiment stageBOLTORPT
Training BO trajectories18.90M18.90M
Preference candidates07.09M
Preference BO rollouts034.04M
Held-out initialization and BO1.05M1.05M
Accounted total19.95M61.08M

Preference construction accounts for 41.13M additional oracle evaluations than BOLT. Nevertheless, ORPT achieves stronger initialization quality with fewer completed training tasks (Figure 2), highlighting the benefit of rollout supervision despite its higher training cost. We have provided detailed cost accounting in Appendix D.

W1-3. Comparison with other proposal methods

The paper also lacks a smaller learned proposer using the same historical data, preference objective, candidate budget, and downstream BO.

As requested, we have added initializer baseline experiments including Random Init, Best points from the nearest training points (Training Best), and data-driven warmstart (FSBO), BOLT, and ORPT at three-dimensional Quadratic function.

Random samples five points uniformly; Training Best selects the five best distinct points from training trajectories; FSBO uses a deep-kernel GP trained on full training tasks (30 tasks, in this setting) and conducts a genetic algorithm over initialization pools that jointly maximize predicted performance across training tasks.

We have added the detailed experiment setup at revision (Section 4.6, Table 4).

InitializerInitial simple regret ↓Final simple regret ↓
Random35.282 ± 29.0370.204 ± 0.170
Training best8.536 ± 9.9350.211 ± 0.133
FSBO0.286 ± 0.1920.151 ± 0.102
BOLT0.329 ± 0.2610.172 ± 0.126
ORPT0.155 ± 0.1570.101 ± 0.099

With the downstream BO procedure fixed, these results provide direct evidence that ORPT’s learned initialization improves optimization performance relative to the evaluated alternatives, including non-LLM warm-start methods.

W1-4/W2-1/Q1. AMP benchmark specification

The AMP benchmark needs clearer specification and stronger evidence that the gains are reliable.

In response, we have provided the requested AMP benchmark (peptide) specifications below. We have also revised the peptide experiments since the original submission, including the preference data construction. We have reported the complete experimental settings and results in the revision (see section 4.1, appendix C (Table 5, 6)).

(1) Exact reference-sequence dataset and split identifiers

We use BOLT’s library of 1,000 extinct reference peptides. Tasks 0–899 are used for training, and tasks 900–949 are the fixed 50-task evaluation set, drawn from the library’s 100 held-out tasks.

(2) Peptide length constraints, and similarity calculation

Peptide lengths are 10–25 residues overall and 12–25 in the evaluation set. Candidate sequences use the 20 canonical amino acids (A,C,D,E,F,G,H,I,K,L,M,N,P,Q,R,S,T,V,W,Y).

There is no separate candidate-length constraint. A candidate is feasible when

1−dLev(x,s)∣s∣≥0.75,1-\frac{d_{\mathrm{Lev}}(x,s)}{|s|}\geq 0.75,

where s is the task's reference sequence, and dLev is the Levenshtein distance.

(3) Identification of MIC predictor / checkpoint

The MIC predictor is an eight-checkpoint APEX ensemble, and the eight predictions are averaged on the MIC scale. The eight serialized model files are in https://github.com/Yimeng-Zeng/APEXGo/tree/main/optimization/apex_oracle/APEX_pathogen_models.

We use only the first of the model’s 11 pathogen-specific outputs, corresponding to A. baumannii ATCC 19606.

(4) Representative prompt-output example

The instruction for LLM initializer of ORPT for peptide ILAGFLLSNNIL is:

system: You are a specialized assistant that modifies peptide sequences to enhance antimicrobial activity.
Make up to 25% sequence modifications based on known antimicrobial peptide properties such as: positive charge, hydrophobicity, and amphipathicity.

user: ILAGFLLSNNIL  

assistant: 

Example output of ORPT-900 for this peptide was RLAKFLRLSNNIL (Levenshtein distance 3 from the reference; similarity 0.75).

Weakness 2

W2-1 (Refer to Response W1-4/W2-1/Q1).

W2-2/Q2. Generalization and robustness

How were the 900 training tasks and 50 held-out tasks constructed?

Training and held-out task construction is detailed in the response to Q1.

What is the sequence-similarity relationship between their reference peptides?

The median nearest-training-reference similarity is 0.667, with a range of 0.357–0.750. Seven of the 50 evaluation references have a training reference exactly at the 0.75 threshold, and none has one above it. A training reference is therefore itself feasible on those seven tasks only.

How many independent training and optimization runs were performed?

We ran one training chain for BOLT and one for ORPT, followed by one BO evaluation per held-out task for each method. We have added this procedure in the revised manuscript at training reporting in Section 4.1.

Comments on the paper

1 comment

Discussion Summary

We thank all reviewers and the Area Chair for their time and constructive feedback, which have helped us strengthen our work.

We appreciate the reviewers’ recognition of the following strengths of our work:

  • Well-motivated transfer problem: Transferring experience across optimization tasks through learned BO initialization. (Reviewer e7Bf)
  • Outcome-based training signal: Evaluating initialization candidates by their contribution to downstream BO. (Reviewers Z9RR, e7Bf, qQQA)
  • Practical preference construction: Comparing matched initialization pools that differ in one candidate while retaining candidate-by-candidate generation. (Reviewers Z9RR, e7Bf)
  • Informative baseline comparison: Comparing ORPT with BOLT at both initialization and after BO across different training scales. (Reviewer Z9RR)

During the discussion period, we added the following experiments and analyses:

  • BO rollout supervision (H) ablation Compared ORPT (H=0) with the ORPT (H=1) on peptide optimization, showing better final MIC at larger training scales. We also extended this ablation to Quadratic and Ackley-53 tasks. (Reviewers Z9RR, e7Bf)
  • Additional synthetic benchmarks: Conducted experiments on Quadratic tasks with moving optima and Ackley-53 showing higher mean final outcomes than BOLT in both settings. (Reviewers e7Bf, qQQA)
  • Additional real world benchmarks: Molecular lead optimization: Added a second application domain with disjoint training and evaluation target proteins, showing a higher mean final predicted affinity than BOLT. (Reviewers e7Bf, qQQA)
  • Controlled initializer comparisons experiment: Compared diverse initialization stretegies: Random, Training Best, FSBO, BOLT, and ORPT with the same downstream BO procedure on Quadratic tasks. ORPT achieved the lowest mean initial and final simple regret. (Reviewers Z9RR, e7Bf)
  • Peptide diversity and memorization analysis: Measured initialization-pool diversity, finding higher mean pairwise sequence distances for ORPT than BOLT on 42 of 50 held-out tasks. (Reviewer e7Bf). Also examined whether feasible training peptides could explain held-out performance. ORPT’s initial proposals outperformed the best reusable training peptides on all 27 tasks where reuse was feasible. (Reviewers Z9RR, e7Bf)

We also provided the following clarifications and revisions:

  • Additional Cost of ORPT: Reported training-trajectory, preference-construction, and held-out evaluation costs separately, and revised our efficiency claims to acknowledge ORPT’s additional training cost. (Reviewers Z9RR, e7Bf, qQQA)
  • Revised preference construction: Updated the peptide experiments to retain preference pairs whose rollout-based rankings agree with standalone objective rankings, and refined our claim to emphasize their complementary value. (Reviewers Z9RR, e7Bf)
  • AMP benchmark specification: Detailed task splits, sequence constraints, similarity calculations, the APEX predictor ensemble, the optimized pathogen output, and a representative prompt–output example. (Reviewers Z9RR, e7Bf)
  • Experimental reproducibility: Clarified training settings, preference-pair selection, baseline initialization, BO budgets, and run counts, including that reported standard deviations measure variation across tasks. (Reviewers Z9RR, e7Bf)
  • Rollout horizon and BO operator dependence: Discussed the existing horizon ablation and added a theoretical sufficient condition for transfer across BO operators, while clarifying the scope of this result. (Reviewer qQQA)

We believe these additional experiments and clarifications have helped address the reviewers’ concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.