- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper aims to use an LLM to transfer knowledge from prior tasks to new ones for initialisation of the new BO task. Their hypothesis is that a roll-out on a down stream task provides a better prior for transfer. The LLM is trained to propose points based on how they affect a downstream BO task. They also prove the relationship between learned objectives and the rollout BO utility. The validation is done on one synthetic function(but creating related tasks by varying task parameter) and for peptide design.
Critical strengths
The idea of improving BO initialisation through roll out on the downstream BO task is interesting.
Critical weaknesses
• One question is the additional cost of 2M*H extra calls to the oracle for the rollout. The comparison between BOLT should be on equal budget – that is rollout evaluations to spend on more more BOLT BO runs. As far as I can see this has not been demonstrated. • Bolt’s results already showed that the best initial point was a good proxy and beat the baselines. To show this extra work is worth it, the gain must be demonstrated – in this case, almost same in synthetic, and better but close in peptide, but only 1 case is demonstrated. Does this hold in general? It is not clear. • The policy learns what helps a particular BO procedure with this rollout. If the BO procedure changes or horizon changes, this will change, and robustness across operators and horizons is not demonstrated. • The results are minimal – demonstrated on one synthetic, and one real world case.
Questions for authors
No specific questions, other than the issues raised above.
3 comments
W3. Robustness to the rolling horizon H and BO operator
The policy learns what helps a particular BO procedure with this rollout. If the BO procedure changes or horizon changes, this will change, and robustness across operators and horizons is not demonstrated.
Rolling horizon H. Our Branin ablation in our paper (see Table 3) shows that ORPT achieves better final BO performance than the variant across the positive rollout horizons tested. This suggests that the benefit of rollout supervision is robust to the choice of within the tested range. The ablation holds the BO operator fixed, so these results concern sensitivity to under the same operator.
BO operator. We agree that the preferences learned by ORPT depend on the BO operator used during training. However, this does not necessarily restrict their usefulness to that operator.
Let denote the BO operator used to construct training preferences and a different operator used for downstream BO on a held-out task. Following Section 3.5, write for their marginal H-rollout utilities.
Under the population preference model, training with favors candidates with larger Q1(x), corresponding to the proposal reweighting
Differentiating at the reference proposal at gives Equation (15) in the manuscript.
Under the same population assumptions as in Section 3.5, the first-order change in expected pool utility at when downstream BO uses is
Thus, if the two operators assign positively correlated marginal utilities to initialization candidates, the proposal update favored by also improves the expected pool utility under , to first order.
We expect related GP-based BO operators to benefit from similar initialization candidates, although this condition has not been verified empirically across operators. Our result provides a sufficient condition rather than a guarantee for arbitrary BO operators. We have clarified this viewpoint in the revised manuscript (Appendix E).
I thank the responses to W1-W4. Thank you for clarifying the budget issue. Regarding the real utility with respect to demonstration, I still find the current experiments weak. I retain the original score
We thank reviewer qQQA for their constructive reviews and recognizing our work as (1) interesting idea of improving BO initialisation through roll out on the downstream BO task. We have addressed your concerns below.
W1. Additional oracle cost
We clarify that ORPT's additional oracle cost is incurred during training, not during optimization of held-out tasks. Our contribution is to learn better BO initializers from completed tasks using rollout-informed preferences, rather than to reduce the total training-time oracle budget.
We additionally report the computational cost for OPRT. For peptide optimization, the oracle costs through 900 training tasks are:
ORPT incurs additional cost but achieves better initialization (6.54 vs. 8.57 µM) and final BO performance (5.93 vs. 7.15 µM).
These results demonstrate the benefit of additional training-time supervision for learned BO initialization. We agree that allocating the same additional training budget to BOLT is an important efficiency comparison, but it addresses a different question from the initialization quality studied here. We have clarified this distinction and report the additional training costs explicitly in the revision (Appendix D).
W2/W4. Additional experiments on synthetic and a real world dataset
The submission covered Branin and peptide design. To test whether the benefit extends beyond those settings, we have added molecular lead optimization and two synthetic task families in rebuttal. Unlike the submitted Branin family, the optimum in our three-dimensional quadratic family moves with the task parameter. We have also evaluated a mixed 53-dimensional Ackley family with a task-dependent objective as
ORPT improves the mean final outcome in all three added settings. The quadratic result addresses the fixed-optimum limitation of Branin; the molecular result extends the evaluation to a second application domain with target-dependent objectives. We have added the experiment details and results in Table 1, 2 at Section 4.1 of the revision.