acceptodds
Under review as a conference paper at ICLR 2027

Large Language Models as Generative Bayesian Policies

Abstract

Bayesian optimization is the dominant framework for sample-efficient scientific discovery, providing principled uncertainty estimates that guide expensive experiments. However, its reliance on hand-crafted, domain-specific representations limits broad applicability. Large language models (LLMs) offer flexibility as many experimental setups can be described in text, but they lack the calibrated uncertainty required in high-stakes scientific applications. Here we ask whether Bayesian optimization can supply the training signal for preference-based LLM alignment. We train a Gaussian process critic through deep kernel learning on the LLM's own embeddings. The same critic's acquisition function then provides a preference signal that trains the LLM to act as a generative Bayesian policy: candidates favored by the acquisition function become "preferred" completions for preference-based finetuning. The actor learns to generate high-utility candidates directly, thereby amortizing the acquisition search of standard Bayesian optimization into the language model itself. We evaluate on three scientific optimization benchmarks spanning reaction optimization, molecular and peptide design under a budget of  100 experiments. The actor's generation distribution shifts progressively toward higher-utility regions across iterations, and the trained model outperforms feature-based Bayesian optimization, LLM-guided optimization, and their union at different levels of integration. These results suggest that calibrated uncertainty can be the training signal that aligns language models with the decision-making demands of experimental discovery.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian optimization, large language models, preference optimization, deep kernel learning, Gaussian processes, scientific discovery, reaction optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6TYZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper leverages GP to construct preference supervision for fine-tuning LLM to generate high-quality candidates. It evaluates the approach on five scientific sequence optimization benchmarks.

Critical strengths
  1. The motivation is sound: using LLMs as generative Bayesian policies is a natural choice for sequence optimization.
  2. The method is reasonable: GP-derived supervision addresses the scarcity of data for LLM fine-tuning.
Critical weaknesses

Although the authors discuss and compare against ToSFiT, the two approaches appear similar in motivation and methodological design. The authors should clarify the advantages of their method over ToSFiT at the methodological level.

Questions for authors
  1. Could the authors provide a reproducibility statement?
  2. Could the authors evaluate the method’s sensitivity to prompts?
  3. Could the authors evaluate the method on SQL query optimization?
  4. Could the proposed algorithm support recursive self-improvement?
  5. Could the authors compare against LLM-assisted Bayesian optimization methods such as LLAMBO, LGBO, and LABO?

I would be willing to raise my score if the authors adequately address these concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6TYZ (1/4)

We thank the reviewer for a precise review. The ToSFiT question prompted two experiments, and the request for LLM-assisted BO baselines produced the largest addition to this revision: LLAMBO, LGBO and LABO are all implemented and run under our protocol.

The four key points.

  1. The ToSFiT row in our Table 1 gives it our deep-kernel critic and our LoRA adaptation in place of its own, so the comparison isolates the training signal. With that critic it stays at or below GOLLuM, which uses the same critic only to select among the untrained model's samples (W1).
  2. Isolated against its own candidate sampler, LLAMBO's in-context surrogate contributes nothing on GB1 (−0.037 to +0.034, every p ≥ 0.39) while contributing +0.580 (p = 0.0097) on Catechol. It helps on a small parameter space, not on a measured sequence landscape.
  3. LGBO's LLM prior changes the selected candidate in 0–15% of iterations and leaves the BO baseline it sits on statistically unmoved.
  4. A sixth benchmark, GB1, with exact-lookup rewards and a known optimum: 0.707 ± 0.035 against 0.605 ± 0.026 for classical BO.

Budgets are 100 BO iterations after a shared 10-point cold start; every number is absolute best-so-far and every ± a standard error over 20 seeds.

W1. Advantages over ToSFiT at the methodological level

ToSFiT is the closest prior work to ours and addresses the same problem, which is why we compare against it directly rather than only citing it. These details were distributed across a figure caption, the ablation section and the appendix; we consolidate them here for clarity. The paper adapts ToSFiT to the low-evaluation regime before comparing: same deep-kernel critic (Section 2), tuned over learning rate, optimizer and updates per iteration, sample-matched at 16 samples per iteration (Figure 1 caption and the ToSFiT appendix), with two controls already reported: four times the samples does not close the gap, and replacing its advantage with our UCB acquisition changes nothing. It also already reports the result that most directly answers your question: with our critic, ToSFiT stays at or below GOLLuM, the same critic used only to select among the untrained model's samples, with no generator training at all (2.17 vs 2.31 on Catechol, 0.62 vs 0.80 on SMILES-DRD2).

The ToSFiT in our Table 1 is therefore not ToSFiT as published. Every component except the training signal is ours:

componentToSFiT as publishedToSFiT in our Table 1
criticBayesian linear regression on mean-pooled frozen embeddings; kernel never adaptsour deep-kernel GP critic
actor updatefull fine-tuning with SGDLoRA, as for GGOLLuM
budget1,000 evaluations (their protein experiment)tuned and sample-matched to 100 BO iterations

What remains is how the GP posterior becomes a training signal. ToSFiT reduces it to a scalar pseudo-reward and follows that gradient; we construct pairwise preferences over the generated candidates and k measured anchors, and optimize IPO. The signal is therefore relative rather than a scalar read off the acquisition, and partly tied to points whose value is known rather than predicted: removing the anchors lets the policy collapse onto its own samples (pair-pool ablation).

New in this revision: neither substitution disadvantages ToSFiT. Replacing its published critic with our deep-kernel critic increases mean performance on all five benchmarks (everything else fixed, 20 seeds):

benchmark+ our deep-kernel critic+ frozen criticdifference
Catechol2.2061.014+1.19 (p = 1e-5)
SMILES-DRD20.6470.513+0.134 (p = 0.08)
SMILES-DRD2-Synth0.5990.555+0.044 (p = 0.64)
Peptide-Lytic0.5090.475+0.034 (p = 7e-5)
Peptide-AMP0.7680.746+0.022 (p = 0.023)

ToSFiT as published, with full fine-tuning and a frozen critic, gives 0.51-0.53 on SMILES-DRD2 across three learning rates, against 0.513 for that same critic under LoRA, and all of these sit below the 0.62 reported for it in Table 1. The Table 1 comparison therefore isolates the training signal: the critic and adaptation setup are matched, while the policy-gradient update is replaced by IPO. We attribute that to how the Bayesian signal is converted into a policy update rather than to the quality of the critic or the backbone, and we read it as a statement about the ~100-evaluation regime rather than about ToSFiT in the setting it was designed for: one scope limit remains, now stated, in that their published protein experiment uses a 1,000-evaluation budget against our 100.

AuthorsOpenReview ↗

Response to Reviewer 6TYZ (2/4)

Q1. Reproducibility statement

The components are already in the paper: the algorithm box, the configuration table (identical on all tasks), the benchmarks, objectives and cold-start pools, the baselines, constrained decoding and the exact prompts are all given in the appendix. What was missing was a statement pointing to each of them, which we have added, together with the seed-to-seed noise floor so readers can tell which differences between our numbers are meaningful. All base models and objectives are public, so the setup is reconstructible from the paper alone; code and analysis scripts are released with the final version.

Q2. Sensitivity to prompts

Prompt sensitivity can be posed on the two Qwen tasks, where the prompt lists the parameter space and asks for one candidate. On the other four the generator is gpt2-zinc-87M or PeptideGPT, BOS-conditioned samplers with no chat template and no parameter space to describe, so there is no prompt to vary. We measured it on Catechol: four paraphrases × 20 seeds, varying wording rather than output format, with variant 0 byte-identical to the published prompt.

The four prompts differ in exactly two lines: the opening framing and the closing output rule. The parameter block between them, which lists the solvents, SolventB%, residence time and temperature, is byte-identical across all four, so the spread below comes from framing alone and not from different information. The opening line of each (full prompts in the appendix):

variantopening framing linebest-so-far
0 (published)"You are an expert optimizer. Your goal is to suggest a candidate to maximize the objective."2.743 ± 0.073
1 (terse)"Propose one candidate that maximizes the objective."2.742 ± 0.045
2 (verbose)"You are a research scientist planning the next experiment in a costly design campaign. Propose the single candidate you believe will give the highest objective value."2.678 ± 0.068
3 (domain-neutral)"Select one input for a function whose output you are trying to make as large as possible."2.559 ± 0.130

No pair differs significantly (all p ≥ 0.20; ANOVA p = 0.38): 0.18 separates best and worst. Rephrasing is essentially free: dropping the persona changes the result by 0.001 and tripling the verbosity by 0.065. The largest numerical difference arises when the domain framing is removed. Variant 3, the only one that stops describing the task as an experiment, sits 0.18 below variant 0 with the widest seed-to-seed spread, but that difference is not statistically significant at 20 seeds. We therefore read the results as suggesting that task framing may matter, while prompt sensitivity is not something we can demonstrate across these variants. Variant 3's parameter block still names the solvents, so any effect comes from framing rather than from information.

Q3. SQL query optimization

We would rather decline this one than answer it badly. SQL plan optimization is a different problem class: methods there train on one set of query shapes and are scored on held-out ones, so the object optimized is a policy that generalizes across queries, whereas every task in this paper optimizes a single objective within a single budget. A number produced under our protocol would not be comparable to that literature.

We did extend domain coverage during this rebuttal, where the comparison is exact. GB1 (Wu et al., 2016) is all 149,361 measured four-site variants of protein G domain B1, so every reward is a lookup and the optimum is known. GGOLLuM reaches 0.707 ± 0.035 within 100 iterations against 0.605 ± 0.026 for classical BO (p = 0.03), 0.503 for LLM (ICL), 0.437 for GOLLuM and 0.311 for the untrained LLM, on a general-purpose Qwen2.5-7B backbone with no protein-specific pretraining.

Q4. Recursive self-improvement

In a limited but measurable sense the loop already is: the actor's samples become its next training set, the critic refits on what the actor chose, and Section 6.4 shows the critic's ranking accuracy on the actor's own proposals rising over a run. The grounding is what stops it drifting: every update is anchored to measured points, so the critic cannot reinforce a region the oracle has not confirmed. What we do not do is modify the algorithm or objective itself, which we regard as future work.


AuthorsOpenReview ↗

Response to Reviewer 6TYZ (3/4)

Q5. LLAMBO, LGBO and LABO

LLAMBO, LGBO and LABO are implemented and run.

On the backbone. We ran both on Qwen2.5-7B base, matching our backbone so the comparison isolates method from model, and also on Qwen2.5-7B-Instruct. We headline the base arms because they are the stronger result for both baselines, not because they are the faithful ones.

LLAMBO (Liu et al., ICLR 2024)

We transcribed their discriminative in-context surrogate into our stack. The comparison is clean because their components map onto ours: zero-shot warmstarting is disabled, since every method starts from the identical cold start, and their history-conditioned sampler is our LLM (ICL) baseline, reused unchanged. The only thing the baseline adds is the in-context surrogate.

taskarmbest-so-farsurrogate addsp
CatecholLLM (ICL), their sampler without the surrogate1.930 ± 0.106n/an/a
as released2.139 ± 0.124+0.2090.30
format-forced2.510 ± 0.157+0.5800.0097
GGOLLuM2.772 ± 0.055n/an/a
GB1LLM (ICL), their sampler without the surrogate0.503 ± 0.034n/an/a
as released0.467 ± 0.028−0.0370.39
rescaled0.525 ± 0.030+0.0220.54
rescaled + forced0.537 ± 0.037+0.0340.49
GGOLLuM0.707 ± 0.035n/an/a

20 seeds per cell; "surrogate adds" is the seed-paired difference against LLAMBO's own history-conditioned sampler, which is our LLM (ICL) baseline, so that column isolates the in-context surrogate and nothing else.

The surrogate's contribution is task-dependent. On GB1 it adds nothing: −0.037 to +0.034 across the three arms, every p ≥ 0.39, including the arm whose surrogate answers every query. On Catechol it does real work with +0.580 at p = 0.0097 in the format-forced arm, and +0.209 (p = 0.30) as released. This split is consistent with LLAMBO's own positioning as a hyperparameter-optimization method: it helps on a small, well-described parameter space and not on a fully measured sequence landscape. GGOLLuM leads on both (2.772 and 0.707).

The surrogate's rank correlation with the true objective, averaged over a run, is 0.54 (as released) and 0.66 (format-forced) on Catechol, and 0.22 / 0.18 / 0.36 on GB1, with parse rates of 0.24 and 0.97 on Catechol and 0.17 / 0.09 / 1.00 on GB1. Where it ranks best (ρ = 0.66) it only matches our critic with a frozen kernel (0.66) and falls short of our learned deep kernel (0.81).

The Instruct runs. Parse rate rises to 1.000 with zero invalid candidates, yet performance collapses, because the optimizer starves before the budget is spent:

armbest-so-farevaluations consumed
Catechol, base + forced2.510 ± 0.157110/110, all 20 seeds
Catechol, Instruct0.647 ± 0.075median 95; 7/20 reached 110
GB1, base, rescaled + forced0.537 ± 0.037110/110, all 20 seeds
GB1, Instruct0.145 ± 0.033median 22; 0/20 reached 110

Those values are not like-for-like. 87.5% of its GB1 proposals duplicate earlier ones (72.3% on Catechol) against 9.3% and 6.1% for base, novel-candidate rate falls 0.70 → 0.08, and surrogate ρ drops to roughly zero (−0.02 GB1, 0.04 Catechol). The model becomes syntactically well-behaved but highly repetitive.

Two added arms, both of which can only help the baseline: their strict parser accepts 28–38% of base draws on Catechol, hence a format-constrained arm; and on GB1 their six-decimal serialization renders rewards of order 1e-4 as 0.000300, so the model answers 0.000000 and EI goes flat, hence a rescaled arm. Applicability is limited to Catechol and GB1.

LGBO (Yuan et al., ICLR 2026)

LGBO is now cited in Section 2 as an LLM prior that reshapes the acquisition, and run under our protocol. Its mechanism is a region-lifted mean tilt: the LLM returns a point or region plus a confidence c, and the posterior mean shifts along Cov(X, G) by exactly c posterior standard deviations, covariance unchanged, with no free scale parameter.

Because LGBO is a prior injected into standard BO, we gave it our own BO baseline as its starting point with identical featurisation, kernel, 512-candidate pool and acquisition, so the only difference is the LLM prior, and added the same loop with the tilt off as a seed-paired control.

taskarmbest-so-farpaired diff vs no tiltseeds with identical final best-so-far
Catecholno tilt (control)2.028 ± 0.061n/an/a
as released2.003 ± 0.061−0.025 (p = 0.14)80%
format-forced2.073 ± 0.038+0.045 (p = 0.47)75%
GGOLLuM2.772 ± 0.055n/an/a
GB1no tilt (control)0.648 ± 0.029n/an/a
as released0.639 ± 0.022+0.003 (p = 0.94)21%
format-forced0.653 ± 0.023+0.006 (p = 0.86)25%
GGOLLuM0.707 ± 0.035n/an/a
AuthorsOpenReview ↗

Response to Reviewer 6TYZ (4/4)

The LLM prior has no measurable effect on either task. Every seed-paired difference is within noise, and on Catechol most seeds finish on precisely the value they reach with no prior. The tilt changes which candidate the acquisition selects in only 5–15% of iterations on Catechol and ~4–7% on GB1, because a preference moves the decision only when the pool holds candidates the GP already considers strongly correlated with the preferred region.

Paired differences are computed over the seeds both arms completed, so on GB1 they need not equal the difference of the column means.

LGBO accordingly tracks the no-tilt control it sits on rather than closing any of the gap to GGOLLuM, and is indifferent to backbone (2.003 vs 1.969; 0.639 vs 0.630) even as its parse rate improves from 0.14 to 0.50 and 0.65 to 0.94. Letting the prior be heard far more often changes nothing. An LLM prior on the surrogate is not sufficient; what distinguishes our method is that the generator itself moves. Two implementation deviations are disclosed in the appendix.

LABO (Chen et al., ICML 2026)

LABO is a dual-fidelity method rather than a competitor at the same fidelity: it treats LLM predictions as a cheap low-fidelity source, fits a Kennedy–O'Hagan joint GP, and gates when a real experiment is worth running. We ran it at 20 seeds, budget-matched at 110 real-fidelity evaluations and paired per seed with every other method on identical cold starts (its 50-point LHS warm-up is low-fidelity only and spends no real budget):

methodCatecholGB1
LABO1.967 ± 0.0610.613 ± 0.032
classical BO1.907 ± 0.0870.605 ± 0.026
GOLLuM2.308 ± 0.0440.437 ± 0.035
GGOLLuM2.772 ± 0.0550.707 ± 0.035

LABO lands at classical-BO parity on both tasks (paired p = 0.60 and 0.84). It is not a weak baseline: on GB1 it beats GOLLuM by +0.176 (p = 0.001), the gated joint GP recovering what GOLLuM's deep kernel does not on that landscape. GGOLLuM is above it by +0.805 on Catechol (paired p = 3e-9) and +0.093 on GB1 (paired p = 0.033).


Your review asked us to separate the method from ToSFiT at the methodological level and to position it against LLM-assisted BO. Both are now answered with experiments: the ToSFiT comparison is a controlled ablation in which every other component is ours, and all three methods you named are now run: an LLM prior on the acquisition (LGBO) is a measurable no-op on both tasks, an LLM surrogate (LLAMBO) helps only on the small parameter space and not on the measured sequence landscape, and a dual-fidelity joint GP (LABO) lands at classical-BO parity. None closes the gap to GGOLLuM, which is our evidence that moving the generator is what matters. With the reproducibility statement, the prompt study and the new GB1 benchmark, we believe your concerns are addressed and hope you will consider raising your score. If any part is unconvincing, tell us during the discussion period and we will run what is needed.

Reviewer_udzvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LLMs for optimization over discrete spaces such as molecules. At a high level, the LLM is fine-tuned to mimic the preferences that a GP presumably would make. The proposed algorithm trains a GP over embeddings using a deep kernel, generates candidates via autoregressive sampling, uses the GP to construct preferences (ranking with UCB, winner partitioning with posterior mean), then fine tunes the LLM using LoRA with IPO, and the process repeats. The paper highlights some promising uplift over strategies that naively attempt to optimize over discrete spaces.

Critical strengths
  • If I were tasked to make an LLM behave similarly to BO, I think what the authors propose is a natural thing to come up with.

  • The ablations are illuminating for the sources of the approach's uplift

  • There is a decent selection of different model architectures

  • Overall procedure seems quite cheap to runOn Line 102: “Standard BO maximizes an acquisition function over a searchable space – an enumerated library, a box, or a learned latent space with a decoder – none of which exists for molecules or peptides.”.

  • This is clearly false. the author’s own Line 121 and cited work contradict this.

Critical weaknesses

Standard BO maximizes an acquisition function over a searchable space – an enumerated library, a box, or a learned latent space with a decoder – none of which exists for molecules or peptides.

This is clearly false. the author’s own Line 121 and cited work contradict this.

No candidate set, latent space or decoder: the acquisition is maximized by aligning πθ\pi_\theta, and a batch is just b samples from it.

I think this is false. GGoLUM clearly constructs a candidate set from sampling from an LLM. The claim that the acquisition function is maximized is unsubstantiated.


There are several established approaches for BO over structured spaces and unfortunately they are not compared here. There are several pre-trained models for molecules and peptides one could easily invoke several of the LSBO methods. The BO baseline as far as I can understand simply a ablation study to see the sources of uplift for the proposed method.

I’m not entirely convinced that the authors have managed to transform an LLM to become a generative Bayesian policy. I think more weakly, the LLM is trained to reflect GP-induced preferences for some particular heuristic. Managing uncertainty to drive exploration are key components of Bayesian decision making and given the preference optimizations (fine-tuning over winners that are just based on the posterior mean), it feels more like a form of local search. I don’t see a convincing case where the approach would choose a candidate with lower posterior mean than another one for the sake of its exploration value. There are also no supporting experiments that demonstrate the LLM has internalized some Bayesian decision policy.


  • Missing reference on line 193
  • Line 234/235 comment on UCB already stated earlier
Questions for authors
  • Sampling from the LLM should be relatively cheap compared to fine-tuning (e.g. batch generation). Presumably GGOLLuM would perform better if you allow more than just 16 samples per run. Given your ~120 evaluations it seems pertinent to test the setting where you are given a ton of wall time to propose a candidate. How does it do here?

  • I imagine there will be cases where a point has higher UCB score than another (e.g. due to higher posterior variance), but will be selected as a loser because of the posterior mean partitioning choice. What would happen here?

5 comments
AuthorsOpenReview ↗

Response to Reviewer udzv (1/3)

We thank the reviewer for a careful reading, for two precise factual corrections, both of which were right, and for the question of whether the LLM has internalised a Bayesian decision policy, which prompted the largest piece of new analysis in this revision. We are glad the ablations read as "illuminating for the sources of the approach's uplift", since that is where most of this reply lives.

W1 and W2. Two overstated claims, corrected and sharpened

"none of which exists for molecules or peptides" (Fig. 1 caption). The reviewer is right that the sentence is false as written, and right that our own Section 2 contradicts it: enumerated libraries and learned latent spaces with decoders both exist for these domains and we run one of each in W3 below.

What we meant is a claim about construction rather than existence, and it is the one we should have made. For reaction optimization the design space is the enumerable set: a product of parameter levels that can be written down exhaustively, so selecting from it loses nothing. For molecules and peptides, exhaustive enumeration of the complete valid design space is generally computationally infeasible, so existing approaches construct finite libraries or learned latent spaces as a tractable proxy, one whose coverage is fixed before optimization begins, must be built per domain (LOLBO's decoder is trained on 1.27M molecules), and which the optimizer can never leave. Our method generates candidates directly from a pretrained model's distribution, so no such proxy has to be constructed first.

The new baselines make this concrete: under our protocol LOLBO's constructed latent space reaches 0.090 at 100 iterations against our 0.897 from the same ten molecules. The caption now states that standard BO maximizes an acquisition function over an explicitly constructed search space, and that for molecules and peptides such a space has to be built and a decoder trained first.

"No candidate set … the acquisition is maximized." Also correct, and we have fixed both halves. GGOLLuM does construct a candidate set by sampling from the policy. The contribution bullet now states that the policy's own samples are the candidate set, so no library, latent space or decoder has to be built beforehand, and Figure 1 now says the search is amortized into the policy, absorbed into where the policy samples from rather than eliminated, which is what the submitted wording implied. We have made the same correction in the Figure 3 caption, and fixed the missing reference on line 193 and the duplicated UCB sentence at lines 234–235.

W4. Has the LLM internalised a Bayesian decision policy?

Thank you for this question. Our additional experiments suggest that Bayesian uncertainty plays two distinct roles.

First, uncertainty informs representation learning. The LLM encoder is optimized through the GP marginal-likelihood objective, which depends on the kernel covariance structure. Freezing the LLM embeddings degrades both the critic's rank correlation with the true objective (new appendix section) and optimization performance:

frozen featureslearned deep kernel
Catechol2.5652.796 (p = 0.018)
Peptide-AMP0.8470.869 (p = 0.006)
Peptide-Lytic0.5210.539 (p = 0.011)

Second, explicit uncertainty plays a smaller role in preference construction once the representation is learned. Only 0.6–2% of winners are promoted by σ under the learned kernel, against 12–15% under frozen features. Consistently, removing σ from pair construction does not significantly affect final optimization performance (differences between −0.035 and +0.014, all p ≥ 0.18). Uncertainty therefore contributes mainly through learning the representation rather than through an explicit exploration bonus in every preference.

Finally, we tested whether the resulting policy can generate competitive candidates without explicit acquisition maximization. We compared GGOLLuM, which evaluates the first b samples of its own policy, against a variant that selects the acquisition maximizer from the same candidate pool. Seed-paired, 20 seeds, on SMILES-DRD2:

oracle evaluations (incl. 10 cold start)GP selectsGGOLLuM selectspaired difference
350.4460.298+0.148 (p = 0.08)
600.6750.490+0.185 (p = 0.07)
110 (full budget)0.8930.855+0.038 (p = 0.53)

GP selection holds a numerical advantage earlier in optimization, but we find no statistically significant difference at any checkpoint.

Together these results suggest that Bayesian feedback shapes the representation and the proposal distribution, allowing the resulting policy to generate candidates competitive with explicit GP-based selection. GGOLLuM thus amortizes Bayesian search into the generator through sequential GP-derived supervision.

AuthorsOpenReview ↗

Response to Reviewer udzv (2/3)

Q2. A point with higher UCB but lower posterior mean: what happens?

The pair is dropped, and we have ablated that choice. Our pair construction keeps a pair only if the winner's posterior mean also exceeds the loser's (the μ filter), precisely to avoid teaching the policy to prefer an uncertain, worse-looking candidate over a confident, better-looking one. Arm E removes the filter, so exploration alone can set the direction of a preference. In the final configuration 98% of pairs already satisfy it, so removing it changes about 2% of pairs, and it does not change where the search ends up: on SMILES-DRD2 arm E reaches 0.866 against arm B's 0.840, and on Peptide-AMP the two are identical at 0.858. What it changes is how fast: on SMILES-DRD2 arm E has the lowest area under the best-so-far curve of arms A–E (0.501 against 0.526 for arm B), so those flipped pairs slow the early search slightly. Both tables are in the pair-construction ablation appendix.

W3. Structured-space BO baselines

The reviewer is right that our BO row functions as an ablation rather than as an external baseline, and that LSBO methods can be invoked off the shelf for these domains. We have added two of them as external baselines. Reviewer smaK asked for the same comparison, so these runs answer both reviews. We note the outcome up front: under our protocol both land below the BO row the reviewer reads as an ablation. That row is a 2048-bit Morgan-fingerprint GP selecting by UCB among the base LLM's 16 proposals per iteration (Section 5), and on SMILES-DRD2 it reaches 0.637 ± 0.054 against LOLBO's 0.090 and COWBOYS' 0.077. We read this as a statement about the ~100-evaluation regime rather than about either method: a simple GP over an LLM-proposed pool is a stronger starting point at this budget than a latent space that has to be built first.

We ran LOLBO (Maus et al., 2022) as published (their pretrained SELFIES InfoTransformerVAE, trust-region loop and variational deep-kernel GP), with only the objective replaced by our DRD2 oracle, b = 1, and an initial design paired per seed with our own cold start.

BO iterationsLOLBO best-so-far
100 (our budget)0.090 ± 0.018
1,0000.359 ± 0.063
10,0000.730 ± 0.072
GGOLLuM at 1000.897 ± 0.030

LOLBO makes essentially no progress at our budget, and even at 10,000 iterations it remains below what GGOLLuM reaches at 100. We present this as a statement about the ~100-evaluation regime rather than about LOLBO, which climbs steadily with budget; rankings among molecular optimizers are known to be budget-dependent (Gao et al., 2022). LOLBO initializes molecular tasks with 10,000 labelled molecules and, as Maus et al. note in their own limitations, requires tens of thousands of evaluations on Guacamol-style tasks, which they observe is unlikely to be efficient enough for wet-lab applications. That is the regime GGOLLuM targets.

We also added COWBOYS (Moss, Ober & Diethe, 2025), which unlike LOLBO is designed for small budgets. COWBOYS ships with Morgan r3 count fingerprints and a Tanimoto kernel, whereas our own GP baselines use r2 binary fingerprints with a Matérn-5/2 kernel, so that the comparison does not turn on featurisation we ran it in its published configuration and in two variants that bring it onto our features and kernel:

armfingerprints / kernelmean best-so-far
faithfulr3 counts / Tanimoto0.077 ± 0.044
matched featuresr2 binary / Tanimoto0.178 ± 0.070
fully matchedr2 binary / Matérn-5/20.093 ± 0.048
GGOLLuMn/a0.897 ± 0.030

The arms are indistinguishable (pairwise p ≥ 0.23) and all far below GGOLLuM (p < 1e-9).

The two methods exhibit different limitations under our evaluation protocol. LOLBO is budget-starved: it climbs steadily with evaluations. COWBOYS is initialization-starved: doubling its budget does nothing (0.077 at 100 iterations, 0.081 at 200), but our cold start draws from the below-median half of the pool, so all ten of its labels are ~0.003; the GP's log-probability-of-improvement surface becomes nearly flat, leaving the pCN sampler with little direction for exploration. Drawing the same ten points from the full objective range takes COWBOYS from 0.077 to 0.586. We ran GGOLLuM from each initialization, seed-paired on identical molecules:

initializationCOWBOYSGGOLLuMpaired difference
10 points, below-median (our protocol)0.0770.897+0.821 (p < 1e-4)
10 points, full range0.5860.849+0.263 (p = 0.015)

We retain a substantial advantage even under matched initializations.

AuthorsOpenReview ↗

Response to Reviewer udzv (3/3)

Q1. More than 16 samples per iteration

The reviewer asks about the regime where wall time to propose a candidate is generous. We ran the full ladder on Peptide-AMP, 20 seeds per arm, with the oracle budget unchanged: only the number of candidates the policy proposes each iteration grows. We report at 50 oracle evaluations as well as at the full budget, because one arm could not finish within wall clock and 50 is below the shortest run of every arm, so every cell is complete and none is selected for having finished:

candidates / iterationpreference pairs / iterationat 50 evalsat 110 evals
496 (cap)0.805 ± 0.0100.854 ± 0.007
896 (cap)0.786 ± 0.0120.832 ± 0.009
16 (default)96 (cap)0.801 ± 0.0110.854 ± 0.005
6496 (cap)0.828 ± 0.0090.875 ± 0.006
25696 (cap)0.828 ± 0.0070.875 ± 0.006
64384 (proportional)0.819 ± 0.0100.865 ± 0.008
2561,536 (proportional)0.793 ± 0.008n/a (see below)

Sampling more helps up to N = 64 and then stops. 16 → 64 is worth +0.027 at 50 evaluations and +0.021 at 110; 64 → 256 is worth +0.0003 (p = 0.98) at 50 and −0.0003 (p = 0.97) at 110. The ordering is identical at both budgets, so it is not an artefact of where we read the curve. On SMILES-DRD2 no setting separates from any other (all p > 0.9).

We checked the obvious confound, and it points the other way. In the ladder above the preference-pair budget is capped at 96 per iteration regardless of N, so those arms test a better-ranked pool rather than more training signal. Arms that scale pairs with the pool do not help; they hurt: at N = 256, 1,536 pairs gives 0.793 against 0.828 for the cap at the same N (−0.035, p = 0.006), and at N = 64, 384 pairs gives 0.819 against 0.828 (−0.009, p = 0.53). Increasing the preference-pair budget does not improve performance and can hurt it substantially at larger pool sizes.

The N = 256 proportional arm is also why we report at 50. Its 1,536 pairs per iteration are about 240 gradient steps per BO iteration, sixteen times the capped arm, and those runs exceeded a 10-hour wall clock at roughly 65 of 110 evaluations on two separate launches.

More sampling and more preference signal both saturate, and the second begins to hurt. The binding constraint is how well the critic orders candidates it has not yet seen, not how many it is given to order.


This review produced the two corrections and the question that prompted the largest addition to this revision: both overstated claims are fixed, W4 is answered with three new experiments, two structured-space BO baselines are added, and the sampling ladder is run out to N = 256. If you consider these concerns addressed, we would be grateful if you would reconsider your rating; and if anything is unclear, we are happy to discuss more.

Reviewer_smaKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This manuscript proposes a methodology that uses Bayesian optimization (BO) to provide the guiding signal for preference-based LLM alignment. Briefly, the proposed method operates as follows: First, deep kernel GP is fitted using all designs that have been evaluated so far. Then, the method samples a pool of new candidate designs from LLM using the current policy. Policy is implemented as a LoRA fine-tuned version of a base LLM. Method then chooses k anchor points (from the evaluated designs) and preference-ranks new candidates designs and anchors. It then chooses the top ranking candidates and evaluates their score. Finally, the method pdates Lora adapter using identity preference optimization, and then starts a optimization cycle.

The proposed method is evaluated on 5 tasks, and the results show that during iterations the LLM generation moves towards higher-utility regions of the GP model, i.e. the proposed method turns pertained base LLM model to a Bayesian optimizer. While earlier works have shown that LLMs can serve as uncalibrated proposal distributions in experimental design, here the aim is to train an LLM to operate like Bayesian optimization by providing the preference signal from “a real BO”.

Operation of the proposed method is examined by various ablation studies, which pinpoint the important steps of the proposed methods: particularly, the preference policy optimization, which makes the LLM generator to act as the optimizer.

Critical strengths

An interesting methodology to use BO to guide LLM alignment!

Critical weaknesses

While it is to some extent fair to argue that standard BO e.g. for molecule or peptide tasks requires a specific latent variable model together with encoder and decoder that is trained separately for each task. Similarly, LLM baseline models that are specific for different tasks are also needed for the proposed method. It would be meaningful and important to include state-of-the-art BO baseline models for empirical evaluations as well. For example, the not-so-recent-anymore LOLBO from (Maus et al, 2023) that you also cite in your manuscript. The recent COWBOYS (Moss H, et al., ICML 2025, Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces). These methods have been demonstrated on various molecule tasks, for example. For amino acid sequences, e.g. KERMUT from (Groth et al, NeurIPS, 2024, Kermut: Composite kernel regression for protein variant effects). While these baselines do not rely on the same principles as the proposed LLM-based method, it would be highly-valuable to include some very recent high-performing methods to get a better understanding where the proposed methods stands wrt these baselines.

Questions for authors

I missed a detailed description of how the k anchors are chosen per iteration?

Please clarify the dimension of input vectors which are used to fit the deep kernel GP surrogate.

Minor comment: Section 6.5: reference to Fig. 6 is missing.

3 comments
AuthorsOpenReview ↗

Response to Reviewer smaK (1/3)

Thank you for the thoughtful and constructive review, and for recognizing the value of using Bayesian optimization to guide LLM alignment. Your suggestions motivated three additions, which we report below: the two structured-space BO baselines you named, a new GB1 protein optimization benchmark, and an ablation of the critic kernel.

W1. Comparison with state-of-the-art BO methods for structured spaces

We adapted and evaluated both LOLBO and COWBOYS on SMILES-DRD2, starting from exactly the same 10 cold-start molecules as GGOLLuM and using 100 BO iterations, with 20 seeds throughout.

GGOLLuM achieves better optimization performance than LOLBO with 100 times fewer BO iterations, starting from exactly the same 10 initial molecules.

LOLBO. Evaluated with its pretrained SELFIES InfoTransformerVAE, trust-region loop and variational deep-kernel GP, at three budgets:

BO iterationsLOLBO mean best-so-far
100 (our budget)0.090 ± 0.018
1,0000.359 ± 0.063
10,0000.730 ± 0.072
GGOLLuM (100 iterations)0.897 ± 0.030

LOLBO makes essentially no progress at our budget, and even at 10,000 iterations it remains below what GGOLLuM reaches at 100. We present this as evidence of sample efficiency in the low-budget discovery regime rather than as a general statement about LOLBO, which climbs steadily with budget. The regimes are simply different: LOLBO initializes molecular tasks with 10,000 labelled molecules and, as Maus et al. note in their own limitations, requires tens of thousands of function evaluations on Guacamol-style tasks, which they observe is unlikely to be efficient enough for applications run in the wet lab. That is the exact regime GGOLLuM targets.

COWBOYS. COWBOYS is published with Morgan r3 count fingerprints and a Tanimoto kernel, whereas our own GP baselines use r2 binary fingerprints with a Matérn-5/2 kernel. So that the comparison does not turn on featurisation, we ran COWBOYS in its published configuration and in two variants that bring it progressively onto our features and kernel:

ConfigurationFeatures / kernelMean best-so-far
Published configurationr3 counts / Tanimoto0.077 ± 0.044
Matched featuresr2 binary / Tanimoto0.178 ± 0.070
Fully matchedr2 binary / Matérn-5/20.093 ± 0.048
GGOLLuMn/a0.897 ± 0.030

The three COWBOYS configurations are statistically indistinguishable (all pairwise p ≥ 0.23), while each performs significantly below GGOLLuM (p < 1e-9), so the gap does not stem from our featurisation choice.

We further investigated why COWBOYS struggles under our initialization and found a different failure mode from LOLBO. COWBOYS is highly sensitive to the quality of its initial observations. Doubling its budget from 100 to 200 iterations barely changes its performance (0.077 to 0.081). Changing the initialization, while keeping the number of initial points and the optimization budget fixed, changes it substantially.

Our cold-start setting deliberately selects 10 initial molecules from the below-median portion of the objective distribution, mimicking discovery scenarios in which optimization starts from unsuccessful candidates. All 10 initial objective values are then approximately 0.003. The GP fitted to these nearly identical, low-valued observations provides little useful signal: its log-probability-of-improvement surface becomes nearly flat, leaving the pCN sampler with little direction for exploration.

To test whether this limitation comes from the initialization rather than the search procedure itself, we repeated the experiment with 10 randomly sampled initial molecules spanning the full objective range. COWBOYS then improves substantially, reaching 0.586. We also ran GGOLLuM from the identical initial molecules for each seed, allowing a paired comparison:

Initialization (SMILES-DRD2)COWBOYSGGOLLuMPaired difference
10 below-median points0.0770.897+0.821 (p < 1e-4)
10 points spanning the full objective range0.5860.849+0.263 (p = 0.015)

COWBOYS makes substantial progress when given a more informative initialization, but deteriorates when starting exclusively from unsuccessful candidates. This also bears on a point Reviewer udzv raised independently, namely whether our method is doing local search: a local search from ten below-median points stays there, which is precisely COWBOYS' behaviour under the identical initialization. GGOLLuM performs strongly under both and retains a significant advantage even when COWBOYS is given the more informative starting set.

AuthorsOpenReview ↗

Response to Reviewer smaK (2/3)

Together these experiments identify two distinct limitations in the low-budget setting: LOLBO requires substantially more optimization iterations, whereas COWBOYS requires a sufficiently informative set of initial observations. GGOLLuM is less dependent on informative initialization here: its pretrained generator supplies a structured proposal distribution even when the initial objective observations carry no gradient, and the preference feedback lets the policy improve from a small number of evaluations.

One further comparison is worth stating, since it does not depend on our method at all. The classical BO row of Table 1 (a 2048-bit Morgan-fingerprint GP selecting by UCB among the base LLM's 16 proposals per iteration) reaches 0.637 ± 0.054 on SMILES-DRD2 under this protocol, well above both LOLBO (0.090) and COWBOYS (0.077). This completes the picture of what the budget actually rewards. Under this low-budget protocol the base LLM's proposal pool provides a simple fingerprint GP with a strong starting point without additional oracle evaluations, whereas the latent-space methods we evaluated need substantially more evaluations to reach comparable performance: LOLBO's space must be built before optimization begins, and COWBOYS has a capable search procedure but nothing informative to start it from.

KERMUT. We agree that KERMUT is relevant to sequence and protein variant-effect modelling. It is, however, a surrogate construction rather than a complete end-to-end optimization method with its own candidate-generation mechanism, so it is not directly comparable to LOLBO or COWBOYS. Our frozen-versus-learned kernel ablation instead demonstrates the importance of adapting the critic representation to experimental feedback: a critic whose kernel cannot adapt ranks unseen candidates worse (Spearman of the posterior mean against truth, averaged over a run: 0.656 vs 0.808 on Catechol, 0.719 vs 0.781 on Peptide-AMP) and optimizes worse on every task. Integrating a KERMUT-style kernel as an alternative critic is an interesting direction for future work.


W2. Dependence on domain-specialized language models

You note that "LLM baseline models that are specific for different tasks are also needed for the proposed method." This is a fair reading of the main results, but it is not a requirement of the method: we demonstrate GGOLLuM with a general-purpose LLM in three of the four domains considered.

DomainBackboneGeneral-purpose?
Reaction optimization (Catechol)Qwen2.5-7Byes
Peptide design (Peptide-AMP)Qwen2.5-7B / PeptideGPTyes (both reported)
Protein design (GB1, new)Qwen2.5-7Byes
Molecular design (SMILES)GPT-2-ZINC-87Mno (see below)

On Peptide-AMP the submitted manuscript already compares both backbones, and GGOLLuM retains its advantage with either, reaching 0.90 with general-purpose Qwen2.5-7B against 0.87 with domain-pretrained PeptideGPT. The newly added GB1 benchmark (Wu et al., 2016; 149,361 measured four-site variants of protein G domain B1, rewards by exact lookup) uses Qwen2.5-7B with no protein-specific pretraining, and GGOLLuM reaches 0.707 ± 0.035 against 0.605 ± 0.026 for classical BO (p = 0.03).

Molecular design is the exception, due to the challenge of generating valid molecular structures rather than a requirement of GGOLLuM. In the other three domains the design space can be enforced during decoding, which is why a general-purpose backbone suffices there. For peptides and for proteins, every string over the amino-acid alphabet is a valid candidate, so restricting the vocabulary is enough; on GB1 the four mutable sites are additionally fixed by position. In reaction optimization we constrain generation to the specified design space, so the policy can only propose admissible reagents and conditions. The molecular equivalent would be a grammar that guarantees a valid SMILES string from the first token, enforcing valence and ring-closure constraints during generation, and this remains an open problem. Representations that are valid by construction, such as SELFIES, sidestep it only by requiring the backbone to generate in a representation that a general-purpose LLM has barely encountered in pretraining, which forfeits the very prior that motivates using one. The submitted appendix already records that SMILES decoding is unconstrained for this reason; we therefore use a chemistry-pretrained generator for molecular tasks, and the revision states this explicitly as a limitation in the main text.

AuthorsOpenReview ↗

Response to Reviewer smaK (3/3)

Q1. How are the k anchors selected per iteration?

At each iteration we sample k = 10 anchors uniformly at random, without replacement, from all previously evaluated points. They are independently resampled at every iteration, irrespective of their objective values.

This choice is motivated by the pair-pool ablation in Figure 5(b), which compares random anchors, the 10 highest-performing evaluated points, and all evaluated points. At larger preference margins, random anchors maintain training stability, whereas selecting the 10 best historical points leads to substantially more training failures. Using all evaluated points also preserves stability, but requires a larger pair pool. The diversity of historical anchors, rather than their objective quality, is what prevents policy collapse. We have clarified the selection procedure and its motivation in Section 4.

Q2. Dimension of the input vectors for the deep-kernel GP

Each candidate is represented as text and passed through the LLM. The pooled final-layer embedding is the input to the Matérn-5/2 kernel, with the encoder representation optimized through the GP marginal-likelihood objective. Its dimensionality is the backbone hidden size: 3,584 for Qwen2.5-7B, 768 for GPT-2-ZINC-87M, and 1,280 for PeptideGPT. We have clarified this in the implementation details.

Q3. Missing reference to Figure 6

Fixed. Thank you for catching this.

The substantive weakness in your review was the absence of state-of-the-art structured-space BO baselines. Both methods you named are now evaluated under our protocol, the surrogate-side method you named is answered by a new critic ablation, and your three questions are resolved in the text. If you consider that concern addressed, we would be grateful if you would reconsider your rating; and if any part of the comparison is not yet convincing, please tell us during the discussion period and we will run what is needed.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

We thank the three reviewers for detailed and specific reviews. We were encouraged that they found:

A natural and well-motivated approach. "If I were tasked to make an LLM behave similarly to BO, I think what the authors propose is a natural thing to come up with" (udzv). The motivation is "sound: using LLMs as generative Bayesian policies is a natural choice for sequence optimization", and the method "reasonable: GP-derived supervision addresses the scarcity of data for LLM fine-tuning" (6TYZ). "An interesting methodology to use BO to guide LLM alignment" (smaK).

Informative ablations. They are "illuminating for the sources of the approach's uplift" (udzv), and pinpoint "the important steps of the proposed method: particularly, the preference policy optimization, which makes the LLM generator act as the optimizer" (smaK).

A practical, broadly applicable procedure. There is "a decent selection of different model architectures" and the "overall procedure seems quite cheap to run" (udzv); soundness was rated 3, with "no substantive concerns about the paper's soundness" (smaK).

Individual replies are posted in each thread; this comment is an index of what changed, since several additions answer more than one review.

New experiments, with the review that prompted each. All use the protocol of the submission: 100 BO iterations after a shared 10-point cold start, 20 seeds, absolute best-so-far. Wherever two arms are compared, we reran both as a matched sweep on the same machine so that hardware and run-to-run variation cannot confound the contrast, which is why a few absolute values differ slightly between tables.

additionprompted by
LOLBO (Maus et al., 2022) at three budgets, paired cold startsmaK W1, udzv W3
COWBOYS (Moss et al., 2025) in three featurisation arms, plus an initialization-quality studysmaK W1, udzv W3
LLAMBO (Liu et al., 2024), base and Instruct backbones, isolated against its own sampler6TYZ Q5
LGBO (Yuan et al., 2026) with a seed-paired no-tilt control and per-iteration diagnostics6TYZ Q5
LABO (Chen et al., 2026), a dual-fidelity joint GP, implemented and run on Catechol and GB16TYZ Q5
GB1: all 149,361 measured four-site variants, exact-lookup rewards, known optimum6TYZ Q3, smaK W2
Prompt-sensitivity study, four framings × 20 seeds6TYZ Q2
ToSFiT with frozen critic and published actor6TYZ W1
Frozen versus learned critic kerneludzv W4, smaK W1
Critic-selects versus policy-selectsudzv W4
Candidates-per-iteration ladder, with pair budgets capped and scaledudzv Q1

Corrections. Reviewer udzv identified two overstated claims, both of which were right and both of which are corrected in the revision: the Figure 1 caption asserting that no searchable space exists for molecules or peptides, and the contribution bullet stating that the acquisition is maximized over no candidate set. The missing reference on line 193 and the duplicated UCB sentence at lines 234–235 are also fixed. We are grateful for the precision of those comments.

What the new experiments establish. Four results cut across the replies.

  1. Sample efficiency at low budget. On SMILES-DRD2, from the same ten molecules, GGOLLuM reaches 0.897 in 100 BO iterations against 0.730 for LOLBO at 10,000. Rankings here are budget-dependent (Gao et al., 2022), so this is a statement about the ~100-evaluation regime, not about LOLBO.
  2. Discovery from unpromising starting points. COWBOYS is sensitive to its initial observations: 0.077 from ten below-median points, 0.586 from ten spanning the full objective range. GGOLLuM keeps a substantial advantage under the more informative initialization too (0.849, paired +0.263, p = 0.015). Experimental discovery usually begins precisely because the available candidates are unsuccessful.
  3. Amortization of Bayesian search. GP-derived feedback shapes both the learned representation and the proposal distribution. By the end of optimization the policy's own candidate reaches 0.855 against 0.893 for explicit acquisition selection over the same pool, which we cannot distinguish statistically (p = 0.53).
  4. Positioning against LLM-assisted BO. All three methods 6TYZ named are now run under our protocol: LLAMBO's surrogate helps on the small parameter space but not on the measured sequence landscape, LGBO's prior leaves its BO baseline statistically unmoved, and LABO lands at classical-BO parity. None closes the gap to GGOLLuM.

Presentation. All three reviewers rated presentation 2. We are reworking the paper around the new results and would welcome specific comments on how to improve the presentation.

We are glad to run further experiments during the discussion period.

AuthorsOpenReview ↗

Summary for the Area Chair

What the paper shows. Experimental discovery campaigns begin with few measurements and poor ones. Our protocol reflects that: every run starts from ten points drawn entirely from the below-median half of the pool (~0.003 on DRD2) and gets ~100 oracle evaluations. In that regime, GP-derived preference supervision moves a pretrained generator's distribution toward high-value regions and outperforms latent-space BO, LLM-assisted BO, on-policy RL with the same critic, and our own surrogate-only and generator-only ablations, across six tasks spanning reaction, molecular, peptide and protein design. The budget comparison is the clearest statement of it: on SMILES-DRD2, LOLBO reaches 0.359 at 1,000 evaluations and 0.730 at 10,000, while we reach 0.897 at 100, and no LOLBO seed reaches our mean even at 10,000.

What the discussion period added. All three reviews prompted new experiments, all run under the submitted protocol (100 BO iterations after a shared 10-point cold start, 20 seeds, absolute best-so-far).

  • Structured-space BO (smaK W1, udzv W3): LOLBO at three budgets, COWBOYS in three featurization arms, and an initialization-quality study.
  • LLM-assisted BO (6TYZ Q5): LLAMBO, LGBO and LABO, each isolated against its own sampler or a seed-paired control.
  • ToSFiT (6TYZ W1): a controlled comparison in which our critic and adapter replace its own, so only the training signal differs. Our critic improves ToSFiT, so the comparison strengthens the baseline rather than handicapping it, and our method still comes out ahead.
  • A sixth benchmark, GB1, with exact-lookup rewards and a known optimum on a general-purpose backbone.
  • For udzv: a probe regressing the trained policy's implicit reward on the critic's posterior mean and variance, critic diagnostics, a proposal-similarity analysis, and a candidates-per-iteration ladder to N = 256.
  • Plus prompt sensitivity, a reproducibility statement, and critic and pair-pool ablations.

What we corrected. Reviewer udzv identified two overstated claims, both correctly: the figure caption asserting that no searchable space exists for molecules or peptides, and the contribution bullet stating that the acquisition is maximized over no candidate set. Both are fixed. We also agree our BO row functions as an internal ablation rather than an external baseline, which is why LOLBO and COWBOYS were added. Separately, we withdrew our own GP-UCB comparison as evidence about the decision rule: matching an acquisition maximizer in best-so-far does not establish a shared rationale, and the reviewer was right to say so.

What we narrowed. Reviewer udzv drew a distinction we accept, between a policy that uses Bayesian feedback and one whose choices reflect value of information. We claim the former. The new diagnostics explain why: training the kernel through the marginal likelihood produces a far smoother landscape (lengthscale 1.33 vs 0.35 on Catechol; correlation of posterior variance with the objective +0.35 vs +0.04), so the posterior mean becomes a reliable ordering and σ promotes only 0.6–2% of winners. The revision defines "think like a Bayesian optimizer" narrowly, as the policy reproducing the preferences of the critic supervising it, and rewrites the abstract, figure captions and conclusion to match.

Where things stand. Reviewer udzv's follow-up notes the revision is improved and accepts that an LLM's parametric prior yields benefits over a per-domain latent space; the open question is whether the policy's decisions reflect value of information, which we do not claim and which the limitations now state. We believe the remaining disagreement concerns how much to claim rather than the soundness of what we report.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.