acceptodds
Under review as a conference paper at ICLR 2027

Learning Reliable Skills from Sparse Online Feedback for Cold-Start LLM-Based Molecular Optimization

Abstract

Molecular optimization seeks drug-like candidates with strong target-specific properties under small budgets of costly evaluations. LLM-based optimizers are attractive because they propose molecules from natural-language objectives and adapt in context without training a task-specific generator. Yet methods that endow an optimizer with reusable skills typically require large offline trajectories, either to train a policy or to build a retrieved skill library — resources a cold-start target does not have. While simpler cold-start methods such as OPRO present candidates as a score-ranked list of independent molecules, preserving scores but discarding the skills latent in the online history. We propose OS-DEV (Online Skill Distillation with Evidence-Aware Verification), a training-free feedback layer that turns each evaluation into a best-referenced trace and periodically distills improving and degrading traces into target-specific proposed skills. Two independent analysts extract complementary hypotheses about what to pursue and avoid, and a role-independent verifier re-examines each proposed skill against the supporting and contradicting observations accumulated in the current run, attaching an interpretable high/medium/low confidence judgment before the resulting verified skill state guides subsequent generation. On all 58 DOCKSTRING targets with 200 oracle evaluations per target and a shared Qwen3.5-4B generator, OS-DEV reaches an overall score of 0.8487, improving over ACE (0.8254) by 0.0233 and over an agentic reference combining Claude Code with DeepSeek-V4-Pro (0.8270) by 0.0217, and leads on 30 of 58 targets. Ablations isolate the contributions of the trace display, the dual analysts, and the evidence-aware verifier, and confirm that organizing sparse feedback into reusable, evidence-linked skills improves LLM-based molecular optimization without retraining the proposal model.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, Language Models, Reasoning & Test-Time Compute, Prompt Engineering & In-Context Learning, Non-convex Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_u55MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes OS-DEV, a training-free agentic framework for cold-start molecular optimization. OS-DEV utilizes an LLM to propose candidates, and instantiates a pair of analysts to distill skills from feedback for the candidates. Then, a verifier evaluates the evidence for the skills and assigns a confidence label for each skill. The skills then guide the LLM optimizer towards better candidates. Experimental results on DOCKSTRING show that OS-DEV outperforms representative baselines, and ablation studies demonstate that each component of OS-DEV brings a performance gain.

Critical strengths
  1. This paper is easy to follow. The motivation is clearly stated, and the method details and experiments are explained well.

  2. The proposed OS-DEV does not reply on pre-built skill libraries, and does not introduce new trainable parameters, which makes it easy to deploy.

  3. It is a reasonable idea to instantiate a pair of analysts for analyze successes and failures respectively.

  4. OS-DEV outperforms representative baselines.

Critical weaknesses
  1. The quality and reliability of the distilled skills are not directly verified. Although the ablation studies show the performance gain brought by each component, there is no experimental results to support that the distilled skills are reliable, or that the verifier can indeed tell good rules from misleading ones. In addition, whether high-confidence skills are more useful than low-confidence ones is also not validated, and so it is hard to say that the confidence labels work as expected.

  2. The advantage of the proposed components (e.g., the pair of success and failure analysts) over existing techniques with similar functionality is not verified. For example, ACE's reflector extracts experience from both successes and errors, how does such a unified reflector compare with the proposed analyst pair?

  3. The performance gains from some components are modest (e.g. failure analyst).

Questions for authors

See "weaknesses". One more: as stated in Sec.2.1, the oracle returns a scalar per candidate and nothing else, while the analyst prompt includes QED/vina components (Sec.3.2). Could the authors clarify this?

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer u55M [1/3]

Dear Reviewer, thank you for the encouraging assessment and concrete suggestions. We have added controls for the verifier and analyst design and clarify the oracle information below.

W1. Can the verifier distinguish useful skills?

Thank you for this suggestion. Owing to the limited time and compute available during the discussion period, we ran two complementary tests on seven targets (the first listed target from each protein family) with three seeds each; we apologize for the limited coverage. We will continue these experiments and include the results in the paper once completed.

First, without calling the verifier, we independently assign each skill a high, medium, or low label with equal probability. This random-label variant reaches 0.8487, versus 0.9019 for OS-DEV:

TargetOriginal OS-DEVRandom labelsNo verifierOriginal − random
ABL10.95500.89160.94420.0634
ACHE0.95980.87450.93590.0853
AR0.73130.70510.72610.0262
ADAM170.86690.80930.84870.0576
ADORA2A0.93120.86420.92060.0670
CYP2C90.91930.89410.91050.0252
HSP90AA10.94970.90200.89290.0477
Mean0.90190.84870.88270.0532

Second, we run the verifier and retain different groups of skills; the optimizer still sees their original confidence labels. The results are:

Retained skillsRemoved skillsMean score (7 targets, 3 seeds)
High + MediumLow0.8992
Low + MediumHigh0.8664
High onlyMedium + Low0.8915
Low onlyHigh + Medium0.8521
All (original OS-DEV)None0.9019

High + medium reaches 0.8992, compared with 0.8664 for low + medium; high only reaches 0.8915, compared with 0.8521 for low only. These results show that higher-confidence skills provide more useful guidance for optimization, supporting the effectiveness of the evidence-aware verifier.

AuthorsOpenReview ↗

Rebuttal to Reviewer u55M [2/3]

W2. Unified reflector versus two analysts

Thank you for this thoughtful suggestion. A unified reflector can consider successful and failed trajectories together and draw on their contrast; separate analysts instead let each prompt focus on a narrower task: the success analyst identifies promising patterns, while the failure analyst concentrates on degradation and invalid proposals (Section 3.2, lines 225–242). Both are reasonable designs. Trace2Skill similarly separates success and error analysis, although its tasks and consolidation pipeline differ from ours.

We tested the reviewer’s alternative by giving both successful and failed trajectories to a single analyst, keeping the rest of OS-DEV unchanged. On seven targets with three seeds, the unified reflector reaches 0.9004 and the dual analysts reach 0.9019:

Analyst designMean score (7 targets, 3 seeds)
Unified reflector0.9004
Dual analysts (OS-DEV)0.9019

The unified design is a strong alternative: the dual design is higher by a modest 0.0015 in this experiment. We appreciate the suggestion, which helps clarify that specialization is useful here but joint reflection is also a viable choice.

AuthorsOpenReview ↗

Rebuttal to Reviewer u55M [3/3]

W3. Is the failure analyst worth its modest gain?

We agree that the success analyst contributes more in this setting: on 58 targets, success-only reaches 0.8426, while adding the failure analyst yields 0.8487 (+0.0061; Section 4.3, lines 424–427). The failure analyst contributes a complementary signal by examining degraded candidates and invalid SMILES to identify patterns to avoid. With a limited budget of expensive molecular evaluations, we believe this additional guidance is worthwhile.

The two analyst calls run in parallel, limiting the additional elapsed time when concurrent inference is available. Neither analyst uses additional oracle evaluations.

Question. Scalar objective versus QED/Vina in the trace

Thank you for pointing this out; we apologize that our description was unclear. The oracle returns the scalar objective ff in Eq. (1). For the analyst trace, we calculate QED from the candidate SMILES using RDKit, and recover the Vina component from ff and QED using the objective formula. This does not require an additional docking evaluation. We clarify this data flow in Section 2.1, lines 145–151, and the Implementation and Computational Cost appendix.

Specifically, with the normalization in Eq. (1), the normalized Vina component is

f−0.2 QED0.8,\frac{f-0.2\,\mathrm{QED}}{0.8},

and the raw Vina score is

−3−9f−0.2 QED0.8.-3-9\frac{f-0.2\,\mathrm{QED}}{0.8}.

The reconstruction assumes sufficient numerical precision.

Reviewer_zwTVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OS-DEV, a framework that augments existing LLM-based optimizers with skill-based experience extraction, verification, and updating mechanisms. By distilling successful and unsuccessful optimization experiences into reusable skills, OS-DEV enables LLMs to accumulate knowledge from historical feedback and leverage it to guide subsequent molecular generation throughout the optimization process.

Critical strengths
  1. The paper presents a complete pipeline for incorporating skill memory into LLM-based molecular optimization. By extracting insights from both successful and unsuccessful optimization experiences, the framework enables LLMs to accumulate reusable knowledge and use it to guide subsequent molecular generation.

  2. The proposed refresh mechanism allows the skill memory to evolve throughout the optimization process. This design has the potential to prevent outdated skills from overly constraining the model's generation behavior and to help the optimizer adapt its search strategy as optimization progresses.

Critical weaknesses
  1. The paper includes methods such as MR.PEA and Claude Code, which are not directly focused on skill learning or memory-based molecular optimization, while providing limited comparisons with existing skill-learning, memory-based, or training-based approaches. Although the related work discusses several skill memory and reflection methods, the authors exclude them from experimental comparisons because they do not directly satisfy the cold-start setting. However, it remains unclear why stronger skill-memory baselines could not be adapted or constructed under the same experimental constraints. More comprehensive comparisons with relevant methods would help better establish the effectiveness of the proposed framework.

  2. When constructing optimization traces, the proposed method consistently uses the historical best solution as the reference for evaluating new candidates. While this provides a consistent measure of global improvement, it may overlook valuable structural information from comparisons between other candidate molecules. For example, a molecule may contain beneficial structural modifications and achieve a competitive score without surpassing the incumbent best solution. Such candidates may consequently be classified as unsuccessful experiences, potentially obscuring useful local improvements. It would be valuable to investigate whether alternative reference strategies, such as comparisons between structurally similar molecules, could provide more informative experiences for skill distillation.

  3. The proposed framework refreshes its skill memory every two generation rounds through periodic refresh and additionally triggers a refresh after five consecutive rounds without improvement. Although the paper provides sensitivity analyses for the parameters T_update and K, it does not explicitly compare periodic-only, plateau-only, and combined refresh strategies. Since the skill memory is already updated frequently through periodic refresh, the additional benefit of the plateau-triggered mechanism remains unclear. A more targeted ablation study would help establish the necessity and effectiveness of combining these two refresh strategies.

  4. The periodic and plateau-triggered refresh mechanisms may facilitate rapid skill adaptation during the early stages of optimization. However, as optimization progresses and further improvements become increasingly difficult, frequent regeneration of the entire skill memory may introduce unnecessary computational overhead and instability. For example, previously useful skills might disappear during subsequent updates, even when the newly accumulated evidence provides little additional information. The paper would benefit from a more detailed analysis of skill stability and update frequency across different optimization stages, particularly whether frequent skill regeneration remains beneficial during late-stage optimization.

Questions for authors

see weakness

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer zwTV [1/3]

Dear Reviewer, thank you for the constructive suggestions. We have taken a closer look at stronger baselines and refresh behavior, and clarify the reference choice below.

W1. Relevant skill and memory baselines

Thank you for suggesting a comparison with stronger skill-memory methods. ACE is an existing memory-based baseline: it maintains an evolving playbook, whereas OS-DEV refreshes a run-specific skill state from traces and assigns evidence-aware labels. On all 58 targets, ACE and OS-DEV score 0.8254 and 0.8487, respectively (Table 1, lines 325–336).

Following your suggestion, we additionally compare with an adapted version of MolMem (Wang et al., 2026), a molecular optimization method that distills successful trajectories into reusable skills. The original method uses an external exemplar database and skill memory accumulated across trajectories. These resources are unavailable in our cold-start setting, so we substantially adapt MolMem to start with an empty memory and build it only from the current run.

Owing to the limited time and compute available during the discussion period, we evaluated this additional comparison on seven targets (the first listed target from each protein family) with three seeds each; we apologize for the limited coverage. We will continue these experiments and include the results in the paper once completed.

MethodMean score (7 targets, 3 seeds)
Adapted MolMem0.8510
ACE0.8570
OS-DEV0.9019

OS-DEV reaches 0.9019, compared with 0.8510 for adapted MolMem and 0.8570 for ACE.

AuthorsOpenReview ↗

Rebuttal to Reviewer zwTV [2/3]

W2. Why use the best-so-far molecule as reference?

We agree. A below-incumbent molecule can contain a useful local motif, and the sign of its global score difference does not identify every beneficial or harmful structural feature. In a cold-start run, however, the early history is small and a close structural neighbor may not exist for every candidate. The incumbent is always available and puts the candidate’s composite score in the context of the best result achieved at that point; this is why we use it as the reference.

Similarity-based retrieval is a promising complementary direction, especially as the run accumulates more molecules. As a related comparison, adapted MolMem reaches 0.8510 on these seven targets, while OS-DEV reaches 0.9019. Thank you for this insightful suggestion. Adapting the transition from OS-DEV to retrieval-based guidance, or combining the two as experience accumulates, is a promising direction that we have added to the revised Future Work section (lines 488–499), and we will continue to explore it.

AuthorsOpenReview ↗

Rebuttal to Reviewer zwTV [3/3]

W3 + W4. Refresh mechanisms and adaptation over the course of optimization

Thank you for these insightful comments. We agree that varying TupdateT_{\mathrm{update}} and KK separately (Tables 7–8) does not isolate the contribution of the two refresh triggers. We therefore compare periodic-only, plateau-only, and combined refresh on seven targets with three seeds each:

Refresh strategyMean score (7 targets, 3 seeds)
Periodic only0.8937
Plateau only0.8710
Periodic + plateau (OS-DEV)0.9019

The combined strategy improves the mean by 0.0082 over periodic-only and 0.0309 over plateau-only. Periodic refresh incorporates new evidence regularly, while the plateau trigger provides another opportunity to revise guidance when search stalls.

In the cold-start setting, this refresh strategy allows OS-DEV to incorporate newly acquired evidence into its guidance throughout optimization. To further examine its progress, we compare scores at 150 and 200 evaluations on the same seven targets:

MethodScore at 150 evaluationsScore at 200 evaluationsGain
Adapted MolMem0.83990.85100.0111
ACE0.84630.85700.0107
OS-DEV0.89100.90190.0109

OS-DEV achieves the highest score at both budgets and continues to improve from 0.8910 to 0.9019. Together with the refresh comparison above, these results support the effectiveness of our strategy under the cold-start budget. As more molecules and observations accumulate, dynamically adapting the refresh strategy to the search state is a promising direction for further improvement. Thank you for this valuable suggestion. We have added this direction to the revised Future Work section (lines 488–499): TupdateT_{\mathrm{update}}, KK, and the trace-selection thresholds γ\gamma and τ\tau could adapt to the fraction of the oracle budget spent, score progress, and properties of the incumbent molecule. We will continue to explore this direction.

Reviewer_bvnjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OS-DEV, a training-free prompt layer on an OPRO-style LLM optimizer for cold-start molecular optimization. It re-encodes the run history as incumbent-versus-candidate traces, has one analyst distill skills to pursue from the improving ones and another distill skills to avoid from the degrading and invalid ones, then has a separate verifier audit each skill against all observations and attach a confidence label. Experiments show that OS-DEV outperforms the prompt-optimization baselines and wins on more than half the targets.

Critical strengths
  1. The cold-start setting is kept clean. There is no training, no offline data, and no retrieval. The method only uses what the run itself paid for. This makes the comparison with ACE fair, because both methods just change the context on the same base model. It also means the layer can be added to any LLM optimizer without changing the generator or the oracle.
  2. The ablations test the obvious alternative explanations. The trace display is checked both ways. It is taken out of OS-DEV, and it is also added to OPRO. This is the right way to tell the representation apart from the distillation. Table 11 also checks whether the choice of incumbent is doing the work.
Critical weaknesses
  1. The verifier is credited with the largest single-component effect, but there is no mechanism that could produce it. Nothing is filtered out. Skills with low confidence stay in the prompt, and skills whose verification fails to parse are kept at medium. So the only thing the verifier changes is one word appended to each bullet, and that word is written by the same model that wrote the skill, under a different prompt. The paper itself cites work showing that self-reported confidence is unreliable, then builds its main component on exactly that. If the ablation really is measuring the label and not something else, I would expect the effect to be small. If it is large, something other than the label is changing when the stage is removed, and the paper does not say what.

  2. The trace is not really a contrast. The paper states that the model generates freely, so a candidate is usually not an edit of the incumbent it is paired with. The two molecules can be unrelated. The analyst is then asked why the second one scored better than the first, which means the structural difference it picks out is chosen after the fact from a pair that was never a controlled comparison. The verifier is supposed to catch bad hypotheses, but it checks them against more pairs of the same kind, so a pattern that appears across several unrelated pairs will look supported. The round-start incumbent control does not help here, because it changes which incumbent is used and not whether the pairing means anything. This matters because the mechanistic story is what the domain reasoning ablation says is worth the most after the verifier.

Questions for authors
  1. Does the Claude Code + DeepSeek reference get the same filter, warmstart, and per-round cap, and how was the 200-call budget enforced inside the harness?
  2. If you shuffle the high/medium/low labels at random and change nothing else, does the 0.0198 survive? And with max_tokens=512 and a reason-then-answer prompt, how often does the verifier output fail to parse and silently default to medium?
  3. The target name is in every prompt and the verifier rewards target specificity. If you anonymize target names, how much of the gain is left? This goes straight to the claim that skills come from the run rather than from recall.
7 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer bvnj [1/5]

Dear Reviewer, thank you for recognizing the clean cold-start setting and our two-way trace ablations. Your questions helped us clarify both the purpose of the verifier and what the traces can support. We provide additional controls and implementation details below.

W1 + Q2: Confidence labels and the verifier

Thank you for raising this important point. In cold-start optimization, the early trajectory contains only a small number of observations, so some skills extracted from it may be useful while others are supported by limited or conflicting evidence. The optimizer relies on this set of skills to propose new molecules. If it treats a weakly supported skill as dependable, the guidance can steer subsequent proposals towards an unpromising direction, consuming part of the limited oracle budget and leaving fewer opportunities to explore and correct course. This makes the overall reliability of the skill set particularly important in our setting. The verifier examines each skill in light of the observations and attaches a confidence label, helping the optimizer give less emphasis to uncertain guidance (Sections 3.3–3.4, lines 246–274).

How is confidence assessed? In the self-reported-confidence setting discussed by Chen and Yi (2026), the model reports confidence in its own predictions; such confidence need not track downstream accuracy. Our analyst first proposes a skill and cites supporting traces. A separate verifier call, using the same base model but a different role and prompt, then examines the skill against available run history, including potentially conflicting observations. It considers evidence sufficiency, chemical plausibility, and target specificity (Section 3.3 and the verifier prompt in Appendix A). The label is an evidence-conditioned assessment in a separate call, rather than a confidence value emitted alongside the original skill. This separate evidence review is especially valuable when the available observations are sparse, as in our cold-start setting. Thank you for highlighting this point. We have added wording in Related Work to clarify that the cited concern is about a model’s confidence in its own predictions (Section 2.2, lines 188–189).

Why can the label matter? The confidence label helps the optimizer distinguish guidance supported by several observations from a tentative skill with limited or conflicting support. This distinction is particularly useful when each new evaluation is costly: it encourages the optimizer to use the stronger guidance while treating less certain skills with appropriate caution.

What changes in the ablation? In the submitted full-suite study, the dual analysts and trace display are retained when the verifier is disabled (Section 4.3, lines 429–431): the final mean is 0.8487 with verification and 0.8289 without it, a difference of 0.0198 across 58 targets. To examine the role of the labels, we additionally test a random-label baseline: the analyst-generated skills are assigned random high/medium/low labels without calling the verifier, with each skill independently assigned one of the three labels with equal probability. This tests whether evidence-aware labels help relative to random labels. Owing to the limited time and compute available during the discussion period, we evaluated the additional controls on seven targets (the first listed target from each protein family) with three seeds each; we apologize for the limited coverage. We will continue these experiments and include the results in the paper once completed.

The table compares OS-DEV, random labels, and no verifier. The final column shows the difference between OS-DEV and random labels.

TargetOriginal OS-DEVRandom labelsNo verifierOriginal − random
ABL10.95500.89160.94420.0634
ACHE0.95980.87450.93590.0853
AR0.73130.70510.72610.0262
ADAM170.86690.80930.84870.0576
ADORA2A0.93120.86420.92060.0670
CYP2C90.91930.89410.91050.0252
HSP90AA10.94970.90200.89290.0477
Mean0.90190.84870.88270.0532
AuthorsOpenReview ↗

Rebuttal to Reviewer bvnj [2/5]

W1 + Q2: Confidence labels and the verifier (continued)

OS-DEV exceeds random labels on all seven targets, with a mean of 0.9019 versus 0.8487 (+0.0532). On the same targets, removing the verifier gives 0.8827.

In a complementary test, we verify the skills and retain high + medium, low + medium, high only, or low only; their original labels remain visible to the optimizer. The results are:

Retained skillsRemoved skillsMean score (7 targets, 3 seeds)
High + MediumLow0.8992
Low + MediumHigh0.8664
High onlyMedium + Low0.8915
Low onlyHigh + Medium0.8521
All (original OS-DEV)None0.9019

High + medium reaches 0.8992 versus 0.8664 for low + medium; high only reaches 0.8915 versus 0.8521 for low only. These results show that the higher-confidence skill groups provide more useful guidance in these tests, supporting the effectiveness of the verifier’s confidence assessment.

Medium labels and parsing. We apologize that our description did not make the two uses of medium clear. The verifier can assign medium when the evidence is suggestive but not strong enough for high confidence (Section 3.3, lines 246–261). We also use medium as a fallback for the small fraction of responses whose labels cannot be parsed. In a GPT-assisted analysis of 8,563 verifier outputs, 12 (0.14%) were identified as unparseable and fell back to medium.

AuthorsOpenReview ↗

Rebuttal to Reviewer bvnj [3/5]

W2: What does the incumbent–candidate pairing mean?

Thank you for pointing out that our description of this reference was unclear. The candidate is freely generated. The incumbent molecule and its best score provide the models with the search context available at that point: the score indicates how the candidate performed relative to the best result so far, and the molecule provides a reference for analysis. We do not require the candidate to be a structural edit of the incumbent (Section 3.1, lines 213–223).

For a successful trace, the prompt asks which molecular properties or interactions could explain its higher composite score (Appendix A, lines 715–731). This is a score-based reference, not a requirement to construct a structural edit. In a GPT-6.1-Terra-assisted classification of 8,563 skills, 736 (8.6%) were classified as explicitly comparing the structures of two molecules. The reference can therefore provide useful search context without every extracted skill taking the form of a paired structural comparison.

This search context is useful in both directions of the original ablation: removing the trace display reduces OS-DEV from 0.8487 to 0.8357, while adding it to OPRO improves OPRO from 0.8197 to 0.8244 (Section 4.3, lines 419–422). Together, these results suggest that presenting observations relative to the current best helps the optimizer use its online history.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.