acceptodds
Under review as a conference paper at ICLR 2027

CovMOLAgent: Execution-Grounded Policy Learning for Multi-Objective Covalent Molecule Optimization

Abstract

Large language models (LLMs) have shown considerable promise in chemistry, yet existing paradigms mainly focus on the design and single-objective optimization of chemical molecules. Multi-objective optimization of a given covalent molecule presents two challenges. First, tool availability does not equate to strategic tool use: agents struggle to recover from failed edits or decide when to stop, resulting in brittle search trajectories. Second, multi-objective search tends to improve molecular properties at the cost of destroying the key warhead that determines the covalent mechanism, while heterogeneous property scales make coordinated improvement difficult. To address these challenges, we propose Covalent Multi-Objective Lead Optimization Agent (CovMOLAgent), an agentic framework for multi-objective covalent lead optimization. Specifically, we first introduce Covalent Executable Trajectory Reconstruction and Alignment (CETRA), an execution-grounded trajectory reconstruction strategy. CETRA constructs failure-aware search trajectories based on deterministic tool executions, covering behaviors such as invalid trials and negative explorations, thereby grounding decisions in executable evidence rather than hypothesized molecular states. We further develop Warhead-Constrained Policy Optimization (WCPO), a source-relative multi-objective optimization mechanism. It incorporates direction-normalized multi-property rewards and a source-warhead penalty, among other components, to reconcile heterogeneous property scales and favor retention of source-warhead labels. Comprehensive evaluations on a held-out benchmark show that CovMOLAgent achieves the strongest aggregate and joint multi-property performance, with leading results on several individual-property metrics, while retaining all recognized source-warhead labels in 90.9% of benchmark instances.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Covalent Molecular Optimization, AI Agents for Science, LLM Post-Training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qjd9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CovMOLAgent, an agentic framework for multi-objective covalent lead optimization. CovMOLAgent comprises CETRA, an execution-grounded trajectory construction strategy based on deterministic tool execution, and WCPO, a warhead-constrained policy optimization objective. CovMOLAgent shows superior performance to other general LLMS without any fine-tuning.

Critical strengths

Comprehensive experiments: The experiments are sufficiently comprehensive, including joint optimization and property changes (I personally think Table 1 and 2 do not really need to be that large tables, though), a case study, and an ablation study. Specifically, the ablation study in Table 3 makes each component’s contribution clearly visible (WCPO and CETRA).

Critical weaknesses
  • Novelty based on a conjunction of established elements: The paper claims three-fold contributions: source-conditioned optimization, multi-property improvement relative to the starting molecule, and retention of source warheads. Source-conditioned editing that modifies a local site while retaining the rest of the molecule under similarity and property constraints is established in fragment-based optimization [2], and substructure-constrained multi-property objectives already appear in deco_hop, scaffold_hop, and valsartan_smarts of the PMO benchmark [1]. What remains specific is that the retained label set varies per source, but this is a benchmark design choice rather than a methodological advance.
  • The covalent constraint is a simple add-on rather than a design principle: Warhead preservation enters at two points: 1) the feasibility predicate that filters CETRA’s candidate neighborhood and 2) the penalty term in the WCPO reward. Both would accept any substructure constraint unchanged other than covalent. CETRA, which performs one-step edits, caches the outcomes, and reconstructs them into failure-aware trajectories, is agnostic to covalency and applies to tool-based molecular editing in general. The paper should state which component is specific to covalent chemistry.

[References]

[1] Gao, W., et al. Sample efficiency matters: a benchmark for practical molecular optimization. NeurIPS 2022.

[2] Lee et al. Molecule Generation with Fragment Retrieval Augmentation. NeurIPS 2024.

Questions for authors
  • How does CovMOLAgent work compared to other fragment-based or covalent-based methods? I understand that other covalent-based methods are not applicable to the same task as they are mostly de novo, but fragment-based approaches' setting are exactly same so it is still applicable.
  • What would be the significant design principle difference for covalent lead optimization compared to other fragment-based or lead optimization tasks?
3 comments

W1, Q1: Comparison with Fragment Optimization, PMO, and Other Non-LLM Methods

Thank you for pointing out these related works. The key distinction lies in what is learned: CovMOLAgent converts failures, property trade-offs, and endpoint verification from actual tool executions into policy supervision, teaching the agent how to choose subsequent actions and when to stop on new inputs. We clarify its relationship to prior work and validate the benefits through additional experiments.

Relationship to Gao et al.'s PMO (NeurIPS 2022). [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization] evaluates molecular optimization by oracle-query efficiency, covering 23 scalar objectives and primarily measuring the AUC of top-10 scores over query counts. The reviewer's examples, deco_hop, scaffold_hop, and valsartan_smarts, indeed incorporate substructure requirements. We study source-conditioned policy execution: defining a preservation set for each source, measuring property changes relative to that source, and evaluating simultaneous multi-property improvement under structural constraints. PMO provides an evaluation principle for query efficiency, whereas CETRA/WCPO train deployment-time search policies from execution experience. Our contribution is therefore policy learning and measurable query-efficiency gains, not substructure constraints themselves.

Relationship to Lee et al.'s f-RAG (NeurIPS 2024). [Molecule Generation with Fragment Retrieval Augmentation] uses hard fragments for molecular construction and soft fragments to guide new-fragment generation. It freezes SAFE-GPT, trains a fragment-injection module, and combines dynamic vocabulary updates with genetic modification. Its core mechanism is learning how retrieved fragments enter a molecular generator. In contrast, CovMOLAgent learns how tool actions should respond to execution observations: CETRA fixes actual trial outcomes before reconstructing selection, adjustment, and verification, while WCPO trains the tool policy using source-relative endpoint gains. Thus, f-RAG focuses on fragment generation and vocabulary expansion, whereas CovMOLAgent focuses on execution-driven action and stopping-policy learning.

To compare with online molecular search, we added Graph-GA, STONED, BRICS fragment retrieval, and MIMOSA-GNN proposal-adaptation baselines (settings in Reviewer 2, W1/Q1). The BRICS baseline uses 1,985 single-attachment-point fragments built only from training inputs. It retrieves replacements by attachment type, size, and structural similarity, with iterative updates and best-so-far selection. We report it as implemented, rather than as the complete f-RAG framework. Results are shown below (percentages):

Method and query settingJIR≥3JIR_QSTAvg. property queries / CovMOLAgent
BRICS, matched total property-query budget3.46 ± 1.052.82 ± 1.240.98×
BRICS, 20 candidate-query budget14.76 ± 0.5311.02 ± 0.377.78×
Graph-GA, 20 candidate-query budget22.60 ± 1.0914.41 ± 0.568.04×
CovMOLAgent24.7920.971.00×

With matched total property-query budgets, CovMOLAgent achieves 20.97% JIR_QST versus 2.82% for BRICS. Even with approximately eight times as many queries, BRICS and Graph-GA reach only 11.02% and 14.41%, respectively. Against the higher-budget Graph-GA, the QST difference is 6.57 percentage points, with a source-paired bootstrap 95% CI of [2.47, 10.81] (Reviewer 2, W1/Q1). This supports higher three-property simultaneous improvement with fewer online queries. Paired comparisons on the complete pre-filtering held-out set show the same advantage (Reviewer 1, W1/Q3).

The results are consistent with our training and behavioral analyses. Complete-trajectory supervision achieves higher mean utility (+0.001621 vs. −0.040219) and a lower failure rate (39.62% vs. 46.61%) than winner-only supervision (Reviewer 2, Table A). The example in Reviewer 1, Q2, further shows the agent switching strategies after execution failure, identifying property degradation, and stopping after verification. Together, these comparisons support our contribution: converting offline execution experience into a multi-objective optimization policy that works with limited online property queries.

W2, Q2: Which Designs Are Specific to Covalent Molecules, and Which Mechanisms Are Generalizable?

The design principle for covalent optimization is to treat reaction-relevant structures in each source molecule as instance-specific preservation targets, so that property gains and preservation requirements jointly guide data construction, policy training, and endpoint evaluation. Global similarity captures overall resemblance, while local reaction-related fragments require explicit checks. Candidate selection must therefore consider both the source-specific preservation set and source-relative property changes. Our framework connects these requirements at three levels:

LevelSpecific design in our workRole in the optimization process
Data constructionExtract source-specific recognition labels, identify winners within feasible neighborhoods, and retain failed/non-improving trialsSupervises candidate selection under structural constraints and property trade-offs
Policy trainingCETRA reconstructs action–observation–adjustment; WCPO combines source-relative utility with label-loss and degradation penaltiesLearns subsequent tool actions and endpoint selection from execution experience
Endpoint validationJoint Full-WH/JIR evaluation, plus instance-count, local-connectivity, and paired-adduct auditsAssesses property gains, source-structure preservation, and local geometric compatibility

The chemistry-specific component is the identification, preservation, and validation of reaction-relevant structures for each source; CETRA/WCPO's execution-learning mechanisms are more general. Unlike PMO's fixed-task substructure scoring or f-RAG's hard fragments for molecular construction, we define a preservation set and property reference for each complete source, then learn to adjust actions in that task context. Source labels guide CETRA neighborhood filtering, label-loss penalties guide policy training, and Full-WH/JIR jointly evaluate structural preservation and property improvement.

Our experiments support this design at two levels. At the decision level, after removing detailed recovery and stopping instructions, CovMOLAgent retains a JIR≥2 of 51.48%, versus 25.64% for the base model with the same simplified prompt (Reviewer 2, Table B), showing that its tool-decision capability persists with fewer procedural instructions. At the structural level, among 429 Full-WH outputs, 97.44% preserve the number of source-matched instances, and 75.76% also satisfy the one-hop local-environment compatibility requirements for all labels. Across five paired covalent-adduct searches, three have a common-scaffold RMSD≤2 Å, and four preserve at least 85% of source-contacting residues (Reviewer 1, W3). These results support learned execution capability, source-specific local-structure preservation, and geometric compatibility under the tested adduct hypotheses.

Therefore, our contribution connects source-specific chemical requirements, execution-experience supervision, and low-query policy optimization, supported by joint improvement, query efficiency, and structural audits.

Thank you for the response. The authors state that CETRA and WCPO are general, which confirms rather than resolves my concern. My question asked what design principle separates covalent lead optimization from general fragment-based lead optimization, and the answer is just the definition of fragment-based lead optimization itself. I appreciate the new fragment-based baselines (that handled weakness 1) but the concern 2 remains unresolved, so I maintain my rating.

Reviewer_4DCSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CovMOLAgent, a tool-augmented framework for multi-objective covalent molecular optimization. The proposed method consists of CETRA, which constructs execution-grounded trajectories for supervised fine-tuning, and WCPO, which applies GRPO with multi-objective rewards and warhead-preservation penalties. The authors also introduce CovaOpBench, a benchmark of 472 molecular optimization tasks. Experiments show improvements in multi-property optimization, failure rates, and warhead retention compared with several general-purpose and domain-specific language models.

Critical strengths
  1. The paper addresses a practical and meaningful problem in covalent molecular optimization, where improving multiple molecular properties while retaining critical warhead structures is challenging.

  2. The execution-grounded trajectory construction is a reasonable approach to improving tool-use reliability. The reported improvements in optimization success and warhead retention are encouraging within the proposed evaluation setting.

Critical weaknesses

W1. Limited benchmark evaluation and insufficient comparisons.

The evaluation relies primarily on a self-constructed benchmark of only 472 positive-filtered tasks, where each source molecule is guaranteed to have an improving candidate within the predefined editing vocabulary. This raises concerns about generalizability, especially without evaluation on established molecular optimization benchmarks or independent test settings. Furthermore, despite comparing numerous general-purpose LLMs, the paper lacks comparisons with closely related molecular optimization methods, including ChemCRAFT, ToolMol, and C-MORAL. A task-specifically trained direct-generation baseline is also missing, making it difficult to justify the additional complexity and execution cost of the proposed agentic framework.

W2. Unclear methodological novelty and insufficient ablations.

The related work omits numerous relevant studies, including MolSearch, MolAct, RePO, POLO, MolMem, TRACE, MolReAct, and AutoLead, making it difficult to distinguish the proposed contributions from existing molecular optimization, agentic RL, and trajectory-learning approaches. The ablation studies are also insufficient to isolate the contributions of CETRA's key components, including failure-aware trajectories, rationale generation, and trajectory reconstruction. Moreover, the evaluation prompt explicitly prescribes failure recovery, candidate selection, and stopping behaviors, which are central claimed benefits of CETRA. Without more targeted ablations, it remains unclear whether these behaviors are genuinely learned or primarily reflect adherence to predefined instructions.

Questions for authors
  1. Can the authors provide comparisons with representative molecular optimization methods, particularly a direct-generation baseline using the same backbone and comparable task-specific training data?

  2. How well does the proposed method generalize beyond the positive-filtered CovaOpBench and its predefined editing vocabulary? Can the authors provide results on independent or established molecular optimization benchmarks?

  3. Can the authors provide more fine-grained CETRA ablations, particularly removing failure-aware examples, GPT-4-generated rationales, and explicit recovery/stopping instructions from the prompt?

  4. Can the authors provide a detailed comparison with closely related works, including MolSearch, MolAct, RePO, POLO, MolMem, TRACE, MolReAct, and AutoLead? In particular, what are the fundamental methodological differences between CETRA/WCPO and existing approaches to trajectory learning, agentic RL, and multi-objective molecular optimization? Which contributions are genuinely novel beyond adapting existing techniques to covalent molecules?

  5. Since warhead retention is evaluated only through label-level SMARTS matching, how can the authors establish that the original covalent reaction mechanism is preserved? Can they provide broader structural or docking validation beyond the single reported example?

5 comments

W1 (Baselines), Q1: Additional Molecular-Search and Pretrained Molecular-Model Comparisons

We add four categories of molecular-search/proposal baselines and a direct-generation SFT baseline using the same backbone and the same source-winner data to separately examine online search and endpoint supervision.

The search baselines include public Graph-GA, STONED operators, BRICS retrieval based on training fragments, and a frozen proposal adapter using publicly available MIMOSA-GNN weights. Each method starts from the same source molecule and uses the same four-property evaluator and source-label-aware utility to select the best-so-far candidate. BRICS fragments are derived only from the training inputs. Each category covers all 472 inputs, uses three search random seeds, and has a maximum candidate property-query budget of 20. We report results under both a query limit matched to the agent and a relaxed search budget. These methods generate new candidates and update their search states, extending beyond the original neighborhood-cache comparisons.

We first match the total online property-query budget. The agent's main experiment performs 1,230 property analyses. For the baselines, including the initial analysis of each source, we use a SHA256-based blind ordering of the sources to allocate two candidate queries to 286 inputs and one candidate query to 186 inputs, giving the same total budget of 1,230. The allocation is identical across methods and seeds and independent of outcomes.

MethodActual mean property queries, including sourceJIR >= 3JIR_QST
Graph-GA, source-initialized adaptation2.6023.67 +/- 0.652.54 +/- 0.73
STONED, source-conditioned adaptation2.6062.47 +/- 0.241.55 +/- 0.53
BRICS fragment retrieval2.5553.46 +/- 1.052.82 +/- 1.24
MIMOSA-GNN, frozen pretrained-proposal adaptation1.7673.11 +/- 0.962.12 +/- 0.64
CovMOLAgent2.60624.7920.97

JIR values are percentages, and +/- denotes the sample standard deviation across the three baseline search seeds. CovMOLAgent uses the outputs from the main experiment. Actual queries may be fewer if candidates are exhausted or unsupported by the vocabulary. We measure efficiency by online property evaluations.

Under this budget, CovMOLAgent's JIR_QST exceeds Graph-GA and BRICS by 18.43 and 18.15 percentage points. Source-paired bootstrap 95% confidence intervals are [14.83, 22.18] and [14.41, 21.89], supporting its joint-improvement advantage.

Even when the budget is relaxed to 20 candidate queries, Graph-GA achieves 22.60% / 14.41% for JIR >= 3 / JIR_QST, while CovMOLAgent achieves 24.79% / 20.97%. Graph-GA uses approximately 8 times as many property queries on average as CovMOLAgent. The QST difference is 6.57 percentage points, with a bootstrap 95% confidence interval of [2.47, 10.81] obtained by resampling paired input molecules. This further supports the advantage in simultaneous three-property improvement with fewer property queries.

This comparison quantifies the deployment benefit: a higher joint-improvement rate under the same query budget, and a QST advantage even when the baseline uses approximately eight times as many property queries.

Each method is adapted to the source-conditioned task used in this paper. MIMOSA-GNN uses a frozen pretrained proposal module. Its vocabulary supports 223 inputs; for the remaining inputs, it returns the source, which is still included in the denominator. On the 223 commonly supported inputs, under a budget of 20 candidate queries, CovMOLAgent achieves a JIR_QST of 23.32%, compared with 16.44% +/- 1.81% for MIMOSA-GNN. BRICS and MIMOSA-GNN are reported as a retrieval baseline and a frozen proposal adapter, respectively.

The same-backbone direct-generation baseline uses the same source-winner pairs but supervises only the mapping from input to final answer, without a tool-interaction loop at deployment. After 1,000 steps, it achieves a positive optimization rate of 47.25%, JIR >= 2 of 38.56%, and mean utility of -0.000646. The corresponding results for full-trajectory SFT are 47.46%, 39.62%, and +0.001621. The final CovMOLAgent after WCPO achieves 59.11%, 54.03%, and +0.137374, respectively. Compared with direct-generation SFT, its positive optimization rate and JIR >= 2 are higher by 11.86 and 15.47 percentage points, respectively. This comparison demonstrates the gains of the complete training system and shows that we evaluate both interaction initialization and subsequent policy optimization.

The new comparisons cover graph editing, SELFIES editing, fragment retrieval, pretrained graph proposals, and same-backbone direct generation. ChemCRAFT, ToolMol, and C-MORAL are discussed in the related-work response.

W1 (Benchmark Evaluation), Q2: Effectiveness Beyond Filtering

The generalization evaluation has been extended from the original benchmark to pre-filtering inputs, training-unseen scaffolds, and externally generated molecules. Matched comparisons on the complete held-out set also support the advantage in three-property joint improvement and query efficiency; the evaluation setup is described in Reviewer 1, W1/Q3. Among the 1,853 inputs from the external generation pipeline after removing training overlap, the positive optimization rate is 48.03%, Full-WH is 92.55%, and JIR >= 2 is 44.90%. Among the 75 inputs with training-unseen scaffolds, 54.67% still achieve positive utility. Together, these results under the unified four-property evaluation protocol support the applicability of our method to inputs beyond positive-outcome filtering.

W2 (Ablations), Q3: Failure Trajectories, Teacher Rationales, and Prompt Dependence

We add two sets of comparisons: Table A compares supervision content during SFT, while Table B compares inference prompts with model weights fixed. They separately analyze the effects of execution supervision and explicit procedural instructions.

Table A: SFT results with different supervision content. All four variants are initialized from the same Qwen3-14B base model, use the same 5,230 source-winner pairs and a global batch size of 8, and are evaluated on all 472 inputs. The no-teacher-rationale variant retains the original actions, observations, and endpoints. The winner-only variant retains only successful edits and validation steps leading to the final answer, removing failed and non-winning branches. Direct-answer supervision retains only the input and final molecule.

Training methodAll four improved>=3 improved>=2 improvedMean utilityImproveFail
Full trajectory (ours)1.0613.7739.620.00162147.4639.62
Without teacher rationales0.004.0313.56-0.03233014.6262.29
Winner-only trajectories0.6413.0840.89-0.04021944.9246.61
Direct-answer supervision0.6413.1435.38-0.05398639.6237.50

Full-trajectory supervision achieves the highest mean utility and positive optimization rate among the listed SFT results. Compared with winner-only trajectories, its positive optimization rate is higher by 2.54 percentage points, its failure rate is lower by 6.99 percentage points, and mean utility increases from -0.040219 to +0.001621. This shows the value of the supervision design in final-submission gains and failure risks, consistent with CETRA's coverage of exploration, adjustment, and validation.

Without teacher rationales, the positive optimization rate is 14.62% and the failure rate is 62.29%; the corresponding results for full trajectories are 47.46% and 39.62%. This shows the benefit of supervision connecting observation explanations with subsequent decisions. Direct-answer supervision achieves JIR >= 2 of 35.38% and mean utility of -0.053986, compared with 39.62% and +0.001621 for full trajectories. This matches the division of roles in our method: SFT learns executable interaction and submission, while WCPO further aligns the policy with final optimization utility, as also shown by the stage-wise results in Reviewer 1, where the latter further increases positive optimization and substantially reduces numerical degradation.

Table B: Prompt comparison with fixed model weights. The original prompt includes explicit state-machine, recovery/stopping, and workflow instructions. The simplified version removes these procedural details while keeping the four-property objectives, structural requirements, tool schema, answer format, and reasoning budget unchanged. The base model and the final SFT+RL model are each evaluated on all 472 inputs under both prompts.

ModelPromptAll four improved>=3 improved>=2 improvedMean utilityImproveFail
Qwen3-14B base modelOriginal1.066.9912.71-0.17873216.9565.89
Qwen3-14B base modelSimplified1.2712.9225.64-0.43042430.5160.17
CovMOLAgent (SFT+RL)Original2.5424.3653.600.10467959.1111.86
CovMOLAgent (SFT+RL)Simplified1.9121.1951.480.06126260.1716.74

The advantage learned through training remains after removing detailed recovery and stopping instructions. Under the simplified prompt, CovMOLAgent achieves JIR >= 2 of 51.48% versus 25.64%, a positive optimization rate of 60.17% versus 30.51%, a failure rate of 16.74% versus 60.17%, and mean utility of +0.061262 versus -0.430424. Compared with the base model, joint improvement increases by 25.84 percentage points, and the failure rate decreases by 43.43 percentage points. The advantage under both prompts supports the claim that the model has acquired optimization capabilities usable with fewer procedural instructions.

Under the original prompt, the final model has higher JIR >= 3 and a lower failure rate. Under the simplified prompt, it still maintains strong positive optimization and joint improvement, demonstrating complementarity between training and instructions. All four groups in Table B come from the same prompt-comparison inference runs and use consistent statistical definitions; the main paper reports the original run. The actual training comparison of reward designs is provided in Reviewer 1, W2/Q1.

Reviewer_e1BNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CovMOLAgent, a tool-augmented LLM framework for source-conditioned multi-objective covalent molecule optimization. The proposed method combines CETRA, which reconstructs execution-grounded trajectories from molecular editing outcomes for supervised fine-tuning, and WCPO, which applies source-relative multi-property rewards and a warhead-preservation penalty during GRPO. Experiments on the newly constructed CovaOpBench report improvements in joint molecular-property optimization, failure rates, and source-warhead retention compared with several open-weight and proprietary LLM baselines.

Critical strengths

S1. Execution-grounded supervision.

The construction of training trajectories from deterministic chemical-tool executions is a reasonable approach to reducing inconsistencies between generated reasoning and actual molecular transformations. The dataset construction pipeline is substantial and provides a practical basis for tool-augmented molecular optimization.

S2. Strong empirical performance under a shared tool interface.

The proposed method achieves favorable results across multiple joint-improvement and structural-retention metrics. The inclusion of reproducibility experiments, paired comparisons, and search controls further strengthens the empirical evaluation.

Critical weaknesses

W1. Limited external validity of the constructed benchmark.

CovaOpBench is constructed by retaining molecules with at least one known improving action within the supported local-edit vocabulary. Although the authors provide useful diagnostics for the excluded molecules, the main comparisons are restricted to this positively filtered benchmark. This makes it difficult to establish whether the reported gains generalize beyond the particular tool action space and task construction procedure.

W2. Insufficient isolation of the proposed methodological contributions.

The training-stage ablations demonstrate that CETRA and WCPO are beneficial as complete components, but they do not sufficiently isolate the mechanisms claimed to be novel. In particular, the contribution of failure-aware trajectory reconstruction is not distinguished from conventional task-specific SFT. Similarly, the reward sensitivity analysis largely evaluates alternative utility configurations on fixed outputs rather than retraining policies with the corresponding components removed.

W3. Limited evidence connecting the optimization results to practical covalent lead optimization.

The main evaluation relies on computational property proxies and SMARTS-based warhead-label retention. Although these metrics are useful, they do not directly establish preservation of covalent reactivity, target engagement, or experimental feasibility. The single docking case provides preliminary structural evidence but is insufficient to substantiate broader chemistry-level claims.

Questions for authors

Q1. Contribution of failure-aware supervision in CETRA.

As mentioned by the authors in the Introduction as a key motivation, existing agents struggle to recover from failed edits and make appropriate stopping decisions. CETRA explicitly incorporates failed and non-improving editing trials into reconstructed training trajectories. However, the current ablations do not isolate the benefit of these failure-aware examples from conventional supervision on successful trajectories. Could the authors compare CETRA against success-only trajectory supervision under a comparable training budget? This would help clarify whether the reported improvements arise specifically from learning failure recovery and stopping behavior rather than from task-specific SFT alone.

Q2. Qualitative evidence of learned reasoning and policy behavior.

Could the authors provide representative language outputs and tool-use trajectories from the trained CovMOLAgent, alongside those of the untuned base model or competing LLM agents on the same molecules? In particular, examples illustrating how the model interprets failed edits, balances competing molecular properties, preserves source warheads, and decides when to stop would help explain what reasoning or decision-making behaviors are actually improved through the proposed training procedure.

Q3. Reliability and generalizability of CovaOpBench.

CovaOpBench contains 472 held-out molecules selected through positive filtering based on the supported local-edit vocabulary. While the paper provides useful analyses of the excluded molecules and ensures molecule-level disjointness, the representativeness and generalizability of the benchmark remain unclear. How diverse are the benchmark molecules in terms of molecular scaffolds and warhead classes, and how sensitive are the reported results to the specific edit vocabulary and property evaluators? Could the authors further validate generalization through scaffold-disjoint or unfiltered evaluation against the main baselines? This would help establish the scope and reliability of CovaOpBench beyond the constructed optimization setting.

5 comments

W1, Q3: Benchmark Construction, Diversity, and Generalization Beyond Positive Filtering

The effectiveness of our method extends beyond positive-outcome filtering and to inputs supplied by external molecule-generation pipelines. CovaOpBench combines a covalent molecule library with ZINC-derived molecules containing linker reaction groups. Filtering is based on pre-executed edit neighborhoods, rather than on the outputs of the model being evaluated. This design provides 472 tasks with verified opportunities for improvement, allowing the comparison to focus on whether a method can discover and complete valid optimizations. We further evaluate the complete held-out inputs before filtering and inputs from an external molecule-generation pipeline that do not overlap with either the training set or the benchmark. The main and external evaluation results are as follows:

Input setPositive optimization rateFull-WH
472 benchmark inputs, main experiment59.11%90.89%
External generation pipeline (DiffSBDD、Pocket2Mol ), 1,853 inputs after removing training overlap48.03%92.55%

For the external evaluation, we excluded one molecule overlapping with the training inputs from the original 1,854 molecules. The remaining inputs are all outside CovaOpBench. In addition to achieving 48.03% positive utility, 44.90% improved at least two properties under Full-WH, and 17.43% simultaneously improved QED, SA, and toxicity. These results show that CovMOLAgent can connect external molecule generation with subsequent optimization, balancing property gains and source-label retention on a new input distribution.

Regarding structural diversity, the 472 inputs contain 334 distinct non-empty Bemis-Murcko scaffolds, along with 25 acyclic inputs. The recognized labels cover acrylamide, cyanamide, nitrile, methyl acrylate, chloroacetamide, and others. There is no exact molecular overlap between the test inputs and the actual SFT/RL training inputs. Moreover, 75 inputs have non-empty scaffolds that do not appear among the training inputs. On this unseen-scaffold subset, CovMOLAgent achieves a positive optimization rate of 54.67%, higher than GPT-5.5, Gemini, Claude Opus 4.7, and GPT-5.6 Sol, which achieve 48.00%, 46.67%, 52.00%, and 44.00%, respectively. This supports its ability to optimize structures not seen during training.

To examine applicability beyond positive filtering, we evaluate the same checkpoint on the complete held-out inputs and conduct matched comparisons with Graph-GA and BRICS using three search random seeds. Both baselines generate new candidates rather than relying on the original cache of 25 edits, and all methods use the same four-property evaluation protocol. This analysis uses one independent inference run on the complete set, with all methods evaluated on the same inputs. CovMOLAgent still outperforms both baselines in simultaneous improvement of QED, SA, and toxicity, while using fewer online property queries. This supports the extension of its joint optimization gains to inputs before filtering.

W2, Q1: Failure Supervision in CETRA and the Role of WCPO

CETRA supervision covers the actual success or failure of executed actions, property trade-offs, subsequent adjustments, and endpoint validation. The teacher generates rationales around fixed execution records. We analyze its role from three perspectives: training stages, supervision variants, and actual decision-making behavior. First, the stage-wise results on the same test inputs are:

StagePositive optimization rateNumerical degradation rateMean scorable utility
Qwen3-14B before fine-tuning11.23%6.57%-0.171603
SFT-only46.19%38.98%-0.037685
SFT + WCPO59.11%9.96%0.137374

SFT increases the scorable rate from 31.36% to 94.28%. RL then increases the positive optimization rate by 12.92 percentage points, reduces the numerical degradation rate by 29.03 percentage points, and shifts mean utility from -0.037685 to +0.137374. SFT establishes executable tool interaction and answer submission, while WCPO further converts executability into higher endpoint utility and less numerical degradation. The numerical degradation rate and scorable rate together characterize this training process.

Regarding the suggestion to use only successful trajectories, the additional supervision comparison is provided in Table A of Reviewer 2, W2/Q3: among the listed SFT checkpoints, the full-trajectory model has a failure rate of 39.62%, lower than the 46.61% of the winner-only model; its mean utility is +0.001621, higher than -0.040219 for the latter. This shows the stronger overall utility and endpoint-selection reliability of full interaction supervision.

Behavior logs provide corresponding execution evidence. Among tasks with an explicit edit failure, the full model ultimately obtains positive utility in 267/443 cases, compared with 17/145 for the base model. The next section presents the actual adjustment process on the same input. This statistic describes recovery in the trajectories actually visited by each policy. Furthermore, after removing the detailed recovery and stopping instructions from the prompt, the final model still achieves 51.48% JIR >= 2, higher than the base model's 25.64% (Reviewer 2, Table B). This supports the ability of the learned decision-making behavior to operate with fewer procedural prompts.

For WCPO, we additionally provide results from models actually trained with different terminal rewards. The alternative reward uses the same four-property utility, but replaces the original formatting/validity, non-degradation, and piecewise positive/negative optimization shaping with a negative reward for an infeasible terminal output and a tanh transformation of utility otherwise. The two runs share the same SFT starting point, training/validation data, and main hyperparameters: batch=64, mini-batch=32, rollout=8, up to 20 rounds, response limit of 8,192 tokens, context limit of 25,000 tokens, learning rate of 1 x 10^-6, and KL coefficient of 0.001. The following table compares the two reward designs, with training outcomes evaluated using the same source-relative molecular utility:

Training reward designRL stepsMean scorable utilityScoredImproveFail
Feasibility gate + tanh alternative reward1500.12366796.82%62.50%13.56%
Original WCPO1500.13737494.92%59.11%15.04%
Feasibility gate + tanh alternative reward1750.10700797.03%60.38%13.56%
Original WCPO1750.12134594.92%63.77%15.04%

WCPO achieves higher mean utility at both the 150-step setting used in the main experiment and the shared final checkpoint at 175 steps, supporting its contribution to the magnitude of optimization gains. On the 445 inputs that are scorable under both methods at 150 steps, the mean utilities of WCPO and the alternative reward are still 0.139470 and 0.125709, respectively. This matched-input comparison further shows that the utility advantage remains on an identical scorable subset. This experiment compares the overall terminal-reward designs, while the score diagnosis below further explains the role of the negative-optimization term.

The score diagnosis of the negative-optimization term further explains how this design allocates training credit. On the same 47 negative-utility outputs, adding the penalty reduces the mean score from 0.400000 to 0.162974, a reduction of 59.26%; 21 outputs are clipped to zero. The scores of positive-utility outputs remain unchanged. This explains the targeted suppression of degraded endpoints by the reward and complements the actual training results.

Q2: Language Outputs and Tool Decisions on the Same Input

We use the first benchmark input as an example. The source molecule is C=CC(=O)N(CC(=O)OCC)C(=O)Nc1nc[nH]n1. The following table excerpts actual tool observations from the same task. Lower SA and toxicity values are better:

StateQEDSAToxicitySubsequent model behavior
Source0.56863.46542.611864Establishes the comparison baseline
Methyl-edit candidate0.59193.48332.814682Recognizes that the QED gain accompanies degradation in the other two properties and does not select it as the best candidate
Pyridine-edit candidate0.58623.22002.610794Retains the source labels, passes validation, is accepted, and triggers stopping

In the trajectory, ReplaceFunctionalGroup(halo->hydroxyl) first returns success=False, after which the model switches to another editing strategy. The methyl candidate is executable but introduces property degradation. The model explicitly states:

"QED improved, but SA worsened slightly and a toxicity alert appeared."

The model then calls AddFunctionalGroup(pyridine) again from the source. After analyze_covalent, normalization, similarity checks, and reaction-rule checks, it submits CCOC(=O)CN(C(=O)C=Cc1ccncc1)C(=O)Nc1nc[nH]n1, with a final utility of +0.0608. The process includes changing the strategy after an execution failure, rejecting a single-property gain that masks degradation, and reevaluating and validating the endpoint.

The base model on the same input also recognizes the toxicity issue in the methyl candidate. However, it then requests deletion of ethoxy, which is unsupported by the tool, and receives Unknown group name. After switching to deletion of ether, the candidate has a similarity of only 0.4808 to the source. The model ultimately returns the source with zero utility. This paired example shows how the model converts observations into subsequent actions, rather than merely comparing final molecules.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.