acceptodds
Under review as a conference paper at ICLR 2027

MolDeTox: Evaluating Language Model’s Stepwise Fragment Editing for Molecular Detoxification

Abstract

Large language models (LLMs) are increasingly used for molecule design and optimization, but systematic evaluation of their capability for toxicity mitigation remains limited. We present MolDeTox, a benchmark for evaluating reference-based molecular detoxification across multiple fragment-level editing tasks. MolDeTox is built on ToxicityCliff, a collection of closely related molecular pairs that differ in toxicity for the same endpoint. These examples expose small structural changes associated with reduced toxicity while minimizing unrelated changes to the molecule. We convert these pairs into three QA tasks covering fragment identification, fragment substitution, and final molecule generation. This structure enables step-wise analysis of model behavior across the detoxification process. Our analysis further identifies common failure cases and highlights potential directions for improving performance. We also assess whether the benchmark provides a useful training signal. Fine-tuning a small language model on the training split leads to substantial gains on MolDeTox. The fine-tuned model also improves molecular repair success under an independent oracle-based evaluation setting. Together, these results show that MolDeTox serves not only as an evaluation benchmark, but also as a useful resource for developing and analyzing molecular detoxification systems. We release the evaluation code and dataset.

Primary area:
datasets and benchmarks
Keywords:
Molecular Toxicity, Molecular Optimization, Detoxification, Benchmark, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZduoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MolDeTox, a benchmark for evaluating stepwise molecular detoxification using LLMs. The authors construct ToxicityCliff, containing 52,600 structurally similar toxic/non-toxic molecular pairs across 49 toxicity endpoints. Based on SAFE representations, they formulate three tasks: toxic fragment identification, non-toxic fragment generation, and final molecule generation. Experiments across multiple LLMs reveal difficulties in fragment editing and frequent input copying. Supervised fine-tuning improves reference-based performance and shows additional gains under an external toxicity-predictor-based evaluation. The main contribution is a benchmark providing intermediate supervision for molecular detoxification.

Critical strengths
  1. Fine-grained evaluation of molecular detoxification. The three-task formulation enables more detailed analysis than evaluating only the final molecule. It helps identify failures in fragment localization, replacement, and molecule reconstruction. The analysis of input copying is particularly useful, showing that high molecular validity or similarity does not necessarily indicate successful editing.
  2. Evidence of training utility. Fine-tuning substantially improves performance across the benchmark. The external ToxiEval evaluation further shows improvements in Safety (11.5% to 42.2%) and Success (4.8% to 16.5%) for the reported Qwen3-4B configurations. These results provide encouraging evidence that the training data can support molecular editing beyond exact reference recovery, although the external evaluation relies on predicted toxicity.
Critical weaknesses
  1. Validity and ambiguity of the benchmark targets. The toxicity-associated fragment labels are derived from structural differences between molecular pairs with opposite toxicity labels. However, these differences do not necessarily identify the fragments responsible for toxicity. This raises concerns about whether Task 1 evaluates toxicity-related structural understanding or simply recovery of reference-specific fragment differences. Furthermore, the same toxic molecule may have multiple non-toxic counterparts, whereas evaluation relies on matching a designated reference. Valid alternative edits may therefore be penalized, potentially affecting reported accuracy and the stepwise failure analysis. The paper would benefit from quantifying this ambiguity and evaluating against multiple valid references.
  2. Generalization and attribution of training improvements. The scaffold split is performed separately for each toxicity endpoint using the toxic molecule's scaffold. It remains unclear whether molecules or closely related structures overlap across training and test sets through different endpoints or paired counterparts. Moreover, the fine-tuning experiments do not fully isolate the contribution of intermediate supervision. All Tasks and Only Task 3 use the same number of epochs but may differ in training budget. In Table 2, All Tasks performs worse than Only Task 3 on multi-fragment generation without CoT (2.29% versus 3.39%). The external evaluation also omits Task 3-only variants, making it difficult to distinguish the benefit of stepwise supervision from additional training and CoT.
Questions for authors
  1. Benchmark validity: How frequently do toxic molecules have multiple non-toxic references for the same endpoint? Would multi-reference evaluation change the model rankings or failure analysis? What evidence supports the reliability of the automatically derived toxicity-associated fragment labels?
  2. Data independence: What molecular and scaffold overlap remains between training and test sets when considering all endpoints and both sides of each molecular pair?
  3. Training effectiveness: Does joint training on all three tasks still outperform Task 3-only training under matched training budgets? Could the authors report external ToxiEval results for the Task 3-only variants to better establish the contribution of intermediate supervision?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Zduo(W1 & Q1)

W1 & Q1. Target validity and reference ambiguity


Thank you for raising these concerns. We agree that structural differences between a toxic/non-toxic pair alone cannot establish that a fragment causes toxicity. Our labels identify editing locations associated with an observed toxicity difference for the same endpoint, as acknowledged in Section 8 (L480–482).

Existing evidence supporting fragment reliability. Appendix B already compares the extracted fragments with endpoint-specific structural alerts from ToxAlerts, a curated resource based on expert knowledge and published literature. Figures 5 and 6 examine the 20 most frequent toxicity-associated fragments for Ames Mutagenicity and Skin Reaction. Of these, 11 and 12 fragments, respectively, are marked as matching documented alerts, with two additional fragments in each endpoint showing ECFP4 Tanimoto similarity of at least 0.8.

This agreement with independently curated toxicity knowledge supports the chemical relevance of the extracted targets. Task 1 therefore evaluates localization of observed editing differences with supporting evidence of toxicity relevance. The analysis covers frequent fragments in two aligned endpoints, and establishing the causal contribution of each fragment would require further evidence. We will bring this existing validation into the main discussion and clarify the intended interpretation of the labels.

Reference ambiguity in the final test set. For each of the 1,976 final test combinations, defined by dataset, endpoint, and canonical toxic-input structure, we counted distinct non-toxic counterparts in the original pair corpus before reference selection. None had multiple eligible counterparts in that corpus (0/1,976, 0.00%). During dataset construction, combinations with multiple counterparts were assigned entirely to training. Thus, the final test targets are unambiguous within the curated pair corpus.

Additional valid edits may exist beyond the curated pair corpus. We therefore added multi-reference exact-match accuracy (MR-Acc) for Tasks 2 and 3, retaining the designated reference and additionally crediting eligible structures with public non-toxic labels for the same endpoint. Experimental results for new metric can be found in our separate official comment, [To All Reviewers]

Effect on model rankings. Multi-reference evaluation changes some comparisons. In multi-fragment Task 2, All Tasks SFT and GPT-5.2 with four-shot prompting differed by only 0.02 percentage points under Acc., while four-shot prompting performs better under MR-Acc. In Task 3, All Tasks w/ CoT retains the highest Acc. and MR-Acc in both fragment settings. Reporting both metrics therefore exposes reference sensitivity while showing which performance conclusions persist after publicly supported alternatives are accepted.

Effect on the step-wise failure analysis. We re-scored the same three inference runs used in Figure 3, applying MR-Acc at Steps 2 and 3 while retaining the designated fragment targets for Step 1. At Step 2, we retained original fragment-level exact matches and additionally credited eligible reconstructed molecules under MR-Acc.

Trajectory outcomeOriginal evaluation (%)MR evaluation (%)
Failed at Step 155.4755.47
Passed Step 1, failed at Step 230.2329.15
Passed Steps 1–2, failed at final reconstruction0.000.00
Complete-path success14.3015.38

Accepting publicly supported alternatives increases complete-path success by 1.08 percentage points, recovering approximately 21 additional trajectories per run. Replacement remains a substantial source of failure after successful localization, and no final reconstruction failures are observed after successful Steps 1 and 2. The downstream failure pattern therefore persists under the expanded evaluation.

Step 1 remains evaluated against the designated editing location. An average of 17.7 predictions per run receive final-answer MR-Acc credit while failing this Step 1 criterion. This quantifies a distinction raised by the reviewer: a valid alternative output may follow a different editing pathway. We therefore distinguish final-answer MR-Acc from complete-path success, which additionally requires agreement with the designated Step 1 target.

AuthorsOpenReview ↗

Response to Reviewer Zduo (W2-1 & Q2)

W2-1 & Q2. Train–test overlap and generalization

Thank you for requesting this analysis. The original scaffold split was performed separately for each toxicity endpoint using the toxic input molecule. Consequently, the same molecule can appear in training for one endpoint and in testing for another. Prior studies distinguish evaluation on previously observed compounds from evaluation on new compounds [1] and discuss cross-task overlap under task-specific splits [2, 3]. We therefore audited all endpoints jointly, comparing both molecules in each test pair against both sides of every training pair.

We identified three mutually exclusive groups:

GroupDefinitionTest pairsRatio
G1: Seen PairThe same toxic–non-toxic pair appears in training under another endpoint42221.4%
G2: Unseen Pair & Seen MoleculeThe pair is unseen, but at least one constituent molecule appears in training45122.8%
G3: Both Molecules UnseenNeither molecule appears in training1,10355.8%

To assess how training exposure relates to performance, we report Task 3 accuracy separately for each group.

Group-wise Acc.

ModelG1 Single Acc. (%)G1 Multi Acc. (%)G2 Single Acc. (%)G2 Multi Acc. (%)G3 Single Acc. (%)G3 Multi Acc. (%)All Single Acc. (%)All Multi Acc. (%)
GPT-5.2 SAFE Gen.5.320.001.900.133.850.253.780.16
Qwen3-4B-Inst.0.000.000.000.000.060.000.040.00
+ Only Task 3 SFT36.749.8011.113.055.400.5712.653.39
+ Only Task 3 w/ CoT SFT34.773.1310.361.655.750.2512.341.29
+ All Tasks SFT52.696.9419.772.168.100.1919.092.29
+ All Tasks w/ CoT SFT62.7231.8414.125.607.411.2719.589.62

All SFT configurations achieve their highest accuracy on G1, where scores may reflect reuse of previously observed edits. We therefore base the revised main evaluation on G3, excluding test pairs with either molecule present in fine-tuning. We will provide the original full-test results and the group-wise breakdown in the appendix.

We further report Acc. and MR-Acc on G3 to assess performance when neither test molecule appeared in fine-tuning.

G3 (Both Molecules Unseen) Acc. and MR-Acc.

ModelSingle Acc. (%)Single MR-Acc (%)Multi Acc. (%)Multi MR-Acc (%)
GPT-5.2 SAFE Gen.3.855.860.251.14
Qwen3-4B-Inst.0.060.060.000.00
+ Only Task 3 SFT5.406.900.572.98
+ Only Task 3 w/ CoT SFT5.757.010.250.82
+ All Tasks SFT8.109.310.192.03
+ All Tasks w/ CoT SFT7.419.251.272.85

The G3 results show that the training benefit persists after exact molecular overlap is removed. For single-fragment editing, All Tasks exceeds Only Task 3 on both Acc. and MR-Acc, and both outperform prompted Qwen3-4B-Instruct. For multi-fragment editing, All Tasks w/ CoT achieves the highest Acc. among the compared models, while Only Task 3 achieves the highest MR-Acc. These results support the utility of MolDeTox training data on unseen molecules, while showing that the preferred training configuration depends on the editing setting and metric. We address intermediate supervision under matched training conditions separately below.

We also assessed scaffold overlap using Bemis–Murcko scaffolds computed with RDKit MurckoScaffold, comparing both molecules in each test pair against both sides of all training pairs across endpoints. At least one constituent scaffold appeared in training for 997 of the 1,976 original test pairs (50.5%), compared with 139 of the 1,103 G3 pairs (12.6%). Under a generic-framework definition that abstracts atom types and bond orders, the corresponding overlap rates were 67.4% and 43.0%. Restricting evaluation to G3 therefore removes exact molecular overlap and substantially reduces scaffold overlap under both definitions. We will report these statistics and describe G3 as a molecule-disjoint evaluation subset, with its residual scaffold overlap explicitly documented.

[1] Walter et al. “Analysis of the benefits of imputation models over traditional QSAR models for toxicity prediction.” Journal of Cheminformatics, 2022.

[2] Simm et al. “Splitting chemical structure data sets for federated privacy-preserving machine learning.” Journal of Cheminformatics, 2021.

[3] Mun and Fazli. “CheMLT-F: multitask learning in biochemistry through transformer fusion.” Journal of Cheminformatics, 2026.

AuthorsOpenReview ↗

Response to Reviewer Zduo (W2-2 & Q3)

W2-2 & Q3. Contribution of intermediate supervision under matched training data

Following the reviewer’s suggestion, we trained an All Tasks variant using the same total number of examples as Only Task 3 and the same three epochs. On G3, the matched All Tasks model provides modest improvements in single-fragment Acc. and MR-Acc, but performs worse on both metrics for multi-fragment editing. Thus, allocating part of a fixed example budget to auxiliary tasks yields mixed benefits across editing settings.

ModelSingle Acc. (%)Single MR-Acc (%)Multi Acc. (%)Multi MR-Acc (%)
Only Task 3 SFT5.406.900.572.98
All Tasks SFT (matched examples, 3 epochs)6.217.530.251.33

A complementary comparison examines how intermediate targets are incorporated into Task 3 itself. All Tasks jointly trains the three tasks, while All Tasks w/ CoT additionally supervises fragment identification and replacement within the Task 3 output sequence before final molecule generation. On G3, this structured sequence improves both multi-fragment Acc. and MR-Acc, although single-fragment performance decreases slightly.

ModelSingle Acc. (%)Single MR-Acc (%)Multi Acc. (%)Multi MR-Acc (%)
All Tasks SFT8.109.310.192.03
All Tasks w/ CoT SFT7.419.251.272.85

These results distinguish auxiliary-task joint training from supervision on an explicit editing sequence. The matched-example comparison shows a limited, setting-dependent benefit from auxiliary tasks, while the sequence-based comparison supports the usefulness of generating intermediate editing decisions for multi-fragment reference recovery. We will qualify our claims accordingly and report training examples, optimizer steps, and processed tokens. The sequence-based comparison uses longer targets, so its training-token budget must also be considered when interpreting the gains.

Reviewer_CuByOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MolDeTox, a benchmark for evaluating molecular detoxification based on structurally similar toxic/non-toxic molecule pairs. It decomposes detoxification into three tasks: toxic fragment identification, non-toxic fragment generation, and final molecule generation. The authors evaluate various LLMs and demonstrate improvements through fine-tuning.

Critical strengths

The benchmark addresses a useful and practically relevant problem in molecular drug discovery.

Critical weaknesses
  • Evaluation metrics: Tasks 2 and 3 are generation tasks, but their main metrics largely measure agreement with a reference rather than actual toxicity reduction. This raises two concerns: (1) a molecule similar to the reference may still be toxic, as illustrated by the toxicity cliff phenomenon; and (2) a less similar molecule may still be non-toxic.

  • Limited baseline coverage: Tables 1 and 2 evaluate language-model-based approaches using SMILES/SAFE representations, but do not include other categories of molecular generation methods, such as graph-based models. This limits the comparison with existing molecular editing approaches.

Questions for authors
  • Can the authors also evaluate Tasks 2 and 3 using toxicity-based metrics rather than only reference-based metrics?

  • Can the authors include graph-based molecular editing or generation baselines to provide a broader comparison?

4 comments
AuthorsOpenReview ↗

Response to Reviewer CuBy (W1 & Q1)

Thank you for raising these concerns. We agree that reference similarity alone cannot establish toxicity reduction, and that molecules differing from the reference may still be non-toxic. However, our original evaluation uses exact structural matching with observed toxic/non-toxic pairs, rather than similarity alone. An exact match therefore recovers a molecule already labeled as non-toxic for the target endpoint.

1. Additional Evaluation Metrics

To address both concerns, we added two complementary metrics:

  • Multi-reference Accuracy (MR-Acc): For Tasks 2 and 3, we additionally accept alternative molecules that satisfy our editing constraints and have publicly available non-toxic labels for the same endpoint. Task 2 evaluates the molecule reconstructed from the predicted replacement fragment, while Task 3 evaluates the generated molecule directly.

  • TxGemma-Predict Success Rate: For Task 3, we additionally evaluate generated molecules using the toxicity-based Success criteria from ToxiEval [1], allowing us to assess candidates without requiring an exact reference match.

Under MR-Acc, additional valid alternatives are recognized. For example, in multi-fragment Task 2, four-shot GPT-5.2 outperforms All Tasks SFT under MR-Acc, despite their nearly identical original Acc. (0.02 percentage-point difference).

In Task 3, All Tasks w/ CoT achieves the highest Acc. and MR-Acc in both fragment settings and improves predicted Success over prompted Qwen3-4B-Instruct.

The full experimental results for Tasks 2 and 3 are available in our separate official comment, [To All Reviewers]


2. Complementary Evidence for Reference-based Evaluation

We further analyzed 343,133 Task 3 predictions pooled across 18 models and runs, excluding SIDER. Success requires TxGemma-predicted Safety, QED ≥ 0.5, SAS ≤ 6, Lipinski violations ≤ 1, and Tanimoto similarity ≥ 0.4.

Reference-match categoryPredictionsPassed Success criteriaSuccess (%)
Designated-reference match (Acc=1)4,9031,97240.22
Additional public non-toxic match (Acc=0, MR-Acc=1)3,3191,48744.80
Neither reference match (Acc=0, MR-Acc=0)334,91160,24017.99

Acc. measures recovery of the designated non-toxic reference, while MR-Acc additionally recognizes 3,319 predictions matching publicly labeled non-toxic alternatives.

These additional matches achieve a comparable composite Success rate to designated-reference matches (44.80% vs. 40.22%), providing supplementary evidence of consistency between reference-based and surrogate-based evaluations.

Meanwhile, 17.99% of unmatched predictions also satisfy the predicted Success criteria, suggesting potentially useful candidates that require further validation.

Importantly, Acc. and MR-Acc rely on existing endpoint-specific non-toxic labels, whereas Success relies on surrogate predictions. These metrics therefore provide complementary evidence rather than interchangeable measures of detoxification success.

[1] Lin, Gong, Wang et al. "Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?" KDD 2026.

AuthorsOpenReview ↗

Response to Reviewer CuBy (W2 & Q2)

Adding Graph-based Molecular Editing Baselines


Thank you for the suggestion. We additionally evaluated HierG2G[1] and Modof[2] on Task 3. These models were originally designed for molecular generation and property-specific optimization, rather than endpoint-conditioned editing across diverse toxicity endpoints.

To adapt them to our setting, we incorporated endpoint descriptions encoded by BioLORD-2023[3] and projected to 16 dimensions into their latent representations. Both models were trained on molecular pairs from the ToxicityCliff training dataset, covering 49 toxicity endpoints. We followed the original hyperparameters and used one deterministic decoding attempt per input, with the latent prior fixed to zero.

ModelSingle Acc. (%)Single MR-Acc (%)Single Success (%)Multi Acc. (%)Multi MR-Acc (%)Multi Success (%)
Graph-based Models
Modof0.000.006.140.000.008.11
HierG2G4.566.7910.612.814.0710.78
SMILES Generation
GPT-5.23.545.5120.510.191.4922.31
Qwen3-4B-Inst.0.140.325.470.000.194.72
SAFE Generation
GPT-5.23.825.1320.560.160.8120.65‾\underline{20.65}
Qwen3-4B-Inst.0.040.045.770.000.005.10
Qwen3-4B-Inst. SFT (SAFE)
Only Task 312.6514.2122.253.39‾\underline{3.39}6.65‾\underline{6.65}18.15
Only Task 3 w/ CoT12.3413.5423.49‾\underline{23.49}1.292.2916.07
All Tasks19.09‾\underline{19.09}20.25‾\underline{20.25}23.062.295.1619.08
All Tasks w/ CoT19.5821.2824.609.6211.7119.27

HierG2G outperforms zero-shot GPT-5.2 and Qwen3-4B-Instruct on Acc. and MR-Acc in both fragment settings, while Modof achieves no exact matches under our evaluation setup.

However, Only Task 3 SFT outperforms both graph-based baselines on Acc. and MR-Acc in both settings, and All Tasks w/ CoT achieves the highest scores among the compared models. These results support the effectiveness of fine-tuning small LLMs for endpoint-conditioned molecular editing.

We will include these comparisons and implementation details in the revised manuscript.

References

[1] Jin et al. "Hierarchical Generation of Molecular Graphs using Structural Motifs." ICML 2020.

[2] Chen et al. "A deep generative model for molecule optimization via one fragment modification." Nature Machine Intelligence, 2021.

[3] Remy et al. "BioLORD-2023: Semantic Textual Representations Fusing Large Language Models and Clinical Knowledge Graph Insights." Journal of the American Medical Informatics Association, 2024.

AuthorsOpenReview ↗

[Gentle Reminder]

Dear Reviewer CuBy,

We sincerely appreciate your thoughtful feedback and constructive suggestions. We have addressed your concerns in our responses and uploaded a revised manuscript incorporating additional experiments and clarifications.

As the discussion period is approaching its end, we would be grateful if you could kindly review our responses and revisions when you have a chance.

Thank you again for your time and consideration.

Best regards,

The Authors

Thanks the authors the detailed responses, which help solves some of my concerns. After careful evaluation, I would like to maintain my score.

Reviewer_GmFROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper focuses on the problem of molecular detoxification by editing molecular fragments using LLMs. This task is subdivided into three subtasks: 1) identifying the toxic fragment; 2) generating a substituted non-toxic fragment; and 3) generating the final non-toxic molecule (from my understanding, this is the end-to-end process combining the previous two steps). Thus, one of the paper's main contributions is to curate a dataset for these three subtasks from broader data sources.

The curation process is broadly abstracted as finding structurally similar molecule pairs (I believe they should share some common substructures) with very different toxicity properties, i.e., one is toxic and the other is non-toxic. Then, SAFE slicing (actually BRICS) is used to fragment the molecular pairs. By comparing the fragments from both molecules, these fragments can be categorized into common fragments shared by both molecules, non-toxic fragments, and toxic fragments. Using this process on broader data sources, the authors curate a core dataset called ToxicityCliff, which I believe is the core of the paper.

Then, based on the constructed molecular pairs and their fragments, the authors evaluate a number of LLMs under different settings (SMILES/SAFE representations, adding molecular images, and using in-context learning). From the results, it appears that even advanced LLMs perform very poorly on this task. The authors then use the curated dataset to fine-tune a Qwen3-4B model, showing substantially improved performance on this task.

Critical strengths
  1. The problem of utilizing LLMs to detoxify molecules is well motivated and very promising. In my personal view, this topic deserves to be explored.

  2. The paper is well written and neat. The figures, especially Figure 1, are informative and illustrate the whole pipeline well. I enjoyed reading this paper.

  3. Although I am not an expert in drug toxicity, the ToxicityCliff construction appears quite sound to me.

Critical weaknesses
  1. The first concern that emerges for me is whether the reported metrics/performance really reflect the performance of the LLMs on these tasks. For a toxic drug, there might be multiple different edits that lead to non-toxic drugs. However, in Tasks 2 and 3, the authors evaluate agreement/similarity with a single target molecule. I feel that this may create false alarms: the model may actually perform a good edit, but it will be rejected because the evaluation focuses on the specific target molecule. I think this is very hard to deal with at current stage.

  2. Using LLMs to solve this problem is promising, but the current setting in the paper makes me feel that this idea is not yet fully clear. I think the promising aspect of using LLMs is their strong reasoning capability, and I believe the authors agree with this because the prompt explicitly requires the LLMs to reason. However, when fine-tuning the small model, I do not think the current setting actually triggers its reasoning capability because reasoning traces appear to be lacking. The training seems more like providing a prompt and directly using the answer as the fine-tuning target. Alternatively, perhaps I missed something and the authors did use reasoning traces. If so, how these reasoning traces are curated, how are these reasoning traces audited to ensure the correctness. I think all these would be very important and valuable to the research community.

  3. Continuing from the previous point, if there are no reasoning cases, why can reasoning be triggered? Furthermore, considering that each task uses a fixed task/prompt and only the input molecules change, I feel there may be other deep-learning approaches for this problem instead of LLMs. To me, using LLMs is not yet a fully clear idea under the current setting.

Questions for authors

I believe most of my concerns have been described in the weaknesses above. Specifically, I would like to hear the authors' thoughts on the following:

  1. How can multiple possible correct answers (structures) ​​be dealt with without any wet laboratory experiments? I believe even using a surrogate predictive models is far from sufficient, as we cannot be certain whether these predictions are correct, especially for out-of-distribution data not trained in surrogate models. If this is the case, I think only Task 1 is meaningful to the research community.

  2. If no reasoning is involved in the model training process, how do the authors justify the necessity of an LLM for this task?

5 comments
AuthorsOpenReview ↗

Response to Reviewer GmFR (W1 & Q1)

Thank you for the comment. We agree that multiple valid edits may exist, and single-reference evaluation may overlook them. However, our reference-based evaluation uses observed toxic/non-toxic molecular pairs, meaning that an exact match recovers a molecule already labeled as non-toxic for the target endpoint. This provides an evidence-grounded measure of structural edit recovery, rather than relying solely on predicted toxicity.

To address the limitation of single-reference evaluation, we added two complementary evaluations:

1. Multi-Reference Accuracy (MR-Acc) for Tasks 2 and 3.

We now accept alternative molecules with publicly available non-toxic labels for the same endpoint. For Task 2, we evaluate the molecule reconstructed from the predicted fragment, while Task 3 evaluates the generated molecule directly.

With MR-Acc, additional valid alternatives are recognized. Importantly, All Tasks w/ CoT still achieves the highest accuracy in Task 3, confirming that the benefit of fine-tuning remains under the expanded evaluation.

2. TxGemma-Predict Success Rate for Task 3.

We also evaluate generated molecules using TxGemma-Predict, following [1]. All Tasks w/ CoT outperforms prompted Qwen3-4B-Instruct in both fragment settings, further supporting the effectiveness of our training data.

However, toxicity predictors can be unreliable for novel molecules, especially near toxicity cliffs. We therefore treat TxGemma-based Success as a supplementary metric, not experimental evidence of detoxification.

Overall, these additional evaluations address alternative valid edits while preserving the original goal of MolDeTox: evaluating whether LLMs can identify and reproduce structural changes associated with observed toxicity differences.

The full experimental results for Tasks 2 and 3 are provided in our separate official comment, [To All Reviewers]. We will incorporate these results and clarify the limitations of each metric in the revised manuscript.

[1] Lin, Gong, Wang et al. “Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?” KDD 2026.

AuthorsOpenReview ↗

Response to Reviewer GmFR (W2 & W3 & Q2)

Thank you for raising these concerns and the additional question. We clarify the role of CoT supervision, why we use LLMs, and provide additional graph-based baselines.

1. Structured supervision (W2).

Our SFT targets do not include natural-language reasoning traces (step*_reasoning). Instead, the "w/ CoT" variants learn a structured sequence of toxic fragment identification, replacement fragment generation, and final molecule construction. These targets are derived from the structural differences between observed toxic/non-toxic molecular pairs.

This stepwise structure also helps us analyze where errors occur. As shown in Section 5, Table 3, and Figure 3, models may identify the correct fragment but fail to generate the appropriate replacement. We will clarify the supervision terminology and provide the actual SFT format and an example in the revision.

2. Why LLMs? (W3 & Q2)

We focus on LLMs because they can leverage pretrained knowledge and natural-language instructions to perform molecular editing across diverse toxicity endpoints within a unified framework. Unlike models that require task-specific training, LLMs can be evaluated in zero-shot and few-shot settings, allowing us to investigate their generalization and adaptation capabilities. Across 49 toxicity endpoints, MolDeTox examines both these capabilities and the improvements achieved through fine-tuning. We use fixed prompt templates to ensure consistent comparisons across models.

3. Graph-based baselines (W3 & Q2).

We additionally evaluated HierG2G [1] and Modof [2] on Task 3. These models were originally designed for molecular generation and property-specific optimization, rather than endpoint-conditioned editing across diverse toxicity endpoints. To adapt them to our setting, we incorporated endpoint descriptions encoded by BioLORD-2023[3] and projected to 16 dimensions into their latent representations. Both models were trained using molecular pairs from the ToxicityCliff training dataset. We followed their original hyperparameters and used one deterministic decoding attempt per input, with the latent prior fixed to zero.

ModelSingle Acc. (%)Single MR-Acc (%)Single Success (%)Multi Acc. (%)Multi MR-Acc (%)Multi Success (%)
Graph-based Models
Modof0.000.006.140.000.008.11
HierG2G4.566.7910.612.814.0710.78
SMILES Generation
GPT-5.23.545.5120.510.191.4922.31
Qwen3-4B-Inst.0.140.325.470.000.194.72
SAFE Generation
GPT-5.23.825.1320.560.160.8120.65‾\underline{20.65}
Qwen3-4B-Inst.0.040.045.770.000.005.10
Qwen3-4B-Inst. SFT (SAFE)
Only Task 312.6514.2122.253.39‾\underline{3.39}6.65‾\underline{6.65}18.15
Only Task 3 w/ CoT12.3413.5423.49‾\underline{23.49}1.292.2916.07
All Tasks19.09‾\underline{19.09}20.25‾\underline{20.25}23.062.295.1619.08
All Tasks w/ CoT19.5821.2824.609.6211.7119.27

HierG2G outperforms zero-shot GPT-5.2 and Qwen3-4B-Instruct in both fragment settings on Acc. and MR-Acc. However, Only Task 3 SFT outperforms both graph-based models, while All Tasks w/ CoT achieves the best overall results. These findings support the effectiveness of fine-tuning small LLMs for endpoint-conditioned molecular editing.

We will include these results and clarify the SFT targets and evaluation settings in the revised manuscript.

References

[1] Jin et al. "Hierarchical Generation of Molecular Graphs using Structural Motifs." ICML 2020.

[2] Chen et al. "A deep generative model for molecule optimization via one fragment modification." Nature Machine Intelligence, 2021.

[3] Remy et al. "BioLORD-2023: Semantic Textual Representations Fusing Large Language Models and Clinical Knowledge Graph Insights." Journal of the American Medical Informatics Association, 2024.

AuthorsOpenReview ↗

[Gentle Reminder]

Dear Reviewer GmFR,

We sincerely appreciate your thoughtful feedback and constructive suggestions. We have addressed your concerns in our responses and uploaded a revised manuscript incorporating additional experiments and clarifications.

As the discussion period is approaching its end, we would be grateful if you could kindly review our responses and revisions when you have a chance.

Thank you again for your time and consideration.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

[To All Reviewers] Additional Evaluation Metric: Multi-Reference Accuracy and Predicted Success Rate

We thank all reviewers for their valuable feedback regarding the limitations of reference-based evaluation.

To address these concerns, we extended our evaluation with two complementary metrics: Multi-Reference Accuracy (MR-Acc) and TxGemma-Predict–based Success Rate.

  • Multi-Reference Accuracy (MR-Acc): In addition to the designated reference, MR-Acc accepts alternative molecules that satisfy our editing constraints and have publicly available non-toxic labels for the same toxicity endpoint. For Task 2, we evaluate the molecule reconstructed from the predicted replacement fragment, while Task 3 evaluates the generated molecule directly.
  • Success Rate: Following the ToxiEval framework [1], we evaluate generated molecules using TxGemma-Predict for toxicity assessment, together with molecular quality constraints (QED ≥ 0.5, SAS ≤ 6, Lipinski violations ≤ 1, and Tanimoto similarity ≥ 0.4).

Task 2 — Acc. / MR-Acc.

ModelSingle Acc. (%)Single MR-Acc (%)Multi Acc. (%)Multi MR-Acc (%)
GPT-4o5.737.390.230.81
GPT-5.28.1110.400.292.00
↳ w/ Image8.5411.510.231.88
↳ 4-Shot21.2623.284.828.80
Gemini 3.1 Flash-Lite3.224.030.000.45
Qwen3-235B-A22B-Inst.6.337.810.101.23
DeepSeek-Llama-70B3.364.900.100.68
Qwen3-30B-A3B-Inst.1.522.120.000.03
Gemma-3-27B2.183.310.001.33
Qwen3-8B1.942.190.190.38
Llama-3.1-8B-Inst.0.561.020.000.90
Qwen3-4B-Inst.0.851.800.000.00
↳ w/ Image0.140.880.000.68
↳ 4-Shot7.968.870.530.53
Qwen3-4B-Inst. SFT
↳ All Tasks28.8830.894.846.91

Task 3 — Acc. / MR-Acc. / Success Rate

ModelSingle Acc. (%)Single MR-Acc (%)Single Success Rate† (%)Multi Acc. (%)Multi MR-Acc (%)Multi Success Rate† (%)
SMILES Generation
GPT-5.23.545.5120.510.191.4922.31
DeepSeek-Llama-70B2.404.0817.220.451.9516.03
Qwen3-8B0.430.964.600.361.525.22
Qwen3-4B-Inst.0.140.325.470.000.194.72
SAFE Generation
GPT-4o1.822.7219.530.130.6821.96
GPT-5.23.825.1320.560.160.8120.65
↳ w/ Image3.734.8020.170.291.0122.03
↳ CoT3.684.7020.300.261.0721.73
↳ 4-Shot15.9918.2822.023.176.0421.52
Gemini 3.1 Flash-Lite0.453.003.500.261.6810.47
Qwen3-235B-A22B-Inst.1.522.7621.270.100.7720.66
DeepSeek-Llama-70B2.373.5018.970.390.9719.37
Qwen3-30B-A3B-Inst.0.740.8517.600.000.0015.64
Gemma-3-27B1.651.9817.010.060.1917.76
Qwen3-8B0.320.3516.620.160.1615.47
Llama-3.1-8B-Inst.0.000.180.640.000.130.85
Qwen3-4B-Inst.0.040.045.770.000.005.10
↳ w/ Image0.000.1416.410.050.1914.65
↳ CoT1.451.8017.910.100.5518.87
↳ 4-Shot3.475.2614.890.530.699.09
Qwen3-4B-Inst. SFT
↳ Only Task 312.6514.2122.253.396.6518.15
↳ Only Task 3 w/ CoT12.3413.5423.491.292.2916.07
↳ All Tasks19.0920.2523.062.295.1619.08
↳ All Tasks w/ CoT19.5821.2824.609.6211.7119.27

† SIDER samples are excluded only from Success Rate because TxGemma-Predict does not support SIDER endpoints.

Key findings. MR-Acc credits additional public-label-supported alternatives. In Task 2, All Tasks SFT leads three metrics, while GPT-5.2 4-Shot leads multi-fragment MR-Acc. In Task 3 SAFE generation, All Tasks w/ CoT leads Acc. and MR-Acc in both settings and improves predicted Success over prompted Qwen3-4B-Instruct.

[1] Lin, Gong, Wang et al., Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?, KDD 2026.

AuthorsOpenReview ↗

[To All Reviewers] Summary of Manuscript Revisions

Dear Reviewers,

We sincerely thank the reviewers for their constructive feedback and valuable suggestions. We have uploaded a revised version of our manuscript, incorporating additional experiments and evaluation metrics to address the concerns raised during the discussion.

The major revisions are summarized below:

1. Expanded Evaluation Metrics (MR-Acc. and Success Rate)

To address the limitations of single-reference evaluation, we introduce Multi-Reference Accuracy (MR-Acc.) for Tasks 2 and 3, which additionally recognizes alternative molecules with publicly available non-toxic labels for the same toxicity endpoint, subject to the editing constraints.

We also introduce an auxiliary oracle-based Success Rate, following ToxiEval, to evaluate generated molecules beyond reference matching. This metric considers predicted non-toxicity, drug-likeness, synthetic accessibility, Lipinski violations, and structural similarity to the input molecule. We have updated the main results and evaluation details accordingly.

2. Additional Graph-Based Molecular Optimization Baselines

To broaden the baseline comparisons, we additionally evaluate Modof and HierG2G on Task 3. We adapt these models for endpoint-conditioned molecular generation using natural-language toxicity endpoint descriptions and train them on the same ToxicityCliff training split used for SFT. The corresponding results and implementation details have been added to the manuscript.

We believe these revisions strengthen the evaluation framework and broaden the experimental comparisons of MolDeTox.

We sincerely appreciate the reviewers' time and thoughtful feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.