acceptodds
Under review as a conference paper at ICLR 2027

Can Anyone Make Poisons? Systematic Red-Teaming of Molecule Language Models

Abstract

Molecule language models offer an accessible interface for molecule design, but also raise potential dual-use concerns. Despite extensive research on red-teaming large language models, the safety risks of these models remain largely unexplored. In this work, we present the first systematic red-teaming study of molecule language models. We curate toxicity data spanning eight endpoints and develop specialized evaluators for systematic assessment. Our analysis further reveals that toxic molecules exhibit greater similarity to known toxic examples than non-toxic molecules. Building on this structural prior, we develop a decoding-time red-teaming strategy that learns a surrogate scorer to estimate the toxic-neighbor potential of partial SMILES and steer generation toward toxicity-associated regions. Extensive experiments across eight models show that our method substantially increases toxicity rates, yielding an average absolute improvement of up to 41.4%. Qualitative analyses further identify generated compounds with established real-world toxicological hazards. These findings expose an overlooked attack surface in molecule language models and underscore the need for stronger safeguards, monitoring, and traceability mechanisms for frontier systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
red-teaming, molecule generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7qG3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies red-teaming of molecule language models for toxic molecule generation. Its main new contribution is a decoding-time attack that uses a learned surrogate to estimate whether a partial SMILES is likely to lead toward molecules structurally similar to known toxic compounds, and then uses that score to guide beam decoding. The paper also curates toxicity datasets and trains endpoint-specific toxicity evaluators, but these components are mostly supporting infrastructure rather than the main methodological novelty.

I will therefore mainly evaluate the paper on whether the structural-similarity prior is well justified across different toxicity endpoints, whether the surrogate-guided decoding method is sound and genuinely effective, and whether the toxicity evaluators are reliable enough to support the reported conclusions.

Critical strengths
  • The method is conceptually simple and well motivated: it converts an expensive rollout-based toxicity-related objective into a lightweight surrogate signal that can be used during decoding, making the attack practical across multiple autoregressive molecule models.

  • The structural-similarity prior shows strong discrimination for some toxicity endpoints, with ROC-AUC reaching 0.961 at the high end. This supports the idea that structural similarity can provide a useful signal for guiding red-teaming in at least some toxicity settings.

  • The proposed decoding strategy is empirically effective across eight molecule language models, consistently increasing overall toxicity rates and substantially outperforming greedy, top-k, top-p, and standard beam decoding. The guidance-interval analysis also shows that more frequent surrogate guidance generally leads to stronger attacks.

Critical weaknesses
  • The structural-similarity prior is not uniformly strong across toxicity endpoints. The paper reports ROC-AUCs as low as 0.653 for stress response and 0.700 for micronucleus induction, indicating only modest discrimination for some endpoints. This suggests that structural similarity provides useful signal in some cases, but is not a consistently strong toxicity prior across all endpoint types.

  • The ablation analysis is insufficient to explain why the method works. The paper mainly varies the guidance interval, but does not report the surrogate’s held-out prediction quality or isolate key components with controls such as random/unguided scoring, rollout-based oracle guidance, prompt-vs-prefix conditioning, or matched-compute beam search. As a result, the overall attack is clearly effective, but the specific contribution of the learned surrogate is less well established.

  • The benchmarking lacks competing red-teaming or guided-decoding baselines, making it difficult to assess the proposed attack against alternative methods.

Minors

  • Several important datasets/resources are mentioned without a proper citation. Appendix C relies on resources such as HCIS / Safe Work Australia, OECD QSAR Toolbox, REACH/ECHA, CPDB, ISS ISSTOX, and OECD eChemPortal, but several of these are only named in prose rather than given a bibliographic reference or URL. The same problem applies to some primary toxicity datasets.

  • In the introduction, the paper states that the toxicity evaluators achieve “over 90% average accuracy across all toxicity endpoints.” However, Table 4 reports MoLFormer accuracies of 95.05%, 83.53%, 95.42%, 65.05%, 82.50%, 85.45%, 75.02%, and 85.04% across the eight endpoints. The simple average is only about 83.4%, not over 90%. The reported 93.43% is explicitly a sample-size-weighted overall accuracy, rather than an average across endpoints. This makes the statement in the introduction misleading and overstates the evaluator performance.

  • There is a wording conflict in the conclusion. The authors state that their method succeeds while “requiring only one forward pass in the target model.” However, the proposed method performs repeated surrogate-guided beam decoding: at each guidance interval, it runs beam search over candidate continuations, selects one, updates the prefix, and repeats this procedure until decoding terminates. This description appears inconsistent with the claim of requiring only one forward pass through the target model.

Questions for authors

Could the authors clarify how the train/validation/test splits were constructed for each toxicity evaluator, especially when combining datasets from multiple sources? In particular, were scaffold- or similarity-based splits used, or were near-duplicate/closely related analogs controlled across splits beyond exact molecule deduplication?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7qG3 (Part 1)

We sincerely thank the reviewer for the constructive feedback, as well as for recognizing the clear motivation, computational efficiency, and empirical effectiveness of our proposed surrogate-guided decoding framework. Below, we provide point-by-point responses to the reviewer's comments and clarify the corresponding aspects of our work.


In response to Weakness 1

While ROC-AUC varies across toxicity endpoints, the similarity distributions of toxic and non-toxic molecules exhibit statistically significant differences across all eight endpoints (all p≤1.0×10−4p \le 1.0 \times 10^{-4}). Notably, even for Stress Response (ROC-AUC = 0.653) and Micronucleus Induction (ROC-AUC = 0.700), the differences remain highly significant (p=1.7×10−50p = 1.7 \times 10^{-50} and p=1.0×10−4p = 1.0 \times 10^{-4}, respectively). These findings demonstrate that structural similarity provides a statistically informative signal across all endpoints. Importantly, our method leverages structural similarity as a guidance signal rather than a highly accurate toxicity classifier, without requiring strong discrimination between toxic and non-toxic molecules. This is further supported by our experimental results, which show that surrogate-guided decoding can improve toxicity rates even for endpoints with relatively low ROC-AUCs, demonstrating the practical utility of structural similarity as a decoding prior.


In response to Weakness 2

We conducted additional experiments and provide detailed responses to each point below.

Surrogate’s held-out prediction quality. To directly assess the predictive reliability of our surrogate scorer, we conducted additional experiments on 1,000 held-out samples, comparing its predictions with rollout-derived reference scores. We report the mean absolute error (MAE) and Pearson correlation across all eight toxicity endpoints:

EndpointMAE ↓Pearson ↑
Cardio.0.050.89
Pneumo.0.050.89
Ophth.0.040.91
Derm.0.050.88
Carc.0.030.89
Mutag.0.040.90
SR0.040.88
MN0.030.88

As shown above, our surrogate scorer consistently achieves low prediction errors (MAE: 0.03–0.05) and strong correlations (Pearson: 0.88–0.91) with the rollout-derived reference scores across all eight endpoints. These results demonstrate that the learned surrogate reliably approximates the completion-based scoring signal, validating its effectiveness in estimating the toxicity-related potential of partial SMILES without requiring additional target-model rollouts during decoding.

Random scoring. To further isolate the contribution of the learned surrogate, we conducted additional experiments by replacing surrogate-guided scoring with random scoring during decoding. We evaluated this control on ChemAligner-T5 and ChemAligner-T5-Pro, comparing it against greedy decoding and our proposed method across all eight toxicity endpoints. The results are summarized below.

ChemAligner-T5

EndpointGreedyRandomOurs
Cardio.8.058.4949.03
Pneumo.23.8841.4575.94
Ophth.51.3743.9855.09
Derm.14.4646.7968.39
Carc.4.2234.1127.24
Mutag.7.4120.9546.27
SR14.5222.7680.09
MN10.4241.1263.12
Avg.16.7932.4658.15

ChemAligner-T5-Pro

EndpointGreedyRandomOurs
Cardio.30.85%18.48%49.15%
Pneumo.39.15%39.64%71.45%
Ophth.82.97%53.61%74.42%
Derm.43.85%54.12%76.03%
Carc.11.15%36.27%26.03%
Mutag.18.12%37.27%48.45%
SR31.24%18.09%77.24%
MN32.58%38.55%64.09%
Avg.36.24%37.00%60.86%

As shown above, our surrogate-guided decoding achieves substantially higher average toxicity rates than random scoring. These results demonstrate that the effectiveness of our method stems from the learned surrogate guidance rather than random candidate selection during beam decoding, further validating the contribution of our surrogate scorer.

Rollout-based oracle guidance. The primary motivation for our learned surrogate is to avoid the substantial computational overhead of repeated target-model rollouts during decoding. While surrogate-guided generation typically completes within a few seconds per molecule, rollout-based oracle guidance can require several hours per molecule due to repeated candidate completions at each guidance step. This substantial computational cost makes evaluation at scale impractical, particularly given the limited time and computational resources available during the rebuttal period. Nevertheless, our surrogate prediction results demonstrate strong agreement with rollout-derived reference scores, supporting its effectiveness as a computationally efficient approximation.

AuthorsOpenReview ↗

Response to Reviewer 7qG3 (Part 2)

In response to Weakness 2 (Continued)

Prompt-vs-prefix conditioning. We retrained a prefix-only surrogate model using the same molecular prefix encoder and training protocol, while removing the prompt-conditioning branch. We evaluated this variant on ChemAligner-T5 and ChemAligner-T5-Pro across all eight toxicity endpoints.

ModelGreedyPrefix-onlyPrompt + Prefix (Ours)
ChemAligner-T516.79%56.64%58.15%
ChemAligner-T5-Pro36.24%55.18%60.86%

As shown above, the prefix-only surrogate already achieves substantial improvements over greedy decoding, while joint prompt-prefix conditioning achieves better performance on average. However, when initially designing our surrogate model, we recognized that relying solely on prefixes would overlook the semantic information in the input captions. In particular, the same molecular prefix may require different continuations under different prompts, whereas a prefix-only surrogate cannot distinguish these contexts, potentially limiting the diversity and flexibility of prompt-conditioned generation. Joint conditioning addresses this limitation by considering both molecular structure and input semantics.

Furthermore, prompt-only conditioning cannot provide meaningful guidance during decoding, as all candidate continuations receive identical surrogate scores under the same input prompt, causing the decoding process to revert to standard beam search. Taken together, these considerations motivate our joint prompt-prefix conditioning design, which leverages both semantic and structural information for effective generation guidance.

Matched-compute beam search. As reported in Table 5 of the supplementary material, we have already compared our method against standard beam search across all eight molecule language models. Our surrogate-guided decoding consistently achieves substantially higher average toxicity rates, demonstrating that the improvements are not simply attributable to beam search itself, but rather to the effectiveness of our proposed framework.

We believe these results provide a clearer understanding of why our method works and will incorporate them into the revised manuscript or supplementary materials, as appropriate. We hope these additional analyses and controlled experiments adequately address the reviewer's concerns.


In response to Weakness 3

To the best of our knowledge, our work is the first to systematically investigate red-teaming of molecule language models for toxicity-oriented generation, which represents a key novelty of this study. Existing red-teaming methods are primarily designed for natural language models and cannot be applied to molecule language models due to the distinct characteristics of SMILES-based molecular generation. Therefore, there are no established baselines for this setting, as explicitly discussed in Section 5.1 (Experimental Settings). Instead, we compare our method against widely adopted decoding strategies, including greedy, top-k, top-p, and standard beam decoding in the original Table 5, across eight molecule language models, demonstrating the consistent effectiveness of our approach.

AuthorsOpenReview ↗

Response to Reviewer 7qG3 (Part 3)

In response to Minor Weakness 1

We thank the reviewer for pointing out the missing references. We will carefully review the resources highlighted by the reviewer, as well as the primary toxicity datasets used in our study, and provide the corresponding bibliographic references or official URLs in the revised version. The references we will add are listed below.

Resources to be cited. The Hazardous Chemical Information System of Safe Work Australia will be cited as [1]. The OECD QSAR Toolbox will be cited as [2]. The skin-irritation records will cite the REACH Regulation itself [3] together with the ECHA registered-substances database from which they were retrieved [4]. The Carcinogenic Potency Database will be cited as [5], and the ISS Carcinogens Data Bank / ISSTOX resource used for the strain-specific Ames annotations as [6]. The OECD eChemPortal, will be cited as [7]. Bibliographic citations for the primary toxicity datasets (hERGCentral, FDA DICTrank, UniTox, ICCVAM/LLNA, EPA ToxValDB, STopTox, and Tox21) were already provided in the original manuscript.

References to be added in the revised version.

[1] Safe Work Australia. Hazardous Chemical Information System (HCIS). https://hcis.safeworkaustralia.gov.au.

[2] OECD. The OECD QSAR Toolbox. https://qsartoolbox.org.

[3] European Parliament and Council of the European Union. Regulation (EC) No 1907/2006 of 18 December 2006 concerning the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH). Official Journal of the European Union, L 396, 30.12.2006, p. 1.

[4] European Chemicals Agency. Registered substances database. https://echa.europa.eu/information-on-chemicals/registered-substances.

[5] Gold, L. S., et al. Supplement to the Carcinogenic Potency Database (CPDB): results of animal bioassays published in the general literature through 1997 and by the National Toxicology Program in 1997–1998. Toxicological Sciences, 85(2):747–808, 2005. Database available at https://www.leadscope.com/CPDB.

[6] Benigni, R., et al. New perspectives in toxicological information management, and the role of ISSTOX databases in assessing chemical mutagenicity and carcinogenicity. Mutagenesis, 28(4):401–409, 2013. Resource available at https://www.iss.it/web/iss-en/databases.

[7] OECD. eChemPortal: The Global Portal to Information on Chemical Substances. https://www.echemportal.org.


In response to Minor Weakness 2

As explicitly stated in the caption of Table 4, the overall accuracy is weighted by the number of test samples for each toxicity endpoint. Nevertheless, we acknowledge that the phrase "over 90% average accuracy across all toxicity endpoints" in the Introduction may be ambiguous. To avoid any misunderstanding, we will revise it to “over 90% overall accuracy across all toxicity endpoints (weighted by the number of test samples per endpoint)” in the revised version.


In response to Minor Weakness 3

We acknowledge that the phrase "requiring only one forward pass in the target model" may be imprecise. Our intended meaning is that the proposed method performs a single autoregressive decoding procedure without requiring additional target-model rollouts for toxicity evaluation. We recognize that, as with standard beam search, multiple forward computations are involved in token generation. To avoid ambiguity, we will revise the statement in the Conclusion to "without requiring additional target-model rollouts" in the revised version.


In response to Question

As described in Section 3.2 (Toxicity Assessment), we preserve the original train/validation/test splits of existing toxicity datasets whenever available. For datasets without predefined splits, as well as newly collected samples, we incorporate them exclusively into the training set. For duplicate molecules with conflicting toxicity labels, we conservatively assign the toxic label and remove duplicate entries. Finally, we remove training molecules that overlap with the validation or test sets to prevent exact-molecule leakage.

We clarify that scaffold- or similarity-based splits were not employed, and closely related molecular analogs were not explicitly excluded across splits. Our primary objective is to systematically investigate the red-teaming vulnerabilities of molecule language models, rather than develop toxicity prediction models with optimized out-of-distribution generalization. Importantly, both our attack method and all baseline decoding strategies are evaluated using the same toxicity evaluators, ensuring a fair and consistent comparison of their relative effectiveness. Nevertheless, we recognize the value of evaluating toxicity predictors under more stringent structural splits and leave this direction for future work. Upon acceptance, we will publicly release our curated toxicity benchmark to facilitate further research and the development of more robust toxicity evaluators.

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns.

The rebuttal has addressed several of my original comments:

  • Surrogate evaluation: The held-out prediction results provide good evidence that the surrogate can approximate the rollout-based scores.
  • Ablation studies: The random-scoring and prefix-only experiments help clarify the contribution of the learned surrogate and prompt conditioning.

However, I still have a few remaining concerns and questions:

  1. Structural-similarity prior: I agree that a guidance signal does not need to be a strong classifier. However, statistical significance does not necessarily mean strong discrimination, and the method still shows inconsistent improvements for some endpoints. I think this limitation should be acknowledged more clearly.

  2. Matched-compute comparison: The authors refer to Table 5, but my original concern was about matched-compute beam search, not standard beam search. Could the authors clarify whether the computational budgets are comparable?

  3. Competing baselines: I understand that there are no established red-teaming baselines for this specific setting. However, comparisons with alternative guided-decoding approaches would still help establish the effectiveness of the proposed method.

  4. Toxicity evaluator reliability: Thank you for clarifying the dataset splits. My remaining concern is that the attack favors molecules structurally similar to known toxic compounds, while the evaluators were not tested under scaffold- or similarity-based splits. Using the same evaluators across all methods ensures consistency, but does not rule out structural bias in the reported toxicity rates. Could the authors provide further evidence or discuss this limitation more explicitly?

  5. Code availability: I could not find a code repository in the manuscript or rebuttal. Have the authors released their implementation or provided an anonymous repository for reviewers? I would like to examine the code before finalizing my assessment.

Overall, the rebuttal has strengthened the paper and resolved several of my concerns. I will also examine the released code before deciding whether to update my rating.

Reviewer_HxwnOpenReview ↗
Rating
3(+1)
Confidence
2(−2)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper explores a new attack surface: an attack on molecule language models. The authors first point out that safety risks may occur in these kinds of domain-specific language models and propose an inference-time attack to make these models generate toxic molecules. The authors discover that toxic molecules exhibit greater similarity to known toxic examples, and they leverage this similarity to achieve their attack goal. Extensive experiments show good performance of their attack on MolT5, BioT5, etc.

Critical strengths
  1. The red-teaming idea is interesting. I have worked on adversarial attacks and backdoor attacks on language models, and I have seen many papers trying to explore topics that are already overcrowded. This paper, however, proposes new ideas. I think that the red-teaming purpose is intuitive and very easy to understand: misleading molecule language models to generate toxic molecules. This may remind the AI4science community to pay attention to safety risks that are largely ignored.

  2. The observation --> motivation of the method is clear. I can see that in Figure 3 (Distributions of structural similarity), the authors intuitively show the differences in structural similarity between toxic and non-toxic molecules. And this motivates the method: if the representations of toxic molecules can fall into a small subspace, it is easier to inject a similar known toxic representation into the molecule language models. In sum, the presentation of this part is natural and sound.

Critical weaknesses
  1. I think the core weakness is the reality of the threat model. I agree that this is a novel attack; however, the authors claim in their paper: " The adversary has local inference access to multiple candidate continuations at each decoding step". So how can the attack really do harm to benign users? I think that the pipeline needs steering/surrogate guidance at decoding steps, and benign users may not do such things when using molecule language models. If the red-teaming cannot be deployed in real-life usage, the strength of the proposed attack surface may not be so good. In other words, it is not so meaningful to make the molecule language models generate toxic molecules only on the attacker's side, but not on other benign users' side. You may reorganize what the attacker and benign users will do during inference.

  2. Additionally, there is another gap. The paper mainly discusses molecule language models; however, writing SMILES is different from generating real toxic or poisonous molecules. Table 2 shows the toxicity rate (which does not increase consistently). I wonder if molecules with a toxicity rate of 20+ or 30+ can be regarded as "toxic". The authors may adjust the claim of the whole work: the attack may not make really "toxic molecules".

Questions for authors

Besides the weaknesses, I also have some questions:

  1. Have you ever tried any domain-specific large language models? Nowadays, many users may use large language models for text-based molecule generation. ChemDFM and ChemLLM [1][2] are some of them. Can the attack be generalized to larger models?

  2. I see that the paper does not discuss defenses against this red-teaming process. There have been many defenses against inference-time attacks, including adaptive defenses against surrogate red-teaming models. Since this is an AI security paper, it is important to discuss defenses. I regard this as a weakness that should be added to the manuscript.

  3. Will the capabilities of molecule language models in generating "novel" molecules degrade? I mean that the model shouldn't be only able to generate things like benzene rings, etc. If so (degrade), benign users will discover the attack very easily.

According to the current version, I may recommend a weak rejection.
[1] Zhang, Di, et al. "Chemllm: A chemical large language model." arXiv preprint arXiv:2402.06852 (2024).
[2] Zhao, Zihan, et al. "Developing ChemDFM as a large language foundation model for chemistry." Cell Reports Physical Science 6.4 (2025).

9 comments
AuthorsOpenReview ↗

Response to Reviewer Hxwn (Part 1)

We appreciate the reviewer's thoughtful comments. We are particularly encouraged by the reviewer's recognition of the novelty and significance of exploring an overlooked safety risk in molecule language models, as well as the clear motivation and soundness of our structural-similarity-guided approach. We provide point-by-point responses below.


In response to Weakness 1

First, we would like to clarify that decoding-time attacks constitute a well-established threat model in LLM safety research. Several representative studies have demonstrated that manipulating the generation process can expose safety vulnerabilities without modifying model parameters or input prompts. Specifically:

  • Enforced Decoding (EnDec) [1] directly manipulates the generation process by forcing LLMs to generate specific tokens at designated positions.
  • Emulated Disalignment (ED) [2] accesses the token distributions of pretrained and safety-aligned models and employs contrastive sampling to shift generation away from safety-aligned behavior.
  • Weak-to-Strong Jailbreaking [3] leverages the token distribution differences between smaller safe and unsafe models to modify the decoding probabilities of a larger safety-aligned model.

We recognize that our statement, "The adversary has local inference access to multiple candidate continuations at each decoding step," may have caused confusion. Specifically, our method requires access to the target model's token probability distributions during inference. We evaluate the top-nn candidate continuations using our surrogate scorer and select the highest-scoring one for subsequent decoding. Our threat model therefore follows the same established setting of adversarial control over the inference process as previous decoding-time attacks. We also note that the reviewer mentions existing defenses against inference-time attacks in Question 2, which further highlights that inference-time manipulation is a recognized and actively studied attack surface.

Furthermore, our objective is to investigate adversarial misuse of molecule language models, rather than to compromise benign users' inference sessions. This is consistent with established LLM jailbreak research, where adversaries deliberately manipulate their own interactions with target models to elicit harmful outputs. Such attacks do not require benign users to experience the same behavior to constitute a meaningful misuse risk. The fact that the attack operates on the adversary's side does not diminish its relevance, as the resulting outputs can still have downstream safety implications.

[1] Zhang et al., Jailbreak Open-Sourced Large Language Models via Enforced Decoding, ACL 2024.

[2] Zhou et al., Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!, ACL 2024.

[3] Zhao et al., Weak-to-Strong Jailbreaking on Large Language Models, ICML 2025.


In response to weakness 2

We acknowledge that our study does not involve wet-lab experiments or physical synthesis, and therefore does not claim to experimentally verify the toxicity of newly generated molecules. However, our objective is to uncover the potential safety risks of AI-generated molecular structures, which is itself a meaningful and well-established research direction, as demonstrated by prior work such as MolSafeEval [1]. In addition to our quantitative evaluation, we provide case studies identifying generated molecular structures corresponding to known hazardous compounds with established toxicological evidence, which provides additional support for the practical relevance of the identified safety risks beyond computational toxicity predictions.

We would also like to clarify a potential misunderstanding regarding Table 2. Toxicity rate measures the frequency of predicted toxic outputs, not the severity of toxicity. A toxicity rate of 20–30% means that 20–30 out of 100 generated molecules are classified as toxic for a given endpoint, rather than indicating that each molecule is only "20–30% toxic." Although some individual endpoint exhibit decreases, the average toxicity rate increases for all eight evaluated models, with an average absolute improvement of 19.1 percentage points and a maximum improvement of 41.4 percentage points, which demonstrates the consistent effectiveness of our method.

Finally, we emphasize that throughout the manuscript, we do not claim to make "real toxic molecules". Nevertheless, to avoid any ambiguity, we will further clarify this distinction throughout the manuscript and revise potentially overstated expressions to emphasize predicted toxicity where appropriate.

[1] Xu, Tong, et al. "MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

AuthorsOpenReview ↗

Response to Reviewer Hxwn (Part 2)

In response to Question 1

We thank the reviewer for this valuable suggestion. Due to time constraints, we conducted additional experiments on 200 randomly selected test samples. Specifically, we evaluated three domain-specific LLMs, ChemDFM-v1.0-13B, ChemLLM-7B-Chat, and TxGemma-9B-Chat [1], as well as a larger general-purpose LLM, Qwen3.8-27B [2], to further assess the generalizability of our approach. The results are presented below.

ChemDFM-v1.0-13B

EndpointBaselineOurs
Cardio.4.5%40.5%
Pneumo.43.5%47.0%
Ophth.40.0%63.0%
Derm.67.5%87.5%
Carc.5.0%35.5%
Mutag.21.5%65.5%
SR27.0%47.0%
MN59.5%68.0%
Avg.33.56%56.75%

ChemLLM-7B-Chat

EndpointBaselineOurs
Cardio.45.0%52.5%
Pneumo.65.0%64.0%
Ophth.74.5%79.0%
Derm.74.5%70.5%
Carc.8.5%43.5%
Mutag.29.5%65.0%
SR42.5%50.5%
MN45.0%57.0%
Avg.48.06%60.25%

TxGemma-9B-Chat

EndpointBaselineAttack
Cardio.62.0%63.5%
Pneumo.26.5%38.0%
Ophth.85.0%92.0%
Derm.89.5%93.0%
Carc.25.0%30.5%
Mutag.37.5%62.0%
SR15.0%22.0%
MN40.0%75.5%
Avg.47.6%59.6%

Qwen3.8-27B

EndpointBaselineAttack
Cardio.18.0%39.5%
Pneumo.50.0%68.5%
Ophth.53.5%73.5%
Derm.63.0%81.5%
Carc.5.5%30.5%
Mutag.20.0%49.0%
SR44.0%43.5%
MN51.0%71.5%
Avg.38.1%57.2%

As shown above, our method consistently improves the average toxicity rate across all four additional models. These results provide further evidence of the effectiveness and generalizability of our approach across both domain-specific and larger general-purpose LLMs.

[1] Wang, Eric, et al. "TxGemma: Efficient and Agentic LLMs for Therapeutics." arXiv preprint arXiv:2504.06196 (2025).

[2] Qwen Team. "Qwen3.8-Max: A New Bar for Coding and Cowork." Qwen Blog (2026). Available at: https://qwen.ai/blog?id=qwen3.8


In response to Question 2

We agree that discussing defenses against inference-time attacks would further strengthen our work. Previous studies have explored several defense strategies against decoding-time manipulation. For example, Han et al. [1] evaluate Probability Smoothing, which mixes token probability distributions with uniform noise to reduce the influence of adversarial logit manipulation, and Safety Fine-tuning, which adjusts model parameters using safety-aligned examples to mitigate harmful generation.

Inspired by these approaches, we further conducted preliminary defense experiments using Gradient Ascent (GA). Specifically, for each toxicity endpoint, we selected 100 experimentally annotated toxic molecules and performed one epoch of gradient ascent to discourage the model from generating toxicity-associated structures. We then evaluated the effectiveness of this defense against our attack on 200 samples across four toxicity endpoints, with the results summarized below.

EndpointBaselineOurs w/ GA defenseOurs w/o GA defense
Cardio.23.5%38.0%39.5%
Pneumo.66.5%81.0%86.0%
Ophth.56.5%71.5%77.0%
Derm.52.5%70.5%75.0%
Avg.49.75%65.25%69.38%

As shown above, GA reduces the attack-induced predicted toxicity rates across all four endpoints, suggesting that it effectively alters model behavior and partially mitigates the effects of the attack. More comprehensive and adaptive defense mechanisms remain an important direction for future work. We will incorporate these findings and discussions into the revised manuscript.

[1] Han, Shuaibiao, et al. "Dictionary Guided Sparse Logit Editing for Reliable Jailbreak Attacks." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

AuthorsOpenReview ↗

Response to Reviewer Hxwn (Part 3)

In response to Question 3

To investigate whether our method compromises the diversity of generated molecules, we follow prior molecular generation studies [1][2] and evaluate structural diversity using the mean pairwise fingerprint Tanimoto distance (Morgan fingerprints, radius 2, 2048 bits), where higher values indicate greater molecular diversity. For our method, we compute this metric separately for each of the eight toxicity endpoints and report the average results below.

ModelBaselineOurs
BioT50.9100.930
BioT5+0.9110.932
ChemAligner-T50.6940.910
ChemAligner-T5-Pro0.8590.910

As shown above, our method consistently improves molecular structural diversity across all four models, indicating that the generated molecules do not collapse into a limited set of similar structures. These results demonstrate that our decoding strategy can steer molecular generation while maintaining diverse molecular outputs.

[1] Polykovskiy, Daniil, et al. "Molecular sets (MOSES): a benchmarking platform for molecular generation models." Frontiers in pharmacology 11 (2020): 565644.

[2] Wang, Zichao, et al. "Retrieval-based controllable molecule generation." International Conference on Learning Representations (ICLR) (2023).

Reviewer_mzUXOpenReview ↗
Rating
2
Confidence
2(−2)
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces the first red-teaming method to investigate whether molecular LLMs can be guided to generate toxic molecules.

The authors construct a benchmark with eight toxicity endpoints grouped into two categories and train a MolFormer-based toxicity prediction model that achieves over 90% accuracy.

The authors propose a red-teaming method that guides molecular generation without additional training or changes to the generative model. Specifically, they train a Surrogate Scorer to predict whether a partially generated SMILES sequence is likely to produce a molecule structurally similar to toxic compounds. During generation, beam search generates multiple candidate sequences at fixed token intervals. The Surrogate Scorer evaluates these candidates, and the highest-scoring sequence is selected to continue generation. This process is repeated until a complete molecule is generated. Finally, a separate Toxicity Evaluator assesses whether the generated molecule is toxic.

The results show that the proposed method achieves a toxic molecule generation rate of over 41.4% compared to greedy decoding. The authors also provide additional evidence that the generated molecules may be toxicologically harmful.

Critical strengths

Novelty and Timeliness: This paper presents the first systematic red-teaming study of molecular language models, highlighting the potential risks of generating toxic molecules. This is a timely and important research direction given the growing use of molecular generative models.

Contribution to Toxicity Research: The authors collect and curate toxicity datasets covering eight different endpoints from multiple sources. These datasets provide a valuable contribution to toxicity research and support systematic evaluation of molecular generative models.

Critical weaknesses

W1. Limitations of Structural Similarity as a Toxicity Signal

The proposed method relies on the assumption that molecules structurally similar to known toxic compounds are more likely to be toxic. However, previous studies on activity cliffs and toxicity cliffs have shown that small structural modifications can lead to substantial changes in biological activity or toxicity[1,2]. Although the authors provide statistical evidence supporting their assumption, it remains unclear whether this relationship is sufficiently reliable to guide toxic molecule generation, especially for molecules exhibiting toxicity cliffs.

[1] Van Tilborg, D., Alenicheva, A., & Grisoni, F. (2022). Exposing the limitations of molecular machine learning with activity cliffs. Journal of chemical information and modeling, 62(23), 5938.

[2] Li, C., Qian, J., Zhou, Y., Wang, S. L., & Chen, C. (2026). Activity cliff-guided feature selection for improved toxicity prediction using machine learning. Computational Toxicology, 100428.


W2. Limited Methodological Novelty Compared to GeneBreaker

The proposed red-teaming method is conceptually similar to GeneBreaker (Zhang et al., 2026), which also uses beam search guided by an external scoring model to steer sequence generation toward harmful outputs without modifying the target generative model. Although the authors introduce a Surrogate Scorer to handle incomplete SMILES sequences, the overall decoding strategy closely follows the approach used in GeneBreaker. Therefore, the methodological novelty beyond adapting an existing red-teaming framework to molecular generation is somewhat limited. A clearer discussion of the differences and additional experiments demonstrating the benefits of the Surrogate Scorer would strengthen the paper.


W3. Limited Validation of the Surrogate Scorer

The authors highlight that incomplete SMILES prefixes cannot be directly evaluated and that the same molecule can have multiple SMILES representations. Although the proposed Surrogate Scorer provides an indirect way to estimate the potential of partial SMILES, it does not fully address these limitations. Furthermore, the paper provides limited evidence of how accurately the Surrogate Scorer predicts the toxic-neighbor potential of partial sequences. Additional experiments evaluating its prediction accuracy and robustness would strengthen the proposed method.


W4. Limited Reproducibility and Unclear Release Policy

The authors do not release the code or toxicity benchmark data. Since this study aims to highlight safety risks, independent verification of the results is important. Although releasing the attack code may raise safety concerns, the authors should clarify their release policy and explain how other researchers can reproduce or validate the results.

Questions for authors

Q1. Evaluation on More Advanced Language Models

Could the authors extend their experiments to more powerful general-purpose open-source LLMs, such as Qwen3.5-122B/397B and DeepSeek-R1-70B, or more recent domain-specific models such as TxGemma? This would help assess whether the proposed method is effective on more advanced models.


Q2. Class Distribution of the Toxicity Benchmark

Could the authors provide the ratio of toxic to non-toxic molecules for each toxicity endpoint in the benchmark dataset? This information would help readers better understand the dataset characteristics and potential class imbalance.


Q3. Additional Evaluation Metrics for Toxicity Evaluators

In Table 4, the authors report only accuracy for the toxicity evaluators. Could the authors provide additional metrics, such as AUROC, AUPRC, F1-score, MCC, precision, and false positive rate (FPR), for each toxicity endpoint? Accuracy alone may overestimate model performance, especially on imbalanced datasets. In particular, precision and FPR are important because false positive toxicity predictions could lead to overestimated red-teaming success rates.


Q4. Potential Application as a Safety Mechanism

Could the authors apply the proposed method in reverse to guide the generation of non-toxic molecules? Could this approach serve as a safety mechanism for molecular language models?

8 comments
AuthorsOpenReview ↗

Response to Reviewer mzUX (Part 1)

We sincerely thank the reviewer for recognizing the novelty and timeliness of our red-teaming framework for molecular language models, as well as the broader value of our work to toxicity research. We appreciate the constructive feedback and address the reviewer's concerns below.


In response to Weakness 1

We fully acknowledge that structurally similar molecules may exhibit substantially different biological activities or toxicological properties, as exemplified by activity cliffs. However, such cliffs represent particularly challenging exceptions that do not invalidate the broader statistical trend between structural similarity and toxicity observed in our study.

Importantly, our method leverages structural similarity as a guidance signal rather than a highly accurate toxicity classifier. It does not assume a deterministic relationship between structural similarity and toxicity, but instead exploits the population-level statistical trend to guide molecular generation. As demonstrated in Figure 3, despite these exceptions, toxic molecules generally exhibit higher similarity to known toxic compounds than non-toxic molecules across all eight toxicity endpoints. These differences are statistically significant (Mann–Whitney U test, p≤10−4p \leq 10^{-4} for all endpoints), supporting the discriminative value of structural similarity as an aggregate statistical signal.

Motivated by this empirical observation, we develop our red-teaming framework, which consistently improves toxicity rates across all eight molecular language models. Together, these findings provide complementary evidence that structural similarity remains a useful statistical prior for guiding molecular generation, despite the existence of activity cliffs. As also noted by Reviewer Hxwn and 7qG3, the structural similarity distributions in Figure 3 provide a clear and intuitive motivation for our proposed method.


In response to Weakness 2

While both approaches employ decoding-time guidance, our method addresses a fundamentally different challenge in evaluating intermediate molecular generations, which prevents the direct application of GeneBreaker's strategy to molecule language models. We elaborate on these differences below.

  • Distinct challenges in evaluating intermediate generations. GeneBreaker operates on DNA sequences, where intermediate prefixes remain syntactically valid nucleotide sequences. For example, both ACGTAC... and its prefix ACGT are syntactically valid DNA sequences that can be meaningfully evaluated by an existing pathogenicity predictor (PathoLM) during generation. In contrast, intermediate SMILES prefixes do not correspond to valid, fully specified molecular structures. For example, while CC(=O)Oc1ccccc1C(=O)O represents a valid molecule, its intermediate prefix CC(= cannot be parsed into a complete molecular structure due to an unclosed branch and an incomplete bond specification. Consequently, this reveals a fundamental methodological challenge that remains unaddressed by GeneBreaker.

  • A distinct scoring formulation based on molecular structural priors and rollout-based evaluation. We first investigate the relationship between molecular structural similarity and toxicity, revealing a statistically significant association that establishes a meaningful structural prior for guiding molecular generation. However, structural similarity cannot be directly computed for incomplete SMILES. In light of this, we introduce a rollout-based scoring formulation that estimates the potential of each partial SMILES by sampling valid molecular completions and measuring their similarity to known toxic compounds. This enables meaningful evaluation of incomplete molecular representations, a key methodological challenge not encountered in GeneBreaker.

  • Efficient completion-aware scoring through surrogate learning. While rollout-based evaluation provides a meaningful scoring signal for partial SMILES, repeatedly generating and evaluating molecular completions during decoding incurs substantial computational overhead. To overcome this limitation, we develop a surrogate scorer that directly predicts rollout-derived structural similarity scores from partial SMILES and their corresponding textual prompts, enabling efficient evaluation of incomplete molecular representations without additional rollouts during inference.

AuthorsOpenReview ↗

Response to Reviewer mzUX (Part 2)

In response to Weakness 2 (Continued)

  • Additional experiments validating the surrogate scorer. To further substantiate the effectiveness of our surrogate scorer, we conducted additional experiments using 1,000 held-out evaluation samples, comparing its predictions against the corresponding rollout-derived reference scores. We report the mean absolute error (MAE) and Pearson correlation across all eight toxicity endpoints below.
EndpointMAE ↓Pearson ↑
Cardio.0.050.89
Pneumo.0.050.89
Ophth.0.040.91
Derm.0.050.88
Carc.0.030.89
Mutag.0.040.90
SR0.040.88
MN0.030.88

As shown above, our surrogate scorer consistently achieves low prediction errors (MAE: 0.03–0.05) and strong correlations with rollout-derived reference scores (Pearson: 0.88–0.91) across all eight endpoints. These results demonstrate that the learned surrogate can accurately approximate the completion-based scoring signal, supporting the reliability of our proposed scoring formulation for evaluating partial SMILES.

  • Summary of methodological distinctions. In summary, while we acknowledge the shared use of decoding-time guidance, our contribution extends beyond applying GeneBreaker to a different biological domain. Specifically, our work (i) identifies the challenge of evaluating incomplete SMILES representations, (ii) establishes an empirically supported molecular structural prior, (iii) introduces a rollout-based scoring formulation to evaluate incomplete SMILES through their possible molecular completions, and (iv) develops an efficient surrogate scorer that learns to approximate rollout-derived scores without additional rollouts during inference. Together, these contributions address distinct methodological challenges in molecular generation that are not considered in GeneBreaker.

In response to Weakness 3

As reported in our response to Weakness 2, we conducted additional experiments on 1,000 held-out samples, comparing the surrogate scorer's predictions against the corresponding rollout-derived reference scores. Across all eight toxicity endpoints, the surrogate scorer achieves consistently low MAEs (0.03–0.05) and high Pearson correlations (0.88–0.91). These results demonstrate that our learned scorer can accurately approximate the completion-based scoring signal, providing direct empirical support for the effectiveness of our proposed approach.

Regarding the reviewer's concern about incomplete SMILES, the inability to directly evaluate incomplete SMILES is precisely the methodological challenge motivating our approach. As further explained in our response to Weakness 2, we address this challenge by introducing a rollout-based scoring formulation and learning a surrogate scorer to approximate the resulting scores. Consequently, our method enables direct prediction of the potential of partial SMILES. We therefore respectfully clarify that our framework is specifically designed to overcome this evaluation challenge, rather than assuming that incomplete SMILES can be directly assessed by conventional molecular scoring functions.

Regarding multiple SMILES representations, we acknowledge that the original manuscript did not sufficiently investigate this issue and thank the reviewer for highlighting it. To address this concern, we conducted additional experiments to evaluate the robustness of our surrogate scorer across different SMILES representations of the same molecule. Specifically, we used the same 1,000 held-out evaluation samples as in the preceding experiment and generated an alternative valid SMILES representation for each molecule. From each alternative representation, we extracted a prefix with the same number of characters as the original prefix. We then evaluated both prefixes and compared their predicted scores. The results are summarized below.

EndpointMAE ↓Pearson r ↑
Cardio.0.00800.9981
Pneumo.0.00800.9981
Ophth.0.00970.9977
Derm.0.00880.9983
Carc.0.00680.9974
Mutag.0.00800.9970
SR0.01210.9964
MN0.00670.9961
Avg.0.00850.9974

As shown above, our surrogate scorer achieves highly consistent predictions across alternative SMILES representations, which demonstrates strong robustness to the tested SMILES serialization variations. One possible explanation is that our surrogate scorer jointly encodes molecular captions and partial SMILES, allowing the shared semantic information in the caption to provide consistent contextual guidance across different representations of the same molecule. We will incorporate these additional experimental results and corresponding discussions into the revised version.

AuthorsOpenReview ↗

Response to Reviewer mzUX (Part 3)

In response to Weakness 4

We fully agree that transparency and reproducibility are essential for trustworthy safety research. However, given the dual-use nature of our work, unrestricted release of the complete red-teaming pipeline could potentially facilitate the misuse of molecular generative models, as discussed in our Ethics Statement.

To balance reproducibility and responsible disclosure, we plan to adopt a tiered release policy. Specifically, upon acceptance, we will publicly release our curated toxicity benchmark datasets and trained toxicity evaluator checkpoints. For potentially sensitive resources, particularly the complete attack implementation, we will consider providing controlled access to qualified researchers for legitimate academic and safety research purposes. Interested researchers would be required to submit an access request specifying their institutional affiliation, research objectives, and intended use of the resources. Access would be evaluated on a case-by-case basis, considering the legitimacy of the research purpose, potential biosecurity implications, and appropriate institutional oversight. Approved researchers would be subject to a research-use agreement restricting unauthorized redistribution, commercial exploitation, and potentially harmful applications.

We believe this tiered release policy can support independent verification of our findings while avoiding potential misuse risks. We have also further revised our Ethics Statement to explicitly clarify our release policy, responsible disclosure considerations, and measures for balancing scientific reproducibility with biosecurity concerns.


In response to Question 1

Due to limited GPU resources, we could not evaluate the extremely large models suggested by the reviewer currently. Instead, we conducted additional experiments on Qwen3.8-27B, DeepSeek-V2-Lite-Chat, and TxGemma-9B-Chat using 200 randomly selected test samples to further assess the generalizability of our approach. The results are summarized below.

Qwen3.8-27B

EndpointBaselineAttack
Cardio.18.0%39.5%
Pneumo.50.0%68.5%
Ophth.53.5%73.5%
Derm.63.0%81.5%
Carc.5.5%30.5%
Mutag.20.0%49.0%
SR44.0%43.5%
MN51.0%71.5%
Avg.38.1%57.2%

DeepSeek-V2-Lite-Chat

EndpointBaselineAttack
Cardio.39.5%60.0%
Pneumo.30.5%63.5%
Ophth.78.0%78.5%
Derm.42.5%77.0%
Carc.32.5%29.5%
Mutag.12.5%45.5%
SR26.5%55.5%
MN30.0%62.0%
Avg.36.5%58.9%

TxGemma-9B-Chat

EndpointBaselineAttack
Cardio.62.0%63.5%
Pneumo.26.5%38.0%
Ophth.85.0%92.0%
Derm.89.5%93.0%
Carc.25.0%30.5%
Mutag.37.5%62.0%
SR15.0%22.0%
MN40.0%75.5%
Avg.47.6%59.6%

As shown above, our method consistently increases the toxicity rate across all three additional models, achieving absolute improvements of 19.1, 22.4, and 12.0 percentage points on Qwen3.8-27B, DeepSeek-V2-Lite-Chat, and TxGemma-9B-Chat, respectively. These results further demonstrate the effectiveness and generalizability of our approach across both general-purpose and domain-specific language models.


In response to Question 2

To provide a clearer understanding of the dataset, we report the class distributions for all eight toxicity endpoints across the training, validation, and test sets, as summarized in the table below. Each entry reports the approximate non-toxic:toxic ratio (%), and the 'Overall' column includes all three splits.

EndpointOverallTrainValidationTest
Cardio.92:892:891:992:8
Pneumo.46:5446:5446:5444:56
Ophth.28:7227:7334:6636:64
Derm.53:4753:4731:6943:57
Carc.58:4257:4356:4480:20
Mutag.40:6039:6144:5644:56
SR75:2573:2773:2777:23
MN57:4356:4456:4465:35

As shown above, class distributions vary across toxicity endpoints. In particular, cardiotoxicity and stress response exhibit relatively pronounced class imbalance, whereas pneumotoxicity and dermatotoxicity have more balanced class distributions.

We appreciate the reviewer's suggestion and will include these detailed class distribution statistics in the supplementary materials of the revised version.

Comments on the paper

1 comment

Dear Reviewers, AC, SAC, and PCs,

We sincerely thank you for your careful evaluation and constructive feedback. The discussion has helped us strengthen the experiments and improve the manuscript.

Following the rebuttal, all three reviewers assigned the highest score of 3 to both Presentation and Soundness. Reviewers also recognized the following strengths:

  • Novel and timely safety problem [mzUX, Hxwn]
  • Valuable toxicity benchmark resources [mzUX]
  • Clear motivation grounded in empirical observations [Hxwn, 7qG3]
  • Simple, efficient surrogate-guided decoding [7qG3]
  • Strong empirical effectiveness [7qG3]

We have responded to every concern through clarifications, additional experiments, or explicit discussion. We have uploaded a revised manuscript incorporating these changes into the main text and supplementary material, including:

  • Additional validation of the surrogate scorer [mzUX, 7qG3]
  • Evaluation on more advanced language models [mzUX, Hxwn]
  • Discussion of potential applications to safety mechanisms [mzUX]
  • Discussion of defenses against the proposed red-teaming process [Hxwn]
  • Additional ablation studies [7qG3]

We have also expanded the ethical considerations. All follow-up concerns in mzUX’s second-round comments and 7qG3’s third-round comments have received point-by-point responses.

We are grateful to each reviewer for helping improve this work. We believe the paper makes a pioneering and practically relevant contribution to exploring the vulnerabilities of open-weight molecule language models. We respectfully ask that the final decision take into account the additional evidence and the revised manuscript.

Thank you very much for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.