acceptodds
Withdrawn from ICLR 2027

How Creative Are Large Language Models in Generating Molecules?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Molecule generation requires satisfying chemical and biological constraints while searching a large and structured chemical space. This makes molecule generation a non-binary problem: many distinct molecular structures can satisfy the same constraints, and effective models must identify non-obvious solutions while maintaining exploration. From this perspective, creativity is a functional requirement in molecular generation rather than an aesthetic notion. Large language models (LLMs) can generate molecular representations directly from natural language prompts, but it remains unclear what kind of creativity they exhibit when generating molecules. Through a systematic evaluation of 10 general-purpose LLMs across 15 molecular generation tasks, we study creativity along two complementary dimensions: convergent creativity, which captures validity and constraint satisfaction, and divergent creativity, which captures novelty, uniqueness, and structural exploration. Our central finding is that higher convergent creativity is consistently associated with lower divergent creativity: the relationship is negative within all 15 evaluation tasks. We next examine how molecular creativity varies with three factors: task structure, model family, and inference-time conditioning. In the tested compatible multi-constraint tasks, adding constraints does not necessarily reduce success; across size-varied checkpoints, the LLaMA families gain convergent creativity while losing divergent creativity, whereas Qwen3 improves both; and in-context demonstrations can sharply increase activity-task success while reducing exploration and directly reproducing supplied examples. We further test whether creativity enhanced in language transfers to molecular generation. In a controlled same-parent comparison, the three creativity-oriented variants fail to reproduce their targeted molecular gains.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Large Language Models, Creativity, Molecule Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Gar6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates how 10 general-purpose LLMs balance constraint satisfaction and structural exploration across 15 molecular-generation tasks. It combines validity, success, novelty, uniqueness, and diversity into convergent (CC) and divergent creativity (DC), examining scale, constraints, ICL, temperature, and prompting. It reports negative CC–DC associations, model-family differences, ICL copying, and failed transfer of language-creativity preference training from a shared SFT parent.

Critical strengths
  1. Multiple models/tasks, within-task comparisons, fixed effects, and alternative metrics provide more insight than a single-metric leaderboard. The authors acknowledge Qwen3’s simultaneous CC/DC gains and inconsistent transfer of some interventions.
  2. ICL copying diagnostics have practical value: exact copies constitute 30.5%–65.5% of valid outputs despite improved activity-task success and near-ceiling reference-library novelty.
Critical weaknesses
  1. I could not reproduce the reported FC results from the definitions and aggregation procedure described in the paper. FC = E/G counts successful, novel, unique outputs among all generations. Averaging the five LogP targets first and weighting 11 families equally gives LLaMA3-8B FC = 0.344618: 3.622 sums ten non-LogP tasks; 0.1688 averages five LogP tasks. Fifteen-task weighting also disagrees beyond three-decimal rounding error. FC ≤ SR should hold under identical weights, yet the reported FC = 0.556 exceeds recalculated SR = 0.4344. Page 26 discloses no alternative counts, denominator, or task subset; Figure 9’s run-level view and Table 8’s legacy-corpus note concern other analyses. The scaling-related FC decline survives, but its magnitude and traceability need correction. Component means/SDs cannot recover mean nonlinear CC/DC scores.
  2. Threshold and valid-sample-size dependence remain unresolved. QED ≥ 0.6, SA ≤ 4, LogP error ≤ 1, and BBB/HIA/DRD2/JNK3/GSK3β scores ≥ 0.5 need protocol or validation support. Thresholding is reasonable, but metric references and high AUC do not establish cutoff-specific positive-prediction reliability. The authors acknowledge conventional cutoffs; their QED scan yields rank correlation as low as 0.067 and shifts of nine positions. Sensitivity of the central association and key model differences remains unclear, as does “absolute success rates are unaffected.” Expected Uniqueness = U/V can decrease with valid-sample count V even under an unchanged conditional molecular distribution, while CC also depends on validity. The fixed-size check covers only Diversity, leaving possible measurement-induced negative association in Uniqueness, DC, and the core regressions untested.
Questions for authors
  1. Reconcile all four main-text FC values with Tables 19–22, specifying data version, tasks, denominator, and aggregation. Explain FC > SR and whether corrections affect scaling comparisons or other conclusions.
  2. Identify prior protocols or independent validation data supporting the cutoffs above; provide threshold-specific predictor performance/calibration, especially positive-prediction reliability, and specify how single-task criteria combine in multi-constraint tasks. Using saved continuous scores, extend the QED analysis to associations, key differences, and uncertainty intervals; clarify the statement about unchanged absolute success rates.
  3. Recompute Uniqueness, Diversity, DC, within-task correlations, and family/model fixed-effects analyses at a common valid-sample count, e.g., n = 40 given the reported minimum of 43. Persistence would strengthen the central claim.
  4. Please clarify the analysis units and aggregation procedures underlying the regression results. In addition, please clarify how the “six tasks” referenced in the CRE–SFT analysis relate to the sample sizes
Reviewer_bRj7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the ability of general-purpose large language models to generate molecules that simultaneously satisfy chemical constraints and explore diverse regions of chemical space.

The authors propose evaluating molecular creativity along two dimensions: convergent creativity (CC), combining molecular validity and constraint satisfaction, and divergent creativity (DC), combining novelty, uniqueness, and structural diversity.

Through experiments involving ten language models and fifteen molecular generation tasks, the paper reports a recurring negative association between the two dimensions. Additional analyses investigate the effects of model family, model size, inference-time conditioning, and creativity-oriented preference tuning.

The main conclusion is that improvements in constraint satisfaction are frequently accompanied by reduced chemical-space exploration, motivating joint evaluation of these objectives.

Critical strengths
  1. Relevant evaluation question.

Understanding the relationship between constraint satisfaction and chemical-space exploration is important for molecular generation. The paper addresses a meaningful limitation of evaluations that focus exclusively on optimization performance or molecular diversity.

The attempt to characterize this relationship specifically for general-purpose language models is relevant given their increasing use in molecular design.

  1. Broad experimental investigation.

The study considers multiple language models, molecular objectives, and inference-time configurations. The inclusion of in-context learning and creativity-oriented model variants provides useful observations about how different generation strategies influence molecular outputs.

Critical weaknesses
  1. The central conclusions rely heavily on insufficiently validated aggregate creativity metrics.

Convergent creativity is defined as the geometric mean of validity and success rate, while divergent creativity combines novelty, uniqueness, and diversity through another geometric mean.

Although the authors provide a motivation for geometric aggregation, the scientific interpretation of these composite quantities remains insufficiently established.

In particular, success rate already incorporates molecular validity. Consequently, the CC metric effectively gives additional weight to validity, potentially changing model rankings and the strength of the reported relationship between constraint satisfaction and exploration.

More generally, the observed negative association between CC and DC may depend on the selected metrics, their relative weighting, and their aggregation.

The paper would be substantially more convincing if the central conclusions were shown to remain stable when considering the individual metrics, alternative aggregation schemes, and success rates computed directly over all generated molecules.

Without such an analysis, it is difficult to determine whether the reported trade-off reflects a fundamental property of molecular generation or partly results from the proposed evaluation framework.

  1. Aggregation across heterogeneous models and tasks obscures the generality of the main finding.

The paper emphasizes a recurring negative relationship between convergent and divergent creativity. However, the evaluated models exhibit substantially different capabilities, and the manuscript itself reports cases where model scaling improves both dimensions.

This raises an important question about whether the observed relationship represents a general trade-off or an aggregate statistical pattern that masks different behaviors across model families.

The main figures do not sufficiently expose the individual components of CC and DC or the heterogeneity of the model-specific relationships. Some analyses also focus on a subset of models without fully establishing how representative these are.

Consequently, the results do not clearly distinguish an intrinsic trade-off from differences in model quality, sampling behavior, or the selected aggregation procedure.

Questions for authors
  1. Metric robustness: Does the reported negative CC–DC association remain when CC is replaced by the overall molecular success rate, and when novelty, uniqueness, and diversity are analyzed separately rather than through their geometric mean?
  2. Model heterogeneity: Can the authors report the relationship between constraint satisfaction and exploration separately for each model family, including confidence intervals? Is the negative association observed consistently, or is it driven by differences between weaker and stronger models?
  3. Aggregation sensitivity: How sensitive are the main conclusions to alternative weighting schemes, arithmetic aggregation, and different novelty reference databases?
Reviewer_8QSdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper talks about how to assess an LLM’s ability to produce molecules, and for this they introduce creative and divergence convergence metrics. Convergent Creativity (CC), measures molecular validity and constraint satisfaction, and Divergent Creativity (DC), measures novelty, uniqueness, and structural diversity. They evaluate 10 different LLMs on 15 different tasks and note that there is a negative correlation with CC and DC metrics.

The paper further investigates how these metrics vary across model sizes, model families, task categories, and inference-time strategies, including temperature control, diversity-oriented prompting, and in-context learning. Finally, they trained an LLM trained for creativity orientation with DPO on to see if it improves the metrics.

Critical strengths
  • Clear presentation and evaluation: The paper provides a clear comparison of different LLMs using intuitive visualizations. The presentation of the CC–DC relationship across tasks, model families, and inference-time strategies makes the experimental observations easy to follow.

  • Rigorous and extensive evaluation: The paper evaluates 10 different LLMs of varying sizes and families across 15 molecular generation tasks. The authors also conduct statistical analyses, including within-task correlations, fixed-effects models, and robustness checks, to support their observations.

  • Interesting observations regarding in-context learning: The paper demonstrates that providing molecular examples through in-context learning can substantially improve success rates on biological activity tasks. The additional analysis showing that these improvements are accompanied by increased copying of demonstrations is particularly useful. This highlights an important limitation of using in-context learning for molecular generation, where improvements in constraint satisfaction may arise from reproducing known examples rather than exploring new molecular structures.

Critical weaknesses
  • Limited novelty and no justification behind the motivation for CC and DC metrics:

My primary concern is regarding the novelty about these metrics. Both CC and DC is the direct geometric mean of already existing widely used metrics in molecular generation evaluation. In particular, CC is defined as:

CC = sqrt(Validity × Success Rate)

where validity is the fraction of valid generated molecules and success rate is the fraction of all generated molecules that are both valid and satisfy the task constraints. This makes the metric rely more on validity than hit rate. Similarly, DC combines novelty, uniqueness, and diversity, which are already standard molecular generation metrics. The paper does not sufficiently demonstrate what additional insight is obtained by combining these metrics under the terminology of divergent creativity.

  • Unclear training details of CrPo-SFT and DPO. Insufficient discussion of related work on DPO and temperature controlled molecule sampling:

In section 6, creativity-oriented preference tuned llama model was used, but it is unclear how this was trained, and DPO was also used on top of this model, it is further unclear how it was performed. Furthermore, prior work such as SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration (Cavanagh et al., 2024) has already explored adapting LLMs for molecular generation using SFT and DPO, with evaluation of molecular generation validity, success rate and diversity. SmilleyLlama also reports results related to diversity, validity and novelty changes with temperature and DPO. This work appears to be missing from the references and the comparison.

  • Insufficient differentiation from existing molecular generation benchmarks

Existing works like PMO (Gao et al., 2022) and guacamole (Brown et al., 2019) benchmark already evaluate on diversity, validity and success rate metrics. Several tasks considered in this paper, including QED, DRD2, JNK3, and GSK3β, also overlap with objectives studied in established molecular optimization benchmarks and their metrics. The authors should more clearly distinguish their evaluation protocol from GuacaMol and PMO and explain why the selected tasks are appropriate for assessing molecular creativity rather than using guacamole and PMO benchmark.

  • Divergent creativity does not necessarily capture diversity among successful molecules

Another important concern is that DC is calculated over all valid generated molecules, regardless of whether they satisfy the requested molecular constraints. As a result, a model that generates many structurally diverse but unsuccessful molecules may receive a high DC score. This is particularly relevant for the biological activity tasks, where several models exhibit low success rates but high divergent creativity.

Questions for authors

Questions for the authors:

  1. Motivation behind CC and DC: Can the authors demonstrate cases where the composite metrics lead to conclusions that would not be apparent from the existing metrics?

  2. Definition of convergent creativity: Since success rate already includes molecular validity, why is validity incorporated again through the geometric mean? Would using constraint satisfaction conditional on validity provide a more balanced measure?

  3. Diversity among successful molecules: Have the authors considered computing DC only over molecules satisfying the requested constraints? Would the negative CC–DC relationship still hold under this definition, particularly for biological activity tasks?

  4. Relationship to existing benchmarks: How does the proposed evaluation framework differ from GuacaMol, PMO, and MOSES beyond the aggregation and interpretation of established metrics? What aspects of molecular creativity are uniquely captured by the proposed evaluation?

  5. Comparison with SmileyLlama: How do the findings on sampling temperature and preference optimization differ from those previously reported in SmileyLlama (Cavanagh et al., 2024)? Could the authors discuss this work and clarify the additional insights provided by the present study?

  6. Post-training methodology: Could the authors provide more details on the CrPO-SFT parent model, the preference datasets, and the training configurations of the DPO and creativity-oriented variants? To what extent might the observed decline in validity depend on the specific training setup?

Overall Assessment

The paper presents an empirical evaluation of general-purpose LLMs for molecular generation and provides useful observations about the relationship between constraint satisfaction and structural exploration. The broad evaluation across models and tasks, together with the analysis of in-context learning and demonstration copying, is valuable.

However, my main concern is the limited methodological novelty of the proposed creativity metrics, which are largely aggregations of established molecular generation measures. The paper does not sufficiently demonstrate that these composite scores capture meaningful aspects of molecular creativity beyond conventional evaluation metrics. In addition, several experimental findings overlap with prior work on molecular generation, and the relationship to existing benchmarks and molecular LLM adaptation methods could be better established. I also believe that evaluating structural diversity among successful molecules is important for interpreting the central CC–DC trade-off. Without this analysis, high divergent creativity may reflect unsuccessful exploration rather than the discovery of diverse molecules satisfying the target constraints.

Addressing these concerns would substantially strengthen the paper's contribution and clarify the significance of its findings for molecular generation research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.