- Rating
- 3(+1)
- Confidence
- 2(−2)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MolRGen, a benchmark aiming at text-based DE NOVO molecule generation. The benchmark consists of 4500 structures and 12 classical molecule properties. Based on the proposed dataset, the author trains a specific LLM, RL-Molstral, which gains better performance than baselines. The results support that GRPO training can benefit reasoning capabilities of the LLM on these DE NOVO tasks.
Critical strengths
-
The writing structure of the paper is clear. The authors follow a "new DE NOVO problem --> new dataset proposed --> train a specific model on it --> evaluation from many aspects" pipeline. This makes the paper easier to follow.
-
The scale of the proposed dataset is large. This may reduce the problem of randomness.
-
The main evaluation consists of both general LLMs and chemistry-specific LLMs. This is necessary in evaluations of text-based molecule generation.
Critical weaknesses
-
In fact, the current story makes the novelty of this paper limited. The authors claim that they want to explore performing "open-ended molecular design", and this has been explored in [1][2]. The "molecule verifier", like RDKit calculations or similarity calculations, is also not a new technique. [3][4] also utilizes a GRPO-style training process to teach general LLMs how to learn knowledge of molecule generation. [5] has explored the topic of RL helping open-ended generation. Indeed, verifiers help LLMs learn, but this is not new, compared with previous work. Regarding the works mentioned above, I think that "novel, new" observations from the proposed benchmark seems to be weak. While I appreciate the artifacts the authors propose, the authors may point out the key differences between this work and previous benchmarks & RL models.
-
Additionally, some important ablations and baselines are missing. Chem-R and MolReasoner I mentioned, and some other baselines should be compared. Additionally, you may add supervised fine-tuning as an ablation. The evaluation does not seem strong enough without these comparisons. You may have some important results in the supplementary material, however, following ICLR 2027 instructions, you may either put them in your main text, or use an "Appendix" just after the references to show these results.
Here is another small suggestion: you may redesign Table 1, as the current version is not clear enough.
[1] Li, Jiatong, et al. "Speak-to-structure: Evaluating llms in open-domain natural language-driven molecule generation." Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2026.
[2] Hao, Li, et al. "Beyond chemical qa: Evaluating llm's chemical reasoning with modular chemical operations." Advances in Neural Information Processing Systems 38 (2026).
[3] Wang, Weida, et al. "Chem-r: Learning to reason as a chemist." Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2026.
[4] Zhao, Guojiang, et al. "Molreasoner: Toward effective and interpretable reasoning for molecular llms." arXiv preprint arXiv:2508.02066 (2025).
[5] Narayanan, Siddharth, et al. "Training a scientific reasoning model for chemistry." Advances in Neural Information Processing Systems 38 (2026): 157671-157710.
Questions for authors
See the weaknesses part. I think the core problem is the limited novelty compared to previous works (open-ended design, GRPO training for LLMs on molecule generation). So I recommend rejection. I have some other questions:
Do you use the same prompt template in your training and evaluation? If so, will it influence the results? If not, can you show the different prompt templates?
Have you ever tried applying your trained model on some other open-source text-based molecule generation benchmarks? If so, you can prove your concept that your verifier "really helps".
4 comments
Response to Rebuttal
Thank you for your response!
I will wait for the additional results as you mentioned.
Best,
Reviewer hsxS
Dear reviewer hsxS,
We just obtained the results of the evaluation of the MolReasoner checkpoint (post-trained on reference-based molecular generation) on our tasks. The performance is lower than of ChemDFM-R, mostly due to the model struggling with instruction following (the model mistakes SMILES and SELFIES notations for instance, or does not generate the compounds in parseable form).
This could be due to its size (only 7B), making it less robust when evaluated on slightly out-of-distribution tasks. You can find below the performances of MolReasoner:
| Model | Size | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| top-1 | top-16 | top-1 | top-16 | top-1 | top-16 | top-1 | top-16 | ||
| Gemma-4 | 31B | 0.65 | 0.37 | 0.67 | 0.48 | 0.68 | 0.52 | 0.7 | 0.56 |
| Mistral-Small-4 | 128B | 0.6 | 0.32 | 0.62 | 0.44 | 0.64 | 0.48 | 0.66 | 0.52 |
| MiniMax-M2 | 229B | 0.61 | 0.31 | 0.64 | 0.43 | 0.66 | 0.48 | 0.68 | 0.42 |
| RL-MolStral-g16 | 128B | 0.64 | 0.45 | 0.66 | 0.53 | 0.67 | 0.55 | 0.69 | 0.57 |
| ChemDFM-R | 14B | 0.62 | 0.31 | 0.65 | 0.41 | 0.66 | 0.46 | 0.69 | 0.5 |
| Ether0 | 24B | 0.55 | 0.22 | 0.58 | 0.31 | 0.59 | 0.36 | 0.62 | 0.42 |
| MolReasoner | 7B | 0.39 | 0.08 | 0.42 | 0.12 | 0.45 | 0.15 | 0.48 | 0.22 |
We will include this new baseline with a discussion of its behavior (similarly to the Ether0 section in the appendix) in the revised version of the paper.
Response to Authors' Rebuttal
Thank you for your extra experiments. I would like to raise my score to 3; however, the novel contributions of this paper are still not so convincing, and I will lower my confidence. I could not strongly support acceptance of this paper.
1. Novelty and positioning ([1]–[5]). We agree that none of the individual components of MolRGen is new in isolation — RDKit descriptors, GRPO, and verifier-based rewards all have precedents. Our claim is not any single component but the composition: a reference-free RL environment where rewards for de novo molecules are computed at generation time, and the findings it enables. Concretely, we see MolRGen as complementary to the cited works:
The distinction we consider essential: in [1]–[5], the verifier checks agreement with a known answer (a described molecule, a QA label, a reaction outcome). MolRGen's verifier instead scores novel molecules against a protein pocket and a property profile at generation time, which is what makes it usable as an RL environment rather than an evaluation set. We also note that [1]–[5] contain no structure-based objective: no cited work conditions on a protein pocket or runs docking inside the training loop. To the best of our knowledge, the combination of (i) reference-free pocket-conditioned prompts, (ii) docking-in-the-loop rewards for GRPO, and (iii) a diversity-aware evaluation exposing the diversity–exploitation trade-off of RL fine-tuning is new as a whole.
2. Missing baselines and ablations.
(All supplementary materials will be concatenated to the main paper for the final submission)
Prompt templates in training and evaluation. We support the use of any prompts, and provide tools to generate custom textual prompts given a task’s metadata, but we used the same prompt templates across training and evaluations up to simple adaptations for multi-property optimization.
Transfer to other benchmarks. We have not evaluated the trained models to other benchmarks, however regarding the generalization abilities of the model, we did evaluate it, since all proteins in the test set are unknown to the model (none leak from the training to the validation and test set).
We hope these answers clarify the contributions of MolRGen relative to prior work, and we remain at the reviewer's disposal for any further discussion.