- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper extends the rule-based IUPAC parser OPSIN to produce enriched XML metadata covering ring topology, atom labeling, substituent attachment, and stereochemistry. This metadata is fed to GPT-5.2 to generate structure descriptions, from which the authors build MolLangData, a set of about 163k molecule–description pairs. On a 2,000-sample subset checked by three LLMs plus expert reconstruction, they report 98.6% precision. They then train Qwen3.5-4B with SFT followed by DAPO-based RL for description-to-SMILES generation. Exact match rises from 12.53% to 84.99% on the internal test set, and to 65.0% and 72.0% on the MolLangBench core and extended sets.
Critical strengths
-
Transparent validation of description correctness. The validation protocol is unusually thorough for an LLM-generated dataset. It combines three-model reconstruction consensus, an explicit false-positive study (200 consensus-accepted samples were checked by humans), and dual-expert reconstruction of every sample that fails automatic validation, for roughly 270 expert-hours in total. This matters to my assessment because most existing molecule–text datasets (e.g., Mol-LLaMA- or KnowMol-style descriptions) give no reliable measure of correctness. A verified 98.6% precision makes the dataset credible as a supervision signal, whatever one thinks of its downstream value.
-
Open release of a reusable tool and artifacts. The authors release the OPSIN extension, which is designed to stay compatible with upstream OPSIN and can be checked against it, along with the dataset, the SFT corpus, and model checkpoints. The community can regenerate descriptions on demand beyond PubChem. This gives the work some value as infrastructure even if the scientific claims are only partly supported.
Critical weaknesses
-
The claimed motivation and the evaluated task do not match, and closely related work is missing. The abstract and introduction say the goal is to let LLMs "generate molecules from natural-language design intent." The only generation task evaluated, however, is exact reconstruction from a complete, unambiguous, atom-by-atom description that averages 261 words and can exceed 1,300. Because each description comes from an IUPAC name that OPSIN already converts to a structure deterministically, the task amounts to translating a verbose paraphrase of an IUPAC name back into SMILES. Real design intent is underspecified and one-to-many.
-
Prior work has argued explicitly that one-to-one molecule–text mappings measure recall of a predefined answer rather than generation ability, and has proposed benchmarks and instruction data for open-domain, text-driven molecule editing, optimization, and customized generation: S²-Bench with OpenMolIns, KDD 2026. This line of work is neither cited nor discussed, and no evaluation shows that models trained on MolLangData transfer to open-ended design tasks. This weakness is central to my recommendation because the paper's main claim of significance, that MolLangData is "a reliable foundation for molecule–language alignment" for chemical reasoning and design, rests on a task that does not exercise those abilities.
-
The novelty is mainly engineering, and the LLM step is not shown to be necessary. The core contribution is case-by-case "trial-and-error engineering" (Sec. 3.2) that adds back information already handled inside OPSIN. Once the enriched XML fully encodes topology, labels, attachments, and stereochemistry, the obvious baseline is deterministic, template-based verbalization. That baseline would be nearly 100% correct by construction and would cost almost nothing, compared with the more than $20k spent on API calls. Without a comparison to such a baseline, it is unclear what the LLM contributes beyond linguistic variety, and whether that variety helps downstream.
Questions for authors
- Template baseline. What precision and downstream training performance does a deterministic, rule-based verbalizer of your enriched XML achieve? If it matches LLM-generated descriptions, what is the justification for the LLM step?
- Split and generalization. The train/test split is random within the same PubChem pool. What is the post-RL exact-match rate under a scaffold split? How should we read the drop from 84.99% on the internal test set to 65.0% and 72.0% on human-written MolLangBench: as a description-style shift or a structural shift?
- Shared models across the pipeline. GPT-5.2 generates the descriptions and is also one of the three validators. GLM-5.1 is both a validator and an SFT teacher. Do the precision estimates and the SFT gains still hold if validation uses model families not involved in generation or distillation?
- Transfer to open-domain generation. Does the MolLangData-trained model improve over the base Qwen3.5-4B on open-domain, text-driven editing, optimization, or customized generation benchmarks such as S²-Bench (MolEdit, MolOpt, MolCustom)? A positive result would substantially strengthen the claim that structure-grounded alignment supports design-oriented tasks.
5 comments
Response to Reviewer NNno (2/4)
W3 and Q1:
A rule-based verbalizer of the XML would indeed be nearly 100% correct as reviewer noted, but its output is still a fixed notation written in text, much like any other rule-based chemical format. It is not how an AI system expresses its candidates (use 2) or how people talk to a drawing tool (use 1). Using a template would require yet another layer to translate between free-form language and the template. A model trained on template output would learn the template rather than how people and LLMs actually describe structures. This is why we use LLMs to write the descriptions, and why this step is necessary.
Regarding novelty, we hope the scope described above already makes our contribution clearer, and shows that it goes well beyond engineering. To our knowledge, MolLangData is the first large-scale dataset of natural-language molecular descriptions that are structurally complete and go through strict quality calibration.
On the engineering itself, we'd like to highlight how challenging this part actually is. IUPAC nomenclature is a very complex system (the IUPAC Blue Book takes more than 1000 pages to introduce its rules). Even OPSIN, maintained for more than a decade, still leaves many corner cases unhandled, and so does our tool. We spent a lot of effort working through these branches and corner cases to build a rigorous XML for guiding description generation, and it differs substantially from OPSIN's native output. Fig. 2 and Fig. B.2 show this difference, comparing the two XMLs for the same molecule. That said, we don't consider this engineering our main contribution.
We would also like to share how we see this work beyond the specific tasks discussed above.
Frontier LLMs have been improving quickly on this task. On the earlier MolLangBench, GPT-5 reached only about 40%, while in this paper GPT-5.6-Sol already exceeds 90%. It's quite possible that future LLMs will solve it entirely. But we think this is far from the end of the story.
One reason is efficiency. Even our model, which is far smaller and more token-efficient, still uses around 1K tokens per structure, as reported in the paper. A chemist looking at a description just draws the structure, without reasoning through it step by step. Deployment matters too. It's hard to imagine putting a frontier-scale LLM inside drawing software like ChemDraw, or calling an external API every time someone wants to draw a molecule. A small, specialized model trained on MolLangData feels like a much more natural fit there.
We also think the dataset has broader uses. Molecules are graphs, so this dataset could help with graph–language alignment more generally, beyond chemistry.
We feel this scope is really the key to understanding our paper, and we'd be very happy to keep discussing any of these points with the reviewer.
Response to Reviewer NNno (3/4)
Q2. How does the model do under a scaffold split, and is the drop from the internal test set to MolLangBench due to a description-style shift or a structural shift?
We thank the reviewer for this good question. We first discuss the drop from the internal test set to MolLangBench, then the scaffold split.
The drop on MolLangBench. We'd first like to note that, although performance on MolLangBench is lower than on our internal test set, training on MolLangData still brings a large improvement there. The base Qwen3.5-4B reaches only 12.5% and 17.0% on MolLangBench, and after training on our dataset it reaches 65.0% and 72.0%. As reviewer noticed, these numbers are still below the 84.99% on our internal test set. We noticed this gap also and looked into it carefully.
We believe the drop comes mainly from a difference in description style rather than a structural shift. Descriptions written by humans differ in style from those generated automatically, and when generating MolLangData we did not prompt for any particular range of linguistic styles.
To test this, we ran a simple experiment. We asked GPT-5.6-Sol to rewrite each MolLangBench description, using the original description and the SMILES as references, so that the molecule stays the same and only the writing style changes. On these rewritten descriptions, performance rises a lot and comes close to the internal test results:
| Core | Extended | |
|---|---|---|
| Original MolLangBench | 65.00% | 72.00% |
| Rewritten by GPT-5.6-Sol | 79.50% | 85.00% |
Since the molecules are identical and only the style changes, this suggests the gap is mainly a description-style shift, and our model currently does better on AI-style descriptions. We see this as an interesting direction for future work. But MolLangData provides a natural base for it, since LLMs can rewrite our descriptions into different styles.
Scaffold split. We thank the reviewer for this suggestion. In our current split, we make sure that no molecule appears in both the training and test sets, but we did not separate them by scaffold.
We'd like to share our view on this setting. MolLangData is designed as a broad, general-purpose dataset that covers most common structures in chemical space, so the main goal is coverage rather than extrapolation to unseen topologies of chemicals. The task is also different from property prediction, where scaffold splits are most important because a model can exploit correlations between scaffolds and properties. In our task, the description specifies the full structure, so the model has to assemble the molecule from the text rather than recall it from a familiar scaffold.
But anyways, we agree that a scaffold split would be a useful additional analysis. Because each training run takes about two weeks on 16 H100 GPUs, we couldn't complete it within the rebuttal period, but we will consider it in future work. We believe this does not affect the core contribution of the paper.
Response to Reviewer NNno (1/4)
We apologize for the delay in our response, and we would like also thank the reviewer for taking the time to review our work and for the detailed comments. Below, we group the weaknesses and questions and address each of them.
We thank the reviewer for these comments. We think they come mainly from a misunderstanding of what our dataset is meant to do, partly due to our wording, so we first clarify the scope and then answer each point.
What MolLangData is for:
We agree that real molecular design is one-to-many. If we want a molecule with a certain property, many structures can satisfy it. With LLMs, though, design works in two sequential steps:
The LLM reasons about the intent and proposes one or more candidate structures. Because it is a language model, it describes these candidates in natural language (which rings, which groups, where they connect), not directly in SMILES, IUPAC names, or another format.
Each candidate description then has to be turned into a concrete structure (SMILES or a graph). This step is one-to-one.
MolLangData does not target step 1. It targets step 2: the interface between natural-language descriptions and molecular structures, in both the perception and generation directions.
The language-conditioned generation experiment evaluates the description-to-structure direction. We see two practical uses for it (also discussed in the experiment section):
Natural-language (or voice) chemical drawing. The most direct application is chemical drawing software that takes natural-language or voice instructions. A user could describe the target, e.g., "draw a benzene ring with a hydroxyl and a methoxy in the meta position" (real instructions can be arbitrarily complex), and the tool renders it directly, without the user drawing each component and connection by hand.
LLM-centric AI systems and automated labs. This is the two-step process discussed above. An AI reasons about a chemical problem, ends up with a candidate described in language, and then needs it as a concrete structure for downstream tools. A reliable description-to-structure model is what makes this step possible. Text-to-image generation works the same way: the LLM first reasons about the user’s intent and writes a detailed prompt, and the image model then renders that prompt faithfully.
We hope this clarifies the scope of our work. With it in mind, the reviewer's points can be answered more directly.
W1:
The one-to-many part of design is step 1. The one-to-one part, which we target, is step 2. Our language-conditioned generation experiment tests exactly this step: a complete description goes in, and the model has to return that exact molecule. Both use cases we discussed above need precisely this.
W2 and Q4:
We thank the reviewer for pointing us to S²-Bench. With the scope above, the distinction between the two works becomes clear. As shown in their Figure 1, S²-Bench covers optimization and some structure manipulation tasks. The optimization task, such as "decrease the logP of this molecule," falls under step 1. It also covers customized generation tasks such as "generate a molecule with 6 carbon atoms." These look closer to our step 2, but there is a difference. A prompt like this describes a constraint rather than a concrete design: a molecule with six carbon atoms could be a chain, a ring, branched, or substituted in different ways. When an LLM reaches a concrete design, the design needs to be unambiguous, since the downstream process requires one specific structure. MolLangData targets this setting, translating a complete description into exactly the intended molecule.
For the same reason, S²-Bench is not a direct test of what our dataset is meant to train, and results on it would mostly reflect the base model's step-1 reasoning ability rather than the quality of the language–structure interface.