- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SREV-Mol, a molecular generation framework designed to satisfy natural-language structural requirements and multiple property constraints. The framework first translates structural descriptions into SMARTS and generates verified seed molecules. It then uses matched molecular pair (MMP) statistics to construct a structure-preserving edit space, while Structured Edit Reasoning (SER) and candidate-aligned Auxiliary Feature Tokens (AFTs) assist the language model in selecting molecular modifications through iterative verification.
Experiments on 900 in-distribution (ID) and 300 scaffold out-of-distribution (OOD) tasks report joint success rates of 67.8% and 29.0%, respectively, outperforming the strongest reported baselines at 59.1% and 24.0% in the corresponding settings.
Critical strengths
S1. Meaningful problem setting and competitive end-to-end performance.
The paper addresses a practically relevant molecular constraint-satisfaction problem by combining natural-language structural requirements with multiple property intervals. Tables 2 and 3 demonstrate that SREV-Mol achieves the highest joint success rates among the evaluated methods on both ID and Scaffold-OOD benchmarks. These results provide positive evidence for the effectiveness of the complete system on the proposed tasks.
S2. Empirical evidence supporting MMP-based search-space reduction and structural preservation.
Figure 3(a) shows that random editing within the MMP candidate space achieves 64.7% success when reference SMARTS and ground-truth seeds are provided, substantially outperforming the evaluated generic random-editing strategies. Table 4 further demonstrates that removing structure-preserving filtering reduces ID joint success from 67.8% to 47.6%. These results provide convincing evidence that incorporating structural constraints and property-informed edit statistics is beneficial for molecular refinement.
Critical weaknesses
W1. The independent contribution and necessity of SER/AFT-based agentic review are not sufficiently established through end-to-end evaluations.
SER and AFTs are presented as central technical contributions. However, the ablations in Section 4.3.2 and Figure 3(b) are conducted only on a separate Qwen3-1.7B model and evaluate step-level edit-selection accuracy rather than the final success of the complete Qwen3-8B system. Although the feature perturbation experiments in Figure 11(c) indicate that the model uses candidate-specific AFT information, they do not establish how much SER and AFTs independently improve end-to-end generation success.
This concern is particularly important because Table 2 shows that, compared with Claude-Opus-5.0 + SREV, SREV-Mol improves structural satisfaction from 70.8% to 82.8%, whereas complete property satisfaction increases only from 79.9% to 80.6%. Consequently, the overall improvement may be primarily driven by the specially trained structural interpretation and seed-generation stage rather than more effective edit selection.
Furthermore, Appendix C.4 already introduces a pretrained candidate-value predictor for estimating downstream editing success, yet the paper does not evaluate a non-LLM policy that directly ranks candidates using this predictor. Figure 3(a) reports 94.5% success for MMP-based beam search. Although this controlled experiment uses reference SMARTS and ground-truth seeds and is therefore not directly comparable to the end-to-end setting, it suggests that the underlying MMP search space already provides substantial optimization capability.
The paper therefore does not adequately answer a fundamental question: What additional benefit does LLM-based Review provide beyond simpler MMP search or learned value-based candidate selection?
W2. Baseline coverage and evaluation design are insufficient to fully substantiate the claimed methodological advantages and generalization.
First, although the related-work section discusses OptADMET, M4olGen, MolReAct, and PROBE, the main experiments do not directly compare against these closely related editing or optimization approaches under matched seeds, property evaluators, and verification budgets. Strong non-agent optimization baselines are also missing. Moreover, the evaluated chemistry agents exclude some inaccessible tools from their original implementations. Consequently, their near-zero joint success rates do not, by themselves, establish superiority over related molecular optimization methods. Even if these methods cannot directly handle the complete natural-language task, controlled optimization experiments starting from common seeds would provide a more meaningful comparison.
Second, Appendix E.3 explicitly states that the edit-selection dataset is randomly split at the individual-round level rather than by source task or complete trajectory. Since each example retains its full preceding trajectory context, highly correlated states originating from the same task may appear in both training and test sets. This does not demonstrate actual data leakage, but it weakens the independence of the evaluation used to support SER/AFT generalization.
Finally, property constraints are constructed around feasible reference molecules, and test structural descriptions undergo selection, with approximately 25–35% of candidate descriptions passing round-trip verification. While this construction is reasonable for ensuring benchmark consistency, it limits the conclusions that can be drawn about unfiltered natural-language requests. Moreover, despite emphasizing support for up to 14 simultaneous property constraints, the paper does not report success rates stratified by constraint count. Paired uncertainty estimates or statistical significance tests for the main OOD differences are also absent.
These limitations do not invalidate the reported benchmark results, but they substantially restrict the strength of the broader methodological and generalization claims.
Questions for authors
Q1. Can the authors provide full Qwen3-8B end-to-end ablations comparing Direct Review, SER-only, SER+AFT, direct AFE value-predictor ranking, and MMP greedy selection under matched Stage 1, training data, initial seeds, and verification budgets? Such experiments are essential for identifying the independent contribution of the proposed review components.
Q2. Can the edit-selection evaluation in Figure 3(b) be repeated using task-disjoint or trajectory-disjoint train/test splits to prevent correlated states from the same trajectories from appearing across splits? Do the gains from SER and AFTs remain consistent?
Q3. Can the authors report performance by the number of simultaneous property constraints, particularly for 10–14 constraints, provide paired statistical analyses on Scaffold-OOD, and clarify the absence of task-aligned non-agent or related fragment-editing baselines?
7 comments
2. Task-Aligned Optimization Baselines (W2, Q3)
First, although the related-work section discusses OptADMET, M4olGen, MolReAct, and PROBE, the main experiments do not directly compare against these closely related editing or optimization approaches under matched seeds, property evaluators, and verification budgets. Strong non-agent optimization baselines are also missing. … and clarify the absence of task-aligned non-agent or related fragment-editing baselines?
→ We first clarify that our task aims to jointly satisfy complex natural-language structural requirements and multiple property constraints, rather than optimizing molecular properties alone.
Although OptADMET provides public code, its required databases, model weights, and some implementation files are unavailable. Similarly, executable implementations or checkpoints for M4olGen and MolReAct were unavailable. We therefore additionally evaluated PROBE, structure-based drug design agent, by initializing its InteractionAgent with ground-truth seed SMILES, replacing its Vina-based objective with our normalized property-gap objective, and excluding edits that violate the reference SMARTS. Since each PROBE loop involves approximately ten editing and verification operations, we allowed up to six loops rather than six individual edits.
| Method | Succ.↑ | Struct.↑ | STRatio↑ | Prop.↑ | PTRatio↑ | Gain↑ |
|---|---|---|---|---|---|---|
| PROBE | 17.0 | 97.9 | 97.9 | 17.0 | 75.7 | 35.2 |
| SREV-Mol | 67.8 | 82.8 | 85.2 | 80.6 | 96.6 | 90.5 |
Table R15. Comparison with task-adapted PROBE on 900 ID instances.
PROBE achieves lower property satisfaction despite even starting from ground-truth seeds and using additional internal edits(Table R15).
We additionally evaluated GraphGA, REINVENT, and GPBO, the top three methods in the PMO benchmark. We adapted them to start from ground-truth seed SMILES while preserving the required structure. REINVENT used seed-conditioned generation with varying attachment sites, while GraphGA and GPBO restricted modifications to non-seed regions using ZINC-derived BRICS fragments. All methods used our normalized property-gap objective and evaluators. We measured the evaluation budget by the number of times the property evaluators were called to assess candidate molecules during optimization.
| # Property Evaluations | GraphGA | REINVENT | GPBO | SREV-Mol |
|---|---|---|---|---|
| 6 | 6.2 | 11.8 | 8.6 | 67.8 |
| 10 | 10.0 | 16.3 | 11.2 | - |
| 100 | 37.6 | 45.1 | 36.2 | - |
| 500 | 66.8 | 67.1 | 73.7 | - |
| 1,000 | 78.9 | 77.8 | 80.2 | - |
| 5,000 | 95.8 | 88.0 | 94.0 | - |
| 10,000 | 98.2 | 88.2 | 96.4 | - |
| 50,000 | 98.9 | 88.2 | 96.4 | - |
Table R16. Table R16. Joint success rates (Succ.) under varying property evaluation budgets during optimization on 900 ID instances. Structural satisfaction is enforced at 100%.
Although we adapted these baselines to compare their optimization capabilities under as closely matched conditions as possible, a fully fair comparison remains challenging due to differences in their optimization mechanisms and computational costs. In particular, the reported evaluation budgets only account for property evaluations during optimization, excluding those used for SREV-Mol's MMP library construction and training-data generation. We will clarify these limitations in the revised paper.
3. Clarification of Excluded Chemistry Agent Tools
Moreover, the evaluated chemistry agents exclude some inaccessible tools from their original implementations.
Regarding the excluded chemistry-agent tools, we omitted RXN4Chem (forward synthesis and retrosynthesis), ChemSpace (reactant-price lookup), and Tavily (web search) due to API accessibility or usage limitations. Specifically, RXN4Chem and ChemSpace were not accessible through available API credentials, while Tavily's limited API quota was insufficient for our evaluation. These tools are primarily intended for synthesis-related tasks or external information retrieval, rather than direct molecular generation and property optimization.
4. Train-Test Independence in Edit-Selection Evaluation (W2, Q2)
Can the edit-selection evaluation in Figure 3(b) be repeated using task-disjoint or trajectory-disjoint train/test splits?
→ To address potential correlations in the original round-level split, we repeated Figure 3(b) using trajectory-disjoint training and test sets, sampling 100K and 5K rounds, respectively. All other settings remained unchanged, and results were averaged over four random seeds.
| Method | Oracle Acc. (%) | Reference Acc. (%) | Oracle Acc. Hard (%) | Reference Acc. Hard (%) |
|---|---|---|---|---|
| Direct (SREV-Mol* (w/o SER, w/o AFT)) | 48.2 ± 0.8 | 35.7 ± 2.7 | 45.8 ± 1.4 | 35.6 ± 2.0 |
| Freeform CoT | 44.0 ± 1.2 | 30.5 ± 0.5 | 41.8 ± 4.5 | 30.2 ± 5.1 |
| Freeform CoT (w/ features) | 42.1 ± 1.3 | 30.9 ± 0.4 | 35.8 ± 3.0 | 23.9 ± 4.1 |
| SREV-Mol* (w/o AFT) | 51.7 ± 1.1 | 40.9 ± 0.6 | 49.4 ± 1.2 | 40.1 ± 0.9 |
| SREV-Mol* | 52.6 ± 1.3 | 41.6 ± 1.0 | 51.8 ± 1.7 | 42.5 ± 2.1 |
Table R17. Edit-selection accuracy under trajectory-disjoint train/test splits.
The performance trends remain consistent with Figure 3(b): SER-only outperforms Direct and Freeform CoT, while AFTs further improve all four accuracy metrics(Table R17). These results confirm that the benefits persist under trajectory-disjoint evaluation.
5. Effect of Round-Trip Verification Filtering (W2)
Test structural descriptions undergo selection, with approximately 25–35% of candidate descriptions passing round-trip verification. … it limits the conclusions that can be drawn about unfiltered natural-language requests.
→ We introduced round-trip verification to ensure that natural-language structural descriptions faithfully correspond to their reference SMARTS, enabling reliable evaluation of structural satisfaction. To examine the effect of this filtering process, we constructed two sets of 1,000 instances each: RTPass, containing descriptions that passed round-trip verification, and RTFree, containing descriptions generated without applying round-trip verification.
We first compared the structural distributions of the two datasets.
| Structural Feature | Dataset | Mean | Max |
|---|---|---|---|
| # Stereocenters | RTPass | 1.20 | 10 |
| RTFree | 3.17 | 26 | |
| # Heavy atoms | RTPass | 33.1 | 71 |
| RTFree | 38.1 | 73 | |
| Molecular weight (g/mol) | RTPass | 466 | 983 |
| RTFree | 537 | 999 | |
| # Rings (SSSR) | RTPass | 4.11 | 11 |
| RTFree | 4.82 | 30 |
Table R18. Structural distribution comparison between RTPass and RTFree.
RTFree contains more complex structures, particularly in stereocenters and ring systems (Table R18). However, RTPass still includes molecules with up to 71 heavy atoms, 10 stereocenters, and 11 rings, indicating that round-trip filtering does not restrict evaluation to simple molecules.
To further examine whether round-trip filtering affects the types of structural errors encountered by SREV-Mol, we evaluated Stage 1 (Translation and seed generation) on both datasets and categorized seed-generation failures relative to the reference structures.
| Error Type | RTPass | RTFree |
|---|---|---|
| Correct composition, incorrect arrangement | 260 (72.4%) | 391 (50.6%) |
| Failed to generate a molecule | 32 (8.9%) | 119 (15.4%) |
| Incorrect ring count | 32 (8.9%) | 112 (14.5%) |
| Incorrect ring composition (size/aromaticity/heteroatoms) | 33 (9.2%) | 98 (12.7%) |
| Other | 2 (0.6%) | 52 (6.8%) |
Table R19. Distribution of Stage 1 structural failure types for RTPass (359 failures) and RTFree (772 failures). Percentages are calculated over failed instances.
Stage 1 achieves 64.1% structural success on RTPass, compared with 22.8% on RTFree. This gap may reflect both the greater structural complexity of RTFree and potential inconsistencies between unfiltered descriptions and their reference SMARTS. Nevertheless, incorrect structural arrangement remains the dominant failure type in both datasets(Table R19).
These findings highlight the importance of round-trip filtering for ensuring reliable structural evaluation, while also showing that it affects the difficulty and distribution of test instances. Therefore, our filtered benchmark provides a more controlled evaluation but does not fully represent unrestricted natural-language requests. We will clarify this trade-off and its limitations in the revised paper.
We sincerely thank the reviewer for the detailed and constructive feedback and valuable suggestions for strengthening our component-level validation, baseline comparisons, and generalization analysis.
1. Independent Contributions of SER and AFTs Beyond Stage 1 (W1, Q1)
→ To isolate the contribution of Review to property optimization, we conducted an additional controlled evaluation on all 1,200 instances (900 ID and 300 Scaffold-OOD). Specifically, we first trained each method end-to-end using Qwen3-8B under identical training settings, and matched Stage 1 to ensure that all methods started from the same initial seed molecules and compared their optimization performance. We then evaluated only the Stage 2 optimization loop with a maximum of six edits, using the same Suggest candidate space and verification procedure.
We compared five methods: MMP Greedy, which selects the top-ranked Suggest candidate w/o Review; direct AFE value-predictor ranking, which selects the candidate with the highest pretrained success score; Direct Review, which fine-tunes the LLM to select edits without SER or AFTs; SER-only, which incorporates structured reasoning without AFTs; and SER+AFT, which represents the full SREV-Mol model. All LLM-based variants used the same Qwen3-8B backbone, training data, and training settings, varying only the Review components.
Table R14. Stage 2 optimization performance under matched structural inputs and verification budgets. The values separated by "/" within the parentheses represent the ID and Scaffold-OOD scores, respectively.
With Stage 1 fixed, SREV-Mol outperforms MMP Greedy by 10.25 percentage points (72.17% vs. 61.92%), demonstrating the benefit of learned candidate selection (Table R14). Direct AFE value-predictor ranking also achieves strong performance (70.42%), highlighting the utility of pretrained candidate-value prediction. The full model further improves success by 1.75 percentage points overall, with incremental gains from Direct Review (69.75%) to SER-only (70.50%) and SER+AFT (72.17%). Although AFE ranking performs slightly better on Scaffold-OOD (41.33% vs. 40.33%), SREV-Mol shows a larger advantage on tasks with 10–14 simultaneous property constraints (70.8% vs. 65.0%, +5.8 points; Table R20, Response 6). These results suggest that SER and AFTs provide additional benefits for challenging multi-property optimization beyond Stage 1 structural generation.
We will include these ablation results in the revised paper.