- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper applies a looped Transformer to SMILES generation, reporting 95.3% validity on ZINC-250K with 53K parameters. It also analyzes errors, internal representations, and attention-head ablations to argue that repeated computation resolves brackets, rings, and valence in a fixed order.
Critical strengths
The 53K model achieves high validity with fewer parameters than the reported unshared baselines. The novelty and diversity checks, together with the additional architectural controls, make this a useful empirical demonstration of compact SMILES generation.
Critical weaknesses
-
Limited contribution and an unclear central concept. The paper does not clearly explain what "small" means in "Small Molecular Grammar." It neither defines grammar size nor connects this phrase to a measurable research question. The demonstrated result is primarily a favorable parameter–validity trade-off on ZINC-250K. Although the additional generation metrics are useful, it remains unclear what broader insight they provide into molecular grammar. This distinction could also be made clearer in the presentation.
-
The claimed bracket–ring–valence computational order is not sufficiently supported. Table 1 shows that all three error types decrease sharply between depths 1 and 2. This supports the narrower observation that bracket errors become rare earlier, while valence errors remain more prevalent and continue to improve, but does not establish that the model first processes brackets, then rings, and finally valence. Table 7 suggests timing differences but does not establish a computational dependency. Since each invalid string is assigned to a single error category, the classification procedure also needs clarification. Head ablation provides a targeted intervention for bracket-related behavior, but does not test the proposed three-stage order. The single-head localization is also reported for the 206K model, whereas the abstract emphasizes the 53K model without clearly distinguishing the scope of these results.
Questions for authors
-
Could the authors clarify what "small" means in "Small Molecular Grammar," and what insight into grammar learning is supported beyond parameter-efficient generation on this dataset?
-
What evidence distinguishes a fixed computational order from different rates of error reduction?
2 comments
Thank you for the response. My main remaining concern is the novelty and significance of the contribution. With the original claim of a fixed bracket-ring-valence ordering withdrawn, I find it difficult to identify the central new insight into grammar learning beyond applying an established weight-shared architecture to SMILES and analyzing its intermediate representations. The revised abstract and introduction also remain difficult to follow, and I find the term "decodability" hard to understand. Given the acknowledged use of AI writing assistance, I suggest that the authors review the revised text more carefully.
Thank you for your review and for recognizing that the 53K model achieves high validity with fewer parameters than the unshared baselines and that our novelty, diversity, and architectural controls provide a useful empirical demonstration of compact SMILES generation. New multi-label error, dependency, prefix-count probe, and 53K/206K ablation controls test each concern directly and calibrate our claim to a depth- and pass-dependent progression.
1. Error classification and computational order vs. error-reduction rates (Weakness 2, Question 2)
We agree that depth error rates and probe timing alone do not distinguish a fixed computational order from different rates of error reduction, which requires causal dependency across passes and decodability above prefix token counting. To test all three points directly, we replaced single-label buckets with a multi-label classifier, intervened on pass-1-to-pass-4 dependency, and added prefix-count probe controls. Together, these tests show pass-specific ring computation beyond surface error rates without uniform stage-to-stage dependency across seeds. We revised the abstract and Section 4 to replace the fixed three-stage order claim with this depth- and pass-dependent progression.
2. Scope of causal head localization across 53K and 206K (Weakness 2)
We resolved both points directly by running the full 32-cell per-head, per-pass ablation on the 53K model across three seeds on ZINC and ChEMBL and by restricting our head-ablation claim to bracket matching (having tested stage-to-stage dependency above). Single-head pass-1 bracket localization replicates in the 53K model (5/6 WS-53K runs, +10 pp bracket errors on ZINC and +18.4 pp on ChEMBL) at the same rate as the 206K model (5/6 WS-206K runs, +23 pp on ZINC and +19.3 pp on ChEMBL), with one non-localizing seed per size (ZINC WS-53K seed 42, ChEMBL WS-206K seed 43). We revised the abstract and Section 4.3 (Table 4, Appendix H Table 14) to report the 53K and 206K models side by side, state this 10/12 localization scope explicitly, and restrict our causal ablation claim to bracket matching.
3. “Small-molecule grammar” and contribution beyond compactness (Weakness 1, Question 1)
“Small-molecule” refers to the standard drug-like compound class (MW ≤ 500 Da, as in ZINC, MOSES, and ChEMBL) rather than formal grammar size, and we hyphenated the title to make this explicit. Beyond compact generation across ZINC, MOSES, and ChEMBL, reusing the same weights at every pass separates computation across depth from layer-specific weights, showing rising ring-state and pass-8 ring-closure decodability above prefix-count controls and a reproducible early bracket head. We revised the title to “SMolLM: Small Language Models Learn Small-Molecule Grammar” and Section 1 to define this term directly.
We hope the multi-label classifier, dependency test, probe controls, and 53K/206K localization resolve your concerns, and we welcome your feedback.