- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper asks whether one input rule can represent scientific strings across domains. GCMS uses grammar-constrained policy-gradient search and a noise-normalized soft-min reward. The rule is learned on materials and protein tasks, then evaluated on held-out properties and biological notations. Results show improved material predictions but mixed biological transfer.
Critical strengths
- The noise-normalized soft-min reward addresses conflicts between domains. Ablations show its advantages over mean and hard-min objectives.
- The paper compares multiple tokenizers and search strategies and includes ablations on reward design, grammar choices, and cross-domain supervision.
- GCMS reduces band-gap MAE from 0.6161 to 0.4918 compared with WordPiece and performs well across several materials benchmarks.
Critical weaknesses
- On ClinVar, GCMS only improves accuracy from 0.8183 to 0.8221. Peptide performance also requires manually relaxing the learned length bound from the search setting.
- GCMS performs worse than WordPiece on BRCA1 (0.5874 vs. 0.6092). Simple character splitting also matches or exceeds GCMS on several materials tasks.
Questions for authors
1.GCMS does not clearly outperform identity on ClinVar (0.8221 vs. 0.8183), performs worse than WordPiece on BRCA1, and requires changing the learned length bound for its best peptide result. Do these results support a general transfer claim, or only transfer under specific conditions? 2. Figure 5 suggests that much of GCMS's search advantage comes from behavioral deduplication and diversity. Can the authors show that policy learning itself provides additional gains over uniform random search with the same deduplication and number of unique rule evaluations? 3. The grammar already includes operations for preserving numeric spans and separating letters. How much of the discovered structure comes from learning rather than these predefined choices? What evidence shows that the learned segments are scientifically meaningful?
We demonstrate that GCMS learns structural transformations that generalize across scientific domains and held-out tasks. Comparisons with uniform sampling show improved proposal quality, while grammar ablations and expanded search experiments demonstrate that the learned behaviour does not depend on domain-specific heuristics or narrowly defined grammar choices.
We thank the reviewers for the questions. The answers are given below.
1. GCMS learns a shared input transformation using a noise-normalized soft-min reward across two scientific domains, discouraging gains in one domain at the expense of another and encouraging structural choices that generalize across scientific notations. For
Cs6Na2Nb6H36O37, WordPiece fragments include##2n,##h3, and##o3, mixing element symbols and stoichiometric coefficients. GCMS separates symbolic units while preserving numeric spans such as 36 and 37. For protein mutationA123G, WordPiece gives['a1', '##23', '##g'], compared to GCMS['a', '123', 'g']which preservs the residue position.Further, the rule learned from crystal classification and ProteinGym PSEAI transfers to held-out tasks: band-gap MAE decreases from 0.6161 to 0.4918 and modified-peptide iRT Spearman correlation improves from 0.8907 to 0.9649 after relaxing the detector length bound for longer sequences, without changing the learned segmentation policy. Additionally, applied without re-search to molecular SMILES on BBBP, the rule improves accuracy from 0.8846 to 0.9024 over WordPiece. The authors would like to mention that we do not claim universal improvements. We acknowledge that the ClinVar gain is within evaluation variability, and GCMS underperforms WordPiece on BRCA1. Empirically, the results show transfer when the learned structural primitives apply to the target notation.
2. We compared GCMS with true uniform sampling, holding grammar, behavioural deduplication, caching, exclusions, and evaluation budgets identical across three search seeds. For crystal + PSEAI, mean proposal reward improves from -2.581 (uniform) to -0.863 (GCMS). Proposals within 1.5 reward units of the best observed reward rise from 7.6% to 31.2% and proposal-reward standard deviation decreases by a factor of 2.09 during GCMS optimization versus 1.21 for uniform, showing concentration toward higher-reward rules (table below).
In the table, Δ denotes the gain from GCMS and the standardized gains use pooled proposal-reward SD. Each of the four settings uses three matched-budget search seeds. Importantly, GCMS raises mean proposal reward in all 12 runs, with standardized gains in the range 0.158 to 0.562. Further, for case-aware crystal search, proposals with normalized reward z≥14 increase from 8.2% to 17.9%. From the first to second half of optimization, mean proposal reward improves by 3.71 for GCMS versus 1.08 for uniform sampling. The results support the finding that although the best of budget rewards remain comparable, policy learning improves sampled-rule quality beyond shared caching and deduplication.
3. The authors would like to mention that the grammar contains no domain-specific knowledge, chemical element lists, residue definitions, or scientific parsing heuristics. It constrains search through generic string operations and GCMS learns useful operations and combinations from downstream scientific task performance. Interestingly, even after removing four digit-handling operations (
group_digits,numeric_suffix,by_prev_class,preserve_numeric) while retaining generic separation and\d+matching, GCMS is able to recover the same functional segmentation behaviour, showing it does not depend on explicitly providing thepreserve_numericgrammar.Across 2,900 evaluations, 2,225 distinct rules, and 50 soft-min searches, GCMS achieves a maximum reward of 2.729 via multiple digit-handling strategies:
preserve_numeric,separate, andgroup_digits. Their probabilities of appearing among the top 10% of evaluated rules are 17.1%, 21.7%, and 21.0% respectively. Thus, useful numeric-span preservation can be recovered via different grammar configurations rather than one predefined operation.Further, we enlarged the grammar with five generic, case-blind distractor transitions (
letter_punct,punct_letter,alnum_punct,punct_alnum,punct_punct) and eight distractor regex pattern sets:[aeiou]+/[^aeiou]+;.{3,4};.{1,3};[a-m]+/[n-z]+;[0-4]+/[5-9]+;[^0-9]+;.{2,3};[a-z]{1,2}/[^a-z]+. These have no domain-specific knowledge and expand reachable behaviours from 842 to 1,096. Despite this larger search space, GCMS recovers the same digit-preserving rule family in all three search seeds and achieves higher mean evaluated-rule reward than uniform sampling (-0.869 vs. -2.182). The evidence supports recovery of useful structural transformations without a narrowly specified grammar by GCMS.