- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GRASP, a 93.5M-parameter graph Transformer for molecular property prediction. It combines large-scale replaced-token detection (RTD) pretraining on ZINC20 with subsequent multi-assay bioactivity adaptation on ChEMBL. GRASP achieves competitive performance on 23 OpenADMET endpoints and 22 TDC ADMET tasks. The paper also evaluates RTD against MLM and examines the contribution of ChEMBL adaptation.
Critical strengths
1. Careful and Transparent Evaluation
The paper evaluates GRASP across diverse ADMET endpoints using strong baselines and consistent data splits. The authors also provide statistical analyses, structural-overlap checks, and a candid discussion of limitations.
2. Consistent Benefit of ChEMBL Adaptation
The ChEMBL adaptation stage reduces mean MAE from 0.407 to 0.374 and improves performance on 21 of 23 OpenADMET endpoints and 19 of 22 TDC tasks, demonstrating the practical value of the staged training approach.
Critical weaknesses
1. Limited Methodological Novelty
The main components are drawn from prior work, including graph-relative attention, ELECTRA-style replaced-token detection, and multi-task bioactivity supervision. Although their integration is useful, the methodological novelty remains limited.
2. Modest Gains over the Strongest Baseline
GRASP improves mean MAE over CheMeleon by only 2.4%, with better results on 12 of 23 OpenADMET endpoints. CheMeleon is also absent from the paired statistical comparisons in Table A3. Given GRASP's substantially larger model size, the practical significance of this improvement remains unclear.
3. Confounded Training Comparisons
The ChEMBL adaptation stage introduces assay labels, additional training, and exposure to new molecules simultaneously, making it difficult to isolate the benefit of assay supervision. The RTD–MLM comparison also changes both the training objective and prediction targets, and its improvement on OpenADMET is not statistically significant after correction (p = 0.097).
Questions for authors
Please address the concerns raised in the Critical Weaknesses section, particularly regarding the improvement over CheMeleon and the independent contributions of ChEMBL adaptation and RTD pretraining.
Additionally, since radius-0 atom tokens may contain information that can be inferred from graph topology, could the authors clarify how much of the RTD task can be solved using structural cues alone, and whether topology-constrained token replacement would affect the results?
Thank you for recognizing the evaluation's transparency and the value of ChEMBL adaptation. We have completed the requested continuation control, added the paired CheMeleon comparison and completed the closer radius-0 objective comparison.
1. Methodological contribution
Revised GRASP changes the molecular representation and structural task jointly: seven explicit atom attributes, separate generator heads and whole-bundle replacement, together with explicit bond/stereochemical encoder inputs and removal of absolute atom-index embeddings. The equations and cumulative ablations are given in response to reviewers KZ5q and pwYW . We credit the established attention, RTD and two-stage training foundations; our contribution is their specific molecular formulation and controlled evaluation.
The reference/A1/A2/revised-GRASP means are 0.3702/0.3686/0.3662/0.3589. Separate reference-architecture GDES/ES/NES means are 0.3702/0.3638/0.3655.
2. Accuracy relative to CheMeleon and practical significance
The requested paired comparison is complete and the retained predictions have been audited. Revised GRASP reaches 0.3589 versus 0.3834 mean MAE, a 6.39% reduction(a significant improvement in this context over the SOTA ), and achieving lower error on 87% of endpoints (20/23) across three distinct organizations (ASAP, Biogen, and ExpansionRx) proves the gain is systemic rather than driven by one or two outlier datasets.
The paired difference is−0.0245 (pointwise 95% CI [−0.0379,−0.0119]); exact Wilcoxon raw p=0.000552 and expanded Holm-25 p=0.00883. The sign-test sensitivity also remains significant (p=0.01025).
This strengthens the accuracy evidence beyond the submitted comparison. Model size remains relevant: the encoder has approximately 93M parameters, and we distinguish accuracy from parameter or compute efficiency. We also completed revised-model FT, LoRA and frozen-mixing evaluations, with mean MAEs 0.3589, 0.3746 and 0.3821, respectively. LoRA uses 986,374 trainable parameters; the FT–LoRA difference is not statistically resolved after correction.
For a comparison between frozen systems, revised GRASP achieves 0.3821 versus 0.4123 for Mol-JEPA, with lower error on 20/23 endpoints (Holm-4 Wilcoxon p=0.000511). GRASP uses learned layer mixing/attention pooling while Mol-JEPA uses CLS features, so this result compares the evaluated frozen systems rather than identical readouts.
3. Contributions of ChEMBL adaptation and RTD
The continuation control matches the structural starting checkpoint,511,897 ChEMBL structures, 50k updates, effective batch 512 and learning-rate schedule. Label-free continuation loads no assay labels. Mean MAEs are 0.3915 before adaptation, 0.3871 after continuation and 0.3589 after supervised adaptation. Supervised adaptation improves over continuation by 7.30% on 19/23 endpoints, with Δ−0.0283, 95% CI [−0.0532,−0.0122] and Holm-3 p=0.0000987. This directly answers whether additional label-free exposure accounts for the supervised-stage improvement at the matched update budget.
The completed original-architecture radius-0 RTD/MLM control gives FT means 0.3658/0.3663, with no detected paired difference (Holm-2 p=0.3146). Frozen MLM is better (0.3797 versus 0.3989;18/23 endpoints; Holm-2 p=0.0271, expanded Holm-25 p=0.1353). We therefore separate the evidence for the complete revised design from claims about objective superiority. Nominal exposure and adaptation budgets are matched; FLOPs and distributed numerical trajectories are not.
4. Structural cues and topology-constrained replacement
We completed a diagnostic on 10,000 validation-partition molecules and 217,489 atoms, using the trained revised structural checkpoint. A deterministic detector checks whether the realized atom description's degree or ring membership contradicts bond adjacency, without seeing the original attributes or corruption mask. These rules identify 14.75% of realized replacements, with a molecule-bootstrap 95% CI of 12.99–16.63%.
After excluding atoms with those contradictions, the discriminator retains AUROC 0.9364. Under replacements constrained to degree/ring consistency, its AUROC is 0.9444, versus 0.9331 for a positive-rate-matched unconstrained diagnostic. These observations show that discrimination persists beyond the two tested inconsistencies; they quantify shortcut availability rather than establish the model's causal reliance or exclude other structural cues.
We respectfully ask that the stronger baseline evidence, completed attribution controls and direct response to the structural-cue question be considered for upgrading the rating of revised manuscript.