- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates whether procedural pretraining on synthetic tasks can improve molecular property prediction. It introduces a three-stage pipeline combining procedural pretraining, molecular pretraining, and downstream fine-tuning. The results show improvements on Lipophilicity, particularly with REVERSE, while additional analyses examine data scarcity, procedural structure, training budgets, and component-wise transfer.
Critical strengths
1. Interesting Research Direction
Applying procedural pretraining to molecular learning is an interesting extension of prior work, suggesting that useful inductive biases can be learned without molecular data.
2. Useful Experimental Analyses
The paper includes several informative analyses, including shuffled-token controls, data-scarcity experiments, procedural-budget sweeps, and attention-only versus feed-forward-only transfer.
Critical weaknesses
1. Limited Generalization of the Main Results
The strongest improvement in the full three-stage pipeline is concentrated on Lipophilicity with REVERSE pretraining (4.8%). Improvements on QM9 are modest, while FreeSolv shows no consistent benefit. The evidence is therefore insufficient to establish broad effectiveness across molecular prediction tasks.
2. Unclear Mechanism Behind Procedural Transfer
REVERSE achieves the strongest improvement despite having little apparent structural correspondence with SMILES. The paper does not sufficiently explain why this task transfers better than more structurally aligned alternatives. Furthermore, the component-wise analyses suggest where transferable information resides but do not fully establish the underlying mechanism.
3. Limited Validation in the Full Training Pipeline
Much of the analysis is conducted without molecular pretraining, whereas the main contribution concerns transfer through a three-stage pipeline. It remains unclear whether the observed training-budget and component-wise effects generalize to the full setting.
Questions for authors
Please address the concerns raised in the Critical Weaknesses section, particularly regarding generalization across molecular tasks, the transfer mechanism of REVERSE, and the applicability of the analyses to the full training pipeline.