- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper asks whether an LLM can be post-trained for molecular lead optimization against an expensive scoring function that is too slow to train on directly. The authors build two tiers of cheap synthetic design tasks: Tier-1, 200,000 randomly generated tasks over 116 RDKit/OpenEye descriptors with a BRICS scaffold constraint, and Tier-2, 999 structure-based tasks on PLINDER pairs scored by POSIT docking plus Chemgauss4. They then post-train Qwen3.6-35B-A3B with asynchronous RLVR (prime-rl, IPO loss) under the 12 recipes of Table 1,each for a total of 500 steps split across two stages, and evaluate on 40 held-out PLINDER systems scored by SQM2.20, which costs tens of minutes per call. The reported findings are: Tier-1-only training already moves the base model a long way on the held-out SQM task; adding a Tier-2 stage helps further; training on Tier-2 alone from the base model collapses to a single-turn policy; and the best curricula exceed five frontier models evaluated zero-shot in the same harness. The appendix adds failure-mode, rollout and chemical analyses.
Critical strengths
-
The paper provides useful evidence of transfer from cheap synthetic tasks to an expensive structure-based design task. This is practically relevant because direct RL training with the evaluation oracle is too expensive. The use of the official PLINDER training and test splits, recipes matched at 500 steps on identical hardware, and a common five-call SQM budget for trained and zero-shot models helps make the comparisons interpretable. In particular, a model trained only on descriptor-based tasks, without protein structures, improves on the base model by 2.89 kcal/mol in the SQM-based relative binding-affinity metric and raises the constraint reward from 36.2% to 94.0% (Table 2). This supports using cheap synthetic tasks to prepare a model for tasks with expensive feedback, within the scoring setup studied here.
-
The appendix provides useful diagnostics for evaluating molecular-design agents. Table A3 separates formatting failures from scoring failures, which helps readers assess how much reliability contributes to the comparison. Fig. A6 examines whether the model submits the best molecule it scored, making final candidate selection an explicit part of the evaluation. This matters because a model can find a good molecule but submit a worse one. Fig. A9 separates results for anionic and neutral seed ligands, which helps identify where the gains are concentrated. These analyses help explain the aggregate scores and identify limitations that an average score alone would hide.
Critical weaknesses
W1. The aggregate advantage over frontier models is concentrated in anionic seeds, and the chemical meaning of these gains needs independent validation. This limits how broadly the result can support improved molecular design.
Fig. A9 separates 14 anionic and 26 neutral seed ligands. On neutral seeds, my reading of the figure is that GPT-5.6 Sol and the best curriculum model, T1(400)-T2(100), both achieve about +3.1 kcal/mol relative to T1(500). For the recipe with the largest aggregate SQM improvement, T1(300)-T2(200), the gap over GPT-5.6 Sol is +3.88 - 1.44 = +2.44 kcal/mol. Digitising the subgroup means gives +2.59 from the anionic systems and -0.15 from the neutral ones. Thus, this aggregate advantage comes entirely from 35% of the test set. The overall average remains a valid benchmark result, but the subgroup dependence should be clear when discussing its scope.
The authors report that "removing an anionic group improved the relative binding affinity by 26.1 kcal/mol" (p. 32) across anionic modifications. This is an association across edits, so it does not establish that anion removal causes the model advantage. However, a large score change associated with charge modification makes an independent check important. SQM2.20 supplies both the optimization feedback during evaluation and the final affinity assessment, with no second scoring function or free-energy spot-check reported for the generated designs. It therefore remains unclear how much of the gain reflects improved binding versus behavior specific to SQM. Subgroup results in the main text and an independent check on a small set of charged designs would directly address this concern.
W2. The size and stability of the reported advantage are difficult to assess because the metric combines task completion with score improvement, while setting selection and training variability are not fully evaluated.
Fig. 2 states that "invalid rollouts are assigned an improvement of 0". The failure rates differ substantially: 0-2 of 200 rollouts fail for the trained models, compared with 19 for GPT-5.6 Sol. The cause also matters: GPT-5.6 Sol has 2 formatting failures and 17 scoring failures. Reliability is a relevant capability, but results over valid rollouts are needed to assess improvement among successfully scored designs. Table 2's headline metric is normalized by subtracting the average of the T1(500) model, and no per-model mean improvement over the seed is reported for the 40 systems. Reporting that mean alongside the valid-rollout results would clarify the effect of assigning zero to failed rollouts.
The reported +/-1 SE error bars describe evaluation uncertainty, while no independent repetitions of the training runs are reported. They therefore do not establish whether differences among the best schedules are stable across training runs. This matters for interpreting which curriculum choices are useful, rather than only whether staged training helps in general.
The procedure for choosing the settings is also unclear. Sec. 4 describes the preferred reward weights as "the best performing combination", without stating what data were used to choose them, and no separate validation split is described. D.3 reports that none of the models trained with alternative reward weights exceeds 2.5 kcal/mol relative to T1(500). Some of those models still exceed frontier baselines, so this sensitivity concerns the size of the reported margin. The schedules in Table 1 use predefined switching points and task mixtures, and the authors report no clear cost-performance scaling law. A clear selection criterion would help readers assess whether the chosen setting is independent of the final evaluation and how to choose a schedule for a different model or training budget.
Questions for authors
Clarifications that could change my assessment:
Q1. Please report the model means and uncertainties behind Fig. A9, particularly on the 26 neutral systems. Does any trained model improve reliably over the strongest frontier comparator on this subset? This would clarify the scope of the aggregate advantage.
Q2. What explains the 26.1 kcal/mol mean score gain associated with removing an anionic group? Is it sensitive to the proton-transfer or solvation terms of Eq. A11? An independent check on a small set of the charged designs,using another scoring method or a free-energy calculation, would help establish whether these gains correspond to improved binding.
Q3. Please report results over valid rollouts alongside the zero-filled metric, together with each model's mean SQM score improvement over the seed. Does the frontier comparison change with a more lenient answer parser or a worked format example? The Tier-1 prompt includes a worked example on p. 39, while the evaluation prompt does not. This would help separate molecule quality from answer-format reliability.
Q4. What data were used to select the curriculum schedule and reward weights? In particular, was w_obj = 0.75 / w_constraint = 0.25 selected using the 40 SQM evaluation systems? If so, can the selected setting be checked on data independent of that choice? What criterion would guide the switching point and task mixture for a different training budget?
Q5. Were any recipes trained repeatedly with independent random seeds? If so, are the differences among the best recipes stable? Limited repetition of a few representative recipes would help assess the training variability that the evaluation error bars do not measure.
Q6. How was the tool-call penalty warmup handled in T2(500)? The reported collapse is a single-turn policy that does not iterate with tool calls, while B.3 reports conservative tool use when the warmup is omitted. The standard warmup spans 1000 group scoring steps. Was the collapse robust to a modest change in this schedule? This would help distinguish a failure caused by the training settings from difficulty learning Tier-2 tasks directly.
Suggestions for presentation, separate from the clarification requests above:
The title's emphasis on synthetic task scaling would be clearer if the paper explicitly distinguished corpus size from training duration. At 500 steps with 64 unique tasks per batch, a run uses at most 32,000 of the 200,000 Tier-1 tasks. Clarifying what is varied would help readers understand which scaling claim the experiments support.
The prompt text in Figure 1 is too small to read clearly at normal print size. Please enlarge the text or use a shorter excerpt so the task setup is easier to understand.
4 comments
Author rebuttal (part 2)
Q3. Results under different criteria
Please report results over valid rollouts alongside the zero-filled metric, together with each model's mean SQM score improvement over the seed. Does the frontier comparison change with a more lenient answer parser or a worked format example? The Tier-1 prompt includes a worked example on p. 39, while the evaluation prompt does not. This would help separate molecule quality from answer-format reliability.
Response: While models do exhibit some failure to adhere to answer formatting, we note that our trained models and stronger frontier models like GPT-5.6 Sol are sufficiently competent such that these failures are rare and a non-issue. Furthermore, we note that failure to adhere to the correct answer formatting is a real failure mode that is corrected through the curriculum training procedure and should be factored into the reporting of the results.
Q4. Curriculum selection
What data were used to select the curriculum schedule and reward weights? In particular, was = 0.75 / = 0.25 selected using the 40 SQM evaluation systems? If so, can the selected setting be checked on data independent of that choice? What criterion would guide the switching point and task mixture for a different training budget?
Response: The weighting factors and used for the results presented in the main text were chosen based on the evaluation set performance, with the results for the other weighting combinations reported in the appendix. For a different training budget, the specific weighting factors would need to be determined empirically, but the task mixture should stay skewed towards the harder task during the curriculum training stage. A good assessment of whether a mixture can continue to deliver training signal without inducing catastrophic forgetting in the model is to examine the training dynamics of the rewards, as is shown in Figure A1 and Appendix D.1. Monitoring this quantity during tuning can help decide a more optimal mixture or curriculum structure. To clarify the selection process for the weighting factor, we have added the following text to the manuscript:
Page 5: We explored various and and found for Tier-1 tasks and for Tier-2 tasks to be the best performing combination based on the evaluation set performance, and so we focus our analysis on this condition and its results ...
Q5. Seed repetition and statistics
Were any recipes trained repeatedly with independent random seeds? If so, are the differences among the best recipes stable? Limited repetition of a few representative recipes would help assess the training variability that the evaluation error bars do not measure.
Response: All recipes were trained using a single random seed due to constraints on the compute available for this work. While a single run cannot characterize the training variability with regards to the evaluation performance metric, the trends we uncover in terms of model performance are robust and the more important result. To clarify this point, we have added the following text:
Page 5: We train each model using a single seed due to compute constraints, and while we do not obtain a clear "scaling law" where increased cost leads to increased performance, we do see that the more expensive curriculum training approaches outperform the cheaper Tier-1 only training, demonstrating robust trends which are the focus of this work.
Author rebuttal (part 3)
Q6. Tool call penalty warmup
How was the tool-call penalty warmup handled in T2(500)? The reported collapse is a single-turn policy that does not iterate with tool calls, while B.3 reports conservative tool use when the warmup is omitted. The standard warmup spans 1000 group scoring steps. Was the collapse robust to a modest change in this schedule? This would help distinguish a failure caused by the training settings from difficulty learning Tier-2 tasks directly.
Response: The tool call penalty warmup was handled in the same way for T2(500) as for all other curriculum experiments, as we found the choice of 1000 scoring steps to be robust for ensuring models learned to economically use the tools provided without becoming exploitative across all of the settings we investigated. We did find from early experiments that T2(500) without a tool penalty similarly does not work, reflecting that the model collapse is a consequence of the task difficulty as opposed to a miscalibrated training setting. To clarify this point, we have added the following text to the manuscript:
Appendix D.2, Page 29: Additionally, we find that while removing the tool call penalty can help the model exploit tools more, this alone did not correct the failure mode noted in Figure A2, indicating that the failure is due to the task difficulty and not a miscalibrated tool penalty warmup during training.
Clarification about task consumption
The title's emphasis on synthetic task scaling would be clearer if the paper explicitly distinguished corpus size from training duration. At 500 steps with 64 unique tasks per batch, a run uses at most 32,000 of the 200,000 Tier-1 tasks. Clarifying what is varied would help readers understand which scaling claim the experiments support.
Response: To clarify this point, we have added the following text to the manuscript:
Page 4: A single 500-step run consumes 32000 of these, at 64 unique tasks per step, so the pool is sized to supply non-repeating tasks rather than to be exhausted.
Fixing prompt text size in Figure 1
The prompt text in Figure 1 is too small to read clearly at normal print size. Please enlarge the text or use a shorter excerpt so the task setup is easier to understand.
Response: We will enlarge the text in the revised version of the paper.
Thank you for the replies and the subgroup table. I continue to find the transfer from cheap synthetic tasks to the SQM evaluation useful. The reported check that removing the tool-call penalty did not correct the Tier-2-only collapse addresses my specific concern about the penalty. The clarification of the task-pool size is also helpful.
My main concerns remain:
-
On the neutral systems, the reported means are +3.06 kcal/mol for T1(400)-T2(100) and +3.08 kcal/mol for GPT-5.6 Sol, relative to T1(500). Their mean performance is similar. The overall advantage of the best recipes over GPT-5.6 Sol is concentrated in the anionic systems. The explanation of the opposing interaction and solvation contributions clarifies how a large score gain can arise. However, it does not establish whether the proposed designs bind more strongly. The added discussion acknowledges that errors in these contributions can exaggerate the predicted gain, and the reply provides no independent check of the designs. This remains my main concern about the interpretation of the gains.
-
Thank you for confirming that the reward weights were chosen using evaluation-set performance. The reported comparison is therefore not independent of that choice. Reporting the other weight settings is helpful, but results on data not used for selection would be needed to assess the selected margin independently. I also understand the compute constraints behind using one training seed per recipe. This leaves variation across training runs, including the stability of the recipe ordering, unmeasured.
-
I agree that reliability should be included in the main results and that formatting failures are rare for GPT-5.6 Sol. However, the original Table A3 lists 2 formatting failures and 17 scoring failures out of 200 rollouts for this model. My question also concerned these scoring failures. The requested results over valid rollouts and mean SQM-score improvement over the seed are not provided in the reply. Reporting these alongside the main metric would help clarify how much of the comparison reflects successful task completion and how much reflects score improvement.
The rebuttal clarifies several points, but my main reservations about the chemical interpretation of the gains and the independence of the evaluation remain. I therefore maintain my rating of 2.
Author rebuttal
We thank the reviewer for their feedback and suggestions and address the questions below, which also cover the weaknesses raised.
Q1. Reporting model means and uncertainties
Response: The means and uncertainties for both the trained and zero-shot models are already included in Figure A9, but we provide the table for the means and uncertainties behind the Figure here for convenience:
These results further reinforce the trained models' large improvement in the cases with anionic seed structures when compared to the zero-shot models, outperforming the zero-shot baselines and leading to better average improvement across all 40 systems. For neutral seed structures specifically, we see that the trained models achieve competitive performance to the zero-shot frontier models, substantially improving upon the performance of the Qwen 3.6 (base) model and keeping pace with far stronger models such as GPT-5.6 Sol. While the degree of increase in the relative binding affinity improvement for the trained models on the neutral seed systems is lower than that for the anionic seeds, the fact that a weaker Qwen 3.6 (base) model can still be trained to match the performance of frontier models is a testament to the effectiveness of the curriculum training strategy investigated in this work.
Q2. 26.1 kcal/mol mean score gain
Response: The 26.1 kcal/mol mean score gain comes from how charged groups affect the gas-phase interaction energy ΔEint and the change in the solvation free energy ΔΔGsolv, where for a charged ligand both of these terms in Equation A11 are large and opposite in sign, each on the order of tens of kcal/mol for a monovalent ion with the physical binding contribution being a small residual between these terms. Removing the anion reduces both the favorable interaction energy and the opposing solvation penalty, and when the reduction in the solvation penalty outweighs the loss of favorable interactions, we see a large gain in the score. To clarify this point, we have added the following text to the manuscript: