- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RETROPLANNER, which uses a 20B LLM to select a subset of opening molecules during retrosynthesis planning, then querying several judges to rank those selected molecules and expand multiple molecules at the same time. It reaches success rate comparable to other LLM baselines while achieving better route quality (judged by other machine learning models on reaction plausibility, round-trip consistency, and starting-material price). This work is evaluated on USPTO-190 for ID test and Chembl-1000 for OOD test, and the training corpus is collected on PaRoutes after deduplication.
Critical strengths
- This work proposes "multi-route expansion" which expands multiple subtrees at each step, which accelerates retrosynthesis planning on modern hardware
- This work achieves better route quality in terms of round-trip validity compared to other LLM-based baselines, which maintaining comparable success rates
Critical weaknesses
1.The added value of the LLM is insufficiently established. The LLM selects frontier molecules and ranks reaction candidates using signals supplied by external models, including reaction plausibility, round-trip consistency, and predicted precursor prices. However, the paper does not establish whether a 20B LLM provides sufficient benefit over a non-LLM planner using the same signals, multi-frontier expansion mechanism, and restart policy. The Score-only baseline partially addresses this question, but does not establish whether the gains persist against a stronger search procedure combining multi-objective scoring and diversity-aware selection or simply a search algorithm on filtered SSR results after some quality thresholds. Figure 4(b) compares expansion strategies, rather than demonstrating a favorable runtime–quality trade-off against such alternatives. The evidence for the reasoning components is also limited: Table 2 reports composite BTR but not its constituent quality metrics, and adding DISCONNECTIONS to L1 decreases BTR from 0.769 to 0.742. Since bond-change evidence and reaction-class evidence are introduced together, their individual contributions remain unclear. These results do not convincingly establish the benefit of the full evidence-grounded reasoning schema.
- The evaluation does not establish a consistent expansion-budget comparison or account for computational cost.
The supplementary implementation appears to apply different accounting rules across methods. In evaluation/board/eval_board_agent_forced_iterative.py,
seenis initialized outside the episode loop (line 137), and only previously unseen molecules increase the cumulative budget (lines 164–166). Re-expanding a molecule in a later RetroPlanner rollout is therefore free under this accounting. In contrast, from_pooled in evaluation/eval_protocol/metrics/table_geo.py accumulates the scalar expenditure of independent baseline runs (off += cost[r], line 167), without deduplicating molecules across those runs. Thus, overlapping expansions can be charged repeatedly for the baselines but only once for RetroPlanner, potentially favoring RetroPlanner under the claimed common unique-molecule budget. The authors should clarify whether this code produced the reported results and recompute the comparison using a consistent accounting rule. Separately, expansion counts do not capture the additional cost of serving the 20B planner; end-to-end runtime and GPU usage should be reported to establish the practical efficiency trade-off.
Questions for authors
- Compared with expanding all currently eligible frontier molecules and scoring their reaction candidates, how much computation does LLM-guided OPEN selection save? Please report end-to-end wall-clock time and measured or estimated FLOPs, including LLM inference, single-step prediction, and candidate scoring, together with the resulting route quality.
- Is 20B model really necessary and did you try smaller models in the same training paradigm?
- How does search algorithms perform in route quality metrics given ? Please include those route quality metrics in the appendix.
- How's really calculated, does it count duplicate queries and is it consistent across different methods?
- How's the distribution of molecules in the PaRoutes training set compared to USPTO-190 and ChemBL-1000, could you please provide a visualization of such distribution? Please include both molecular fingerprint comparison, scaffold comparison and so on to address potential data leaking concerns.
- How competitive are non-LLM search methods when given the same chemical evidence? Beyond the reported equal-weight scalarization, have you evaluated search baselines that filter candidate reactions using the same plausibility and round-trip criteria, then rank the remaining candidates using price and diversity? Please match the candidate generator, stock, and expansion-budget accounting, and report both success rate and route quality.
11 comments
W1-3. Contribution of Reasoning Schema Components
We provide detailed component ablations in Table R5 of our response to Reviewer 4Uvy (W1 & Q1).
The (MOLECULE) block grounds the planner in the structural relationship between the current molecule and the target, likely contributing to its higher BTR. Although achieves lower BTR than (76.32% vs. 80.95%), it recovers more routes (16.76 vs. 13.35) and achieves higher HV (0.2563 vs. 0.2445).
We also separately evaluate the two subfields of DISCONNECTIONS. bond changes achieves higher BTR (75.99% vs. 71.84%) and BTR Dom. (43.50% vs. 39.62%) than reaction class, while the latter achieves slightly higher HV (0.2461 vs. 0.2416). Combining both yields the highest HV (0.2478) among the three variants.
These results highlight the distinct contributions of reasoning components to route quality and multi-objective coverage.
Q1. LLM inference cost
We report per-target end-to-end accounting across all three stages. LLM inference, single-step prediction, and round-trip scoring all run on GPU and are reported in Table R12. Candidate scoring comprises three objective proxies. Round-trip prediction runs on GPU and is reported in Table R12, while plausibility and molecular price are small models over fingerprints whose cost is negligible, at PFLOP per target for the planner. FLOPs use with 3.6B active parameters for the planner, which is an MoE model.
- Table R12. Per-target computation and route quality (USPTO-190, )
| Method | Succ. ↑ | Expansions | LLM (PFLOP) | Single-step (PFLOP) | Round-trip (PFLOP) | Total (PFLOP) ↓ | Wall-clock (s/target) ↓ | HV ↑ |
|---|---|---|---|---|---|---|---|---|
| Expand-all | 62.63% | 181.4 | 0 | 0.693 | 1.246 | 1.940 | 14,639 | 0.1520 |
| MCTS | 74.21% | 239.1 | 0 | 0.210 | 0 | 0.210 | 50 | 0.1562 |
| Retro*-0 | 77.89% | 300.0 | 0 | 0.264 | 0 | 0.264 | 96 | 0.1892 |
| Retro* | 87.89% | 300.0 | 0 | 0.264 | 0 | 0.264 | 109 | 0.2107 |
| MO-MCTS | 79.47% | 300.0 | 0 | 0.264 | 1.068 | 1.332 | 464 | 0.1171 |
| MORetro* | 91.58% | 250.2 | 0 | 0.220 | 0.891 | 1.110 | 2,739 | 0.2905 |
| RetroPlanner | 98.42% | 140.1 | 3.24 | 0.117 | 0.147 | 3.504 | 3,385 | 0.2563 |
- Table R13. Measured GPU use and energy (USPTO-190, )
| Method | GPU-hours/target | Active Wh/target | Including idle |
|---|---|---|---|
| Expand-all | 0.103 | 22.7 | 88.8 |
| RetroPlanner | 0.047 | 17.4 | 27.5 |
All runs use the same node of eight NVIDIA H100 GPUs, with four cards serving the single-step and forward models and four serving the LLM. GPU-hours represent utilization-weighted GPU time rather than elapsed wall-clock time, measured using per-card utilization counters.
Table R12 shows that Expand-all performs no pruning, opening every eligible frontier molecule and processing candidate reactions in first-seen order until the expansion budget is exhausted. Compared with Expand-all, RetroPlanner reduces molecule expansions by 22.8% (140.1 vs. 181.4) while achieving substantially higher success (98.42% vs. 62.63%) and HV (0.2563 vs. 0.1520). Although LLM inference dominates RetroPlanner's computational cost (3.24 of 3.504 PFLOPs), Table R13 shows 2.2× lower utilization-weighted GPU time and 23% lower active energy consumption compared with Expand-all.
Compared with MORetro*, RetroPlanner achieves higher success (98.42% vs. 91.58%) with 44% fewer molecule expansions, although MORetro* achieves higher HV at lower computational cost. Overall, LLM-guided OPEN selection achieves the highest success rate with the fewest molecule expansions, while maintaining competitive multi-objective route quality and reducing measured GPU resource consumption relative to Expand-all.
W2 & Q4. Budget counting
We thank the reviewer for identifying the inconsistent budget accounting. The evaluation pipeline for the LLM planners is shared across all of them, so no method is charged differently under it. The inconsistency was between that convention and table_geo.py (line 167), which accumulated the expenditure of independent baseline runs without deduplicating molecules across them. We re-evaluated all methods using unique molecule expansions as the common budget definition, and Tables R14 and R15 report the corrected results.
- Table R14. Main results under matched budget counting (USPTO-190, )
| Method | Counting | Succ. ↑ | Routes ↑ | BTR ↑ | BTR Dom. ↑ | HV ↑ |
|---|---|---|---|---|---|---|
| Retro-R1 | original | 82.11% | 2.99 | 38.42% | 10.53% | 0.1085 |
| matched | 91.05% | 4.73 | 45.26% | 13.16% | 0.1348 | |
| RetroAgent | original | 96.84% | 3.07 | 46.84% | 9.47% | 0.1074 |
| matched | 98.42% | 3.52 | 44.21% | 13.16% | 0.1243 | |
| RetroPlanner | matched | 92.11% | 7.62 | 66.84% | 33.68% | 0.1973 |
- Table R15. Search baselines under matched counting, with original values in parentheses (USPTO-190, )
| Method | Success ↑ | Routes ↑ | BTR ↑ | BTR Dom. ↑ | HV ↑ |
|---|---|---|---|---|---|
| MCTS | 51.05% (44.74) | 19.79 (16.88) | 29.47% (27.89) | 10.53% (10.00) | 0.0897 (0.0817) |
| Retro*-0 | 59.47% (52.63) | 69.62 (29.55) | 37.89% (34.21) | 24.74% (23.16) | 0.1475 (0.1271) |
| Retro* | 63.68% (57.89) | 68.15 (33.34) | 44.74% (38.95) | 28.95% (24.74) | 0.1598 (0.1410) |
Under matched counting, every method gains. All tables in the revised paper will follow this counting convention.
Q2. Does the approach require a large backbone?
We applied the same training recipe to two smaller backbones and report the results in Table R16.
- Table R16. The same training recipe on smaller backbones (USPTO-190, )
| Backbone | Active params | Succ. ↑ | Routes ↑ | BTR ↑ | BTR Dom. ↑ | HV ↑ | LLM PFLOP/target ↓ | Wall-clock (s/target) ↓ |
|---|---|---|---|---|---|---|---|---|
| Qwen3-4B | 4.02B | 85.26% | 7.69 | 60.00% | 32.63% | 0.1938 | 4.22 | 2,195 |
| Qwen3.5-9B | 9.0B | 94.21% | 7.80 | 69.47% | 37.89% | 0.2088 | 12.14 | 6,268 |
| RetroPlanner (GPT-OSS-20B) | 3.61B | 92.11% | 7.62 | 66.84% | 33.68% | 0.1973 | 3.24 | 3,385 |
Table R16 shows that the training recipe transfers across different backbones. Qwen3-4B achieves 85.26% success with lower route quality but faster inference, while Qwen3.5-9B achieves the highest success rate and route quality, at 3.75× the FLOPs and 1.85× the runtime of GPT-OSS-20B. GPT-OSS-20B activates only 3.61B parameters per token, achieving competitive performance with the lowest FLOPs and a favorable performance–compute trade-off, which is why we keep it as the main configuration.
We sincerely thank the reviewer for the careful review and detailed comments. We will address each concern and revise the paper accordingly. Please find our responses to the reviewer’s key concerns below.
As stated in §3.1, the search baselines in Table 1 use the same objectives only for post-search route selection, not during search. Specifically, we retain the Pareto non-dominated routes from each target's route pool and select the highest-scoring route using an equal-weighted composite of plausibility, round-trip accuracy, and price utility.
To test whether these objectives also help during search, we re-ran the non-LLM baselines with the composite in their search signals, using λ∈{0,1,4}, where λ=0 is the original setting. The composite f is the mean of the three objectives expressed as percentile ranks among the candidates offered for the same molecule, so it reorders candidates within a molecule rather than removing them. For Retro* it is added to the edge cost as −λlogf, while for MCTS it scales the terminal reward as an exponent.
We additionally evaluate MO-MCTS, which selects from the Pareto front of per-objective confidence bounds, and following Reviewer RJWv’s suggestion report BTR Dom. and hypervolume(HV) alongside BTR.
Table R10 shows that incorporating these signals into search does not consistently improve route quality and reduces success (e.g., Retro*: 63.68% → 60.00%). For Retro*, the added cost also penalizes longer routes, making deeper targets harder to solve.
Among these configurations, RetroPlanner reaches the highest success (92.11%) and BTR Dom. (33.68%) from the smallest route pool (7.62). We will include these results and the metric formulation in §A.4.
The reviewer asks for a search procedure combining multi-objective scoring with diversity-aware selection.
RetroPlanner considers route diversity through its
DECIDINGschema, wherediverging fromidentifies alternative disconnections relative to existing routes andordered byrecords the evidence behind the choice, so selection is conditioned on route context rather than on numerical scores alone.To test whether an explicit diversity incentive benefits non-LLM search in the same way, we added the reviewer’s term directly to the Retro* edge cost,
cost=−logpmodel−λlogf−νlog(max(1−maxσ∈SJ(prec,σ),0.05))
where J is the Jaccard overlap of starting-material sets and S contains the leaf sets of previously recovered routes. We use r=3 search restarts under the same total expansion budget, following the restart policy the reviewer asks us to match, which is also what lets the term act at all.
Adding a diversity bonus to Retro* search yields limited improvements in diversity. Increasing ν from 0 to 4 raises first-cut diversity from 0.518 to 0.537, while dissimilarity slightly moves (0.732 to 0.741), with comparable success and HV. Treating diversity as a fourth objective improves both diversity metrics (0.769 and 0.634), but reduces success to 60.70% and HV to 0.1457.
In contrast, RetroPlanner achieves the highest starting-material dissimilarity (0.801), first-cut diversity (0.670), success (92.11%), and HV (0.1973). Selection conditioned on route context therefore appears to balance diversity and quality better than a numerical bonus.
We note that the ν=0 row uses a single pass and the ν=4 row three restarts.