acceptodds
Under review as a conference paper at ICLR 2027

RetroPlanner: Multi-Route Expansion and Multi-Objective Reasoning for LLM-Based Retrosynthesis Planning

33 views

Abstract

Multi-step retrosynthesis planning is often considered solved once a route to purchasable starting materials is found, but such a route should also be practical in real-world chemistry. Recent LLM-based planners formulate this process as sequential decision-making over an AND-OR graph, yet typically expand a single molecule per turn, which limits global comparison across alternative routes before the search budget is committed. We introduce RetroPlanner, an LLM-based planner that jointly compares multiple routes for efficient search toward those of higher synthesizability as measured by reaction plausibility, round-trip consistency, and starting-material price. RetroPlanner is trained to reason over chemical evidence: structural progress from the target molecule, bond changes and reaction classes, where candidate routes diverge from one another, and how they rank across multiple objectives. Experiments on multi-step retrosynthesis benchmarks demonstrate that RetroPlanner recovers more alternative routes than prior LLM-based planners while improving synthetic route quality and maintaining a high success rate.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Retrosynthesis Planning, LLM, Agent, Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Qw2mOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RETROPLANNER, which uses a 20B LLM to select a subset of opening molecules during retrosynthesis planning, then querying several judges to rank those selected molecules and expand multiple molecules at the same time. It reaches success rate comparable to other LLM baselines while achieving better route quality (judged by other machine learning models on reaction plausibility, round-trip consistency, and starting-material price). This work is evaluated on USPTO-190 for ID test and Chembl-1000 for OOD test, and the training corpus is collected on PaRoutes after deduplication.

Critical strengths
  1. This work proposes "multi-route expansion" which expands multiple subtrees at each step, which accelerates retrosynthesis planning on modern hardware
  2. This work achieves better route quality in terms of round-trip validity compared to other LLM-based baselines, which maintaining comparable success rates
Critical weaknesses

1.The added value of the LLM is insufficiently established. The LLM selects frontier molecules and ranks reaction candidates using signals supplied by external models, including reaction plausibility, round-trip consistency, and predicted precursor prices. However, the paper does not establish whether a 20B LLM provides sufficient benefit over a non-LLM planner using the same signals, multi-frontier expansion mechanism, and restart policy. The Score-only baseline partially addresses this question, but does not establish whether the gains persist against a stronger search procedure combining multi-objective scoring and diversity-aware selection or simply a search algorithm on filtered SSR results after some quality thresholds. Figure 4(b) compares expansion strategies, rather than demonstrating a favorable runtime–quality trade-off against such alternatives. The evidence for the reasoning components is also limited: Table 2 reports composite BTR but not its constituent quality metrics, and adding DISCONNECTIONS to L1 decreases BTR from 0.769 to 0.742. Since bond-change evidence and reaction-class evidence are introduced together, their individual contributions remain unclear. These results do not convincingly establish the benefit of the full evidence-grounded reasoning schema.

  1. The evaluation does not establish a consistent expansion-budget comparison or account for computational cost. The supplementary implementation appears to apply different accounting rules across methods. In evaluation/board/eval_board_agent_forced_iterative.py, seen is initialized outside the episode loop (line 137), and only previously unseen molecules increase the cumulative budget (lines 164–166). Re-expanding a molecule in a later RetroPlanner rollout is therefore free under this accounting. In contrast, from_pooled in evaluation/eval_protocol/metrics/table_geo.py accumulates the scalar expenditure of independent baseline runs (off += cost[r], line 167), without deduplicating molecules across those runs. Thus, overlapping expansions can be charged repeatedly for the baselines but only once for RetroPlanner, potentially favoring RetroPlanner under the claimed common unique-molecule budget. The authors should clarify whether this code produced the reported results and recompute the comparison using a consistent accounting rule. Separately, expansion counts do not capture the additional cost of serving the 20B planner; end-to-end runtime and GPU usage should be reported to establish the practical efficiency trade-off.
Questions for authors
  1. Compared with expanding all currently eligible frontier molecules and scoring their reaction candidates, how much computation does LLM-guided OPEN selection save? Please report end-to-end wall-clock time and measured or estimated FLOPs, including LLM inference, single-step prediction, and candidate scoring, together with the resulting route quality.
  2. Is 20B model really necessary and did you try smaller models in the same training paradigm?
  3. How does search algorithms perform in route quality metrics given NB=300N_B=300? Please include those route quality metrics in the appendix.
  4. How's NBN_B really calculated, does it count duplicate queries and is it consistent across different methods?
  5. How's the distribution of molecules in the PaRoutes training set compared to USPTO-190 and ChemBL-1000, could you please provide a visualization of such distribution? Please include both molecular fingerprint comparison, scaffold comparison and so on to address potential data leaking concerns.
  6. How competitive are non-LLM search methods when given the same chemical evidence? Beyond the reported equal-weight scalarization, have you evaluated search baselines that filter candidate reactions using the same plausibility and round-trip criteria, then rank the remaining candidates using price and diversity? Please match the candidate generator, stock, and expansion-budget accounting, and report both success rate and route quality.
11 comments

We sincerely thank the reviewer for the careful review and detailed comments. We will address each concern and revise the paper accordingly. Please find our responses to the reviewer’s key concerns below.

W1-1. Non-LLM Search Baselines Using the Same Objective Signals

As stated in §3.1, the search baselines in Table 1 use the same objectives only for post-search route selection, not during search. Specifically, we retain the Pareto non-dominated routes from each target's route pool and select the highest-scoring route using an equal-weighted composite of plausibility, round-trip accuracy, and price utility.

To test whether these objectives also help during search, we re-ran the non-LLM baselines with the composite in their search signals, using λ∈{0,1,4}\lambda \in \{0,1,4\}, where λ=0\lambda=0 is the original setting. The composite ff is the mean of the three objectives expressed as percentile ranks among the candidates offered for the same molecule, so it reorders candidates within a molecule rather than removing them. For Retro* it is added to the edge cost as −λlog⁡f-\lambda \log f, while for MCTS it scales the terminal reward as an exponent.

We additionally evaluate MO-MCTS, which selects from the Pareto front of per-objective confidence bounds, and following Reviewer RJWv’s suggestion report BTR Dom. and hypervolume(HV) alongside BTR.

  • Table R10. Objective equal-weighted signals in the search cost (USPTO-190, NB=50N_B=50)
Methodλ\lambdaSucc. ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑
MCTS051.05%19.7929.47%10.53%0.0897
450.00%20.8531.05%11.58%0.0904
Retro*-0059.47%69.6237.89%24.74%0.1475
454.74%70.3637.89%25.26%0.1431
Retro*063.68%68.1544.74%28.95%0.1598
460.00%65.6343.16%27.37%0.1580
MO-MCTS–45.79%9.6325.26%8.95%0.0886
RetroPlanner–92.11%7.6266.84%33.68%0.1973

Table R10 shows that incorporating these signals into search does not consistently improve route quality and reduces success (e.g., Retro*: 63.68% → 60.00%). For Retro*, the added cost also penalizes longer routes, making deeper targets harder to solve.

Among these configurations, RetroPlanner reaches the highest success (92.11%) and BTR Dom. (33.68%) from the smallest route pool (7.62). We will include these results and the metric formulation in §A.4.

W1-2. Multi-Objective Search with Diversity-Aware Selection

The reviewer asks for a search procedure combining multi-objective scoring with diversity-aware selection.

RetroPlanner considers route diversity through its DECIDING schema, where diverging from identifies alternative disconnections relative to existing routes and ordered by records the evidence behind the choice, so selection is conditioned on route context rather than on numerical scores alone.

To test whether an explicit diversity incentive benefits non-LLM search in the same way, we added the reviewer’s term directly to the Retro* edge cost,

cost=−log⁡pmodel−λlog⁡f−νlog⁡(max⁡(1−max⁡σ∈SJ(prec,σ),0.05))\mathrm{cost} = -\log p_{\text{model}} - \lambda \log f - \nu \log \Big( \max\big(1 - \max_{\sigma \in S} J(\mathrm{prec}, \sigma), 0.05\big) \Big)

where JJ is the Jaccard overlap of starting-material sets and SS contains the leaf sets of previously recovered routes. We use r=3r=3 search restarts under the same total expansion budget, following the restart policy the reviewer asks us to match, which is also what lets the term act at all.

  • Table R11. Diversity-aware selection in Retro* (USPTO-190, NB=50N_B=50)
SetupSucc. ↑RoutesDissim ↑1stCutDiff ↑BTR Dom. ↑HV ↑
ν=0\nu=0, r=1r=170.00%83.490.7320.51829.47%0.1698
ν=4\nu=4, r=3r=371.23%73.960.7410.53729.47%0.1698
Diversity as 4th objective60.70%47.320.7690.63423.68%0.1457
RetroPlanner92.11%7.620.8010.67033.68%0.1973

Adding a diversity bonus to Retro* search yields limited improvements in diversity. Increasing ν\nu from 0 to 4 raises first-cut diversity from 0.518 to 0.537, while dissimilarity slightly moves (0.732 to 0.741), with comparable success and HV. Treating diversity as a fourth objective improves both diversity metrics (0.769 and 0.634), but reduces success to 60.70% and HV to 0.1457.

In contrast, RetroPlanner achieves the highest starting-material dissimilarity (0.801), first-cut diversity (0.670), success (92.11%), and HV (0.1973). Selection conditioned on route context therefore appears to balance diversity and quality better than a numerical bonus.

We note that the ν=0\nu=0 row uses a single pass and the ν=4\nu=4 row three restarts.

W1-3. Contribution of Reasoning Schema Components

We provide detailed component ablations in Table R5 of our response to Reviewer 4Uvy (W1 & Q1).

The L1L_1 (MOLECULE) block grounds the planner in the structural relationship between the current molecule and the target, likely contributing to its higher BTR. Although L3L_3 achieves lower BTR than L1L_1 (76.32% vs. 80.95%), it recovers more routes (16.76 vs. 13.35) and achieves higher HV (0.2563 vs. 0.2445).

We also separately evaluate the two subfields of DISCONNECTIONS. bond changes achieves higher BTR (75.99% vs. 71.84%) and BTR Dom. (43.50% vs. 39.62%) than reaction class, while the latter achieves slightly higher HV (0.2461 vs. 0.2416). Combining both yields the highest HV (0.2478) among the three variants.

These results highlight the distinct contributions of reasoning components to route quality and multi-objective coverage.

Q1. LLM inference cost

We report per-target end-to-end accounting across all three stages. LLM inference, single-step prediction, and round-trip scoring all run on GPU and are reported in Table R12. Candidate scoring comprises three objective proxies. Round-trip prediction runs on GPU and is reported in Table R12, while plausibility and molecular price are small models over fingerprints whose cost is negligible, at 8.8×10−68.8\times10^{-6} PFLOP per target for the planner. FLOPs use 2NP2NP with 3.6B active parameters for the planner, which is an MoE model.

  • Table R12. Per-target computation and route quality (USPTO-190, NB=300N_B=300)
MethodSucc. ↑ExpansionsLLM (PFLOP)Single-step (PFLOP)Round-trip (PFLOP)Total (PFLOP) ↓Wall-clock (s/target) ↓HV ↑
Expand-all62.63%181.400.6931.2461.94014,6390.1520
MCTS74.21%239.100.21000.210500.1562
Retro*-077.89%300.000.26400.264960.1892
Retro*87.89%300.000.26400.2641090.2107
MO-MCTS79.47%300.000.2641.0681.3324640.1171
MORetro*91.58%250.200.2200.8911.1102,7390.2905
RetroPlanner98.42%140.13.240.1170.1473.5043,3850.2563
  • Table R13. Measured GPU use and energy (USPTO-190, NB=300N_B=300)
MethodGPU-hours/targetActive Wh/targetIncluding idle
Expand-all0.10322.788.8
RetroPlanner0.04717.427.5

All runs use the same node of eight NVIDIA H100 GPUs, with four cards serving the single-step and forward models and four serving the LLM. GPU-hours represent utilization-weighted GPU time rather than elapsed wall-clock time, measured using per-card utilization counters.

Table R12 shows that Expand-all performs no pruning, opening every eligible frontier molecule and processing candidate reactions in first-seen order until the expansion budget is exhausted. Compared with Expand-all, RetroPlanner reduces molecule expansions by 22.8% (140.1 vs. 181.4) while achieving substantially higher success (98.42% vs. 62.63%) and HV (0.2563 vs. 0.1520). Although LLM inference dominates RetroPlanner's computational cost (3.24 of 3.504 PFLOPs), Table R13 shows 2.2× lower utilization-weighted GPU time and 23% lower active energy consumption compared with Expand-all.

Compared with MORetro*, RetroPlanner achieves higher success (98.42% vs. 91.58%) with 44% fewer molecule expansions, although MORetro* achieves higher HV at lower computational cost. Overall, LLM-guided OPEN selection achieves the highest success rate with the fewest molecule expansions, while maintaining competitive multi-objective route quality and reducing measured GPU resource consumption relative to Expand-all.

W2 & Q4. Budget counting

We thank the reviewer for identifying the inconsistent budget accounting. The evaluation pipeline for the LLM planners is shared across all of them, so no method is charged differently under it. The inconsistency was between that convention and table_geo.py (line 167), which accumulated the expenditure of independent baseline runs without deduplicating molecules across them. We re-evaluated all methods using unique molecule expansions as the common budget definition, and Tables R14 and R15 report the corrected results.

  • Table R14. Main results under matched budget counting (USPTO-190, NB=50N_B=50)
MethodCountingSucc. ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑
Retro-R1original82.11%2.9938.42%10.53%0.1085
matched91.05%4.7345.26%13.16%0.1348
RetroAgentoriginal96.84%3.0746.84%9.47%0.1074
matched98.42%3.5244.21%13.16%0.1243
RetroPlannermatched92.11%7.6266.84%33.68%0.1973
  • Table R15. Search baselines under matched counting, with original values in parentheses (USPTO-190, NB=50N_B=50)
MethodSuccess ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑
MCTS51.05% (44.74)19.79 (16.88)29.47% (27.89)10.53% (10.00)0.0897 (0.0817)
Retro*-059.47% (52.63)69.62 (29.55)37.89% (34.21)24.74% (23.16)0.1475 (0.1271)
Retro*63.68% (57.89)68.15 (33.34)44.74% (38.95)28.95% (24.74)0.1598 (0.1410)

Under matched counting, every method gains. All tables in the revised paper will follow this counting convention.

Q2. Does the approach require a large backbone?

We applied the same training recipe to two smaller backbones and report the results in Table R16.

  • Table R16. The same training recipe on smaller backbones (USPTO-190, NB=50N_B=50)
BackboneActive paramsSucc. ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑LLM PFLOP/target ↓Wall-clock (s/target) ↓
Qwen3-4B4.02B85.26%7.6960.00%32.63%0.19384.222,195
Qwen3.5-9B9.0B94.21%7.8069.47%37.89%0.208812.146,268
RetroPlanner (GPT-OSS-20B)3.61B92.11%7.6266.84%33.68%0.19733.243,385

Table R16 shows that the training recipe transfers across different backbones. Qwen3-4B achieves 85.26% success with lower route quality but faster inference, while Qwen3.5-9B achieves the highest success rate and route quality, at 3.75× the FLOPs and 1.85× the runtime of GPT-OSS-20B. GPT-OSS-20B activates only 3.61B parameters per token, achieving competitive performance with the lowest FLOPs and a favorable performance–compute trade-off, which is why we keep it as the main configuration.

Reviewer_RJWvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RETROPLANNER is an LLM-based multi-step retrosynthesis planner. Unlike existing agents that expand only one frontier molecule per turn, it opens multiple frontier molecules simultaneously, cross-compares their candidate reactions all at once, ranks them, and then outputs completed routes. The authors attempt to support this through a variety of experiments.

Critical strengths
  • This paper has good motivation and analysis. Existing LLM-based retrosynthesis planners expand only one frontier molecule per turn, so they have no opportunity to globally compare alternative routes before the search budget is exhausted. On a 60-target subset of USPTO-190, the authors present a motivating analysis showing that for "78.6% of the 42 targets solved by RetroAgent, a better route dominating that route exists within the search space, and 64.3% of these lie at nodes that the agent had already opened but did not expand."
  • Overall, extensive experiments were conducted, including ablation and efficiency-related experiments.
Critical weaknesses
  • An introduction to representative prior work on multi-objective retrosynthesis search, such as Lai et al. [1], is needed, along with a discussion of how this work differs and a performance comparison. An evaluation framework has already been explored in multi-objective retrosynthesis search [1], yet this paper does not use it at all and reports only its own self-defined, equally weighted scalar composite score (BTR). The differences need to be addressed.
  • The core claim (parallel > sequential) does not appear to be statistically significant.
  • While the paper claims to be multi-objective (the motivation, learning, and environment are all based on Pareto criteria), the evaluation uses equal-weight scalarization. Can the current approach verify a set that simultaneously satisfies diversity and quality?
  • The planner selects only among candidates proposed by the single-step model. It is still bounded by the single-step model’s top-K, and there is no sensitivity analysis on this.

[1] Lai, Helen, et al. "Multi-objective synthesis planning by means of Monte Carlo Tree search." Artificial Intelligence in the Life Sciences 7 (2025): 100130.

Questions for authors

Is p the expansion policy or the filter policy of AiZynthFinder? If it is the expansion policy, it is template-based, whereas the single-step model in this paper is template-free. How is p computed for template-free candidates, and what value is assigned to candidates that do not match the template library?

5 comments

We thank the reviewer for the helpful comments. In particular, the suggestion of a Pareto-based metric provided a useful perspective for our evaluation, and we will incorporate it into the revised paper. Please find our responses to the reviewer’s key concerns below.

W1 & W3. Beyond BTR: Pareto-Based Evaluation

We agree that a Pareto-based view is the appropriate one for this setting, and we have adopted it. We report hypervolume (HV) over each target’s route pool, and add MO-MCTS [1], the method the reviewer pointed us to, using our three objectives(plausibility, round-trip, starting-material price) with an additional reward for successful route completion.

BTR’s equal-weighted scalarization was intended to select a representative route from the Pareto front by balancing the three objectives. A dominance criterion is closer to the Pareto view, so we add BTR Dom., which counts a route as better only when it dominates the reference on every objective. Table R6 reports both.

Following Reviewer Qw2m’s suggestion, we also standardized budget accounting across all methods, counting each unique molecule expansion toward the budget. Table R6 reports the Pareto-based metrics under this matched budget definition.

  • Table R6. Pareto-based evaluation (USPTO-190, NB=50N_B=50)
MethodSucc. ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑
MCTS51.05%19.7929.47%10.53%0.0897
Retro*-059.47%69.6237.89%24.74%0.1475
Retro*63.68%68.1544.74%28.95%0.1598
MO-MCTS45.79%9.6325.26%8.95%0.0886
MORetro*67.89%43.1652.11%38.95%0.1979
RetroPlanner92.11%7.6266.84%33.68%0.1973

Pareto-based metrics provide insights beyond BTR. While MORetro* achieves slightly higher hypervolume and BTR Dom., RetroPlanner reaches comparable hypervolume while solving substantially more targets.

All methods use ReactionT5v2 for round-trip evaluation during search. To assess robustness, we re-scored the recovered routes using an alternative model (RXNG) under Table R7 and recomputed the Pareto metrics.

  • Table R7. Hypervolume under independent round-trip scoring (USPTO-190, NB=50N_B=50)
MethodHV (original proxy) ↑HV (alternative) ↑
MORetro*0.19790.1780
RetroPlanner0.19730.1819

Under the original proxy, MORetro* achieves slightly higher hypervolume, whereas the ordering reverses under alternative scoring, with RetroPlanner maintaining substantially higher success rates.

W2. Statistical Significance of Parallel vs. Sequential Expansion

To address the reviewer's concern, we ran both expansion strategies four times without explicit multi-objective reasoning supervision to isolate the effect of the expansion strategy itself, under matched settings on USPTO-190, reporting per-target paired differences with 95% confidence intervals to assess statistical significance in Table R8.

  • Table R8. Paired comparison of expansion strategies (USPTO-190, NB=300N_B=300)
AxisMulti-ParMulti-SeqΔ (Par - Seq), 95% CIz
Succ.99.74%99.47%+0.26% [−0.53, +1.32]+0.58
BTR Dom.42.24%35.92%+6.32% [+2.76, +10.00]+3.39
HV0.23430.2251+0.0091 [+0.0052, +0.0131]+4.55

Parallel expansion improves BTR Dom. by 6.32% and HV by 0.0091, while maintaining comparable success rates (99.74% vs. 99.47%), demonstrating better objective-space coverage.

W4. Top-k Analysis

We vary the candidate width KK at inference and report results in Table R9.

  • Table R9. Sensitivity to the single-step model(R-SMILES)’s top-KK (USPTO-190, NB=50N_B=50)
MethodKSucc. ↑BTR Dom. ↑HV ↑
RetroPlanner584.74%24.74%0.1596
1092.11%33.68%0.1973
2095.79%40.00%0.2442
5098.95%43.68%0.2840

Although the planner is trained with K=10K=10, increasing KK at inference consistently improves success and objective-space coverage without retraining, demonstrating its ability to benefit from a wider candidate pool.

Q1. p - plausibility?

It is a filter policy. AiZynthFinder [2] separates expansion and filter policies: the former proposes disconnections, while the latter evaluates generated reactions. We use template-free R-SMILES for expansion and AiZynthFinder’s filter policy for plausibility scoring.

[1] Lai, Helen, et al. "Multi-objective synthesis planning by means of Monte Carlo Tree search." Artificial Intelligence in the Life Sciences 7 (2025): 100130.

[2] Saigiridharan, Lakshidaa, et al. "AiZynthFinder 4.0: developments based on learnings from 3 years of industrial application." Journal of cheminformatics 16.1 (2024): 57.

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

We thank the reviewer again for the thoughtful and constructive feedback, particularly for suggesting a Pareto-based evaluation of multi-objective retrosynthesis planning. We briefly summarize how the concerns were addressed through additional experiments and analyses.

W1 & W3: We introduced two Pareto-based metrics (BTR Dom. and hypervolume) and added MO-MCTS as a multi-objective search baseline. At a budget of 50, RetroPlanner achieves 92.11% success, 66.84% BTR, and 0.1973 HV, compared with MORetro* (67.89%, 52.11%, and 0.1979 HV). Although MORetro* achieves slightly higher HV under the original evaluator, RetroPlanner achieves higher HV under an alternative round-trip model (0.1819 vs. 0.1780), demonstrating competitive multi-objective coverage and robustness across evaluators.

W2: We repeated parallel and sequential expansion experiments four times under matched settings and computed paired 95% confidence intervals. Parallel expansion improves BTR Dom. by 6.32 percentage points (95% CI: +2.76 to +10.00) and HV by 0.0091 (95% CI: +0.0052 to +0.0131), while maintaining comparable success rates. These results provide statistically significant evidence for improved multi-objective coverage through parallel expansion.

W4: We conducted a top-K sensitivity analysis without retraining the planner. Increasing the single-step candidate width from K=5 to K=50 improves success from 84.74% to 98.95%, BTR Dom. from 24.74% to 43.68%, and HV from 0.1596 to 0.2840, demonstrating that RetroPlanner benefits from a broader candidate pool despite being trained with K=10.

Q1: We clarified that AiZynthFinder's filter policy evaluates reaction plausibility, while the template-free R-SMILES model generates reaction candidates. These two components operate independently, resolving the ambiguity regarding plausibility scoring for template-free reactions. We will revise the corresponding description in the Appendix §A.2 to clarify this distinction.

We hope these additional experiments address the concerns regarding Pareto-based evaluation, statistical significance, and candidate-width sensitivity. Please let us know if any concern remains unresolved or if further clarification would be helpful.

Reviewer_4UvyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper introduces RetroPlanner, an LLM-based framework for multi-step retrosynthesis planning that aims to recover diverse and synthesizable routes rather than merely finding a single feasible solution. Its main contributions are multi-route expansion, which jointly considers multiple frontier molecules within an AND–OR graph, and evidence-grounded multi-objective reasoning, which guides reaction selection using plausibility, forward round-trip consistency, and starting-material price. The planner is supervised on Pareto-filtered search trajectories augmented with verified reasoning traces. Experiments on USPTO-190 and ChEMBL-1000 demonstrate improved route diversity and several route-quality metrics compared with existing LLM-based planners, while maintaining relatively high search success rates.

Critical strengths

S1. Well-Motivated Multi-Route Planning with Promising Empirical Results

The paper identifies a meaningful limitation of single-molecule frontier expansion and introduces a joint multi-route planning strategy that coordinates exploration across competing branches. The results demonstrate substantially improved route diversity over existing LLM-based planners, together with stronger round-trip consistency and competitive overall route quality. The expansion-strategy ablations further support the benefits of multi-route exploration while showing improved search efficiency through parallel decision-making.

S2. Well-Engineered Planning Framework with Evidence-Grounded Supervision

The shared AND–OR DAG representation and structured chemical reasoning framework are thoughtfully designed for multi-step synthesis planning. The cheminformatics-based verification pipeline provides useful consistency checks for teacher-generated reasoning traces. The extensive implementation and robustness experiments with alternative single-step and forward reaction models further strengthen the technical contribution.

Critical weaknesses

W1. Insufficient Isolation of the Proposed Contributions

The empirical gains are difficult to attribute to the proposed planning mechanisms because RetroPlanner explicitly observes the chemical proxy signals used in evaluation, whereas competing LLM-based planners do not optimize the same objectives. Although the existing ablations partially address this issue, they do not fully disentangle the benefits of multi-route planning, additional proxy signals, and Pareto-filtered supervision under matched LLM training and inference settings. Moreover, Table 2 shows that the full reasoning schema (L3) achieves lower Better-Than-Ref than molecule-level reasoning alone (L1), raising questions about the independent contribution of the additional reasoning components. The expansion-strategy variants also use partially different training corpora, further complicating attribution.

W2. Route Evaluation Implementation and Protocol Consistency

The released evaluation drivers call _route_rxns and _route_leaves without passing the route-specific argument. Consequently, distinct solved pathways sharing a root reaction but diverging downstream can be serialized using the same closed_via choices, potentially affecting route counts, diversity, and quality metrics. The extent to which this issue affects the reported results remains unclear. Additionally, Better-Than-Ref values appear consistent with different reference-target denominators in the main and ablation experiments. The expansion-strategy and reasoning-schema ablations also report substantially different performance under the same nominal search budget while using different training configurations. These issues warrant clearer documentation and validation of the evaluation protocol.

Questions for authors

Q1. Contribution Attribution

Under matched proxy signals, training supervision, and inference budgets, what are the independent contributions of multi-route expansion and the full reasoning schema? In particular, why does the full L3 schema underperform the simpler L1 variant on Better-Than-Ref?

Q2. Route Export and Metric Validity

Were the reported results generated using the released route export implementation, in which _route_rxns and _route_leaves are called without the route-specific argument? If so, how do the route-count, diversity, and quality metrics change after correcting the exported pathways?

Q3. Ablation Comparability and Evaluation Denominators

Could the authors clarify the differences in training corpora, checkpoints, and evaluation protocols between Figure 4 Parallel and Table 2 L3, which exhibit substantially different performance under the same nominal search budget? Were different reference-target denominators used for Better-Than-Ref across the main results and ablation studies?

7 comments

We thank the reviewer for the helpful comments and for identifying an issue in our implementation. Correcting this issue allowed us to obtain accurate results, which we will report in the revised paper. Please find our responses to the reviewer’s key concerns below.

W2 & Q2. Route Export Bug and Corrected Results

The issue is limited to route export for ours, RetroPlanner. During search, each registered route stores its own reaction choice for every molecule (route.choices), so routes that diverge at interior nodes are correctly distinguished and registered as separate routes. We corrected the export code and re-evaluated all results. The main results in Table R1 are only marginally affected. However, the route-level metrics change more substantially in the ablations, reported in Tables R1–R2.

We have revised the corresponding analyses accordingly, and all results in the revised paper will reflect the corrected evaluation.

  • Table R1. Main Results - Table 1 (USPTO-190, NB=50N_B=50)
MethodSucc. ↑Routes ↑Better-Than-Ref(BTR) ↑Plausibility ↑Round-Trip ↑Price ↓
RetroPlanner (reported)original92.11%7.4166.84%86.03%61.11%167.80
RetroPlanner (corrected)corrected92.11%7.6266.84%89.78%63.37%160.90
  • Table R2. Ablation Studies - Multi-Route (Par.) (Figure 4(a) in the paper) and L3L_3 (Table 2 in the paper) (USPTO-190, NB=300N_B=300)
MethodSucc. ↑Routes ↑1stCuts ↑1stCutDiff ↑BTR ↑
Multi-Route (Par.)original100.00%7.753.540.65482.63%
corrected100.00%11.523.470.61678.42%
L3L_3 (RetroPlanner)original98.42%5.593.350.74572.63%
corrected98.42%16.764.410.72776.32%

W1 & Q1. Disentangling Component Contributions of RetroPlanner

The reviewer questions whether RetroPlanner's improvements come from multi-objective planning or simply from access to chemical signals unavailable to competing LLM planners.

Our motivation experiments show that existing LLM planners do not explicitly incorporate multiple optimization objectives(plausibility, round-trip, and starting-material price) into their planning decisions. RetroPlanner addresses this by reasoning over chemical signals representing these objectives, motivating the joint comparison of multiple candidate routes within a single planning step. Multi-route expansion enables such comparisons, while Figure 4(b) shows that parallel expansion reduces computational cost compared with sequential expansion. We further disentangle the contributions of planning strategy and chemical signals through the component ablations in Table R3.

Following a suggestion from Reviewer RJWv, we also add two Pareto-based metrics to these tables: hypervolume (HV) over each target’s route pool, and BTR Dom., which counts a route as better than the reference only if it Pareto-dominates across all three objectives. We report them alongside BTR, which compares the route selected from the Pareto front using an equal-weighted composite score, to provide a more comprehensive evaluation of reference-based route quality and multi-objective coverage.

  • Table R3. Component attribution (USPTO-190, NB=50N_B=50)
MethodParallel Multi-Route expansionChemical signalPareto-filtered supervisionSucc. ↑Routes ↑BTR ↑BTR Dom. ↑HV ↑
RetroAgent✗✗✗98.42%3.5244.21%13.16%0.1243
gpt-oss-20b (backbone)✓✓✗54.21%4.5536.84%23.68%0.1386
RetroPlanner (w/o signal at inference)✓✗✓88.94%8.3757.37%24.21%0.1733
RetroPlanner (Seq.+Reasoning)✗✓✓85.26%4.1052.11%26.32%0.1712
RetroPlanner✓✓✓92.11%7.6266.84%33.68%0.1973

Table R3 shows that planning strategy, chemical signals, and Pareto-filtered supervision jointly contribute to multi-objective route quality. Sequential expansion with multi-objective reasoning limits the planner's ability to jointly compare multiple routes. The model trained with Pareto-filtered supervision also substantially outperforms the zero-shot prompted backbone. This combination is absent in prior works.

W2 & Q3. Clarifying Ablation Settings and Evaluation Consistency

The reviewer asks whether differences in training corpora, checkpoints, or evaluation protocols explain the performance gap between Figure 4(a) and Table 2 under the same search budget.

Following the correction of the route export issue described above(W2&Q2), Both Tables R4 (Figure 4(a) in the paper) and R5 (Table 2 in the paper) report the corrected ablation results under the same maximum budget of 300, using the same training instances and action fields, differing only in whether multi-objective reasoning supervision during training.

Regarding the reviewer's question about BTR denominators, both the main results and ablations use the same 190 USPTO-190 targets, each with one reference route, ensuring identical denominators for BTR and BTR Dom. across all tables.

  • Table R4. Effect of Expansion strategy (USPTO-190, NB=300N_B=300)
MethodSuccessRoutes1stCuts1stCutDiffBTRBTR Dom. ↑HV ↑
Random87.89%15.041.710.29846.84%11.05%0.1339
Score-only95.26%15.901.690.30474.21%48.42%0.2447
Single-Route98.42%6.082.250.43768.42%35.79%0.2260
Multi-Route (Seq.)99.47%14.983.510.53075.26%45.79%0.2407
Multi-Route (Par.)100.00%11.523.470.61678.42%49.47%0.2550

The corrected results clarify the distinction between route count and route diversity. Multi-Route (Par.) achieves the best overall performance in success, route quality, and first-cut diversity, despite recovering fewer routes than some alternatives.

W1 & Q1. Contribution of Additional Reasoning Components

The reviewer questions the contribution of additional reasoning components, as the full schema (L3L_3) achieves lower BTR than molecule-level reasoning alone (L1L_1).

The L1L_1(MOLECULE) block grounds the planner in the structural relationship between the current molecule and the target. For example, it explicitly identifies structural features retained from the target and additional groups introduced along the synthesis route. This target-relative reasoning likely contributes to the strong BTR performance of L1L_1 (80.95%, the highest among the variants).

MOLECULE
target: oxadiazole bearing a dimethylamide, a cyclohexyl-substituted carbon, ...
this piece: the same oxadiazole and cyclohexane, plus a methyl ester, ...
not in the target: a methyl ester and a tetrahydropyran ring.

In Table R5, we additionally report Pareto-based metrics following Reviewer RJWv's suggestion to evaluate multi-objective route quality beyond BTR. We also separately ablate bond changes and reaction class within DISCONNECTIONS, addressing Reviewer Qw2m's W1.

  • Table R5. Effect of Reasoning schema (USPTO-190, NB=300N_B=300)
VariantParant.Succ. ↑Routes ↑1stCuts ↑1stCutDiff ↑BTR ↑BTR Dom. ↑HV ↑
L0L_0 (Multi-Route Par.)100.00%11.523.470.61678.42%49.47%0.2550
L1L_1 (+MOLECULE)L0L_0100.00% (+0.00)13.35 (+1.83)4.64 (+1.17)0.710 (+0.094)80.95% (+2.53)40.15% (−9.32)0.2445 (−0.0105)
L2-bondL_{2\text{-bond}} (bond changes only)L1L_198.42% (−1.58)13.23 (−0.12)4.53 (−0.11)0.724 (+0.014)75.99% (−4.96)43.50% (+3.35)0.2416 (−0.0029)
L2-rxnL_{2\text{-rxn}} (reaction class only)L1L_197.89% (−2.11)13.23 (−0.12)4.49 (−0.15)0.722 (+0.012)71.84% (−9.11)39.62% (−0.53)0.2461 (+0.0016)
L2L_2 (+DISCONNECTIONS)L1L_198.42% (−1.58)13.19 (−0.16)4.39 (−0.25)0.709 (−0.001)76.29% (−4.66)37.69% (−2.46)0.2478 (+0.0033)
L3-divL_{3\text{-div}} (+diverging from)L2L_298.95% (+0.53)13.18 (−0.01)4.73 (+0.34)0.725 (+0.016)74.41% (−1.88)40.77% (+3.08)0.2476 (−0.0002)
L3-ordL_{3\text{-ord}} (+ordered by)L2L_298.95% (+0.53)13.66 (+0.47)4.58 (+0.19)0.727 (+0.018)77.18% (+0.89)39.72% (+2.03)0.2483 (+0.0005)
L3L_3 (RetroPlanner)L2L_298.42% (+0.00)16.76 (+3.57)4.41 (+0.02)0.727 (+0.018)76.32% (+0.03)46.84% (+9.15)0.2563 (+0.0085)

The two DECIDING subfields contribute to route diversity: diverging from achieves the highest 1stCuts (4.73), while ordered by achieves the highest 1stCutDiff (0.727). Although L3L_3 has lower BTR than L1L_1, combining both subfields yields more routes (16.76 vs. 13.35) and higher HV (0.2563 vs. 0.2445), supporting their contribution to route diversity and multi-objective coverage.

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Summary of Revisions

We thank all reviewers for their thoughtful and constructive feedback. Their comments helped us clarify our contributions, strengthen the experimental evaluation, and better characterize the scope and limitations of our approach.

Contribution and scope

RetroPlanner addresses multi-step retrosynthesis planning by jointly expanding multiple frontier molecules and reasoning over reaction plausibility, round-trip consistency, and starting-material price. Our primary contribution is the efficient allocation of limited single-step expansion budgets to recover diverse, high-quality synthetic routes while maintaining high search success. We revised the manuscript to clarify this focus and distinguish the contributions of multi-route expansion and evidence-grounded multi-objective reasoning.

Summary of revisions

We expanded the manuscript with the following experiments and analyses.

  • Multi-objective evaluation. Added Pareto-dominance-based Better-Than-Reference (BTR Dom.), hypervolume (HV), and structural diversity metrics to evaluate route quality beyond scalarized scores and route counts (Section 3.1, Appendix A.4).
  • Matched search baselines. Added MO-MCTS and evaluated conventional search methods with matched chemical signals, composite scoring, and diversity-aware selection. These experiments show that directly incorporating chemical objectives into search priorities does not consistently improve success or route quality (Appendix A.5, Tables 6–8).
  • Budget scaling and candidate sensitivity. Extended search-budget comparisons from NB=50N_B=50 to 500500, evaluated route quality at NB=300N_B=300, and varied the single-step top-KK candidates. The results characterize RetroPlanner's limited-budget performance, MORetro*'s stronger Pareto quality at larger budgets, and the influence of candidate availability (Appendix A.5–A.6, Tables 9–11).
  • Expansion strategy and statistical validation. Compared random, score-only, single-route, sequential multi-route, and parallel multi-route expansion under controlled settings. Parallel expansion achieves the highest structural diversity and HV, with paired experiments confirming improvements in Pareto quality over sequential expansion (Section 4.1, Appendix A.8, Tables 2 and 14).
  • Reasoning schema and planning behavior. Disentangled molecule-level reasoning, bond changes, reaction classes, and the DECIDING subfields. The full schema achieves the highest route count and HV, while attention analyses provide evidence that reasoning content is associated with decisions over explored routes (Section 4.2, Appendix A.8, Table 3, Figure 7).
  • Reaction-model robustness. Replaced R-SMILES with LocalRetro without additional planner training and ReactionT5v2 with RXNGraphormer for independent round-trip evaluation. RetroPlanner maintains the highest HV and round-trip consistency among the compared planners in both settings (Section 4.3, Appendix A.7, Tables 12–13).
  • Computational cost and backbone sensitivity. Added end-to-end FLOPs, wall-clock time, utilization-weighted GPU-hours, and energy comparisons, explicitly accounting for LLM inference overhead. Evaluated additional LLM backbones to characterize performance–compute trade-offs (Appendix A.9, Tables 15–17).
  • Reproducibility and supplementary code. Clarified budget accounting, representative-route selection, Pareto-based evaluation, and training-data construction. We also completed debugging of the implementation and uploaded the updated code to the supplementary materials (Sections 2–3, Appendices A.3–A.4).

We have updated the manuscript and highlighted all revisions in blue. Our individual reviewer responses provide further experimental details and clarifications corresponding to their specific concerns.

We hope these revisions address the reviewers' principal concerns and provide a clearer assessment of RetroPlanner's contributions and limitations. We would be grateful if the reviewers and Area Chairs considered the additional evidence and revised manuscript in their final assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.