- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RetroRoute, a multistep retrosynthesis framework combining a planning-conditioned MolT5 proposal model, utility-filtered positive refinement, and a ChemDFM-based state-aware listwise reranker. The method achieves improved Top-1 reference-route recovery on RetroBench, with particularly strong gains over proposal-only variants under greedy search.
Critical strengths
-
The framework addresses a relevant limitation of single-step retrosynthesis: a locally plausible reaction may be unhelpful for completing a synthesis route.
-
The controlled experiments show consistent gains from the planning scaffold, URPO, and the trained Analyzer. The greedy-search experiments are particularly useful in demonstrating improved first-choice branch ordering.
Critical weaknesses
-
The current evidence does not convincingly establish that the improvement comes from route-level or state-aware decision making. The Analyzer receives not only route-level information such as the frontier and history, but also strong local candidate features including proposal rank, structural validity, and purchasable-reactant count. The comparison against an unadapted ChemDFM therefore does not isolate the contribution of the evolving search state. A trained local reranker with the same candidate-level features but without route history/frontier would be a much more informative control.
-
The evaluation is strongly tied to reference-route recovery, making the claimed improvement in general route quality unclear. The main “route exact match” metric is based on matching the terminal starting-material set to a recorded reference, while the training procedure also uses route annotations and a reference-alignment utility term. It is therefore difficult to determine whether RetroRoute learns general long-term synthesis utility or primarily becomes better at recovering benchmark-compatible reference routes.
Questions for authors
- Can the authors provide an ablation that removes route-level state information (e.g., frontier, history, and remaining budget) while retaining the same trained reranker and local candidate features?
- How exactly are the positive actions used to train the Analyzer constructed? In particular, are they defined by reference-route membership, URPO utility, downstream search success, or some combination of these signals?
- Can the authors report reference-independent measures of route quality or chemical validity?
- What is the contribution of the individual URPO utility terms, especially and ?
- Can a smaller non-LLM reranker trained with the same listwise supervision achieve comparable performance?
5 comments
Thanks for the review - Weakness2
Response to Weakness2:
We sincerely thank the reviewer for raising this important concern. We agree that reference-route exact match alone cannot establish general synthesis quality, particularly when route annotations are used during training. We would therefore like to clarify the distinction between reference-derived training supervision and reference-independent route evaluation, and provide additional evidence beyond reference-route recovery.
1. The role of reference information in training and inference. We acknowledge that the Analyzer uses reference-reaction supervision at teacher-forced training states, and that URPO incorporates a reference-alignment utility signal. These annotations provide supervision for learning candidate preferences, while inference proceeds using model-generated candidates and the dynamically evolving search state.
2. RetroRoute explicitly connects sequential reaction decisions through iterative route-context updates. At each planning step, the search state incorporates the unresolved frontier, accumulated route history, and remaining search budget. After a reaction is selected, the controller updates the frontier and records the decision in the route history. The Analyzer then evaluates subsequent candidates using this updated context. Consequently, later decisions are conditioned on the partial synthesis route constructed by preceding model decisions, rather than being independently matched against a fixed reference trajectory. This iterative mechanism is central to our route-level planning design.
3. Additional reference-independent evaluation metrics. We evaluate the quality of routes generated by RetroRoute on 5,838 test targets from multiple reference-independent perspectives, including route diversity, average number of reaction steps, and route complexity, as summarized in Table R3-3. Overall, these metrics indicate that RetroRoute produces diverse, structurally consistent, and moderately complex routes, providing complementary evidence of route generation quality beyond reference-route matching.
Table R3-3. Reference-independent evaluation of generated routes under beam search.
| Metric | Result |
|---|---|
| Completed targets | 5,826 / 5,838 |
| Beam search completion rate | 99.79% |
| Top-5 reaction diversity | 0.791 |
| Top-5 starting-material diversity | 0.581 |
| Distinct reaction routes (Top-5) | 4.91 |
| Average reaction steps (Top-1) | 3.13 |
| Average maximum route depth (Top-1) | 3.02 |
| Average distinct starting materials (Top-1) | 3.46 |
| Valid SMILES rate (Top-1) | 100% |
| Acyclic decomposition graph rate (Top-1) | 99.47% |
Overall, we position RetroRoute as a framework for state-aware sequential retrosynthetic planning. Importantly, RetroRoute is not directly trained using rollout-derived route-completion rewards. Instead, URPO uses predefined chemical and route-related utility signals as offline reward surrogates, while the Analyzer learns state-conditioned candidate preferences through teacher-forced reference-action supervision. Our contribution therefore lies in integrating these complementary supervision signals with dynamically updated planning states, rather than explicitly learning a calibrated long-horizon return from observed synthesis outcomes. We hope this clarification addresses the reviewer's concern about the distinction between reference-derived training supervision and reference-independent route evaluation, and shows that RetroRoute is not merely recovering reference routes but performing state-aware sequential planning.
Thanks for the review - Question1 & 2
Response to Question1:
We thank the reviewer for this helpful suggestion. We are very sorry for unclearly demonstrating our experimental results. The original comparison against an unadapted ChemDFM did not isolate the contribution of route-state information from that of local candidate features.
To address this concern, we introduce a matched Local-only ChemDFM control. It uses the same pretrained initialization, LoRA adaptation procedure, training examples, and listwise supervision as the full Analyzer. It retains all local candidate descriptors, including proposal rank, structural validity, and purchasable-reactant count, while excluding the unresolved frontier, route history, and remaining search budget.
Table R3-1: Contribution of route-state information in the Analyzer.
| Analyzer | Local features | Route-state features | Beam Top-1 | Beam Top-3 | Beam Top-5 | Greedy DFS Top-1 |
|---|---|---|---|---|---|---|
| Local-only ChemDFM | Yes | No | 36.6% | 46.3% | 49.8% | 35.52% |
| Full-state ChemDFM | Yes | Yes | 37.6% | 46.7% | 50.1% | 36.67% |
The full-state Analyzer achieves consistent improvements across both beam search and greedy DFS, with gains of 1.0 and 1.15 percentage points in Top-1 accuracy, respectively. The improvements in Top-3 and Top-5 are smaller, suggesting that local candidate descriptors already provide a strong basis for candidate assessment.
These results indicate that route-state information serves as complementary planning context. By jointly considering candidate-specific properties and the evolving synthesis state, the full Analyzer achieves more effective branch evaluation across different search strategies. This supports the value of integrating local candidate assessment with state-aware decision making in RetroRoute.
Response to Question2:
We appreciate the opportunity to clarify the construction of the Analyzer's positive actions.
The Analyzer is trained using teacher-forced states derived from the annotated training synthesis routes. At each training state , the Reviewer generates a candidate set . The positive subset is defined as
Thus, positive labels are determined by reference-reaction matching at the corresponding training state. Neither URPO utility scores nor downstream rollout/search success is used to construct .
The Analyzer is optimized using the multi-positive listwise objective:
=-\log\frac{\sum_{i\in P_t}\exp(z_i)} {\sum_{j\in C_t}\exp(z_j)}.$$ The two types of positive supervision serve different purposes. URPO uses complementary chemical and route-related utility signals to identify additional feasible alternatives for Reviewer refinement, whereas the Analyzer uses reference-route supervision to learn state-conditioned ranking among generated candidates. In particular, the Analyzer's positive labels are not inherited from URPO and do not represent observed downstream route-completion rewards. We hope this clarification resolves the reviewer's question about how $P_t$ is constructed and makes clear that the Analyzer's positive supervision is based solely on reference-reaction matching at teacher-forced training states, rather than on URPO utility scores or downstream search outcomes.
Thanks for the review - Weakness1
Dear Reviewer PmE8, We sincerely appreciate the time and effort you have taken to provide valuable feedback on our work. Your insightful comments have been immensely helpful in improving the clarity and quality of our manuscript. We have carefully considered each of your suggestions and have made the appropriate revisions to address your concerns. Below, we present detailed, point-by-point responses to your observations.
Response to Weakness1:
We agree that the original comparison with an unadapted ChemDFM does not fully isolate the contribution of route-state information. We therefore provide an additional controlled ablation and further clarify the motivation behind our framework.
First, the central motivation of RetroRoute is to bridge the gap between local reaction plausibility and sequential route-level decision making. Local descriptors, such as proposal rank, structural validity, and purchasable-reactant count, provide valuable evidence about individual reactions. However, they do not fully characterize the evolving synthesis problem, where the suitability of a reaction also depends on unresolved intermediates, previous decisions, and the remaining search budget. Accordingly, our contribution is not simply to introduce additional features into a reranker, but to establish a unified planning framework that separates candidate generation from state-conditioned branch evaluation. The Reviewer expands the candidate space through planning-conditioned generation and URPO refinement, while the Analyzer integrates local reaction evidence with the evolving search context to guide branch selection.
Second, we conduct a controlled ablation to explicitly assess the contribution of route-state information. As detailed in our response to Question 1, the Local-only ChemDFM retains the same pretrained initialization, LoRA adaptation, training examples, listwise supervision, and candidate-level descriptors as the full Analyzer, but excludes the unresolved frontier, route history, and remaining search budget.
Table R3-1: Contribution of route-state information in the Analyzer.
Table R3-1 indicates that local descriptors already provide a strong foundation for candidate evaluation, while route-state information contributes additional, complementary planning context. We therefore emphasize that the advantage of RetroRoute arises from integrating complementary reaction-level and route-level information within a sequential decision-making framework, rather than attributing the overall improvement to any single input component. This distinction is central to our methodological contribution and is further supported by the controlled ablation.