acceptodds
Under review as a conference paper at ICLR 2027

Coverage Before Control: Route-Instruction Grounding and Steering for Controllable Retrosynthesis

Abstract

Single-step retrosynthesis models are commonly evaluated by their ability to recover recorded reactions. In practice, chemists may need to choose among several precursor sets for the same product, for example to preserve a particular motif. Recovering a recorded answer alone does not establish this ability to follow a preference. Satisfying such requests requires both coverage of relevant alternatives and control over which alternatives are favored. We introduce Route-Instruction Grounding and Steering (RIGS), a two-stage framework for instruction-conditioned retrosynthesis. Stage A trains a language projector, teaching it which alternatives an instruction favors or discourages. Stage B uses the projector learned in Stage A to steer a frozen generative model through lightweight residual adapters. We construct nested one-to-many training supports by pairing each product with increasing numbers of candidate precursor sets. Broader support helps the model generate a wider range of alternatives, and RIGS learns to guide generation according to instructions. The relationship between coverage and control is consistent across model scales but non-monotone.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Retrosynthesis, language guided generation, free form steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9UJROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Route-Instruction Grounding and Steering (RIGS) as a steering adapter to improve sampling frequency of specific retrosynthetic precursors according to a language guidance input. This is done with a frozen Qwen3 embedding layer for the language input, their two-stage RIGS steering adapter aligning the language embeddings and steering generation, and then generation with GraphDiT.

Critical strengths

The authors show that, compared to matched reference and matched shuffled results, performance is generally better by around 10-20 percentage points, but with a large variance, so instruction following does work. Increasing the precursor training pool isn’t monotonic, with the best results at top30. They show that their stage A alignment does improve performance over just stage B interventions, but primarily in the avoided set. They also compare their RIGS generation-time guidance to reranking afterward, with RIGS P@1 performance better than reranking, but N@1 performance worse. They finish by showing that their results generalize, to varying degrees, to other datasets.

Critical weaknesses

The main contribution of RIGS seems to be its steering performance, but it is unclear that RIGS performs better than a simple reranking. Additionally, the evaluation metric used to benchmark RIGS is lacking completeness and may be skewing results. The preferred set used to evaluate performance contains a set of reasonable preferred reactions, but it doesn’t contain all possible paths that should be preferred, so it’s possible to generate a precursor that follows the language guidance instructions, but is penalized for not being in the preferred set. Generalization, particularly to the Pistachio dataset, is also somewhat lacking.

Questions for authors
  • Why did you choose to interfere at generation-time if the primary goal is to rerank, given that RIGS doesn’t show coverage improve?
  • You claim that coverage is not compromised. In addition to the HIT@100 in table 3, it would be helpful to include total percentage of preferred candidates rather than asking it it just contains a single preferred candidate.
  • You include a comparison to simple reranking in table 3, which is helpful, but the comparison to RIGS doesn’t seem fair. The reranking employed is much simpler than RIGS, and may not be accurately representing a fair comparison. For a perfect reranking, P@1 should be equal to HIT@100, rather than the 22.23 to 64.80 difference. Further, reranking for P@1 does even worse than the baseline. Ablations targeting this reranking, perhaps focusing on P@1 performance rather than N@1 performance which the current reranking seems to be doing, would be very helpful.
  • The preferred set is a set of at most 100 precursors, but how well does this span the space of all precursors that fit the guidance? In particular, the RDKit oracle used for reranking, which should provide perfect ranking, achieves a P@1 of 26.99 in table 16 instead of equaling the HIT@100 of 64.80. Since this is the case, how do you rationalize using this constructed set of 100 precursors when there are many other valid precursors that will be penalized in evaluation, which may be skewing benchmark results?
  • Why is the performance on the Pistachio set so much worse than USPTO?
6 comments
AuthorsOpenReview ↗

Generation-Time Steering, Reranking, and Preferred-Set Evaluation

We thank the reviewer for recognizing the instruction-following gains and Stage A’s suppression of conflicting alternatives. We clarify generation-time steering, LLM selection, and evaluation beyond reference membership.

1. Why steer at generation time

Coverage asks whether a relevant alternative appears; steering changes how frequently it is proposed. RIGS can therefore improve instruction control with little change in Hit@100. In multistep planning, single-step proposals supply search expansions [1]. Concentrating proposals on compatible alternatives can reduce the effective branching factor when the planner prioritizes or limits expansions using this distribution.

2. Reranking and candidate-pool composition

A perfect preferred-membership selector would attain P@1 = Hit@100 on a fixed pool. Reliable reranking remains a challenging research problem in retrosynthetic planning [2,3]. Our appendices assess complementary requirements using RDKit validity and descriptor utility, relaxed round-trip accuracy and coverage, SCScore-based complexity reduction, procurement burden, and reaction-class ambiguity. Forward consistency offers partial evidence of chemical feasibility [4].

We additionally compare Reference, Reference + LLM, RIGS, and RIGS + LLM using four direct property evaluators: procurement burden, synthetic complexity difference, halogen count, and ring count. These assess instruction satisfaction beyond offline Top100 membership. Compared with Reference + LLM, RIGS + LLM reduces mean procurement burden from 9.86 to 8.93 under lower-burden guidance, increases halogen count from 2.75 to 3.17 under higher-halogen guidance, and increases ring count from 3.04 to 3.38 under higher-ring guidance. Steering changes the proposed distribution; LLM selection assesses completed candidates.

Separately, we compute product-weighted empirical quantiles for frequency-ranked Top-1 and Top-10 outputs on 963 paired Reference–RIGS products. Under lower-complexity guidance, all five quantiles decrease for both rankings: the Top-1 median falls from 0.401 to 0.157 and Q75 from 0.860 to 0.613. Under lower-halogen guidance, both rankings’ medians decrease from 1 to 0 and Q75 from 2 to 1. Zero-halogen output fractions increase from 38.14% to 55.93% for Top-1 and from 31.57% to 51.55% for Top-10.

Top-10 quantiles. Each line gives Reference → RIGS, with values ordered Q10/Q25/Q50/Q75/Q90; arrows beside metrics indicate the requested direction.

Procurement ↓: 4.33/10.00/10.69/13.00/20.00 → 4.14/10.00/10.69/13.18/20.00
Procurement ↑: 3.47/10.00/10.69/12.30/20.00 → 3.78/10.00/10.69/13.04/20.00
Complexity difference ↓: −0.418/0.000/0.416/0.872/1.261 → −0.594/−0.138/0.325/0.794/1.221
Complexity difference ↑: −0.441/−0.005/0.412/0.888/1.382 → −0.379/0.030/0.421/0.926/1.366
Halogen count ↓: 0/0/1/2/3 → 0/0/0/1/3
Halogen count ↑: 0/0/1/2/3 → 0/1/1/2/3
Ring count ↓: 1/1/2/3/4 → 1/1/2/3/4
Ring count ↑: 1/1/2/2/3 → 1/1/2/3/4

These results complement Hit@100 by showing changes across the returned alternatives, beyond the appearance of a single favorable candidate.

3. Preferred-set coverage

The finite inventory provides a reproducible multi-reference benchmark of known instruction-compatible alternatives. Exhaustively enumerating all compatible precursor sets is impractical. Reference, Matched, and Shuffled use the same frozen inventory, enabling paired comparisons of instruction-dependent control. At 65M Top30, Matched improves P@1 over Shuffled by 18.52 percentage points. Direct property evaluation additionally credits improvements outside this inventory.

4. Performance on Pistachio

Our templates [5] are extracted from USPTO. Differences in Pistachio’s reaction distribution may reduce their coverage and applicability, limiting planning performance. Extending our data pipeline to extract and curate templates from more diverse reaction sources is an important direction for future work.

References

[1] Chen, B., Li, C., Dai, H., & Song, L. (2020). Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search. ICML, PMLR 119, 1608–1616.

[2] Lin, M. H., Tu, Z., & Coley, C. W. (2022). Improving the performance of models for one-step retrosynthesis through re-ranking. Journal of Cheminformatics, 14, 15.

[3] Li, J., Fang, L., & Lou, J.-G. (2023). RetroRanker: leveraging reaction changes to improve retrosynthesis prediction through re-ranking. Journal of Cheminformatics, 15, 58.

[4] Torren-Peraire, P., et al. (2024). Models Matter: the impact of single-step retrosynthesis on synthesis planning. Digital Discovery, 3, 558–572.

[5] Yu, K., et al. (2024). Double-Ended Synthesis Planning with Goal-Constrained Bidirectional Search. Advances in Neural Information Processing Systems, 37, 112919–112949.

Thank you for your response. However, there are still some unanswered questions, primarily pertaining to reranking and the RDKit oracle.

(1) Although RIGS can improve control with little change in Hit@100, this can also theoretically be done with a reranking. Specifically, reranking can’t affect coverage, but RIGS doesn’t seem to affect coverage either. Choosing to steer at generation time when coverage isn’t affected is still unclear, unless RIGS does substantially better than reranking, but this is also unclear.

(2/3) Reranking is, as you say, also a challenging problem. However, it doesn’t seem to be fairly compared to. There is still the huge discrepancy between the RDKit oracle going from 64.80 for Hit@100 to 26.99 for P@1, the magnitude of which remains unaddressed. Although enumerating through all precursor sets is impractical, it is by itself concerning that the oracle goes from 64.80 to 26.99, when an appropriately matched objective and evaluation should have Hit@100 be similar to P@1. This suggests a deep mismatch between the preferred set evaluation and RDKit property objectives. Additionally, the scalar head reranking reported in (Table 3) is trained off of RDKit, not the precursor sets. So, even with perfect optimization of the RDKit properties, this method of reranking achieves 26.99 with the oracle, which is lower than the RIGS reporting, purely because of the evaluation mismatch. Notably, the reranking decreases RDKit regret substantially, unlike RIGS, indicating that the reranking does work if evaluated on the metric it’s trained on. In essence, reranking is much less aligned to the evaluated metric than RIGS and doesn’t constitute a fair comparison of RIGS with reranking.

(4) Although Pistachio is a different reaction set, how much different is it than USPTO? Do they both cover similar classes of reactions, and are the precursor sets still applicable? The performance on Pistachio is substantially worse, to the point of barely improving P@1 at all. Can you quantify the overlap, or otherwise identify why the generalization is so poor, especially considering that RIGS is trained on Pistachio data as well?

I choose to maintain my rating.

Thank you for your detailed follow-up and for clarifying your remaining concerns. We address these points below with additional analyses and explanations.

(1) Generation-time control beyond Hit@100

RIGS changes instruction-conditioned proposal probabilities. Hit@100 only records whether a preferred candidate appears in 100 draws, so similar scores can coexist with different probability allocations. Table 1 reproduces central quantiles from the frequency-ranked Top-10 table in Section 2 of our first-round response, providing direct-property evidence beyond preferred-set membership.

Table 1. Direct-property quantiles of frequency-ranked Top-10 candidates.

We directly scored up to ten distinct valid, frequency-ranked precursor sets for each of 963 paired products, including outputs outside the preferred inventory. Each product was scored on its assigned property and guidance direction and contributed equal total weight, shared equally among its candidates. Entries are Q25 / Q50 (median) / Q75; arrows indicate the requested direction.

\hline \text{Property and direction} & \text{Reference} & \text{RIGS} \\\\ \hline \text{Procurement burden}\\ \downarrow & 10.00\\,/\\,10.69\\,/\\,13.00 & 10.00\\,/\\,10.69\\,/\\,13.18 \\\\[3pt] \text{Procurement burden}\\ \uparrow & 10.00\\,/\\,10.69\\,/\\,12.30 & 10.00\\,/\\,10.69\\,/\\,13.04 \\\\[3pt] \text{Complexity difference}\\ \downarrow & 0.000\\,/\\,0.416\\,/\\,0.872 & -0.138\\,/\\,0.325\\,/\\,0.794 \\\\[3pt] \text{Complexity difference}\\ \uparrow & -0.005\\,/\\,0.412\\,/\\,0.888 & 0.030\\,/\\,0.421\\,/\\,0.926 \\\\[3pt] \text{Halogen atom count}\\ \downarrow & 0\\,/\\,1\\,/\\,2 & 0\\,/\\,0\\,/\\,1 \\\\[3pt] \text{Halogen atom count}\\ \uparrow & 0\\,/\\,1\\,/\\,2 & 1\\,/\\,1\\,/\\,2 \\\\[3pt] \text{Ring count}\\ \downarrow & 1\\,/\\,2\\,/\\,3 & 1\\,/\\,2\\,/\\,3 \\\\[3pt] \text{Ring count}\\ \uparrow & 1\\,/\\,2\\,/\\,2 & 1\\,/\\,2\\,/\\,3 \\\\ \hline \end{array}$$ Under lower-complexity and lower-halogen guidance, the Top-10 medians decrease from 0.416 to 0.325 and from 1 to 0, respectively. These changes demonstrate instruction-aligned shifts across the returned alternatives, evaluated directly on their properties rather than solely through preferred-set membership. These probabilities matter when a planner repeatedly requests candidate expansions under a finite sampling budget. For example, a region with 1% probability under the unguided generator has a 63.4% chance of appearing in 100 independent draws. Even a perfect reranker cannot recover it from a pool in which it never appears. If instruction conditioning raises that probability to 20%, just ten independent draws give an 89.3% chance of reaching the region. Our setting addresses a practical need in multistep synthesis planning: a single-step generator should respond to route-specific requirements expressed in free-form language. RIGS learns how these instructions favor or discourage precursor alternatives for the same product and transfers this grounding into the proposal distribution itself. This provides a trainable single-step component for more selective multistep search, with the potential to reduce the effective branching factor under limited sampling budgets.
Reviewer_u348OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper construct a benchmark named USPTO-MR, where every product have multiple solution. Based on this benchmark, the author trained the a language guided flow matching for single step retrosynthesis. The multi-solution dataset successfully achieve the goal of evaluating whether the model can correctly predicts the reactant candidate while following the language instructions.

Critical strengths
  • Using the language to guide the single-step retrosynthesis prediction shows great practical value, and shows great potential in the LLM-guided or agentic multiple step retrosynthesis planning when the complex constraints are proposed.
  • The presentation is clear, the paper is easy to follow.
Critical weaknesses

The paper's evaluation have some problem, see the questions for details.

Questions for authors
  • The paper follows the evaluation of GraphDit. I already communicated with the authors of GraphDit and confirmed that the the evaluation protocol of the GraphDit only checks the topological correctness while ignoring the stereo information and the formal charges. However, using stereochemistry-aware canonical SMILES exact matching for evaluation are the common agreement for single-step retrosynthesis prediction[1, 2, 3]. Since approximately 20% of USPTO-Full reactions involve stereochemistry, ignoring it during evaluation may substantially inflate the reported performance making the experimental results a weak evidence for support the contributions. So
    • could the authors clarify whether stereochemistry is considered during evaluation and if so, how it is recovered from stereochemistry-free graph predictions?
    • Or the authors should provide results under a consistent stereochemistry-aware evaluation protocol.
  • Given the inconsistent evaluation protocol, the GraphDit might not be the SOTA guidance-free model, The authors should provide the performance of strong baselines like RSmiles[1] too, as the guidance-free reference under the stereochemistry-aware canonical smiles evaluation protocol. If the improvement on the P@K of the proposed method do not shows the clear improvement on the P@K, the practical value of this paper will be largely weakened, as we can use a post-filtering or ranking to get the preferred answer.

[1] Zhong, Zipeng, et al. "Root-aligned SMILES: a tight representation for chemical reaction prediction." Chemical Science 13.31 (2022): 9023-9034.

[2] Han, Yuqiang, et al. "Retrosynthesis prediction with an iterative string editing model." Nature Communications 15.1 (2024): 6404.

[3] Zeng, Kaipeng, et al. "Ualign: pushing the limit of template-free retrosynthesis prediction with unsupervised SMILES alignment: K. Zeng et al." Journal of Cheminformatics 16.1 (2024): 80.

3 comments
AuthorsOpenReview ↗

Stereochemistry-Aware Evaluation, Backbone Transfer, and Multistep Planning

We appreciate the reviewer’s recognition of the practical value of instruction-guided retrosynthesis and its potential for multistep planning. We clarify the strict evaluation protocol below and provide an additional experiment using R-SMILES.

1. Stereochemistry and formal charges in evaluation

The main USPTO results use the same canonical isomeric SMILES exact matcher (isomericSmiles=True) for Reference, Matched, and Shuffled, preserving stereochemistry and formal charges. The current graph decoder leaves stereochemical configurations unspecified. Consequently, a prediction with correct connectivity but missing a configuration specified by the reference fails the strict match.

Although approximately 20% of USPTO-Full reactions contain stereochemistry, this does not mean that all of them involve stereochemical changes. In the USPTO-50k test set, Zhong et al. [3] identified only 157 reactions (approximately 3%) annotated with such changes and manually reviewed them, finding incorrect reference stereochemistry in 51.6% of these cases.

2. A stronger sequence-model reference: R-SMILES

We trained R-SMILES [1] from scratch on our USPTO-derived Top30 routes, then trained the transferred Stage-A projector and guidance adapters with the backbone frozen. Reference is unguided; Matched and Shuffled share parameters and differ only in instruction.

All use 2,048 test groups with R-SMILES ranking and no property filtering/reranking. P@k/N@k are preferred/avoided any-hit rates among the first k distinct valid precursor sets, macro-averaged over four axes (%; P↑, N↓).

Condition  P@1    P@3    P@5    P@10   N@1    N@10
Reference  35.55  63.33  75.05  88.77  35.55  88.77
Matched    59.72  82.18  89.36  95.41  9.91   69.19
Shuffled   33.59  60.21  73.14  86.23  30.96  84.86

Matched raises P@1 by 24.17 points and P@10 by 6.64, and reduces N@1 by 25.63 versus Reference. Its advantage over Shuffled demonstrates instruction-dependent control.

Direct property distributions of Top-10 candidates

Using the same four evaluators described under Direct property evaluation in our response to Reviewer hteW, we directly score procurement burden, synthetic complexity difference, halogen count, and ring count on the same groups.

Top-10 quantiles. Quantiles pool the first up to ten distinct valid precursor sets in native rank order. Entries are Q25/Q50/Q75; arrows indicate the requested direction.

Metric              Reference             Matched               Shuffled
Procurement ↓       10.000/10.693/11.792  5.011/10.000/10.693   10.000/10.693/11.946
Procurement ↑       10.000/10.693/11.792  10.693/10.693/14.554  10.000/10.693/11.946
Complexity delta ↓  0.056/0.514/1.008     -0.351/0.019/0.524    0.091/0.526/1.006
Complexity delta ↑  0.056/0.514/1.008     0.319/0.694/1.137     0.121/0.569/1.023
Halogen count ↓     0/1/2                 0/0/1                 0/1/2
Halogen count ↑     0/1/2                 1/1/2                 0/1/2
Ring count ↓        1/2/3                 1/2/2                 1/2/3
Ring count ↑        1/2/3                 1/2/3                 1/2/3

Lower-complexity guidance reduces the Top-10 median from 0.514 to 0.019; lower-halogen guidance reduces it from 1 to 0. Lower-procurement guidance reduces Q25 from 10.000 to 5.011, showing property shifts across returned alternatives. Preferred Hit@100 rises 97.66%→99.17%. These results demonstrate that instruction-dependent gains extend to a sequence backbone with broad candidate coverage.

3. Value for multistep planning

RIGS can make multistep retrosynthetic search more selective. Single-step predictions supply expansions to planners such as Retro* [2]. Reallocating probability toward constraint-compatible precursors offers a way to reduce the effective branching factor while retaining access to viable routes. Local reductions can compound: a simplified search tree with branching factor bb and depth dd has bdb^d candidate paths. Reducing branching by 20% per decision leaves 0.810≈10.7%0.8^{10}\approx 10.7\% of the paths after ten decisions, an 89.3% reduction. This illustrates the potential when a planner uses the guided distribution to focus expansions. Table 20 and Appendix F.3 provide probability-mass analysis and multistep examples supporting this direction.

References

[1] Zhong, Z., et al. (2022). Root-aligned SMILES: a tight representation for chemical reaction prediction. Chemical Science, 13, 9023–9034.

[2] Chen, B., Li, C., Dai, H., & Song, L. (2020). Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search. ICML, PMLR 119, 1608–1616.

[3] Zhong, W., Yang, Z., & Chen, C. Y.-C. (2023). Retrosynthesis prediction using an end-to-end graph generative architecture for molecular graph editing. Nature Communications, 14, 3009.

The fact that the dataset contains only a small amount of stereochemistry-related data does not mean that such data are unimportant. Handling chemical stereochemical properties is an important part. Compared with previous models, the model is completely unable to handle stereochemistry-related data at all. This makes the proposed algorithm entirely lacking in realistic application value. Even if your benchmark scores are pushed ever higher, AI4Science is ultimately a field in which practical value must be examined; therefore, I will retain my original score.

Thank you for emphasizing the importance of stereochemistry. We agree that its significance should not be judged by its frequency in a benchmark.

But we would want to clarify that this work is tackling a completely different problem: our work is focused on two connected questions: how to broaden the generator's coverage of precursor alternatives, and how to make generation responsive to users' preferences among them. Our contribution thus concerns both the range of alternatives a generator can propose and the user's ability to guide those proposals. While existing work has extensively studied the recovery of recorded reactions, learning to ground free-form instructions in preferences among precursor alternatives for the same product and transfer this grounding into direct single-step generation remains less explored. We believe this also addresses a significant problem in parallel with stereochemistry.

Generation-time control addresses a limitation that remains even with an ideal selector: it can only choose among alternatives already in the candidate pool. If instruction-aligned alternatives have low probability under an unguided generator, finite sampling may miss them. RIGS makes the proposal distribution responsive to the instruction, allowing user preferences to shape which candidates are generated. RIGS complements selection by shaping the candidate pool. Our comparison using the same LLM selector supports the value of this control.

RIGS also offers an alternative to inference-time guidance through an auxiliary property classifier. It learns instruction-dependent control through grounding and adapters, without requiring a separate classifier to score candidate continuations during decoding. This provides another way to incorporate language preferences into the generation process.

The R-SMILES experiments show that our control approach also applies to a model that supports stereochemical prediction. On this backbone, the grounding and steering approach improves preferred-set P@1\mathrm{P}@1 from 35.55% to 59.72%, compared with 33.59% under shuffled instructions. These results support the contribution of learned instruction grounding beyond the original graph model. The original decoder's lack of stereochemical prediction is therefore not an inherent constraint of the control framework.

Our results demonstrate that the control framework makes precursor generation responsive to user preferences across the two evaluated backbones, including R-SMILES, which supports stereochemical prediction. This addresses a practical need in synthesis planning: directing candidate generation toward users' objectives under a finite sampling budget. We will address this concern more fully in the revised paper by clarifying the stereochemical capabilities and limitations of each backbone and revising our discussion of practical applicability accordingly. We appreciate your emphasis on practical relevance and recognize the importance of further evaluating stereochemical accuracy, experimental executability, and planning efficiency in the intended applications.

We hope these clarifications help convey the scope and practical value of our contribution. Thank you again for your feedback and consideration.

Reviewer_hteWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies instruction-conditioned single-step retrosynthesis, aiming to favor or avoid precursor alternatives according to a user's request. It proposes RIGS, which first grounds language in candidate preferences and then transfers the learned projector to residual adapters on a frozen graph generator. Experiments examine how training support affects candidate coverage and instruction control, with additional tests on structural instructions and recorded reaction alternatives.

Critical strengths
  1. The technical writing is generally clear, and the distinction between candidate coverage and instruction-dependent control is useful.

  2. In terms of technical performance, RIGS provides a concrete way to transfer preferences learned from offline candidate comparisons into direct graph generation. The Stage-A ablation identifies a specific benefit in suppressing conflicting alternatives, while appropriately leaving its promotion benefit unresolved.

Critical weaknesses
  1. Scope and reliability of language guidance. The introduction motivates target-specific preferences, and the structural-instruction experiments provide validation evidence. However, the experiments do not directly establish robustness to paraphrases or ambiguous wording, and the structural preference labels lack comprehensive independent expert validation. How sensitive is the model to paraphrases or ambiguous wording? Since the structural labels rely on LLM judgments, a robustness analysis with domain experts' input would help establish whether the reported alignment reflects reliable interpretation of chemists' requests.

  2. Incremental contribution and comparison with simpler alternatives. Stage A helps suppress unwanted alternatives, but its additional benefit for recovering preferred alternatives is not clearly established. Recovering more candidates labeled as preferred does not necessarily mean that the outputs better satisfy the requested molecular property. Also, how does RIGS compare with a frozen retrosynthesis generator followed by training-free filtering or reranking with a pretrained LLM? A comparison at a comparable generation budget, including selection cost and chemical feasibility, would clarify when the additional training is worthwhile.

Questions for authors
  • Is there concrete evidence of robustness to paraphrases or ambiguous instructions, or an independent expert audit of the structural preference labels?

  • For case-specific structural instructions, what advantage does generation-time steering provide over a frozen generator followed by a pretrained LLM selector, considering instruction satisfaction, computational cost, and chemical feasibility?

1 comment
AuthorsOpenReview ↗

Instruction Robustness and Complementarity with LLM Selection

We thank the reviewer for recognizing the distinction between coverage and control. Our additional experiments examine instruction robustness, Stage A, and complementarity with LLM selection.

1. Robustness to paraphrases

On 982 USPTO Metric4 products, we tested six paraphrases per instruction: T1–T4 were eligible for task training; H1–H2 were excluded. For 8M, P@1 differs from canonical wording by at most 0.87 percentage points. Preferred-Top1 outcome agreement across paraphrases is 90.95% for 8M and 86.17% for 65M. Scores below are percentages.

Wording    8M P@1  N@10   65M P@1  N@10
Canonical  30.17   51.20  40.12    34.93
T1         29.74   49.90  41.22    34.83
T2         29.76   52.27  40.86    35.36
T3         30.04   51.35  39.87    34.22
T4         30.45   49.90  40.55    34.70
H1         30.24   50.41  38.85    35.46
H2         29.30   53.21  38.11    45.98

2. The contribution of Stage A

Stage A grounds the language projector in favored and conflicting alternatives before transfer to generation. In the 8M Top30 ablation, it improves avoided-route suppression at Top-10 by 6.82 percentage points under difference-in-differences analysis (Table 15), helping the generator reject instruction-incompatible routes.

3. Instruction satisfaction and LLM selection

For case-specific structural instructions, 65M RIGS improves case-balanced P@1 by 8.09 percentage points and reduces N@10 by 18.44 points relative to Shuffled (Table 19).

Setup. We compare four configurations on the same 200 distinct USPTO Top30 Metric4 products. Reference is unguided GraphDiT-8M; RIGS adds generation-time guidance. Both rank by generation frequency. Their +LLM variants rerank the corresponding unchanged candidate pools. DeepInfer’s deepseek-v4.1-flash receives the target product, instruction, and candidate SMILES, scoring each candidate from 0 to 100 for instruction satisfaction before Top-k selection.

Direct property evaluation. We measure procurement burden [1], synthetic complexity difference [2], halogen count, and ring count [3]. These assess property improvements beyond membership in the offline Top100 inventory. Entries are mean property values of returned precursor sets; arrows indicate the requested direction, and * marks the best mean.

Metric                   Ref    Ref+LLM  RIGS   RIGS+LLM
Procurement burden ↓     11.16  9.86     11.31  8.93*
Procurement burden ↑     11.72  15.23*   11.94  14.82
Complexity difference ↓  0.30   0.56     0.06*  0.36
Complexity difference ↑  0.26   0.87     0.35   0.91*
Halogen count ↓          1.12   0.54*    0.62   0.54*
Halogen count ↑          1.29   2.75     1.42   3.17*
Ring count ↓             2.21   1.58*    2.17   1.58*
Ring count ↑             2.08   3.04     2.21   3.38*

Instruction Satisfaction. RIGS and a pretrained LLM selector provide complementary forms of instruction following. RIGS shifts the precursor proposal distribution toward the requested structural preferences, while the LLM evaluates completed candidates. Our supplementary comparison across four property axes illustrates this complementarity. Relative to Reference + LLM, RIGS + LLM lowers mean procurement burden from 9.86 to 8.93 when lower burden is requested, and increases mean halogen count from 2.75 to 3.17 when more halogens are requested.

Computational cost. RIGS requires upfront training and adapter computation, while cached instruction features can be reused across sampling and planning expansions. A pretrained LLM selector avoids task-specific training but adds an evaluation pass over each candidate pool.

Chemical feasibility. Appendix B.4 reports round-trip Top-1 of 63.0% for Reference and 71.0% for RIGS on 200 test groups. This evaluates reference agreement or forward consistency, while forward-model coverage and bias affect its interpretation [4]. Separately, we assess all four saved rankings with the same ReactionT5 checkpoint. A candidate passes if any of the five predicted products matches the canonical target. RT@k is the percentage of products with at least one passing candidate among the first k:

Method           RT@1 (%)  RT@10 (%)
Reference        60.5      76.0
Reference + LLM  45.5      73.0
RIGS             62.0      75.5
RIGS + LLM       45.0      71.5

References

[1] Tu, Z., et al. (2025). ASKCOS: Open-Source, Data-Driven Synthesis Planning. Accounts of Chemical Research, 58, 1764–1775.

[2] Coley, C. W., Rogers, L., Green, W. H., & Jensen, K. F. (2018). SCScore: Synthetic Complexity Learned from a Reaction Corpus. Journal of Chemical Information and Modeling, 58, 252–261.

[3] RDKit. (n.d.). RDKit: Open-source cheminformatics.

[4] Torren-Peraire, P., et al. (2024). Models Matter: the impact of single-step retrosynthesis on synthesis planning. Digital Discovery, 3, 558–572.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.