acceptodds
Under review as a conference paper at ICLR 2027

Steering Autoregressive Retrosynthesis with Calibration

Abstract

Synthesis planning seeks an efficient sequence of chemical reactions building up to a target molecule. Such sequences are typically built by repeatedly invoking a pretrained autoregressive single-step model. Using classifier guidance, we can steer said model toward reactions with desired properties at inference time, without retraining. We show that cross-entropy trains a classifier to estimate a property's probability given the prefix, but to change an overconfident generator's ranking, that estimate must be extremely confident. For substructural properties like reaction type, where a specific, rare combination of tokens decides the label, no realistic corpus gives the classifier enough examples of each combination to reach that confidence. We overcome this issue with a novel method called Calibration for Autoregressive Guidance (CAG), which employs contrastive augmentation and a margin-based loss to calibrate the classifier so that it can meaningfully discriminate between continuations during decoding. Empirically, on USPTO-190 with chemist-specified guidance targets at every node, CAG returns a route meeting all the specified reaction types for of targets, against for an identically trained cross-entropy classifier, and for an unguided model. Guidance also more than doubles the number of distinct routes found per target, from for the unguided baseline to with CAG, outperforming most template-based methods.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
retrosynthesis, steerable synthesis, classifier guidance for an autoregressive model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_27w6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAG (Calibration for Autoregressive Guidance) to guide reaction prediction models towards specific user-specified reaction types during generation. The primary contribution of the paper is the guidance mechanism, validated on USPTO-190 and USPTO-50k.

Critical strengths
  1. The method overall seems to work (e.g. on USPTO-50k the average performance does increase)
  2. The case against classifier-free guidance is fairly well-argued.
Critical weaknesses
  1. Baselines: a few things stood out to me.

    First, one obvious baseline is sampling larger numbers of reactions and filtering by constraint satisfaction. Table 3 seems to show this working fairly well: e.g. the row around line 1872 is a generator with reaction filtering and seems to be the best performing method in the whole table, including CAG. That seems to undercut a bit part of the paper's narrative.

    Second, there are likely other simpler methods which might accomplish similar thing, e.g. stochastic sampling with a scaled temperature (probably followed by filtering).

    So overall it would be good to show that the complexity of CAG is justified.

  2. Reaction validity: a potential consequence of the guidance is producing invalid reactions. Figure 5a seems to show that increasing adherence comes with lower validity. So I'm concerned that the extra "solved reactions" are actually just invalid ones, which wouldn't actually solve the problem.

Questions for authors

From what I wrote above, my biggest questions are:

  1. Does this actually perform better than baselines like reaction filtering
  2. Are the new reactions found by CAG actually valid (sorry if it's in the paper and I missed it)

More generally, I would love to see the writing be improved a bit. In particular I think the arguments in §3.1-3.2 could be shortened and made clearer (giving more high-level intuition).

4 comments
AuthorsOpenReview ↗

New baselines, validity, and improved writing

We thank the reviewer for their insightful suggestions and answer their questions below.

W1. Sampling larger number of reactions

We have run the reviewer's suggestion and report it in a new appendix section "D.3. Sampling more candidates from the generator" (lines 1774-1820, Table 3). We hold the generator, the checkpoint and the data of the main steering experiment fixed and change only the beam width, sweeping omega in {5, 10, 20} over the 640 products of USPTO-190-Steps against all 11 requested classes (7,040 pairs). We vary the beam width because in this generator the requested count is bounded by the beam width and number of augmentations [5].

decodingpooladherenceclass diversityvaliditymean ranktop-10
beam, omega=516.40.0900.3450.3425.981.6
beam, omega=10 (paper)36.40.0890.4950.28310.859.9
beam, omega=2078.70.0880.6320.22818.948.1
CAG, lambda=2—0.1460.6920.22310.965.9

A wider beam enlarges the candidate pool without making any individual candidate more likely to carry the requested class. Per-sample adherence is flat across the sweep (0.090, 0.089, 0.088), while the pool and class diversity grow (from 16.4 to 78., and 0.345 to 0.632 respectively), and the rank of the first adherent samples grows (5.9 to 18.9). A planner that expands only the leading candidates of each call therefore benefits less from the adherent samples at a beam size as high as 20. CAG at lambda = 2 instead raises the per-sample rate itself to 0.146 and reaches a higher class diversity, 0.692, from the omega = 10 pool.

W2. Simpler methods accomplishing similar things (e.g. stochastic sampling with a scaled temperature)

We have also run this baseline: stochastic sampling from the generator with temperature scaling. We report our results in a new appendix section "D.4 Stochastic Sampling with Scaled Temperature" (lines 1821-1861, Table 4). The main takeaways from these results is that changing the temperature scaling in stochastic sampling leads to a wider selection pool, without targeting a specific property value. This is reflected in the adherence, which is essentially equal to that of the beam search sampler (0.09 for all temperatures), while the achievable class diversity (ACD) is higher (from 0.495 for beam search to 0.619 for stochastic sampling with temperature T=2.), matching also an increase in the average candidate pool per (product, requested class) pair from 36 to 53 samples out of the 100 requested. CAG (at lambda=2) outperforms both the unguided beam search and the stochastic sampler with higher temperature with an adherence of 0.146 and an ACD of 0.692, at a comparable round-trip validity (0.223 against 0.250 for T=2.).

W3+Q2 validity of the solved guided reactions

Evaluating the validity (i.e. plausibility) of a computational retrosynthetic prediction is a known challenge in the field [1, 2, 3]. The best proxy (round-trip accuracy, using a forward model to predict the product back from the proposed reactants) is biased for favoring the standard reactant set for each product since it was trained on a single reaction per product. This is why we report, in Fig. 5b, both round-trip accuracy and the rxn-insight name identification. On the reviewer's question on whether guidance recovers valid precursors at all, we point to the result reported in Fig. 5a, which states that out of the 287 products where the unguided baseline finds no round-trip correct samples, CAG finds at least one round-trip valid precursor for 231 of them, in addition to tying with RetroKNN [4], one of the best template-based methods available, on achievable class diversity filtered by round-trip validity.

In the multi-step setting, the reviewer's concern is that the extra targets CAG solves are solved with worse chemistry than the ones the unguided model already reaches. We therefore split CAG's adherent routes into the targets it shares with the unguided model and the targets only CAG solves, and compare both against the unguided model's own adherent routes. All numbers below are per-step rates within adherent solved routes: round-trip is the share of steps whose product appears in the forward model's top-10 predictions, and specific RXN-Insight name the share of steps assigned a name other than OtherReaction. Brackets are 95% intervals obtained by resampling the targets with replacement (10,000 draws).

round-tripspecific RXN-Insight name
unguided's 18 adherent targets0.171 [0.065, 0.316]0.343 [0.183, 0.525]
CAG, the 16 shared targets0.118 [0.045, 0.239]0.194 [0.062, 0.374]
CAG, the 36 new targets0.149 [0.078, 0.239]0.253 [0.155, 0.374]

On both proxies, the targets CAG alone solves come out slightly ahead of the targets it shares with the unguided model, so the extra solves are not degraded.

AuthorsOpenReview ↗

Q1, improving the writing, and references.

Q1. Do we outperform baselines like reaction filtering

Our method complements (positively) search-level mechanisms like hard filtering (F) and value function guidance (G) rather than competing with them. On the USPTO-50k checkpoint, CAG alone matches the filtering row the reviewer points to on adherent solve while returning five times as many adhered routes per target, and adding CAG to filtering raises adherent solve further (Table 5). On the stronger USPTO-Full checkpoint, now reported separately in Table 6, CAG improves every search-level mechanism it combines with on the share of requested steps, and of whole routes whose class matches the request. On this stronger checkpoint, solve rate declines with CAG, since it fails to solve more complex targets within the time budget given that it's a slower method. The row the reviewer cites comes from the USPTO-Full runs of the submitted Table 5: we found a labelling error in the classifier's training data and reran that whole block, which is what Table 6 now reports.

Improved writing

We have rewritten §3.1 and §3.2 around the high-level argument the reviewer asks for, moving the formal statements to the appendix. The rewritten sections now say, in plain terms, (i) that guidance only changes the generator's output if it voices its preference as confidently as the generator does, (ii) that this is hardest for properties decided at a few specific token positions, which we now name substructural and contrast with global properties such as Tanimoto similarity in a new figure (Figure 2), and (iii) that reaching the required confidence on such a property needs a prohibitive number of samples per (product, disconnection) because sharp class boundaries block the interpolation a network would normally rely on. We have additionally added Figures 3 and 4 to the Methods section (Section 3) to illustrate both how the training corpus does not contain the prohibitive number of samples needed to achieve the desired confidence (Figure 3), and the mechanisms of CAG (Figure 4). We have also applied minor edits to the other sections of the paper (Abstract, Intro, Limitations and Discussion, and Conclusion) to better reflect our clarified claims.

Score

If the reviewer finds our answers and updated manuscript satisfactory, we hope they would consider raising their score accordingly. We remain happy to address any further concerns during the discussion period.

References

[1] P. Schwaller, R. Petraglia, V. Zullo, V. H. Nair, R. A. Haeuselmann, R. Pisoni, C. Bekas, A. Iuliano, T. Laino. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy. Chemical Science 11(12), 3316-3325, 2020. (introduces round-trip accuracy, and argues no single metric is sufficient on its own)

[2] F. Hastedt, R. M. Bailey, K. Hellgardt, S. N. Yaliraki, E. A. del Rio Chanona, D. Zhang. Investigating the reliability and interpretability of machine learning frameworks for chemical retrosynthesis. Digital Discovery 3, 1194-1212, 2024. (benchmarks how often single-step predictions are chemically infeasible)

[3] J. Choe, S. Chen, Y. Jung. Assessing the extrapolation capability of template-free retrosynthesis models. Digital Discovery 5(6), 2591, 2026. DOI 10.1039/D6DD00072J. (round-trip scores are bounded by the forward model and can penalise chemically reasonable out-of-distribution predictions; the authors resort to manual inspection to assess plausibility)

[4] S. Xie, R. Yan, J. Guo, Y. Xia, L. Wu, T. Qin. Retrosynthesis Prediction with Local Template Retrieval. AAAI 2023, 5330-5338.

[5] Z. Zhong, J. Song, Z. Feng, T. Liu, L. Jia, S. Yao, M. Wu, T. Hou, M. Song. Root-aligned SMILES: a tight representation for chemical reaction prediction. Chemical Science 13(31), 9023-9034, 2022. DOI 10.1039/d2sc02763a.

Reviewer_27w6OpenReview ↗

Rebuttal acknowledged

Thank you for these changes- I think the new experiments strengthen the work and resolves most of my concerns. My initial score was 3, and although I view the paper more favourably, the small overall effect size means I don't want to raise my score to 4 (the max possible score).

We thank the reviewer for taking the time to go through the revised manuscript, and are glad the new experiments resolved most of their concerns. We understand the reservation about the overall effect size. We remain available to answer any further questions.

Reviewer_18kgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors provide a modification to the classifier guidance approach for guiding autoregressive chemical language models for retrosynthesis planning. Through careful and domain-motivated analysis, the authors justify how traditional classifiers based on the cross-entropy fail for ranking different pathways, and how this is a severe failure mode when considering the substructure-oriented nature of retrosynthesis. They then go on to introduce contrastive autoregressive guidance (CAG), which combines a data augmentation strategy with a modified loss function that contains a margin term. They then demonstrate state of the art performance on the USPTO-190 evaluation set, showing that they can generate reaction routes that are adherent to specified reaction types.

Critical strengths

The key strength of this paper is the diagnostic analysis that the authors provide, proving first that standard classifier guidance does not incentivize ranking among different paths. Their analysis is very theoretically grounded and well-motivated by the inherent constraints of molecular retrosynthesis, and this author found their presentation of the derivations and the structure of the paper to be very logical. Additionally, the diagnostic work in the first half of the paper provides ample justification for both the contrastive data augmentation strategy that the authors employ and the margin-based term appended to the loss function. This author views the theoretical analysis and contribution as the main selling point of this work.

The authors provide convincing results on their evaluation for the USPTO-190 dataset compared to other baselines, and the in-depth case studies and illustrative examples of the reactions (e.g. Figure 4) help reinforce their point and gives the reader a clearer understanding of the problem. The additional ablations of the ranking loss is a nice addition that helps isolate the effects of modifications to the loss functions. This shows thoughtful experiment and analysis design, and further supports the acceptance of the paper.

Critical weaknesses

One weakness is the design of the figures. Figure 1 is difficult to interpret and takes a few passes to understand exactly the structure of the problem and approach, and the caption feels very uninformative. Figure 2 is also poorly formatted and should be more legible/space filling as opposed to its overly sparse design currently.

Another weakness is that theoretical analysis, while sound and logically presented, is very dense and makes it difficult to disentangle the specific algorithmic/data augmentation contributions that are used in the experimental section. It would be good to have a section or some other visual way to highlight the specific objective for optimization.

These issues all relate to the clarity of the paper, and should be addressed.

Questions for authors

Questions for clarification:

  • The abstract cites the following numbers: 27.4% of targets for CAG, 22.6% for cross entropy classifier, and 9.5% for unguided model. However, Figure 3c has the unguided model listed as 0.14, so those numbers should be made consistent
  • Two hyperparameters γ\gamma and μ\mu are set as 1.0 throughout the paper, but they are never swept over or otherwise tuned. Ablations should be presented for these parameters.
  • The junta framing assumes no structure linking token combinations, which is not true for sequence-based representations of molecules such as SMILES strings. Further justification for the junta analysis, and why it is appropriate for the SMILES case, should be provided.
  • The cost of running the classifier guidance method should be more clearly presented in the main text and expanded upon, since the cost of using the classifier is a big limitation.

In terms of future work, all the directions the authors have already provided are reasonable. Additional work should also look at the optimization of global properties.

4 comments
AuthorsOpenReview ↗

Updated figures, improved writing, and a new ablation

We thank the reviewer for their constructive feedback. Below we answer their questions and clarify any confusing details from our work.

W1. Design of the figures

We have redesigned Figure 1, and added new figures 2, 3 and 4 to illustrate the difference between substructural and global properties (2), show how the composition of our corpus does not meet the sample size requirement for learning substructural properties (3), and illustrate the components of CAG (4). We hope the new figures are to the level of clarity the reviewer expects.

W2. Clarity of the paper and theoretical analysis

We have rewritten Sections 3.1 and 3.2 around a single plain-language argument, and moved the formal material to the appendix. The claims the rewritten sections now make, in order, are (i) that guidance only changes the generator's output if it voices its preference as confidently as the generator does, (ii) that this is hardest for properties decided at a few specific token positions, which we now name substructural and contrast with global properties such as Tanimoto similarity in a new figure (Figure 2), and (iii) that reaching the required confidence on such a property needs a prohibitive number of samples per (product, disconnection) because sharp class boundaries block the interpolation a network would normally rely on. We have also applied minor edits to the other sections of the paper (Abstract, Intro, Limitations and Discussion, and Conclusion) to better reflect our clarified claims, and added a few new appendix sections to answer concerns by other reviewers.

Q1. Consistency in the numbers cited in the figure and the abstract

We thank the reviewer for pointing out this inconsistency, which we fixed in the current draft (abstract lines 023-027, and Figure 5c with its caption, lines 332-338; the matching numbers are quoted in the Results paragraph at lines 417-419). The mismatch in numbers came from the fact that Figure 5c reported a multi-step run with 100 single-step model calls, while the abstract quoted numbers from a budget of 10 calls. We changed Figure 5c to match the abstract, with the following numbers: adherent solve 0.095 (95% CI [0.058, 0.142]) unguided against 0.274 [0.211, 0.337] for CAG, and 2.09 [1.80, 2.44] against 4.45 [3.92, 5.02] routes per target, with 95% intervals from resampling the 190 targets. The broader comparison against every baseline and guidance mechanism, at 100 model calls, remains in the appendix table.

Q2. Ablations for the parameters γ\gamma and μ\mu

We have now retrained the USPTO-190 reaction-type with the following values: γ∈{0,1,3,5}\gamma \in \{0, 1, 3, 5\} at μ=1\mu = 1 and the ranking weight μ∈{0.1,1,10}\mu \in \{0.1, 1, 10\} at γ=3\gamma = 3. We re-ran guided generation with each classifier over the 7,040 (product, requested class) pairs formed by the 640 USPTO-190 test products and the 11 reaction classes.

γ\gammaμ\muaccuracyadherenceΔ\Delta adherenceany-adherentround-tripΔ\Delta round-trip
010.2870.108−0.0032±0.0009-0.0032 \pm 0.00090.7340.187+0.016±0.001+0.016 \pm 0.001
110.3540.109−0.0017±0.0009-0.0017 \pm 0.00090.7410.172+0.001±0.001+0.001 \pm 0.001
310.3550.111—0.7460.171—
510.4730.110−0.0011±0.0009-0.0011 \pm 0.00090.7780.115−0.056±0.001-0.056 \pm 0.001
30.10.5140.121+0.0096±0.0010+0.0096 \pm 0.00100.8070.111−0.060±0.001-0.060 \pm 0.001
3100.1690.103−0.0080±0.0014-0.0080 \pm 0.00140.7180.042−0.129±0.002-0.129 \pm 0.002

γ\gamma is the parameter the paper's argument is about, and it behaves as the argument predicts: removing it entirely (γ=0\gamma = 0) costs adherence while gaining round-trip (the classifier stops moving the generator at all). Raising it past 1 buys accuracy and candidate coverage (any-adherent 0.746 →\to 0.778) but no further adherence. μ\mu trades adherence against validity rather than improving both. The most accurate value (μ=0.1\mu = 0.1) gains +0.0096+0.0096 adherence and gives up −0.060-0.060 round-trip, and the least accurate (μ=10\mu = 10) collapses round-trip to 0.042. We therefore keep γ=3\gamma = 3, μ=1\mu = 1 for the ideal tradeoff. All these ablations vary far less than the CAG-vs-CE gap the paper is about, so the method's conclusions do not rest on this parameter choice.

Q3. Junta argument and structure linking token combinations

We thank the reviewer for the opportunity to clarify this point. We use the junta argument as a tool to analyze the worst case scenario, i.e. how many samples an estimator needs to see after it has exploited joint structures. We essentially argue that for substructural properties, the shared structure between token combinations is not what determines the property value (e.g., for reaction types, a single atom value is what differentiates between acylation and C-C coupling), and to bound how many samples are needed to learn these sharp boundaries with enough confidence, we treat the structures as independent, which is the extreme case. We have now made this argument clearer in sections 3.1. and 3.2., and added a measurement in Appendix B.6 that tests whether samples from chemically similar reactions can replace samples of the exact combination. They cannot: when a combination has no samples of its own, adding similar neighbours does not improve the classifier's agreement with a counting estimator.

Q4. Present the cost of running the classifier in the main text

We have moved the information on compute resources from appendix D.8 of the submitted version to section 4.1 (experimental setup, lines 400-403); the full compute appendix is now C.9. The cost of the running the classifier in brief is 4-5 times the inference time when using 72 tokens (the size of the 50k full vocab) in guidance, and 1.08-1.14 times the inference time at the cost of 1.2 percentage points in achievable class diversity compared to the optimal setup.

Score

We hope our answers have address the reviewer's concerns, and that they would consider raising their score accordingly. We remain happy to address any additional questions during the discussion period.

Reviewer_18kgOpenReview ↗

Rebuttal acknowledged

I thank the authors for their response to the initial points in my reviews. While the additional experiments and clarifications enhance the quality of the paper, I will maintain my current score of 3.

AuthorsOpenReview ↗

Thank you for your reply

We thank the reviewer for taking the time to go through our response and the revised manuscript, and are glad the additional experiments and clarifications improved the paper. We remain available to answer any further questions.

Reviewer_9iKJOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

In this paper the author steers a retrosynthesis model toward reaction types a chemist wants, one known example like C–C coupling, without retraining it. A small classifier adjusts the generator’s token choices as it decodes. The authors argue that ordinary training leaves this classifier too weak to overrule a very confident generator, so they train it with wrong-token examples and a ranking loss CAG. On USPTO benchmarks, this hits the requested type more often and finds more routes. It keeps the generator frozen but slows inference, and it is shown mainly for reaction type.

Critical strengths

The paper addresses a practical and important problem by incorporating chemists’ preferences into retrosynthesis planning, such as favoring reliable cross-coupling reactions and avoiding hazardous transformations. This better reflects how routes are selected in real medicinal and process chemistry settings. Another key strength is the plug-and-play design: guidance can be applied selectively during generation without retraining the underlying model, allowing users to switch between guided and unguided modes or optimize for properties not included during training. The approach also goes beyond simply re-ranking existing solutions. By steering generation at the token level, it can discover synthetic routes that baseline methods fail to propose, enabling access to new chemistry and improving coverage on challenging targets.

Critical weaknesses

Only the Schneider reaction classes are tested. The authors partly concede this: the theory is restricted to substructural properties, and the limitations note that on a global property Tanimoto similarity the margin loss gives no gain over cross-entropy. That is an honest negative result, but it means the method’s scope is narrow and leaving-group or functional-group objectives, which the theory claims to cover, are never run.

The gains over the authors’ own cross-entropy classifier are modest (adherent solve 0.274 vs. 0.237, about 7 of 190 targets, without error bars), and on USPTO-190 search-level filtering or value guidance alone outperforms CAG alone.

Questions for authors

The multi-step results rest on 190 targets, so CAG’s edge over the cross-entropy classifier 0.274 vs. 0.237 is about 7 targets. Can you give confidence intervals or multiple seeds? The same applies to the 640-product single-step set and the 40-target misspecification study.

USPTO-190-Steps is described as test products, yet the classifier’s training data is built from “training and validation subsets” of it. How exactly are products split, and is any evaluated product, or its ground-truth reactants, seen in classifier training?

2 comments
AuthorsOpenReview ↗

A new substructural property and the significance of the empirical gains

We thank the reviewer for taking the time to comment on our manuscript. Below we address their concerns and answer their questions.

W1. "the method’s scope is narrow and leaving-group or functional-group objectives, which the theory claims to cover, are never run."

We have now added an additional application based on leaving group choice. The full experiments details and results are in Appendix D.5. We specifically focus on Acylation reactions, for which a free acid or an acyl halide are possible leaving groups (illustrative examples are given in the appendix, Figure 14). The experiment therefore asks: "For an acylation reaction, can we guide the generator towards a free acid instead of an acyl halide (and vice versa)?". We compare a CAG-classifier, a CE-classifier, and the unguided generator. We evaluate on a test corpus made up of extracted acylation reaction + their counterfactual: i.e. we duplicate the precursor set and change only the leaving group, leading to 1,318 evaluation rows (659 products x 2 requested donors) that differ only in which donor is requested, and on exactly half of which the request runs against the donor recorded in USPTO. All runs use the USPTO-50k checkpoint. A snippet of our results is given below (full results in E.5, Table 6):

requested donorarmtop-1 adherenceround-trip
free acidunguided0.2580.509
free acidCE, λ=0.50.3820.436
free acidCAG, λ=0.50.4070.430
acyl halideunguided0.1290.509
acyl halideCE, λ=0.50.3810.478
acyl halideCAG, λ=0.50.5100.472

Paired over products, CAG's lead over CE is significant in both directions (CAG alone correct on 94 products against 9 for CE, p = 6e-19; free acid, 24 against 8, p = 0.007). The lead is largest at small λ and closes by λ = 2, where both experiments saturate and both pay more round-trip accuracy for it, which is what the calibration argument predicts.

W2. "The gains over the authors’ own cross-entropy classifier are modest"

We would like to highlight that the difference in performance between the two classifiers is substantial at the single-step level (where guidance is applied). CAG offers more valid-adherent precursors, and the separation holds at every guidance strength the two classifiers can reach, now with error bars demonstrating the significance of the empirical gain (Figure 7a).

We admit that the benefits of CAG get watered down in the multi-step setting: this is partly due to the fact that most of the guidance effect is carried by search-level guidance mechanisms, and partly because we bound all our search baselines with the same time limit on the single-step model, which gives the slower CAG-steered methods less time to properly explore the harder targets. We have now added new appendix sections to show that CAG still has advantages in multi-step:

  • In D.6 "Steering rare reaction types", we show that the advantage of CAG lies in better handling request to rare reaction classes specifically.
  • In D8. "Search results" (Table 9), we show that CAG improves the route adherence over solved routes when paired with any additional search-level guidance mechanism (G, F), and only fails on coverage (i.e. adherent solve rate), which we attribute to CAG being slower and therefore requiring more time on the harder targets.

Q1. Missing error bars in all experiments

We have added errors bars to all the experiments requested by the reviewer (multi-step, single-step and the misspecification study). The bars are obtained by resampling with replacement (10,000 samples) the products in each evaluation (paired across methods), compute the metrics on each resampling, and reporting the 2.5th and 97.5th percentiles. The results are reported in the updated manuscript, and reported briefly below for completeness:

  • In our single-step experiments, every difference between CAG and CE excludes 0, with separation widening with guidance strength, notably CE's adherence falls below that of unguided (0.387 [0.374, 0.398] vs 0.536) at lambda=10, while it holds at 0.732 [0.722, 0.741] for CAG.
  • In multi-step, the gap is smaller and not significant for the reasons given in W2.
  • In misspecification, the adherent solve rate computed is within tight bounds, confirming the observation that solve rate degrades smoothly with corruption.
AuthorsOpenReview ↗

Q2 and references

Q2. How the data is split for classifier training

We train the classifier on data collected from the training data split of USPTO-190, the one released by Retro*[1]. Specifically, we sample 4,994 products, stratified by reaction type (up to 454 products per type across the 11 classes), out of the 199,104 unique products appearing in those training routes. We generate 15 precursors for each with the unguided model (trained on USPTO-Full with the test-route reactions removed), assign a reaction type to each precursor with Rxn-Insight [2], and produce partial sequences from each precursor. No test reaction is seen in the training data generated this way. We added this information to the new draft under Appendix C.4, "Reaction type partial-sequence data" (lines 1576-1582), which now states the product pool each classifier is built from and how many samples per product it contributes.

Score

We hope our answers have address the reviewer's concerns, and that they would consider raising their score accordingly. We remain happy to address any additional questions during the discussion period.

References

[1] B. Chen, C. Li, H. Dai, L. Song. Retro*: learning retrosynthetic planning with neural guided A* search. ICML 2020, 1608-1616. (source of the USPTO-190 route splits we train and evaluate on)

[2] M. R. Dobbelaere, I. Lengyel, C. V. Stevens, K. M. Van Geem. Rxn-INSIGHT: fast chemical reaction analysis using bond-electron matrices. Journal of Cheminformatics 16(1), 37, 2024. DOI 10.1186/s13321-024-00834-z. (assigns the reaction class and name used as the guided property and as the adherence label)

Comments on the paper

1 comment

Dear reviewers,

As the authors have posted their responses, I would encourage you to actively parcitipate in the dicussion.

Thank you.

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.