- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SOLVENT, a 8B LLM trained on top of R^3-pretrained 8B model with 5 training phases for direct multistep retrosynthesis route generation without the need of a single-step model. SOLVENT outperforms all LLM based methods with 8B parameters. SOLVENT also outperforms conventional search-based methods when comparing LLM call budgets against single-step model call budgets. Additionally, SOLVENT is shown to perform long-horizon reasoning since the performance increases as the feedback turn proceeds.
Critical strengths
-
Competitive Performance SOLVENT achieve better performance compared to other LLM baselines on multiple benchmarks with 8B of parameters. When constraining the number of single-step model calls in search-based baselines the same as the number of LLM calls in SOLVENT, SOLVENT also outperforms conventional search-based algorithms.
-
Comprehensive Experiments SOLVENT conducts detailed ablation experiments with detailed description about the experiment setup in the appendix.
Critical weaknesses
-
Limited Chemical Viability The primary success criterion provides limited evidence of chemical feasibility, as template substructure support and atom-source accounting do not establish that the proposed transformations produce the intended products under compatible conditions. Though additional analysis on route quality judged by ChemSensor, SAScoure and round trip validity is provided in Table 21, it's limited to 400 shared targets and SOLVENT failed to outperform KimiK3 and other proprietary LLMs in terms of round trip validity.
-
Over-claimed Superiority The search-budget comparison uses different accounting units for Solvent and conventional planners (number of LLM calls for SOLVENT or other LLM based methods and number of single-step model calls for conventional planners). Consequently, the reported advantage at equal numerical budgets does not establish superior computational or search efficiency. Moreover, route quality metrics in Table 21 show that SOLVENT provide worse routes compared to other LLMs. While the training stage ablation is provided, the 5-stage training pipeline is still a sign of over-engineering compared to conventional planners with no clear advantage. The "long-horizon reasoning" capability is shown via the increasing performance as the turn goes up, but it's still unclear whether the historical interaction played important role in its reasoning or just added noise to the current iteration.
Questions for authors
- Is SOLVENT really history-dependent, or can SOLVENT really do long-horizon reasoning? Currently it's shown that the performance (in terms of success rate) increases as the turns goes up while maintaining all interactions in the context, what if only the latest turn is kept in the context? How many of prior history are kept due to the context window limitation?
- Could you provide extended ChemSensor, SAScore, Round-trip evaluation to other experiments?
- Can SOLVENT still outperform conventional baselines or other LLM-baselines with single-step models if the compute is the same?
- The design of the current 5-stage training pipeline is not intuitive, could you share more about the design choices and why the starting point ( pretrained Qwen3-8B) performs differently with (0.18% of Initial Backbone in table 3 v.s. 19.03% of R3 in table 1)?
8 comments
Question 2
Could you provide extended ChemSensor, SAScore, Round-trip evaluation to other experiments?
We extended the ChemCensor, SA score, and forward round-trip evaluations to RETRO-R1 and EG-MCTS from Table 2, using the same shared 400-target subset and evaluation configurations as Table 23. The results show complementary strengths across methods.
| Method | Target–leaf ΔSA | SA-reducing steps (%) | RC routes/100 steps | FG routes/100 steps | Stock-complete & route RT@20 (%) |
|---|---|---|---|---|---|
| RETRO-R1 | 0.362 | 67.17 | 14.77 | 6.75 | 10.63 |
| EG-MCTS | 0.560 | 71.80 | 18.12 | 8.08 | 16.01 |
| Solvent | 0.827 | 84.69 | 8.62 | 4.11 | 9.02 |
Solvent achieves the largest target–leaf ΔSA (0.827) and the highest proportion of SA-reducing steps (84.69%). Solvent's template-free generation allows flexible reaction proposals, while verifying chemical feasibility remains a challenge for template-free methods. In this comparison, RETRO-R1 and EG-MCTS obtain higher ChemCensor RC/FG and round-trip scores. We have added the expanded evaluation and discussion in Appendix F.2.4 (Table 25).
Question 3
Can SOLVENT still outperform conventional baselines or other LLM-baselines with single-step models if the compute is the same?
To address the question about computational efficiency, we complemented the search-budget results in Table 2 with runtime measurements of Solvent and RETRO-R1 on all 1,440 targets under matched hardware and inference configurations. As discussed in Appendix C.2, computational effort is difficult to align across LLM generation, single-step model prediction, and tree search.
Solvent has lower mean per-target termination times than RETRO-R1 across all four benchmarks. The table reports mean termination time and mean per-target effective output rate, including tool calls.
| Benchmark | Solvent time (s) | RETRO-R1 time (s) | Solvent effective tokens/s | RETRO-R1 effective tokens/s |
|---|---|---|---|---|
| USPTO-190 | 741.89 | 1,845.82 | 11.45 | 0.77 |
| ChEMBL-1000 | 551.80 | 1,369.59 | 14.24 | 0.77 |
| Pistachio Reachable | 462.76 | 763.40 | 11.44 | 0.78 |
| Pistachio Hard | 718.18 | 1,532.15 | 11.30 | 0.78 |
Solvent's higher token usage is compatible with lower overall planning time. For an LLM planner coupled with a single-step model, the total cost includes LLM generation, single-step prediction, and search. Solvent directly generates and revises complete routes.
Solvent uses more extensive reasoning to improve route planning, naturally requiring additional tokens. It allocates computation to global synthesis strategy, precursor availability, and feedback-guided route revision rather than explicit search over single-step disconnections. This route-level reasoning supports the higher success reported in Table 2 under the stated search budgets. We have added the runtime measurements in Appendix F.2.3 (Table 22) and clarified the scope of budget alignment in Appendix C.2.
Question 4
What motivates the five-stage training pipeline? Why does the Initial Backbone (pretrained Qwen3-8B) achieve 0.18% in Table 3, compared with 19.03% for R3 in Table 1?
We designed our two-round curriculum based on the insight that stable behavior and sufficient exploration capacity are needed for effective RL. Round 1 establishes the foundation for subsequent capability improvement. Warm-up builds retrosynthetic knowledge, teacher SFT introduces multi-turn planning, and RL consolidates the model's own successful behaviors into a stable seed policy, addressing malformed queries and truncated responses. Round 2 builds on this foundation to prepare a stronger starting policy for final-stage RL. We bootstrap the warmed-up policy using successful trajectories from the seed policy and control bootstrapping depth to retain sufficient exploration capacity (Appendix F.1.2, Figure 6). Applying the same final-stage RL directly to the teacher-SFT policy yields substantially lower validation success, supporting the importance of this preparation (Appendix F.1.3, Figure 7). The five stages thus progress from behavioral stabilization to task-performance improvement. We have clarified this rationale in Section 4.
The 0.18% and 19.03% results correspond to different checkpoints from the R3 work [3]. The Initial Backbone in Table 3 (0.18%) is the Qwen3-8B-Base checkpoint after R3's continued pretraining (CPT), which we use to initialize Solvent. The R3 baseline in Table 1 (19.03%) is their final model after the complete post-training procedure. We have clarified these checkpoint identities in Section 5.1 and the Table 3 caption.
[1] Choe, J., Chen, S., and Jung, Y. Assessing the extrapolation capability of template-free retrosynthesis models. Digital Discovery, 2026. https://doi.org/10.1039/D6DD00072J
[2] Kimi Team. Kimi K3: Open Frontier Intelligence. 2026. Section 4.2.2, Figure 9. https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf
[3] Wang et al. R3: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning. ACL 2026. https://aclanthology.org/2026.acl-long.1745/
Response to Reviewer nQj2
We thank the reviewer for the detailed feedback and suggestions for additional evaluation.
Weakness 1 & Weakness 2
Solvent's template-free generation allows joint planning of complete routes and exploration of new substrate combinations, supported by additional chemical checks. Template-based planners constrain reaction proposals through template application. These checks assess chemical plausibility, with compatible reaction conditions requiring further expert and experimental validation. Forward round-trip evaluation also depends on a learned predictor and can produce both false positives and false negatives [1]. We therefore use complementary diagnostics to assess different aspects of route quality.
The route-quality results show a mixed profile, with competitive chemical-support metrics and room to improve forward-model consistency. Table 23 uses the same fixed 400 targets for all models. Solvent has lower round-trip recovery than Kimi K3 (9.02% vs. 13.44%), alongside stronger synthetic-accessibility improvement and ChemCensor reaction-center and functional-group support.
Our budget alignment is designed to compare search efficiency across different planning methods. For Table 2, Solvent's search budget N counts the cumulative number of reaction steps in submitted routes, accounting for the reaction-level work used to find a complete route. We have clarified this definition and comparison scope in Appendix C.2.
Our contribution is a practical, reproducible post-training recipe that develops long-horizon retrosynthetic reasoning in an 8B model to rival frontier LLMs on this task. Kimi K3 is a strong reference model whose post-training task synthesis includes chemistry [2]. The experiments explain how behavior stabilization and controlled bootstrapping depth prepare the model for effective RL. We will release the stage-wise checkpoints and complete training pipeline to help the community reproduce and build on these findings. We have clarified the training rationale in Section 4. Appendix F.2.4 presents the complementary route-quality comparisons.
Question 1
Solvent retains the complete interaction history within its 32k-token context window. We tested the suggested latest-turn-only setting, retaining the fixed prompt and the most recent model response with its corresponding tool feedback. The comparison covers all 1,440 targets with up to five interaction turns per trajectory:
The 6.12 percentage-point drop in macro-average success supports the value of earlier interactions for route revision. We have added this ablation in Appendix F.1.6 (Table 18) and two case studies in Appendix G.4 showing how Solvent uses earlier interaction history to guide route revision.