acceptodds
Under review as a conference paper at ICLR 2027

Before the Furnace: Training LLM Agents for Inorganic Retrosynthesis

15 views

Abstract

Inorganic retrosynthesis connects materials discovery with synthesis planning by identifying precursor sets for a target composition. Effective recommendations require models to acquire evidence beyond composition and use it to assess and revise precursor candidates. We introduce PreWeave, an LLM agent trained for inorganic precursor recommendation through tool interaction. We train its interaction policy through supervised fine-tuning on distilled trajectories followed by agentic reinforcement learning. The policy learns to select tools and use their feedback to guide subsequent precursor decisions. Agentic training improves precursor-set Pass@1 accuracy from 53.81% for the base agent to 73.02%. PreWeave achieves competitive performance with task-specific models and tool-augmented frontier LLMs. Further analyses show that training makes reference-matching recommen- dations more consistent across runs and shifts tool use toward candidate construction, with retrieval contributing most to accuracy among the available tools. We will release tools, interaction trajectories, and training and evaluation code to support further research on agent policy training for materials synthesis.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Inorganic retrosynthesis, LLM agents, Material discovery

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_it7EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents PreWeave, a Qwen3.5-9B agent for inorganic precursor recommendation with four tools: retrieval of similar recipes and a conditional precursor completion model (both from He et al. 2023), plus valence and element-coverage checks. The agent is trained by SFT on teacher trajectories that match a reference answer, followed by online RL (GSPO) with an exact-match reward. On the He et al. benchmark (2,654 test targets), Pass@1 goes from 53.81 (untrained agent) to 71.89 (SFT) to 73.02 (SFT+RL); answer-only SFT gives 68.88, and MSP-LLM on the same backbone gives Hit@1 73.89. Analyses show that tools add 3.6 points, that gains saturate at four calls, and that training mainly makes answers consistent across runs while coverage stays flat.

Critical strengths
  1. The experiments have the right controls: an untrained agent with tools, answer-only SFT without tools, the same 9B backbone under a task-specific method (MSP-LLM), and frontier LLMs with the identical tools. The appendix is complete (sampled MSP-LLM, a fixed-tool-order control, per-target gain and loss counts, RL dynamics, dataset accounting and costs).

  2. The pipeline is specified well enough to reproduce, on a public benchmark, with tools, trajectories and code to be released. The analysis is candid even where it undercuts the story: tools add little, the chemical checks do not help, and training does not expand coverage.

Critical weaknesses
  1. The headline 53.81 to 73.02 is against an untrained agent. Against answer-only SFT the gain is 4.1 points at Pass@1 and 1.9 at Pass@8, and that control was decoded with settings tuned for Pass@4, so the matched gap is probably 2-3 points. MSP-LLM on the same backbone, with no tools, no teacher and no RL, has Hit@1 73.89 vs Pass@1 73.02 and Hit@8 85.34 vs 78.33; under sampling (Table 7) PreWeave leads by 0.64 points, i.e. 17 targets. RL adds 1.13 points on a single run. Pass@1 is the first run only (App. B.1), and no standard error or paired test appears anywhere although eight runs exist. With N=2654N = 2654 one standard error is about 0.9 points, so most of the differences discussed in Sec. 4-5 (the RL gain, the tool ablations of 0.1-0.2 points, the budget effects) is inside noise.

  2. Tools add 3.58 points at inference, about half from a single retrieval call; the SFT agent with tools removed (68.31) is below answer-only SFT (68.88), so trajectory supervision by itself gives nothing; removing either chemical check slightly raises Pass@1. The consistency result (8-of-8 hits from 14% to 66%) comes with an unchanged zero-hit set (578, 577, 575 targets), so training sharpens the output distribution rather than solving new targets; a low-temperature or majority-vote evaluation of the base agent, which costs nothing extra, would show how much of the headline gain is sharpening. Finally, the recipe (distilled SFT, then group-relative RL with an outcome reward) is the standard one from Search-R1, ToolRL and ReTool, none of which is cited, and there is no RL-only run. The real contribution is the environment, the data and the characterization, and the framing should say so.

Questions for authors
  1. Please report mean ±\pm SD over the eight runs for every row in Table 2 Panel B and Table 3, and a paired test for SFT vs SFT+RL.
  2. What does the base agent reach with greedy or low-temperature decoding, or with majority vote over its eight runs?
  3. How often does the final answer equal the top retrieved recipe or the top MPC suggestion?
  4. MSP-LLM (Qwen3.5-9B) is your own re-training on this split; please give its training details, and say why SynReason is cited but not compared.
Reviewer_LvJMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces PreWeave, an LLM agent for inorganic precursor recommendation from a target composition. The agent integrates four tools: Retrieval, conditional MPC, Valence analysis, and Element coverage. On a 512 formula target subset, PreWeave shows performance comparable to the tool-augmented frontier LLMs.

Critical strengths
  • Use agent to similuate the synthesis planning process obtained a relatively good performance.
  • The presentation is clear, the paper is easy to follow.
Critical weaknesses
  • According to the ablation, MPC, Valence analysis, and Element coverage show a little improvement to the pass rate, and removing Valence analysis, which is presented as an important chemical check, even improves the performance. Moreover, the paper does not introduce a new dataset or provide experimental synthesis validation, the overall contribution and novelty is limited.
  • Agent design is evaluated with only four predefined tools. As the authors discuss in the related work, scientific agents can interact with a broader range of models and tools, especially the precursor Retrieval and MPC tool are from the same paper. It is unclear how this framework can incorporate other precursor predictors or tools for reaction conditions.
  • Authors use the element coverage case to help check chemical valid, but the proposed NH4H2PO4 introduces other elements: N and H. Though the authors explicitly permits C, H, and N as potential byproduct elements. These rules may be overly restrictive for less conventional precursor choices, and may affect the performance on OOD cases and open-vocabulary design.
  • The comparison on the 512-target subset does not show outstanding performance compared to current LLMs. Given that online RL scales up to 128 H200 GPUs, I am not convinced that the additional training cost is justified by the reported gains. The benefit may also become less compelling if compare to the latest LLMs. More convincing evidence would include systematic evaluation on precursors outside the MPC vocabulary, or broader synthesis tasks involving temperature, reaction time, and other conditions.
Questions for authors

See weaknesses.

Reviewer_HSnWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce PreWeave, an LLM agent for inorganic precursor recommendation. Given a target composition, a Qwen3.5-9B policy calls four tools over several turns, then outputs a precursor set. The tools are a retrieval tool, a conditional precursor model (both the released models of He et al. 2023), a valence check, and an element-coverage check (both pymatgen with rules). The policy is trained by SFT on DeepSeek V4 Flash trajectories, filtered to those matching the reference, followed by 200 steps of GSPO with an exact-match reward and a protocol-violation penalty. On the He et al. 2023 benchmark, Pass@1 rises from 53.81% (untrained agent) to 73.02%. The paper reports parity with tool-augmented frontier LLMs on a 512-target subset, plus analyses of tool ablations, call budgets, and tool-use patterns across training stages.

Critical strengths

The architecture is ablated extensively over tool access, call budget, model-selected versus fixed tool order, length recovery, and training stage, and the comparison with an answer-only SFT is convincing. These ablations make it possible to attribute the gains to specific components, and the weaknesses below rely on them. The authors make an effort to evaluate beyond headline accuracy, discussing the 0 / 1–7 / 8-of-8 hit breakdown and the per-target gain/loss counts between stages, thus evaluating consistency and coverage.

Critical weaknesses

This paper should be rejected for the following main reasons. I am open to increasing the score provided the authors respond to these concerns and to the more detailed points under Questions.

Limited contribution, both methodologically and for materials synthesis.

The method is an unmodified application of existing agentic SFT+RL recipes. Trained agentic retrosynthesis (Retro-R1, organic) and RL-trained inorganic precursor prediction (SynReason) both precede it. I do not see which parts of the methodology are novel (multi-turn tool-calling agents, SFT on rejection-sampled teacher trajectories with observation masking, group-relative outcome-reward RL with trajectory-level advantage and a KL penalty to the SFT policy, difficulty-filtered target sampling). The tools are the He et al. 2023 models and pymatgen, the benchmark and metric are those of He et al. 2023, and the GSPO-specific machinery reduces to REINFORCE with a group-mean baseline + KL, because one optimizer step per fresh batch makes the importance ratio 1 at gradient evaluation, so clipping never engages (A.4).

The task also has unfortunately limited value for materials synthesis. It selects precursors without synthesis conditions, ignores thermodynamics and kinetics, and scores against a ground truth whose identity depends on synthesis conditions the benchmark discards. For a materials scientist, precursor choice is not separable from temperature, atmosphere, dwell time, and processing route. The same target is often made from different precursor sets under different conditions, and the same precursor set succeeds or fails depending on conditions. The paper does not discuss this dependence. This also makes the “ground truth” labels ambiguous, as a reported precursor set is only correct relative to the conditions it was used with. Finally, “retrosynthesis” here is single-step precursor selection, not recursive route search, so the comparison to organic retrosynthesis methods (RetroChimera, Retro-R1, RetroAgent, R3) is not robust.

The empirical results do not support the main claims.

Gains over the appropriate baselines are small, come from single training runs, and sit below the coverage of the agent’s own underlying tool. The headline improvement (53.81 to 73.02) is measured from an untrained base agent. Against answer-only SFT (68.88), the gain is 4.1 points, of which agentic SFT accounts for about 3.0 and RL for 1.13; removing tools from the SFT agent at inference costs 3.58 (Table 3). Pass@1 seems to tie MSP-LLM. At k=8, the agent reaches 78.33 Pass@8, but the He et al. 2023 ranked list, which is the model the agent calls as a tool, reaches 84.74 Hit@8. Therefore, the agent produces less coverage than its own tool’s top-k. The RL gain (+1.13) is also close to the gain from conditional length recovery alone (+0.87, Table 8). On N=512, the standard error at 73% accuracy is about 2 points. The frontier block spans 68.55–74.22 Pass@1, so the results do not separate PreWeave from the frontier models. The teacher, DeepSeek V4 Flash, is also one of the comparators, so parity with that model class is expected.

The central claim, that the policy learns to revise precursor decisions in response to tool feedback, is contradicted by the paper’s own behavioral analysis. In Figures 1–2 and the introduction, PreWeave is framed as proposing, assessing, and revising candidates based on tool feedback. However, cross-round set switches fall from 58.9% (base agent) to 3.4% (SFT) and 3.9% (RL), same-set rechecks fall from 58.3% to 0.1% and 0.3%, and element coverage is followed by End in 81% of transitions after training (Fig. 6). Therefore, the “revision” is base-agent behavior, and the case studies (Section 5.4, C.4) might be hand-picked instances of what training removes.

Questions for authors

Clarifications

Fig. 7 and C.5 describe the reported run as the “first trial” of online RL, and A.5 states the run scaled from 64 to 128 GPUs partway through. How many RL trials were run, was the reported one selected among them, and did the resize involve a restart? The RL prompt seems to add an MPC-vocabulary clarification that the teacher demonstrations don’t have (A.4), so part of the SFT vs. RL difference may come from the prompt. Which prompt produced the reported SFT number? Teacher trajectories were generated with up to 16,384 tokens per reply and up to 10 calls / 12 replies during recovery. The student is evaluated at 5,120 tokens per generation and 8 calls, so it is trained on trajectories it cannot reproduce at test time. Please explain how this mismatch affects the results. PreWeave runs in its native harness while frontier models run in a smolagents ToolCallingAgent with different token limits and finalization logic. The “same tools and prompt” claim does not cover the runtime. Please clarify these differences and their effect on the frontier comparison.

Requested analyses

The tool ablation removes tools at inference time from a policy trained with them, so the no-tools score (68.31) is a distribution-shifted policy, not a policy trained without tools. The ablation also uses the SFT checkpoint, not the reported RL model. Please report the comparison against a policy trained without tools, and the ablation on the RL model. Compute is not matched: the agent generates several turns of up to ~5k tokens each, while answer-only SFT generates 256 tokens. Please account for this in the comparison. Given that the RL gain (+1.13) is close to the gain from conditional length recovery alone (+0.87), please elucidate the benefits of RL post-training beyond performance. All results come from single training runs. Please report variance across runs for the main comparisons.

Editorial (no response needed)

The SynReason reference is incomplete: no venue or year, a stray affiliation marker (“Pengfei Cai1”), and a broken hyphenation (“reason-ing”). “Retrieval tools has the largest individual effect” should be “have” (Section 1). Section 2.2 pairs the citations in the wrong order: “RETRO-R1 and RetroAgent (Zhu et al., 2026; Liu et al., 2026)”. In the reference list, Liu et al. is Retro-R1 and Zhu et al. is RetroAgent. The Retro-R1 entry lists NeurIPS vol. 38 with year 2026; volume 38 is the 2025 proceedings. Internal names remain in the draft: “chemistry-v2 harness” (B.2), “checkpoint-1313” (C.2, C.3), and references to pilot/smoke runs and imported attempts (A.3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.