- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the MSP problem. It points out that existing methods determin different components in isolation, and limited experimental trial budgets and equipment availability require choosing which of the many possible routes to test. Correspondingly, this paper proposed CondMSP, which extends MSP from recipe prediction to execution-aware planning.
Critical strengths
- The studied problem is very important and interesting.
- The motivation of this paper is clear.
Critical weaknesses
- The background is not clear. The MSP problem is not a common machine learning task, the definition of the background in Introduction is not clear. Maybe the authors can provide a concrete example to describe the studied problem. This makes readers hard to understand the following text of this paper.
- No detailed related work discussion provided.
- The problem setting is hard to understand, especially in preliminaries. Are there train set and test set? What should a model learn on train set, and what it should do on test set? In other words, what is the complete data flow for a model? And how to evaluate them?
- The used LLM baselines are only light version of GPT, which can not reflect the strong planning ability of modern LLMs. What is the performance of using GPT 6, rather than using the mini version? And what about Claude, and other LLMs?
Questions for authors
Please see weaknesses above.
6 comments
Author Response (2/3)
[W3] Training and test sets, data flow, and evaluation.
We apologize that the problem setting and the data flow were not clearly described in the submission. There are separate training, validation, and test sets. Following MSP-LLM [7], the main benchmark is built from text-mined solid-state synthesis recipes [9] and split into 8,295 training, 932 validation, and 918 test targets. No target is shared across the splits, and no test recipe comes from a paper that appears in the training set. The condition model additionally uses 27,110 training records annotated with at least one of temperature, time, or atmosphere. A second set of 2,344 new targets [10] is used only for evaluation. On the training set, each module of CondMSP learns one mapping. On the test set, the modules are applied in sequence, starting from the target composition alone (Table B).
Table B. Task and data flow
| Module | What it learns from the training set | Input at test time | Output at test time |
|---|---|---|---|
| Precursor prediction (from MSP-LLM) | Target → precursor set | Target composition | Top-10 precursor sets with probabilities |
| Condition prediction | (Target, precursor set) → maximum heating temperature, maximum heating time, and atmosphere | Target and each predicted precursor set | Predicted conditions for each precursor set |
| Operation prediction (generation and retrieval) | (Target, precursor set, conditions) → operation sequence; the retrieval index stores the training procedures | Target, predicted precursor set, and predicted conditions | Ranked operation sequences for each precursor set |
| Route selection (MILP) | Nothing is learned; the route utility combines the precursor probability and the operation rank | Candidate routes, their utilities, predicted conditions, and the budget | Routes to try and their assignment to furnace runs |
At test time, every module uses the predicted outputs of the preceding modules, and the reference route of the test target, i.e., the route reported in the literature, is used only for scoring. The validation set is used to select the combination thresholds of the condition model and to calibrate the fusion of generated and retrieved operation sequences.
The main metric, recipe recovery, is the fraction of test targets for which at least one selected route has exactly the reference precursor set and operation sequence. It is measured under three kinds of budgets: the number of routes tried (trials), the number of furnace runs, each holding up to eight samples with compatible predicted conditions, and the total furnace-hours, including a 2-h overhead per run. In addition, Table 1 (paper) evaluates condition prediction, and Table 2 (paper) evaluates operation prediction with the literature precursors given as input, so that each module can also be assessed on its own. In the revised manuscript, Section 3.4 (End-to-End Pipeline) describes this data flow, a condensed version of Table B appears as Table 22 in Appendix G, and the data splits and evaluation metrics are described in the Datasets paragraph of Section 4 and the Metrics paragraph of Section 4.3.
Author Response (3/3)
[W4] Comparison with stronger LLMs.
Following the reviewer's suggestion, we evaluated GPT-6.1-sol and Claude Opus 5.5 under the protocol of the GPT-5.4 mini baselines in Table 3 (paper): the same fixed 100-shot examples, 30 ranked candidates per response, and the same scorer and furnace planner on all 918 test targets. Table C reports PP→SOP prompting, in which a model first predicts a precursor set and then 30 operation sequences for it, because it gave higher recovery than end-to-end prompting for both models.
Table C. Recipe recovery (%) of frontier LLMs under trial and furnace budgets (918 test targets). The rows for GPT-5.4 mini (100-shot), MSP-LLM, and CondMSP are from Table 3 (paper).
| Method | 1 trial | 10 trials | 30 trials | 1 run | 3 runs | 10 runs | 48 h |
|---|---|---|---|---|---|---|---|
| GPT-5.4 mini (100-shot) | 7.19 | 12.69 | 18.08 | 12.64 | 14.81 | 18.08 | 15.31 |
| GPT-6.1-sol (100-shot) | 6.75 | 16.88 | 27.23 | 15.47 | 20.81 | 27.23 | 22.00 |
| Claude Opus 5.5 (100-shot) | 6.86 | 17.21 | 27.89 | 16.99 | 22.22 | 27.89 | 23.09 |
| MSP-LLM (LLaMA-3.1-8B) | 7.81 | 22.55 | 34.68 | 20.04 | 27.89 | 34.68 | 27.92 |
| CondMSP (LLaMA-3.1-8B) | 8.46 | 26.18 | 40.01 | 21.71 | 32.90 | 42.85 | 31.99 |
As shown in Table C, these models are much stronger than GPT-5.4 mini, recovering 27.23% and 27.89% of the targets at 30 trials compared with 18.08%, but they remain below CondMSP at every budget: at 30 trials and with three furnace runs, Claude Opus 5.5 reaches 27.89% and 22.22%, compared with 40.01% and 32.90% for CondMSP. They are also below MSP-LLM, which fine-tunes an 8B open model on the same training data. Strong general-purpose LLMs therefore improve substantially over the light versions, but a planner trained for this task remains clearly better. In the revised manuscript, we have added these results to Appendix I (Table 26) and referred to them in Section 4.3.
[1] He, Tanjin, et al. "Precursor recommendation for inorganic synthesis by machine learning materials similarity from scientific literature." Science advances 9.23 (2023): eadg8180.
[2] Noh, Heewoong, et al. "Retrieval-retro: retrieval-based inorganic retrosynthesis with expert knowledge." Advances in Neural Information Processing Systems 37 (2024): 25375-25400.
[3] Prein, Thorben, et al. "Retro-rank-in: a ranking-based approach for inorganic materials synthesis planning." arXiv preprint arXiv:2502.04289 (2025).
[4] Huo, Haoyan, et al. "Machine-learning rationalization and prediction of solid-state synthesis conditions." Chemistry of Materials 34.16 (2022): 7323.
[5] Karpovich, Christopher, et al. "Interpretable machine learning enabled inorganic reaction classification and synthesis condition prediction." Chemistry of Materials 35.3 (2023): 1062-1079.
[6] Na, Gyoung S. "Artificial intelligence for learning material synthesis processes of thermoelectric materials." Chemistry of Materials 35.19 (2023): 8272-8280.
[7] Noh, Heewoong, et al. "MSP-LLM: A Unified Large Language Model Framework for Complete Material Synthesis Planning." arXiv preprint arXiv:2602.07543 (2026).
[8] Szymanski, Nathan J., et al. "An autonomous laboratory for the accelerated synthesis of inorganic materials." Nature 624.7990 (2023): 86-91.
[9] Kononova, Olga, et al. "Text-mined dataset of inorganic materials synthesis recipes." Scientific data 6.1 (2019): 203.
[10] Lee, Sanghoon, et al. "Text-mined dataset of solid-state syntheses with impurity phases using Large Language Model." Scientific Data 12.1 (2025): 1969.
Author Response (1/3)
We sincerely thank the reviewer for the helpful feedback on the clarity of our paper. We address each point below with clarifications and additional experiments.
[W1] Background and a concrete example of MSP.
We apologize for the lack of background, which made the paper difficult to read. Following the reviewer's suggestion, we describe the task with a concrete example. Materials synthesis planning (MSP) asks how a target inorganic material, specified only by its chemical composition, can be synthesized. In solid-state synthesis, a synthesis plan consists of the starting chemicals (precursors), the sequence of operations applied to them (e.g., mixing, heating, quenching), and the processing conditions (heating temperature, time, and atmosphere). A target can usually be synthesized from several precursor sets and procedures, and each attempt occupies a furnace for hours, so a planner must also decide which of the candidate routes to try within a limited budget. Table A illustrates the task with a target from our test set.
Table A. An example of the task (test target Ba₄Mn₃O₁₀, budget of three furnace runs of up to eight samples each)
In this example, the precursor model ranks BaCO₃ + Mn₂O₃ above the reference precursor set BaCO₃ + MnO₂. CondMSP predicts the conditions for each set, generates operation sequences, and, with three furnace runs, tries routes of both sets, among which is the reference route. A pipeline that tries candidates only for the top-ranked precursor set, as MSP-LLM does, misses it. The example also shows the two roles of the predicted conditions in CondMSP: they serve as process context for operation prediction (step 3), and they determine which routes can share a furnace run (step 4), since routes with the same predicted conditions can be fired together.
In the revised manuscript, we have added this example as Section 2.4 (An Example of MSP), which presents the task itself, and we have added Section 3.4 (End-to-End Pipeline), which follows how CondMSP processes the same target step by step. Section 1 now points to the example when MSP is first defined, and Appendix G describes the basic background of solid-state synthesis and data-driven MSP and gives the full example as Table 23.
[W2] Related work.
A related-work section is provided in Appendix A of the submission. Because of the page limit, it was placed in the appendix rather than in the main text, which made it easy to miss. It discusses materials synthesis prediction, including thermodynamics-guided route ranking, text-mined recipe datasets, and the prediction of precursors, synthesis conditions, and operation sequences, as well as self-driving laboratories.
In brief, prior data-driven methods predict precursors [1, 2, 3], synthesis conditions [4, 5], or operation sequences [6], and MSP-LLM [7] connects precursor prediction with precursor-conditioned operation prediction. These methods output candidate recipes, but they neither use predicted conditions as context for operation prediction nor decide which candidates to try under a limited budget; CondMSP addresses both. Self-driving laboratories such as the A-Lab [8] execute candidate recipes in furnace batches, and CondMSP provides the per-target step of choosing which routes to try and which can share a furnace run. In the revised manuscript, we have added a related-work section to the main text as Section 5, which condenses Appendix A and refers to it for the extended discussion.