- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Focus on material search, this work introduces an agentic framework to refine a material structure targting on a specific design requirements. The major novelty lies in two aspects: multi-fidelity material reasoning design (inner loop) and evolution loop (outer loop) with self-reflection. Evaluations on two tasks (ie, search task and stable structure discovery benchmark) show its superiority compared to traditional methods such as MLIP, and SOTA LLMs such as GPT 5.6. Finally, the analysis and case study further demonstrate its effectiveness.
Critical strengths
- Presentation: The work is well structured and clearly written; the challenges are clearly presented, and the solutions are stated to reasonably address them.
- Novelty: This work not only introduces an effective multi-fidelity framework for material structure discovery, but also proposes an efficient reflection method for agent harnessing, which may further generalize this method to other scientific domains.
- Evaluation: The experiments are elaborated to demonstrate its strong superiority. For example, it improves the overall success rate from about 50% to 90% on search performance.
Critical weaknesses
- Novelty: Although the overall framework is novel, the major solutions, ie, multi-fidelity solutions that combine MLIP and DFT, and self-reflection solutions that utilize historical trajectory (records in this work), are not new. The authors should compare and discuss with some similar works, including but not limited to examples such as multi-fidelity work [1,2] and self-reflection on material discovery [3].
- The authors demonstrate its effectiveness in two benchmarks, ie, computational Raman and phonon property search tasks and retrospective literature case studies on well-studied materials (eg, Si, etc), but they claim a broader effectiveness for general, autonomous materials discovery and arbitrary multi-perturbation (strain, vacancy, substitution, and compositional doping) structure search (for example, the authors claim "General "materials structure discovery" or ""combinatorially large" perturbation space"). The authors should narrow down their claims (and research focus) or include much more comprehensive experiments.
[1] Hernández-García, A., Saxena, N., Jain, M., Liu, C. H., & Bengio, Y. "Multi-Fidelity Active Learning with GFlowNets". 2024. TMLR.
[2] Chen., Zhan ., et al. "PhyVer: Physics-Grounded Material Claim Verification with Multi-Fidelity Physical Evidence. ACL. 2026.
[3] Abhyankar,et al.. "LLEMA: Evolutionary search with LLMs for multi-objective materials discovery." ICLR. 2026.
Questions for authors
Try to address the weaknesses with concrete experiments or clear revisions.
5 comments
Response to the questions - part 2
Weakness 2: Scope of the generality claim
We appreciate the reviewer’s comment on the generalizability of Struct-R1. We agree that the scope of our empirical validation should be stated more precisely. Our current experiments cover five target properties (Raman shift, elastic energy, charge transition level, phonon decay, and Born effective charge) on Si, Ge, GaAs, and MgO, with point defects and uniaxial strain as perturbations. They also include the Matbench Discovery test and two case studies. We will revise the Abstract, Introduction, and Conclusion so that the claims match this empirical scope.
Part of what broader search requires is exercised in our experiments. Struct-R1 models the role of the cheap tier through structured tier records. In our experiments, this role differs across tasks. Cheap tiers can be used directly for measurement calibration, or serve only as trend indicators, or be uninformative or even unavailable in some cases. The tier records and self-evolution let the agent identify and apply the right role for each task and optimize the search strategies. We also added transfer experiments in our response to Question 2 of Reviewer VkDf, where search hints learned on one material or property were frozen and applied to unseen tasks. In several cases, the transferred hints improved success rates and reduced DFT costs, showing the potential of such learned experience to generalize across materials and properties. Struct-R1 combines agent reasoning with this harness design, so it is designed to handle these situations as the search space broadens.
We agree that such generalization requires further systematic validation. Extending the evaluation to more complex materials, broader property targets, and larger perturbation spaces remains an important direction for future work. We will add a discussion of these limitations and future directions in the revised paper.
We thank the reviewer again for the valuable and constructive feedback. We would be happy to provide further clarification or additional experimental results if the reviewer has any remaining questions.
Follow-up on the rebuttal responses
Dear Reviewer Xkgq,
Thank you again for your constructive feedback. We have added further comparisons and revised the manuscript to clarify our contributions and empirical scope. We sincerely hope these revisions can addresse your concerns.
We would greatly appreciate hearing if you have any remaining concerns. As the discussion period is already more than halfway through, we still have time to provide further experiments or clarification if needed.
Thank you again for your time and consideration.
Best regards,
The Authors
Response to the questions - part 1
We thank the reviewer for the detailed and constructive feedback. We will add new comparison results and adjust the imprecise claim in the paper. Below are the responses.
Weakness 1: Baselines and related work
We thank the reviewer for pointing out these related works. We believe our core contribution lies in the agent harness design for materials search, which enables efficient utilization and continuous evolution of cross-fidelity knowledge to balance search accuracy and measurement cost. Existing multi-fidelity methods, such as MF-GFN [1], rely on statistical or learned models that can be unreliable when low-fidelity measurements are biased or uninformative. Meanwhile, LLM-based approaches such as PhyVer [2] and LLEMA [3] focus on claim verification and evolutionary materials discovery, respectively, without explicitly enabling the agent to maintain and update cross-fidelity relationships for cost-aware search decisions. We also added new baselines experiments, using the same results in Question 5 in Review ehTF, including a transferred version of LLMatDesign, LLEMA, MF-MES, and a simple ReAct Agent. MF-GFN [1] requires training a GFlowNet policy, while Struct-R1 and all other baselines do not train a neural policy. We therefore did not compare with it directly. Instead, we adopted MF-MES, the cost-aware multi-fidelity acquisition function used by MF-GFN.
LLMatDesign and LLEMA were designed for composition search and do not directly support our tasks. Following their released code, we adapted their core search strategies to our tools, budgets, and questions. LLMatDesign proposes one modification per step and uses self-reflection and search history to guide subsequent steps. LLEMA evolves a population using successful and unsuccessful candidates, and generates five new candidates per iteration. MF-MES is a cost-aware multi-fidelity Bayesian optimization baseline that uses a Gaussian process to model target-property errors across configurations, strains, and fidelity tiers. It selects the next candidate and measurement fidelity based on expected information gain per unit time cost, and updates the model after each measurement. We also build a ReAct agent that can call MLIP, MLIP pre-relax and DFT as tools. All experiments use GPT-5.6 Terra and the same tasks as Table 2. We will report details of these new baselines in the revision.
LLMatDesign performs sequential refinement and can be misled when MLIP feedback is biased, so the accuracy is low despite the lower DFT call numbers. LLEMA is more robust because it maintains a population of candidates, but evaluating five candidates per generation increases DFT cost. MF-MES achieves the highest success rate among non-LLM baselines through cost-aware fidelity selection, but performs less effectively with biased or uninformative MLIP feedback. A ReAct agent with the same tools can reach better balance, but because of the lack of relationship records between the tiers, it still has room for improvement.
Please notice that LLMatDesign and LLEMA were not designed for the same multi-fidelity setting as Struct-R1, so our goal is not to show that these methods fail. Instead, the comparison highlights the need for a dedicated harness system that explicitly manages measurement cost, fidelity, and cross-tier relationships for the multi-tier tools in the materials science research.