acceptodds
Under review as a conference paper at ICLR 2027

Harness Meets Diffusion: Single-ReAct-Guided Crystal Generation and Knowledge-Grounded Superconductivity Search

Abstract

Scientific crystal discovery requires generators that can adapt their search to explicit property targets. We introduce a single-ReAct controller for a frozen LLM-diffusion crystal generator. The controller uses batch summaries and retrieved materials knowledge to guide subsequent LLM proposals. Selected candidates then undergo diffusion refinement. In open-ended generation, it raises the stable, unique, and novel fraction () from to across three matched seeds, with a gain in every seed and a decline in reference-distribution fidelity. We then apply the controller to superconductivity screening using a frozen predictor and a separate structural predictor for evaluation. In comparisons with the same generator and candidate budget, the controller achieves across four seeds, compared with for best-of- and for verifier-only selection. Across six seeds, it generates as many candidates with predicted K and nearest-training-composition Jaccard distance at least as a knowledge-only policy. Adding a superconductivity- tuned generator further increases the joint stable, unique, novel, and predicted- fraction from to in a matched run. These results show that feedback-driven control improves the computational screening yield of an existing crystal generator across two scientific objectives.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Crystal Generation, Diffusion Models, Large Language Models, Materials Discovery, Superconductivity Search, Verifier-Guided Generation, ReAct Agents, Scientific Machine Learning, Inverse Materials Design

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_vHq6OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors present an harness to control a fronze LLM-Diffusion crystal generator. They propose an implementation of such controller, where it can assess the provisional LLM-generated structures, retrieve materials knowledge, and ultimately adaptively update the generator prompt. They perform experiments against best-of-k and verifier-selection baselines, showing that indeed the implemented harness can increase materials functionality in both a open-ended material generation, and a knowledge-grounded superconductivity search tasks.

Critical strengths
  1. This work seems to show that control of provisional (not refined) LLM ‘structures’ alone allows to control (at least certain) desired functional quantities of the final structures. I believe this to be a valuable observation for this class of methods.
Critical weaknesses
  1. (1st contribution - Controller) The first contribution of a batch-level controller seems somewhat trivial, why such a controller would not be possible to be defined in <1 day by any scientist? I work with related agentic frameworks for discovery and I am struggling to understand what really is the value and/or novelty of this first contribution.
  2. (2nd contribution - Experiments / Evaluation) The considered baselines seem primarily trivial approaches and do not seem to involve currently used methods such as reward-based fine-tuning methods as well as inference-time methods. The lack of such standard methods renders the experimental assessment in this work quite unconvincing, and from a more scientific standpoint, does not allow to gain understanding how the proposed method actually relates to more standard ways (developed over the last 3-4 years) to potentially tackle the same problem class.
  3. (3rd contribution - screening experiments) The baselines employed are again somewhat trivial, similarly, the fact that in-context learning works for online adaptation is very established (e.g., see all literature on Thompson sampling interpretations of in-context learning / LLM conditioning). While specific applications of these ideas might be of relevance to tackle open problems e.g., in material design, I struggle to understand the value/novelty of this experimental assessment in the context of this conference. In other words, the new information it brings seems very limited to me, at least from a methodological perspective. If the application tackled is an open problem in material sciences, there might be a better venue in my opinion to show that this methodology allows to make progress on that specific open problem.
Questions for authors
Reviewer_zSAgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces an LLM-based controller around the frozen CrysLLMGen crystal generator. Candidate scores, search history, and predefined materials knowledge guide iterative prompt updates, after which selected candidates undergo diffusion refinement. Experiments on open-ended crystal generation and superconductivity screening report improved yields of candidates meeting joint stability, novelty, and predicted-property criteria.

Critical strengths

Under matched candidate budgets, the controller improves joint screening yields over fixed prompting and direct candidate selection. These comparisons provide evidence that evaluation feedback can improve the use of an existing crystal generator.

Critical weaknesses
  1. The methodological contribution needs clearer support. The framework combines knowledge retrieval, search memory, prompt updates, and heuristic selection around an existing generator, but comparisons mainly involve fixed prompting, direct selection, and internal variants. Its advantages over related LLM-guided materials search methods are therefore difficult to assess. Reliance on a fixed knowledge pack and task-specific rules also leaves unclear how much of the approach transfers beyond the evaluated tasks or turns accumulated experience into reusable knowledge.
  2. Screening yield alone does not resolve important questions about the generated candidates. The paper does not assess whether the planner's structural requirements are preserved in the final refined crystals, or whether higher yields concentrate generation in a few chemical systems or closely related structures. Distance from training compositions and aggregate SUN scores do not establish diversity within the generated set. These gaps limit the evidence for reliable structural control and broad search coverage.
Questions for authors
  1. How does the control strategy differ from GenMS [1] and MatLLMSearch [2]? Please clarify the methodological differences and, where a direct comparison is feasible, use consistent tasks, evaluation criteria, and clearly specified search budgets.
  2. What changes to the knowledge pack, prompts, and rules are required for a new materials system or property objective? Please distinguish demonstrated transfer or knowledge reuse from extensions that would require further development.
  3. For a small random sample of planner outputs with explicit structural requirements, how often do the final crystals satisfy them? Please report representative examples and satisfaction rates before and after refinement, covering requirements such as coordination, layering, or connectivity.
  4. Using the existing samples, can the authors compare chemical-system coverage, distributional concentration, and structural diversity across search strategies, including the subset meeting the target-property threshold? This would help establish whether the screening gains involve a substantial loss of coverage. [1] Yang et al. Generative Hierarchical Materials Search. NeurIPS, 2024. [2] Gan et al. MatLLMSearch: Crystal Structure Discovery with Evolution-Guided Large Language Models. arXiv:2502.20933v2, 2025.
Reviewer_amHROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses open-ended crystal generation and superconductivity search through a harness-based ReAct workflow, with a generator component that is an LLM fine-tuned for crystal generation. The workflow loop consists of other components, including a controller that plans, scores, selects, summarizes, and rewrites prompts for the subsequent generation. The selected structures are refined by a frozen diffusion model (similar to Khastagir et al. 2025) and then evaluated. The performance is measured by standard crystal generation metrics (validity, SUN, MSUN), and TcT_c-based metrics for superconductivity search. The overall idea is reasonable for this problem, and it seems to improve certain metrics when compared to some baselines and to previous crystal generation models, and also to alternative control policies within the approach. However, the draft at this stage suffers from poor presentation and lack of clarity in several places.

Critical strengths
  1. The overall methodology aligns with current trends in LLM research – self-improvement, harness optimization.
  2. The reported metrics for the proposed approach are aggregated over multiple seeds.
  3. Some major limitations (lack of DFT validation, approximate nature of the TcT_c proxy) are acknowledged.
Critical weaknesses

Presentation and Clarity

The current draft is not easy to follow because of lack of clarity and missing information. Some examples are provided below.

  1. There is no background section providing a brief overview of the domain and the problem, especially the superconductivity search part, which is new for ML audience: Even TcT_c is not labeled as critical temperature anywhere in the draft.
  2. The abstract reports four-decimal figures with standard deviations and per-experiment seed counts, and the main takeaways are hard to follow.
  3. It is totally unclear what the knowledge pack is. Additionally, much of the details of the controller’s working are not provided in the main draft and are buried in the Appendix (which also seems incomplete). TcProxy is not defined properly in the main text.
  4. The space available is not used effectively despite the 9-page limit.
  5. Some of the baselines are not explained well in the main text. For example, the details of best-of-k, knowledge-only, verifier-only, control-only, etc. baselines only have ambiguous 1-line explanations.
  6. Details of the “generator fine-tuning” experiment are not provided – e.g. what dataset, model, and method was used to fine-tune?
  7. No examples of generated crystals or trajectories of the ReAct loop are provided.
  8. Equation 1 does not define all variables used.

Experiments

  1. The evaluation of generated crystals follows an older protocol (with CHGNet model for stability). MatterGen [1] and LeMat-GenBench [2] provide newer protocols with newer MLIPs.
  2. It is not clear why different experiments use different numbers of seeds while reporting results.
  3. The metric values of the baselines are directly taken from Khastagir et al. 2025, and it is not fully clear if the evaluation protocols exactly match with them (e.g. number of sampled crystals, denominator in SUN percentages, rejection sampling, etc.).
  4. The summary passed to the planner is never specified and appears not to include validities (cheap to compute on provisional structures), which may explain the lower compositional validity compared to CrysLLMGen.
  5. Uniqueness and novelty percentages are not separately reported.

References

[1] Zeni, Claudio, et al. "A generative model for inorganic materials design." Nature 639.8055 (2025): 624-632.

[2] Betala, Siddharth, et al. "Lemat-genbench: A unified evaluation framework for crystal generative models." arXiv preprint arXiv:2512.04562 (2025).

Questions for authors
  1. What is the training dataset of the tuned generator for TcTc-optimization?
  2. Were the compositions of the generated samples inspected for rare-earth elements?
  3. Is there a challenge in including the diffusion refinement step inside the controller’s self-improvement loop? How will this affect the wall-clock time of the run?
Reviewer_gAPEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a harness consisting of a feedback loop that aims to improve the yield of a frozen LLM–diffusion crystal generator on open-ended generation and superconductivity screening. The pipeline is simple: the authors take the outputs from the LLM, score and select the promising candidates, and ask an LLM to revise the prompt for the next round of generation, guided by the history and prior knowledge. Only the most promising candidates are eventually passed on to the diffusion model for refinement.

Critical strengths
  1. In open-ended generation, the authors report increases of about 61% and 32% in SUN and MSUN, respectively, over the baseline. In their knowledge-grounded superconductivity search, the harness increases the number of candidates with a predicted Tc >= 20 K from 226 to 859, as evaluated by the final structural verifier.

  2. The reported improvements over best-of-k and verifier-only are great to have.

Critical weaknesses
  1. In Table 4, Knowledge-only and Control-only have higher mean TcProxy scores than the full controller, but their SUN, MSUN, and joint SUN and Tc metrics are absent. Table 9 does not help either, as it uses an earlier controller with four prompts, and even there I do not see a clear benefit of using the full controller. This is a major gap that needs to be addressed to gauge whether the whole harness performs better than either component alone. Additionally, Table 5 shows a 2.26× React advantage over Knowledge-only in the number of candidates meeting predicted Tc and element-set distance thresholds, but does not report how many of those candidates also satisfy SUN.

  2. The general novelty of the method is modest, as prior works cited in the paper already combine language models, diffusion, and guided search. The contribution therefore needs to be particularly clear. Without evaluating Knowledge-only and Control-only on the final joint objective, the benefit of combining these components remains unclear.

In its current form, I cannot suggest acceptance.

Questions for authors
  1. Can you provide the SUN, MSUN, and joint SUN and Tc metrics for Knowledge-only and Control-only in Table 4, so that the full harness can be compared with either component alone?

  2. How do the results in Table 9, which uses an earlier four-prompt controller, establish a benefit from using the full controller?

  3. How many of the candidates underlying the 2.26× React advantage over Knowledge-only in Table 5 also satisfy SUN?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.