acceptodds
Under review as a conference paper at ICLR 2027

C4: Evidence-Aware Search for LLM-Guided Materials Discovery

Abstract

Historical results guide candidate generation in materials discovery, but a limited evaluation budget leaves many directions in the vast composition and structure space rarely or never tried. Search must therefore use successful experience while distinguishing insufficient observation from low potential. We introduce C4, a framework that coordinates four complementary tools: Open expands candidate support, Element identifies promising chemical directions, Explore probes underexplored alternatives, and Refine develops promising parents. C4 combines evaluated outcomes and shared visit records as search evidence, drawing on the exploration–exploitation principle of considering observed returns together with the evidence behind them. An LLM proposes tool-call proportions and branch-scoring weights from structured feedback. The Harness turns these proposals into budgeted calls and concrete substitution choices. Across fourteen multi-property tasks and three seeds with Qwen3-8B, C4 averages 448.33 new feasible compositions, compared with 105.67 for LLEMA, and leads all five baselines in each seed’s total. Component studies and parent reuse illustrate the roles of the tools and their coordination. The advantage also extends to GPT-4o and longer search, while knowledge-guided cases show how additional context redirects the same search process.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM-guided materials discovery, multi-granularity search, adaptive search, materials discovery, large language models, finite-evidence decision making

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_LZKnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The work proposes an LLM-guided framework for materials discovery under a limited evaluation budget. The method uses a coordination of 4 search mechanisms: Open, which proposes new candidates; Element, which looks for substitution directions; Explore, looks for underexplored alternatives; and Refine, which looks for parent structures. The objective, as highlighted by the authors is to discover as many distinct feasible compositions as possible within a fixed evaluation budget.

The empirical results are good. C4 outperforms the reported LLM-based baselines, as well as MCTS and simulated annealing, across the 14 materials tasks. The ablations are also good and indicate that the different components are complementary and that adaptive coordination performs better than fixing the allocation and scoring parameters.

Overall, I think the problem is relevant and the results are interesting. However, I have two main concerns that currently make me lean towards rejection: I was not really able to find a methodological novelty to the work, there is a very broad literature which seems either missing, or just simply that the authors are unaware that this problem has been addressed by whole other communities, that of sequential search/optimisation literature, and I am not fully convinced that the paper establishes why an LLM is needed for the adaptive search-control decisions.

Critical strengths
  • The problem is relevant. Searching a very large materials space under a limited evaluation budget is important.
  • The empirical advantages over the reported baselines are good and over several tasks.
  • The Open/Element/Explore/Refine decomposition is intuitive, and an ablation was done over them.
  • The paper is well written and the framework is relatively easy to follow.
Critical weaknesses
  1. I find the methodological novelty and positioning relative to well known and established sequential search and optimisation methods unclear.

C4 combines several ideas that are individually quite well established: exploration based on visitation, adaptive allocation between different search operators, sharing information across related branches, broad generation, and local refinement. There is substantial literature around these ideas in bandits, evolutionary optimisation, adaptive operator selection, dynamic algorithm configuration, and more generally sequential optimisation.

The paper does cite UCB, MCTS, simulated annealing, and some work on adaptive operator selection. However, I was quite surprised by how little discussion there is of the much broader literature on Bayesian optimisation, active learning / experimental design, categorical and combinatorial optimisation, and related sample-efficient optimisation methods. Even some LLM-BO combinations.

I appreciate that the materials space here is discrete, structured, and can also be dynamically expanded through Open, so I agree that applying a standard GP-BO might not be straightforward. However, I do not think this makes the broader literature irrelevant. For example, one could be using an LLM or another generative model to construct or expand the candidate space, while using an uncertainty-aware optimisation method to decide which of these candidates should actually consume the evaluation budget.

Because of this, I still struggle somewhat to understand what the fundamentally new search principle in C4 is, beyond the particular decomposition into Open/Element/Explore/Refine and the way information is shared between them. I think the paper needs to position this considerably more carefully against the existing sequential optimisation literature and, in particular, explain what limitation of these approaches C4 is addressing that cannot be dealt with by existing methods.

  1. I am not fully convinced is why the LLM should be responsible for the adaptive search-control decisions. I see a motivation for using an LLM for candidate generation, particularly early in a search, as several LLM-BO papers have recently suggested that is a good idea.

I find the motivation much less clear for using the LLM to decide how the budget should be allocated between Open/Element/Explore/Refine, or to adapt the numerical branch-scoring weights. Once the problem has been reduced to allocating budget between four operators and adapting a small number of weights based on previous search statistics, this starts to look much more like an online algorithm-selection or meta-optimisation problem. There are several standard approaches for doing this using bandits, online learning, adaptive operator selection, or related methods.

The Fixed-C4 ablation is useful, but I think what it shows is that adaptive coordination is better than fixed coordination. It does not show that LLM-based adaptive coordination is better than algorithmic adaptive coordination. To me this distinction is quite important. The current results could equally be interpreted as showing that the C4 search architecture itself is strong, with the LLM being one possible mechanism for adapting its control parameters.

I think a particularly useful, and relatively limited, additional experiment would therefore be to keep exactly the same C4 architecture and the same LLM-based candidate-generation mechanism, but replace the LLM controller with a simple adaptive non-LLM controller that receives exactly the same search statistics. This would much more directly test whether the LLM is contributing something specific to the search-control problem.

Questions for authors
  1. What do the authors consider to be the closest existing sequential optimisation formulation to C4? In particular, why would structured/categorical Bayesian optimisation, Bayesian active learning, bandit methods, or adaptive operator-selection methods not address essentially the same problem, potentially together with an LLM or generative model for constructing the candidate space?
  2. Could the authors compare the LLM-based adaptation of the tool allocation and branch-scoring weights against an adaptive non-LLM controller that receives the same observations? As far as I can tell, the current Fixed-C4 comparison demonstrates the benefit of adaptation, but not the benefit of LLM-based adaptation specifically.
  3. More generally, can the authors disentangle how much of the improvement comes from the chemical/semantic prior of the LLM, and how much comes from the search architecture and adaptive-control mechanism itself?
Reviewer_XHqjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents C4, an LLM-guided adaptive exploration/exploitation framework that decides where to spend a fixed candidate-evaluation budget; for this, C4 relies on four complementary search operators and shared search history.

Critical strengths
  • Clear system specifications, appears reproducible;
  • Well-motivated decomposition into four operators that address complementary needs;
  • Strong empirical gains across a fairly broad task set.
Critical weaknesses
  • Ablations do not cleanly isolate the claimed evidence-aware/adaptive coordination mechanism: the component study is only one seed, and removing a tool does not cleanly remove its underlying mechanism (e.g., if I understood details of the framework correctly, Explore is not the only place where C4 uses visitation counts, Refine is affected too).

  • Baseline comparisons are not fully apples-to-apples because methods differ in model calls, output budgets, and implementation details. In particular, the comparisons to LLMatDesign and MatLLMSearch are harder to interpret because the paper evaluates locally adapted variants rather than the original systems (so, it is difficult to know whether poor performance reflects the underlying methods or the adaptation).

  • The evidence for generality is somewhat limited. The main experiments use one backbone (Qwen3-8B) and only three seeds, while the GPT-4o transfer and longer-budget analyses are even more limited. The detailed cross-tool case study is also based on a selected trajectory, so it is hard to tell how broadly the observed behavior generalizes.

Questions for authors
  • Can you provide multi-seed ablations that remove the adaptive allocation/ranking components?
  • How do results change under matched LLM-call or token budgets across baselines?
Reviewer_2tUmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

C4 searches for distinct material compositions that satisfy fixed property constraints under a candidate budget. It combines broad proposals, element substitution, visit-aware exploration, and local composition refinement. The tools share parents and search records, while an LLM adjusts their allocation and branch-scoring weights. A separate generator builds structures from the selected formulas. The paper reports more compositions passing fixed proxy tests than the baselines, with further experiments on longer searches, added knowledge, and another backbone. Novelty is measured within each search archive.

Critical strengths

I like the shared search records. Parents and outcomes found by one tool can inform later choices by another. The scoring, scheduling, and prompt details make it possible to inspect how this information is used.

The extra experiments add useful support. Longer searches and a different backbone still show an advantage. These checks are limited, but suggest that the result is not confined to the original short-budget setting.

Critical weaknesses

The LLM controller is not isolated. Fixed C4 freezes both allocation and branch weights at their initial settings. This does not separate adaptive allocation from better branch scoring, or establish an advantage over a tuned static policy. I would also compare with a simple controller using observed tool rewards.

Element-family and route checks do not verify exact target stoichiometry. A successful but unintended composition can therefore reinforce the selected substitution. I would like an audit and results with credit restricted to correctly realized actions.

Site-level substitutions determine coverage and branch selection, but the structure generator receives only a formula. Different intended edits can therefore lead to the same input. I would test formula-level visit tracking before attributing the gains to finer-grained exploration or local refinement.

More proxy hits do not yet establish more useful materials. For example, the electrolyte predicate does not measure ionic conductivity. I accept the stated proxy scope, but the scientific significance remains uncertain.

Questions for authors

Weaknesses section

Reviewer_TEp5OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces C4, an evidence-aware search and control framework for LLM-guided materials discovery. It coordinates four complementary tools, Open, Element, Explore, and Refine, within a generation–evaluation loop, sharing parent structures, property-evaluation outcomes, and visitation records. Inspired by the UCB exploration–exploitation principle, C4 combines observed promise with insufficient exploration to guide subsequent search decisions. The LLM proposes tool allocations and branch-scoring weights, while a deterministic Harness validates these proposals, selects concrete actions, and manages the candidate budget. Experiments on fourteen multi-property tasks show improved discovery of distinct proxy-feasible compositions over five baselines, with further validation through ablations, longer searches, and transfer to GPT-4o.

Critical strengths
  1. C4 shares structures, evaluation results, and visit counts across its four tools, allowing both successful and failed attempts to guide later choices. This helps balance promising directions with less-explored alternatives under a limited budget. The Harness provides explicit rules for action selection and budget management, while the LLM proposes candidates and adjusts search priorities.

  2. The empirical gains are substantial. Across fourteen tasks and three seeds, C4 finds an average total of 448.33 new proxy-feasible compositions, versus 105.67 for LLEMA and 260.33 for MCTS under matched candidate and model-call budgets. Its advantage persists in longer searches and with GPT-4o, supporting the framework’s usefulness beyond the main setting.

Critical weaknesses

The physical significance of the discoveries remains unclear. Novelty is assessed only against the run’s archive, and feasibility is determined largely by surrogate predictions, without reported independent physical validation. Some tasks omit key application properties, such as ionic conductivity for electrolytes. The structure generator also receives the formula rather than parent coordinates, so local structural preservation during refinement is not guaranteed.

Questions for authors
  1. Do you have controlled comparisons of shared vs. per-tool evidence and adaptive allocation versus adaptive scoring? The cross-tool reuse statistics come from one selected trajectory and show that reuse occurs, not that sharing improves outcomes.
  2. What fraction of candidates generated by each tool, particularly Open and Refine, pass the validity checks, and what fraction of those valid candidates satisfy the task constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.