- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors present an harness to control a fronze LLM-Diffusion crystal generator. They propose an implementation of such controller, where it can assess the provisional LLM-generated structures, retrieve materials knowledge, and ultimately adaptively update the generator prompt. They perform experiments against best-of-k and verifier-selection baselines, showing that indeed the implemented harness can increase materials functionality in both a open-ended material generation, and a knowledge-grounded superconductivity search tasks.
Critical strengths
- This work seems to show that control of provisional (not refined) LLM ‘structures’ alone allows to control (at least certain) desired functional quantities of the final structures. I believe this to be a valuable observation for this class of methods.
Critical weaknesses
- (1st contribution - Controller) The first contribution of a batch-level controller seems somewhat trivial, why such a controller would not be possible to be defined in <1 day by any scientist? I work with related agentic frameworks for discovery and I am struggling to understand what really is the value and/or novelty of this first contribution.
- (2nd contribution - Experiments / Evaluation) The considered baselines seem primarily trivial approaches and do not seem to involve currently used methods such as reward-based fine-tuning methods as well as inference-time methods. The lack of such standard methods renders the experimental assessment in this work quite unconvincing, and from a more scientific standpoint, does not allow to gain understanding how the proposed method actually relates to more standard ways (developed over the last 3-4 years) to potentially tackle the same problem class.
- (3rd contribution - screening experiments) The baselines employed are again somewhat trivial, similarly, the fact that in-context learning works for online adaptation is very established (e.g., see all literature on Thompson sampling interpretations of in-context learning / LLM conditioning). While specific applications of these ideas might be of relevance to tackle open problems e.g., in material design, I struggle to understand the value/novelty of this experimental assessment in the context of this conference. In other words, the new information it brings seems very limited to me, at least from a methodological perspective. If the application tackled is an open problem in material sciences, there might be a better venue in my opinion to show that this methodology allows to make progress on that specific open problem.