acceptodds
Under review as a conference paper at ICLR 2027

TritonDFT: An Agentic Framework and Benchmark for Automated DFT Workflows

Abstract

Density Functional Theory (DFT) is a cornerstone of materials science, yet executing DFT in practice requires coordinating a complex, multi-step workflow. Existing tools and LLM-based solutions automate parts of the steps, but lack support for full workflow automation, diverse task adaptation, and accuracy–cost trade-off optimization in DFT parameter choices. To this end, we present TritonDFT, an agentic framework adopting an expert-curated, extensible workflow design, with Pareto-aware parameter inference and automated parallelization, to enable efficient and accurate DFT execution. We further introduce DFTBench, a benchmark for evaluating the agent's multi-dimensional capabilities, spanning accuracy-cost trade-off optimization, high-performance computing knowledge, and monetary cost efficiency. Our code and benchmark suite can be found at https://anonymous.4open.science/r/TritonDFT-43C7/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, Material Science, LLM Agent, DFT, Scientific Benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kpbeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper demonstrates the effectiveness of several agent components through a series of ablation studies, including empirical guardrails against over-configuration in automated parallelization, reference-guided self-reflection, and self-reflection with lightweight probes. It also compares TritonDFT with the LLM-based framework DREAMS and rule-based parameter-selection baselines derived from Atomate and AiiDA, showing advantages in the reported evaluations.

However, the paper’s characterization of DREAMS is inaccurate. Section 5.3 contrasts a “fixed prompt template” with task-dependent planning and execution, whereas DREAMS already supports dynamic planning, systematic convergence testing, and feedback-driven error recovery. Its execution examples show the planner adding missing calculation steps and revising the workflow after failures. A fixed system-prompt template does not imply a fixed execution plan. Furthermore, the manuscript does not clearly describe the comparison protocol, including whether the official DREAMS implementation and prompts were used, which underlying model was selected, and whether DREAMS could readjust its parameters for downstream SCF and DOS calculations. These omissions weaken the persuasiveness of the comparison.

Moreover, several existing studies have proposed highly similar agent frameworks, including DREAMS [1], VASPilot [2], and AutoDFT [3]. These systems already combine task planning, domain-specific tools, parameter selection, execution monitoring, and feedback-driven recovery. Although the manuscript cites these works, it does not adequately address their architectural overlap with TritonDFT or establish clear methodological differences through direct comparisons. This substantially limits the novelty of the proposed architecture.

References

[1] Wang, Z., et al. (2025). DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulation. arXiv:2507.14267.

[2] Liu, J., et al. (2025). VASPilot: MCP-Facilitated Multi-Agent Intelligence for Autonomous VASP Simulations. arXiv:2508.07035.

[3] Yang, P., et al. (2026). AutoDFT: A Closed-Loop Multi-Agent Framework for Autonomous DFT Calculations. arXiv:2605.26179.

Critical strengths
  1. The paper presents extensive ablation studies demonstrating the effectiveness of several components, including reference-guided self-reflection, lightweight execution probes, and expert-informed guardrails for parallelization, which improve parameter-selection accuracy and computational efficiency.
Critical weaknesses
  1. Limited novelty. Several existing frameworks, such as DREAMS, VASPilot, and AutoDFT, adopt similar designs involving task planning, tool use, and feedback-driven refinement. The paper does not sufficiently distinguish its architecture from these approaches or demonstrate the contribution of its specific design choices.

  2. Insufficient experimental detail. The paper does not consistently report the effective test-set size and pass-rate denominator for each experiment. The comparison with DREAMS also lacks key details, including the underlying model, whether the official implementation and prompts were used, and whether both methods had comparable computational budgets and opportunities to adjust parameters. These omissions make the results difficult to reproduce and the fairness of the comparison difficult to assess.

Questions for authors
  1. Could a stronger model, equipped with an automated execution loop as in Codex or Claude Code, achieve comparable results with appropriate prompting, without the proposed framework?

  2. Are the observed improvements statistically significant given the current test-set sizes? How sensitive are these results to variations in prompt wording?

1 comment

We thank the reviewer for the detailed feedback and for recognizing the effectiveness of improvement.

Weakness 1: Novelty and Distinction from Existing Agent Frameworks

We agree that existing frameworks, including DREAMS, VASPilot, and AutoDFT, have demonstrated the potential of LLM agents for automated DFT workflows. Our contribution focuses on additional challenges in scientific computing rather than workflow automation alone. In particular, TritonDFT introduces Pareto-aware parameter inference to balance numerical accuracy and computational cost, lightweight probing for feedback-driven refinement, and hardware-aware parallelization. DFTBench further evaluates these capabilities across 67 materials with diverse physical and computational characteristics. These dimensions remain not fully explored in existing agentic DFT studies.

We will clarify these distinctions and better position our contributions relative to prior work in the revision.

Weakness 2: Experimental Details and DREAMS Comparison

Thank you for highlighting these issues. We acknowledge that our description of DREAMS as using a fixed prompt template could be misleading. DREAMS supports dynamic planning, and we will revise this characterization.

Our comparison focuses on the numerical accuracy of configurations selected by each framework and their suitability for downstream calculations. We will provide additional details on the DREAMS implementation, underlying LLM, prompts, computational budgets, and opportunities for parameter refinement. We will also clarify the effective test-set sizes and pass-rate denominators across experiments to improve reproducibility.

Question 1: Comparison with General-Purpose Coding Agents

General-purpose coding agents may face challenges in scientific computing without sufficient guidance from domain experts, particularly in selecting physically appropriate parameters and balancing numerical accuracy with computational cost.

We will include additional baselines using general-purpose coding agents, such as Codex and Claude Code, in the revision to further evaluate their capabilities on DFTBench and compare them with TritonDFT.

Question 2: Statistical Significance and Prompt Sensitivity

We thank the reviewer for raising this concern. DFTBench includes 67 materials across 10 categories and three accuracy thresholds. Our experiments demonstrate consistent improvements from reference guidance and lightweight probing across different material types.

In the revision, we will clarify the sample sizes and evaluation protocols. We will also expand the statistical analysis and examine sensitivity to prompt variations to better characterize the robustness of our findings.

Thank you again for your constructive feedback.

Reviewer_av5pOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces TritonDFT, an agentic system built to perform end-to-end density functional theory (DFT) workflows. DFT is a complex, multi-step computational procedure in chemistry-adjacent computational fields that is used to compute a variety of molecular properties including geometry, reaction energies, and binding affinities. The authors’ proposed system decomposes the DFT process into a pipeline accomplished by discrete agentic steps. Their results show that TritonDFT obtains better cost efficiency on the Pareto frontier of cost vs. accuracy in DFT calculations compared to previous non-agentic automated DFT approaches as well as a baseline DFT agent. They examine these tradeoffs in-depth and provide several case studies with domain-specific insights for automation of DFT workflows.

Critical strengths
  1. The paper is very well communicated and organized. The authors do a great job of introducing the reader, even those without domain knowledge, to the problem in an understandable way. I also appreciate how the authors are explicit about their parameters and controls; this lends credibility to the study design.
  2. The focus on Pareto optimization of cost vs. performance in the DFT automation seems novel, and the authors appropriately explore this direction.
  3. DFT is a practical and common workflow in many chemistry-adjacent fields, so the focus on this workflow is one of great impact to many scientific disciplines.
  4. I appreciate that authors released their code and files to run the benchmark and agent.
Critical weaknesses
  1. My primary critique of this work is that it is more of an application-based work rather than a fundamental leap in methodology. The proposed agent follows a fairly-standard multi-step workflow for agentic systems. Further, the benchmark is a fairly standard application-based benchmark, and I don’t see how it yields greater insights into agentic design or LLM performance in scientific settings. The novelty of this work is limited from an AI and/or machine learning perspective.
  2. The authors only test against two primary baselines, AiiDA, an 2016 algorithm for DFT automation, and DREAMS, another agent. The latter isn’t ran on the proposed DFTBench but rather only a more limited benchmark. I find it hard to evaluate the contribution of a benchmark if it’s only ran on one harness. I appreciate authors running TritonDFT on multiple base LLMs on the DFTBench, but more harnesses would be needed, such as standard coding agent benchmarks, for a proper comparison.
  3. I believe this is a rigorous paper, but it doesn’t necessarily fit the scope of ICLR. This paper is a better fit for a computational chemistry conference or journal, given its takeaways are more specific to computational chemistry and related fields rather than machine learning or AI. Similarly, the novelty in the method seems to be the more comprehensive control for practical simulations and environments in which the agent can operate rather than core algorithmic or evaluation novelties.
  4. The core takeaways from the paper are difficult to parse for non-domain experts. I would recommend communicating in more domain-agnostic terms and elaborating on general AI takeaways.
Questions for authors
  1. It’s unclear whether the structures and energies provided in the paper for the DFTBench can be obtained online or parametrically by the LLMs. Did the authors observe during benchmarking that agents accessed any of these data online? This could undermine the credibility of the benchmark.
  2. How should practitioners be expected to tune these parameters in your TritonDFT system? There’s no discussion in the paper about how practitioners should practically use the system and when to accept the cost tradeoff.
1 comment
AuthorsOpenReview ↗

Response to Review av5p

We thank the reviewer for the thoughtful feedback and for recognizing the rigor and practical value of our work.

Weakness 1: Methodological Novelty and General AI Contributions

Our primary contribution to AI lies in investigating whether and how LLM agents can reliably handle complex scientific computing tasks. Beyond workflow automation, TritonDFT investigates agent reasoning in accuracy-cost trade-off optimization, feedback-driven parameter refinement, and HPC resource configuration. DFTBench further provides a systematic evaluation of these capabilities across diverse scientific workloads. We believe these contributions have broader implications for AI for Science, particularly for developing and evaluating agents that interact with complex scientific software under accuracy and computational constraints.

Weakness 2: Baseline Comparisons and Benchmark Evaluation

We thank the reviewer for this suggestion. AiiDA and Atomate are established workflow tools widely used in practical DFT research, while DREAMS is a representative recent LLM-based agentic framework for DFT automation. We therefore consider them meaningful and competitive baselines among the available systems. DREAMS was evaluated on its original benchmark because its prompts and workflow implementation are tailored to its supported DFT tasks. Although DREAMS supports dynamic planning and refinement, directly extending its existing implementation to the broader tasks in DFTBench would require modifications to its prompts, tool interfaces, and execution workflow.

In the revision, we will clarify the baseline selection criteria and comparison protocols. We will also expand the evaluation to include additional baselines, such as general-purpose coding agents, to better assess the capabilities of TritonDFT and the broader applicability of DFTBench.

Weakness 3: Relevance to ICLR and AI Research

Our work investigates the capabilities and limitations of LLM agents in complex scientific computing tasks. We believe this is an important research direction within AI for Science and is relevant to the ICLR community.

Our evaluation goes beyond demonstrating practical DFT automation. It examines how agents handle numerical accuracy requirements, computational cost constraints, execution feedback, and HPC task efficiency improvement in scientific workflows. These challenges are relevant to the broader development of reliable scientific agents.

Weakness 4: Accessibility to Non-Domain Experts

Thank you for this suggestion. In the revision, we will further reduce domain-specific terminology and provide clearer explanations of technical concepts. We will also highlight the general AI takeaways to make the paper more accessible to readers without a background in DFT.

Question 1: Potential Data Leakage in DFTBench

Throughout the benchmark evaluation, agents were not allowed to browse the Internet. The only permitted external access was through a restricted Materials Project API interface to retrieve initial crystal structures and basic structural information. Agents did not have access to the benchmark's reference energies or expert-curated parameter configurations.

Furthermore, published DFT results often use different pseudopotentials, exchange-correlation functionals, and convergence settings. These differences can lead to inconsistent parameter configurations. Therefore, agents cannot reliably infer the benchmark's target configurations directly from existing literature.

Question 2: Practical Parameter Tuning and Human Interaction

TritonDFT supports human-in-the-loop interaction to allow practitioners to inspect and refine agent-generated configurations. As detailed in Appendix D, we provide two interfaces: a web interface and an advanced interface for HPC environments.

Both interfaces allow users to review the proposed workflow and parameter configurations before execution. Users can also provide feedback, modify parameters, and request revisions from the agent based on their accuracy requirements and computational budgets. This allows practitioners to retain control over the accuracy-cost tradeoff rather than relying entirely on automated decisions.

Reviewer_ZogYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TritonDFT proposes an agentic framework and associated benchmark for Density Functional Theory workflows for materials sciences. The authors present the associated DFTBench for evaluating capabilities in automated DFT across 67 materials, with a focus on Pareto-aware methods that focus on cost-accuracy tradeoffs. They show improvements over DREAMS, an open-source agentic framework.

Critical strengths
  1. TritonDFT take the important perspective of incorporating Pareto-aware approaches. Balancing cost vs accuracy is an important tradeoff that is often more practically useful than optimizing just accuracy, as real-world workflows have cost budgets.

  2. DFTBench appears valuable for future development. It incorporates a range of material types and complexities.

  3. Lightweight probing and reference-guided refinement produce improvements over single-pass parameter selection.

Critical weaknesses
  1. The automatic parallelization experiment raises concerns about the agent architecture. The LLM is tasked with configuring several coupled, parallelization controls and often activates too many of them. Guardrails improve performance, suggesting that the reliability may come from manually encoded expert policy rather than agent reasoning.

  2. The lightweight probing improves pass rate, but total cost is hard to interpret. Figure 5c indicates six probes account for a substantial fraction of compute, while it is unclear if Table 2 includes all exploratory and failed probe runs in its cost factors.

Questions for authors

What feedback and stopping criteria does lightweight probe refinement use?

1 comment
AuthorsOpenReview ↗

Response to Review ZogY

We thank the reviewer for the constructive feedback and for recognizing the value of Pareto-aware optimization and lightweight probing.

Weakness 1: Automatic Parallelization and Expert Guardrails

We agree that expert knowledge plays an important role in reliable scientific workflows. In automatic parallelization, expert-designed guardrails prevent inefficient combinations of coupled parameters. These constraints complement LLM reasoning rather than replace it.

Importantly, TritonDFT does not rely solely on fixed expert policies. In particular, lightweight probing enables the agent to refine its parameter choices based on actual execution feedback. The agent adapts its decisions to different materials and computational requirements. Our ablation results demonstrate the effectiveness of this feedback-driven refinement.

Weakness 2: Lightweight Probing Overhead and Cost Accounting

We agree that lightweight probing introduces additional computational overhead. However, this overhead is relatively small in most cases. In particular, lightweight probes are typically short, one-shot test runs that take considerably less time than full DFT calculations. Although Fig. 5(c) shows that repeated probing can increase computational cost, this does not represent the typical overhead.

We will add more detailed runtime measurements and clarify the cost accounting in Table 2, including the overhead of exploratory and failed probes, to provide a more complete picture of the end-to-end execution cost.

Question 1: What feedback and stopping criteria does lightweight probe refinement use?

Lightweight probe refinement is guided by the LLM's reasoning rather than fixed stopping rules. The agent examines execution feedback, including convergence behavior, computational cost, and potential numerical issues. Based on these observations, it determines whether the current parameters are sufficient or require further refinement. The agent stops probing when it judges that the configuration provides an acceptable balance between accuracy and computational cost. In our experiments, we additionally impose a maximum budget of six lightweight probes to prevent unbounded exploration and control computational overhead.

Thank you again for your valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.