- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents MALOQ, a framework for training and running MLH models on large atomic systems. Combined with a quantum transport solver, the authors simulate a MoS₂/HfO₂ transistor and report a >150× speedup over a DFT+QT pipeline.
Critical strengths
- The problem of bridging ML Hamiltonians and device-scale quantum transport is important and relevant to TCAD.
- The systems work is solid, with scaling experiments on up to hundreds of GPUs and micro-benchmarks for each component.
Critical weaknesses
The related issues will be elaborated in the "Questions" section.
Questions for authors
- I am less familiar with quantum transport and device simulation, but I am familiar with the literature on machine-learned Hamiltonians. The contribution is primarily one of systems engineering and application, and it is unclear how much transferable knowledge the paper offers. The backbone is taken directly from HELM. Kernel fusion, block packing, and graph partitioning (METIS, edge-wise) are standard techniques, and distributed GNN inference for MLIPs already exists. The paper does not clearly articulate what is specific to MLH that makes these solutions non-trivial, nor does it compare against existing distributed approaches in terms of communication volume or scaling. As a result, the main novelty is first to combine these pieces for MLH at scale, which is valuable but incremental from an ML-methods perspective.
- Large-scale DFT is indeed challenging, but linear-scaling DFT methods have also been developed. Compared with such methods, what advantages does MALOQ offer in terms of efficiency and accuracy?
- For systems too large for DFT to provide reference results, can the authors provide any form of validation, for example against experimental data, or via indirect checks such as convergence of key observables with system size?
- Is training on large graphs truly necessary for the systems studied here? Could the authors compare against a locality-based, small-cell training strategy (e.g., DeepH-style) to show that distributed large-graph training is genuinely required?
6 comments
(Q1.1) The scope of our work ranges from computational innovations to real deployment in currently relevant scientific applications: We want to underline that the distributed graph implementation required for H prediction is only a sub-set of what our paper covers. MALOQ provides an integrated computational framework spanning several aspects of large-scale ML Hamiltonian learning and its deployment in scientific applications. What we present ranges from efficient distributed implementations over massive edge-wise graphs (Section 3.3), accelerated data processing pipelines over large-scale molecular datasets (Appendix F), demonstration of strong scaling performance across both large molecules (Appendix E) and much larger materials systems than have been treated in any previous work (Fig 4), connection to an advanced quantum transport modeling pipeline (Methodology in Appendix A), and an end-to-end application in an industry-relevant transistor modeling workflow (Section 5).
(Q2) Linear scaling DFT is demonstrably slower here, and generally does not provide an advantage across all material systems: We used the linear scaling DFT algorithm of CP2K to recompute the Hamiltonian matrix of structures made of 2,088 and 4,176 atoms (see the table below). It has the same accuracy as DFT. As compared to the standard O(N3) solver of DFT, the time with the linear scaling algorithm increases by a factor of 4x (93 sec per iteration with linear-scaling instead of 23 sec with the O(N3) approach), which corresponds to 2.6 node hours (NHs) instead of 0.64 NHs for 100 SCF iterations (the number of iterations is model-independent in our case). Since inference on the 2,088-atom structure with MALOQ takes 2e-3 NHs, at the same accuracy, the speed up we obtain with ML compared to linear scaling DFT is in the order of 1,300x (3 orders of magnitude).
The reason why linear scaling DFT is slower than conventional DFT for our structures is that such algorithms exploit a very specific property present in large enough structures and with a significant bandgap. If these conditions are satisfied, the corresponding density matrix becomes sparse, and linear scaling algorithms can take advantage of this characteristic. In many device applications, which include metallic layers or small-gap semiconductors, linear scaling algorithms may not show any computational advantage or even break down. We will include this discussion in a revised version of the paper since this is an important point that may not be widely known.
| # atoms | DFT, 1 SCF iter (NH) | LS-DFT, 1 SCF iter (NH) | Speedup of LS-DFT over DFT |
|---|---|---|---|
| 2,088 | 0.00639 | 0.0258 | 0.25× (4.0× slower) |
| 4,176 | 0.0239 | 0.160 | 0.15× (6.7× slower) |
(Q3) Device-level trends in downstream transport modeling observed for smaller structures hold true for the largest ones. The MoS2/oxide structures used here are a subset of a larger applications-focused direction, and since the submission of this paper, MALOQ has been deployed on structures at the 100k atom scale on similar geometries, matching the physical dimensions of fabricated transistors under consideration as future technology nodes in the semiconductor industry. While a direct comparison with experimental data is not yet conducted, we have clear evidence that our results are sensible and interpretable.
We simulated MoS2/oxide nanoribbons with increased width w (out-of-plane axis in Fig. 6(b)) between 1.5 and 12 nm, predicted their Hamiltonian matrix with MALOQ, computed the electronic current that flows through them, and normalized this current with w. From experiments, we expect that the normalized current would increase at small w and then saturate, i.e., becomes width-independent. This is exactly the behaviour we observe in our simulations, and the length scale of saturation can be quantified and will be compared with experimental measurements. This demonstrates convergence of a 'key downstream physical observable' with increasing system size.
(Q4) Please see our response to Q1, where we explain why large-graph training cannot be circumvented.
We are interested to hear your opinions after our response, and look forward to a reply.
[1] Zhanghao Zhouyin, Zixi Gan, Shishir Kumar Pandey, Linfeng Zhang, and Qiangqiang Gu, Learning local equivariant representations for quantum operators ICLR 2025. URL https://arxiv.org/pdf/2407.06053v1
[2] Chen Hao Xia, Manasa Kaniselvan, Alexandros Nikolaos Ziogas, Marko Mladenovic, Rayen Mahjoub, Alexander Maeder, and Mathieu Luisier. Learning the electronic hamiltonian of large atomic structures, ICML 2025. URL https://arxiv.org/abs/2501.19110.
We are waiting to hear your opinions on our comments. Our responses to your original review are further summarized below:
(1) We articulated the differences between MLIPs and MLH which necessitated our work and make it non-trivial, eg the edge-wise (vs. node-wise) nature and higher spherical degree of the data.
(2) We showed that linear scaling DFT is slower than standard DFT algorithms for our atomic structures. This is not unexpected, and originates from the nature of linear scaling DFT, which takes advantage of properties (wide bandgap, sparsity) that are not generally present across all material systems.
(3) To offer the "indirect checks such as convergence of key observables with system size" suggested, we described our indirect validation of reproducing experimentally-relevant trends from scaling the transistor channel widths. To further directly validate our work against DFT, in our response to reviewer EE76 we also included a full IV-curve comparison.
(4) We clarified why the training structures cannot be made smaller without losing inference accuracy, and that this is what necessitates distributed training.
We look forward to further exchanges during the discussion period.
Thank you for writing a thoughtful review. We have tried to structure our response below to stress that (1) Hamiltonian prediction at scale is a highly relevant application (as we submitted to the track 'applications to the physical sciences'), (2) the challenges involved in doing so are significant and fundamentally different than previously encountered by MLIPs, (3) and that our objective is to design the computational innovations to tackle them, and demonstrate how a new applications of ML in the physical sciences (specifically, semiconductor device modeling) has been enabled as a result.
(Q1) ML Hamiltonian prediction has a fundamentally different computational structure: We realize that this is important information that should have been in the main paper (and not in the appendix).
First, we want to preface our response with a general point about applications: Many, if not most of the “big ticket” directions for Hamiltonian prediction, involve structures with a large number of atoms, from electronic materials engineering to semiconductor device modeling. However, due to inherent computational difficulties (explained below), moving to larger systems is not straightforward. The community’s efforts in this direction started with small molecule or crystalline material benchmarks, such as MD17, QH9, and crystalline MoS2 structures in the first DeepH work. Before MALOQ, the few MLH models which attempted to scale up to a large number of atoms either (1) gave up important features in the ML architecture or (2) remained limited to modest system sizes. SLEM [1] and Ref. [2] both turned to “local” approaches, with a single message-passing/interaction layer. Other models like DeepH, QHflow, UniHamGNN, or HELM, all carefully select the size of their training systems to fit within the memory of a single GPU. Practically, the challenge of generalizing to significantly larger numbers of atoms, as required for practical scientific applications, has not yet been addressed.
The reason Hamiltonian prediction poses a different set of challenges than MLIPs boils down to a key point: MLIPs are a node-wise, scalar/vector prediction problem, while ML Hamiltonians take the form of an edge-wise, tensor-prediction challenge. In MLIPs, we predict a scalar (energy) per structure or a vector (force) per atom. Conversely, in ML Hamiltonians, each edge, including self-edges, carries an orbital-pair block that decomposes into a set of irreps up to lmax = 6-8, depending on the orbital basis chosen. This means:
We need to include long range interactions during training in order to generalize to arbitrary sizes during inference. If two atoms interact with each other, the corresponding nodes are connected in the underlying graph neural network, and their interaction is represented by a block in the Hamiltonian matrix. In realistic systems, atoms located 10A apart can still interact with each other (see Fig. G.1). Without having ever ‘seen’ such long-range interactions, a model cannot predict them. Meanwhile, small errors in matrix elements compound and create larger errors in the eigenvalues, so artificially truncating the matrix by imposing smaller cutoffs corrupts downstream physical quantities, e.g., transmission functions, total energies, density-of-states, or electronic currents. In practice, we can't escape the necessity of training on large graphs.
ML Hamiltonian models need to consider much larger interaction cutoffs than MLIPs. This higher cutoff leads to significantly increased edge/node ratios compared to MLIPs, where typically cutoffs are restricted to 6A (UMA). For reference, we present in the table below how the edge:node ratios would scale with interaction cutoffs for one of the training structures we used for the example in Section 5:
Together, these differences between ML Hamiltonian and MLIP prediction translate into massive data volumes that exceed the memory of a single GPU node, even for moderately sized systems (~2000 atoms), and a complex implementation to handle dense, edge-wise communication patterns. This is what makes the problem challenging and not comparable to what is encountered in MLIPs.