- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This manuscript presents Pairformer-H, an E(3)-equivariant neural network architecture for predicting electronic Hamiltonians of periodic materials. Its main innovation is the incorporation of orbital-operator matrix paths into the internal message-passing process, where information from paths through intermediate atoms is aggregated to update atom-pair features. The method is physically motivated and achieves competitive prediction accuracy. The ablation studies also demonstrate that the matrix-path module significantly improves the performance of the baseline architecture.
However, several important issues remain to be addressed. First, the computational complexity and additional overhead of the network have not been systematically evaluated, leaving its scalability to larger orbital basis sets and more complex systems unclear. Second, although the ablation studies demonstrate the effectiveness of the proposed module, they do not sufficiently establish its advantages over existing higher-order equivariant networks. In particular, the comparison with MACE-H does not adequately control for the angular momentum channels of the hidden representations and model capacity, making it difficult to attribute the observed performance differences specifically to the matrix-path mechanism. Furthermore, the current benchmarks mainly focus on configuration-dependent predictions within individual material systems, with limited validation of cross-material and out-of-distribution generalization.
Overall, Pairformer-H introduces a meaningful approach to machine-learning Hamiltonians. However, more systematic evaluations of computational efficiency, fair comparisons with existing methods, and broader generalization tests are needed to convincingly establish its advantages over existing approaches.
Critical strengths
-
The network architecture has a clear physical motivation. The orbital-operator matrix-path mechanism introduces a physics-informed inductive bias tailored to electronic Hamiltonians. By explicitly accounting for the effects of intermediate atoms and orbital hybridization on the coupling between target atom pairs, the proposed approach incorporates the orbital structure of the Hamiltonian more directly into the internal message-passing process.
-
Pairformer-H achieves competitive Hamiltonian prediction accuracy. Among the compared models, Pairformer-H achieves the lowest prediction errors on five of the seven datasets without spin–orbit coupling (SOC) and on both datasets with SOC. These results demonstrate the effectiveness of the proposed architecture.
-
The authors provide validation of downstream physical properties. Beyond evaluating errors in Hamiltonian matrix elements, the authors further calculate band structures, densities of states, and interband optical conductivity. The results show that the predicted Hamiltonians reproduce the main features of the electronic structure reasonably well, providing additional support for the physical reliability of the method.
Critical weaknesses
- The computational complexity and practical cost of the matrix-path mechanism have not been sufficiently analyzed. The proposed matrix-path mechanism explicitly enumerates two-hop atomic paths through intermediate atoms. Assuming an average local coordination number of z, the number of candidate two-hop paths scales approximately as z^2, whereas conventional local message-passing networks primarily process edges, whose number scales approximately linearly with z. In addition, each matrix path involves explicit matrix contractions in orbital space. For local orbital bases of comparable size, the computational cost of a naive matrix multiplication scales cubically with the number of local orbitals. Consequently, the matrix-path module may introduce substantial computational and memory overhead for highly coordinated systems, larger cutoff radii, larger orbital basis sets, heavy elements, and calculations involving SOC. The authors should systematically evaluate how computational time and memory consumption scale with system size, local coordination number. In particular, an analysis of the trade-off between prediction accuracy and computational cost is needed to establish whether the proposed architecture offers a practical overall advantage.
- The advantages of the matrix-path mechanism in representing higher-order many-body interactions remain unclear. The current ablation studies demonstrate the effectiveness of the matrix-path module within Pairformer-H but do not establish its advantages over existing higher-order equivariant networks, such as MACE-H. It remains unclear whether the improved accuracy originates from greater representational capacity or a more effective physics-informed inductive bias. The authors should clarify this distinction and provide more direct comparisons to establish the specific advantages of the proposed mechanism.
- The comparison with MACE-H does not adequately control for differences in model representation capacity. In the no-SOC benchmarks for bilayer Bi2Te3 and bulk Td-WTe2, the hidden-layer irreducible representations of MACE-H are truncated at l=2, whereas Pairformer-H retains higher-order angular momentum channels. Consequently, the observed accuracy improvements may partly reflect differences in representation capacity rather than the matrix-path mechanism itself. A more controlled comparison, with comparable hidden-layer angular momentum channels, model sizes, and computational budgets, is needed to establish the advantages of Pairformer-H.
Questions for authors
-
How does the computational cost of the matrix-path module scale with local coordination number, orbital basis size, and system size? Can the authors provide direct comparisons of runtime, GPU memory consumption, and prediction accuracy with MACE-H, SLEM, and DeepH-E3?
-
What specific advantages does the matrix-path mechanism offer over existing higher-order equivariant message-passing approaches? Is the accuracy improvement primarily due to greater representational capacity or to a more effective physics-informed inductive bias?
-
Can Pairformer-H generalize to materials or chemical compositions not included in the training data? The current benchmarks primarily involve material-specific training. Could the authors provide cross-material or out-of-distribution tests to demonstrate the transferability of the proposed architecture?
6 comments
We compared the runtime and GPU memory consumption of DeepH-E3, SLEM, MACE-H, and Pairformer-H on the bilayer Bi$_2$Te$_3$ noSOC dataset using an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. The results are presented in the table below.
| Model | Trainable parameters (M) | Hamiltonian MAE (meV) ↓ | Training time (s/step) | Training time (s/epoch) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|
| DeepH-E3 | 0.768578 | 0.5739 | 0.452 | 93.05 | 3.10 |
| SLEM | 4.565572 | 0.6114 | 0.229 | 47.32 | 2.98 |
| MACE-H | 1.574434 | 0.5159 | 0.332 | 68.69 | 4.35 |
| Pairformer-H | 9.490700 | 0.2430 | 0.938 | 192.22 | 28.72 |
As shown in the table, SLEM is the fastest model. It improves computational efficiency through local SO(2) convolutions and strictly localized message passing, although this speedup comes at some cost to prediction accuracy. DeepH-E3 and MACE-H achieve a balance between speed and accuracy, whereas Pairformer-H runs more slowly but achieves the highest prediction accuracy. Future work will focus on improving the computational efficiency of Pairformer-H and reducing its GPU memory consumption.
-
We thank the reviewer for pointing out shortcomings in our experimental setup. Regarding Weaknesses 2 and 3 and Question 2, we have briefly introduced the advantages of Pairformer-H’s orbital-operator matrix-path mechanism in Section 1 of the main text. Below, we describe its specific advantages over existing higher-order equivariant message-passing methods in detail.
(i) Explicitly constructing coupling paths through intermediate atoms for each target atom pair. Existing higher-order equivariant message-passing methods first aggregate neighborhood information and construct higher-order equivariant node features through tensor products. They then update atom-pair representations using the endpoint node features, existing edge features, and geometric information. Consequently, intermediate atoms primarily influence target-pair updates through aggregated environmental representations.
In Pairformer-H’s orbital-operator matrix paths, for a target atom pair , the model constructs individual matrix paths through intermediate atoms :
Here, and are the orbital operators of the two constituent edges, and is the orbital operator of the intermediate atom. This allows the intermediate atom’s environmental information to directly influence the target-pair update, enabling the model to learn how the surrounding environment affects interatomic orbital couplings. The element identities, orbital features, and local environments of different intermediate atoms contribute to the update through their respective matrix paths, with adaptive aggregation determining the weight of each path. Thus, for the same target atom pair, the model can account for the effects of different intermediate atoms separately and adjust its orbital-coupling prediction accordingly.
(ii) Organizing internal network interactions according to the orbital-block structure of the Hamiltonian. Existing higher-order equivariant message-passing networks construct higher-order many-body features through tensor products of equivariant features and angular-momentum coupling, and expand angular-momentum channels to accommodate Hamiltonian-block prediction. These methods primarily combine environmental information in the feature space of irreducible representations before mapping atom-pair features to orbital matrix blocks. In contrast, Pairformer-H constructs orbital operators from atomic and atom-pair representations during feature interactions and contracts over the intermediate atom’s orbital indices through . The rows and columns of the matrix generated by each path correspond to the orbitals at the two endpoints of the target atom pair. Thus, whereas existing methods learn many-body environments by combining equivariant features, our matrix paths further organize these interactions according to the connections between orbital spaces. The advantage is that the orbital-block structure of the Hamiltonian directly guides intermediate-layer updates, providing a structural prior aligned with the prediction target for learning multicenter orbital couplings.
To address the unfairness in the baseline configurations, we first increased the parameter counts of SLEM, DeepH-E3, and MACE-H to values comparable to that of Pairformer-H. Meanwhile, we increased the maximum hidden-layer angular momentum of MACE-H to 4. The specific configurations of the four models are shown below.
Model Hidden-layer irreducible representations (irreps) Maximum hidden-layer angular momentum Backbone layers Trainable parameters (M) SLEM 107x0e+107x1o+54x2e+27x3o+27x4e4 4 9.515205 DeepH-E3 260x0e+130x1o+65x2e+32x3o+32x4e+16x5o5 3 9.456159 MACE-H 150x0e+150x1o+150x2e+150x3o+150x4e4 3 9.448156 Pairformer-H 64x0e+32x1o+16x2e+8x3o+8x4e4 3 9.490700 We retrained DeepH-E3, SLEM, and MACE-H on the bilayer Bi$_2$Te$_3$ noSOC dataset and compared their runtime and GPU memory consumption with those of Pairformer-H. We used an NVIDIA RTX 4090D 48GB GPU for these experiments. The comparison results are shown below.
Model Trainable parameters (M) Hamiltonian MAE (meV) ↓ Training time (s/step) Training time (s/epoch) Peak GPU memory (GiB) DeepH-E3 9.456159 0.4993 0.432 89.12 9.36 SLEM 9.515205 0.5006 0.291 59.95 4.35 MACE-H 9.448156 0.3842 0.603 123.93 18.52 Pairformer-H 9.490700 0.2370 0.938 192.22 28.72 As shown in the table, increasing the representational capacity of the three models indeed improves their prediction accuracy, but their accuracy remains below that of Pairformer-H.
We sincerely thank the reviewer for the detailed and constructive comments and for the time and effort devoted to evaluating our work. The reviewer’s valuable suggestions and insights will help us further improve the clarity of our study.
We thank the reviewer for raising the issue of computational complexity and GPU memory consumption. Regarding Weakness 1 and Question 1, we acknowledge that orbital-operator matrix paths implemented using naive matrix multiplication incur greater computational costs than conventional local message-passing networks.
We theoretically analyze the time and space complexity of the orbital-operator matrix paths. For time complexity in the noSOC setting, let the system contain N atoms, with an average local coordination number z, an actual local orbital dimension d used in matrix multiplication, and P valid paths satisfying the target-edge and periodic closure constraints. For each path i→k→j, the model computes Pikj=(BikGk)Bkj.
When all three operators are dense d×d matrices, each path involves two standard matrix multiplications. The computational complexity of these contractions is therefore TBGB=O(Pd3).
For real-valued operators, counting each multiplication and addition as a separate floating-point operation, the forward contractions require approximately 4Pd3 floating-point operations. Regarding local coordination number, let zkin and zkout denote the in-degree and out-degree of atom k in the neighbor graph, respectively, where neighbors include periodic images within the cutoff. Before filtering by the closure constraints, the number of candidate two-hop paths is
Pcandidate=∑k=1Nzkinzkout,P≤Pcandidate.
For reciprocal neighborhoods, this number becomes ∑kzk2. When coordination numbers are approximately uniform, the number of candidate paths is approximately Nz2, yielding a candidate-path computational complexity of O(Nz2d3). Thus, for fixed N and d, the candidate-path computational cost grows approximately quadratically with the local coordination number.
Regarding orbital basis size, with the neighbor graph and valid path set fixed, the two standard matrix multiplications per path make the computational cost grow cubically with the actual orbital dimension d, giving TBGB=O(d3) for fixed P.
Regarding system size, when atomic density, cutoff radius, basis set, and the distribution of local environments remain unchanged, the average numbers of neighbors and valid paths per atom remain of the same order as system size increases. The total number of paths therefore increases approximately linearly with the number of atoms N. Consequently, for fixed d, the BGB contractions have approximately linear computational cost with system size:
P=O(N),TBGB=O(Nd3)=O(N).
For space complexity, memory consumption mainly arises from node and edge operators, output accumulation matrices, and path intermediates required for backpropagation. Let E denote the number of stored atom-pair blocks. The node and edge operators and output matrices occupy O((N+E)d2) memory, while automatic differentiation may retain O(Pd2) matrix intermediates across the paths. The matrix-storage complexity of a single orbital-operator matrix-path layer during training is therefore
SBGB,train=O((N+E+P)d2).
When path intermediates dominate memory consumption, this cost is O(Pd2), scaling linearly with the number of paths and quadratically with the orbital dimension.
The SOC and noSOC settings have the same asymptotic complexity orders. However, the spin-dependent branch in the SOC setting uses complex-valued 2d×2d operators, and the model involves both real and complex arithmetic. Consequently, the expanded spin dimension and complex arithmetic introduce additional computational and memory costs relative to the noSOC setting.