acceptodds
Under review as a conference paper at ICLR 2027

Pairformer-H: An E(3)-Equivariant Pairformer for Electronic Hamiltonian Prediction

Abstract

Learning electronic Hamiltonians from atomic structures offers a route to electronic-property prediction without costly self-consistent density functional theory (DFT) calculations. However, accurately capturing how the surrounding atomic environment shapes orbital couplings between atom pairs remains a central challenge. We introduce Pairformer-H, a sparse E(3)-equivariant architecture that incorporates the Hamiltonian’s orbital-block structure into pairwise information exchange through learned orbital-operator compositions. Along each path connecting a target pair through an intermediate atom, the model composes two pair operators with an environment-dependent operator acting on the intermediate orbital space. This composition contracts over intermediate orbital indices, producing a contribution with the orbital dimensions of the target pair. Adaptive aggregation combines contributions across periodicity-consistent paths to update the target-pair representation. Across seven datasets without spin–orbit coupling and two with spin–orbit coupling, Pairformer-H achieves Hamiltonian matrix-element mean absolute errors of 0.166–0.356 meV and the lowest reported errors among the evaluated baselines on five without spin–orbit coupling datasets and both with spin–orbit coupling datasets. Relative to the best baseline for each dataset, errors decrease by 51.7% on bilayer BiTe without spin–orbit coupling and 35.4% on bulk \-WTe. Representative band structures, densities of states, and optical responses closely agree with DFT references. These results show that aligning a network’s internal interactions with the orbital-block structure of the Hamiltonian yields accurate predictions across materials and spin settings.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Electronic-structure Hamiltonian prediction, Equivariant neural networks, AI for materials science

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vMzcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript presents Pairformer-H, an E(3)-equivariant neural network architecture for predicting electronic Hamiltonians of periodic materials. Its main innovation is the incorporation of orbital-operator matrix paths into the internal message-passing process, where information from paths through intermediate atoms is aggregated to update atom-pair features. The method is physically motivated and achieves competitive prediction accuracy. The ablation studies also demonstrate that the matrix-path module significantly improves the performance of the baseline architecture.

However, several important issues remain to be addressed. First, the computational complexity and additional overhead of the network have not been systematically evaluated, leaving its scalability to larger orbital basis sets and more complex systems unclear. Second, although the ablation studies demonstrate the effectiveness of the proposed module, they do not sufficiently establish its advantages over existing higher-order equivariant networks. In particular, the comparison with MACE-H does not adequately control for the angular momentum channels of the hidden representations and model capacity, making it difficult to attribute the observed performance differences specifically to the matrix-path mechanism. Furthermore, the current benchmarks mainly focus on configuration-dependent predictions within individual material systems, with limited validation of cross-material and out-of-distribution generalization.

Overall, Pairformer-H introduces a meaningful approach to machine-learning Hamiltonians. However, more systematic evaluations of computational efficiency, fair comparisons with existing methods, and broader generalization tests are needed to convincingly establish its advantages over existing approaches.

Critical strengths
  1. The network architecture has a clear physical motivation. The orbital-operator matrix-path mechanism introduces a physics-informed inductive bias tailored to electronic Hamiltonians. By explicitly accounting for the effects of intermediate atoms and orbital hybridization on the coupling between target atom pairs, the proposed approach incorporates the orbital structure of the Hamiltonian more directly into the internal message-passing process.

  2. Pairformer-H achieves competitive Hamiltonian prediction accuracy. Among the compared models, Pairformer-H achieves the lowest prediction errors on five of the seven datasets without spin–orbit coupling (SOC) and on both datasets with SOC. These results demonstrate the effectiveness of the proposed architecture.

  3. The authors provide validation of downstream physical properties. Beyond evaluating errors in Hamiltonian matrix elements, the authors further calculate band structures, densities of states, and interband optical conductivity. The results show that the predicted Hamiltonians reproduce the main features of the electronic structure reasonably well, providing additional support for the physical reliability of the method.

Critical weaknesses
  1. The computational complexity and practical cost of the matrix-path mechanism have not been sufficiently analyzed. The proposed matrix-path mechanism explicitly enumerates two-hop atomic paths through intermediate atoms. Assuming an average local coordination number of z, the number of candidate two-hop paths scales approximately as z^2, whereas conventional local message-passing networks primarily process edges, whose number scales approximately linearly with z. In addition, each matrix path involves explicit matrix contractions in orbital space. For local orbital bases of comparable size, the computational cost of a naive matrix multiplication scales cubically with the number of local orbitals. Consequently, the matrix-path module may introduce substantial computational and memory overhead for highly coordinated systems, larger cutoff radii, larger orbital basis sets, heavy elements, and calculations involving SOC. The authors should systematically evaluate how computational time and memory consumption scale with system size, local coordination number. In particular, an analysis of the trade-off between prediction accuracy and computational cost is needed to establish whether the proposed architecture offers a practical overall advantage.
  2. The advantages of the matrix-path mechanism in representing higher-order many-body interactions remain unclear. The current ablation studies demonstrate the effectiveness of the matrix-path module within Pairformer-H but do not establish its advantages over existing higher-order equivariant networks, such as MACE-H. It remains unclear whether the improved accuracy originates from greater representational capacity or a more effective physics-informed inductive bias. The authors should clarify this distinction and provide more direct comparisons to establish the specific advantages of the proposed mechanism.
  3. The comparison with MACE-H does not adequately control for differences in model representation capacity. In the no-SOC benchmarks for bilayer Bi2Te3 and bulk Td-WTe2, the hidden-layer irreducible representations of MACE-H are truncated at l=2, whereas Pairformer-H retains higher-order angular momentum channels. Consequently, the observed accuracy improvements may partly reflect differences in representation capacity rather than the matrix-path mechanism itself. A more controlled comparison, with comparable hidden-layer angular momentum channels, model sizes, and computational budgets, is needed to establish the advantages of Pairformer-H.
Questions for authors
  1. How does the computational cost of the matrix-path module scale with local coordination number, orbital basis size, and system size? Can the authors provide direct comparisons of runtime, GPU memory consumption, and prediction accuracy with MACE-H, SLEM, and DeepH-E3?

  2. What specific advantages does the matrix-path mechanism offer over existing higher-order equivariant message-passing approaches? Is the accuracy improvement primarily due to greater representational capacity or to a more effective physics-informed inductive bias?

  3. Can Pairformer-H generalize to materials or chemical compositions not included in the training data? The current benchmarks primarily involve material-specific training. Could the authors provide cross-material or out-of-distribution tests to demonstrate the transferability of the proposed architecture?

6 comments

We sincerely thank the reviewer for the detailed and constructive comments and for the time and effort devoted to evaluating our work. The reviewer’s valuable suggestions and insights will help us further improve the clarity of our study.

  1. We thank the reviewer for raising the issue of computational complexity and GPU memory consumption. Regarding Weakness 1 and Question 1, we acknowledge that orbital-operator matrix paths implemented using naive matrix multiplication incur greater computational costs than conventional local message-passing networks.

    We theoretically analyze the time and space complexity of the orbital-operator matrix paths. For time complexity in the noSOC setting, let the system contain NN atoms, with an average local coordination number zz, an actual local orbital dimension dd used in matrix multiplication, and PP valid paths satisfying the target-edge and periodic closure constraints. For each path i→k→ji\rightarrow k\rightarrow j, the model computes Pikj=(BikGk)Bkj.P_{ikj}=(B_{ik}G_k)B_{kj}.

    When all three operators are dense d×dd\times d matrices, each path involves two standard matrix multiplications. The computational complexity of these contractions is therefore TBGB=O(Pd3).T_{\mathrm{BGB}}=O(Pd^3).

    For real-valued operators, counting each multiplication and addition as a separate floating-point operation, the forward contractions require approximately 4Pd34Pd^3 floating-point operations. Regarding local coordination number, let zkinz_k^{\mathrm{in}} and zkoutz_k^{\mathrm{out}} denote the in-degree and out-degree of atom kk in the neighbor graph, respectively, where neighbors include periodic images within the cutoff. Before filtering by the closure constraints, the number of candidate two-hop paths is

    Pcandidate=∑k=1Nzkinzkout,P≤Pcandidate.P_{\mathrm{candidate}}=\sum_{k=1}^{N}z_k^{\mathrm{in}}z_k^{\mathrm{out}},\qquad P\leq P_{\mathrm{candidate}}.

    For reciprocal neighborhoods, this number becomes ∑kzk2\sum_k z_k^2. When coordination numbers are approximately uniform, the number of candidate paths is approximately Nz2Nz^2, yielding a candidate-path computational complexity of O(Nz2d3)O(Nz^2d^3). Thus, for fixed NN and dd, the candidate-path computational cost grows approximately quadratically with the local coordination number.

    Regarding orbital basis size, with the neighbor graph and valid path set fixed, the two standard matrix multiplications per path make the computational cost grow cubically with the actual orbital dimension dd, giving TBGB=O(d3)T_{\mathrm{BGB}}=O(d^3) for fixed PP.

    Regarding system size, when atomic density, cutoff radius, basis set, and the distribution of local environments remain unchanged, the average numbers of neighbors and valid paths per atom remain of the same order as system size increases. The total number of paths therefore increases approximately linearly with the number of atoms NN. Consequently, for fixed dd, the BGB contractions have approximately linear computational cost with system size:

    P=O(N),TBGB=O(Nd3)=O(N).P=O(N),\qquad T_{\mathrm{BGB}}=O(Nd^3)=O(N).

    For space complexity, memory consumption mainly arises from node and edge operators, output accumulation matrices, and path intermediates required for backpropagation. Let EE denote the number of stored atom-pair blocks. The node and edge operators and output matrices occupy O((N+E)d2)O((N+E)d^2) memory, while automatic differentiation may retain O(Pd2)O(Pd^2) matrix intermediates across the paths. The matrix-storage complexity of a single orbital-operator matrix-path layer during training is therefore

    SBGB,train=O((N+E+P)d2).S_{\mathrm{BGB,train}}=O\left((N+E+P)d^2\right).

    When path intermediates dominate memory consumption, this cost is O(Pd2)O(Pd^2), scaling linearly with the number of paths and quadratically with the orbital dimension.

    The SOC and noSOC settings have the same asymptotic complexity orders. However, the spin-dependent branch in the SOC setting uses complex-valued 2d×2d2d\times2d operators, and the model involves both real and complex arithmetic. Consequently, the expanded spin dimension and complex arithmetic introduce additional computational and memory costs relative to the noSOC setting.

We compared the runtime and GPU memory consumption of DeepH-E3, SLEM, MACE-H, and Pairformer-H on the bilayer Bi$_2$Te$_3$ noSOC dataset using an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. The results are presented in the table below.

ModelTrainable parameters (M)Hamiltonian MAE (meV) ↓Training time (s/step)Training time (s/epoch)Peak GPU memory (GiB)
DeepH-E30.7685780.57390.45293.053.10
SLEM4.5655720.61140.22947.322.98
MACE-H1.5744340.51590.33268.694.35
Pairformer-H9.4907000.24300.938192.2228.72

As shown in the table, SLEM is the fastest model. It improves computational efficiency through local SO(2) convolutions and strictly localized message passing, although this speedup comes at some cost to prediction accuracy. DeepH-E3 and MACE-H achieve a balance between speed and accuracy, whereas Pairformer-H runs more slowly but achieves the highest prediction accuracy. Future work will focus on improving the computational efficiency of Pairformer-H and reducing its GPU memory consumption.

  1. We thank the reviewer for pointing out shortcomings in our experimental setup. Regarding Weaknesses 2 and 3 and Question 2, we have briefly introduced the advantages of Pairformer-H’s orbital-operator matrix-path mechanism in Section 1 of the main text. Below, we describe its specific advantages over existing higher-order equivariant message-passing methods in detail.

    (i) Explicitly constructing coupling paths through intermediate atoms for each target atom pair. Existing higher-order equivariant message-passing methods first aggregate neighborhood information and construct higher-order equivariant node features through tensor products. They then update atom-pair representations using the endpoint node features, existing edge features, and geometric information. Consequently, intermediate atoms primarily influence target-pair updates through aggregated environmental representations.

    In Pairformer-H’s orbital-operator matrix paths, for a target atom pair i,ji,j, the model constructs individual matrix paths through intermediate atoms kk:

    Pikj=BikGkBkj.P_{ikj}=B_{ik}G_kB_{kj}.

    Here, BikB_{ik} and BkjB_{kj} are the orbital operators of the two constituent edges, and GkG_k is the orbital operator of the intermediate atom. This allows the intermediate atom’s environmental information to directly influence the target-pair update, enabling the model to learn how the surrounding environment affects interatomic orbital couplings. The element identities, orbital features, and local environments of different intermediate atoms contribute to the update through their respective matrix paths, with adaptive aggregation determining the weight of each path. Thus, for the same target atom pair, the model can account for the effects of different intermediate atoms separately and adjust its orbital-coupling prediction accordingly.

    (ii) Organizing internal network interactions according to the orbital-block structure of the Hamiltonian. Existing higher-order equivariant message-passing networks construct higher-order many-body features through tensor products of equivariant features and angular-momentum coupling, and expand angular-momentum channels to accommodate Hamiltonian-block prediction. These methods primarily combine environmental information in the feature space of irreducible representations before mapping atom-pair features to orbital matrix blocks. In contrast, Pairformer-H constructs orbital operators from atomic and atom-pair representations during feature interactions and contracts over the intermediate atom’s orbital indices through BikGkBkjB_{ik}G_kB_{kj}. The rows and columns of the matrix generated by each path correspond to the orbitals at the two endpoints of the target atom pair. Thus, whereas existing methods learn many-body environments by combining equivariant features, our matrix paths further organize these interactions according to the connections between orbital spaces. The advantage is that the orbital-block structure of the Hamiltonian directly guides intermediate-layer updates, providing a structural prior aligned with the prediction target for learning multicenter orbital couplings.

    To address the unfairness in the baseline configurations, we first increased the parameter counts of SLEM, DeepH-E3, and MACE-H to values comparable to that of Pairformer-H. Meanwhile, we increased the maximum hidden-layer angular momentum of MACE-H to 4. The specific configurations of the four models are shown below.

    ModelHidden-layer irreducible representations (irreps)Maximum hidden-layer angular momentum lmax⁡l_{\max}Backbone layersTrainable parameters (M)
    SLEM107x0e+107x1o+54x2e+27x3o+27x4e449.515205
    DeepH-E3260x0e+130x1o+65x2e+32x3o+32x4e+16x5o539.456159
    MACE-H150x0e+150x1o+150x2e+150x3o+150x4e439.448156
    Pairformer-H64x0e+32x1o+16x2e+8x3o+8x4e439.490700

    We retrained DeepH-E3, SLEM, and MACE-H on the bilayer Bi$_2$Te$_3$ noSOC dataset and compared their runtime and GPU memory consumption with those of Pairformer-H. We used an NVIDIA RTX 4090D 48GB GPU for these experiments. The comparison results are shown below.

    ModelTrainable parameters (M)Hamiltonian MAE (meV) ↓Training time (s/step)Training time (s/epoch)Peak GPU memory (GiB)
    DeepH-E39.4561590.49930.43289.129.36
    SLEM9.5152050.50060.29159.954.35
    MACE-H9.4481560.38420.603123.9318.52
    Pairformer-H9.4907000.23700.938192.2228.72

    As shown in the table, increasing the representational capacity of the three models indeed improves their prediction accuracy, but their accuracy remains below that of Pairformer-H.

Reviewer_78rNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose Pairformer-H, an E(3)-equivariant model for predicting electronic Hamiltonians. They introduce orbital-shell token encoding and a pair-update operation that composes learned orbital operators through intermediate atoms, inspired by pairformer architecture. The model also incorporates spin-orbit coupling (SOC) by learning corrections to pretrained noSOC model. Validation uses material-specific models on seven noSOC and two SOC datasets, with improvements over the evaluated baselines on five noSOC and both SOC datasets.

Critical strengths

1. Orbital-space pair updates

The model updates each pair representation by composing two pair operators with an intermediate atom operator and aggregating contributions across valid paths. The operation contracts over intermediate orbital indices. Ablation study on this orbital-operator matrix paths in Table 10 further supports the contribution of it to the performance gain.

2. Extension to spin-orbit coupling

The model extends a pre-trained noSOC model through separate spin-independent and spin-dependent branches. The corrections are then assembled in the Pauli basis while preserving joint spin-orbital equivariance and Hermiticity, enabling prediction of spin-resolved Hamiltonians.

Critical weaknesses

1. Limited evaluation in terms of dataset, properties, and baselines

While the suggested architecture shows performance gain over DeepH-E3, MACE-H, and SLEM, the experiments use material-specific models evaluated on random splits of small datasets and only hamiltonian MAE is quantitatively compared. Considering the amount of other works on electronic hamiltonian prediction, it is not clear to assess the efficacy of the suggested architecture because of limited dataset, properties, and baselines.

Questions for authors
  1. Could you evaluate the model on a larger and more diverse materials dataset, with a shared model and a split that tests transferability across material systems? Comparisons with method like NextHAM [1] would help establish whether the improvements extend beyond the current small material-specific settings.

  2. If broader materials experiments are computationally prohibitive, could you evaluate the architecture on a molecular Hamiltonian dataset such as QH9, including an unseen molecule or size OOD split? This Would enable comparisons with a wider range of recent models such as QHNetV2 [2], QHFlow2 [3], and SAIL [4].

  3. Please provide quantitative comparisons beyond Hamiltonian MAE to downstream properties like band energies, band gaps, or HOMO-LUMO and orbital energies for molecular systems, and atomic energy and forces. Comparisons of these properties would provide further evidence of the performance and practical utility.

[1] Advancing Universal Deep Learning for Electronic-Structure Hamiltonian Prediction of Materials. ICLR. (2026)

[2] Efficient prediction of so (3)-equivariant hamiltonian matrices via so (2) local frames. arXiv. 2506.09398

[3] Machine Learning Hamiltonians are Accurate Energy-Force Predictors. ICML. (2026)

[4] Transferable SCF-Acceleration through Solver-Aligned Initialization Learning icml. ICML. (2026)

3 comments

We sincerely thank the reviewer for the detailed and constructive comments and for the time and effort devoted to evaluating our work. The reviewer’s valuable suggestions and insights will help us further improve the clarity of our study.

  1. We thank the reviewer for raising the question of model generalization. Regarding Weakness 1 and Questions 1 and 2, unfortunately, our current computational resources do not allow us to complete a large-scale cross-material evaluation within a short period. To further evaluate the architecture on larger and more diverse datasets, we have added molecular Hamiltonian prediction experiments. We use QH9 Stable-ID and QH9 Stable-OOD and train a separate model for each split to assess the architecture’s in-distribution and out-of-distribution generalization capabilities. Regarding the comparison methods suggested by the reviewer, the publicly released QH9 models of QHFlow2 rely on an initial Hamiltonian for residual prediction, which differs from our setting of predicting Hamiltonians directly from molecular structures. We have not found corresponding pretrained weights for models trained without an initial Hamiltonian. SAIL primarily targets self-consistent field (SCF) acceleration and does not directly report Hamiltonian MAE on QH9 Stable-ID and Stable-OOD. Given our current computational resources and the revision timeline, we have not included these two methods in this quantitative comparison. We compare Pairformer-H with QHNet and QHNetV2, and the results are presented in the table below.

    ModelStable-ID Hamiltonian MAE (μHartree) ↓Stable-OOD Hamiltonian MAE (μHartree) ↓
    Pairformer-H73.2769.72
    QHNet76.3172.11
    QHNetV231.5022.97

    As shown in the table, Pairformer-H outperforms QHNet but remains less accurate than QHNetV2. These supplementary experiments aim to evaluate the architecture’s in-distribution and out-of-distribution generalization on large and diverse datasets. The results demonstrate that the new Pairformer-H architecture can generalize across large and diverse datasets while achieving high prediction accuracy.

  2. We thank the reviewer for the valuable suggestions. Regarding Question 3, we have added comparisons of downstream property errors across 10 materials datasets. We used the Hamiltonian matrices predicted by Pairformer-H, DeepH-E3, MACE-H, and SLEM to calculate band energies and band gaps, and quantitatively evaluated each model’s prediction errors against DFT reference results. SLEM was included only in the noSOC comparisons. For the test set of each dataset, we used the same overlap matrices and downstream calculation settings for the predicted and DFT reference Hamiltonians. We calculated the band energy MAE and the absolute band gap error for each configuration, then averaged these values to obtain the band energy MAE and band gap MAE reported in the tables. The band energy MAE results are presented in the table below.

    DatasetPairformer-H MAE (meV) ↓DeepH-E3 MAE (meV) ↓SLEM MAE (meV) ↓MACE-H MAE (meV) ↓
    Monolayer graphene noSOC0.45560.64520.64710.6137
    Bilayer graphene noSOC1.81922.27263.35091.4174
    Monolayer MoS₂ noSOC0.57450.99921.10741.7633
    Bilayer bismuthene noSOC0.60721.37181.88946.1060
    Bilayer Bi₂Se₃ noSOC1.02294.22494.40013.8336
    Bilayer Bi₂Te₃ noSOC1.24032.426412.78281.4840
    Bulk TdT_{\mathrm d}-WTe₂ noSOC2.76195.26924.50604.2055
    Bilayer bismuthene SOC2.82591.7303—1.5593
    Bilayer Bi₂Se₃ SOC5.28364.8489—2.3527
    Bilayer Bi₂Te₃ SOC4.57584.4611—6.8526

    The band gap MAE results are presented in the table below.

    DatasetPairformer-H MAE (meV) ↓DeepH-E3 MAE (meV) ↓SLEM MAE (meV) ↓MACE-H MAE (meV) ↓
    Monolayer graphene noSOC0.67710.54940.53440.5658
    Bilayer graphene noSOC1.58431.46861.74890.6260
    Monolayer MoS₂ noSOC1.16120.97941.19641.1744
    Bilayer bismuthene noSOC0.65211.51292.07762.2625
    Bilayer Bi₂Se₃ noSOC2.01433.58673.88501.5202
    Bilayer Bi₂Te₃ noSOC1.46512.85117.80712.7019
    Bulk TdT_{\mathrm d}-WTe₂ noSOC3.92936.63304.59253.3986
    Bilayer bismuthene SOC0.97150.5582—0.6056
    Bilayer Bi₂Se₃ SOC3.90711.6196—1.7717
    Bilayer Bi₂Te₃ SOC4.74763.3427—1.4568

For molecular systems, we calculated the MAEs of occupied orbital energies, HOMO energies, LUMO energies, and HOMO–LUMO gaps relative to DFT reference values for Pairformer-H and QHNet on the complete test sets of QH9 Stable-ID and QH9 Stable-OOD. Each error metric is averaged over molecules. Both models use the same reference overlap matrices, and errors are consistently evaluated against the DFT energy levels in the original full basis. For QHNetV2, we have not yet obtained the official implementation and corresponding trained model weights needed to reproduce its results from the public repository provided by the authors. Therefore, we are unable to calculate its downstream property errors using the same test sets and energy-level calculation settings. The QH9 Stable-ID results are presented in the table below.

PropertyPairformer-H MAE (μHartree) ↓QHNet MAE (μHartree) ↓
Occupied orbital energies483.63884.65
HOMO energy728.69815.72
LUMO energy1512.251414.86
HOMO–LUMO gap2008.031995.48

The QH9 Stable-OOD results are presented in the table below.

PropertyPairformer-H MAE (μHartree) ↓QHNet MAE (μHartree) ↓
Occupied orbital energies574.15697.26
HOMO energy543.44566.08
LUMO energy2422.692523.64
HOMO–LUMO gap2364.382359.98

Overall, Hamiltonian MAE and downstream property errors exhibit similar trends across the evaluated datasets. In the materials experiments, some test sets are relatively small, and large errors in individual configurations may noticeably affect the mean. In the molecular experiments, the test sets are relatively large, and Pairformer-H’s downstream property prediction results on QH9 Stable-ID and Stable-OOD are broadly similar. This is also generally consistent with the conclusion that Pairformer-H achieves slightly better Hamiltonian prediction accuracy than QHNet on these two data splits.

Thank you for conducting the additional experiments and for sharing the new results. However, I feel that my main concerns are only partially addressed, and the rebuttal still does not provide clear evidence that Pairformer-H offers a meaningful advantage in either architectural effectiveness or predictive performance.

My main concerns were: (1) whether the proposed architecture remains competitive beyond small, material-specific benchmarks, and (2) whether it can accurately predict practically meaningful downstream properties (e.g., band gaps, energies, forces, and SCF acceleration), which are more directly relevant to the utility of Hamiltonian prediction.

The additional QH9 results partially address the first concern, but they do not show a clear advantage of Pairformer-H. Pairformer-H performs only slightly better than QHNet, which was introduced in 2023, while QHNetV2 achieves substantially lower Hamiltonian MAE on both Stable-ID (31.50 vs. 73.27 μHartree) and Stable-OOD (22.97 vs. 69.72 μHartree).

I also do not find the reasons for excluding QHFlow2 and SAIL fully convincing.

The authors state that QHFlow2 relies on an initial Hamiltonian, but I do not think this difference alone is sufficient to exclude it from comparison. If the use of an initial Hamiltonian is considered important, the same initialization could also be incorporated into Pairformer-H. Alternatively, if retraining under that setting is computationally expensive, Table 13 of QHFlow2 already reports results without initialization, including Hamiltonian MAE and several downstream properties, which could serve as a direct reference point. In particular, QHFlow2 achieves a Stable-ID Hamiltonian MAE of 25.50 μHartree without initialization, compared with 73.27 μHartree for Pairformer-H.

The fact that SAIL primarily focuses on SCF acceleration does not make it irrelevant as a Hamiltonian-prediction baseline. SAIL uses QM9 with the same size-based split corresponding to QH9 Stable-OOD, so the evaluation setup is directly comparable. Since pretrained checkpoints are publicly available, evaluating SAIL on QH9 should mainly require inference rather than retraining. Therefore, I do not think its focus on SCF acceleration is a sufficient reason to exclude it from comparison.

Regarding the second concern, I appreciate the newly added downstream evaluations. However, they also do not show a consistent advantage of Pairformer-H. In particular, Pairformer-H does not consistently outperform the existing baselines in band energy or band gap, and several materials have lower band-gap errors with other methods. Since electronic properties near the frontier bands are directly relevant to many practical applications of Hamiltonian prediction, I think these results weaken rather than strengthen the evidence for a clear practical advantage.

I appreciate the additional experiments and clarifications, but the new results do not establish competitiveness against stronger recent methods, the exclusion of relevant recent comparisons remains insufficiently justified, and the downstream results do not show a consistent practical advantage. Therefore, I would like to maintain my original score.

Reviewer_cCAFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors modify pairformer such that the orbital interactions between two atoms are well represented and include equivariance constraints. Notably the design enables learning both no spin-orbit coupling and a modifier layer which learns a spin-orbit coupling. Most of the paper is extrapolation on this architecture, but they authors also introduce a few benchmark examples with spin-orbit couplings computed as an extension of an existing benchmark. Their method performs SOTA compared to others they compare with.

Critical strengths
  • The authors provide a fairly clear description of a two novel architectures one for spin-free and one with spin-coupled representations. The performance is generally the best among models they compare with, except on two simpler cases. Their optical response matches DFT nearly exactly.
  • The authors produce a small extended dataset for benchmarking with spin orbit coupling. It is small, but it's a good start. Especially since they began with an existing dataset and a model to compare to.
Critical weaknesses
  • There are limited benchmark examples to compare to, especially for the spin orbit coupling examples.
Questions for authors
  • It was not fully obvious to me, you are training the model on each example... right? it isn't trained generally? This should be specified clearly as a limitation.
  • You ultimately train using what seems to me to be the the element-wise error, although there is a complex weighting. The justification for this loss could use some more extrapolation. Furthermore, do other losses such as the WALoss apply in this setting? Would it positively affect your results to use it? My understanding is that element-wise losses are ideal for learning hamiltonians because low elementwise MAE does not imply that you will get a good approximation to the eigenvalues and vectors of the true hamiltonian.
  • Manasa Kaniselvan, et. al. 2026's model and follow up work are applicable across the periodic table, despite being trained in a non-periodic setting. Although direct comparison can be hard due to different functionals, I would think it would make sense to mention this work.
2 comments

We sincerely thank the reviewer for the detailed and constructive comments and for the time and effort devoted to evaluating our work. The reviewer’s valuable suggestions and insights will help us further improve the clarity of our study.

  1. We thank the reviewer for pointing out the issues in our paper. To address the insufficient number of SOC datasets mentioned in Weakness 1, we have added experiments on the bilayer bismuthene SOC dataset, expanding the SOC evaluation to three material datasets. The results are presented in the table below.

    ModelHamiltonian MAE(meV)↓
    DeepH-E30.2537
    MACE-H0.2356
    Pairformer-H0.2213

    Pairformer-H continues to achieve the best prediction accuracy on the bilayer bismuthene SOC dataset, further demonstrating that this architecture can achieve highly accurate Hamiltonian prediction in SOC systems. The corresponding results have been added to Section 3.2 of the main text.

    Regarding Question 1, we train a separate model for each material dataset and evaluate it on independent test configurations of the same system. This was unclear in the original manuscript. We have clarified this setup in Section 3.1 of the main text. In Section 5, we acknowledge the lack of cross-material generalization evaluation as a limitation and prioritize such evaluation for future research. Given limited computational resources, we cannot complete a large-scale cross-material evaluation within a short period. We added molecular Hamiltonian prediction experiments to evaluate the architecture on larger, more diverse data. We use QH9 Stable-ID and QH9 Stable-OOD, two splits of QH9-Stable, which contains 130,831 stable molecular configurations. Stable-ID uses a random split to evaluate predictions for unseen molecules from the same distribution. Stable-OOD splits molecules by total atom count, with 3–20, 21–22, and 23–29 atoms in the training, validation, and test sets, respectively, to evaluate size extrapolation to larger molecules. We train a separate model on each split to assess the architecture’s in-distribution and out-of-distribution generalization. The table below compares our Pairformer-H with QHNet and QHNetV2.

    ModelStable-ID Hamiltonian MAE(μHartree)↓Stable-OOD Hamiltonian MAE(μHartree)↓
    Pairformer-H73.2769.72
    QHNet76.3172.11
    QHNetV231.5022.97

    As shown in the table, Pairformer-H outperforms QHNet but remains less accurate than QHNetV2. These supplementary experiments aim to evaluate the architecture’s in-distribution and out-of-distribution generalization on large-scale, diverse data. The results demonstrate that the proposed Pairformer-H architecture can generalize on large-scale, diverse datasets while achieving high prediction accuracy.

  2. We thank the reviewer for the valuable suggestions. Regarding Question 2, we use element-wise mean squared error (MSE) as the loss function for the following reasons.

    (i) Element-wise MSE directly supervises the prediction target. Our model predicts the Hamiltonian represented in a given localized orbital basis. Element-wise MSE directly constrains its valid matrix elements, including intra-atomic and interatomic orbital couplings, thereby providing supervision for the entire target operator. Unlike fitting a particular spectrum or property alone, this approach helps preserve the matrix information required for various downstream electronic-structure calculations.

    (ii) Element-wise MSE can be computed directly on the predicted atom-pair matrix blocks, without assembling the full reciprocal-space Hamiltonian or solving its eigenvalue problem at each training step. It also requires no additional reference eigenstate labels. This makes it a direct and computationally efficient baseline training objective for Hamiltonian prediction in periodic systems.

    (iii) We also follow established practices in this field. Methods such as DeepH-E3, SLEM, and MACE-H use element-wise error as their loss function.

    We did not use WAloss in the original experiments for the following reasons.

    (i): The publicly released DeepH-E3 data used in this work do not include the corresponding reference overlap matrices. Although the original paper reports some DFT calculation settings, the available information does not allow us to verify that overlap matrices recomputed for WAloss would use exactly the same basis functions and numerical settings as the original Hamiltonian calculations.

    (ii): Compared with using element-wise MSE alone, WAloss additionally evaluates the projected Hamiltonian error in the reference eigenstate basis. This introduces extra matrix operations and reference-state data loading, increasing the training time per epoch.

  1. We thank the reviewer for the valuable suggestion. Regarding Question 3, we have added citations to the 2026 work by Manasa Kaniselvan et al. and subsequent work in Section 4 of the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.