- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether, when the prediction error of a learned electronic Hamiltonian is small, the topological correctness of the subspace it occupies can be guaranteed. The authors construct smooth, bandgap-bearing examples in which topological discrepancies are concentrated within arbitrarily small regions, allowing the error in finite p-order integrals to be arbitrarily small. On the other hand, the paper links topological inconsistency to the consistency distance of the projection operators, and provides sufficient guarantees based on the consistency projection distance or the Hamiltonian error relative to a reference energy gap. Finite-point sampling requires additional coverage conditions and independently proven regularity bounds.
Controlled four-band learning experiments show that the band MAE may rank a set of topologically incorrect models ahead of correct ones. Real-World Hamiltonian Cases The paper distinguishes between topological sensitivity caused by structured perturbations and the loss of numerical or spectral validity. It also reports negative results: cyclic finite-point samplin shows no stable advantage within the same budget, and in exploratory 10 meV material studies, all 276 evaluations with validity at both ends preserved topological properties. Therefore, the contribution lies in distinguishing between average accuracy and strict guarantees, rather than estimating the prevalence of errors in models used in actual deployments.
Critical strengths
-
The paper clearly establishes a connection between the evaluation criteria and the downstream properties for which no guarantees can be provided. The localization construction intuitively demonstrates the limitations of finite-order errors, while the consistency conditions explain which stronger information is sufficient to provide guarantees. The paper distinguishes between families of smooth functions subject to no additional uniform constraints and cases with verifiable regularity bounds, while also acknowledging that positive stability tools stem from classical results.
-
The scope of the empirical conclusions is defined with considerable rigor. The paper distinguishes between valid topological inconsistencies and invalid invariant computations, performs numerical resolution checks, and reports both positive examples and null results. In particular, the authors retain the results from the 10 meV study that revealed no topological inconsistencies, as well as the results showing that ring sampling lacks stability advantages, without using these to imply general failure or a universal solution. These qualifications significantly enhance the credibility of the interpretation of the results.
Critical weaknesses
-
The failure labels in the evaluation benchmark need to be explained more clearly in the main text. Section 5 reports 75 failures out of 216 instances, while Appendix I.5 explains that the benchmark uses composite failure labels, which differ from inconsistencies with valid topologies. This distinction is important because the projection witness results do not apply to all bandgaps or numerical validity failures.
-
There is one clear editorial issue in the manuscript. The Ethics Statement on page 10 describes the distribution of language model responses, pre-trained language models, and registered representation sets, which is inconsistent with this Hamiltonian study and should be replaced with a statement that accurately describes this research. This affects confidence in the quality of the final review of the manuscript.
Questions for authors
-
Existing Failure Labels (Request for Clarification): Please specify in Section 5 which scenarios are covered by the composite failure label, and, based on the existing labels, briefly describe the proportion of the 75 failures attributable to valid topology inconsistencies versus validity-related failures. Please maintain a distinction between these and the conditions under which the projection witness results apply.
-
Numerical Validity (Request for Additional Explanation): The validity rules are provided in Appendix I.5. Could you add an explicit reference in Section 6 to clarify that 0.25 meV is an operational choice specific to the numerical settings used, which must be combined with convergence and conditional checks, rather than a universal physical threshold?
-
Ethical Statement (request for revision of the manuscript): Please replace the statement on page 10 regarding language model responses and representation groups with a statement describing the Hamiltonian data and research methods actually used in this paper.
5 comments
Response to Reviewer RTLq (2/3)
Q1 / W1 (continued). The benchmark conclusion is unchanged under either labeling (App. J.1, J.3):
| Instances | Labels | AUROC loop / stratified IID, | Difference [95% family-cluster interval] | One-sided |
|---|---|---|---|---|
| All 216 | historical | 0.8445 / 0.8456 | −0.0010 [−0.039, 0.042] | 0.51 |
| 198 valid at both endpoints | current status | 0.8470 / 0.8415 | +0.0055 [−0.031, 0.056] | 0.36 |
| 198 valid at both endpoints | historical | 0.8486 / 0.8450 | +0.0036 [−0.034, 0.056] | 0.42 |
Changes in the paper. Sec. 5, ll. 308–315:
... 75 of which are labeled failures by family-specific Chern-label disagreement rules. The labels do not fold in gap or conditioning failures. [...] 69 of the 75 failures are valid mismatches and none is an invalid prediction; the other six are unresolved, five of them the exact constructions of Sec. 3, recorded on a single grid (Appendix J.3). The witness of Proposition 2 applies to the valid mismatches.
App. I.5 (ll. 2793–2798) keeps the distinction you asked for:
This taxonomy is distinct from the historical family-specific topology-label disagreement used in the finite-query benchmark [...]. The pointwise witness theorem applies to two continuous, equal-rank valid bundles with different topology; it cannot automatically be applied to all 75 recorded label failures without those assumptions.
Q2. Could you add an explicit reference in Sec. 6 clarifying that 0.25 meV is an operational choice specific to the numerical settings, combined with convergence and conditioning checks, rather than a universal physical threshold?
Short answer. Done, in Sec. 6, in the Fig. 4 caption, and consistently with App. I.5. We also audited every stored Wilson flow with a stricter, Z2Pack-style gap-clearance criterion (App. J.11).
Details. The guardrail is one of four joint checks: gap above 0.25 meV, Kramers error below 0.01, Wilson-link singular value above , and agreement of the two finest Wilson-loop resolutions. Key outcomes of the clearance audit (criterion: clearance at least 0.3; every case is listed in App. J.11):
| Case | Gap-clearance outcome |
|---|---|
| ZrSiPt, NaCaTaTiO₆ | all planes pass at both endpoints |
| Ta₃AlC₂ | the four , planes pass; the adaptive flows pass |
| Ca₃Si₄Ir₄ | all planes fail, consistent with its invalid classification |
| Strained BiTeI, , | all 144 dense flows pass (minimum 0.42 / 0.49) |
| GeBi₄Te₇ mismatch records (6) | both endpoints dense-verified; minimum prediction clearance 0.497 |
The audit is reported in full, including the cases where a stored flow does not by itself establish its label under this stricter criterion; the stress-study counts keep that study's three-grid protocol of record, with the dense statuses reported alongside (App. J.11).
Changes in the paper. Sec. 6, ll. 341–344:
[...] with on the six time-reversal-invariant planes and a validity protocol: an operational 0.25-meV gap guardrail plus Kramers, link and two-finest-grid checks (Appendix I.5; Table 2).
Fig. 4 caption (l. 458):
the 0.25-meV line is the operational validity guardrail, not a phase boundary.
App. I.5 (ll. 2789–2792):
The exact 0.25-meV cutoff is an operational, conservative heuristic for the frozen grids. It is neither a universal validity threshold nor a physical phase boundary; the diagnosis is supported jointly by gap collapse, convergence behavior, Kramers/link checks, and resolution stability.
Response to Reviewer RTLq (3/3)
Q3 / W2. Please replace the Ethics Statement on page 10 with a statement describing the Hamiltonian data and methods actually used.
Short answer. Thank you for catching this; it is replaced.
Changes in the paper. Ethics Statement, ll. 548–552:
This work studies computational electronic Hamiltonians using controlled models, released electronic-structure data and released predictor artifacts. It involves no human participants or personal data. Data and software sources are identified, and no third-party model weights or raw datasets are redistributed.
You noted that this error affected confidence in the final review of the manuscript. For this revision we therefore re-verified every number in the submission against its source data. The resulting corrections are listed in full in the general response (part 2/3). Table 20 now gives, for each empirical result, its evidence, validity protocol and status.
The scope statements and null results you valued are kept.
| Kept in the revision | Where |
|---|---|
| The 10-meV study: 276 evaluations valid at both endpoints, 0 valid mismatches, 0 newly unsafe. Now extended to 20 meV, with still no valid mismatch | Sec. 7, ll. 424–426; App. H.3, Tables 17–18 |
| Loop-structured queries show no stable detection advantage at equal cost | Sec. 5, ll. 322–323; App. F, Table 13 |
| New: the same holds design by design | Sec. 5, ll. 316–317; App. F.3 |
| The stress counts locate vulnerable planes rather than estimate prevalence | Sec. 7, ll. 461–462 |
| All realistic-model controls reported in full | App. E.2, J.4, J.7 |
The equal-cost AUROC across budgets is unchanged (Table 13):
| Budget | Loop-structured | IID | Stratified IID | Random paths |
|---|---|---|---|---|
| 16 | 0.7647 | 0.7351 | 0.7096 | 0.7068 |
| 32 | 0.8663 | 0.7788 | 0.8398 | 0.8519 |
| 64 | 0.8445 | 0.8421 | 0.8456 | 0.8778 |
| 128 | 0.8880 | 0.9008 | 0.8789 | 0.8624 |
Scored design by design (16 designs at ), the mean AUROC is 0.589 (loop), 0.650 (IID), 0.647 (stratified IID) and 0.682 (random path). The loop-minus-stratified difference is positive in 4 of 16 designs on the 216 instances, and in 4 of 16 on the 198 valid at both endpoints (App. F.3).
The general response summarizes the evidence added for the other reviewers, including a released predictor that returns a converged wrong class on strained BiTeI (Sec. 6, Table 3).
If these changes address your questions, we would be grateful if you would consider raising your score.
Response to Reviewer RTLq (1/3)
Thank you for the careful and constructive review. You found that the paper "clearly establishes a connection between the evaluation criteria and the downstream properties for which no guarantees can be provided", and you wrote: "The localization construction intuitively demonstrates the limitations of finite-order errors" and "The scope of the empirical conclusions is defined with considerable rigor." Your request to separate valid topological mismatches from validity failures led to a full validity audit of the benchmark (Sec. 5; App. J.3). We kept, in the main text, the scope statements and null results you valued.
Q1 / W1. Please specify in Sec. 5 which scenarios the composite failure label covers, give the proportion of the 75 failures attributable to valid topological mismatches versus validity failures, and keep these distinct from the conditions of the projector witness.
Short answer. Done. Tracing the frozen benchmark code showed that the 75 labels are family-specific Chern-label disagreements. Gap, link and convergence failures were never folded in, so "composite" was the wrong word. A validity audit of both endpoints finds that 69 of the 75 are valid topological mismatches, none is an invalid prediction, and 6 are unresolved.
Details. Label rules and composition (App. J.1, Table 21):
Validity gates, fixed before evaluation and applied at both endpoints:
An endpoint recorded on a single grid counts as unresolved, not as passing. Results by family (Table 24):
No instance is an invalid prediction. Five of the six unresolved failures are the exact two-band constructions of Sec. 3: they were recorded on a single grid, and their Chern difference is known analytically. Current status over all 216: 70 valid mismatches, 128 valid agreements, 18 unresolved.