- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
This paper proposes HyPI, a method for learning higher-order interactions from resting-state fMRI data. It aims to identify brain interactions that cannot be explained by pairwise relationships. The method compares the activity of groups of brain regions with a pairwise maximum-entropy model. It then uses statistical tests and reliability checks to select hyperedges. These interactions are added to a frozen pairwise prediction model through a residual branch.
The authors also propose three criteria to evaluate higher-order interactions: statistical reliability, additional predictive value, and whether the model uses the correct subject-specific interactions. They evaluate the method on synthetic data and four clinical fMRI datasets. The reported results show improved classification accuracy and stronger evidence of higher-order interaction use compared with the tested methods.
Critical strengths
-
The research question is meaningful. The paper aims to distinguish whether higher-order interactions contain information beyond pairwise relationships, whether they improve prediction, and whether the model actually uses this information.
-
The evaluation framework is valuable. The authors go beyond classification accuracy by introducing capacity-matched controls and counterfactual interventions to examine whether the model depends on subject-specific higher-order interactions.
Critical weaknesses
-
The paper is difficult to follow. Throughout the manuscript, the authors introduce many concepts, technical terms, and mathematical notations without sufficiently explaining their roles in the overall framework and how they support the main scientific claims. The transitions between different parts are often unclear, making it easy to lose track of the main argument. The lengthy appendices add many technical details but do not sufficiently help readers understand the overall logic.
-
Insufficient theoretical and experimental support for the proposed certification protocol. The paper proposes P1–P3 as criteria for certifying higher-order information use. However, I do not think the theoretical analysis and experiments provide sufficient support for this protocol. The theoretical results mainly establish properties of individual components, but do not clearly justify why the three criteria together can establish genuine higher-order information use. For example, Appendix C.3 acknowledges that the temporal-null test does not directly test pairwise irreducibility. The experiments evaluate HyPI using the proposed criteria, but do not adequately validate the criteria themselves under known ground-truth conditions.
-
Incomplete discussion of related work. The paper overlooks two relevant studies: Zhan et al. (NeurIPS 2025) on higher-order brain network modeling, and Liu et al. (Nature Methods, 2025) on the effects of different pairwise interaction measures. Although Santoro et al. (Nature Communications, 2026) is cited, its systematic comparison of higher-order methods is not discussed sufficiently. A clearer discussion of these studies would help establish the contribution of HyPI and justify its choice of pairwise representations.
Questions for authors
-
Could the authors explain more clearly why the main components of HyPI are needed and why they are combined in this way? In particular, how do the design choices in Section 3.4 contribute to the paper's main objectives?
-
What theoretical arguments support the use of P1–P3 together as a certification protocol? How does P1 establish pairwise irreducibility when Appendix C.3 acknowledges that the temporal-null test does not directly test this hypothesis?
-
How are the synthetic datasets generated, including the planted interactions and labels? Do these experiments provide evidence that the proposed certification protocol can correctly distinguish genuine higher-order information use from cases where it is absent?
-
The full certification comparison with confidence intervals is reported for ABIDE (AAL116), while related analyses are presented for other settings. Could the authors clarify whether the joint certification results are consistent across the remaining datasets and parcellations?
-
Liu et al. (Nature Methods, 2025) systematically compared 239 pairwise interaction statistics and showed that their choice can substantially affect downstream results. Given these findings, how robust are HyPI's reported higher-order predictive advantages and certification results to the choice of pairwise representation?
9 comments
Response to NoaZ · Comment 2/5
Q2 & W2. What justifies using P1–P3 together, and how does P1 establish pairwise irreducibility?
Response.
P1–P3 together test one null, and each rules out an explanation that the other two cannot. P1 defines irreducibility by the score and uses the temporal null only to calibrate it.
(a) Target null and its connection to conditional independence.
We added to Section 3.5 the null that P1–P3 test together:
- Higher-order use means that the profile carries label information that pairwise statistics do not, so the null is .
- Conditional randomization would test it by resampling profiles from . This conditional is unknown, so we give each subject the profile of a pairwise-matched donor.
- The capacity-matched Null is then a permute-and-relearn control, and the Counterfactual is a matched form of model reliance.
- For a Bayes-optimal predictor and exact replacement, equals .
The protocol is thus related to conditional randomization but is not an exact conditional randomization test, so certification is evidence relative to the chosen profiles, surrogates, learner, and matching.
(b) Why the three criteria are needed together.
Each rules out an explanation that the other two cannot:
- P1 rules out profiles explained by pairwise statistics or estimation artifacts. Without it, a model could pass P2 and P3 with an auxiliary input computed from connectivity.
- P2 rules out gains from added capacity or from re-optimizing the pairwise model. Without it, a model can depend on an input (P3) that adds no label information, as MvHo-IB does in most settings.
- P3 rules out a model that ignores whose profile it receives. P2 compares models trained on true and reassigned profiles, so a gain can arise during training although the final predictions do not change when the profile is replaced. P3 tests the fixed checkpoint directly.
(c) The decision rule.
A trained predictor can also gain from pairwise information that its backbone underuses, as the synthetic negative control of Q3 shows. We thus added , the risk of the same branch trained on surrogate-scan profiles, which keep each scan's spectra and connectivity but remove higher-order structure, so that (Section 3.5).
- Certification requires the 95% lower bounds of and to exceed zero, which makes it an intersection–union test over four comparisons that share the bootstrap resamples and need no adjustment.
- Under the idealization of (a), for the surrogate-scan profile , so a positive value still implies .
- With the fitted models fixed, HyPI is certified under this rule in all eight settings, and the rule lowers its bounds by at most 0.03.
(d) How P1 establishes pairwise irreducibility.
We added to Section 3.2 that the score defines irreducibility and the temporal null calibrates it:
- is the KL divergence from a group's empirical joint-state distribution to its projection onto the pairwise log-linear family. By the Pythagorean relation, it is the divergence that no pairwise model can remove, and it is zero if and only if the distribution lies in the pairwise family.
- Finite scans, autocorrelation, and binarization produce positive divergences even for a stationary Gaussian process, so the temporal null calibrates the score against them.
- Non-Gaussian marginals, nonstationarity, or motion could also exceed the null, so we tested them directly (Appendix F.2). Profiles on spectrum-preserving surrogates, with or without each ROI's amplitude distribution, are never certified. Under the submitted rule, certification holds under drift-preserving and time-varying nulls, FD censoring, motion-balanced classes, and same-site donors (lower bounds of 0.23 to 1.36).
Response to NoaZ · Comment 3/5
Q3 & W2. Does the protocol separate higher-order use from its absence under known ground truth?
Response.
Yes. HyPI profiles are certified in at most 1% of null cohorts and in 80% of positive-control cohorts. We changed the synthetic setup of the submission (50 subjects, 50 time points) to longer, fMRI-like scans, and added this validation to Section 4.2 and Appendix G.
(a) Generator.
- Each cohort has 240 subjects, 30 ROIs, and 200 time points of autoregressive background activity (lag-one coefficient 0.5).
- We plant four triplets and two quadruplets. In each planted group, the binary state of one member agrees with the parity of the other members with a subject-specific probability drawn from , so every planted group has zero pairwise correlation.
- Hyperedges are fixed to the planted groups to isolate the protocol from discovery.ixed to the planted groups to isolate the protocol from discovery.
(b) Scenarios.
In the positive control, the label shifts the interaction strength of three planted groups. In four null scenarios, the planted interactions are label-free:
- (i) The label shifts the correlation of an ROI pair outside every planted group.
- (ii) The label shifts the correlation of two non-target members inside three planted groups.
- (iii) The label sets this correlation to with a random sign for one class and to 0 for the other, which a linear predictor on connectivity cannot read.
- (iv) The label is drawn from a logistic model on the scan connectivity of these member pairs.
Scenarios (i)–(iii) act on population parameters, so holds for the population pairwise structure, but interaction estimates could still carry information about the latent coupling beyond the scan connectivity. In (iv), holds exactly for the model input. We run 200 cohorts per null scenario and 100 for the positive control.
(c) Results.
- HyPI profiles are certified in at most 1% of the cohorts of every null scenario, consistent with the 2.5% level of each one-sided bound, and in 80% of the positive-control cohorts (see Table R-N2 below).
- The table also shows a limitation of the submitted rule. Profiles built from the connectivity of the same groups, for which holds exactly, pass that rule in 56.0% to 76.5% of the cohorts with label information inside the groups, because the frozen predictor on all 435 connectivity entries underuses the few informative ones.
- We thus added (Q2(c)). These connectivity profiles equal their surrogate-scan profiles, so and the revised rule excludes them by construction, which is not calibration evidence. The calibration evidence is the rate for HyPI profiles, which keep 80% in the positive control because their surrogate-scan branch gains at most 1.5% of over the pairwise predictor.
- In the full pipeline with discovery, 10 of 12 cohorts are certified when the planted interactions carry the label, and HyPI is 19.4% more accurate than the pairwise predictor. When the same interactions are present but only connectivity carries the label, all six cohorts retain the interactions and none is certified, so discovery alone does not produce certification.
Table R-N2. Percentage of synthetic cohorts certified (95% Clopper–Pearson intervals) under the two-way rule of the submission, for surrogate-scan profiles, and under the three-way rule. Higher-order structure carries no label information in all but the last row, and holds exactly for the model input in (iv). In the two-way and three-way columns, lower is better (↓) for the null rows and higher is better (↑) for planted interactions. The surrogate-scan column should be low in every row.
| Label carried by | Profile | Better | Two-way | Surrogate-scan | Three-way |
|---|---|---|---|---|---|
| Connectivity of other regions | HyPI | ↓ | 0.5 [0.0, 2.8] | 0.0 [0.0, 1.8] | 0.0 [0.0, 1.8] |
| Group connectivity | ↓ | 0.0 [0.0, 1.8] | 0.0 [0.0, 1.8] | 0.0 [0.0, 1.8] | |
| Coupling inside groups | HyPI | ↓ | 1.0 [0.1, 3.6] | 0.0 [0.0, 1.8] | 1.0 [0.1, 3.6] |
| Group connectivity | ↓ | 69.0 [62.1, 75.3] | 67.5 [60.5, 73.9] | 0.0 [0.0, 1.8] | |
| Sign-symmetric coupling | HyPI | ↓ | 0.0 [0.0, 1.8] | 0.5 [0.0, 2.8] | 0.0 [0.0, 1.8] |
| Group connectivity | ↓ | 76.5 [70.0, 82.2] | 77.0 [70.5, 82.6] | 0.0 [0.0, 1.8] | |
| Model input (iv) | HyPI | ↓ | 1.0 [0.1, 3.6] | 0.5 [0.0, 2.8] | 1.0 [0.1, 3.6] |
| Group connectivity | ↓ | 56.0 [48.8, 63.0] | 54.5 [47.3, 61.5] | 0.0 [0.0, 1.8] | |
| Planted interactions | HyPI | ↑ | 80.0 [70.8, 87.3] | 0.0 [0.0, 3.6] | 80.0 [70.8, 87.3] |
Response to NoaZ · Comment 1/5
We thank the Reviewer for the careful review and for finding the research question meaningful and the evaluation framework valuable.
We have uploaded a revised PDF with changes in blue. Below, AHO and DHO are in 10−3 nats, and a method is certified when the 95% lower bounds of both are above zero.
Q1 & W1. Why is each component needed, and how do the components fit together?
Response.
Each component serves one of three requirements, and the revision now states this mapping throughout.
(a) What changed for readability.
We revised the paper so that its logic is easier to follow:
(b) One argument.
A claim that a model uses higher-order information requires three things:
Each component of HyPI serves one of these (see Table R-N1 below).
Table R-N1. Role of each component of HyPI. Ablation values are accuracy (%) / AHO / DHO (point estimates, 10−3 nats) on ABIDE (AAL116). The full model gives 72.84 / 5.48 / 6.47, and higher is better (↑) for all three.
(c) Design of the residual branch.
By Bayes' rule,
logp(y∣S(2),FHO)=logp(y∣S(2))+logp(FHO∣S(2))p(FHO∣y,S(2)),
so optimal use of higher-order information is an additive log-space correction to the best pairwise predictor, and its expected reduction in log-loss is the conditional mutual information I(Y;FHO∣S(2)). HyPI follows this identity: