- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AXON, which uses attention learned through masked reconstruction of fMRI signals as an edge-specific regularization prior for VAR connectivity estimation. Experiments cover connectome recovery, reproduction of empirical dynamics, and downstream prediction, with AXON achieving a mean behavioral prediction correlation of 0.210, compared with 0.169 for FC and 0.197 for VAR.
Critical strengths
-
AXON is simple and practically appealing, combining a learned attention backbone with each subject’s own timeseries through edge-specific regularization.
-
The evaluation spans animal tract-tracing connectomes, human simulations, lifespan cohorts, and multiple downstream tasks, providing complementary evidence for the approach.
-
The ablations on region embeddings, encoder layers, attention activations, and causal masks provide useful insights into the attention representations learned under the proposed setup.
Critical weaknesses
-
Step-1 may attribute to attention a gain that stems from population-level information. The pooled attention is learned across the pretraining cohort and is largely subject-invariant: with the BOLD input ablated, it still correlates with the original at r = 0.71–0.96 (Fig. 10B). FC and partial correlation, by contrast, appear to be estimated from individual held-out recordings. The mismatch is most consequential in the Step-1 simulations, where all subjects share the same structural connectome and differ only in noise seed (App. B.1). Population-average FC and a tuned regularized partial correlation estimated on the pretraining data are needed for a like-for-like comparison.
-
The contribution of the edge-specific prior is not isolated. Appendix D argues that because γ = 0 recovers VAR at the same λ₀, any difference “arises solely from the attention prior.” This does not follow. Under Eq. 5, each row of A sums to 1/P, so every entry is well below 1 and (A_ij + δ)^(−γ) is large. Unless A is rescaled, AXON therefore applies a much stronger overall penalty than VAR at the same λ₀, not merely a differently structured one. In addition, γ is tuned while VAR’s λ₀ does not appear to be. An independently tuned VAR, a uniform prior matched in penalty scale, and priors built from population FC or shuffled attention are needed to show that the edge-specific structure itself drives the gains.
-
The directionality claims need qualification. Figure 2B suggests higher direction (r) on two atlases but slightly lower d-accuracy on all three relative to VAR, while the animal directionality results claimed in the abstract are not reported in the presented evaluations.
-
The mechanistic conclusions extend beyond the evaluated setup. The ablations do not establish a fundamental limitation of self-attention, and unmasked patches from the same region remain available, weakening the argument that reconstruction must rely on other regions.
-
The downstream advantage over VAR is limited, and key evaluation details remain unclear. The gains are not significant for age or in five of eight behavioral categories, and brain-state balanced accuracy is identical to VAR at the reported precision. Of particular concern, each condition has its own pretrained encoder (l. 340); if brain-state features are extracted with the encoder matching each scan’s condition, label information enters the features. The split protocol and the configurations of several near-chance BFM baselines (e.g., 0.17–0.20 on 8-class decoding, vs. 0.97 for FC) also require clarification.
Questions for authors
-
Are Step-1 FC and partial correlation estimated from individual held-out recordings? How would population-average FC and tuned regularized partial correlation using the pretraining data compare?
-
Is (A) rescaled before Equation 7? Could the authors compare AXON with independently tuned VAR, a penalty-scale-matched uniform prior, and priors based on population FC or shuffled attention?
-
How are brain-state features extracted without knowing the test condition? Please also clarify the relationship between pretraining and the ten evaluation splits, family grouping, and the treatment of dependence across splits in statistical testing.
-
Could the authors provide numerical results and uncertainty estimates for Figure 2B, and clarify whether animal directionality was evaluated? How are connections with zero ground-truth asymmetry handled in d-accuracy?
-
Do the current ablations distinguish the effects of the reconstruction objective, patch resolution, and attention pooling on directionality? If not, could the conclusion be restricted to the evaluated setup?
-
Which checkpoints, input representations, spatial/TR adaptations, feature readouts, and probe-tuning procedures were used for each BFM baseline?
-
The animal direct (r) values in Figure 2A appear to differ from Table 9. Could the authors clarify whether the models, evaluation subsets, or aggregation procedures differ?
10 comments
Response to Q2 and W2, part 1 (2/8)
Q2: Is (A) rescaled before Equation 7? Could the authors compare AXON with independently tuned VAR, a penalty-scale-matched uniform prior, and priors based on population FC or shuffled attention?
W2: Unless A is rescaled, AXON therefore applies a much stronger overall penalty than VAR at the same λ₀, not merely a differently structured one. In addition, γ is tuned while VAR’s λ₀ does not appear to be.
Key summary: "AXON's gain over VAR may reflect a stronger overall penalty or unequal tuning rather than the edge-specific structure of the attention."
Response for Q2 and W2:
We thank for this critical comment. The Reviewer pointed out that AXON and VAR may differ not only in the edge-specific structure of the attention but also in the overall penalty strength and in tuning; it therefore needs to be shown that the edge-specific structure itself drives the gain once these two factors are matched. Below, we first clarify that the penalty strength was already matched in our implementation, which answers the Reviewer's question on rescaling; we then report the results of new analyses in which we tuned λ0 independently for each prior and, with both factors matched, varied only the edge-specific structure.
-
Penalty strength: In our implementation, the penalty strength was already matched, because A is rescaled before Eq. 7: Specifically, A is scaled to a maximum of 1, and the penalties are normalized by their off-diagonal mean, so that every prior has the same mean penalty as VAR; VAR thus corresponds to the scale-matched uniform prior. At the same λ0, AXON therefore does not penalize more strongly than VAR. We now state this rescaling explicitly below Eq. 7 (Section 3.2) and in Appendix D.1, and have removed the claim that any difference "arises solely from the attention prior" in the revised manuscript.
-
Tuning: In the original comparison, AXON and VAR shared the same λ0; with the mean penalty matched, the two estimators therefore differed only in how the penalty was distributed across connections. As the Reviewer requested, however, we further tuned λ0 independently for VAR and for every other prior (see the results in 3) below and Appendix D.1).
-
Edge-specific structure: With penalty strength and tuning matched, we compared four priors that have the same average penalty but differ in which connections are penalized more or less: (i) VAR, which applies the same penalty to every connection; (ii) shuffled attention, which uses the attention values but assigns them to random connections (1,000 permutations under each of 2 null models); (iii) a population-average FC prior, which uses the group-average FC of the pretraining subjects in place of the attention; and (iv) AXON, which uses the attention itself. We evaluated each prior on the directed simulations (100 subjects), scoring both the detection of direct connections (direct r, F1) and their direction (direction r, d-accuracy). The table below lists values for DK-68 / Glasser-360 / Schaefer-400, with higher values indicating better performance; † and ‡ mark values significantly lower and higher than AXON, respectively (tests in Appendix D.5). For shuffled attention, we report the best of the 2,000 permutations for each metric, i.e., the most favorable case for this control:
| method | direct r | F1 | direction r | d-accuracy |
|---|---|---|---|---|
| AXON | 0.633 / 0.680 / 0.695 | 0.535 / 0.498 / 0.498 | 0.186 / 0.172 / 0.163 | 0.594 / 0.611 / 0.614 |
| VAR (tuned) | 0.447† / 0.339† / 0.333† | 0.461† / 0.299† / 0.295† | 0.194‡ / 0.084† / 0.080† | 0.609‡ / 0.606† / 0.608† |
| Shuffled attention prior | 0.494 / 0.401 / 0.394 | 0.476 / 0.347 / 0.339 | 0.151 / 0.078 / 0.077 | 0.593 / 0.607 / 0.609 |
| Population-average FC prior | 0.624† / 0.576† / 0.695 | 0.534 / 0.425† / 0.502‡ | 0.179† / 0.120† / 0.153† | 0.598‡ / 0.609† / 0.614 |
Response to Q2 and W2, part 2 (3/8)
Response for Q2 and W2 (continued)
- Even with its own tuned λ0, VAR had lower direct r and F1 than AXON at all three atlases, and this pattern was largely preserved for the direction r and d-accuracy, except for DK-68. At this DK-68 parcellation, VAR achieved rather higher performance than AXON. We attribute this, at least in part, to the small number of regions: with 68 predictors per regression and about 1,200 time points, VAR is already well determined, leaving little room for an edge-specific prior. At Glasser-360 and Schaefer-400, where VAR must estimate roughly 130,000 and 160,000 coefficients, the problem becomes ill-posed, which is the regime AXON was designed for; here AXON roughly doubled VAR's direction r (0.172 vs 0.084 and 0.163 vs 0.080).
- Shuffled attention, with the same values in random positions, fell below AXON on every metric, even when its highest value among the 2,000 shuffles was taken (1,000 from each of 2 shuffling schemes, node permutation and row shuffle), indicating that the gain arises largely from where the values are placed rather than from the values themselves. The population-average FC prior matched AXON's F1 at DK-68 (0.534 vs 0.535) and was slightly but significantly higher at Schaefer-400 (0.502 vs 0.498), but had lower direction r at all three atlases. Thus, AXON's gain over VAR in recovering direct connections does not stem from tuning, and most of it is lost when the attention values are shuffled. Against the population-average FC prior, the advantage was clear for direction r but not for F1, suggesting that while a population-level prior captures part of the benefit for detecting connections, the attention prior yields better direction estimates. Revised PDF: Section 3.2 and Appendix D.1 (penalty scaling); Appendix D.5, Table 15.
Response to Q1 and W1 (1/8)
We thank the Reviewer for recognizing the simplicity and practicality of AXON, the breadth of our evaluation, and the insights from our ablations. We ran all the analyses the Reviewer requested and answered the 7 questions below in order. At each answer, we also discussed the corresponding "Critical Weakness" raised by the Reviewer.
Q1: Are Step-1 FC and partial correlation estimated from individual held-out recordings? How would population-average FC and tuned regularized partial correlation using the pretraining data compare?
W1: The pooled attention is learned across the pretraining cohort and is largely subject-invariant ... FC and partial correlation, by contrast, appear to be estimated from individual held-out recordings.
Key summary: "Step-1 may attribute to attention a gain that stems from population-level information."
Response for Q1 and W1:
The Reviewer is correct that FC and partial correlation in Fig. 2A were estimated from each held-out subject's recording, whereas the attention is learned from the pretraining subjects and, as shown in Fig. 10B (Fig. 11b in the revised PDF), is largely consistent across subjects. Fig. 2A alone therefore cannot determine whether the attention's advantage stems from its learned structure or simply from averaging across multiple subjects.
For a like-for-like comparison, we computed the population-average FC and the tuned regularized partial correlation as the Reviewer suggested (the latter with two standard estimators, Ledoit-Wolf shrinkage and graphical lasso) from the same 100 pretraining subjects whose attention was averaged, and scored each against the ground-truth connectome (values are DK-68 / Glasser-360 / Schaefer-400; higher is closer to the ground truth; † significantly lower than the attention, tests in Appendix C.5):
Even FC averaged over 100 subjects did not match the attention: the attention outperformed it on F1, AUC and direct r at all three atlases (e.g., DK-68 F1 0.695 vs 0.506), and exceeded both partial correlations on F1 and AUC at all three atlases. The one exception was direct r at DK-68, where the attention was similar to the Ledoit-Wolf partial correlation (0.800 vs 0.795) and slightly, though not significantly, below the graphical lasso (0.822). Averaging over subjects alone therefore does not explain most of the attention's advantage. Revised PDF: Appendix C.5, Table 11.