acceptodds
Under review as a conference paper at ICLR 2027

Beyond Site Agreement: Re-estimation for Brain Network Generalization

Abstract

Cross-site out-of-distribution (OOD) generalization in resting-state functional magnetic resonance imaging (rs-fMRI) often relies on learning task-discriminative representations from full-scan functional connectivity (FC) graphs and promoting invariance across source sites. However, FC graphs are estimated from finite, temporally correlated blood-oxygen-level-dependent (BOLD) sequences. Cross-site agreement therefore does not necessarily imply that predictive evidence remains supported under FC re-estimation within the same scan. In this paper, we propose Brain Network Re-estimation-Informed OOD Learning (BRIO), a framework that uses within-scan FC re-estimation to guide cross-site alignment. BRIO maps fullscan graphs and their re-estimates into consistently indexed connectome factors, enabling comparisons of their predictive contributions. It assesses re-estimation support from changes in these contributions relative to within-class subject variability and class separation. For each source-site pair and class, this task-calibrated support from both sites is combined with predictive relevance to form pairwise qualifications, which determine relative factor weights and overall alignment strength. Leave-one-site-out experiments on four real-world datasets (ABIDE, REST-metaMDD, SRPBS, and ABCD) show that BRIO consistently outperforms competitive baselines, with relative improvements of up to 3.8% in accuracy. These gains also persist under an alternative brain parcellation on ABIDE.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Brain Networks Analysis, Functional Connectome Re-estimation, Cross-site Out-of-Distribution Generalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_j4pBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BRIO for cross-site generalization of resting-state fMRI classifiers. Its central motivation is that agreement between source-site representations does not necessarily imply stability under re-estimation of functional connectivity from the same scan. BRIO generates FC re-estimates through temporal block resampling, maps them into consistently indexed connectome factors, and measures changes in each factor’s predictive contribution. Predictive relevance and task-calibrated re-estimation support jointly determine factor weights and the strength of class-conditional cross-site alignment. Leave-one-site-out experiments cover four datasets and an additional parcellation on ABIDE.

Critical strengths
  1. FC is estimated from finite, temporally dependent observations, yet cross-site alignment often treats it as fixed. BRIO evaluates changes in predictive contributions rather than raw edge variability alone, and combines stability with task relevance rather than assuming that every stable feature is useful. The shared factor representation makes these comparisons well defined within the model.

  2. The paper evaluates unseen-site performance across ASD, depression, and ADHD datasets, with an additional atlas on ABIDE. The controlled alignment-strength sweep, factor-retention analysis, block-length sensitivity, and reduced-scan experiments help test the proposed mechanism. In particular, the comparison at similar site discrepancy provides evidence that the method offers more than simply stronger alignment.

Critical weaknesses
  1. The ablation studies show that BRIO’s components are useful. However, it remains unclear whether the full weighting method works better than simpler approaches, such as using re-estimated FC graphs for data augmentation or encouraging consistent predictions across them. These methods should be compared using the same backbone and computational budget. The theoretical results explain the weights but do not guarantee better performance at unseen sites.

  2. Table 1 shows how performance varies across test sites, but not how consistently BRIO improves over the strongest baseline. The accuracy gains over CORE range from 1.17 to 2.27 percentage points. Please report paired comparisons on the same test sites, results from multiple random seeds, and details of model selection. The results should distinguish variation across sites from variation across random seeds.

Questions for authors
  1. How does BRIO compare with the same backbone trained using block-resampled FC augmentation or a prediction-consistency loss, with matched re-estimation and tuning budgets? This would help determine whether selective alignment is necessary for the reported gains.
  2. Can the authors provide paired per-site improvements over CORE and the strongest BRIO ablation, including uncertainty across repeated seeds? Please also specify the number of seeds and how they are aggregated in Table 1.
  3. How are the two validation sites selected in each LOSO fold, and what hyperparameter-search and checkpoint-selection procedures are used for BRIO and its baselines? Please clarify whether the same validation protocol and comparable tuning budgets are applied throughout.
  4. Equation 16 assigns higher support when the reference scale increases, which can occur through larger within-class variability as well as stronger class separation. Can the authors explain or empirically test when this normalization distinguishes genuinely stable predictive evidence from noisy, heterogeneous factors, particularly at small sites?
  5. Table 5 reports the default block as exactly 12.0 s on ABIDE, SRPBS and ABCD. Yet ABIDE and SRPBS have heterogeneous TRs across sites, and the block length is defined in time points at the fold level. A single duration in seconds is therefore ill-defined, and the paper’s TR-transfer argument does not hold as stated. How is the block length handled across sites with different TRs? Why is a single duration in seconds reported?
  6. Several values look unusually regular. In Table 4 (ID results), almost every mean and SD ends in the digit 5. BRIO’s SDs are smaller than every baseline’s in nearly every column. Could you confirm how they were computed and release the per-run logs and code?
14 comments
AuthorsOpenReview ↗

Response to Reviewer j4pB (1/10)

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: The ablation studies show that BRIO's components are useful. However, it remains unclear whether the full weighting method works better than simpler approaches, such as using re-estimated FC graphs for data augmentation or encouraging consistent predictions across them. These methods should be compared using the same backbone and computational budget. The theoretical results explain the weights but do not guarantee better performance at unseen sites.

A1: Thanks for your comment. To test whether the full weighting works better than simpler approaches under the same backbone and computational budget, we compared BRIO with five variants that use the same functional connectivity (FC) re-estimates for data augmentation, prediction consistency, or both on top of unqualified alignment, on the same network, and recorded the number of re-estimates and the training time of every method. In BRIO, the re-estimates are used only to compute the qualification. At each refresh, the calibration subjects are passed through the network with their re-estimates without gradients, and training mini-batches contain only full-scan FC. The variants are constructed as follows:

  • (1) Empirical risk minimization (ERM) trains with the site–class-balanced binary cross-entropy (BCE) only.
  • (2) ERM with re-estimate augmentation (ERM+Aug) feeds each sampled subject's full-scan FC together with kk of its re-estimates into the BCE.
  • (3) ERM with prediction consistency (ERM+Cons) adds to the BCE on the full-scan FC λcons\lambda_{\mathrm{cons}} times the binary Jensen–Shannon divergence between the predictions on the full-scan FC and on one re-estimate [1].
  • (4) Unqualified alignment (UA) is BRIO without qualification guidance (w/o QG) in the paper, which uses uniform factor composition and unit pair strength.
  • (5) UA+Aug+Cons adds both training signals to UA. It uses the same number of re-estimates as BRIO in the same fold, spread evenly over its training steps, on average 0.97 per sampled subject per step on ABIDE, and differs from BRIO only in the qualification.

All variants share the BRIO network, edge support, fold-level block rule, re-estimate generator, warm-up, optimizer, number of training epochs, validation sites and seeds. In each fold and seed, the hyperparameters shared with BRIO take the values selected for BRIO, and each variant tunes its own parameter over three values with the same selection metric, with k∈{1,2,4}k\in\{1,2,4\} for ERM+Aug and λcons∈{0.3,1,3}\lambda_{\mathrm{cons}}\in\{0.3,1,3\} for ERM+Cons and UA+Aug+Cons. A run of BRIO generates R=NcalKnrefreshR=N_{\mathrm{cal}}K n_{\mathrm{refresh}} re-estimates, where NcalN_{\mathrm{cal}} is the number of calibration subjects and nrefreshn_{\mathrm{refresh}} is the number of refreshes. We ran all methods, including BRIO, on ABIDE, REST-meta-MDD and SRPBS with three new seeds shared by all methods. The table below reports the leave-one-site-out (LOSO) area under the ROC curve (AUC) and accuracy (ACC) in %. For each seed, we average the metric over all target sites of a dataset, and each entry is the mean ± standard deviation (SD) of the three seed-level values. Re-est. is the number of re-estimates passed through the network per training run on ABIDE, and Wall-h is the wall-clock training time in hours of one configuration and seed on ABIDE. Both are averaged over folds, with eight runs sharing two GPUs.

MethodABIDE AUCABIDE ACCMDD AUCMDD ACCSRPBS AUCSRPBS ACCRe-est.Wall-h
ERM61.84±0.3456.09±0.4862.36±0.2758.21±0.3177.93±0.7172.61±0.6600.059
ERM+Aug62.45±0.2956.85±0.3762.88±0.3158.65±0.3478.67±0.4973.44±0.61219,5040.146
ERM+Cons62.71±0.3157.18±0.2762.87±0.2259.02±0.2978.79±0.5373.48±0.49158,5760.122
UA63.24±0.3657.30±0.4663.88±0.3359.69±0.3379.89±0.6274.64±0.5400.136
UA+Aug+Cons64.81±0.2659.68±0.4365.94±0.2861.12±0.2481.12±0.3976.54±0.59154,2860.197
BRIO67.35±0.3062.09±0.3268.90±0.1864.27±0.3083.62±0.4478.94±0.64154,2860.162
AuthorsOpenReview ↗

Response to Reviewer j4pB (2/10)

As shown in the above table, the full weighting outperforms the simpler approaches under a matched backbone and budget. UA+Aug+Cons, the strongest variant, uses the same number of re-estimates as BRIO and a longer training time of 0.197 hours on ABIDE, compared with 0.162 hours for BRIO, and BRIO exceeds it by 2.4 to 3.2 points in every column. ERM+Aug and ERM+Cons train slightly faster than BRIO and improve ERM by only 0.4 to 1.1 points, and ERM+Aug passes the most re-estimates through the network without a larger gain. Across the methods, performance grows with the use of alignment and qualification and does not follow the number of re-estimates or the training time: on all three datasets and both metrics, both training-signal variants outperform ERM, UA outperforms both of them, UA+Aug+Cons outperforms UA, and BRIO performs best.

To test whether this improvement holds across target sites, we computed the paired gain of BRIO over UA+Aug+Cons on the same target sites and seeds, with seeds paired across methods. For each dataset and metric, the table below reports the mean over target sites of the seed-averaged gain in percentage points and an unadjusted 95% confidence interval (CI) from a two-level bootstrap that resamples target sites and then seeds within each site with 10,000 replicates [2]. It also reports the number of target sites with a positive gain and the pp-value of an exact Wilcoxon signed-rank test over target sites, Holm-corrected across the six dataset–metric tests [3, 4]. SD${\mathrm{site}}$ is the SD of the seed-averaged gain across target sites, and SD${\mathrm{seed}}$ is the SD of the gain across seeds within a target site, averaged over target sites.

SettingMetricGain95% CISites > 0ppSD$_{\mathrm{site}}$SD$_{\mathrm{seed}}$
ABIDEACC2.41[1.52, 3.36]19/20<0.0011.981.36
ABIDEAUC2.54[1.43, 3.67]16/200.0022.571.27
REST-meta-MDDACC3.15[2.32, 4.01]24/25<0.0012.140.92
REST-meta-MDDAUC2.96[2.01, 3.90]23/25<0.0012.361.14
SRPBSACC2.40[0.71, 3.93]9/110.0192.601.28
SRPBSAUC2.50[1.06, 3.98]9/110.0142.541.31

As shown in the above table, the gain of BRIO over UA+Aug+Cons is positive on 80% to 96% of the target sites, its CI excludes zero in all six dataset–metric settings, and it remains significant after Holm correction with p≤0.019p\le0.019. The gain varies more across target sites than across seeds in every setting, which indicates that site differences account for most of the remaining variation. We have added these results to Appendix J.7 (Tables 8 and 9) of our manuscript.

The theoretical results characterize what the weights estimate and how they respond to re-estimation changes. The performance at unseen sites is established empirically, through the LOSO experiments, where each target site is excluded from training and model selection, and through the paired comparisons above. The evidence for unseen-site performance is empirical. It comes from the LOSO experiments, where each target site is excluded from training and model selection, and from the paired comparisons above.

AuthorsOpenReview ↗

Response to Reviewer j4pB (3/10)

W2: Table 1 shows how performance varies across test sites, but not how consistently BRIO improves over the strongest baseline. The accuracy gains over CORE range from 1.17 to 2.27 percentage points. Please report paired comparisons on the same test sites, results from multiple random seeds, and details of model selection. The results should distinguish variation across sites from variation across random seeds.

A2: Thanks for your comment. To show how consistently BRIO improves over the strongest baseline, we computed paired comparisons with CORE on the same target sites and seeds, separated the variation across sites from that across seeds, and describe the model-selection protocol.

(1) Paired comparison with CORE. Table 1 of the paper reports one run per target site with a single seed, and its ± is the SD across target sites. For this response, we ran BRIO and CORE on all five settings with three new seeds shared by all methods. The LOSO splits and the two validation sites of each fold are shared by all methods and seeds, so all results are paired by target site and seed. For each setting and metric, we computed the gain of BRIO over CORE for every target site and seed. The table below reports the mean over target sites of the seed-averaged gain in percentage points and an unadjusted 95% CI from a two-level bootstrap that resamples target sites and then seeds within each site with 10,000 replicates [2]. It also reports the number of target sites with a positive gain and the pp-value of a two-sided exact Wilcoxon signed-rank test over target sites [3], Holm-corrected across the ten settings [4]. Following [5], we separate the two sources of variation. SD${\mathrm{site}}$ is the SD of the seed-averaged gain across target sites, and SD${\mathrm{seed}}$ is the SD of the gain across seeds within a target site, averaged over target sites.

SettingMetricGain95% CISites > 0ppSD$_{\mathrm{site}}$SD$_{\mathrm{seed}}$
ABIDEACC1.29[−0.16, 2.72]13/200.463.191.76
ABIDEAUC1.89[0.29, 3.43]14/200.243.611.30
ABIDE (CC200)ACC2.22[1.11, 3.35]15/200.0112.471.27
ABIDE (CC200)AUC2.52[1.24, 3.79]15/200.0102.921.32
REST-meta-MDDACC1.52[0.19, 2.91]17/250.243.481.54
REST-meta-MDDAUC1.35[−0.41, 3.38]15/251.004.941.09
SRPBSACC1.08[−1.97, 4.21]6/111.005.241.86
SRPBSAUC1.25[−0.64, 3.49]6/111.003.661.13
ABCD (ADHD-task)ACC1.35[−1.05, 3.82]6/101.004.061.18
ABCD (ADHD-task)AUC1.97[−1.01, 5.25]6/101.005.391.40

As shown in the above table, the mean gain over CORE is positive in all ten settings and ranges from 1.1 to 2.5 points, in line with the single-seed gains in Table 1. Its CI excludes zero in four settings, and both metrics on ABIDE (CC200) remain significant after Holm correction. The variation of the gain comes mainly from the target sites. SD$_{\mathrm{site}}$ is 1.8 to 4.5 times SD$_{\mathrm{seed}}$ in every setting, and the intervals are widest on SRPBS and ABCD, which have 11 and 10 target sites. When the 66 target sites of the four AAL116 settings are pooled, the gain over CORE is 1.35 points in ACC with a 95% CI of [0.43, 2.27], positive on 42 sites and with an unadjusted Wilcoxon p=0.008p=0.008, and 1.59 points in AUC with a 95% CI of [0.56, 2.63], positive on 41 sites and with p=0.009p=0.009. The single-seed results in Table 1 also show a lower across-site SD for BRIO than for CORE in all ten settings, by a factor of 1.3 to 2.2. We have added these results to Appendix J.8 (Table 16) of our manuscript.

Reviewer_ev6POpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BRIO for cross-site generalization of functional connectivity (FC) models. It uses within-scan block resampling to measure changes in the predictive contributions of learned connectome factors. These changes are combined with task relevance to guide class-conditional alignment between source sites. The method is evaluated on four rs-fMRI datasets under leave-one-site-out settings. It achieves consistent improvements over the compared baselines.

Critical strengths
  1. The paper evaluates BRIO on four multi-site datasets and two parcellations of ABIDE. It includes competitive baselines, ablation studies, and sensitivity analyses. The results show consistent improvements across datasets.

  2. Figure 3 examines the relationship between site agreement and re-estimation stability. Figure 9 shows that BRIO is less sensitive to reduced scan length. These experiments provide useful evidence beyond standard classification results.

Critical weaknesses
  1. It is unclear why within-scan re-estimation stability should indicate cross-site transferability. The experiments show that the method works, but they do not fully establish this connection. Comparisons with simpler perturbation-based approaches would help clarify the contribution.

  2. The connectome factors are jointly learned and are not constrained to be independent. Their prediction changes may also compensate for one another. It is unclear whether factor-wise support provides a reliable measure of predictive stability or whether the resulting qualifications are robust to different learned decompositions. The extensive theoretical analysis does not directly resolve these questions.

Questions for authors

The final score will be lowered or raised depending on how the authors respond to these questions

  1. Why should stability under within-scan FC re-estimation indicate cross-site transferability? What evidence supports this choice over other sources of variation?

  2. How does BRIO compare with a simpler approach that directly regularizes prediction consistency under FC re-estimation, without factor-wise qualification?

  3. How robust are the learned factor qualifications to different initializations and factor decompositions? How do dependencies and compensating changes between factors affect the support scores?

  4. Are the improvements over the strongest baseline statistically reliable across the evaluated sites and runs? Could the authors provide paired comparisons or confidence intervals?

10 comments
AuthorsOpenReview ↗

Response to Reviewer ev6P (1/8)

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: It is unclear why within-scan re-estimation stability should indicate cross-site transferability. The experiments show that the method works, but they do not fully establish this connection. Comparisons with simpler perturbation-based approaches would help clarify the contribution.

A1: Thanks for your comment. To clarify why within-scan re-estimation stability indicates cross-site transferability, we first state the reasoning and its scope, and then test the connection in three ways. We relate the support of each factor to its transfer drop on the held-out site under the block bootstrap and simpler perturbations, compare BRIO with perturbation-based training, and induce a mismatch in scan length between source and target sites.

Functional connectivity (FC) computed from a finite scan equals the connectivity of the subject plus an estimation error whose variance depends on the number of time points, the repetition time (TR) and the temporal autocorrelation of the blood-oxygen-level-dependent (BOLD) signal [1]. These acquisition parameters differ systematically between sites, so the estimation error is a component of the observed FC whose size depends on the site. A factor whose predictive contribution depends on this component transfers poorly to a site with a different protocol. The block bootstrap keeps the temporal dependence within blocks [2, 3], so it estimates from a single scan how much the contribution of a factor changes under this error. Re-estimation stability targets one acquisition-dependent source of cross-site shift rather than site invariance as a whole. A factor can be stable within every scan and still differ between sites, for example because of scanner-specific signal properties or population differences. BRIO uses stability to decide which factors the class-conditional alignment acts on, and the alignment addresses the systematic part of the site shift. Re-estimation support identifies factors whose predictive contribution is not driven by finite-scan estimation error, which is one acquisition-dependent source of cross-site shift.

(i) Support and transfer drop on the natural site shift. To test the connection on the natural site shift, we related the support of each factor on the source sites to its transfer drop on the held-out target site. For each fold and seed, we took the selected BRIO checkpoint and computed the support of each factor on the training source sites under three perturbations: (1) the block bootstrap used by BRIO; (2) an independent and identically distributed (i.i.d.) frame bootstrap with block length 1, which ignores temporal dependence; (3) additive symmetric Gaussian noise on FC, scaled so that the mean squared FC change equals that of the block bootstrap.

The transfer drop of a factor is the AUC of its attribution on the two validation sites minus that on the held-out target site, with the attribution oriented by the sign of its source class contrast. We kept the factors in the upper half of task relevance, since factors without task signal have a drop close to zero. Within each dataset, we pooled these factors over folds and seeds and computed the partial Spearman correlation between support and transfer drop, controlling for the source AUC and the fold–seed pair. A more negative correlation means that higher support predicts a smaller drop. For the block bootstrap, the pp-value tests a negative correlation by permuting support within fold–seed strata. For the other two perturbations, it tests whether the block bootstrap gives a more negative correlation by swapping the two supports of each factor. Both tests use 10,000 permutations pooled over ABIDE, REST-meta-MDD and SRPBS. The table below reports the partial correlations, the number nn of pooled factors in each dataset and the pooled pp-values.

PerturbationABIDEREST-meta-MDDSRPBSpp
nn24060066
Block bootstrap−0.19−0.34−0.27<0.001
i.i.d. frame bootstrap−0.11−0.16−0.150.003
Gaussian noise on FC−0.030.04−0.06<0.001
AuthorsOpenReview ↗

Response to Reviewer ev6P (2/8)

As shown in the above table, higher support under the block bootstrap goes with a smaller transfer drop on all three datasets, with partial correlations of −0.19 to −0.34. The association weakens as the perturbation keeps less of the temporal structure of the BOLD signal. The i.i.d. frame bootstrap, which removes temporal dependence, keeps about half of it, and Gaussian noise on FC, which ignores the time series, gives correlations close to zero. Both differences from the block bootstrap are significant with p≤0.003p\le0.003. Among the perturbations we tested, support computed with the temporal structure preserved is the most informative about transfer to an unseen site. We have added these results to Appendix J.4 (Table 7) of our manuscript.

(ii) Comparison with perturbation-based training. To compare BRIO with simpler perturbation-based training, we trained five variants that use the same block-bootstrap re-estimates in different ways. In BRIO, the re-estimates are used only to compute the qualification. At each refresh, the calibration subjects are passed through the network with their re-estimates without gradients, and training mini-batches contain only full-scan FC. The variants are constructed as follows:

  • (1) Empirical risk minimization (ERM) trains with the site–class-balanced binary cross-entropy (BCE) only.
  • (2) ERM with re-estimate augmentation (ERM+Aug) feeds each sampled subject's full-scan FC together with kk of its re-estimates into the BCE.
  • (3) ERM with prediction consistency (ERM+Cons) adds to the BCE on the full-scan FC λcons\lambda_{\mathrm{cons}} times the binary Jensen–Shannon divergence between the predictions on the full-scan FC and on one re-estimate [4].
  • (4) Unqualified alignment (UA) is BRIO without qualification guidance (w/o QG) in the paper, which uses uniform factor composition and unit pair strength.
  • (5) UA+Aug+Cons adds both training signals to UA. It uses the same number of re-estimates as BRIO in the same fold, spread evenly over its training steps, on average 0.97 per sampled subject per step on ABIDE, and differs from BRIO only in the qualification.

All variants share the BRIO network, edge support, fold-level block rule, re-estimate generator, warm-up, optimizer, number of training epochs, validation sites and seeds. In each fold and seed, the hyperparameters shared with BRIO take the values selected for BRIO, and each variant tunes its own parameter over three values with the same selection metric, with k∈{1,2,4}k\in\{1,2,4\} for ERM+Aug and λcons∈{0.3,1,3}\lambda_{\mathrm{cons}}\in\{0.3,1,3\} for ERM+Cons and UA+Aug+Cons. A run of BRIO generates R=NcalKnrefreshR=N_{\mathrm{cal}}K n_{\mathrm{refresh}} re-estimates, where NcalN_{\mathrm{cal}} is the number of calibration subjects and nrefreshn_{\mathrm{refresh}} is the number of refreshes. We ran all methods, including BRIO, on ABIDE, REST-meta-MDD and SRPBS with three new seeds shared by all methods. The table below reports the leave-one-site-out (LOSO) AUC and accuracy (ACC) in %. For each seed, we average the metric over all target sites of a dataset, and each entry is the mean ± standard deviation (SD) of the three seed-level values. Re-est. is the number of re-estimates passed through the network per training run on ABIDE, and Wall-h is the wall-clock training time in hours of one configuration and seed on ABIDE. Both are averaged over folds, with eight runs sharing two GPUs.

MethodABIDE AUCABIDE ACCMDD AUCMDD ACCSRPBS AUCSRPBS ACCRe-est.Wall-h
ERM61.84±0.3456.09±0.4862.36±0.2758.21±0.3177.93±0.7172.61±0.6600.059
ERM+Aug62.45±0.2956.85±0.3762.88±0.3158.65±0.3478.67±0.4973.44±0.61219,5040.146
ERM+Cons62.71±0.3157.18±0.2762.87±0.2259.02±0.2978.79±0.5373.48±0.49158,5760.122
UA63.24±0.3657.30±0.4663.88±0.3359.69±0.3379.89±0.6274.64±0.5400.136
UA+Aug+Cons64.81±0.2659.68±0.4365.94±0.2861.12±0.2481.12±0.3976.54±0.59154,2860.197
BRIO67.35±0.3062.09±0.3268.90±0.1864.27±0.3083.62±0.4478.94±0.64154,2860.162
AuthorsOpenReview ↗

Response to Reviewer ev6P (3/8)

As shown in the above table, performance rises steadily from ERM to BRIO: on all three datasets and both metrics, both training-signal variants outperform ERM, UA outperforms both of them, UA+Aug+Cons outperforms UA, and BRIO performs best. Training with the re-estimates as augmentation or consistency improves ERM by 0.4 to 1.1 points, and ERM+Aug passes the most re-estimates through the network yet gains less than one point. UA performs better than both, and adding augmentation and consistency to it brings a further 1.2 to 2.4 points. BRIO achieves the best result in every column and exceeds UA+Aug+Cons by 2.4 to 3.2 points with the same number of re-estimates and 18% less training time. The re-estimates contribute more when they decide which factors to align than when they serve as training signals. We have added these results to Appendix J.7 (Table 8) of our manuscript.

(iii) Controlled shift in scan length. To test the connection under a controlled shift, we induced a mismatch in finite-scan estimation error between source and target sites. All models were trained on full-length scans. At test time, each target scan was shortened to a random contiguous window of p∈{100,50,25}p\in\{100,50,25\}% of its length, with three windows per scan and predictions averaged over the windows. Compared with the scan-length experiment in Appendix J.11 (Figure 9), this test adds UA+Aug+Cons and CORE and a shorter length of 25%. We evaluated the trained checkpoints of ERM, UA, UA+Aug+Cons, CORE and BRIO on ABIDE, REST-meta-MDD and SRPBS with the three new seeds. The table below reports the LOSO ACC in %, averaged over target sites and seeds. The last row of each dataset gives the paired gain of BRIO over UA+Aug+Cons with its 95% confidence interval (CI) from a two-level bootstrap over target sites and seeds.

SettingMethod100%50%25%
ABIDEERM56.0952.1950.94
ABIDEUA57.3053.2951.68
ABIDEUA+Aug+Cons59.6856.4353.88
ABIDECORE60.8056.6153.69
ABIDEBRIO62.0960.2758.12
ABIDEGain over UA+Aug+Cons [95% CI]2.41 [1.52, 3.36]3.84 [2.55, 5.06]4.24 [2.54, 6.02]
REST-meta-MDDERM58.2153.4151.60
REST-meta-MDDUA59.6954.8852.94
REST-meta-MDDUA+Aug+Cons61.1258.0255.21
REST-meta-MDDCORE62.7558.2455.29
REST-meta-MDDBRIO64.2762.0159.68
REST-meta-MDDGain over UA+Aug+Cons [95% CI]3.15 [2.32, 4.01]3.99 [3.00, 5.00]4.47 [3.02, 5.92]
SRPBSERM72.6164.9558.98
SRPBSUA74.6468.6461.89
SRPBSUA+Aug+Cons76.5471.8167.46
SRPBSCORE77.8670.9966.52
SRPBSBRIO78.9476.5173.53
SRPBSGain over UA+Aug+Cons [95% CI]2.40 [0.71, 3.93]4.70 [2.42, 7.14]6.07 [3.43, 8.91]

As shown in the above table, all methods lose accuracy as the target scans become shorter, and the advantage of BRIO grows as the estimation error at the target sites departs further from that at the source sites. Its gain over UA+Aug+Cons rises from 2.4 to 3.2 points at full length to 4.2 to 6.1 points when a quarter of each target scan is kept, and its own accuracy drops the least at every scan length on all three datasets. UA+Aug+Cons, which sees re-estimates during training, degrades less than CORE at every scan length and stays below BRIO throughout. The largest gain appears on SRPBS, where the 25% setting keeps about 27 frames of the shortest scan. We have added these results to Appendix J.11 (Table 19) of our manuscript.

Reviewer_sYoNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BRIO uses block-bootstrap re-estimates of functional connectivity to qualify task-relevant factors before cross-site alignment. It combines factor-specific re-estimation support with task relevance, using source data during training and full-scan inputs at inference.

Critical strengths
  1. The method connects alignment strength to finite-scan reliability through an explicit, task-aware qualification mechanism.
  2. Source-only leave-one-site-out evaluation spans four cohorts and two ABIDE atlases. Component ablations, alignment-strength sweeps, and shortened-scan tests support the empirical contribution.
Critical weaknesses
  1. The ablations support qualification within this architecture, but omit a matched re-estimation-augmentation or prediction-consistency baseline. It remains unclear whether qualifying alignment is more effective than using identical re-estimates directly for robustness.
  2. The warm-up length T_w and refresh interval M determine when factors are frozen and qualifications updated, yet their numerical defaults and search ranges are missing (Algorithm 1; Appendix H). This limits reproducibility and interpretation of the reported runtime.
Questions for authors
  1. Does qualification retain its advantage over augmentation/consistency training when backbone, re-estimates, and compute are matched?
  2. Please specify T_w/M and tuning ranges, and report paired per-site gains with uncertainty against CORE and the unqualified-alignment variant.
10 comments
AuthorsOpenReview ↗

Response to Reviewer sYoN (1/6)

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: The ablations support qualification within this architecture, but omit a matched re-estimation-augmentation or prediction-consistency baseline. It remains unclear whether qualifying alignment is more effective than using identical re-estimates directly for robustness.

A1: Thanks for your comment. To test whether qualifying the alignment is more effective than using the identical re-estimates directly, we compared BRIO with five variants that use the same functional connectivity (FC) re-estimates in different ways. In BRIO, the re-estimates are used only to compute the qualification. At each refresh, the calibration subjects are passed through the network with their re-estimates without gradients, and training mini-batches contain only full-scan FC. The variants are constructed as follows:

  • (1) Empirical risk minimization (ERM) trains with the site–class-balanced binary cross-entropy (BCE) only.
  • (2) ERM with re-estimate augmentation (ERM+Aug) feeds each sampled subject's full-scan FC together with kk of its re-estimates into the BCE.
  • (3) ERM with prediction consistency (ERM+Cons) adds to the BCE on the full-scan FC λcons\lambda_{\mathrm{cons}} times the binary Jensen–Shannon divergence between the predictions on the full-scan FC and on one re-estimate [1].
  • (4) Unqualified alignment (UA) is BRIO without qualification guidance (w/o QG) in our manuscript, which uses uniform factor composition and unit pair strength.
  • (5) UA+Aug+Cons adds both training signals to UA. It uses the same number of re-estimates as BRIO in the same fold, spread evenly over its training steps, on average 0.97 per sampled subject per step on ABIDE. It differs from BRIO only in the qualification.

All variants share the BRIO network, edge support, fold-level block rule, re-estimate generator, warm-up, optimizer, number of training epochs, validation sites and seeds. In each fold and seed, the hyperparameters shared with BRIO take the values selected for BRIO, and each variant tunes its own parameter over three values with the same selection metric, with k∈{1,2,4}k\in\{1,2,4\} for ERM+Aug and λcons∈{0.3,1,3}\lambda_{\mathrm{cons}}\in\{0.3,1,3\} for ERM+Cons and UA+Aug+Cons. A run of BRIO generates R=NcalKnrefreshR=N_{\mathrm{cal}}K n_{\mathrm{refresh}} re-estimates, where NcalN_{\mathrm{cal}} is the number of calibration subjects and nrefreshn_{\mathrm{refresh}} is the number of refreshes. We ran all methods, including BRIO, on ABIDE, REST-meta-MDD and SRPBS with three new seeds shared by all methods.

The table below reports the leave-one-site-out (LOSO) area under the ROC curve (AUC) and accuracy (ACC) in %. For each seed, we average the metric over all target sites of a dataset, which gives one value per seed. Each entry is the mean ± standard deviation (SD) of these three seed-level values. Re-est. is the number of re-estimates passed through the network per training run on ABIDE, and Wall-h is the wall-clock training time in hours of one configuration and seed on ABIDE. Both are averaged over folds, with eight runs sharing two GPUs.

MethodABIDE AUCABIDE ACCMDD AUCMDD ACCSRPBS AUCSRPBS ACCRe-est.Wall-h
ERM61.84±0.3456.09±0.4862.36±0.2758.21±0.3177.93±0.7172.61±0.6600.059
ERM+Aug62.45±0.2956.85±0.3762.88±0.3158.65±0.3478.67±0.4973.44±0.61219,5040.146
ERM+Cons62.71±0.3157.18±0.2762.87±0.2259.02±0.2978.79±0.5373.48±0.49158,5760.122
UA63.24±0.3657.30±0.4663.88±0.3359.69±0.3379.89±0.6274.64±0.5400.136
UA+Aug+Cons64.81±0.2659.68±0.4365.94±0.2861.12±0.2481.12±0.3976.54±0.59154,2860.197
BRIO67.35±0.3062.09±0.3268.90±0.1864.27±0.3083.62±0.4478.94±0.64154,2860.162
AuthorsOpenReview ↗

Response to Reviewer sYoN (2/6)

As shown in the above table, performance rises steadily from ERM to BRIO: on all three datasets and both metrics, both training-signal variants outperform ERM, UA outperforms both of them, UA+Aug+Cons outperforms UA, and BRIO performs best. Using the re-estimates as training signals gives modest gains. ERM+Aug and ERM+Cons improve ERM by 0.4 to 1.1 points, and ERM+Aug passes the most re-estimates through the network yet gains less than one point. Alignment contributes more, as UA outperforms both, and adding augmentation and consistency to UA brings a further 1.2 to 2.4 points. Qualification brings the largest step. BRIO achieves the best result in every column and exceeds UA+Aug+Cons by 2.4 to 3.2 points with the same number of re-estimates and a shorter training time of 0.162 hours on ABIDE, compared with 0.197 hours. When used to qualify the alignment, the same re-estimates yield a much larger improvement.

To test whether this improvement holds across target sites, we computed the paired gain of BRIO over UA+Aug+Cons on the same target sites and seeds, with seeds paired across methods. For each dataset and metric, the table below reports the mean over target sites of the seed-averaged gain in percentage points, together with an unadjusted 95% confidence interval (CI) from a two-level bootstrap that resamples target sites and then seeds within each site with 10,000 replicates [2]. It also reports the number of target sites with a positive gain and the pp-value of an exact Wilcoxon signed-rank test over target sites, Holm-corrected across the six dataset–metric tests [3, 4]. SD${\mathrm{site}}$ is the SD of the seed-averaged gain across target sites, and SD${\mathrm{seed}}$ is the SD of the gain across seeds within a target site, averaged over target sites.

SettingMetricGain95% CISites > 0ppSD$_{\mathrm{site}}$SD$_{\mathrm{seed}}$
ABIDEACC2.41[1.52, 3.36]19/20<0.0011.981.36
ABIDEAUC2.54[1.43, 3.67]16/200.0022.571.27
REST-meta-MDDACC3.15[2.32, 4.01]24/25<0.0012.140.92
REST-meta-MDDAUC2.96[2.01, 3.90]23/25<0.0012.361.14
SRPBSACC2.40[0.71, 3.93]9/110.0192.601.28
SRPBSAUC2.50[1.06, 3.98]9/110.0142.541.31

As shown in the above table, the gain of BRIO over UA+Aug+Cons is positive on 80% to 96% of the target sites, its CI excludes zero in all six dataset–metric settings, and it remains significant after Holm correction with p≤0.019p\le0.019. The gain varies more across target sites than across seeds in every setting, which indicates that site differences account for most of the remaining variation. REST-meta-MDD, with the most target sites, shows the highest share of positive sites and the narrowest intervals, and SRPBS, with the fewest, shows the widest intervals. We have added these results to Appendix J.7 (Tables 8 and 9) of our manuscript.

AuthorsOpenReview ↗

Response to Reviewer sYoN (3/6)

W2: The warm-up length TwT_w and refresh interval MM determine when factors are frozen and qualifications updated, yet their numerical defaults and search ranges are missing (Algorithm 1; Appendix H). This limits reproducibility and interpretation of the reported runtime.

A2: Thanks for your comment. To make the training schedule reproducible and the runtime interpretable, we report the value or search grid of every hyperparameter of BRIO, split the training time of every fold into qualification refreshes and the remaining training, and measure the sensitivity of the results to TwT_w and MM. The schedule controlled by TwT_w and MM works as follows:

  • (1) During the first Tw=15T_w=15 epochs, the network is trained with the site–class-balanced BCE only, without alignment or qualification, and the assignment of regions of interest (ROIs) to factors is learned.
  • (2) At epoch TwT_w, the ROI-to-factor assignment is frozen. The factor queries and the key projection stop updating, so factor jj refers to the same ROIs in every later qualification, while the graph encoder, the factor head and the classifier continue to train.
  • (3) After warm-up, the qualification is recomputed every M=20M=20 optimizer steps. At each refresh, all training subjects serve as calibration subjects and are passed through the network with their full-scan FC and KK block-bootstrap re-estimates without gradients, with K=8K=8 on all datasets except ABCD, where K=16K=16. Between refreshes, the alignment uses the most recent qualification.
  • (4) TwT_w, MM and the Sinkhorn regularization ε=0.05\varepsilon=0.05 are fixed across datasets and folds and are not selected on any data, and the sensitivity to TwT_w and MM is reported below. JJ and KK are selected once per dataset on the validation sites, and the alignment weight λenv\lambda_{\mathrm{env}} and the edge-support ratio ρg\rho_g are selected per target site and seed on the two validation sites.

The table below lists each hyperparameter with its value or search grid and the level at which it is tuned.

HyperparameterSymbolValue or gridTuned
Warm-up length in epochsTwT_w15, sensitivity over {8, 15, 30}No
Refresh interval in optimizer stepsMM20, sensitivity over {10, 20, 40}No
Re-estimates per calibration subjectKK{1, 2, 4, 8, 16} on ABIDE and REST-meta-MDD, {2, 4, 8, 16, 32} otherwise, 8 selected, 16 on ABCDPer dataset
Calibration subjects per site–class cell–All training subjects of the cellNo
Block lengthℓb\ell_bEstimated per fold by Eq. (82) of the paper, τac=0.10\tau_{\mathrm{ac}}=0.10, [ℓmin⁡,ℓmax⁡]=[2,30][\ell_{\min},\ell_{\max}]=[2,30]No
Number of factorsJJ{2, 4, 8, 16, 32}, 8 selected, 16 on REST-meta-MDD, 4 on SRPBSPer dataset
Alignment weightλenv\lambda_{\mathrm{env}}{0.005, 0.01, 0.05}Per target site
Edge-support ratioρg\rho_g{0.10, 0.20, 0.30}Per target site
Sinkhorn regularizationε\varepsilon0.05No
Learning rate / weight decay–3×10−43\times10^{-4} / 10−410^{-4}No
Batch–8 subjects per site–class cell, sampled with replacement when the cell has fewer than 8, forward micro-batch 32No
Max epochs / patience / min post-warm-up epochs–80 / 15 / 10, 120 / 20 / 10 on SRPBSNo

As shown in the above table, most hyperparameters are shared across datasets, and only λenv\lambda_{\mathrm{env}} and ρg\rho_g are tuned for each target site and seed. Their selections concentrate on the middle of each grid, with λenv=0.01\lambda_{\mathrm{env}}=0.01 chosen in 58% of the runs and ρg=0.20\rho_g=0.20 in 53%. The per-dataset choices of JJ and KK settle on 8 for most datasets.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

We sincerely thank all reviewers for their constructive comments. We have revised the manuscript accordingly, and all changes are highlighted in blue. The main revisions are summarized below.

1. Comparison with re-estimate augmentation and consistency training

  • Added ERM, ERM+Aug, ERM+Cons, BRIO w/o QG and w/o QG+Aug+Cons under a matched backbone and re-estimate budget, with training cost and paired per-site gains, in Appendix J.7 (Tables 8 and 9). (Reviewers sYoN, ev6P, j4pB)

2. Paired statistics across sites and seeds

  • Added paired per-site comparisons of BRIO with CORE, BRIO w/o QG and the strongest ablation over three seeds, with bootstrap confidence intervals, Holm-corrected Wilcoxon tests, and site-level and seed-level standard deviations, in Appendix J.8 (Table 16). (Reviewers sYoN, ev6P, j4pB)

3. Hyperparameters, model selection and runtime

  • Added the value or search grid of every hyperparameter, including TwT_w and MM, in Appendix H (Table 5). (Reviewer sYoN)
  • Added the model-selection protocol and a comparison with and without per-target hyperparameter search for BRIO and CORE in Appendix H (Table 6), and the tuning procedure of the baselines in Appendix F. (Reviewer j4pB)
  • Added the sensitivity to TwT_w and MM in Appendix J.10 (Table 18). (Reviewer sYoN)
  • Added the split of training time into qualification refreshes and training steps in Appendix J.12 (Table 21). (Reviewer sYoN)

4. Re-estimation support and qualification

  • Added the relation between factor support and transfer drop under the block bootstrap, an i.i.d. frame bootstrap and Gaussian noise in Appendix J.4 (Table 7). (Reviewer ev6P)
  • Added the compensation between factors, factor removal by support, and the robustness of qualifications across seeds and numbers of factors in Appendix J.5 (Tables 10–12). (Reviewer ev6P)
  • Added the analysis of the reference scale in Eq. (16), including correlation, injected heterogeneity and subsampling of calibration subjects, in Appendix J.6 (Tables 13–15). (Reviewer j4pB)
  • Added a test-time scan-length shift with BRIO w/o QG+Aug+Cons and CORE in Appendix J.11 (Table 19). (Reviewer ev6P)

5. Corrections and clarifications

  • Corrected the ID results in Table 4 (Appendix J.1) with values recomputed from the per-fold logs. (Reviewer j4pB)
  • Revised Table 17 to report the block length in time points, and clarified in Appendix E.2 that the block length is defined in time points and applied to all sites of a fold. (Reviewer j4pB)

We hope these revisions address the reviewers' concerns, and we are happy to provide further clarification during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.