- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors recognize that brain foundation models are lacking relative to much simpler baselines such as kernel ridge regression (KRR) on functional connectivity (FC). A diagnosis is presented showing that the FC eigenvalues are inappropriate for trait prediction, with global components being overweighted by KRR, while some potentially interesting structure that is specific to individuals is captured by intermediate ranks instead, which the paper suggests to calibrate towards. These adapted FCs are in turn used to train an encoder model on a varied dataset with strong downstream results, especially given the limited complexity of the approach.
Critical strengths
- The authors present a simple finding that is relevant to ML for neuroscience, which I genuinely can imagine to be adopted in the field. This finding is clearly diagnosed, is given a plausible explanation, and is shown to be likely useful for pretraining encoder models. As such, the paper is valuable as it manages to provide something novel and valuable to the community, without introducing much extra complexity.
Critical weaknesses
- The evaluations are currently somewhat lacking, which prevents clearly drawing some of the conclusions the paper attempts to make:
a) I believe pretraining always uses the FC^a* teacher. As such, its unclear to what extend other parts of the pipeline are responsible for the downstream gains, or whether its a* specifically. Ideally, the pretraining pipeline is repeated by only setting a=1 (and perhaps an intermediate value or two, depending on cost).
b) two of the models (BrainLM and BrainJEPA) appear to score very poorly. I'm concerned they were not appropriately reproduced, in which case the BFM comparison is thin.
Questions for authors
Primary questions that would upgrade my score when addressed:
-
Would it be possible to use a non-a* teacher so its role for pretraining an encoder can be discerned?
-
For BrainLM/BrainJEPA, are the authors confident the models were evaluated correctly? (small note: those models do not z-score individual parcels and in my experience in order to function they require mean-level offsets between parcels to survive preprocessing - is this the case for your datasets?)
Clarifications:
-
Could the authors provide a case for the significance of improved fingerprinting (given that its presented as a major eval in the paper)?
-
Could the authors speculate on why FC wins over the encoder model with larger sample sizes?
3 comments
Thanks to the authors for the additional analyses.
The answers to Q1 and Q4 are especially appreciated.
I'm not quite as convinced about the response to Q2 however. The comparison models' robust scaling use statistics computed across subjects instead of per recording. While it shifts subject-specific amplitude differences, they are very much still present, so from the response I can't tell if the authors still provided OOD inputs. Regarding the empirical check; was this done on z-scored data? If so, there is no way to recover the between-subject offset information and this would not tell us anything.
Q2. We thank the reviewer for this remark. Our check used inputs that were already z-scored per recording, so it could not test the reviewer's point. For the camera-ready version, we will update all results accordingly: Table 3, the other cohorts in the appendix and every corresponding table.
We re-evaluated both models exactly as their released code prepares the data: regional BOLD level kept, robust scaling across subjects, and each model's own region order (Brain-JEPA: Tian-S3 then Schaefer-400 in 17-network order; BrainLM: A424 sorted by coordinate).
Both models are now well above chance, but F-BERT remains significantly better. HCP-YA cognition composite, same subjects, folds and probe as Table 3; mean std over the 200 folds:
| Encoder | Scan | Cognition |
|---|---|---|
| BrainLM-111M | 2.4 min | |
| BrainLM-650M | 2.4 min | |
| Brain-JEPA | 5.8 min | |
| F-BERT-1M (ours) | 2.4 min | |
| F-BERT-8M (ours) | 2.4 min | |
| F-BERT-1M (ours) | 57.6 min | |
| F-BERT-8M (ours) | 57.6 min |
On full scans, F-BERT-1M exceeds BrainLM-650M by (), BrainLM-111M by () and Brain-JEPA by (). At 2.4 minutes F-BERT-1M is level with BrainLM-650M, which has almost 800 times more parameters, and above the other two (). Tests are the corrected resampled -test on the 200 paired folds.
We will correct every affected row.
We thank the reviewer for the positive assessment and for the questions, which we address with new experiments.
Q1. A pretraining run with a non-α∗ teacher. As suggested, we pretrained fMRI-BERT using the FC-distill objective with the raw FC teacher and everything else unchanged. The student does learn from a raw-FC teacher, but the FC$^{\alpha^\ast}$ teacher is significantly better at both model sizes, on cognition and on fingerprinting, and the raw-FC encoders fall below raw FC at 2.4 minutes. The full tables and paired tests are in the general response. We will add this control to Section 5.4 and Appendix J and reword the contribution statement.
Q2. Were BrainLM and Brain-JEPA evaluated correctly?
Each model runs from its released checkpoint and released feature-extraction code, at its own parcellation and window length. The reviewer is right that our parcel timeseries are band-passed and z-scored per region, so between-parcel mean levels are absent from our inputs. Both models' own training pipelines, however, rescale every parcel before the encoder: Brain-JEPA applies robust scaling per ROI (Dong et al., 2024, Sec. 4.1) and so does BrainLM's UK Biobank preparation (Ortega Caro et al., 2024, Sec. 2.1 and model card). Rescaling each parcel removes the mean offsets between parcels, so the trained models never saw them either.
An empirical check. We encoded the same HCP-YA segments three ways, with the model's own robust scaling, with a per-parcel z-score and as stored, and probed each frozen embedding with the KRR of Table 3. BrainLM-111M and Brain-JEPA stay at chance in all three cases (∣r∣<0.04).
In addition, our linear probing results align with those reported by the authors. They, too, scored well below on linear probing compared to fine-tuned models. As we mention in Section 7, fine-tuning could reorder Table 3.
Q3. Why does better fingerprinting matter?
Test–retest identification measures how much subject-specific information a representation keeps. Reliable prediction of individual traits such as cognitive ability requires that sessions of the same subject be more similar to each other than to sessions of other subjects.
The BFMs do not separate subjects better than sessions: their embeddings are stable across sessions, yet they identify the right subject on retest only 1–2% of the time, mapping every subject onto nearly the same point and discarding the individual information that cognition prediction relies on. Identification thus measures how much individual, rather than group-level, variance a representation keeps. We will add this point to Section 5.3.
Q4. Why does Ridge fitted on FC outperform the fMRI-BERT with more subjects and longer scans? Table 4 shows the gain of the encoder over FC is largest with short scans and few subjects and reverses at 38 minutes and ≥500 subjects. We see three contributing reasons and will add them to Section 5.4. (i) Capacity. KRR on the 101,025 connectome edges can fit more as the number of training subjects grows. A frozen 128- or 384-dimensional embedding cannot. With few subjects the compression helps as a regulariser; as the number of train subjects increases, it becomes a bottleneck. (ii) What the teacher denoises. The student is trained to encode the whole-recording connectome from an 80-volume window, so it helps most when the FC estimate is short and noisy. At 57.6 minutes the FC estimate is already precise. (iii) Teacher mismatch. The student matches similarities between subjects within batches drawn from 162 datasets that do not include HCP. It therefore reproduces the specific structure of HCP-YA only approximately, while KRR fitted on HCP connectomes uses it directly once there are enough subjects.