The Variance Brain Foundation Models Forgot: A Third-Order Basis for Predicting Individual Differences

Brain foundation models (BFMs) are self-supervised Transformers pretrained on fMRI data. We posit that these models should capture each subject's cognitive performance from their fMRI signal. Yet across three published BFMs and every readout we test, they predict cognition worse than a linear regression from the $\sim$80K parameters of the functional connectivity matrix (FC). Scale does not help: BrainLM's 650M model predicts cognition worse than its 111M. We attribute this to a variance allocation problem: BFM pretraining captures the variance components that dominate fMRI but not the higher-order structure that predicts cognition. Our per-cumulant analysis of the reconstructed signal shows that the second-order covariance is partially preserved, while the third-order co-skewness tensor is largely destroyed. To recover what BFMs lose, we design a linear pipeline that projects the fMRI signal into the subspace that best preserves its co-skewness and computes FC there. This improves over raw FC and exceeds every pretrained BFM on both datasets and both parcellations we test ($r$ from $0.303$ to $0.368$ on AOMIC AAL-424), matching or exceeding the closest published FC results on comparable targets with no pretraining and no GPU. We recover the raw-FC ceiling on BrainLM's forward pass by finetuning with a loss targeted at this same subspace, which the same loss outside the subspace does not reach. This points to the pretraining objective, not the model size, as the bottleneck.