Are Foundation Models Enough for Data-Scarce Functional Connectivity Diagnosis? Revisiting Task-Specific Learning with BrainLRR

Functional connectivity (FC) derived from Resting-state fMRI captures disruptions in brain-network organization, making FC-based classification a promising non-invasive approach to disease diagnosis. However, FC classification is constrained by small, heterogeneous, and noisy datasets. Whether foundation models (FMs) can transfer effectively under such label scarcity remains unclear. We benchmark general-purpose LLMs, vision backbones, and neuroimaging FMs under zero-shot, few-shot, and target-dataset fine-tuning protocols. Learned alignment improves LLM-based classification from a mean AUROC of 53.3 to 70.8, but encoder-only models using the same learned FC representation achieve comparable performance, suggesting pretrained language representations do not provide a consistent advantage. We therefore introduce Brain Low-Rank Regularization (BrainLRR), a compact task-specific model combining region-of-interest (ROI)-profile masking with low-rank regularization of subject representations. Across four classification tasks spanning Autism spectrum disorder (ASD), Parkinson’s disease (PD), and Alzheimer’s disease (AD), BrainLRR demonstrates favorable sample efficiency and outperforms the evaluated FM baselines across the examined data-scarcity settings. The design components of BrainLRR also generalize to other architectures, improving BrainNetCNN across all four tasks. These results show that, in data-scarce FC classification, FM utility depends on modality alignment and pretraining domain, while carefully designed task-specific models can provide an effective alternative.