Representation Over Capacity: Sparse Temporal Access for Functional Connectivity

Resting-state fMRI is inherently temporal, yet most brain-network models operate on a static summary of the signal. Standard functional connectivity (FC) compresses an entire BOLD scan into a single matrix, averaging away how interactions between brain regions evolve over time. We argue that this creates a representational bottleneck: once temporal variation is discarded, increasing downstream model capacity cannot recover it. We introduce **TempLasso**, a deliberately low-capacity temporal FC model that selects a small set of discriminative connections, tracks them across local FC windows, and models their trajectories with a lightweight BiGRU. Across three rs-fMRI cohorts and five static FC backbones, TempLasso outperforms 13 of 15 substantially more complex static models. When used as a plug-and-play temporal branch, **X+TempLasso** improves all 15 backbone–dataset combinations, with an average gain of +4.63 AUC points. In contrast, directly applying the full static backbone to every temporal window is markedly less reliable, showing that greater per-window model capacity does not necessarily translate into better temporal modeling. A complementary theoretical analysis formalizes static FC as a many-to-one temporal compression: Bayes-optimal prediction with temporal access cannot be worse than prediction restricted to static FC, and can be strictly better when discarded temporal structure is task-relevant. Together, these results suggest that progress in brain-network learning may depend less on increasingly elaborate models of a static connectome and more on preserving and exploiting the temporal structure already present in the signal.