CoLaP: Coupled Latent Prediction for a Synchronized EEG–fNIRS Foundation Model

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) capture the neuronal electrical activity and cortical hemodynamic changes of the same underlying brain activity, respectively. Existing brain signal foundation models rely almost exclusively on EEG as input, while synchronous EEG-fNIRS modeling methods are mostly designed for a single task on a fixed dataset, with the model architecture bound to the signal channel configuration and therefore difficult to adapt to datasets with different electrode and optode layouts. To address this problem, this paper proposes the Coupled Latent Prediction (CoLaP) foundation model. The pretraining of CoLaP consists of two objectives, intra-modality masked latent prediction and cross-modality temporal latent prediction. The former infers the latent representation of masked positions from visible context within each modality, and the latter predicts the latent representation of adjacent time windows on the cross-modality fused representation, so that the learned EEG-fNIRS representations preserve the structure of each modality while encoding the coupling between the two modalities. CoLaP is pretrained on 96,305 ten-second synchronized windows from 16 public datasets and evaluated on 7 tasks across 4 downstream datasets. The results show that CoLaP ranks among the top three on 6 of the 7 downstream tasks and outperforms all baselines on mental arithmetic, valence, and arousal recognition. Adaptation protocol experiments show that pretraining yields an average improvement of 7.74% over training from scratch. Modality ablation shows that the coupling information between EEG and fNIRS learned during pretraining transfers to different downstream tasks. For reproducibility, the code and pretrained weights are submitted anonymously with the manuscript.