How pretraining objectives shape EEG representations: a controlled comparison

EEG foundation models learn transferable representations from large unlabelled recordings, but differences in datasets, architectures, and evaluation protocols obscure how pretraining objectives shape what they learn. We address this gap through a controlled comparison of masked autoencoding (MAE), autoregressive prediction (AR), and subject-level contrastive learning (SLCL), using a shared frozen tokenizer and GPT-2 backbone with pretraining data drawn from 50,894 hours of four-channel consumer EEG. Across seven downstream tasks, MAE shows the clearest performance gains on sleep staging, age prediction, gender classification, and EEG event classification, while SLCL best learns subject-specific features. Increasing pretraining data generally improves downstream performance, while LoRA fine-tuning largely preserves the relative performance of the three objectives observed under frozen probing. To interpret these objective-specific strengths, we probe how established EEG features are encoded in the learned representations. MAE retains strong band-power information from the tokenizer and correlates most with cross-channel coherence features, as it reconstructs masked tokens from visible context across channels and time. AR most clearly exposes band power trends, since it predicts subsequent activity from preceding context. SLCL favors alpha-peak frequency and alpha asymmetry, suggesting that aligning recordings from the same participant across sessions emphasizes participant-related characteristics. Layer-wise probing further reveals that task-relevant EEG information is distributed differently across model depth depending on the pretraining objective, with all seven tasks reaching their highest probe scores before the final layer. Together, these findings connect pretraining formulations to distinct spectral, spatial, and temporal feature profiles, providing an interpretable basis for understanding their downstream strengths and trade-offs.