Adaptive Multi-Objective Alignment in Pretraining Time-Series Foundation Models for EEG tasks

Time-series foundation models have become a powerful approach for analyzing EEG data, showing strong performance across healthcare and neuroscience. However, most models rely on a single self-supervised pretraining objective, such as masked reconstruction, which may limit their ability to capture diverse features and hinder generalization across downstream tasks. Incorporating multiple learning objectives during pretraining is a promising direction, though disparities in gradient magnitudes can cause certain objectives to dominate the optimization, which reduces the benefits of joint training. In this work, we systematically study this phenomenon and explore the benefits of multi-objective pretraining of time-series foundation models. We introduce MAPE (Multi-objective Alignment in Pretraining time-series foundation models for EEG tasks), a theoretically grounded framework that efficiently aligns multiple objectives within a single training pipeline by analyzing their pretraining losses. Our straightforward, low-cost approach is applicable to any model. Our empirical analysis across three different encoders demonstrates that MAPE leads to improved performance compared to unaligned multi-objective pretraining and single self-supervised pretraining settings, highlighting its effectiveness in learning robust and generalizable EEG representations.