Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach

Heterogeneity in time series data is far more pronounced than in vision or language, manifesting at two coupled levels: inter-domain discrepancies in sensing modalities, sampling rates, and physical processes, and intra-domain variability such as latent sub-domains and temporal concept drift. We empirically show that this bi-level heterogeneity is the root cause of two failures of mixed-batch centralized pretraining of time series foundation models (TSFMs), namely representation collapse and gradient conflict, which existing federated learning (FL) methods only partially address since they assume each client is internally homogeneous. We propose FedTRL, a heterogeneity-aware FL pretraining method whose two components target the two levels respectively: a local objective that combines diffusion-based reconstruction with sub-domain adversarial regularization and prototype alignment to reduce intra-domain conflict without disturbing temporal modeling, and a domain-aware aggregation (DaG) on the server that re-weights clients by prototype discriminability and global semantic alignment to control inter-domain interference. Extensive experiments across point/probabilistic forecasting show FedTRL matches or surpasses centralized and federated TSFM baselines and degrades least under domain imbalance, drift, and unseen-domain transfer, turning bi-level heterogeneity from an obstacle into a controllable training signal.