PLARE: Feature Replacement with a Mixture of Experts for Self-Supervised ECG Representation Learning

Self-supervised models learn electrocardiogram (ECG) representations from unlabeled recordings. ECG waveform shapes vary within and across recordings: an isolated beat may differ from its neighbors, whereas a distinctive shape may recur across beats. This variation motivates selecting feature-processing parameters according to the content of each short waveform segment (patch). To this end, we introduce PLARE (Patch-Level Adaptive Replacement with Experts), a self-supervised ECG model with a patch-level mixture-of-experts (MoE) module. Content-dependent routing (CDR) selects experts based on each patch's convolutional features, and patch feature replacement uses their outputs in place of the original features before token projection. A temporal Transformer then models relationships among the resulting patch tokens to produce a record-level ECG representation. Across four PTB-XL label sets and CPSC2018, PLARE achieved higher AUROC and AUPRC than the evaluated baselines on four of the five tasks under both linear probing and fine-tuning. In ablations, feature replacement achieved a higher average of AUROC and AUPRC than residual addition on four of five tasks under each protocol. Content-dependent routing improved this average over fixed random expert assignment on all five tasks under both protocols. Inference costs were lower than those of larger baselines. Using INCART's beat-level annotations, we found that expert selection distributions in the frozen pretrained model differed between contexts surrounding normal beats and premature ventricular contractions. These findings support content-dependent feature replacement for ECG representation learning. Code is available at https://anonymous.4open.science/r/PLARE-5FC6.