acceptodds
Under review as a conference paper at ICLR 2027

Beyond Patch-Level EEG Reconstruction: Contrastive Pretraining and Multiscale Convolutional Transformer for EEG Decoding

26 views

Abstract

Self-supervised pretraining has shown early promise for improving non-invasive electroencephalogram (EEG) decoding. Recently, many large-scale EEG foundation models have converged on the recipe of raw signal tokenization followed by masked reconstruction pretraining. However, this approach may be suboptimal for data with low signal-to-noise ratio and sample-level dependencies at multiple timescales. To test this hypothesis, we develop a novel contrastive-pretrained EEG model with multiscale sample-level convolutions and Transformer encoder blocks (CoCoT-EEG). CoCoT achieves state-of-the-art performance on extensive benchmark decoding tasks with heterogeneous experimental configurations. Moreover, we systematically dissect the impact of architecture and pretraining objective through ablation experiments and mechanistic analyses. Consistent with our hypothesis, models benefit from contrastive pretraining, as well as sample-level feature extraction across patch boundaries and longer convolution kernels. Thus, we demonstrate the viability of contrastive learning for EEG modeling while suggesting key design considerations that complement existing architectural advances, prompting further investigations in alternative large-scale pretraining strategies.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, EEG decoding, EEG foundation models, self-supervised learning, contrastive learning, neural decoding, representation learning, brain-computer interfaces

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VPbaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoCoT-EEG, a contrastive-pretrained EEG foundation model that combines multiscale temporal convolutions with a Transformer encoder. Motivated by the limitations of conventional patch-first tokenization and masked reconstruction, the authors adopt a convolution-first strategy to better capture continuous temporal dependencies in EEG signals.

Critical strengths
  1. The motivation is clear. This paper raises a relevant question about the widely adopted patch-first tokenization and masked reconstruction paradigm in current EEG foundation models. The proposed convolution-first approach, which extracts features before patchification, is well motivated by the continuous and multiscale nature of EEG signals.

  2. The paper evaluates the proposed method on 11 EEG benchmarks covering diverse downstream tasks. The results are competitive with recent EEG foundation models.

Critical weaknesses
  1. The overall technical novelty is relatively limited, as both multiscale convolutions and contrastive pretraining have been widely explored in prior work. The main architectural contribution is essentially reversing the order of convolution and patchification. While this is an intuitive design choice, the results in Tables 2 and 8 show that its benefits are generally modest or inconsistent, despite notable improvements on a few datasets. In my view, the empirical gains are not sufficiently compelling to justify such a relatively incremental architectural modification.

  2. Since the main contribution lies in changing the order of convolution and patchification, I would be interested to see whether this design choice can also improve existing EEG foundation models, rather than only the proposed architecture. For example, applying the convolution-first strategy to established models such as CBraMod or LaBraM would help assess whether the observed benefits are generalizable. Demonstrating consistent improvements across different architectures would substantially strengthen the contribution of this work.

Questions for authors

See in Weakness . 2

3 comments

Thank you for your concise, constructive feedback and a clear request for how to strengthen the contribution of our work. We would like to clarify that our main contribution does not lie in changing the order of convolution and patchification (Weakness 2), but rather the systematic investigation of pretraining objective while controlling for model architecture in a new proposed model. Nevertheless, we test the effect of the conv-first tokenizer on CBraMod and CSBrain below and find significant performance boosts in the from-scratch (no pretraining) setting, thus providing evidence for a potentially generalizable design principle and further strengthening our architectural contribution.

1. Clarifying contributions beyond novel architecture
First, while the proposed architecture is intuitive and “well motivated”, our architectural contribution is not only to introduce the model, but to systematically understand its impact through ablation and interpretability studies (e.g., Fig. 2), in addition to showing performance gains across extensive benchmark tasks. More broadly, the main contribution of the paper is our systematic comparison between reconstruction and contrastive objectives under architecture-matched settings.

As the reviewer points out in Table 2, Conv-1st indeed introduces small (but consistent) performance benefits over Patch-1st (on par or better on 10/11 tasks). But the more striking improvement comes from contrastive pretraining (MoCo) compared to MAE, which can be seen in the same Table.

As two recent review papers found (Fig. 1A), while contrastive pretraining has been explored in prior works such as BENDR and BIOT, reconstruction pretraining represents an overwhelming majority of the approaches in the last 5 years (~90%). Moreover, as a consequence of adopting MAE pretraining, almost all papers directly inherit a patch-first modeling approach, which makes direct comparison of design contributions more challenging. To study them separately, we make a further contribution here by proposing essentially a second novel and competitive architecture in mask-aware, sparse-conv CoCoT as a control, which enabled the systematic study of tokenization order and pretraining objective.

Thus, we want to highlight our investigation of both model architecture and pretraining objective, especially in systematically disentangling these two factors, as a key contribution, which is to our best knowledge the first paper to do so at scale. We therefore very much welcome any feedback from the reviewer regarding the novelty, extensiveness, and depth of these contributions relative to the current literature.

2. Conv-first modification for existing model architectures
Nevertheless, we agree that demonstrating consistent improvements across different architectures would substantially strengthen the contribution of this work. Thus, we test the effect of our Conv-1st tokenizer on modified variants of CBraMod and CSBrain in the from-scratch setting, removing the confound of reproduce heterogeneous pretraining recipes and datasets while bypassing compute resource limitations during the rebuttal period.

Compared to the native configuration of CBraMod and CSBrain, which both employ within-patch convolutions after tokenization, we reverse this order by swapping in the same convolution-first tokenizer of CoCoT (which also enables cross-patch, long convolutional kernels).

In summary, we find that the conv-first tokenizer significantly boosts both CBraMod and CSBrain on several tasks, including +5-7% on FACED and around +10% on SEED-VIG (Table 1 below). Thus, we conclude that the advantages of the conv-first recipe may be generalizable beyond our CoCoT architecture, and we thank the reviewer for this suggestion.


Additional details

Table 1: from-scratch performance of modified Conv-1st baselines Results are balanced accuracy except for SEED-VIG, where we report Pearson correlation ×100. Values are mean ± 95% Student-t CI (4 seeds).

BackboneTaskNativeConv-first tokenizer
CBraModFACED52.63 ± 2.5457.69 ± 0.75
PHYSIO59.52 ± 2.2060.07 ± 0.56
BCI-IV46.03 ± 2.2255.21 ± 10.62
SHU-MI58.73 ± 2.5760.65 ± 1.81
SEED-VIG54.57 ± 3.2063.88 ± 1.41
CSBrainFACED44.81 ± 1.2952.26 ± 0.85
PHYSIO58.00 ± 1.9658.49 ± 1.31
BCI-IV48.55 ± 1.3256.55 ± 6.31
SHU-MI59.08 ± 1.6261.28 ± 2.83
SEED-VIG43.76 ± 8.5254.47 ± 3.37

I thank the authors for their response and additional experiments. I understand the computational constraints during the rebuttal period. However, the current experiments on CBraMod and CSBrain do not provide sufficient evidence that changing the tokenization order leads to consistent improvements.

Given these computational constraints, a more feasible and rigorous alternative would be to pretrain CBraMod and CSBrain on smaller datasets, keeping their original architectures and pretraining objectives unchanged while modifying only the order of convolution and patchification. Such controlled experiments would more clearly isolate the contribution of the proposed method. However, the authors' current experiments do not evaluate the proposed modification under pretraining, which is particularly important for EEG foundation models, especially given the substantial performance gap between pretrained and from-scratch CSBrain. More concerningly, even in the from-scratch setting, the effect of tokenization order is not properly isolated: the authors retain the original tokenizers for the Patch-first baselines but replace them with the proposed CoCoT tokenizer for the Conv-first variants. This introduces additional architectural differences beyond tokenization order, making the comparison insufficiently controlled and the observed improvements difficult to attribute to the claimed contribution.

Since my primary concerns regarding methodological novelty and generalizability remain unresolved, I do not find the current experimental evidence sufficiently rigorous or convincing to revise my assessment. I therefore maintain my initial rating.

We appreciate your continued engagement and the opportunity to clarify our overall contribution.

To clarify: our contribution on the architectural side is a multiscale convolutional tokenizer that extracts continuous features before aggregating them into tokens. Within this design, the manuscript separately examines tokenization order using matched truncated kernel banks, and the additional benefit of extending the kernel bank (Section 4.3.1, Table 2) which is only enabled by the reversed tokenization order.

As a result, our contribution on this matter is the tokenizer module itself, and not merely the reversed order of convolutional embedding. The experiment above demonstrates that simply importing our tokenizer to existing architectures can result in substantial improvements when trained from scratch, including +2.6% on FACED, +4% on BCI-IV and +8% on SEED-VIG for CBraMod compared to the published result of the pretrained CBraMod (see Table 1 in main paper).

Thus, the tokenizer itself—which includes the multiscale convolutions enabled by switching the order—is the contribution, which we show above can be flexibly introduced to other architectures.

Furthermore, we did not intend to claim that conv-before-patch is better in every single architecture, nor is it feasible (or valuable, for that matter) to run such an experiment. Nevertheless, we acknowledge that some of our wording maybe have suggested a broader claim about reversing the operations of arbitrary existing tokenizers, and we will clarify this scope.

But to further test your specific hypothesis, we extend the Native versus Conv-first comparison provided in our initial response and have added an intermediate Patch-first configuration for both CBraMod and CSBrain. All configurations retain each model’s Transformer backbone and spectral and positional encoding paths and use the same supervised training protocol. The expanded table therefore compares the native tokenizer, Patch-first tokenizer, and Conv-first tokenizer. The Conv-first tokenizer still achieves the highest mean performance on all five evaluated tasks for both backbones.

BackboneTaskNativePatch-first tokenizerConv-first tokenizer
CBraModFACED52.63 ± 2.5452.91 ± 1.1457.69 ± 0.75
PHYSIO59.52 ± 2.2057.92 ± 1.4260.07 ± 0.56
BCI-IV46.03 ± 2.2251.32 ± 7.1555.21 ± 10.62
SHU-MI58.73 ± 2.5758.33 ± 4.6260.65 ± 1.81
SEED-VIG54.57 ± 3.2058.45 ± 1.6663.88 ± 1.41
CSBrainFACED44.81 ± 1.2948.20 ± 1.2852.26 ± 0.85
PHYSIO58.00 ± 1.9656.69 ± 1.5058.49 ± 1.31
BCI-IV48.55 ± 1.3250.30 ± 4.0356.55 ± 6.31
SHU-MI59.08 ± 1.6260.69 ± 1.7161.28 ± 2.83
SEED-VIG43.76 ± 8.5252.11 ± 6.4654.47 ± 3.37

Furthermore, within the constrained rebuttal compute budget, we prioritized an additional small-scale pretrained comparison for CBraMod. We pretrained the Native, Patch-first and Conv-first variants for 25,000 steps under a common MoCo protocol, using the same corpus, augmentation recipe, and optimization settings, followed by finetuning on the same five tasks with four seeds. That setting preserves the observations made in the from-scratch setup: the full Conv-first tokenizer achieves higher mean performance than the truncated Patch-first variant on all five tasks after pretraining.

BackboneTaskNativePatch-first tokenizerConv-first tokenizer
CBraModFACED54.83 ± 0.8756.21 ± 1.3058.11 ± 0.58
PHYSIO62.49 ± 0.6462.27 ± 0.2362.90 ± 0.73
BCI-IV46.94 ± 6.8151.87 ± 0.8055.13 ± 6.43
SHU-MI60.39 ± 1.5162.11 ± 0.7662.31 ± 1.23
SEED-VIG62.60 ± 1.5862.85 ± 2.0863.86 ± 1.54

These results extend the observed advantage of the full tokenizer over its truncated Patch-first alternative and the Native tokenizer to pretrained CBraMod. Together with the matched architectural ablations within CoCoT, these experiments support the effectiveness of the proposed tokenizer and its transfer to additional encoder backbones.

Finally, we again emphasize that our main contribution is much more than simply the order of operation within the tokenizer. We hope the reviewer will also consider these architectural results in the context of the paper’s central contribution: a systematic, architecture-matched investigation of contrastive versus reconstruction-based pretraining for EEG foundation models, supported by extensive downstream evaluation and analyses of the learned representations. If you still have concerns regarding methodological novelty and generalizability on those broader contributions, we are happy to clarify further.

Reviewer_DUqYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CoCoT-EEG, an EEG representation model combining continuous multiscale temporal convolutions, L2-pooling tokenization, a Transformer encoder, and MoCo contrastive pretraining. It challenges the widespread combination of early patchification and masked reconstruction in EEG foundation models. Across eleven downstream datasets, the model reports the highest primary-metric mean among the included baselines on ten tasks. Architectural ablations and spectral probes provide useful evidence for convolution-first tokenization and contrastive pretraining. The empirical results are promising, although the attribution of improvements to individual design choices, comparison fairness, and model-capacity accounting require further clarification.

Critical strengths
  1. The paper addresses an important design question: whether modeling choices inherited from masked language and image modeling suit continuous, noisy EEG signals. Separating tokenization order from the pretraining objective gives the work value beyond introducing another architecture.

  2. Evaluation covers eleven datasets spanning motor imagery, emotion recognition, seizure detection, abnormality detection, sleep staging, and vigilance estimation. Improvements are substantial on several tasks, including CHB-MIT and HMC. The HBN pretraining experiment provides additional evidence that transfer benefits extend beyond a single pretraining corpus.

  3. The controlled diagnostics are more informative than qualitative visualizations alone. Translating bursts across patch boundaries directly examines the proposed tokenization mechanism, while the matched-RMS spectral experiment tests oscillatory information accessibility. Appendix M carefully describes probe selection, subject separation, and paired bootstrap procedures.

Critical weaknesses
  1. The central objective comparison changes several factors simultaneously. Appendix B states that MoCo receives six stochastic augmentations, whereas MAE receives an unaugmented waveform. Downstream evaluation also uses MoCo's momentum backbone and MAE's directly optimized backbone. In addition, visibility-aware MAE changes the tokenizer's input support and normalization statistics during pretraining. Matching the underlying architecture and optimizer therefore compares two training recipes without fully isolating the loss. The results support the effectiveness of the proposed MoCo recipe, but broader conclusions about contrastive versus reconstruction objectives need augmentation controls, an EMA control, and some sensitivity analysis for the MAE configuration.

  2. The state-of-the-art comparison relies on published baseline values whose equivalence to the current evaluation is insufficiently documented. Section 4.1 refers to the same splitting process, while Appendix E says baseline values follow their respective evaluation settings. Exact downstream subject/session assignments, preprocessing differences, adaptation access, and tuning budgets are not consolidated. Explicitly marking Euclidean Alignment on BCI-IV is helpful, but it does not resolve comparability across the full benchmark. Re-evaluating representative strong baselines on identical partitions would substantially strengthen the headline result. This is a reporting and verification gap; the manuscript does not establish that leakage occurred.

  3. The downstream head may contribute substantial unreported capacity. Section 3.3 flattens all channel-time embeddings before projection, and Table 5 specifies a hidden dimension of 1,024. Calculated from these stated dimensions, the first projection alone would contain 64 × 8 × 512 × 1,024 = 268,435,456 weights on PHYSIO, assuming the described direct projection. This makes parameter counts and head-matched comparisons essential. Flattening also introduces dependence on channel order and input dimensions, so a topology-agnostic backbone does not by itself demonstrate robustness to missing, reordered, or changed electrodes.

  4. The novelty positioning needs qualification and closer empirical comparisons. Section 1 claims an absence of earlier fixed-architecture objective studies, but MAEEG (Chien et al., 2022) already compared contrastive and waveform-reconstruction pretraining using closely related convolution–Transformer backbones. The present paper's broader evaluation and continuous multiscale tokenizer should define the distinction more precisely. BENDR, BIOT, and CoMET are discussed but omitted from the main comparison, leaving the advantage over existing contrastive and hybrid approaches unresolved.

  5. Some architectural and mechanistic conclusions exceed the controls provided. Comparing the full kernel bank with a truncated bank examines longer temporal context, but does not establish the benefit of multiple scales over a parameter-matched single-scale tokenizer. Likewise, Figure 2 demonstrates accessibility of injected spectral information; it does not directly measure removal of acquisition artifacts or preservation of task-relevant neural activity under realistic noise. Randomly initialized tokenizer controls, overlapping-patch alternatives, and downstream corruption experiments would help distinguish learned specialization from architectural filtering effects.

Questions for authors
  1. How much of the MoCo–MAE gap remains when augmentation exposure is controlled? A useful experiment would apply selected identical corruptions to both methods while retaining the clean reconstruction target for MAE. Reporting both online and momentum MoCo encoders would clarify the contribution of checkpoint averaging.

  2. Provide a benchmark protocol table with exact subject/session partitions, normalization and filtering, checkpoint selection, adaptation procedures, and hyperparameter-search budgets. For BCI-IV, specify whether alignment statistics use unlabeled target-subject or target-session data. For TUAB exclusion, clarify whether filtering also checks subject overlap across other TUEG recordings.

5 comments

Thank you for your constructive feedback and suggestions. We appreciate your comment pointing out the importance of the design question, the breadth of the pretraining and finetuning evaluation and the resulting performance improvement, as well as the informative, quantitative diagnostics.

To address your concerns, we have performed several additional large-scale pretraining and evaluation experiments to isolate the impact of objective, augmentations, and architecture, and further clarify model design choices and evaluation settings.

1. Augmentation controls (Question 1):
We have now added three complementary pretraining experiments to control for augmentations in MoCo and MAE:
Exp 1: MoCo with random masking only. MoCo receives two independently sampled 50% random masks and no additional augmentations, using the same masking distribution and sparse visibility-aware tokenizer as MAE.
Exp 2: MoCo and MAE with matched masking and augmentations. We adapt the channel dropout and time masking in the paper’s MoCo augmentation recipe into patch-aligned masks hiding 50% of patches. Both methods use these masks and the same four remaining signal-processing augmentations, applied to visible waveform samples before sparse tokenization to prevent information leakage. MAE reconstructs the clean waveform at masked positions.
Exp 3: Denoising reconstruction with CoCoT augmentations as corruption. A denoising autoencoder (DAE) uses the same augmentation recipe and dense encoder as the paper’s MoCo model (identical base CoCoT in Table 1), reconstructing the clean waveform with an L1 loss over all samples.

In particular, in Exp 2, MoCo achieves higher mean performance on nine of the ten evaluated tasks, while MAE remains slightly better on TUAB. The random-mask and denoising controls provide complementary evidence (see Tables 1–3 in our response to Reviewer xUb9).

These results drastically expand the scope of our ablation studies while supporting the original conclusion that MoCo outperforms MAE pretraining even under matched masking and augmentation conditions.

Nevertheless, we also observe that the additional augmentations in MoCo pretraining provide a noticeable improvement from masking only (Exp 1 vs. Exp 2). Reconstruction also benefits from MoCo’s augmentations, compared to its original random masking, though improvement is lesser than for MoCo (Exp 2 vs. Exp 3). We will thus report these findings in the main text to provide further context and adjust the generality of our claims accordingly in the revised paper. Thank you for the suggestion.

2. EMA control and MAE sensitivity analysis (Question 1):
We additionally investigate 1) the choice of MoCo’s momentum versus online encoder for finetuning 2) the sensitivity of MAE to masking percentage (from 50% to 75% masked).
Finetuning from MoCo’s online encoder yields similar performance to finetuning from its momentum encoder (base CoCoT), indicating that the observed advantage does not depend on selecting the momentum checkpoint (Table 1 below). We also find that MAE pretraining can be sensitive to masking strength, though it does not change the overall conclusions as MoCo keeps the broader advantage (Table 2 below). We will include these controls in the appendix.

3. Reevaluation of baselines beyond reported numbers (Question 2):
We apologize for the lack of reproducible documentation. To clarify: all downstream task splits are identical for all baseline models, since the original publications either re-use the same splits from CBraMod when evaluating their own models (e.g., CSBrain), and/or report previously published values using the same splits. We follow the exact same practice and do both here. The split details for each task are reported below in Table 3 and will be included in the Appendix.

Furthermore, following the reviewer’s suggestion, we re-run finetuning of baseline models using their published pretrained weights and under a common evaluation setting for a subset of tasks. Moreover, we conduct evaluation with both flattened and mean-pooled embeddings for all models (see next point). Overall, the default CoCoT achieves higher mean performance than the reevaluated baselines on all four rerun tasks with flattened readouts, a trend that also holds with mean pooling for the 3D-CoCoT (see below) variant. To keep alignment conditions consistent across models and avoid extra confounding factors, we did not apply Euclidean Alignment (EA) in these reruns. See Table 4 below for details.

4. Downstream decoder head dimension (Question 2):
The reviewer correctly points out the increased dimensionality of the downstream decoder head due to flattened encoder embeddings, though we note that this is standard in the field, with other baseline models composed of similarly sized readout heads (e.g., REVE). The default CoCoT accommodates different channel counts and montages without requiring electrode coordinates, which may be unavailable for some datasets. Its ordered, flattened readout preserves channel identity within each downstream task, allowing the task-specific decoder to exploit electrode-dependent patterns without explicit spatial positional encoding.

Nevertheless, to isolate readout size impact, we additionally evaluate mean-pooled embeddings, including the 3D spatial PE CoCoT (section 4.3.1 of the paper) variant that encodes electrode location before aggregation.. This results in a significant reduction in CoCoT’s readout parameters (e.g., 11x for PHYSIO), and similar parameter count across models (~26M for CoCoT, REVE, CSBrain, and CBraMod).

We compared the flatten and mean-pooling CoCoT variants against the same readouts for the baselines: in full finetuning (Table 4) and with a simpler linear probing (Table 5). Overall, we find that CoCoT with mean-pooled embeddings—and, as a result, much smaller readout heads—still outperformed baseline models with mean-pooled embeddings. Furthermore, flattened representation and bigger downstream heads improved performance for the baseline models, but CoCoT remains the most performant under this matched setting. Thus, we conclude that CoCoT’s strong performance relative to existing baselines does not arise purely from the downstream decoder head.

5. Clarifying contribution and earlier contrastive models:
We apologize for the lack of precision in our claims and clarify here: we did not intend to claim a general lack of comparison between reconstruction vs. contrastive pretraining, but only in the recent line of work surrounding large-scale pretrained EEG foundation models. As the reviewer noted, our work provides a number of novel contributions relative to existing approaches in addition to the pretraining objective comparison, including the extensive evaluation and architectural innovations such as the multiscale tokenizer. But to further detail the key differences:

MAEEG (Chien et al., 2022) takes BENDR’s existing architecture and applies reconstruction instead of contrastive pretraining, but with much narrower downstream evaluations (only sleep stage classification) and baseline comparisons, a simpler architecture, limited pretraining dataset, architectural, and augmentation ablations, and in general predates the recent advances of self-supervised pretraining. Nevertheless, we consider this early investigation an intellectual precursor and will cite it appropriately in an updated version of our paper.

As for empirical comparisons, we did not include models such as BENDR and BIOT since they were already surpassed by earlier reconstruction models (e.g., reported in CBraMod). Moreover, to the reviewer’s point, the lack of a matched-architecture study leaves the contrastive vs. reconstruction (or hybrid) comparison unresolved in any case. Thus, they were omitted due to space constraint, though we agree they are important for context and completeness.

We report below the performance of BIOT and COMET (BENDR was evaluated on a different set of downstream tasks) on the common evaluation tasks compared to CoCoT (Table 6): CoCoT achieves better performance on 3 of 4 tasks, and thus sets a new standard for contrastive-pretrained EEG foundation models. We will include these comparisons in the Appendix in the revised paper.

6. Further mechanistic analyses and controls:
We additionally pretrained a parameter-matched single-scale MoCo control using 37-sample kernels. The multiscale model achieves higher mean performance on six of ten tasks, with an average balanced-accuracy gain of 1.31 percentage points across the nine classification tasks and the largest improvement on CHB-MIT (+6.68 points). Although the single-scale model performs better on some tasks, these results support an average benefit of the multiscale design (see Table 7 for details).

We added architecture-matched random-initialization controls to all three Figure 2 diagnostics (see Tables 8A - C). Conv-first’s reduced patch-boundary sensitivity is already present before training, demonstrating an architectural contribution, while pretraining improves absolute frequency-decoding accuracy (Fig. 5A in the revised manuscript). In contrast, pretraining substantially strengthens the spectral specialization of short and long tokenizer branches (Fig. 5D–E). Random encoders also exhibit declining high-frequency accessibility, which MoCo pretraining mitigates much more strongly than MAE (Fig. 5F; numerical results in Table 8 below).

We further repeated Figure 2C using the matched structured-mask and augmentation checkpoints. The high-frequency accessibility gap persists: low- peak-height decoding reaches R2 = 0.927 for MoCo versus 0.757 for MAE (see table 9). This shows that MAE’s high-frequency decoding disadvantage persists even when masking and augmentation distributions are matched.

Finally, adding overlapping context to the frozen Patch-first tokenizer substantially reduces its error at the original patch boundary. However, the Patch-first-minus-Conv-first error gap increases again on either side of the shared overlap region, with broad maxima outside that region. Conv-first retains lower error averaged across positions, including when both tokenizers use overlapping context. Overlap therefore mitigates the original boundary effect without fully closing the frequency-decoding gap between the two tokenizers (Fig. 5B–C; numerical results in Table 10 below).

Reviewer_xUb9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper challenges the dominant recipe of raw signal tokenization followed by masked reconstruction for EEG foundation models. It proposes CoCoT-EEG, which applies multi-scale one-dimensional convolutions to the raw continuous signal before L2 pooling based patchification and encodes the resulting tokens with a Transformer stack, and pretrains the model with MoCo contrastive learning on TUEG of around 27,000 hours. The pretrained model reportedly achieves state-of-the-art performance on 10 out of 11 downstream tasks against LaBraM, CBraMod, REVE, and CSBrain, and the architecture also outperforms many existing single-task models when trained from scratch. Beyond the benchmarks, the authors run architecture matched ablations under matched pretraining conditions, namely Conv-first versus Patch-first tokenizers with truncated kernels, and MoCo versus MAE using the same full kernel encoder, where the masked variant employs a sparse visibility-aware convolutional tokenizer to prevent leakage. Mechanistic analyses complement these results, showing lower and more uniform frequency decoding error across patch boundaries for the Conv-first tokenizer, frequency specialization of kernels by length, and more accurate linear decoding of spectral peak height from MoCo than from MAE embeddings across all five evaluated bands, with a smaller decline toward higher frequencies.

Critical strengths
  1. The paper provides what appears to be the first architecture matched comparison of pretraining objectives for EEG foundation models, consistent with the authors' survey reporting that 49 of 55 existing models use reconstruction and that no prior study has isolated the objective under a fixed architecture. The masked variant uses a technically nontrivial sparse visibility-aware convolutional tokenizer to prevent leakage, and the Conv-first versus Patch-first ablation controls receptive field size with truncated kernels, which makes the comparison meaningful.
  2. The evaluation breadth and analysis depth exceed typical work in this space. The paper covers eleven datasets across six task types, pretrained, from-scratch, and second corpus conditions, a leakage controlled TUAB evaluation through a corpus filtered checkpoint, and mechanistic probes that help explain rather than merely accompany the performance differences. The two main pretraining runs together took 20 hours on a single GPU according to Appendix G, so the study is also modest in compute.
Critical weaknesses

The MoCo versus MAE comparison does not isolate the pretraining objective. MoCo is trained with six augmentations while MAE uses none, and the augmentation ablation in Section 4.3.3 shows that augmentations carry much of the contrastive benefit. No control separates the loss from the augmentations, although contribution 3 claims that the two are isolated.

Questions for authors
  1. Table 1 reports the TUAB AUROC of CBraMod as 86.06 while Table 6 reports 91.56 for the same baseline. Which value is correct?
  2. To isolate the effect of the pretraining objective itself, could you train MoCo with the same masking as MAE as its only augmentation, that is, using two random maskings of each input as the two contrastive views, so that both arms in Table 2 receive identically masked inputs and differ only in the loss?
  3. The pretraining corpus uses 19 EEG channels at 100 Hz according to Appendix B, while downstream datasets differ in channel count and montage. How is the pretrained checkpoint adapted to these datasets, and is any part of the encoder re-initialized?
  4. Will the code and pretrained checkpoints be released?
2 comments

Thank you for your positive and constructive feedback. We also appreciate your suggestion to better isolate the pretraining objective from augmentations. We have now added three complementary pretraining experiments, where the results broaden and further strengthen the overall conclusion of the paper.

1. MoCo with MAE-matched random masking and no additional augmentations (directly answering Question 2 and critical weakness): we pretrained CoCoT with the MoCo objective using two independently sampled 50% random masking and no other augmentations, matching the MAE pretraining setting. Both methods used the same sparse visibility-aware convolutional tokenizer. We find that MoCo achieves higher mean performances on 7 tasks, and lower performances on 3, suggesting that additional augmentations are not necessary for all of MoCo’s advantages (see Table 1 below). Nevertheless, view construction remains an integral part of contrastive learning, and masking alone does not impose the same invariances as our EEG-specific signal-processing augmentation recipe, motivating the additional experiment below.

2. MoCo with MAE-matched masking plus all augmentations: building on the previous experiment, we tuned the time masking and channel dropout augmentations of our MoCo augmentation recipe to mask 50% of patches. Both methods receive the same four remaining signal-processing augmentations from the original recipe, applied to the visible waveforms in order to avoid leakage from hidden to visible tokens (which would detriment MAE). MoCo uses two independently sampled views, while MAE reconstructs the uncorrupted waveform at masked positions. First, we find that the augmented MAE method largely preserves the baseline MAE performances, while significantly improving on some tasks. Under this matched recipe, MoCo reaches the best performances on 9 out of 10 tasks, while MAE remains slightly better on TUAB (see Table 2 below). This shows that while reconstruction can benefit from the revised augmentation scheme, MoCo remains the overall stronger approach.

3. Denoising reconstruction with all augmentations: lastly, we trained a denoising autoencoder (DAE) using the same augmentation recipe as the paper’s MoCo recipe. Both methods use the dense full input encoder, and the DAE reconstructs the clean waveform from the augmented input, using an L1 loss over all waveform samples. We find that DAE slightly improves over the baseline MAE with matched results on most tasks and clearer improvements on BCI-IV and CHB-MIT. Nonetheless, MoCo retains higher performances on 9 out of 10 tasks when compared with DAE, under the same signal-processing augmentation recipe (see Table 3 below).

Together, these experiments further isolate the contribution of pretraining objective from augmentations, and show that MoCo’s downstream advantages cannot be solely attributed to the additional augmentations. We will revise the manuscript and promote the matched augmentations MAE vs MoCo experiments to

Responses to questions:

  1. TUAB AUROC: Thank you for identifying this reporting error. 86.06 is from CSBrain’s reported values for CBraMod, while 91.56 is from the original CBraMod publication. The original CBraMod publication’s TUAB-excluded pretraining condition reports balanced accuracy of 82.49 ± 0.25 and AUROC of 91.56 ± 0.17, consistent with our Table 6. We will harmonize Table 1 accordingly. The overall ranking based on the primary metric is unaffected.
  2. Objective vs. augmentation isolation: As detailed above, we have implemented the requested independent-mask MoCo control and added two complementary comparisons under matched signal-processing corruption. In summary, we find that MoCo retains better empirical performance on most tasks even with matched masking and no other augmentations, while showing even broader gains under the matched augmentation settings. Augmentations do provide a notable benefit for both reconstruction and MoCo pretraining, and we will acknowledge this accordingly. We will incorporate these controls into the revised manuscript.
  3. Adaptation to different channel count and montage: No adaptation or reinitialization of the tokenizer and transformer backbone are needed. The tokenizer applies the same temporal filters independently to each EEG channel, and the Transformer processes the resulting tokens using temporal positional encoding only, allowing the backbone to process different channel counts and sequence lengths.
    The backbone is finetuned with newly initialized task-specific prediction heads, which are sized for the downstream input dimensions. A CoCoT model variant with 3D spatial PE (detailed in section 4.3.1) and mean-pooled embeddings is discussed in response to Reviewer DUqY (paragraph 4).
  4. Code release: Yes, the code and checkpoints will be released on GitHub and HuggingFace, respectively.

Supporting details

Results are balanced accuracy except for SEED-VIG, where we report Pearson correlation ×100. Values are mean ± 95% Student-t CI (4 seeds). BCI-IV results use no Euclidean Alignment.


Table 1. Random masking without additional augmentations.

TaskMAE, 50% random maskingMoCo, independent 50% masks
BCI-IV54.84 ± 2.2959.09 ± 3.72
PHYSIO64.34 ± 1.0866.03 ± 0.61
SHU64.45 ± 0.3164.46 ± 1.37
FACED56.81 ± 1.4155.26 ± 2.72
SEED-V42.29 ± 0.7342.37 ± 0.44
HMC76.68 ± 0.3778.15 ± 0.48
ISRUC79.45 ± 0.0581.74 ± 0.07
CHB-MIT84.78 ± 2.0388.39 ± 0.31
TUAB82.02 ± 0.4581.57 ± 0.46
SEED-VIG64.02 ± 2.4263.22 ± 3.96

Table 2. Matched structured masking and signal-processing augmentations.

TaskMAE, structured masks + augmentationsMoCo, structured masks + augmentations
BCI-IV57.25 ± 1.4158.55 ± 5.81
PHYSIO64.58 ± 0.3467.04 ± 0.59
SHU64.79 ± 1.3665.71 ± 1.30
FACED57.05 ± 1.8158.55 ± 2.90
SEED-V42.28 ± 0.4043.30 ± 1.19
HMC76.66 ± 0.3078.55 ± 0.30
ISRUC79.60 ± 0.2980.65 ± 0.72
CHB-MIT87.58 ± 0.5988.05 ± 0.67
TUAB81.54 ± 0.5281.20 ± 0.32
SEED-VIG64.01 ± 3.6664.99 ± 1.96

Table 3. Denoising reconstruction with the original MoCo augmentation recipe.

TaskDAE, full SP-only recipeMoCo, full SP-only recipe
BCI-IV55.82 ± 4.9959.25 ± 2.12
PHYSIO64.67 ± 0.7466.90 ± 0.37
SHU64.40 ± 1.6464.93 ± 1.79
FACED56.92 ± 1.5258.76 ± 3.06
SEED-V42.38 ± 0.3343.40 ± 0.33
HMC77.15 ± 0.3178.51 ± 0.26
ISRUC80.02 ± 0.1280.99 ± 0.26
CHB-MIT85.90 ± 0.8187.98 ± 0.53
TUAB82.32 ± 0.5481.35 ± 0.91
SEED-VIG63.71 ± 0.5565.63 ± 1.32

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank you for the feedback and suggestions

We thank the reviewers for their constructive feedback and suggestions. We especially appreciate that the reviewers highlighted the novelty and importance of our research question, the breadth of our evaluation, as well as the depth of our analyses.

To reiterate: through an extensive benchmarking suite, architecture-matched ablation studies isolating the impact of pretraining loss, as well as several interpretability experiments demonstrating the benefits of our architecture, our paper is the first systematic investigation of a critical design consideration—reconstruction vs. contrastive objective—for large-scale pretraining of EEG foundation models.

Nevertheless, we appreciate the reviewers’ suggestions for additional experiments and clarifications to further strengthen our evidence.

To summarize, we have conducted 9 new computationally intensive experiments, including:

  1. Four pretraining experiments (plus the downstream finetuning runs for each): 3 to further isolate the impact of augmentation vs. pretraining objective (xUb9, DUqY), and 1 to assess MAE’s sensitivity to masking hyperparameters (DUqY). Overall, we find that MoCo’s downstream advantages cannot be solely attributed to the additional augmentations compared to MAE.

  2. Four finetuning experiments (DUqY) where we re-run key comparisons under identical evaluation settings, to verify the impact of finetuning MoCo’s momentum vs. online encoder, as well as to specifically assess the impact of readout head dimension using flattened vs. mean-pooled encoder embeddings for CoCoT and all baseline models under both full finetuning and linear probing evaluation. In a nutshell, we find that CoCoT’s performance gains remain strong under matched evaluation protocols, including with substantially smaller readout heads.

  3. One from-scratch experiment to assess the impact of our convolution-first approach in other models (VPba), thus reversing the order of patchification and convolution in CBraMod and CSBrain and evaluating them in the from-scratch setting. We find that the conv-first tokenizer significantly boosts both CBraMod and CSBrain on several tasks, including +5-7% on FACED and around +10% on SEED-VIG, and is never detrimental.

  4. Additionally, we significantly extended the control conditions of our experiments and analyses (DUqY), by evaluating a parameter-matched single-scale control, adding random-initialization controls to all three Figure 2 diagnostics, repeating the frequency-accessibility analysis under matched masking and augmentations, and examining overlapping-patch controls (added new Fig. 5 in revised manuscript).

We also provide additional information such as the evaluation setting across all models and datasets (e.g., data splits, DUqY), as well as clarifying our contribution, specifically emphasizing the critical but so-far underexplored investigation of pretraining loss under architecture-matched settings for EEG foundation models (VPba). The latter is particularly important, since such a systematic evaluation has so far not been seen in a field focused on architectural improvements in the default reconstruction pretrained setting.

Thus, in light of the additional empirical evidence and context, we hope the reviewers and AC can recommend our work for acceptance, where we can contribute our novel perspective on rigorously evaluating pretraining objectives as a part of the design considerations in the exploding field of EEG foundation modeling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.