acceptodds
Withdrawn from ICLR 2027

NeuroCodeX: Beyond Homogenization in EEG Joint Pretraining

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning effective EEG representations remains challenging when downstream tasks have limited labeled data. Joint pretraining on diverse large-scale EEG datasets offers a promising solution, but substantial differences in electrode montages, channel configurations, and sampling rates make it difficult to directly combine datasets. Existing approaches typically address these discrepancies by enforcing a unified input structure through signal resampling, channel padding, masking, or fixed channel selection, thereby suppressing dataset-specific structural characteristics and limiting the ability of joint pretraining to exploit complementary information across datasets. In this work, we propose NeuroCodeX, a structure-aware framework that moves beyond homogenization by explicitly modeling structural differences during EEG joint pretraining. NeuroCodeX consists of two components: a Hierarchical Topographic Encoder that captures anatomy-aware spatial dependencies through hierarchical region-level aggregation, and a Multi-Scale Quantizer that provides rhythm-aware discrete supervision across different sampling rates. We evaluate NeuroCodeX on four EEG datasets spanning diverse acquisition settings, channel configurations, and downstream tasks. Experimental results demonstrate consistent improvements over state-of-the-art baselines, with particularly strong benefits under data-scarce conditions. These findings suggest that effective EEG joint pretraining depends not simply on combining more data, but on appropriately preserving and modeling the structural characteristics of individual datasets. Code is available at: https://anonymous.4open.science/r/NeuroCodeX.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
EEG Representation Learning, Heterogeneous EEG Pretraining, Vector Quantization, Topographic Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_CsY1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper looks at joint pretraining on EEG datasets that use different montages and sampling rates. NeuroCodeX has two parts. A Hierarchical Topographic Encoder maps channels into predefined cortical regions at three levels of detail and runs coherence-based graph convolutions over them. A Multi-Scale Quantizer splits each signal into the five classic EEG bands and gives each band its own random-projection pseudo-labels. The method is tested on TUAB, TUEV, CHB-MIT and SEED-IV against ten baselines.

Critical strengths
  1. The problem is real. Anyone who has tried to pool TUH, SEED and CHB-MIT data runs straight into mismatched montages and sampling rates, and most existing work simply resamples and pads. The paper makes a clear case for modeling these differences instead, and Figure 3 illustrates the issue nicely. I think the question matters to the EEG foundation model community even if this method turns out not to be the final answer.

  2. Converting band limits into FFT bins for each sampling rate, as in Equation 10, is a simple and sensible way to avoid resampling. Table 10 supports it well. Resampling everything to 200 Hz cuts TUEV balanced accuracy from 0.410 to 0.361 and SEED-IV from 0.326 to 0.294. The CHB-MIT result is also impressive, with an AUC-PR of 0.596 compared with 0.369 for CBraMod.

Critical weaknesses
  1. The experiments don't isolate the main claim. No baseline is jointly pretrained on the same data, so the gains might just come from more pretraining data. Joint gains are mixed. Pretraining on all four datasets lowers TUEV kappa from 0.371 to 0.355, and adding TUEV hurts CHB-MIT AUC-PR. The abstract stresses data-scarce settings, but no experiment varies the amount of labels. I'd like to see homogenization baselines such as BIOT trained on the same data.

  2. I find the baseline numbers hard to trust. On TUEV and TUAB they sit far below the original papers. LaBraM-Base gets 0.386 and 0.618 balanced accuracy here, but its paper reports 0.641 and 0.814, and CBraMod reports 0.667 on TUEV against 0.407 here. On CHB-MIT, 21 of the 24 baseline means are identical to Table 8 of the CBraMod paper, yet none of the standard deviations match. Which numbers were actually rerun?

  3. Some method details look off. Equation 15 involves no masking, so the encoder predicts codes of its own input and the task may be nearly trivial. In Equation 12 the reference phase uses the first frequency bin, which is zero for every band except delta. Table 13 puts T5 in the temporal region but P7, the same electrode position, in the parietal one. Six of the seventeen fine regions are also empty for TUH data.

Questions for authors

See weaknesses.

Reviewer_B5nEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NeuroCodeX argues that joint pretraining across heterogeneous EEG datasets should model structural differences (montages, channel configurations, sampling rates) rather than homogenize them. Within a random-vector-quantization SSL framework it contributes: (1) the Hierarchical Topographic Encoder (HTE), which aggregates channel embeddings into anatomy-aware regional representations via intra-region graph convolution (adjacency from magnitude-squared coherence) and inter-region graph convolution, at three predefined cortical granularities (6/10/17 regions); and (2) the Multi-Scale Quantizer (MSQ), which factorizes discrete pseudo-labels over five canonical EEG bands, aligning each band's Fourier-bin interval to the physical Hz range per dataset's sampling rate, with per-band random-projection quantizers (non-learnable, following VQ-MTM) and phase-rotated augmentations. Experiments cover four datasets (TUAB, TUEV, CHB-MIT, SEED-IV) under single-dataset and joint pretraining; on TUEV joint pretraining with CHB-MIT reaches 0.4286 bAcc (vs 0.4068 for CBraMod, 0.3900 single-dataset), and ablations remove HTE/MSQ, swap spatial aggregation, and drop individual bands.

Critical strengths
  • The problem framing is useful and the MSQ design is a concrete, well-motivated fix: aligning band masks to physical frequency across sampling rates addresses a real cross-dataset inconsistency that most joint-pretraining pipelines silently ignore, and the phase-rotation augmentation is a sensible way to make band codes sampling-invariant.
  • The ablations isolate meaningful comparisons (graph vs mean vs padding aggregation; per-band removals) and the negative finding — joint pretraining benefits are non-monotonic in dataset composition ("+All" worse than "+CHB-MIT") — is a useful corrective to the "more data is better" assumption in EEG pretraining.
Critical weaknesses
  • The central thesis is never tested against its own alternative. Nothing compares the same NeuroCodeX pipeline under homogenization (resampling to a common rate, channel padding/selection) at matched pretraining data against the structure-aware version; and the SOTA baselines (LaBraM, EEGPT, CBraMod) are pretrained on their own corpora, so "consistent improvements over state-of-the-art baselines" confounds architecture with pretraining data and scale — the very confound the paper criticizes in prior work.
  • The evidence is thin for the claimed scope. Four datasets only; SEED-IV bAcc is 0.3127 (4-class, near-chance territory), removing δ or β improves some metrics there, and "+All" degrades TUEV kappa relative to single-dataset pretraining. The headline "particularly strong benefits under data-scarce conditions" has no label-fraction or few-shot analysis in the main text, gains over CBraMod on TUEV are ~0.6–2.2 points with only three seeds and no significance tests, and key pretraining details (per-dataset hours, sampling rates, parameter counts, MSC cost per patch) are deferred to the appendix.
Questions for authors
  • Can you run the decisive control: the identical pipeline trained on the same joint corpus but homogenized (common sampling rate + channel padding/selection), compared against the structure-aware version?
  • What are the per-dataset pretraining hours and were any baselines given joint pretraining on the same corpora?
  • Where is the data-scarce advantage quantified? Learning curves over labeled fractions or few-shot settings would directly support the abstract's claim.
Reviewer_4Dx2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper tried to address the problem of joint pretraining on EEG datasets with different electrode configurations and sampling rates. The authors hypothesize that preserving structural differences across datasets can help learn better transferable representations. The proposed NeuroCodeX has two main components, HTE for learning region-level representations and MSQ for generating frequency-band pseudo-labels. The authors test their method on TUAB, TUEV, CHB-MIT and SEED-IV. Some results are better than the selected baselines, but I don't think the paper explains well why these components are needed for heterogeneous joint pretraining.

Critical strengths

None.

Critical weaknesses
  1. I don't see much novelty in the proposed solution, because basically the method is just combining hierarchical graph aggregation with frequency-band quantization. There are previous works that have already investigated hierarchical spatial modeling for EEG [1], and random vector quantization with phase alignment [2]. Additionally, the paper misses comparisons with strong EEG FMs such as REVE [3] and CSBrain [4]. I don't understand what is really new from this combination.

  2. The experiment appears somewhat questionable. The authors want to show that preserving heterogeneous EEG structures is better than homogenizing them. However, Tab. 10 only considers resampling all signals to 200 Hz. I don't think this is enough to validate the main hypothesis. Why don't the authors compare with a conventional homogenization pipeline, using the same model and training data? Also, Appendix F states that the authors still select 19 canonical channels for TUAB and TUEV. I don't understand to what extent the proposed method actually moves beyond homogenization.

  3. I don't feel convinced about the explanation for the improvement. For example, in Tab. 7, using CHB-MIT improves SEED-IV from 0.3127 to 0.3392 BA, but using all datasets decreases it to 0.3031. I understand that more data is not always better, but why does CHB-MIT help while the other datasets don't? The authors didn't really investigate this. I think this is important because the whole motivation is about learning from heterogeneous datasets.

  4. I also find the comparison with existing EEG FMs somewhat unfair. In Appendix H, BIOT is pretrained from scratch, but LaBraM, EEGPT and CBraMod use their released checkpoints. NeuroCodeX also uses the target datasets for pretraining. I don't think we can draw a strong conclusion from comparing models with different pretraining data. I suggest the authors at least conduct a matched comparison with one of the stronger EEG FMs to show whether the proposed method is actually better.

  5. I find the presentation somewhat careless. Even Figure 1, which is supposed to explain the whole method, contains very small annotations and a typo ("Surpervise"). Figure 4 also contains "Pesudo-labels", while Section 2.3 repeats the same citations. (minor)

References

[1] Hierarchical Dynamic Graph Convolutional Network With Interpretability for EEG-Based Emotion Recognition. IEEE TNNLS, 2022.

[2] Vector Quantization Pretraining for EEG Time Series with Random Projection and Phase Alignment. ICML 2024.

[3] REVE: A Foundation Model for EEG - Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects. NeurIPS 2025.

[4] CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding. NeurIPS 2025.

Questions for authors

See Weaknesses.

Reviewer_bJYZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes NeuroCodex, an EEG foundation model designed to address the challenges posed by heterogeneous electrode montages during pretraining. The proposed framework incorporates a coarse-to-fine HTE module and frequency-aware MSQ for vector-quantized signal reconstruction. The authors evaluate NeuroCodex under both single-dataset and joint pretraining settings, reporting improvements over the evaluated baselines.

While the problem is relevant and the experimental results are promising, the methodological novelty appears limited, as similar ideas have been explored in prior literature. More importantly, the evaluation protocols are insufficiently specified, and comparisons with closely related methods are missing or inadequate.

Critical strengths

The paper provides extensive architectural ablations and relatively detailed implementation information, which helps assess the contribution of individual components and facilitates reproducibility.

Critical weaknesses
  1. The dataset splitting and pretraining protocols are not clearly described. In particular, it remains unclear whether the single-dataset pretraining experiments strictly follow the official training splits, or whether recordings from downstream validation and test sets are included during pretraining. The protocol used for validation and model selection also requires clarification. This is a significant concern because even label-free pretraining on held-out evaluation data may introduce information leakage and bias downstream performance estimates under an inductive evaluation setting [1]. Without a clear description of the data partitions and their usage across pretraining, model selection, and downstream evaluation, it is difficult to assess the reliability of the reported improvements.
  2. The key ideas underlying HTE and MSQ are not entirely novel. Frequency-aware reconstruction and multi-scale signal modeling have been explored in previous studies [2,3], while approaches to handling heterogeneous EEG montages and channel layouts have also been investigated [4,5]. However, the paper does not adequately discuss how the proposed components differ from these existing approaches, nor does it provide sufficiently comprehensive comparisons with closely related EEG foundation models, particularly LUNA and REVE. These comparisons are important for establishing whether the reported gains arise from genuinely new methodological contributions rather than alternative implementations of existing ideas.
  3. The downstream evaluation covers a relatively narrow range of tasks, making it difficult to assess the broader generalization capabilities of NeuroCodex. Given that the primary motivation is to address heterogeneous EEG channel layouts, evaluations involving unseen montages, cross-dataset transfer, and a more diverse set of downstream tasks would be particularly valuable.

[1] Apicella, A., Isgró, F., & Prevete, R. (2024). Don’t push the button! Exploring data leakage risks in machine learning and transfer learning. Artificial Intelligence Review, 58. [2] Barmpas, K., Lee, N., Chalatsis, D., Raftery, W.E., Panagakis, Y., Adamos, D.A., Laskaris, N.A., Koliousis, A., Farina, D., & Zafeiriou, S.P. (2025). NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models. [3] Sukhbaatar, J., Imamura, S., & Tanaka, T. (2026). BandVQ: Band-Wise Vector-Quantized EEG Foundation Model. ArXiv, abs/2605.24921. [4] Döner, B., Ingolfsson, T.M., Benini, L., & Li, Y. (2025). LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis. ArXiv, abs/2510.22257. [5] El Ouahidi, Y., Lys, J., Thölke, P., Farrugia, N., Pasdeloup, B., Gripon, V., Jerbi, K., & Lioi, G. (2025). REVE: A Foundation Model for EEG - Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects. ArXiv, abs/2510.21585.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.