acceptodds
Under review as a conference paper at ICLR 2027

BrainMamba: A Scalable EEG Foundation Model for Coupled Multi-Channel Spatiotemporal Evolution

Abstract

Large-scale representation learning from heterogeneous electroencephalography (EEG) data is essential for building generalizable EEG foundation models across diverse brain states, subjects, and downstream tasks. However, building scalable EEG foundation models faces two fundamental challenges: (1) data heterogeneity, as diverse channel configurations, acquisition protocols, and task settings hinder unified large-scale pretraining and cross-dataset transfer; and (2) neural dynamics modeling, as existing backbones struggle to capture the coupled spatial interactions and temporal evolution intrinsic to multi-channel brain activity. To address these challenges, we propose BrainMamba, a scalable foundation model that formulates large-scale EEG modeling as structured neural state evolution. At its core, BrianMamba models multi-channel EEG as a coupled dynamical system, where latent neural states evolve over time through structured state-space transitions while being continuously shaped by cross-channel interactions. By jointly capturing temporal evolution and inter-channel coupling, this formulation enables coherent modeling of distributed brain dynamics over long temporal contexts. To further accommodate the substantial heterogeneity of EEG recordings, the state evolution is adaptively specialized through expert routing, allowing different neural patterns across subjects, tasks, and recording conditions to be captured within a unified backbone. This state-centric design naturally scales across to heterogeneous channel configurations and sequence lengths, providing a unified foundation for large-scale EEG pretraining. BrainMamba adopts a large-scale mask-reconstruction-based pretraining paradigm, where masked spatiotemporal segments are reconstructed to learn EEG contextual semantic representations. For evaluation, we propose a novel validation paradigm for EEG foundation models, comprising task-constrained pretraining, channel-consistent transfer, and xwself-evaluation, to rigorously evaluate model generalization and robustness. Extensive experiments on more than 10 datasets across emotion, cognition, and disorder benchmarks demonstrate that BrainMamba consistently outperforms existing EEG foundation models, exhibiting superior cross-scenario generalization and strong scalability, highlighting its potential for real-world deployment.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Foundation Model, Representation Learning, Self-supervised learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3695OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed method, BrainMamba, integrates a coupled multi-channel Mamba state update, periodic spatial/temporal Transformers, mixture-of-experts routing, and masked reconstruction, pretrained on heterogeneous EEG corpora. Evaluations cover emotion, motor imagery, cognition, and clinical datasets.

Critical strengths
  • Multi-corpus pretraining and heterogeneous montage handling.
  • Comparisons with LaBraM, CBraMod and other EEG models, plus corpus-sampling analyses.
Critical weaknesses
  • The central generalization claim is confounded by pretraining on the unlabeled target datasets before fine-tuning. The paper reports strong performance across more than ten datasets. There is a fundamental problem in the experimental setting: the pretraining was done on all available unlabeled EEG datasets followed by downstream evaluation on those same datasets. This causes a serious data leakage issue. The basic rule for machine learning is: test data should never be used in any stage of the training, no matter whether their labels are used or not; otherwise, the purpose of evaluating generalizability is forfeited. Authors should do a leave-one-dataset-out pretraining experiment, where the target dataset is excluded entirely from pretraining.
  • Eq. 4 collapses cross-channel interactions into a shared transformed sum, potentially modeling common-reference and volume-conduction effects instead of neurophysiological connectivity.
  • The rank argument for a later Transformer recovering information already discarded by a state bottleneck is not convincing without residual information paths.
Questions for authors
  1. Does BrainMamba genuinely model spatial interactions between EEG channels? Or, how can the model learn meaningful spatial connectivity when the coupling operator does not distinguish individual channel pairs?
  2. Does the experimental protocol demonstrate genuine foundation-model generalization?
  3. How does your proposed architecture (an integration of several other known components) bring new insights?
Reviewer_wpf1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a BrainMamba architecture for training EEG foundation models. The paper identifies two key challenges faced by existing EEG foundation models: data heterogeneity and coupled neural dynamics modeling. These challenges are addressed by adding expert routing to the model architecture and incorporating cross-channel interactions into the state updates based on the existing Mamba framework. This architecture consistently achieves improved performance on various downstream tasks after completing the pre-training phase.

Critical strengths
  • The article is written clearly and is easy to follow. It clearly identifies the two challenges faced by existing methods and provides solutions to these challenges. The effectiveness of the design was also proved through experiments.
Critical weaknesses
  • In the appendix of this paper, the ablation experiments for each module have been provided. However, there is a lack of ablation experiments for the core "cross-channel state coupling" of the module. Therefore, it is not possible to effectively explain where the benefits of the article come from. The author needs to add experiments where this module is replaced with a regular Mamba or where the strength of cross-channel coupling is changed, in order to demonstrate the effectiveness of this module.
  • This paper lacks some sensitivity experiments on key hyperparameters. The most crucial issue is whether the low-rank mentioned by the authors is caused by dstaste=16 for the state dimension. Can increasing the state dimension solve the low-rank problem? And will the spatio-temporal transformer still bring the same benefits? The paper does not explain whether multiple seeds are used during fine-tuning for downstream tasks, which cannot demonstrate the robustness of the results.
Questions for authors

See weeknesses.

3 comments
AuthorsOpenReview ↗

Kind Reminder to Review Our Rebuttal Responses

Dear Reviewer, thank you very much for your time and consideration. As the rebuttal period will end soon, we would like to kindly remind you that we have provided detailed responses to your comments and concerns. We would greatly appreciate it if you could take a moment to review our rebuttal before the deadline. Thank you again for your valuable feedback.

Thanks for the response. Most of my previous concerns have been resolved. I will keep the postive rating unchanged.

AuthorsOpenReview ↗

Reply to wpf1: Thank You for Your Positive Feedback

Thank you very much for your positive feedback and for taking the time to carefully consider our responses. We are very pleased to hear that most of your previous concerns have been resolved.

We sincerely appreciate your recognition of our efforts and your decision to maintain a positive rating. Your thoughtful comments and constructive suggestions have been invaluable in helping us improve the quality and clarity of our work.

Thank you again for your time!

Reviewer_tuz3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BrainMamba, an EEG foundation model that combines coupled state-space modeling, spatiotemporal Transformers, and Mixture-of-Experts (MoE) layers. The model is pretrained on heterogeneous EEG datasets using masked reconstruction and different data sampling strategies. Experiments cover 10 datasets, multiple pretraining and transfer settings, including leave-one-dataset-out (LODO) evaluation, and an independently collected sleep EEG dataset.

Critical strengths

S1. Extensive experimental evaluation

The paper evaluates multiple pretraining and transfer settings across 10 EEG datasets, including LODO experiments. This provides a relatively comprehensive empirical assessment of cross-dataset transfer.

S2. Additional validation on independently collected EEG data

The independently collected sleep EEG dataset is a valuable addition, providing further evidence of transferability across tasks and recording conditions.

Critical weaknesses

W1. Insufficient discussion and comparison with important related work

The paper overlooks relevant EEG foundation models such as MMM (Learning Topology-Agnostic EEG Representations with Geometry-Aware Modeling), REVE, and LUNA. Given the substantial existing research on heterogeneous electrode configurations and spatial modeling, their connections and differences deserve clearer discussion and comparison.

W2. Inconsistent EEGMamba baseline reference

The experimental tables identify EEGMamba as a Neural Networks 2025 model, while the corresponding citation refers to Gui et al. (2024), which is a different work. The baseline implementation and citation should be clarified.

W3. Limited methodological novelty relative to prior work

The core state-space formulation closely resembles the earlier State Mamba. Moreover, hybrid Transformer–Mamba–MoE architectures have already been explored in works such as Jamba. The paper does not sufficiently establish its distinctive methodological contribution.

W4. Insufficient theoretical justification for the low-rank bottleneck

The low-rank analysis concerns the rank of an individual state-transition mapping. It does not directly establish a corresponding representational bottleneck in the full model or justify the claim that periodically inserted Transformer layers can recover high-rank information.

W5. Limited expressiveness of the proposed cross-channel coupling

The coupling formulation can be expressed using a shared global state aggregation and channel-specific self-state terms, without explicit electrode-pair-specific transition matrices. Its ability to capture complex cross-channel coupling dynamics therefore requires stronger justification.

W6. Inconsistent descriptions of the pretraining loss

Equation (7) specifies a squared L2 reconstruction loss, whereas Appendix A.4.2 states that Huber loss (β = 1.0) is used during pretraining. This inconsistency should be resolved.

W7. Insufficient details on data splitting and reproducibility

Appendix A.4.2 outlines LOSO, cross-session, and cross-subject evaluation strategies. However, the exact training, validation, and test partitions, validation procedures, and related reproducibility details remain insufficiently specified.

W8. Insufficient evidence for the scalability claim

The paper emphasizes scalability but lacks direct computational complexity and efficiency evaluations, such as FLOPs, memory usage, and runtime scaling with sequence length and channel count.

Questions for authors

Q1. What are the key methodological differences between the proposed State Mamba module and the original State Mamba? How does the theoretical analysis establish that Transformer layers overcome the claimed low-rank bottleneck?

Q2. Could the authors clarify the actual pretraining loss, the EEGMamba baseline used in the experiments, and the exact data partitions and validation procedures?

Q3. Could the authors provide computational complexity or efficiency measurements across different sequence lengths and channel counts to substantiate the scalability claim?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.