acceptodds
Under review as a conference paper at ICLR 2027

CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models

Abstract

Electroencephalography (EEG) signals are a critical, non-invasive method to monitor electrical brain activity. They play a major role in neuromuscular and biomechanical settings, as they are used in clinical settings to monitor sleep, seizure activity, muscular response, etc. However, the length of EEGs is limitless - they can span anywhere from a couple seconds to multiple hours. This serves as a major hurdle for existing deep learning methods due to two major factors: (1) existing EEG models are predominantly built upon the attention mechanism, incurring quadratic scaling as the sequence length increases, and (2) raw EEG signals must be processed in a sliding-window fashion due to current models' requirements of fixed-length input, preventing global understanding across the entire signal as a whole. To this extent, we propose CaMBRAIN, a Causal, Mamba-based EEG model for real-time, streaming inference of brain EEG signals. We argue that bidirectional approaches to processing EEG signals are counterintuitive and needlessly expensive due to the causal, unidirectional nature of EEG. Thus, we propose a unidirectional, causal, state space model (SSM) for EEG understanding. However, training such a model is not straightforward, as crucial EEG events can be extremely brief - occurring within fractions of a second - yet may be separated by long intervals spanning minutes. While current EEG methods use self-supervised objectives to optimize for signal reconstruction, these objectives are not well suited for streaming state space models, they fail to explicitly train the hidden state to retain the salient long-range context needed for streaming inference. Therefore, we introduce a multi-stage self-supervised training pipeline specifically tailored to encourage long-range memory retention and strong performance on EEG signals, while preserving the linear-time complexity of state space models. CaMBRAIN achieves state-of-the-art (SOTA) results on multiple EEG benchmarks with 6–158 less sustained compute than EEG foundation model baselines, enabling long-range, continuous inference over variable-length EEG signals.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Mamba, Pre-Training, Foundation Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_yr1yOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper propose a new model using State Space Model for EEG tasks. The model introduces different pre-training stage using masked reconstruction, ARM and JEPA-style loss. Their model is applied on different EEG task and show better results with less computational resources.

Critical strengths
  • The paper is well written and very easy to follow. Each step of the method is clear and come with nice figure to illustrate them.
  • The motivations are clearly introduced and the results show that CaMBRAIN achieves good results with less computation time.
  • The method based on State Space Models are perfectly fit for EEG data.
Critical weaknesses

If all the introduction and methodology part is clear and well written, the experimental part lacks a bit a details:

  • The paper should mention in the main part the pre-training dataset used.
  • One claim is that CaMBRAIN is more computationally efficient. However the GFLOPS are given only for one table. I am wondering why we don't have a study of compute resource for all dataset?
  • CaMBRAIN is the best for all datasets except TUAB, but the authors pass through this without explaining why. It would be interesting to have a discussion about why potentially it is failing.
  • On the ablation study the case where only stage 2 is tested is not on the table. It could be nice to have this last ablation study to see if stage 1 is important. Also stage 1 and 2 comprise both of them two losses. Did you try to remove one of them ?
  • My last point is on the claim "real time". If I understood properly, the signal is passed entirely to the model. So for each new record t, how do CaMBRain works ? Do it need to repass all the signal to the model? I don't really see the difference with the other model. Each model is taking a part of the signal to represent the last step. The differences comes from what you are taking to represent the last token no?
Questions for authors
  • Do you re-train all the model are take the value of the orignal papers. If yes which values ? I'm wondering how comparable are this results. If CamBRAIN needs to be retrain for each new datasets, the computational time can be high against FM.

  • If you chose a step delta, you have a delay to detect anomalies no ? Is it possible to get rid of this step and directly give time step to the model rather than passed to an encoder first ?

  • Do you think having a bigger pre-training set will increase the results ? Did you check the results with increasing number a training data to see if you reach a plateau ?

5 comments
AuthorsOpenReview ↗

Response to Reviewer yr1y

We thank the reviewer for the constructive review and the detailed questions. We address each point below.

W1 -- Pretraining dataset in the main text

Thank you for the suggestion, we have added details of our pre-training dataset into the main text (highlighted in green).

W2 -- Compute across datasets

We have extended compute reporting to MAT, TUAB and TUAR in Table R3 below. Note that the only difference in latency comes from the numbers of EEG channels in each dataset: MAT uses 19 channels, whereas TUAB and TUAR use 22. Besides that, CaMBRAIN and all baselines are dataset-agnostic with respect to compute cost.

Table R4: GLOPS/s measurements on all datasets reported in the paper.

ModelMATTUABTUAR
BIOT17.7420.5420.54
LaBraM19.2322.4922.49
CBraMod15.0717.4817.48
LUNA-L45.8145.9946.00
EEGMamba12.2814.2214.22
REVE-base216.78252.19252.20
CaMBRAIN1.231.241.24

W3 — TUAB performance gap

Published TUAB results improve with larger bidirectional models. Our focus is continuous, real-time inference, so we prioritize a lightweight causal readout and low computation per update. Please note Parameter count alone is not a like-for-like comparison: at any time step a bidirectional model sees both past and future samples in its window, while a causal model sees only the past. We discuss causality and bidirectional processing further in the paper in §3.1 and Appendix A.19 (Table 22).

Table R5: Comparison between model size and TUAB performance.

ModelParameters (M)TUAB AUROCGFLOPs/s ↓
LUNA-Base7.100.88688.33
LUNA-Large43.110.892445.99
LUNA-Huge311.810.8957321.95
CBraMod — standard classifier4.880.889217.48
CBraMod — large classifier69.290.915627.47
CaMBRAIN37.000.86681.24

W4 -- Stage-2-only

We provide a Stage-2-only experiment on the TUAR dataset below in Table R6. The table shows that combining both of our proposed stages of training yields the best results - we have added this row to our ablation table in the edited rebuttal pdf (highlighted in green).

Table R6: Ablation table with Stage 2-only training included. Using both stages of our proposed method yields best results.

InitializationAUROCAUPR
Random0.9370.750
Stage 1 only0.9430.778
Stage 2 only0.94260.7895
Both stages0.9600.859

We have not separately ablated the individual loss terms. As described in §§3.2.1–3.2.2, the masked objectives train the model to recover missing EEG content or representations from causal context, while the autoregressive objectives encourage the current hidden state to retain information predictive of future activity. We give further motivation for our training strategy and its alignment with state space models in our response to W1(a) of Reviewer HmMm.

W5 -- CaMBRAIN Streaming

CaMBRAIN processes only the EEG patch when it arrives, updates its hidden state (which compresses all previously seen context), and produces the next prediction. Earlier EEG samples are not re-computed since the recurrent hidden state are carried forward over time, keeping per-update computation constant and independent of recording duration (see Fig. 1).

Q1 -- Baseline results and evaluation

Thank you for pointing this out - we fine-tuned LUNA and REVE on CHB-MIT and have added this note in the edited rebuttal pdf. All other baseline scores are taken directly from the original papers. Please note LUNA and REVE also perform downstream fine-tuning, which we also identically follow.

Moreover, CaMBRAIN achieves higher reported AUROC on CHB-MIT than LUNA and REVE under both fine-tuning and frozen-encoder linear probing (Table 1b; Table 18).

Q2 -- Step size and anomaly-detection delay

We follow the same patch tokenization method as LUNA and REVE to enable channel-agnostic EEG inference. Bypassing this tokenizer would remove the mechanism that supports performing EEG inference on any number of channels (as discussed in W2 above). This tokenization is still causal: each token uses the current, already acquired patch, and temporal processing accesses only current and past tokens.

Q3 -- Pretraining-data scaling

Larger and more diverse pretraining data could potentially further improve performance. We trained on the full TUEG corpus (~21,000 hours), while REVE uses over 60,000 hours across 92 datasets and reports stronger TUAB performance. Despite this, our current results demonstrate strong TUAR, CHB-MIT and MAT performance (Tables 1–2) alongside compute 158× lower than REVE (Table 1b). Using a larger pre-training corpus could serve as one promising avenue of future work.

I thank the authors for their answers that clarify most of my points. I am still concerned about the choice of fine-tuning Luna and Reve on CHB-MIT. Why only these two for this dataset ? Having different models for different datasets seems like cherry picking the good competitors for each datasets. Having a clear discussion for all these choices could help to strengthen the trust in the results.

AuthorsOpenReview ↗

Reply to comment by Reviewer yr1y

We thank the reviewer for the prompt follow-up and the opportunity to clarify our baseline choices.

We chose LUNA and REVE because they are two of the strongest available EEG encoders. We fine-tuned them on CHB-MIT because we wanted to evaluate their performance in continuous streaming. Adding causal EMA or trained GRU memory to their window representations did not close the forecasting gap to CaMBRAIN (Table 14). We would also like to point out LUNA and REVE do not report on CHB-MIT

On other datasets, we compare with their published results on MAT, TUAB and TUAR.

CaMBRAIN also achieves higher AUROC than LUNA and REVE in linear probing experiments (Table 18), supporting the fact that CaMBRAIN's training regiment led to better quality representations without encoder fine-tuning.

Reviewer_HmMmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CaMBRAIN, a causal Mamba-3 model that updates a persistent hidden state from multi-channel EEG patches. The model is pretrained using an initial causal autoregressive/reconstruction stage followed by an EMA teacher–student latent-prediction stage, and is then fine-tuned with a linear task head. Experiments cover artifact detection, abnormality classification, mental stress, and seizure detection, with competitive accuracy and substantially lower claimed sustained computation than window-based foundation models.

Critical strengths
  1. The paper targets a meaningful deployment regime that is often neglected. Maintaining a causal persistent state is a natural formulation for bedside monitoring and wearable EEG, where future samples are unavailable and repeatedly encoding overlapping windows is wasteful. The recurrent SSM implementation provides a clear computational advantage for sufficiently frequent online updates.

  2. The architecture and learning objectives are coherently motivated. A causal Mamba backbone, representation-level teacher–student refinement, and prediction from the current state all align with the stated goal of online inference. The paper also distinguishes architectural persistence from simply attaching a downstream GRU or EMA to windowed features.

  3. The appendix contains several useful diagnostic experiments. The authors report subject overlap, a subject-disjoint TUAR result, causal-preprocessing checks, streaming/parallel numerical agreement, event-level seizure statistics, reset-interval experiments, state half-lives, latency, memory, and pretraining ablations. This transparency materially improves the paper.

Critical weaknesses
  1. The training setup provides limited support for the claimed long-range memory. Both pretraining stages operate on five-second windows, the Stage-2 future horizon is only four 62.5 ms patches, and the positional embedding repeats every 80 patches, or five seconds. Thus, the state is never directly trained to preserve information over minutes or hours. The CHB-MIT reset study suggests useful context beyond ten seconds, but intermediate horizons such as 30 seconds or several minutes are not tested, and accuracy over multi-hour streams is not evaluated; constant computational cost alone does not establish arbitrarily long effective memory.

  2. The headline experiments are not fully causal end to end. The reported benchmark pipeline uses zero-phase filtering and window-level robust quartile normalization, both of which access future samples within an evaluation window. The appendix provides a causal-filter rerun on CHB-MIT and a causal-normalization result for only one seed, but no equivalent end-to-end causal evaluation on TUAR, TUAB, or MAT.

Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer HmMm

We thank the reviewer for the careful and constructive review. We address each weakness below.

W1(a) — Training for long-range memory

This is precisely our motivation in using a state space model. In the original Mamba paper [1], the authors explain in Sec 3.5.2 how its selection mechanism retains relevant information while ignoring intervening inputs, and further demonstrate successful extrapolation from 256-token training sequences to over one million tokens on inference in Sec 4.1.2.

The idea behind our pretraining follows this same intuition: Stage 1 (§3.2.1) learns EEG temporal structure through causal prediction and masked reconstruction, while Stage 2 (§3.2.2) predicts masked and future teacher representations directly from the current hidden state, encouraging it to retain information useful for subsequent EEG activity.

Building on Table 3(a) from the main paper, we have ablated the performance of pretrained CaMBRAIN vs fine tuning only on persistant inference. As seen in Table R2, CaMBRAIN gets a substantial boost in both AUROC and AUPR when performing continuous inference longer than 5s due to the benefits of our pretraining strategy.

Table R2: AUROC and AUPR while streaming inference on CHB-MIT finetuned when pretrained vs. when not pretrained (mean over 3 seeds).

TrainingAUROC (%)AUPR (%)
Fine-tuning only87.4932.24
Pretrained + fine-tuned88.92 (+1.43 pp)47.59 (+15.35 pp)

Moreover, stateful retention beyond the training-window duration is demonstrated both in our paper through continuous-inference and reset experiments (Table 3(a), Appendix A.12 and Table R2) and in [1].

Please see Table R1 in our response to W2/Q1 of reviewer vwMn for additional longer-window training results.

W1(b) -- Long-range memory evaluations

As discussed in Sec 3.1 and Table 5 in the appendix, SSMs parallel computation during training and sequential state updates during streaming inference. In Appendix A.12, we examine how the hidden state evolves over time, specifically how long previous context stays in the hidden state. Our analysis reveals that despite training on 5s windows, the median half-life context decay from previous windows is still 18.7s.

Moreover, we provided results on multi hour-long streaming evaluations in Table 3(a) of the main paper. CHB-MIT contains EEG seizure recordings of up to 4 hours. Furthermore, the benefit of maintaining a persistent state over long streams is demonstrated through the improved AUROC from 0.874 to 0.889 and BAC from 0.666 to 0.723, compared to resetting the state every 5s.

W2 -- End-to-end causal preprocessing

We have extended the end-to-end causal preprocessing evaluation to all four datasets, with three seeds per dataset in Table R3 below. The original preprocessing followed previous works to ensure fairness. Table R3 shows that causal preprocessing achieves nearly identical signal characteristics as non-causal processing. Thus, this has a negligible impact on the results reported in the paper.

Table R3: Similarity between causal and non-causal EEG preprocessing on all datasets. Similarities are extremely high, with negligible impacts on performance.

DatasetFilter / spectral similarityChange in AUROC: non-causal → causal
CHB-MITSpectral-feature correlation: 0.99950.9205 → 0.9150 (−0.0055)
MATLog-bandpower correlation: 0.9948–0.99830.8758 → 0.8776 (+0.0018)
TUABLog-bandpower correlation: 0.9590–0.99940.8668 → 0.8665 (−0.0003)
TUARLog-bandpower correlation: 0.9704–0.99950.9362 → 0.9351 (−0.0011)

[1] Gu, A., & Dao, T. (2023). Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752.

Thanks to the authors for the detailed response and additional experiments. However, the results still do not provide sufficient evidence that the model effectively retains and uses long-range information beyond the short training windows. Therefore, my original score remains unchanged.

We thank the reviewer for the follow-up and the opportunity to clarify our experiments.

We designed certain experiments in the main paper to demonstrate the benefit of context beyond the 5s training window. Our experimental setup is as follows:

We compare resetting the hidden state every 5s (training context) against preserving it throughout complete EEG recordings, including recordings lasting several hours.

Here are the relevant comparisons in the paper and rebuttal:

  • Sec. 4.3.1, Table 3(a): Retaining state across complete recordings increases sparse-patch AUROC from 0.874 to 0.889 and BAC from 0.666 to 0.723, compared with resetting every 5s.
  • Appendix A.4, Figure 3: Persistent and windowed predictions are illustrated on a held-out recording, showing greater prediction stability during a segment of ictal noise.
  • Appendix A.11, Table 12: Across three held-out patients and 26 seizure onsets, retaining state reduces false alarms from 44.0 to 10.5 per hour at matched 100% event sensitivity.
  • Appendix A.12, Table 13: Persistent inference produces higher seizure-onset probabilities than resetting state every 5, 7 or 10s.
  • Appendix A.13, Table 14: Adding causal EMA or trained GRU aggregation to windowed LUNA and REVE representations does not close the reported continuous-forecasting gap to CaMBRAIN.
  • Table R2: We compare pretrained + fine-tuned models with fine-tuning-only models during continuous inference on CHB-MIT, reporting AUROC and AUPR. In both settings, the hidden state is preserved throughout each complete recording, including recordings lasting several hours.

As mentioned in our previous reply this intuition is also discussed in the original Mamba paper [1], where input-dependent selection enables the model to learn which information to preserve and which intervening inputs to ignore. The original mamba paper also demonstrates long-context generalization a model trained on short context. Our pretraining follows the same principle.


[1] Gu, A., & Dao, T. (2024). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. First Conference on Language Modeling (COLM). arXiv:2312.00752.

Reviewer_vwMnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a causal, Mamba-based state space model (CaMBRAIN) for real-time, streaming inference on EEG. The authors argue that existing models are poorly suited to streaming because they do not explicitly train hidden states to retain long-range context. They propose a multistage self-supervised training pipeline to encourage long-range memory retention. They report performance comparable or superior to major existing models on several EEG benchmarks, with substantially lower sustained compute.

Critical strengths
  1. High practical relevance. The work addresses an important and long-standing need in brain research and clinical practice: continuous, real-time EEG monitoring. Clinical and practical settings, such as ICU and epilepsy monitoring, long-term ambulatory recording, and wearable or home-based applications, require models that operate causally, efficiently, and over long durations. The proposed approach aims to directly address these requirements.
  2. Clear motivation and efficiency. The paper identifies a specific limitation of existing approaches for streaming applications and proposes a training strategy to address it. The reported computational savings could make continuous EEG monitoring considerably more feasible.
  3. Valid tests. The paper tests the model on several well-suited tasks—artifact detection, seizure detection, abnormality detection, and stress detection. These are indeed very important aspects EEG application in neuroscience research and practice.
Critical weaknesses

Moderate Concern--I don't think it supports rejection. Fixed-window evaluation and continuous EEG processing. The model is trained and evaluated on fixed windows of 62.5 ms. While this is efficient, it raises questions about how well the experimental protocol supports the paper's central motivation of continuous, streaming inference with long-range memory. EEG phenomena relevant to monitoring, such as seizures, stress, and artifacts, are highly dynamic, with artifacts being be rather random, varying widely in duration and evolving over seconds to hours. Given this temporal variability, it is important to consider whether dynamically adapting the effective temporal context to EEG signal characteristics could further improve performance. however, I suspect that it would increase the compute load.

I encourage the authors to clarify how hidden states are handled between windows and justify the chosen window length. Ideally, they would also evaluate whether performance benefits from accumulated temporal context or the use of different sampling windows that may better capture the dynamics. Such an evaluation would provide stronger support for the proposed application to real-world EEG monitoring.

Questions for authors

Clarify how hidden states are handled between windows and justify the chosen window length. Would performance benefit from accumulated temporal context or the use of different sampling windows that may better capture the dynamics.

2 comments
AuthorsOpenReview ↗

Response to Reviewer vwMn

We thank the reviewer for the constructive review and for recognizing the practical relevance of continuous EEG monitoring. We address each point below

W1 / Q1 — Fixed-window evaluation and continuous EEG processing

The 62.5 ms duration refers to an input patch. Training uses 5s windows consisting of 80 patches. The central benefit of CaMBRAIN being SSM-based is its hidden state, which continuously retains EEG signal information from previous windows during inference. This key distinction enables CaMBRAIN to make predictions based on a learned summary of the EEG history up until the current timestep, beyond the limited view of independently processed windows used by previous methods (see Fig. 1 and Lines 86–93). To support fair comparison with windowed baselines, we report 5s window-level evaluations in Tables 1–2, while separately demonstrating continuous streaming and the benefits of preserving state in Table 3(a) of the main paper.

We thank the reviewer for reconciling the potential benefits of longer temporal context with its computational implications. Windowed encoders can incorporate longer context through downstream aggregation, which we examine in Appendix A.13, Table 14. Adding a causal EMA or trained GRU over LUNA and REVE’s window representations does not close the forecasting-performance gap to CaMBRAIN in this experiment. We agree that it should be possible to dynamically adapt the effective temporal context but as our experiment demonstrates it is non-trivial.

W2 / Q1 -- Effect of accumulated temporal context

During training, CaMBRAIN ingests a 5s EEG window to make a prediction. In the main paper, we show results on two different evaluation protocols: standard 5s windows in Tables 1-2 (as an exact comparison to previous works), and continuous inference where the entire EEG signal is passed continuously to CaMBRAIN in Table 3(a). Note that continuous inference is a unique ability of CaMBRAIN only. We discuss this in Lines 206 - 240 and Sec. 4.3.1. To explicitly clarify this, we have added details regarding our training and evaluation setups in Section 4 of the edited rebuttal pdf (highlighted in green).

Regarding window length generalization, we show in Sec. A.12 of the appendix that CaMBRAIN generalizes to longer EEG sequences despite being pre-trained on 5s windows.

To further highlight this, we show results in Table R1 below when training CaMBRAIN with various window sizes on CHB-MIT, the dataset with the longest raw EEG signals (hours-long). SSMs can already generalize past the context length they are trained with, due to their hidden state, hence the minimal AUROC gains when training CaMBRAIN with longer window sizes. We would also like to point out that it becomes expensive to train as the window size increases.

Please see our response to W1(a) from reviewer HmMm for more details.

Table R1: AUROC on CHB-MIT when fine-tuning on various window sizes.

Training window sizeAUROC
5s — paper benchmark0.9205
30s0.9162
60s0.9326

We also examine patch-size sensitivity in Appendix A.16, Table 19, comparing 8, 16, 32 and 64 samples. The best patch size varies across datasets, with no consistent advantage from increasing it. Our 16-sample choice provides a 62.5 ms update interval, while the persistent state accumulates context across patches.

We thank the reviewer for their time and consideration. With less than 24 hours left in the discussion, we would appreciate hearing whether our responses have addressed their concerns. We are happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.