acceptodds
Under review as a conference paper at ICLR 2027

NeuroOnline: Bridging Pretraining and Online Adaptation for EEG Foundation Models

Abstract

EEG foundation models have shown strong potential in learning generalized representations across subjects and tasks. However, most existing approaches follow a pretraining–static deployment paradigm, which suffers from two key limitations: (1) misalignment between pretraining objectives and downstream tasks, and (2) limited adaptability to distribution shifts in online settings. We propose Online Neural Adaptation (NeuroOnline), a unified framework that enables continuous adaptation in online scenarios. NeuroOnline integrates two complementary mechanisms: (1) multi-view consistency learning, which enforces cross-view alignment to promote consistent representations and improve robustness to distribution shifts; (2) context-aware representation modulation, which uses cross-attention between a learnable prompt and EEG representations to enable adaptive modulation under evolving data distributions. Together, these mechanisms unify representation alignment and dynamic adaptation. Experiments on multiple EEG benchmarks demonstrate that NeuroOnline consistently outperforms strong baselines in online settings and achieves robust performance under distribution shifts. Extensive experimental results further validate the effectiveness and robustness of the proposed framework. The code is available at https://anonymous.4open.science/r/NeuroOnline-25FB.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG Foundation Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9DwcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NeuroOnline, a framework for supervised online adaptation of EEG foundation models to address pretraining-downstream task misalignment and non-stationary distribution shifts. The framework integrates two components, including Multi-view Consistency Learning and Context-aware Representation Modulation. Experiments across 8 EEG datasets on 4 foundation models demonstrate improvements over offline fine-tuning, conventional supervised online adaptation, and a test-time training baseline.

Critical strengths

1.The motivation is convincing

2.The evaluation is comprehensive, covering 8 datasets across 4 pretrained backbones.

Critical weaknesses

1.Although the paper emphasizes strictly causal, predict-then-update learning, Table 6 reports three training epochs and a batch size of 64 for online methods. It remains unclear how these settings are implemented in a sequential data stream, whether previously observed samples are revisited, and whether predictions are always evaluated before the corresponding labels become available.

2.Multi-view consistency regularization and prompt-conditioned feature modulation are conceptually related to established representation learning and feature-wise adaptation techniques. It would be helpful to show more evidence on how such techniques reduce pretraining-downstream misalignment.

3.The computational analysis focuses on forward GFLOPs and compares NeuroOnline against an offline baseline. It would be better to account for the complete cost of online backpropagation and multi-view augmentation.

Questions for authors

See my weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9Dwc (1/3)

We sincerely thank the reviewer for their helpful suggestions.

W1: Predict-then-Update Protocol.

Response:

We further clarify the online data processing and model update procedures to ensure a more transparent and reproducible evaluation protocol.

Online Update Schedule. NeuroOnline maintains a historical buffer with a capacity of 320 labeled EEG samples, managed using a first-in, first-out (FIFO) strategy. Each incoming sample is first used for prediction, and its label is revealed only afterward. The newly observed sample is then added to the buffer, with the oldest sample removed when the buffer reaches capacity. An online update is triggered every 64 newly observed samples. During each update, all samples in the buffer are used for optimization with a mini-batch size of 64 over 3 epochs. This strategy enables the model to periodically incorporate recent labeled observations while retaining historical information for continuous adaptation.

The complete online optimization configuration is summarized in Table R6. These additional details clarify the online update schedule and historical sample utilization, improving the reproducibility of our evaluation protocol.

Table R6. Hyperparameter Settings for Online Evaluation.

HyperparametersSettingsDescription
Subject OrderingChronological, subject-wiseSubjects are processed sequentially in temporal order
Historical Buffer320Maximum number of previously observed samples stored in the buffer
Update Stride64An online update is triggered after every 64 newly observed samples
Batch Size64All 320 buffered samples are used for optimization, with a mini-batch size of 64
Online Epochs3Samples are used for 3 training epochs
OptimizerAdamWOptimizer used for online adaptation
Learning Rate1e-4Learning rate for online optimization
Weight Decay5e-2Weight decay coefficient
Label Smoothing (Multi-class)0.1Label smoothing coefficient for multi-class classification
AuthorsOpenReview ↗

Response to Reviewer 9Dwc (2/3)

W2: Pretraining–Downstream Task Misalignment.

Response:

Alignment with downstream tasks. To provide more direct evidence of how MCL and CRM mitigate pretraining–downstream task misalignment, we conduct an additional representation-level analysis using LogME (You et al., 2021), which quantifies the compatibility between EEG representations and downstream task labels based on the marginal likelihood of a Bayesian linear model. Higher LogME indicates better alignment between the representations and downstream labels. Under the same evaluation protocol, we compare NeuroOnline with the following four configurations.

(i) Pretrained CBraMod. We extract representations from the pretrained CBraMod backbone before downstream fine-tuning.

(ii) off-FT-MLP (fullfine-tuning). We attach an MLP classification head and fine-tune the entire model end-to-end.

(iii) NeuroOnline w/o MCL. We remove both augmented views together with the consistency objective.

(iv) NeuroOnline w/o CRM. We remove the CRM module.

Results. As shown in Table R10, NeuroOnline achieves higher LogME than all ablated variants. Together with the ablation results in Table 4 (the original manuscript), these results suggest that MCL and CRM help mitigate pretraining–downstream task misalignment by making the learned representations more informative for downstream classification.

Table R10. Downstream Task Alignment of EEG Representations Measured by LogME.

MethodBCIC-2A ↑BCIC-2B ↑MentalArithmetic ↑
Pretrained CBraMod-0.5324 ± 0.0000-0.6823 ± 0.0000-0.3212 ± 0.0000
off-FT-MLP (fullfine-tuning)-0.4958 ± 0.0138-0.6294 ± 0.0153-0.1967 ± 0.0309
NeuroOnline w/o MCL-0.4373 ± 0.0087-0.5612 ± 0.01240.0548 ± 0.0251
NeuroOnline w/o CRM-0.4239 ± 0.0092-0.5580 ± 0.0109-0.0106 ± 0.0264
NeuroOnline-0.3823 ± 0.0096-0.3735 ± 0.01310.2310 ± 0.0275

[1] You, Kaichao, et al. "Logme: Practical assessment of pre-trained models for transfer learning." International conference on machine learning. Pmlr, 2021.

AuthorsOpenReview ↗

Response to Reviewer 9Dwc (3/3)

W3: Computational Cost of Online Adaptation.

Response:

Computational cost. We extend our computational cost analysis to include inference latency, data augmentation time, online update time (including forward and backward propagation), and peak GPU memory consumption. As shown in Table R11, NeuroOnline incurs additional computational overhead compared with conventional online full fine-tuning, primarily due to multi-view augmentation and consistency-based optimization. Nevertheless, the increase in inference latency is relatively modest, while data augmentation introduces less than 0.5 ms of overhead per segment.

Asynchronous model updates. To avoid blocking online prediction during adaptation, we analyze NeuroOnline’s robustness to asynchronous model updates in Section 4.8 (the original manuscript). NeuroOnline can be deployed with separate inference and adaptation processes. The inference process continuously handles incoming EEG segments using the current model, while the adaptation process updates a model replica in parallel and atomically activates the updated model upon completion. Since an update may not finish immediately, inference may temporarily rely on stale parameters. We therefore simulate synchronization delays of s∈0,64,128s \in \\{0, 64, 128\\} EEG segments, where ss denotes the number of subsequent EEG segments processed by the currently deployed model before the updated model is activated. The corresponding delays are approximately 0, 5.3, and 10.7 minutes on Mumtaz2016 and 0, 4.3, and 8.5 minutes on BCIC-2A, respectively. The case s=0s = 0 corresponds to immediate synchronization. As shown in Fig. 3(F-G), NeuroOnline maintains comparable performance for delays of up to 128 EEG segments, with only limited degradation across the evaluated datasets. This indicates robustness to moderate parameter staleness. By decoupling inference from adaptation, the asynchronous design avoids blocking prediction while introducing only a bounded delay in deploying updated parameters, supporting practical deployment in latency-sensitive EEG applications.

Notes. We compute update time as follows:

= \frac{\text{Total online update time (including the original segments only)}}{\text{Number of Segments in the Test Stream (including the original segments only)}}.$$ $$\text{Update Time (NeuroOnline)} = \frac{\text{Total online update time (including the original segment and its time-masked and frequency-masked views)}}{\text{Number of Segments in the Test Stream (including the original segments only)}}.$$ **Table R11. Computational Cost Comparison of NeuroOnline with Offline and Online Full Fine-Tuning on BCIC-2A and Mumtaz2016.** | Method | BCIC-2A | | | | Mumtaz2016 | | | | | :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | | Latency (ms) | Aug. Time (ms/seg.) | Update Time (ms/seg.) | Peak GPU Memory (GB) | Latency (ms) | Aug. Time (ms/seg.) | Update Time (ms/seg.) | Peak GPU Memory (GB) | | off-FT-MLP | 8.58 | — | — | 0.233 | 8.61 | — | — | 0.250 | | on-FT-MLP | 8.80 | — | 6.59 | 0.745 | 8.87 | — | 12.95 | 0.633 | | **NeuroOnline** | **9.99** | **0.43** | **19.15** | **4.352** | **9.43** | **0.42** | **37.66** | **4.617** |
Reviewer_4tn1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed NeuroOnline combines consistency learning across augmented EEG views with prompt conditioned representation modeling. The model predicts incoming samples and subsequently adapts using their labels. Experiments cover widely used datasets in its empirical evaluation.

Critical strengths

The work addresses a relatively important limitation of EEG foundation models: adaptation to changing deployment conditions. The empirical evaluation across several tasks and backbones provides useful evidence of broad applicability in real world scenarios. The component ablation study are pretty extensive, examining masking, consistency etc.

Critical weaknesses

For Mumtaz2016, multiple EEG segments share the same subject-level diagnosis. If this diagnosis becomes available after an early segment, later predictions for that subject can exploit the already available information. Consequently, the reported 99.02% balanced accuracy does not necessarily establish strong diagnostic performance on previously undiagnosed subjects.

The online learning and evaluation schedule is underspecified. The authors need to clarify the update frequency, buffer size, ordering of subjects etc.

The unnormalized squared-distance loss has a potential scale ambiguity: reducing representation magnitudes decreases consistency loss, while compensatory scaling of classifier weights can preserve predictions. This may not be evidence that collapse occurred.

Additional baselines are needed to ablation on the gains. The comparison would be stronger with ordinary online adaptation using the same augmentations, a small replay buffer, and a parameter-matched adapter.

Questions for authors

Please see above

7 comments
AuthorsOpenReview ↗

Response to Reviewer 4tn1 (1/4)

We sincerely thank the reviewer for their helpful suggestions.

W1: Online Evaluation on Previously Undiagnosed Subjects in Mumtaz2016.

Response:

To further assess NeuroOnline's diagnostic capability on previously undiagnosed subjects and its robustness to different subject-sample ordering protocols, we conduct two additional experiments:

(i) Interleaved Subject-sample Evaluation. We randomly shuffle EEG samples across all test subjects to construct an interleaved data stream, rather than processing all samples from the same subject consecutively. All other experimental settings remain unchanged.

(ii) Undiagnosed-subject Evaluation. Before using any data from a test subject for online updates, we evaluate the model on the first 10 EEG samples from that subject. This setting directly assesses the model's initial diagnostic capability on previously undiagnosed subjects, without relying on subject-specific historical supervision.

As shown in Table R5, NeuroOnline achieves the best performance under both evaluation settings. These results demonstrate NeuroOnline's robustness to interleaved subject-sample ordering and its diagnostic capability on previously undiagnosed subjects.

Table R5. Evaluation of NeuroOnline under Different Subject-sample Ordering and Diagnosis Settings on Mumtaz2016.

MethodSubject-wise (Original)Interleaved Subject-sampleUndiagnosed-subject
on-FT-MLP98.70 ± 0.5996.55 ± 0.5393.09 ± 0.73
on-LoRA-MLP98.69 ± 0.6196.56 ± 0.5192.25 ± 0.28
on-MLP98.33 ± 0.5696.11 ± 0.4692.67 ± 0.55
NeuroTTT88.86 ± 0.4887.52 ± 0.3288.33 ± 0.32
NeuroOnline (Ours)99.02 ± 0.2297.33 ± 0.3194.92 ± 0.43
AuthorsOpenReview ↗

Response to Reviewer 4tn1 (2/4)

W2: Clarification of the Online Learning and Evaluation Protocol.

Response:

We further clarify the online data processing and model update procedures to ensure a more transparent and reproducible evaluation protocol.

Online Update Schedule. NeuroOnline maintains a historical buffer with a capacity of 320 labeled EEG samples, managed using a first-in, first-out (FIFO) strategy. Each incoming sample is first used for prediction, and its label is revealed only afterward. The newly observed sample is then added to the buffer, with the oldest sample removed when the buffer reaches capacity. An online update is triggered every 64 newly observed samples. During each update, all samples in the buffer are used for optimization with a mini-batch size of 64 over 3 epochs. This strategy enables the model to periodically incorporate recent labeled observations while retaining historical information for continuous adaptation.

The complete online optimization configuration is summarized in Table R6. These additional details clarify the online update schedule and historical sample utilization, improving the reproducibility of our evaluation protocol.

Table R6. Hyperparameter Settings for Online Evaluation.

HyperparametersSettingsDescription
Subject OrderingChronological, subject-wiseSubjects are processed sequentially in temporal order
Historical Buffer320Maximum number of previously observed samples stored in the buffer
Update Stride64An online update is triggered after every 64 newly observed samples
Batch Size64All 320 buffered samples are used for optimization, with a mini-batch size of 64
Online Epochs3Samples are used for 3 training epochs
OptimizerAdamWOptimizer used for online adaptation
Learning Rate1e-4Learning rate for online optimization
Weight Decay5e-2Weight decay coefficient
Label Smoothing (Multi-class)0.1Label smoothing coefficient for multi-class classification
AuthorsOpenReview ↗

Response to Reviewer 4tn1 (3/4)

W3: Scale ambiguity in the unnormalized squared-distance consistency loss.

Response:

To investigate the potential scale ambiguity in the unnormalized squared-distance consistency loss, we conduct two additional analyses.

(i) L2-Normalized Consistency Analysis. We introduce a variant that normalizes the modulated representations before computing the consistency loss. As shown in Table R7, the normalized variant achieves comparable downstream performance to the original formulation across all three datasets, with only marginal differences. This suggests that the effectiveness of NeuroOnline does not substantially depend on the scale sensitivity of the original consistency objective.

(ii) Effective Rank Analysis[1]. Following Roy and Vetterli (2007), we calculate the effective rank of learned representations and normalize it by the total representation dimension to obtain the normalized effective rank, which is used to assess potential dimensional collapse. As shown in Table R8, the original NeuroOnline maintains a normalized effective rank comparable to the L2-normalized variant, while outperforming off-FT-MLP and NeuroOnline w/o MCL. These results provide additional evidence that the original consistency objective does not induce substantial dimensional collapse.

Together, these results suggest that the performance improvements are not due to representation magnitude shrinkage or dimensional collapse.

Table R7. Performance Comparison between Original and L2-Normalized Consistency Objectives.

Consistency ObjectiveBCIC-2A (Bal.Acc. ↑)BCIC-2B (Bal.Acc. ↑)MentalArithmetic (Bal.Acc. ↑)
L2-Normalized65.32 ± 1.7565.89 ± 1.2463.04 ± 2.18
Original (Ours)66.18 ± 2.1166.74 ± 0.6163.96 ± 2.77

Table R8. Effective Rank Analysis of EEG Representations under Different Consistency Objectives.

MethodBCIC-2A (%) ↑BCIC-2B (%) ↑MentalArithmetic (%) ↑
off-FT-MLP (fullfine-tuning)12.23 ± 0.149.58 ± 0.1110.62 ± 0.17
NeuroOnline w/o MCL12.58 ± 0.229.87 ± 0.1610.86 ± 0.15
NeuroOnline (L2-Normalized)13.22 ± 0.1710.33 ± 0.1211.46 ± 0.13
NeuroOnline (Original)13.62 ± 0.1210.81 ± 0.1011.75 ± 0.11

[1] Roy, Olivier, and Martin Vetterli. "The effective rank: A measure of effective dimensionality." 2007 15th European signal processing conference. IEEE, 2007.

Reviewer_B1qrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses an important problem of adapting EEG foundation models, as they tend to degrade under distribution shifts and misalignment between pretraining objectives and downstream tasks. The authors propose NeuroOnline, which is an online adaptation approach with multi-view consistency learning and context-aware representation modulation.

Critical strengths
  • The paper addresses an important problem in the EEG representation learning domain, as many EEG foundation models do not transfer well to enable clinical deployment.
Critical weaknesses
  • The paper aims to address the limitation of pretraining–downstream task misalignment. However, it is unclear which specific mechanism in the proposed approach addresses this issue. In particular, the multi-view consistency learning component appears similar to existing self-supervised pretraining strategies commonly used in EEG foundation models. Given the similarity between the proposed objective and conventional EEG foundation model pretraining, it is difficult to understand how reusing a similar masking strategy effectively mitigates pretraining–downstream task misalignment.

  • There appears to be a substantial performance gap between the results originally reported for CBraMod (which I understand correspond to the full fine-tuning with the MLP variant) and those presented in this manuscript. For instance, CBraMod reports a balanced accuracy (BA) of 64.17%, whereas this manuscript reports 53.89%. Similar for mental arithmetic task. Could the authors clarify the reasons for these performance differences?

  • Some of the claims made in the manuscript are vague and not grounded in the results. For example, the manuscript mentions "(i) Online updating generally outperforms static offline deployment, highlighting the importance of adapting EEG foundation models to distribution shifts." However, based on Table 2, offline approaches seem to be performing better compared to online.

Questions for authors
  • The paper mentions paradigm (online adaptation of EEG FMs) innovation as one of its major contributions. However, online adaptation of EEG foundation models has been explored and benchmarked in prior related works [1,2]. Could the authors clarify how the proposed paradigm differs from these existing works?

[1] Wang, S., et al. Neurottt: Bridging pretraining-downstream task misalignment in EEG foundation models via test-time training.

[2] Lee, G. J., et al. Test-Time Adaptation for EEG Foundation Models: A Systematic Study under Real-World Distribution Shifts.

  • This work would benefit from a discussion of prior work on test-time adaptation of EEG foundation models (under the related works section), as these studies are directly relevant to the focus of this paper. In my opinion, including these works would help better contextualize the contribution and clarify its novelty relative to existing methods.

  • How was the significant testing conducted? Was it between seeds?

  • The meaning of the *** in Table 2 is unclear. While the manuscript indicates that they denote statistical significance, the asterisks are placed next to the method name rather than individual performance values. Does this indicate that the proposed method achieves statistically significant improvements over the baselines across all evaluated tasks and metrics? For instance, on the mental arithmetic task, NeuroTTT achieves an AUROC of 74.00 ± 5.02, while NeuroOnline achieves 76.38 ± 5.38.

11 comments
AuthorsOpenReview ↗

Response to Reviewer B1qr (1/6)

We sincerely thank the reviewer for their helpful suggestions.

W1: How does MCL mitigate pretraining–downstream task misalignment?

Response:

While NeuroOnline adopts temporal and frequency masking, its key distinction from conventional EEG self-supervised pretraining lies in task-supervised online representation alignment rather than general-purpose representation learning. Specifically, conventional pretraining learns general-purpose representations through self-supervised objectives, whereas NeuroOnline jointly optimizes downstream classification and multi-view consistency using newly observed EEG data. The classification objective guides representations toward downstream task requirements, while MCL enforces consistency across temporal and spectral perturbations. This joint optimization enables MCL to regularize task-specific adaptation rather than simply repeating the pretraining objective.

To further validate this mechanism, we measure LogME (You et al., 2021), which quantifies the compatibility between EEG representations and downstream task labels based on the marginal likelihood of a Bayesian linear model. We compare the pretrained CBraMod, off-FT-MLP (fullfine-tuning), NeuroOnline without MCL, and the complete NeuroOnline framework under identical evaluation settings. As shown in Table R1, NeuroOnline achieves the highest LogME scores, demonstrating improved representation–task compatibility and the additional contribution of MCL beyond task-supervised online adaptation. These results complement the ablations in Table 4 (the original manuscript), where removing MCL reduces Balanced Accuracy by 3.47, 1.24, and 1.74 percentage points on BCIC-2A, BCIC-2B, and MentalArithmetic, respectively.

Table R1. Downstream Task Compatibility of Learned EEG Representations Measured by LogME.

MethodBCIC-2A ↑BCIC-2B ↑MentalArithmetic ↑
Pretrained CBraMod-0.5324 ± 0.0000-0.6823 ± 0.0000-0.3212 ± 0.0000
off-FT-MLP (fullfine-tuning)-0.4958 ± 0.0138-0.6294 ± 0.0153-0.1967 ± 0.0309
NeuroOnline w/o MCL-0.4373 ± 0.0087-0.5612 ± 0.01240.0548 ± 0.0251
NeuroOnline-0.3823 ± 0.0096-0.3735 ± 0.01310.2310 ± 0.0275

[1] You, Kaichao, et al. "Logme: Practical assessment of pre-trained models for transfer learning." International conference on machine learning. Pmlr, 2021.

AuthorsOpenReview ↗

Response to Reviewer B1qr (2/6)

W2: Performance discrepancies with the original CBraMod results.

Response:

We clarify that the reported 53.89% Balanced Accuracy on PhysioNet-MI corresponds to on-FT-MLP (online full fine-tuning), rather than the offline full fine-tuning setting used in the original CBraMod study. The corresponding off-FT-MLP achieves 60.46%, substantially narrowing the discrepancy with the originally reported 64.17%.

The remaining performance difference may primarily arise from two implementation differences: (1) the original CBraMod study does not provide the exact version numbers of its software packages, potentially leading to differences in software environments; and (2) to better reflect real-world online settings, our data preprocessing is performed on individual EEG segments (i.e., epochs or trials), rather than entire continuous EEG recordings, ensuring that preprocessing does not rely on future EEG observations unavailable at inference time. All other experimental settings are consistent with those of the original CBraMod study. Furthermore, all competing methods in our experiments follow identical experimental settings and evaluation protocols to ensure fair comparisons.

AuthorsOpenReview ↗

Response to Reviewer B1qr (3/6)

W3: The claim that online updating generally outperforms offline deployment is not consistently supported.

Response:

We appreciate the reviewer's observation and agree that online adaptation does not necessarily outperform offline fine-tuning, as its effectiveness depends on the specific adaptation strategy. As shown in Table 2 (the original manuscript), conventional online updating can improve performance on several datasets but may also underperform offline methods. For example, on BCIC-2B, off-FT-MLP achieves 65.59% Balanced Accuracy, outperforming on-FT-MLP (64.86%), whereas NeuroOnline achieves 66.74%. These results highlight the importance of effective adaptation mechanisms rather than online updating alone.

We will revise the statement in Section 4.2 as follows: "Online adaptation offers opportunities to improve upon static deployment, but its effectiveness relies on appropriate adaptation mechanisms. NeuroOnline achieves more consistent improvements by integrating multi-view consistency learning and context-aware representation modulation."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.