acceptodds
Under review as a conference paper at ICLR 2027

PRiSE-EEG: A Prior-Guided Foundation Model with Depth-Stratified Experts for Cross-Paradigm EEG Representation Learning

Abstract

EEG foundation models aim to learn reusable representations across heterogeneous paradigms, yet existing approaches often use uniform adaptation mechanisms and are typically reported under separate downstream fine-tuning protocols. In this work, we first analyze dense EEG Transformers from two complementary perspectives. Gradient similarity across six downstream datasets reveals substantial optimization conflicts among EEG paradigms, while CKA analysis on mixed-paradigm batches shows a consistent depth-wise transition: shallow layers preserve stronger cross-paradigm similarity, whereas deeper layers become increasingly specialized. Motivated by these findings, we propose **PRiSE-EEG**, a prior-guided EEG foundation model with CKA-calibrated Depth-Stratified Experts. PRiSE-EEG forms continuous multi-channel EEG patches using weak static cortical and network priors and dynamic short-time channel interactions, then allocates shared and specialized experts across MoE Transformer blocks according to a sigmoid mapping from layer-wise CKA sharedness. This design preserves common EEG regularities in early blocks while assigning more specialized capacity to later task-specific transformations. Experiments on 12 public EEG benchmarks show strong performance. Compact ablations further show that CKA-derived expert allocation improves over dense Transformers, uniform MoE, and manually fixed shared-specific expert ratios. Code is available in an [anonymous repository](https://anonymous.4open.science/r/PRiSE-EEG-A66F).

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG Foundation Models, Brain-Computer Interface, Mixture of Experts

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bjEAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRiSE-EEG, a prior-guided EEG foundation model for cross-paradigm representation learning. Motivated by gradient conflicts across EEG tasks and depth-dependent differences in representation similarity, the authors introduce a continuous multi-channel tokenizer that integrates static spatial priors with dynamic channel interactions. They further propose CKA-calibrated depth-stratified experts to allocate shared and specialized experts across Transformer layers.

Critical strengths
  1. The motivation is good. This paper addresses an important challenge in cross-paradigm EEG representation learning. The analysis of gradient conflicts and layer-wise representation similarity provides a reasonable motivation for the proposed depth-stratified expert design.

  2. The proposed method is evaluated on 12 public EEG benchmarks covering diverse tasks.

Critical weaknesses
  1. Although the CKA-based expert allocation is well motivated by the challenges of cross-paradigm EEG representation learning, as discussed in Strength 1, the proposed solution largely reduces this challenge to determining the number of shared and specialized experts at different network depths. Related ideas have already been explored in existing works, such as NeurIPT (NeurIPS 2025), which investigates progressive expert specialization, and EEGMoE (TNNLS 2026), which introduces shared and domain-specific experts. In particular, NeurIPT is not discussed in the related work, while EEGMoE is acknowledged but not included in the experimental comparisons. A clearer distinction from these methods, together with more direct comparisons, would help establish the novelty of the proposed design.

  2. The ablation results in Figure 3 show that using a fixed number of shared experts ("Fixed 2 Shared Experts") leads to relatively modest performance degradation compared with the proposed CKA-based allocation. Given that depth-stratified expert allocation is a central contribution of this work, these results raise questions about how much the CKA-guided design actually contributes beyond a simpler fixed allocation strategy.

  3. The implementation details of joint fine-tuning need further clarification, particularly how training batches are constructed and how datasets of substantially different sizes are balanced during optimization. More importantly, Table 9 shows that joint fine-tuning consistently outperforms single-task fine-tuning across all 12 heterogeneous EEG datasets, which deserves further investigation. Although positive transfer across tasks is possible, such consistent improvements are not necessarily expected. Given the motivation of this work, the CKA-guided expert design should ideally play an important role in facilitating beneficial cross-paradigm knowledge sharing. However, the relatively modest improvements over fixed expert allocation in Figure 3 make it unclear how much this design contributes to the observed multi-task gains. Although Figure 5 provides some analysis of gradient similarity and expert routing, it does not directly explain these consistent improvements or establish their connection to CKA-guided allocation.

Questions for authors

Please see in Weaknesses

1 comment

I appreciate the authors’ additional experiments and clarifications, which have addressed some of my concerns and strengthened the empirical support for the proposed design. However, my main concern remains.

While I appreciate the motivation of mitigating cross-paradigm conflicts and facilitating knowledge sharing, the methodological advance of this component primarily consists of using offline representation similarity to determine the layer-wise allocation of shared and specialized experts within an existing MoE framework. In my view, this remains a relatively incremental extension of existing expert-allocation designs and this offers limited methodological novelty relative to such broader motivation. And the paper still lacks a sufficiently clear explanation of how this specific CKA-guided allocation mechanism further mitigates cross-paradigm conflicts and facilitates knowledge sharing.

As the CKA-guided expert allocation is presented as a main technical contribution of this work(see in Intro. Contribution. 2), I maintain my original rating. This is my final rating.

Reviewer_3zGoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRiSE-EEG, an EEG foundation model that combines a prior-guided continuous tokenizer with a depth-stratified mixture-of-experts Transformer. The design is motivated by two pilot analyses: gradient similarities reveal conflicts among downstream datasets, while within- and between-paradigm CKA suggest that shallow layers are more shared and deeper layers more specialized. PRiSE-EEG converts the layer-wise CKA ratio into a fixed number of shared versus routed experts, and forms tokens using static sensor-region/network biases together with dynamic short-window channel interactions.

Critical strengths
  1. The paper connects representation analysis to architectural design in an interesting way. Using CKA as a pretraining-time calibration signal for layer-wise expert allocation is more principled than applying an identical MoE configuration at every depth. The combination of shared and specialized capacity is well motivated by cross-paradigm heterogeneity.

  2. The empirical scope is substantial. The pretraining corpus is large and diverse, downstream evaluation spans 12 datasets and multiple task families, and the paper reports ten-run results for the proposed model. The appendices include single-task and multi-task comparisons, scaling experiments, latency measurements, routing analyses, and multiple component ablations.

  3. The model includes several practically useful design choices. Soft rather than hard sensor-group biases, dynamic short-time groups, sparse expert activation, and a compact dual-stream head offer a plausible way to accommodate heterogeneous EEG while controlling activated computation. The availability of an anonymous code repository is also valuable.

Critical weaknesses
  1. The data-splitting description leaves a serious risk of subject leakage. Appendix D states that subject-level splitting is used for most datasets, but Mimul-11, Workload, TUEV/TUSL, and ADFTD are described only as stratified splits over the available data; SEED and SEED-V explicitly split trials while retaining the same subjects across partitions. For datasets containing many windows per participant, sample-level stratification can substantially inflate performance through subject identification. The paper should state the disjoint unit for every dataset and provide strictly subject-disjoint results for claims of cross-subject transfer.

  2. The main comparison is not protocol-matched. Tables 1, 2 compare unified multi-task PRiSE-EEG against baselines adapted separately per task, and even the baseline adaptation modes differ (for example, EEGPT uses a frozen encoder with a linear probe). Appendix Table 10 partially addresses this by jointly training the baselines, but model size, pretraining data, active parameter count, classification heads, and optimization budgets remain unmatched, and PRiSE-EEG is not uniformly best across reported metrics. The uncorrected two-sample t-tests over repeated runs on the same test sets also do not justify the large number of significance markers.

  3. The experiments do not isolate the value of CKA calibration itself. The shared-expert schedule is obtained from one dense pilot and a hand-parameterized sigmoid with fixed threshold and temperature, yet its stability across pilot seeds, batch composition, checkpoints, or corpora is not reported. A fixed-ratio and a reversed-order ablation show that some depth dependence is useful, but they do not rule out a simple monotone heuristic, random schedules with the same layer-wise counts, or schedules selected directly on validation performance. In addition, the dense performance ablation is not matched to the MoE model’s total capacity.

  4. The neuroscientific interpretation of the static priors is insufficiently validated. Mapping scalp electrodes to cortical atlas networks by spatial proximity is not a direct physiological assignment because scalp measurements reflect volume-conducted mixtures of sources. The no-prior comparison shows that an attention bias can help, but there is no downstream performance comparison against random or geometry-matched groupings. Table 12 only verifies that the learned relaxation preserves the supplied initialization, while Integrated Gradients and Grad-CAM provide post hoc associations rather than evidence that the proposed network labels are causally useful.

Questions for authors

See Weaknesses.

1 comment

Thank the author for the responses and additional experiments. I think the new results have addressed most of my concerns, especially regarding the baseline comparisons and CKA-guided expert allocation. I have updated my rating to Weak Accept.

Reviewer_UkzkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses the conflicting task updates across layers and underused weak sensor-space structure in cross-paradigm EEG learning by proposing PRiSE-EEG, a prior-guided EEG foundation model with CKA-calibrated depth-stratified experts.

Critical strengths

The proposed framework was evaluated across various downstream classification and detection tasks. The results demonstrate that the model maintains high efficiency

Critical weaknesses
  1. Lacks Explanation & Inconsistent Terminology: Unaligned keywords are used, such as "multi-task" versus "cross-paradigm." The concept of "multi-task" appears suddenly in the results section without proper introduction or clarification.

  2. Gaps in the Literature: There are noticeable gaps in the citation timeline (in the introduction). For instance, the authors cite sources from 2008, 2010, 2018, and 2025, or jump directly from 2010/2012 to 2025 without acknowledging intermediate research.

Questions for authors

1- Clarify the training procedure for multi-task versus single-task settings.

2- How are the losses balanced across different tasks during multi-task training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.