acceptodds
Under review as a conference paper at ICLR 2027

HiERA: Hierarchical Domain-aware Reusable Adaptation for Continual Personalization of EEG Foundation Models

Abstract

EEG foundation models (EFMs) generalize well across EEG analysis and brain-computer interface (BCI) tasks, yet continually personalizing EFMs to a growing stream of subjects remains costly: full fine-tuning is computationally expensive, while per-subject LoRA adapters cause storage to grow rapidly with the number of subjects. We observe that shallow EFM layers encode more domain-level variation shared across subjects, whereas deeper layers increasingly capture subject-specific characteristics. Building on this observation, we propose **HiERA**, a hierarchical adaptation framework that employs a domain-shared Tensor Ring transformation in shallow layers and a compact, reusable adapter repository in deeper layers. New subjects can reuse existing adapters when appropriate, avoiding unnecessary parameter growth. Experiments on multiple public EEG datasets and EFMs show that HiERA maintains competitive personalization performance while substantially reducing trainable and stored parameters compared with other PEFT baselines. Code will be released upon acceptance.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Foundation model, EEG, Adaptation, PEFT, Tensor decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jKzyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies continual personalization of EEG foundation models (EFMs): subjects from several datasets arrive one at a time, and the model must adapt to each while keeping the added parameters small. Two analyses motivate the design. Layer-wise linear probing suggests shallow layers carry domain information while deeper layers carry subject information. Effective-rank measurements of full fine-tuning updates suggest that a domain-shared update needs high rank. The paper claims comparable or better accuracy than per-subject LoRA/OFT with far fewer parameters, plus ablations on the split point, adapter types, and ranks.

Critical strengths
  1. The problem is practical and timely. Serving a growing stream of users from heterogeneous recording setups with a single EFM is a realistic deployment scenario.
  2. The experiments are broad. The study covers four backbones with different architectures (bidirectional and causal, discrete-token and continuous) and three task types.
Critical weaknesses
  1. The central efficiency and performance claims are not supported by the paper's own evidence. For example, unaccounted replay storage overturns the efficiency claim. The abstract claims HiERA "substantially reduc[es] trainable and stored parameters." However, HiERA stores raw EEG samples in a per-domain replay buffer, and this storage is never counted. LoRA and OFT, which store nothing beyond their adapters, are charged for every parameter. Raw patient EEG is also more privacy-sensitive than adapter weights, which matters in the hospital scenario the introduction uses as motivation. Forgetting is never measured, although the method can cause it. Sec. 2.1 says previous subjects "can be directly served when their data are revisited," but all reported results test each subject immediately after adapting to it.
  2. The motivation analyses are confounded, and the method contradicts them. For example, the domain-probing result is trivial. In Fig. 2(a)(b), the domain-ID probe sits at 100% at every layer, not just shallow ones. So the figure shows domain information is present throughout the network, not that shallow layers are specifically domain-level. The paper's own ablations contradict its motivation. Q2/A2 argue that the domain adapter needs growing, high-rank capacity, and that fixed low-rank LoRA is "capacity-bounded." Table 4(a) shows TR rank 1 is best for the domain adapter, and larger ranks do not help. The authors themselves conclude "a low-rank transform is sufficient to capture domain-level variation" (Sec. 4.4), which directly contradicts A2. In Table 3, LoRA r=1 (16 KB, far more free parameters than the 4.56 KB TR transform) performs worse than no domain adapter at all (SEED 47.66 vs. 49.12). A model with strictly more capacity, initialized at identity and selected on validation data, underperforming no adaptation points to an optimization or protocol artifact, not a rank limitation.
  3. The novelty is incremental. The domain component (TW0T W_0 with a dense TR-parameterized TT, motivated by OFT's sparsity) and the TR-compressed LoRA appear closely related to TLoRA (Tao et al., 2025), which the paper cites. The remaining contributions (layer split, threshold-gated adapter reuse, replay) are standard continual-learning ingredients.
Questions for authors
  1. It’s suggested that authors report total storage for each method in Table 2, including the replay buffer, the domain heads, and any per-subject projection layers. Does HiERA retain its advantage once these are counted? What happens without replay?
  2. Please report accuracy on all previously seen subjects using the final model after the full stream, and compare it to immediate post-adaptation accuracy, for HiERA and the baselines.
  3. How was the 56.97% REVE mean computed, given that Table 2 implies 56.63% (below full FT's 56.87%)? Please correct the 16+9 count and the "consistently improves" statement.
  4. How do you reconcile A2 (domain adapters need growing, high-rank capacity) with Table 4(a) (rank 1 is best) and with LoRA r=1 underperforming no adapter in Table 3?
  5. Can you provide within-domain subject probes and an eRank control that holds the number of optimization steps fixed?
Reviewer_YzUtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes HiERA an adaptation framework as an alternative adapter solution to LoRA. It is based on the finding that EEG FMs exhibit hierarchical structure over blocks and according to the authors LoRA should not be applied.

Critical strengths

The framework is based on the finding that EEG foundation models exhibit a hierarchical structure over blocks. The authors demonstrate that subject-id prediction improves over layers while dataset domain can be predicted even from the first layers. I liked this analysis for the HiERA hypothesis. The authors use TRD and have a very interesting thresholding system for new adapters. The ablation studies are very interesting particularly the tradeoff between performance and parameters.

Critical weaknesses

More recent EEG foundation models are missing from the analysis

Table 2 does not demonstrate very good results – minimal gains compared to LoRA or OFT

Several ablation studies that will help understand the necessity of this framework are missing (see questions below).

Questions for authors

73% of subjects end up with a common adapter. Is there any ablation that show the performance if 73 new adapter had been created?

Does the order that the subject get into the framework matter? Any ablation analysis on that?

What is the purpose of Figure 7? There are no clear dataset boundaries especially since isruc and seed are everywhere on the plot. Maybe I have misunderstood something.

Were any comparison datasets part of the pre-training corpus of these EEG FMs? If yes, how does that affect the probing and performance results?

Any ablation study on the chosen threshold?

Could the authors include more recent EEG FMs?

Reviewer_a87tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on the adaptation of EEG foundation models. The authors observe that shallow layers in EEG FMs learn general domain-level features and deep layers learn subject-specific features, which is an interesting finding. Based on that, they propose HiERA, which is a hierarchical adaptation framework that employs a domain-aware Tensor Ring transformation in shallow layers and a compact, reusable adapter repository in deeper layers.

Critical strengths
  • One of the key strengths of the paper is its finding that shallow layers of EEG foundation models tend to capture domain-level features, while deeper layers learn subject-specific representations. This is an interesting and valuable insight.

  • Another strength is that the proposed adaptation strategies are well motivated by empirical analyses, such as layer-wise probing, which provide a clear rationale for the design choices based on the representations learned at different layers of EEG foundation models.

Critical weaknesses
  • A weakness of the proposed approach is the lack of clarity regarding its practical deployment on unseen subjects, where labeled data may not be available. In particular, the second stage of the subject adapter appears to rely on a validation set, raising concerns about the utility of the method in real-world, label-free settings.

  • The performance gains achieved by the proposed approach appear limited considering the additional computational and storage overhead associated with subject-specific adaptation. In particular, direct fine-tuning of EEG foundation models on downstream tasks can achieve substantially higher performance without requiring subject-specific adapters. For example, on ISRUC, REVE achieves 78.19 as reported in the original paper, compared to 72.82 achieved by HiERA. This raises concerns about the performance–efficiency trade-off of the proposed method, as conventional full fine-tuning appears to achieve better performance with a simpler deployment strategy.

Questions for authors
  • In Table 2, why is there a big difference between the full finetuning and the reported results in the original REVE paper? E.g.: For REVE in ISRUC: 71.73 in this paper vs. 78.19 in the original REVE. Similarly for LaBraM.

  • It is unclear how the proposed method would be deployed in a real-world setting where labeled data from new subjects are unavailable. In particular, the second stage of the subject adapter appears to rely on a validation set. How would this stage operate when adapting to a new subject without access to labeled validation data?

  • Could the authors provide a subject-wise analysis comparing performance before and after adaptation to better understand the improvements achieved by the proposed method across individual subjects? Additionally, it would be valuable to investigate how the number of adapters in the repository affects performance, particularly the minimum number of adapters required to observe meaningful improvements.

  • What is the computational requirement of the proposed approach such as memory and computational time?

1 comment

Since there are no response from the authors, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.