acceptodds
Under review as a conference paper at ICLR 2027

ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Models

Abstract

EEG foundation models (EFMs) capture reusable knowledge from large-scale EEG data, while many EEG recordings also include companion physiological signals that provide complementary information beyond the EEG-only interface. The challenge is to preserve this pretrained knowledge while extending the EFM to heterogeneous multimodal recordings through an adaptation inferred from unlabeled target data. We introduce ZeroMAG, a zero-shot multimodal adapter generation framework that extends a frozen EEG encoder and prediction head using unlabeled target recordings, without target labels or target-side optimization. The target datasets are held out from all model training and selection in the ZeroMAG pipeline. ZeroMAG organizes companion modalities around a configuration-invariant adapter, constructs a modality–subject–task condition from unlabeled recordings and task context, and generates adapter weights in a function-constrained latent space learned from source adapters. Across six held-out target datasets and three EFM backbones, ZeroMAG improves balanced accuracy by 7.22 percentage points over EEG-only inference and 4.89 points over direct weight regression, while coming within 0.50 points of supervised multimodal adaptation on average. Ablations further show that removing functional supervision from either representation learning or conditional generation degrades generated-adapter performance, confirming the contribution of both components.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG foundation models, multimodal physiological signals, zero-shot adaptation, adapter generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tSR3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a zero-shot conditional generative multimodal adapter, ZeroMAG, that uses auxiliary physiological information and unlabeled target-domain signals to adapt the frozen EEG foundation model.

Critical strengths

The main strength of ZeroMAG is that it enhances the model without accessing labeled target-domain data or target-side optimization, and adds multimodal information to frozen EEG foundation models. Also, the evaluation pipeline includes a lot of backbones, datasets, and tasks.

Critical weaknesses
  1. The proposed method is too complex to understand, and the ablation studies can not sufficiently prove the necessity of the proposed design.

  2. Baselines are too weak to prove the strength.

  3. The code is different from the paper’s main text and prevents reproducing the results.

  4. The main results and ablation results have a large difference of 3.40%. This mistake impacts the reliability of the proposed method. The average result is 70.01% in Table 1 and Table C.1, but 73.41% in the ablation analysis.

Questions for authors
  1. ZeroMAG combines the time-frequency branch, conditional encoding, transformer, VAE, diffusion, and MoE. It uses multiple loss terms in stage I and stage II training but does not sufficiently introduce the necessity of this design.

  2. The comparison of baselines is insufficient. The compared baselines now include EEG-only, mean and nearest source weights, and the direct MLP. But the important conditional hypernetworks, adapters, generators, and multimodal alignment approaches on the same source data have not been compared (see Section 4.1 and Table 1).

  3. The proposed code is different from their claim in the main paper, including the architecture and training settings, such as the depth of dit and the number of experts. The training source code in stage I not activate the claimed functional supervision loss introduced in 3.4.1. Also, stage II includes extra balance and diversity losses.

  4. The main results and ablation results have a large difference of 3.40%. This mistake impacts the reliability of the proposed method. The average result of nine CBraMod/ZeroMAG is 70.01% in Table 1 and Table C.1, but 73.41% in the ablation analysis of Table 2 and G.1. The results of three functional loss ablation studies (70.54%, 71.83%, 72.26%) are higher than the full model results (70.01%). Since the original seed and dataset results are not provided, it is difficult to explain these differences.

Reviewer_YS8jOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

In this work, authors proposed a significant question regarding the EEG foundation model: can multimodal capabilities be added to an existing EFM without target-side training? Considering that EEG contains companion signals to capture complementary physiological information, a common approach to exploiting these signals is to train a new multimodal backbone which is expensive. Thus, authors introduce ZeroMAG, a zero-shot multimodal adapter generation framework that extends a frozen EEG encoder and prediction head using unlabeled target recordings.

Critical strengths

A key strength of this work is its focus on a practically important challenge: incorporating new auxiliary signals without modifying the existing EEG encoder. To address this, the authors introduce ZeroMAG, a framework designed to generate personalized adapters from data without target-specific labels, enabling a base EEG model to be adapted to the target task. Experiments with three backbone architectures show that removing the personalized adapters leads to lower reconstruction error but higher prediction error on the target task. These findings support the value of personalized adapters for downstream performance.

Critical weaknesses

The main weakness of this paper is that “unseen modalities” and “unseen tasks” are not clearly defined, making it hard to judge how well the method generalizes. ZeroMAG relies on pretrained prediction heads, pretrained embeddings for different tasks and modalities, and a parameter space learned from source adapters. However, the paper does not explain whether the held-out tasks and modalities appeared in other training stages. It is also unclear how the modules for these held-out tasks and modalities were trained. Do the results therefore show that the generator has not seen the corresponding adapters, or that the entire system has not seen the task or modality? The authors should make this distinction clear.

Questions for authors
  1. Regarding the FoG task which is a task-holdout target with no source adapter, I wonder what data is used to train its prediction head and how its task embedding is obtained. Does “task-holdout” mean that FoG is excluded only from source adapter and generator training, or from all task-specific training?
  2. I wonder is it necessary to generate a separate set of adapters for each target, or could some data be combined to train a shared adapter?
  3. For the same subject, does the performance of the generated adapters remain stable when using recordings from different time periods?
  4. Could the authors test how ZeroMAG performs when the auxiliary signals are noisy or missing?
Reviewer_5ppCOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes ZeroMAG, a framework designed to equip frozen EEG foundation models (EFMs) with multimodal capabilities (incorporating companion signals such as EOG, EMG, ECG, eye movement, ACC, and GSR) without requiring target labels or target-side optimization. The core idea is to generate plug-in adapter weights from an unlabeled calibration recording using a two-stage latent generation pipeline. In the first stage, reference adapters fitted on source datasets are encoded into a structured latent space using a VAE and an MoE decoder regularized by functional KL divergence. In the second stage, a Diffusion Transformer (DiT) generates adapter latents conditioned on modality, subject, and task representations extracted from 64 unlabeled windows. The generated adapter is plugged into the frozen EEG backbone and its frozen prediction head. The approach is evaluated across 6 primary target datasets (plus 3 in the appendix) on three frozen EEG backbones (CBraMod, CSBrain, CodeBrain).

Critical strengths
  • The motivation to extend existing, frozen EEG foundation models to heterogeneous companion physiological signals in a plug-and-play manner without target-side fine-tuning is sensible and practically relevant.
  • The modular 1+N1+N adapter architecture—anchored by the frozen EEG representation and utilizing parallel temporal-spectral encoders for available companion signals—provides a neat and structured way to accommodate arbitrary modality configurations.
  • The evaluation spans multiple foundation model backbones and diverse physiological tasks, showing consistent improvements over simple weight averaging and direct MLP generation.
Critical weaknesses
  • The prediction head setup and label space mapping appear ill-defined and contradictory. First, according to Appendix A.2, the freezing-of-gait (FoG) head is trained on the Daphnet dataset, which contains only accelerometer recordings and no EEG. It is unclear how a classification head taking fused EEG and auxiliary features can be trained on a dataset lacking EEG. Furthermore, FoG is highlighted as an unseen task-holdout, yet a task-specific head already exists, and using an "unspecified-task" condition still yields performance within ~0.55 points of supervised tuning—the same gap seen on in-task targets. Second, the paper assumes a frozen, shared task output layer across datasets, but the label spaces between source and target datasets do not match. For example, the source emotion datasets (DEAP, ASCERTAIN) use continuous valence-arousal dimensions, whereas target SEED-V uses 5 discrete emotion classes; OpenBMI is 2-class while BCI-IV-2a is 4-class; and WAUC vs. EEGMAT have different mental workload/arithmetic task formulations. The paper does not specify how logits are mapped across these disparate label spaces through a frozen head. Third, It is also unclear what the "supervised tuning" baseline actually updates (the adapter only or the entire backbone) and whether this adaptation is performed per-subject or per-dataset.

  • Training a complex pipeline consisting of a VAE, a 4-expert MoE decoder, and a Diffusion Transformer on only 311 source reference adapters seems overkilled and is not adequately justified against more natural baselines. In my opinion, the comparison lacks: (i) a universal, pooled-source multimodal adapter trained with modality dropout; (ii) a standard conditional hypernetwork trained end-to-end directly on the source tasks with the same functional loss; (iii) test-time adaptation (TTA) baselines such as Tent, SHOT, or T3A, which are especially relevant given that the proposed method already requires a per-subject calibration phase of 64 unlabeled windows; (iv) simple kNN retrieval or ensembling in function space. The mean-source baseline is an unconvincing comparison due to weight-permutation symmetries in neural networks.

  • The ablation results indicate that removing or shuffling the subject condition causes performance to drop by 9–11 points, falling back to EEG-only performance. This suggests that nearly all multimodal gains stem from the subject factor. However, the paper does not analyze what this factor actually encodes—whether it captures true individual physiology, hardware/montage artifacts, electrode impedance, or class priors. Missing controls such as auxiliary-only or EEG-shuffled inputs make it hard to verify whether companion signals (e.g., ACC in FoG) are simply acting as shortcut predictors rather than providing genuine multimodal synergy.

  • The experimental results between the main text and the ablation studies do not line up. In Table 1 and Table C.1, the 9-target average B-ACC for CBraMod with ZeroMAG is 70.01%, but in Table 2 (ablation), Full ZeroMAG is reported as 73.41% under the "same protocol"—a 3.4-point shift. In contrast, Direct MLP is virtually identical (62.78% vs. 62.34%), which artificially widens the reported ZeroMAG-vs-MLP advantage from ~7.2 to 11.07 points. Given that this run-to-run discrepancy (3.4 points) is larger than the entire measured effect of removing functional supervision (1.15 to 2.87 points), I am afraid one cannot draw confident conclusions regarding the contribution of the functional loss.

  • The headline claim of trailing supervised adaptation by only 0.50 points appears cherry-picked. This gap holds only for the 6 target datasets selected for Table 1. On the three datasets relegated to Appendix C.1 (CAP, DREAMER, BCI-IV), the supervised gap is substantially wider (0.95 to 2.14 points), making the true 9-target gap approximately 0.91 points. All 9 evaluation targets should be reported together in the main text with paired statistical tests across seeds and subjects.

  • There are several statistical anomalies and plausibility issues in the reported numbers: (i) across all 18 backbone-dataset evaluations in Table 1, the gap between ZeroMAG and supervised tuning falls within an improbably narrow range of 0.38 to 0.61 points, despite seed standard deviations being between 0.4% and 1.6%; (ii) on EEGMAT, the seed standard deviation drops from 2.5–2.8% for the EEG-only baseline down to 0.6–0.9% for adapter-based methods on the exact same splits, with no explanation for this variance collapse; (iii) for SEED-V, the EEG-only zero-shot score (40.62%) is suspiciously close to fully supervised CBraMod fine-tuning (~40.9%). Moreover, SEED-V provides precomputed differential entropy features rather than raw continuous eye-tracking signals, making the application of the temporal-spectral FFT branch technically questionable.

  • The cross-paradigm comparisons with multimodal foundation models are not entirely fair or transparent. Brant-X actually outperforms ZeroMAG across all 6 target datasets in Table 1, yet it is excluded from Figure 1(c). In addition, SleepFM is pre-trained exclusively on sleep data; the paper does not explain how SleepFM was evaluated on emotion recognition, cognitive load, and FoG, nor whether target labels were exposed during its evaluation.

  • The "zero-shot" framing is overstated. While the method avoids target-side backpropagation, it requires 64 calibration windows (which amounts to 32 minutes of continuous data for 30-second sleep epochs) and 50 iterative DDIM sampling steps at deployment, contradicting the claim in Section 2.3 of a "single forward pass."

  • The appendix (~25 pages) contains unnecessary textbook filler (e.g., standard formulas for balanced accuracy, Cohen's kappa, scalar derivative derivations, and basic arithmetic like 15/2=7.515/2=7.5) that obscures the core technical details and should be streamlined.

Questions for authors
  • Could you clarify how the frozen prediction head for FoG was trained using the Daphnet dataset, given that Daphnet contains only accelerometer signals and no EEG?

  • How are mismatches in label spaces handled between source and target datasets (e.g., 2-class OpenBMI vs. 4-class BCI-IV-2a, continuous valence/arousal vs. 5-class SEED-V) when passing adapted representations through a frozen prediction head?

  • Could you comment on the 3.4-point performance discrepancy for CBraMod between Table 1/C.1 (70.01%) and Table 2 (73.41%), and how this run-to-run variation impacts the claims regarding the functional supervision loss?

  • What specific physiological or recording properties does the subject condition vector capture? Have you run control experiments (such as companion-signal-only or EEG-shuffled inputs) to ensure auxiliary signals are not acting as task shortcuts?

  • Why was Brant-X omitted from Figure 1(c) despite outperforming ZeroMAG across the evaluated benchmarks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.