acceptodds
Under review as a conference paper at ICLR 2027

BrainCodes: A Lightweight EEG Foundation Model over Discrete Neural Codes

Abstract

Recent electroencephalography (EEG) foundation models increasingly emphasize model scale, potentially limiting their deployment on resource-constrained devices and in everyday environments. We argue that transferable EEG features can instead be learned with a compact architecture and introduce BrainCodes, a lightweight foundation model pretrained through self-supervised learning on more than 800 hours of recordings from over 1,000 participants. Our architecture combines a quantization-based neural tokenizer with a spatiotemporal encoder that supports arbitrary electrode configurations. The pretraining objective explicitly encourages representations that capture spatial connectivity through channel positional encoding, and sensitivity to narrowband neural oscillations through an auxiliary training objective. Our model matches or surpasses the state of the art across diverse neurophysiological monitoring tasks, including sleep staging, attention assessment, and cognitive load estimation. BrainCodes transfers effectively to low-resource wearable devices while using up to 97% fewer trainable parameters than comparable architectures. Beyond predictive accuracy, our evaluation framework characterizes the geometry and invariances of learned EEG representations across measurable spatial, temporal, and spectral signal properties.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG foundation models, neural tokenization, vector quantization, self-supervised learning, wearable EEG

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9QSdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BrainCodes, a lightweight EEG foundation model that combines discrete neural tokenization, masked-token prediction, spatial encoding, and a synthetic oscillation-based auxiliary objective. The authors evaluate it across 14 downstream benchmarks, including wearable EEG and reduced-electrode settings, reporting competitive performance with a compact trainable backbone.

Critical strengths

The evaluation on consumer-grade EEG hardware, including in-ear EEG and Muse headbands, is a valuable aspect of this work, as such settings remain relatively underexplored in EEG foundation model research.

Critical weaknesses
  1. Methodological novelty. Discrete EEG tokenization, masked prediction, and spatial encoding have already been explored in prior work, including LaBraM, CBraMod, NeuroRVQ, BrainOmni, etc. The synthetic oscillation-based objective appears to be the most distinctive component, but its advantage over simpler frequency-aware alternatives remains unclear.

  2. Motivation for discrete code prediction. The paper argues that discrete code prediction alleviates the spectral bias of waveform reconstruction. However, the prediction targets come from a frozen BioCodec tokenizer that is itself reconstruction-trained and may inherit similar biases. Moreover, although cross-entropy avoids direct amplitude weighting, the losses across RVQ stages are equally weighted despite potentially different prediction difficulties and information content. Does this necessarily encourage the preservation of subtle, low-energy oscillatory features, or could the model still prioritize more easily predictable components? It remains unclear to what extent discrete code prediction addresses the underlying limitation of reconstruction-based learning, rather than simply changing the prediction space.

  3. Effectiveness of synthetic oscillation supervision. Predicting the properties of artificially injected oscillations does not necessarily guarantee better representations of naturally occurring EEG rhythms. Since the auxiliary task only requires predicting the properties of artificially injected oscillations, could the model learn features specialized for detecting these synthetic perturbations rather than capturing naturally occurring neural dynamics? It remains unclear whether the auxiliary task specifically promotes meaningful oscillatory representations or primarily facilitates synthetic perturbation detection. The natural-rhythm analysis in Appendix G is encouraging, but does not directly resolve this distinction.

  4. Attribution of the lightweight performance advantage. BrainCodes achieves impressive performance with substantially fewer parameters than several competing EEG foundation models. However, it remains unclear whether this advantage arises from the proposed method or from differences in model capacity, pretraining data, and training protocols. Parameter-matched comparisons under controlled pretraining conditions would help determine whether competing architectures could achieve similar performance at the same scale. Furthermore, investigating how BrainCodes scales with model size and pretraining data would help clarify whether its lightweight design genuinely enables strong performance with fewer parameters, or instead represents a pragmatic compromise in a setting where increasing model capacity offers limited additional benefits.

  5. Transfer claims. The wearable-device and sparse-montage results are encouraging. However, the sparse-montage experiments involve configuration-specific fine-tuning, making it difficult to disentangle the generalization of pretrained representations from target-specific adaptation. Evaluating transfer to unseen electrode configurations without additional backbone fine-tuning would provide stronger evidence for montage generalization.

Questions for authors

See weakness above.

5 comments
AuthorsOpenReview ↗

Response from the Authors (1/4)

We thank the reviewer for the careful assessment. Each methodological question led to additional experimentation on our part, which is now included in the revised manuscript. In addition, the reviewer may also refer to the Overview response attached as an Official Comment to all.

R3-W1 · Methodological novelty and simpler frequency-aware objectives

Concern. “Discrete EEG tokenization, masked prediction, and spatial encoding have already been explored in prior work … The synthetic oscillation-based objective appears to be the most distinctive component, but its advantage over simpler frequency-aware alternatives remains unclear.”

Response. We agree that the individual ingredients have precedents. G1 explains how BrainCodes differs in their use, particularly its independently released codec and localized oscillation supervision. Unlike objectives on spectral content, the injection loss supervises event frequency, duration and location. BrainCodes without injection is already frequency-aware through BioCodec’s spectrally trained targets (R3-W2). Removing injection causes the largest ablation drop, from 0.498 to 0.383 mean nBAC (Table 6).

Complementary comparisons. BrainCodes exceeds NeuroRVQ under its released downstream protocol (0.597 versus 0.559 mean nBAC; G7). With identical windows and linear probes, its frozen features also outperform five-band power plus the 1/f exponent on 8 of 10 datasets and on average (Table R3-W1).

Table R3-W1. [New experiment] BAC using the same logistic-regression probe and identical windows. Spectral features comprise per-channel power in five bands and the 1/f exponent, estimated from a specparam-style fit of the Welch spectrum.

Dataset (chance level)Spectral parametersBrainCodes
Eyes open/closed (0.50)0.8240.809
EEGMAT (0.50)0.6840.764
EESM23 (0.17)0.3830.469
TUEV (0.17)0.4200.459
ISRUC (0.20)0.6940.746
PhysioNet-MI (0.25)0.3610.519
BCI IV-2a (0.25)0.4390.502
High-Gamma (0.25)0.5480.637
OpenBMI-ERP (0.50)0.5330.598
Pavlov (0.50)0.5970.593
Mean nBAC0.3260.414
AuthorsOpenReview ↗

Response from the Authors (2/4)

R3-W2 · Motivation for discrete code prediction

Concern. “the prediction targets come from a frozen BioCodec tokenizer that is itself reconstruction-trained and may inherit similar biases. Moreover, … the losses across RVQ stages are equally weighted … could the model still prioritize more easily predictable components?”

Response. We tested both concerns, i.e., whether the codec’s training loss inherits the frequency-power bias of waveform reconstruction, and whether BrainCodes preferentially learns the easier RVQ stages. Three findings support our design:

  1. BioCodec’s loss reduces the frequency-power bias. Squared-error reconstruction emphasizes high-power components, which in EEG typically favors slow activity (He, 2014; Donoghue et al., 2020). BioCodec instead uses log-magnitude spectral losses. We measured frequency weighting by increasing the amplitude of each 2-Hz band from 1–45 Hz by 10% and recording the change in loss. The resulting log-log slope is nearly flat for BioCodec’s loss (−0.09), compared with −1.43 for squared error (Table R3-W2a). Thus, although the tokenizer is reconstruction-trained, its loss does not have the same frequency-power weighting as waveform squared error.
  2. Low-energy RVQ stages carry substantial spectral information. Later stages add only 17% of waveform variance, yet increase the linear decodability of beta power from 0.31 to 0.46 R², delta power from 0.14 to 0.26, and the 1/f exponent from 0.58 to 0.75 (Table R3-W2b). Equal stage weighting assigns these stages half of the code-prediction objective, despite their comparatively small contribution to waveform variance. This prevents their contribution from being reduced simply because they encode lower-energy residuals.
  3. BrainCodes learns late stages as well as early ones. On held-out HBN participants, masked-code prediction accuracy is similar across all six RVQ stages and context reduces uncertainty by a similar amount at each stage (Table R3-W2c). Code usage is nearly uniform within each stage, so frequent-code guessing cannot explain these accuracies. These measurements do not support the concern that training concentrates only on easily predictable early stages. Table R3-W2a. [New experiment] Log-log slope of the weight each loss gives to each frequency, measured by raising the amplitude of each 2-Hz band from 1 to 45 Hz by 10%; a slope near 0 means all frequencies are weighted equally.
Loss weight per Hz, slopePhysioNet-MIEEGMAT
EEG spectrum (1/f)−1.43−1.88
Mean squared error−1.43−1.88
BioCodec loss−0.09−0.22

Table R3-W2b. [New experiment] Information (decodability) carried by the first k BioCodec RVQ stages: R² of the waveform reconstruction (first row), and R² of predicting each spectral property (1/f exponent and band power) from the codes (other rows).

First k RVQ stages1236
Waveform R²0.4340.6370.7010.844
1/f exponent0.3730.4650.5810.751
Delta0.1130.1490.1360.258
Theta0.3820.4550.4820.594
Alpha0.7950.8310.8630.892
Beta0.0830.1610.3100.457

Table R3-W2c. [New experiment] Masked-code prediction per RVQ stage on held-out HBN participants, evaluated on our official model. Chance top-1 accuracy is 1/256=0.0039.

RVQ stage123456
Top-1 accuracy0.2320.2380.2430.2300.2380.234
CE below code frequencies (nats)2.112.122.122.092.112.10
AuthorsOpenReview ↗

Response from the Authors (3/4)

R3-W3 · Does BrainCodes capture natural rhythms or only synthetic bursts?

Concern. “could the model learn features specialized for detecting these synthetic perturbations rather than capturing naturally occurring neural dynamics? … The natural-rhythm analysis in Appendix G is encouraging, but does not directly resolve this distinction.”

Response. Yes. BrainCodes represents natural rhythm power and utilizes it in downstream tasks. We establish this in two steps: testing the full model on natural recordings, and repeating the analysis in a model pretrained without injection.

  1. The full model uses natural rhythm information downstream. Appendix G shows that frozen BrainCodes features encode naturally occurring rhythm power. To test whether this information matters for prediction, we erased linearly decodable rhythm power using LEACE and refitted the downstream probe. Balanced accuracy decreased by 0.042–0.212 across eight datasets, more than any matched random erasure (G2, revised Figure 2e). Thus, downstream performance depends on spectral information present in natural EEG, beyond synthetic bursts.
  2. Natural rhythms can emerge without synthetic supervision. We retrained BrainCodes without the injection objective, holding the data, initialization, data order and number of training steps fixed. This model never encountered a synthetic burst during pretraining, yet its frozen features still encode natural rhythm power. Erasing that information again produces substantially larger downstream accuracy losses than random erasure (Table R3-W3). These findings argue against an explanation based on synthetic-perturbation detection. Natural rhythms are represented and used downstream, even without injection. The controlled comparison separately shows that the injection objective increases sensitivity to narrowband rhythms (R2-W3).

Table R3-W3. [New experiment] BrainCodes pretrained without the injection objective. The drops are in the balanced accuracy of a linear probe, after erasing rhythm power (LEACE, as in G2) or after 20 random erasures of the same size.

Dataset (chance BAC)−Rhythm−1/fRandom erasure, largest
EEGMAT (0.50)−0.059−0.024−0.007
EESM23 (0.17)−0.185−0.126−0.011
PhysioNet-MI (0.25)−0.060−0.013−0.005

R3-W4 · Attribution of the lightweight advantage, and scaling

Concern. “it remains unclear whether this advantage arises from the proposed method or from differences in model capacity, pretraining data, and training protocols. Parameter-matched comparisons under controlled pretraining conditions would help … investigating how BrainCodes scales with model size and pretraining data would help clarify …”

Response. Our ablations support a contribution from the method and establish competitive accuracy at a small scale. The same architecture scores 0.270 mean nBAC without pretraining versus 0.498 with pretraining (Table 7); removing injection lowers it to 0.383 (Table 6). BrainCodes also leads under the shared recipe of Table 4 and the NeuroRVQ benchmark’s released protocol (G7). It outperforms the compared 3.2M–5.8M foundation models in Table 2 and EEGPT-Tiny on shared tasks, despite several baselines using orders of magnitude more pretraining data. On scale, scaling LaBraM from 5.8M to 369M parameters improved TUEV by only 0.02 BAC, within one participant-bootstrap s.d. (Appendix H), and downstream gains depend more on participant coverage than on windows per participant (Appendix D).

Reviewer_qDiPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrainCodes is a small EEG foundation model with a 1.9M-parameter backbone. Each channel is tokenized by a frozen BioCodec residual quantizer, and a factorized transformer U-Net with 3D rotary electrode encoding predicts masked codes. An auxiliary task asks the bottleneck to recover the frequency, duration and location of injected narrowband bursts. It is pretrained on about 820 hours of HBN data and tested on 14 tasks, including consumer headsets, plus spatial and spectral probes.

Critical strengths
  1. Getting this level of performance from a 1.9M-parameter backbone pretrained on about 820 hours is notable. On the seven standard benchmarks it reaches a mean chance-normalized accuracy of 0.609 against 0.581 for REVE-Base at 69M, and it leads on TUEV, ISRUC and PhysioNet-MI. Pretraining took about 190 GPU-hours, which puts this kind of model within reach of smaller labs.

  2. The paper goes beyond benchmark tables. The evaluation on the Muse headband and in-ear EEG is practically relevant, and BrainCodes clearly wins on EESM23 with 0.525 against 0.418 for REVE. The ablation also supports the core idea, since removing the oscillation-injection objective drops mean chance-normalized accuracy from 0.498 to 0.383, the largest effect of any component.

Critical weaknesses
  1. The headline ranking rests on balanced accuracy, and threshold-free metrics tell a different story. On TUAB, AUROC is 0.883 against 0.925 for REVE and 0.923 for CBraMod. On Siena, where the largest gain is claimed, AUROC is 0.880, below five baselines, and the test set holds only two patients. BrainCodes uses its own class weights and picks checkpoints by validation balanced accuracy while baselines are quoted, so part of the gain may come from thresholding.

  2. The efficiency claim counts only trainable parameters. BrainCodes needs the frozen BioCodec tokenizer at inference, which adds 3.15M parameters, so the deployed model has about 5.0M, more than CBraMod at 4.0M and BIOT at 3.2M. LaBraM's tokenizer, by contrast, is only used during pretraining. For a paper aimed at wearables, I would expect FLOPs, latency and memory on a realistic device rather than trainable parameter counts.

  3. The grounding analysis is weaker than presented. The rhythm probe injects Hann-windowed narrowband bursts into contiguous electrode regions, which is almost the same as the auxiliary pretraining task, so a higher score there is expected. The preference index also rewards losing broadband information, since BrainCodes ranks last on white versus pink noise. On real EEG in Appendix G, REVE and LaBraM decode oscillatory power better on average.

Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response from the Authors (1/2)

We thank the reviewer for the careful reading of our work and thorough comments. In addition, the reviewer may also refer to the Overview response attached as an Official Comment to all.

R2-W1 · Does the performance advantage depend on thresholding or checkpoint selection?

Concern. “The headline ranking rests on balanced accuracy, and threshold-free metrics tell a different story. On TUAB, AUROC is 0.883 against 0.925 for REVE … On Siena, where the largest gain is claimed, AUROC is 0.880, below five baselines, and the test set holds only two patients. … part of the gain may come from thresholding.”

Response. BrainCodes has the highest mean AUROC (0.883) and AUPRC (0.731) across binary tasks, and the highest mean Cohen’s κ (0.633) and weighted F1 (0.739) across multiclass tasks (Table R2-W1). AUROC and AUPRC are threshold-free; κ and F1 provide complementary classification measures.

TUAB and Siena. BrainCodes trails REVE-Base and CBraMod on TUAB, as reported in the manuscript. On Siena, all compared models use the same subject-disjoint protocol and two test patients. Seizures constitute only 2.3% of test windows, which renders AUPRC the informative metric for the positive-class performance under severe imbalance (Saito and Rehmsmeier, 2015). BrainCodes achieves the highest reported AUPRC despite not being pretrained on Siena, whereas REVE included Siena in pretraining.

Checkpoint selection. Baselines retain their original selection rules: CBraMod, for example, selects by validation AUROC, whereas BrainCodes selects by validation balanced accuracy (BAC). The aggregate AUROC/AUPRC results show that the advantage is not confined to BAC.

Table R2-W1. [New analysis] Mean nBAC and threshold-free metrics for the models of Table 2, averaged from Tables 8–14 of the manuscript: AUROC and AUPRC over the binary tasks (TUAB, Siena, EEGMAT), Cohen's κ and weighted F1 over the multi-class tasks (TUEV, ISRUC, PhysioNet-MI, BCI IV-2a).

ModelMean nBACAUROCAUPRCCohen's κWeighted F1
EEGNet0.4280.8140.5940.4440.602
………………
BIOT0.4780.8460.6200.5080.651
LaBraM-Base0.5010.8520.6030.5480.676
CBraMod0.5400.8720.6540.5740.694
REVE-Base0.581‾\underline{0.581}0.881‾\underline{0.881}0.685‾\underline{0.685}0.620‾\underline{0.620}0.732‾\underline{0.732}
BrainCodes0.6090.8830.7310.6330.739
AuthorsOpenReview ↗

Response from the Authors (2/2)

R2-W2 · Efficiency and the tokenizer at inference

Concern. “The efficiency claim counts only trainable parameters. BrainCodes needs the frozen BioCodec tokenizer at inference, which adds 3.15M parameters, so the deployed model has about 5.0M … I would expect FLOPs, latency and memory on a realistic device rather than trainable parameter counts.”

Response. The reviewer is right that the BioCodec encoder (not decoder) runs at inference, so we now report all executed parameters, FLOPs and latency, even though studies typically report only trainable parameters. The encoder adds 1.59M parameters, whereas the decoder never runs, which gives 3.47M parameters in total. On one CPU thread, BrainCodes processes a 4-s Muse window in 53 ms and 21 MB, same latency as REVE-Base with 69.2M parameters and 278 MB (G4). Its FLOPs are higher than those of small baselines because the encoder processes each channel separately at 250 Hz, and Appendix K gives the full table. Once heads are counted, BrainCodes is also the smallest trained model on every task (G5).

R2-W3 · Circularity of the grounding probes

Concern. “The rhythm probe injects Hann-windowed narrowband bursts into contiguous electrode regions, which is almost the same as the auxiliary pretraining task … The preference index also rewards losing broadband information … On real EEG in Appendix G, REVE and LaBraM decode oscillatory power better on average.”

Response. We agree that synthetic-probe performance alone does not establish grounding to natural rhythms. With respect to Section 5, we therefore distinguish probe generalization, the injection-induced trade-off, and independent evidence from natural EEG.

Probe generalization. The evaluation differs from pretraining: mu and beta are continuous rhythms, the spindle uses a different envelope, and delta lies outside the pretraining range of 4–32 Hz. The gain from injection also persists in a single-electrode probe, removing the shared regional-injection pattern.

Measured trade-off. In two matched retrained models differing only in injection supervision, injection improves rhythm identification but reduces noise-color identification (Table R2-W3). We replace the preference index with these separate measures and distinguish codec from backbone performance (G3).

Natural EEG evidence. Erasing natural rhythm power from frozen features reduces downstream accuracy more than matched random erasure on 8 datasets (G2). This establishes downstream relevance, not uniformly better natural-power decoding than REVE or LaBraM. Appendix G is now also reported relative to the tokenizer to clarify the backbone’s contribution.

Table R2-W3. [New experiment] Bottleneck probes for two matched retrained models, with and without the auxiliary (aux) injection objective. Balanced accuracy averaged over tested SNR levels from 35 to 15 dB; chance is 0.25 for rhythm identification and 0.50 for noise-color identification.

Injected markerBAC with auxBAC without aux
Rhythm, regional0.7330.695
Rhythm, single electrode0.5500.507
Noise color0.7530.835

We hope that the new analyses and efficiency measurements strengthen the reviewer's assessment. We thank the reviewer for motivating these additions and invite them to also consider the experiments presented in responses to R1 and R3 in their final recommendation.

AuthorsOpenReview ↗

Following up on the rebuttal

Dear Reviewer qDiP,

Thank you again for your review. Here is a summary of how we addressed your main points:

  • W1 (thresholding): BrainCodes has the highest mean AUROC, AUPRC, Cohen's κ and weighted F1 (Table R2-W1), and the highest AUPRC on Siena which is, according to established literature, the most informative metric on that task.
  • W2 (tokenizer at inference): we report 3.47M executed parameters plus FLOPs, latency and memory; a 4-s Muse window takes 53 ms and 21 MB on one CPU thread, much faster than real time, and same latency as REVE-Base with 278 MB (G4).
  • W3 (probe circularity): we replaced the preference index with separate measures from two models in matched settings, with and without injection, and LEACE shows that natural rhythm power is used downstream (G2, G3).

We would appreciate knowing whether this rebuttal addresses your concerns.

Thank you for the detailed responses and additional analyses. I appreciate the authors’ efforts to address my concerns and will maintain my original rating of weak acceptance.

Reviewer_gJefOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1(−1)
Read review
Paper summary

BrainCodes is a contribution to the rapidly growing literature on EEG foundation models, focussing on a lightweight design and on encouraging learning of spatial and spectral representations more directly than in pretext tasks using mask prediction. To this aim, BrainCodes uses a range of established architectural components, including a frozen pre-trained tokenizer and factorized transformer layers, with masked-code prediction, as well as a auxiliary tasks using injection of synthetic oscillatory signals into the EEG. Evaluation includes a range of standard benchmarks covering diverse neurophysiological monitoring tasks, including sleep staging, attention assessment, and cognitive load estimation, as well as evaluation on customer-grade EEG, reporting competitive or improved performance compared to the tested set of baselines. In a separate set of analyses, internal representations of spatial and spectral features as intended by the BrainCodes design are probed and compared to baseline models.

Critical strengths

The synthetic oscillatory injection auxiliary task is in my view the most original and interesting contribution. By injecting localized narrow-band oscillations into EEG and asking the model to predict properties of the injected signal (frequency, duration, and spatial location), the authors aim to explicitly encourage representations to preserve physiologically relevant properties reflected by the injected signals. The ablation in Table 6 supports the usefulness of this objective: removing it reduces downstream performance on all three evaluated datasets and produces the largest decrease among the components tested. Under typical self-supervised pre-training assumptions, one would hope that fundamental oscillatory patterns can be directly learned from the EEG data. If it would turn our to be true that even for basic oscillations, self-supervised pre-training on EEG "needs help" from synthetic data, this could be an interesting aspect to investigate further.

So far, the analyses provided in the manuscript and supplement support that learning of representations of spatial and spectral features worked as intended, at least to some degree. This aspect would gain substantially if it were possible to provide evidence that the emergence of these representations played a role in downstream performance.

Critical weaknesses

The authors claim two primary contributions. First, "A compact foundation model on top of discrete codes" - however, the architectural ingredients are mostly established, such as the vector-quantized tokenizer, masked discrete-code prediction, and factorized temporal/channel attention. The number of relevant parameters is not uniquely small, either (especially when also counting the tokenizer; see for example EEGPT). The most original component is the oscillatory injection auxiliary task. More broadly, the idea of pertaining on synthetic data, however, also already exists (Grieger et al., Data-Efficient Sleep Staging with Synthetic Time Series Pretraining, https://arxiv.org/html/2403.08592v1) but this related work is not cited.

The authors second claim: "Transfer to sparse, consumer-grade montages. To our knowledge, we provide the first systematic evaluation of EEG foundation models on consumer-grade hardware". However, previously, EEG-X was "evaluated on seven diverse datasets from four EEG headsets, ranging from portable consumer-grade devices to clinical-grade systems, in both real-world and clinical settings" (https://arxiv.org/html/2511.08861v1). Specifically, EEG-X was evaluated on datasets including three emotive datasets (DREAMER, Crowdsourced, and STEW).

Missing important baselines:

LUNA (Latent Unified Network Architecture), which explicitly targets efficient computation and heterogeneous electrode layouts, close to Braincodes motivation. arxiv.org/abs/2510.22257

CodeBrain, ICLR 2026: a particularly relevant comparison which includes discrete EEG tokenization, a focus on spectral information, and a motivation for efficient backbone design.

EEG-X is another strong missing baseline, as it addresses two central BrainCodes motivations (Transfer across electrode layouts, focus on frequency-domain information) (arxiv.org/html/2511.08861v1#S2.F3).

In general, it would be crucial to evaluate the already included and missing baselines at matched evaluation protocols, for example, with same CV settings. The authors already acknowledge this issue ("We are therefore not directly rank-comparable") but then rank the results anyhow. Evaluation under matched downstream conditions would include identical data splits, electrode subsets, labeled-data and HPO budgets.

In summary, as critical weakness, both the novelty and performance superiority of BrainCodes are substantially less strongly supported than claimed in the manuscript.

Questions for authors

In the opening paragraph of the introduction, the authors state "EEG however presents a different setting, in which a masked interval is only weakly constrained by its neighborhood", contrasting EEG to patch reconstruction in language and vision. This statement is an overstated and insufficiently supported motivation. The ability to reconstruct EEG from the temporal context highly depends on many parameters such as patch size, signal components, signal quality, and is not fundamentally different from the other named domains. Successful representation learning also does not require perfect mask reconstruction. BIOT / Yang et al. also seems wrongly cited in this context. Can you please clarify?

Table 2 reports a higher mean accuracy for BrainCodes than REVE-Base (0.609 versus 0.581). However, the Siena dataset result accounts for a major part of this aggregate advantage. Could the authors complement the mean-based analysis with a median-based report, together with a leave-one-dataset-out sensitivity analysis? This would clarify how consistently the improvements are across tasks and how sensitive the reported overall advantage is to individual datasets/outliers.

Contribution "Evidence of retained spatial and spectral structure" (page 2) in its current form is mostly a check that the model learns spatial and spectral representations as indented by the design. Could the authors also provide evidence that these representations play a role in the downstream performance? (e.g., by running the same analysis on the ablated models, or by interventions targeting these representations directly and measuring downstream effects?)

8 comments
AuthorsOpenReview ↗

Response from the Authors (1/3)

We thank the reviewer for identifying our contributions and for suggesting analyses that strengthen the evaluation. Shared details are provided in G1–G7, while reviewer's questions are addressed below.

R1-W1 · What is novel, and how small is the deployed model?

Concern. “the architectural ingredients are mostly established, such as the vector-quantized tokenizer, masked discrete-code prediction, and factorized temporal/channel attention. The number of relevant parameters is not uniquely small, either (especially when also counting the tokenizer; see for example EEGPT).”

Response. We agree that the individual architectural ingredients are established. The contribution lies in their use within a compact discrete-code framework, particularly oscillation-injection supervision and pretraining on the codes of an independently released codec. G1 explains these distinctions and the rotary encoding over 3D electrode coordinates. The oscillation injection objective is further motivated because removing it causes the largest ablation drop, from 0.498 to 0.383 mean nBAC (Table 6).

Number of parameters. Including the BioCodec encoder, BrainCodes executes 3.47M parameters; the decoder never runs at inference (G4). It remains smaller than EEGPT-Tiny (Wang et al., 2024), a model that scores lower on all shared tasks. Counting task heads, BrainCodes also trains the fewest parameters among the compared foundation models on every task (G5).

Revision. We distinguish the 1.9M trainable backbone from the 3.47M parameters executed at inference and report task-head-inclusive counts separately.

R1-W2 · How does this method differ from prior synthetic pretraining?

Concern. “the idea of pertaining on synthetic data, however, also already exists (Grieger et al., Data-Efficient Sleep Staging with Synthetic Time Series Pretraining) but this related work is not cited.”

Response. We thank the reviewer and now cite Grieger et al. (2024) and distinguish synthetic-only pretraining from our auxiliary objective. Grieger et al. pretrain on synthetic sinusoids. BrainCodes, in contrast, adds short bursts to real EEG and also predicts their duration and location. Removing this objective lowers downstream mean nBAC from 0.498 to 0.383 (Table 6). Our contribution is therefore not synthetic pretraining broadly, but a specific supervision and its demonstrated downstream benefit.

R1-W3 · Distinction from prior consumer-hardware evaluation

Concern. “previously, EEG-X was ‘evaluated on seven diverse datasets from four EEG headsets, ranging from portable consumer-grade devices to clinical-grade systems’”

Response. We thank the reviewer and we now cite the EEG-X preprint (Foumani et al., 2025). We clarify that our contribution concerns systematic evaluation on four-electrode consumer montages, rather than consumer hardware broadly.

Distinction. EEG-X’s consumer datasets—DREAMER, STEW and Crowdsourced—use 14-channel Emotiv headsets. DREAMER and STEW are evaluated in-domain; Crowdsourced tests transfer from TUAB with a frozen encoder and a target-specific classifier. The updated version adds 64-channel PhysioNet-MI but still reports no Muse or in-ear evaluation. BrainCodes evaluates actual four-electrode Muse and in-ear recordings and complements these with systematic electrode removal, including transfer without backbone or classifier retraining, using interpolation for missing channels (R3-W5).

Revision. We narrow the original claim to these specific hardware and transfer settings.

AuthorsOpenReview ↗

Response from the Authors (2/3)

R1-W4 · Missing baselines: CodeBrain, LUNA and EEG-X

Concern. “Missing important baselines: LUNA (Latent Unified Network Architecture) … CodeBrain, ICLR 2026 … EEG-X is another strong missing baseline.”

Response. We thank the reviewer for these valuable suggestions. We now include CodeBrain in Table 2 and discuss LUNA and EEG-X explicitly. BrainCodes exceeds CodeBrain on three of four shared tasks and on average, with approximately 13M fewer trainable backbone parameters (Table R1-W4). LUNA reports only TUAB among our tasks, where it is on par with BrainCodes. EEG-X v2 evaluates TUEV as a binary task and pretrains on it, so that result is not comparable with our six-class evaluation. Its consumer datasets use 14-channel Emotiv headsets, whereas our consumer evaluation targets four-electrode Muse and four-electrode in-ear recordings, thus EEG-X reports neither setting (R1-W3). Methodologically, EEG-X reconstructs rASR-cleaned targets using a dictionary-based convolutional transformation to improve frequency- and shape-aware comparison. BrainCodes instead predicts discrete codec codes and adds localized oscillation supervision; R3-W2 examines its frequency weighting.

Table R1-W4. Balanced accuracy on the four tasks shared with CodeBrain; CodeBrain as published (Ma et al., 2026), BrainCodes from Table 2.

Model (in BAC)TUABTUEVISRUCEEGMAT
CodeBrain0.8290.6430.7840.751
BrainCodes0.8110.6900.7960.772

R1-W5 · Matched evaluation protocols

Concern. “it would be crucial to evaluate the already included and missing baselines at matched evaluation protocols … Evaluation under matched downstream conditions would include identical data splits, electrode subsets, labeled-data and HPO budgets.”

Response. BrainCodes’ advantage persists in evaluations using shared downstream protocols. Table 4 uses identical splits and a shared recipe. We additionally reran BrainCodes under the NeuroRVQ paper’s released code and protocol, obtaining the highest mean nBAC (G7).

Additional matched tests. With identical windows and probes, BrainCodes exceeds REVE-Base in 17 of 18 electrode-removal conditions (R3-W5) and simple spectral features on 8 of 10 datasets (R3-W1).

Comparison scope. In Table 2 we compare BrainCodes with published baseline results on the same data splits. Each baseline retains its original downstream training and checkpoint selection procedure as is common practice, rather than being retrained under a common recipe. Our shared-protocol evaluations provide complementary evidence that the advantage persists.

R1-Q1 · Masked reconstruction in EEG and the BIOT citation

Concern. “‘a masked interval is only weakly constrained by its neighborhood’ … This statement is an overstated and insufficiently supported motivation. … BIOT / Yang et al. also seems wrongly cited in this context.”

Response. We agree that the original statement was too broad. Predictability depends indeed on the masking pattern, signal components and recording conditions. We clarify two distinct motivations below. We also remove the BIOT citation from this argument, since BIOT uses a contrastive objective rather than masked-waveform reconstruction, although its method includes patch masking.

  1. Temporal ambiguity. EEG events may depend on inputs not observed in the recording, in a way that temporal context does not uniquely determine a masked interval. Squared-error patch reconstruction additionally emphasizes high-power components, often favoring slow activity over weaker rhythms (He, 2014; Donoghue et al., 2020). Our frequency-perturbation analysis measures this weighting and shows a flatter profile for BioCodec’s spectral loss (R3-W2).
  2. Spatial redundancy. Nearby electrodes can make masked-channel reconstruction easier through volume conduction. We quantify this by comparing spherical-spline recovery when a channel is masked alone versus together with its nearest electrodes (Table R1-Q1). Our design. BrainCodes combines discrete-code prediction with joint masking of neighboring electrodes. Cross-entropy predicts a distribution over possible code targets rather than a single waveform estimate, while spatial masking reduces direct copying from nearby channels. Speech, also a waveform driven by unrecorded input, moved to discrete targets in the same way (wav2vec 2.0; HuBERT).

Table R1-Q1. [New experiment] Fraction of channel variance recovered by spherical-spline interpolation (Perrin et al., 1989), when the target channel is masked alone or together with its nearest 12.5% of electrodes, as in BrainCodes pretraining.

MaskPhysioNet-MIHBN-EEG
Channel only61%82%
Channel + neighbors (BrainCodes)2%3%
AuthorsOpenReview ↗

Response from the Authors (3/3)

R1-Q2 · Median and leave-one-dataset-out analysis

Concern. “the Siena dataset result accounts for a major part of this aggregate advantage. Could the authors complement the mean-based analysis with a median-based report, together with a leave-one-dataset-out sensitivity analysis?”

Response. Yes, and the model’s ranking holds under both analyses. BrainCodes has the highest median and the best average rank on all benchmarks, and it ranks first in all 21 leave-one-dataset-out (LODO) splits. We follow Demšar (2006), who recommends ranks and Wilcoxon tests across datasets because a mean is sensitive to outliers, as the reviewer noted (see also Agarwal et al., 2021; Dehghani et al., 2021). The tables below give the results per benchmark.

Without Siena, which gives most of the mean margin over REVE-Base on the standard suite, BrainCodes still matches REVE-Base (0.602 against 0.598 mean nBAC) with 1.9M against 69M parameters, and this is the narrowest of the 21 splits. Under linear probing, BrainCodes also ranks first (average rank at 1.43).

Table R1-Q2a. [New analysis] Standard suite, fine-tuning (Table 2, 7 datasets). BrainCodes ranks first by mean nBAC in 7 of 7 leave-one-dataset-out (LODO) splits; Friedman p < 0.001.

ModelMean nBACMedian nBACAverage rank
BrainCodes0.6090.6221.57
REVE-Base0.5810.5321.86
CBraMod0.5400.5232.86
LaBraM0.5010.4894.00
BIOT0.4780.4704.71

Table R1-Q2b. [New analysis] NeuroRVQ suite under the paper’s own code and protocol (Table G7, BrainCodes from the [New] run). BrainCodes ranks first in 4/4 LODO splits; Friedman p = 0.002.

ModelMean nBACMedian nBACAverage rank
BrainCodes0.5970.7071.38
NeuroRVQ0.5590.6692.12
NeuroGPT0.4840.5453.62
CBraMod0.4660.5204.12
LaBraM0.4500.5524.50
BrainOmni0.4080.4465.50
EEGPT0.2640.2107.00
BIOT0.2010.1397.75

Table R1-Q2c. [New analysis] Wearable EEG benchmark, full fine-tuning (Table 4, 3 datasets). BrainCodes ranks first in 3/3 LODO splits; Friedman p = 0.066.

ModelMean nBACMedian nBACAverage rank
BrainCodes0.5130.4301.33
CBraMod0.4180.2793.00
LaBraM0.4130.2932.33
REVE-Base0.3500.3013.33
CSBrain0.3230.2405.00

Table R1-Q2d. [New analysis] Linear probing (Table 3, 7 datasets; with the corrected mean nBAC). BrainCodes ranks first in 7/7 LODO splits; Friedman p = 0.049.

ModelMean nBACMedian nBACAverage rank
BrainCodes0.5020.4861.43
REVE-Base0.4330.4801.86
CBraMod0.3090.2872.71

R1-Q3 · Do the represented properties matter downstream?

Concern. “Could the authors also provide evidence that these representations play a role in the downstream performance? (e.g., by running the same analysis on the ablated models, or by interventions targeting these representations directly and measuring downstream effects?)”

Response. Yes, we ran the suggested intervention (G2): we erased each item from the frozen features and then refitted the downstream probe. Erasing rhythm power lowered balanced accuracy by 0.042 to 0.212 on eight datasets, whereas random erasures of the same size cost at most 0.028. Likewise, erasing the 1/f exponent costs more than every random erasure on 7 of 8 datasets, most on sleep staging. We also ran it on the ablated model: without injection, erasing rhythm power still costs 0.06–0.19 (R3-W3).

We thank the reviewer for motivating these additions. We hope that the additional experiments, benchmark analyses, and revisions address the concerns raised. We also encourage the reviewer to consider the complementary evidence presented in our responses to R2 and R3.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Overview Response from the Authors (6/6)

References

  1. Agarwal, R., et al. (2021). Deep reinforcement learning at the edge of the statistical precipice. NeurIPS.
  2. Avramidis, K., et al. (2025). Neural codecs as biosignal tokenizers. arXiv:2510.09095.
  3. Baevski, A., et al. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. NeurIPS.
  4. Barmpas, K., et al. (2025). NeuroRVQ: Multi-scale biosignal tokenization for generative foundation models. arXiv:2510.13068.
  5. Belrose, N., et al. (2023). LEACE: Perfect linear concept erasure in closed form. NeurIPS.
  6. Borsos, Z., et al. (2023). AudioLM: A language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
  7. Copet, J., et al. (2023). Simple and controllable music generation. NeurIPS.
  8. Dehghani, M., et al. (2021). The benchmark lottery. arXiv:2107.07002.
  9. Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.
  10. Döner, B., et al. (2025). LUNA: Efficient and topology-agnostic foundation model for EEG signal analysis. NeurIPS.
  11. Donoghue, T., et al. (2020). Parameterizing neural power spectra into periodic and aperiodic components. Nature Neuroscience, 23(12), 1655–1665.
  12. El Ouahidi, Y., et al. (2025). REVE: A foundation model for EEG – adapting to any setup with large-scale pretraining on 25,000 subjects. NeurIPS.
  13. Foumani, N. M., et al. (2025). EEG-X: Device-agnostic and noise-robust foundation model for EEG. arXiv:2511.08861.
  14. Grieger, N., Mehrkanoon, S., and Bialonski, S. (2024). Data-efficient sleep staging with synthetic time series pretraining. arXiv:2403.08592.
  15. Heo, B., Park, S., Han, D., and Yun, S. (2024). Rotary position embedding for vision transformer. ECCV, 289–305.
  16. He, B. J. (2014). Scale-free brain activity: Past, present, and future. Trends in Cognitive Sciences, 18(9), 480–487.
  17. Hsu, W.-N., et al. (2021). HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM TASLP, 29, 3451–3460.
  18. Jiang, W.-B., Zhao, L.-M., and Lu, B.-L. (2024). Large brain model for learning generic representations with tremendous EEG data in BCI. ICLR.
  19. Ma, J., et al. (2026). CodeBrain: Bridging decoupled tokenizer and multi-scale architecture for EEG foundation model. ICLR.
  20. Perrin, F., et al. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalography and Clinical Neurophysiology, 72(2), 184–187.
  21. Saito, T., and Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10(3), e0118432.
  22. Wang, G., et al. (2024). EEGPT: Pretrained transformer for universal and reliable representation of EEG signals. NeurIPS.
  23. Wang, J., et al. (2025). CBraMod: A criss-cross brain foundation model for EEG decoding. ICLR.
  24. Warner, C., et al. (2026). ZUNA: Flexible EEG superresolution with position-aware diffusion autoencoders. arXiv:2602.18478.
  25. Yang, C., Westover, M. B., and Sun, J. (2023). BIOT: Cross-data biosignal learning in the wild. NeurIPS.
  26. Zhou, Y., et al. (2025). CSBrain: A cross-scale spatiotemporal brain foundation model for EEG decoding. NeurIPS.
AuthorsOpenReview ↗

Overview Response from the Authors (5/6)

[G6] Can BrainCodes transfer to sparse montages without fine-tuning? Yes. Without any backbone adaptation, BrainCodes keeps 97% (PhysioNet-MI) and 98% (OpenBMI-ERP) of its full-montage accuracy with only 4 electrodes, versus 83% and 87% under full fine-tuning (Appendix C). Our BrainCodes model fitted on the full montage and applied to reduced layouts without retraining outperforms REVE-Base in 17 of 18 conditions, including 8-channel OpenBCI and 4-channel Muse layouts (R3-W5), which supports that montage robustness is inherited in BrainCodes. For these new experiments, we either fit a linear probe per electrode subset (Table R3-W5a), or remove electrodes only at test time (Table R3-W5b).

[G7] Does BrainCodes still lead under the NeuroRVQ benchmark's own protocol? Yes. As R1 suggested, we reran BrainCodes on the NeuroRVQ benchmark's own code and protocol (Barmpas et al., 2025). BrainCodes has the highest mean nBAC of all models (0.597 vs. 0.559 for NeuroRVQ), higher than under our original protocol in Appendix B (0.579). It scores higher on 3 of the 4 tasks and is within one standard deviation on OpenBMI-ERP, and it ranks first in all 4 leave-one-dataset-out splits (Table R1-Q2b). Our original protocol was therefore conservative, as the paper anticipated (R1-W5).

Table G7. [New experiment] BrainCodes rerun under NeuroRVQ benchmark's own protocol. Balanced accuracy, mean ± s.d. over 10 folds per task; NeuroRVQ as published.

ModelHigh-GammaOpenBMI-ERPPavlovPhysioNet eyesMean nBAC
NeuroRVQ0.700 ± 0.0730.876 ± 0.0330.574 ± 0.0270.869 ± 0.0260.559
BrainCodes0.760 ± 0.0680.867 ± 0.0300.597 ± 0.0210.890 ± 0.0230.597

Overall assessment

Recent EEG foundation models have improved mainly by scaling up. LaBraM pretrains on about 2,500 hours of EEG (Jiang et al., 2024), CBraMod on more than 9,000 (Wang et al., 2025), and REVE on more than 60,000, with up to 408M parameters (El Ouahidi et al., 2025). BrainCodes shows a complementary direction: a pretraining objective grounded in physiology lets a 1.9M-parameter backbone, pretrained on about 820 hours, match or outperform these models, including on consumer wearables (G1, G7, R3-W4). The new evidence in this rebuttal shows that this advantage is robust, that the spectral and spatial structure is used downstream, and that the efficiency holds once task heads and inference are counted.

Revisions to the paper. The PDF with changes highlighted is uploaded and available for review.

  • [Sections 1–2] Technical novelty (G1, R1-W1, R3-W1): clarified which components are new to EEG pretraining.
  • [Section 1, Section 5.3, Appendix L] Masking motivation (R1-Q1): clearer reasoning, backed by channel interpolation (Table R1-Q1); BIOT citation corrected.
  • [Section 1, Section 4.3, Appendix A] "First" claim (R1-W3): narrowed to sparse consumer montages, namely the four-electrode Muse and in-ear EEG.
  • [Section 2, Appendix A] (R1-W2, R1-W3, R1-W4): EEG-X, LUNA, Grieger et al., ZUNA now cited.
  • [Section 3.3, Table 2, Appendices I and J] CodeBrain (R1-W4): added as a baseline (Table R1-W4).
  • [Section 4.1, Appendix K] Parameters and inference cost (G4, G5, R2-W2): parameters trained per task, plus parameters executed, FLOPs, memory, and latency (Tables G4, G5).
  • [Table 3] Corrected mean nBAC: 0.309, 0.433, 0.502 for CBraMod, REVE-Base and BrainCodes.
  • [Section 5.2, Figure 2d–e, Appendix G] Downstream relevance (G2, G3, R1-Q3, R2-W3): LEACE erasure and the matched injection probe replace the preference index (Tables G2, G3).
  • [Appendix I] Robustness (R1-Q2, R2-W1): medians, average ranks, leave-one-dataset-out splits and threshold-free metrics (Tables R1-Q2, R2-W1).
  • [Appendix B] NeuroRVQ protocol (G7, R1-W5): rerun under the benchmark's own code and protocol (Table G7).
  • [Section 5.3, Table 5, Appendix L] Codec targets (R3-W2): frequency weighting of the loss and per-stage decoding (Tables R3-W2a–c).
  • [Appendix G] Spectral-feature baseline (R3-W1): BrainCodes features outperform hand-crafted spectral features on 8 of 10 datasets (Table R3-W1).
  • [Appendix E] Model pretrained without injection (R2-W3, R3-W3): it still encodes and uses natural rhythms (Tables R2-W3, R3-W3).
  • [Appendix C] Frozen transfer to unseen montages (G6, R3-W5): frozen-probe retention, and one model outperforming REVE-Base in 17 of 18 reduced layouts (Tables R3-W5a/b).
AuthorsOpenReview ↗

Overview Response from the Authors (4/6)

[G3] Does the backbone retrieve spectral information beyond the codec? Yes. By re-running the injection probe of Section 5.2 in matched settings, we find that weak rhythms become less linearly decodable after BioCodec’s quantizer, and BrainCodes makes them decodable again, to about the level before RVQ. From quantizer output to bottleneck, rhythm identification rises by 0.17–0.37 balanced accuracy, whereas noise-color identification rises by 0.01–0.05, so the backbone adds oscillatory information specifically. This analysis replaces the preference index, which, as R2 noted, rewarded losing broadband information (R2-W3).

Table G3. [New experiment] Linear-probe identification accuracy of signals added to 512 PhysioNet-MI test windows at three different SNR (dB): one of four rhythms (delta, mu, beta, spindle) over a region of electrodes or on a single electrode (chance 0.25), and white vs. pink noise as a broadband control (chance 0.50). Results are reported in BAC and means over 3 draws.

ModelRhythm, 35 / 25 / 15 dBColor, 35 / 25 / 15 dBC=1, 35 / 25 / 15 dB
BioCodec pre-RVQ0.47 / 0.76 / 0.950.92 / 0.98 / 1.000.33 / 0.50 / 0.78
BioCodec post-RVQ0.26 / 0.41 / 0.700.75 / 0.93 / 0.980.26 / 0.29 / 0.47
BrainCodes0.43 / 0.78 / 0.970.80 / 0.94 / 0.990.30 / 0.51 / 0.83
REVE-Base0.63 / 0.90 / 0.990.73 / 0.88 / 0.940.40 / 0.60 / 0.86
LaBraM0.51 / 0.80 / 0.970.93 / 0.96 / 0.980.35 / 0.52 / 0.79
CBraMod0.42 / 0.64 / 0.870.87 / 0.94 / 0.950.29 / 0.39 / 0.61
BIOT0.47 / 0.68 / 0.880.74 / 0.80 / 0.840.28 / 0.31 / 0.37
CSBrain0.36 / 0.51 / 0.740.71 / 0.80 / 0.860.29 / 0.34 / 0.47

[G4] What does BrainCodes cost during inference? 3.47M parameters and real-time CPU latency.

Finding. BrainCodes executes 3.47M parameters at inference: the 1.88M backbone and the 1.59M BioCodec encoder. The 5.0M quoted in the reviews includes the 1.56M BioCodec decoder, which never runs here. On one CPU thread, BrainCodes processes a 4-s Muse window in 53 ms with 21 MB memory and a 10-s, 19-channel window in 523 ms, both well within real time. Its FLOPs exceed those of the other small models because the codec encodes each channel separately, which makes its cost lowest on sparse wearable montages, the setting it targets (R2-W2). The full cost table is added to Appendix K.

Table G4. [New experiment] BrainCodes runs in real time on a single CPU thread. GFLOPs, latency, and peak memory per window. *CSBrain assigns electrodes to anatomical regions and has none for TP9 and TP10, so it cannot run on the Muse layout.

ModelMuse × 4s: GFLOPsCPU, 1 threadGPUMemory (MB)19 ch × 10s: GFLOPsCPU, 1 threadGPUMemory (MB)
BrainCodes1.5553 ms11.9 ms21.319.06523 ms13.8 ms55.8
BIOT0.185.1 ms3.2 ms13.11.9732.9 ms7.4 ms16.8
LaBraM0.209.1 ms7.2 ms17.72.5651.4 ms7.3 ms22.0
CBraMod0.138.3 ms7.4 ms18.91.5136.8 ms7.5 ms20.7
CSBrainn/an/an/an/a3.37162.1 ms47.1 ms33.8
REVE-Base2.2257.1 ms12.1 ms278.030.88399.3 ms12.8 ms291.8

[G5] Does BrainCodes train fewer parameters? Yes. BrainCodes trains 1.9M–2.5M parameters per task, the fewest of all compared foundation models on every task. The sizes in Table 2 count only backbones, as in prior work, which hides task heads: CBraMod trains 19.1M–69.3M parameters per task, CSBrain up to 125.3M and CodeBrain up to 79.6M, all based on each model's code (see also Table G5). Counting task heads therefore strengthens our efficiency claim (R1-W1, R2-W2).

Reporting convention: Models built on neural codecs list a frozen tokenizer separately, even when it runs at inference, as MusicGen (Copet et al., 2023) and AudioLM (Borsos et al., 2023) both do. We followed this convention, but in the revision we also report the parameters at inference.

Table G5. [New experiment] Trained parameters counted from released code; "Size as reported" is quoted from the original papers. The 4.1M in parentheses adds the frozen codec encoder at inference.

ModelSize as reportedTrained per task (based on released code)
BrainCodes1.9M1.9M to 2.5M (4.1M including tokenizer)
BIOT3.2M3.2M
LaBraM-Base5.8M5.8M
CBraMod4.0M19.1M to 69.3M
CSBrain8.9M23.1M to 125.3M
CodeBrain15.2M31.4M to 79.6M
REVE-Base69M69.2M to 70.0M

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.