acceptodds
Under review as a conference paper at ICLR 2027

M²-EEG: A Macro–Micro Pretraining Paradigm for Expanding the Temporal Scope of EEG Foundation Models

14 views

Abstract

Electroencephalography (EEG) foundation models have advanced transferable representation learning, yet pretraining and evaluation remain centered largely on short local segments. For recording-scale decisions, aggregating independently encoded windows leaves local representations unaware of context elsewhere in the recording. We introduce M²-EEG, a Macro–Micro pretraining paradigm built on the principle that waveform resolution and temporal reach should be modeled separately. A high-resolution Micro pathway preserves local waveform evidence, while a lower-bandwidth Macro pathway models recording-wide context. Aligned fusion produces context-aware temporal and recording-level representations within a single whole-recording backbone pass. Using a 5,300-hour quality-controlled EEG corpus, M²-EEG is progressively pretrained from local representation learning to long-context modeling and joint Macro–Micro adaptation using a unified family of multi-scale spectral prediction objectives. Across six long-context EEG datasets, frozen M²-EEG representations outperform those of the compared EEG foundation models under aligned downstream readouts, while retaining strong short-window transfer with the same complete backbone. Controlled context-span experiments further show that expanding contextual support improves predictions even when local EEG evidence is held fixed. Together, these results support Macro–Micro pretraining as a scalable approach to extending EEG foundation learning from local segments toward recording-scale interpretation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG Foundation Models, Long-Context Modeling, Representation Learning, Self-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eoXQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes M²-EEG, a Macro–Micro pretraining paradigm for EEG foundation models that separates high-resolution local waveform modeling (Micro) from lower-bandwidth recording-scale context (Macro), then reconnects them via temporally aligned fusion. The model is progressively pretrained on ~5,300 hours of quality-controlled EEG with multi-scale masked spectral prediction. Under a frozen-backbone protocol with aligned task readouts, M²-EEG outperforms six compared EEG foundation models on six long-context datasets, while retaining competitive short-window transfer. Controlled context-span analyses further argue that broader contextual support helps even when local EEG evidence is held fixed.

Critical strengths

This work provides strong mechanistic evidence beyond benchmark performance: enlarging contextual support improves prediction while keeping local EEG evidence fixed, and aligned joint fusion outperforms dimension‑matched late fusion. These observations deliver generalizable scientific insights instead of only empirical leaderboard gains.

Critical weaknesses

Baselines are short-window pretrained foundation models evaluated without a strong recording-level contextualizer, while M²-EEG was designed and pretrained to supply that context inside the backbone. This comparison shows that M²-EEG provides usable recording context under the stated protocol, but it does not yet establish that Macro–Micro pretraining is necessary or superior to putting a capable temporal aggregator on on top of existing short-window EEG foundation models.

Questions for authors

How much of the long‑context gain remains if Macro inputs are replaced by learned downsampled Micro summaries?

1 comment
AuthorsOpenReview ↗

Response to Reviewer eoXQ

Thank you for the constructive feedback and the specific experimental suggestion. We conducted additional comparisons to address both points. The contextualized-baseline comparison uses the common rebuttal evaluation protocol and is not directly comparable to the main-paper frozen-probe results.

Weakness. Can strong downstream contextualization match M²-EEG?

We equipped frozen REVE-Large and CBraMod backbones with learned full-record Transformers under the same downstream protocol. Across three downstream refits, M²-EEG retains the highest mean performance on NMT (0.8623 vs. 0.8152/0.7772 BAcc) and Sleep (0.6383 vs. 0.5959/0.5885 participant-macro F1). Thus, the observed advantage persists when competing backbones receive learned recording-level context.

Question. How much long-context gain remains with learned downsampled Micro summaries?

We implemented the suggested construction using a learned strided convolution and linear projection of Micro features as Macro inputs. All variants were retrained under the same 8,000-update protocol rather than replacing inputs only at inference. We also trained a corresponding local-only reference for each input; the reported long-context gain is measured relative to its respective local-only reference.

Macro inputNMT BAccSleep pF1Long-context gain (NMT / Sleep, pp)
Spectral descriptors0.87500.6600+9.24 / +10.82
Learned Micro downsampling0.79890.5947+3.80 / +4.48

Learned Micro downsampling retains approximately 41% of the measured long-context gain on both tasks, while spectral descriptors remain higher in absolute terms by +7.61 pp [+3.73, +11.49] and +6.53 pp [+4.20, +8.86]. The percentages are point estimates, and both gains move inference context as well as the input, so they are protocol-specific rather than a fully isolated estimate of the Macro-input contribution. We hope this directly addresses the question.

Reviewer_tbxDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes M$^2$-EEG, a Macro–Micro pretraining paradigm for learning recording-scale EEG context while preserving local waveform detail. A high-resolution Micro pathway processes bounded 10-second regions, while a lower-bandwidth Macro pathway propagates context across the recording; temporally aligned fusion connects the two. The model is pretrained on approximately 5,300 hours using multi-scale masked spectral prediction. Frozen-backbone evaluation covers six long-context datasets spanning abnormality classification, sleep staging, vigilance regression, and seizure detection, with five additional short-window tasks. M$^2$-EEG obtains the best reported mean across the long-context comparisons, including substantial Event-F1 gains on TUSZ and CHB-MIT. Controlled Sleep-EDFx experiments further show that expanding context from local support to the full recording improves subject-macro F1 by 0.035 while holding the local target and Micro evidence fixed.

Critical strengths
  1. The paper is well-motivated. Restricting high-resolution processing to 10-second regions while propagating lower-bandwidth states through sparse region–global attention gives a clear way to increase temporal reach without dense attention over all waveform tokens.
  2. The evaluation is comprehensive: it spans six datasets and four task types. The long-context evidence is broad and consistently favorable.
  3. Controlled analysis provides direct evidence that the model uses nonlocal context rather than merely benefiting from a larger local encoder.
Critical weaknesses
  1. In the comparison experiments with baselines, M$^2$-EEG forms each representation using the complete recording, whereas the other foundation models generally supply locally encoded window features without a learned recording-level contextualizer. Therefore, it is unclear whether Macro–Micro pretraining is better than strong alternatives given the same temporal support. A baseline comparison or ablation in which the alternative model has access to the same recording-level context is needed.
  2. The NMT aggregation experiment evaluates mean, attention, and temporal-Transformer aggregation only on M$^2$-EEG's own Micro features. Testing whether applying the same strong temporal contextualizer to another pretrained backbone could match or exceed M²-EEG is needed.
  3. The stage-wise comparison is not a controlled ablation: the increase in Sleep-EDFx F1 from 0.508 after context learning to 0.567 after joint adaptation coincides with both a change in the optimization regime and additional training steps. The gain therefore cannot be attributed specifically to joint Macro–Micro adaptation without an equal-budget continuation that receives the same number of additional updates but omits joint adaptation.
  4. Four of the six long-context datasets are also sources in M$^2$-EEG's self-supervised corpus, while competing checkpoints have different pretraining data. Although exact test-record matches are excluded, M$^2$-EEG may already be familiar with the target datasets' acquisition domains, including their recording sites, equipment or montages, participant populations, and signal characteristics, whereas the baselines do not necessarily receive comparable exposure.
Questions for authors

Please see weaknesses.

8 comments
AuthorsOpenReview ↗

Response to Reviewer tbxD

Thank you for the constructive review and suggestions. We conducted additional experiments with contextualized external backbones, architecture-matched pretraining controls, equal-update continuations, and target-dataset removal to address the main concerns. The external-backbone and target-domain-removal comparisons use a common rebuttal evaluation protocol; their absolute scores are not directly comparable to the main-paper frozen-probe results.

W1. Providing comparable recording-level context to external backbones.

We equipped frozen REVE-Large and CBraMod representations with learned full-record Transformers, using the same downstream hyperparameter search, training procedure, and evaluation protocol as M²-EEG. Contextualizer widths accommodate the different backbone feature dimensions. Results are mean ± SD across three downstream refits:

Task / metricM²-EEGREVE-Large + TransformerCBraMod + Transformer
NMT / BAcc0.8623 ± 0.00830.8152 ± 0.01090.7772 ± 0.0218
Sleep / participant-macro F10.6383 ± 0.00270.5959 ± 0.00590.5885 ± 0.0063

M²-EEG retains the highest observed means on both tasks even when all backbones receive learned recording-level contextualization. This comparison matches downstream training and evaluation procedures, while backbone capacities and pretraining corpora remain different.

W2. Testing temporal aggregation on external pretrained representations.

The original aggregation experiment used M²-EEG's Micro features. The new comparisons above directly extend contextualization to the external backbones' own representations.

We additionally evaluated bidirectional Mamba, bidirectional SSM, and sparse local–global aggregators on NMT:

NMT BAccTransformerBi-MambaBi-SSMSparse local–global
REVE-Large0.81520.76090.73370.7935
CBraMod0.77720.73370.71200.7554

None of the alternative aggregators exceeds the corresponding Transformer. These alternatives are single fits, so the comparison reflects observed scores rather than replicated differences across aggregator families.

To further isolate the contribution of pretraining, we trained two models with the same Macro–Micro architecture, Stage-I checkpoint, 8,000 updates and downstream heads, varying only the pretraining supervision — whole-recording 10/60-s targets versus 1-s targets in isolated 10-s regions. Both receive full-record inference. Long-context pretraining improves NMT by +8.15 pp [+4.33, +11.97] and Sleep by +9.00 pp [+6.32, +11.68].

These results show that learning recording-scale dependencies during pretraining improves transfer even under identical architecture and full-record inference context, complementing the stronger external-backbone comparisons.

W3. Isolating joint adaptation from additional optimization.

We performed an equal-update control starting from the same checkpoint. Both branches receive 8,000 updates with identical long- and short-input exposure. The only difference is whether gradients from the long-context prediction loss update Micro: J0 allows these gradients, while J1 blocks them.

J0 − J1 (pp)NMT BAccSleep pF1
Direct continuation+4.89 [−0.77, +9.48]+2.97 [−0.96, +5.89]
After shared 2,000-step final refinement+7.61 [+3.95, +11.27]+6.20 [+3.75, +8.65]

Although the direct differences remain uncertain, both post-refinement confidence intervals exclude zero.

We further repeated the full pretraining procedure with three independent random initializations. The post-refinement J0−J1 improvements remain positive across all seeds: +5.98/+6.52/+7.07 pp on NMT and +4.19/+3.90/+3.50 pp on Sleep, with all corresponding paired confidence intervals excluding zero.

These controls demonstrate a reproducible post-refinement benefit from joint Macro–Micro adaptation beyond simply performing additional optimization updates.

AuthorsOpenReview ↗

Response to Reviewer tbxD

W4. Assessing target-domain exposure and transfer independence.

To assess whether familiarity with the acquisition domain drives the improvements, we removed all recordings of the target dataset from the context-training pool, SSL checkpoint selection and the fitting pool, retraining each time; downstream task training sets are unchanged. The control removes an equal number of recordings from the other datasets instead. These scores use a native-head evaluation protocol rather than the contextualized-Transformer protocol above (full data 0.8750 NMT / 0.6600 Sleep; after removal 0.8478 / 0.6248).

Performance change (pp)NMT BAccSleep pF1
Target-dataset removal−2.72 [−7.21, +1.77]−3.52 [−6.41, −0.63]
Matched-size removal elsewhere−6.52 [−9.77, −2.91]−7.17 [−9.46, −4.91]
Target removal − matched removal+3.80 [+0.15, +7.45]+3.65 [+1.33, +5.97]

Target-dataset removal produces a smaller performance decline than removing the same number of recordings elsewhere. The NMT change under target-domain removal is uncertain, while Sleep shows a measurable decline. These controls assess sensitivity to target-dataset exposure within M²-EEG, rather than its margin over external backbones.

Our dataset audit also confirms that no exact test recordings overlap with pretraining. Sleep-EDFx participant separation is verified within its repository, while NMT patient identities remain unresolved. Importantly, CHB-MIT and SEED-VIG are entirely absent from the pretraining corpus, providing complementary dataset-unseen evaluations. Earlier pretraining exposure and cross-repository identity uncertainty remain outside the scope of the removal control.

Overall, the new experiments show that M²-EEG retains its advantage over contextualized external backbones, while architecture-matched and equal-update controls provide direct evidence for the contributions of recording-scale pretraining and joint adaptation. We hope these additional findings address the reviewer's main concerns.

I thank the authors for the additional experiments and clarifications. Some of my concerns have been addressed. Two issues remain:

  1. The rebuttal M$^2$-EEG scores differ substantially from the main-paper results (0.8374 vs. 0.8623 on NMT and 0.5880 vs. 0.6383 on Sleep-EDFx). Please clarify this.
  2. Did target-dataset removal include Stage-I local pretraining, including the cached 10-s windows and the inherited Stage-I checkpoint? If Stage-I retained recordings from the target dataset, target-domain familiarity may already be encoded in the Micro representation. Moreover, the current experiment is an internal comparison between M$^2$-EEG variants and does not control for the pretraining-corpus mismatch with the external backbones. Sleep-EDFx also shows a significant decline after target-dataset removal.
Reviewer_vMoROpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces M²-EEG, a hierarchical EEG foundation model designed to extend self-supervised representation learning from short local segments to complete EEG recordings.

The proposed architecture separates local waveform resolution and long-range temporal context through two pathways. The Micro encoder processes high-resolution waveform patches within fixed ten-second regions, while the Macro encoder operates on lower-bandwidth spectral and amplitude descriptors across the recording. A temporally aligned fusion mechanism combines these representations to produce contextual temporal features and recording-level embeddings.

The model employs a progressive self-supervised training strategy consisting of local representation learning, long-context learning, joint Macro–Micro adaptation, and full-coverage contextual refinement. Its objectives predict spectral information at multiple temporal scales.

M²-EEG is pretrained on approximately 5,300 hours of EEG from ten public data sources and evaluated on six long-context datasets covering recording-level classification, sleep staging, vigilance regression, and seizure event detection. Additional experiments assess short-window transfer across five tasks.

The authors report improvements over six existing EEG foundation models under frozen-backbone long-context evaluation. Controlled context-span experiments provide evidence that extending the available temporal context improves predictions even when the local EEG evidence associated with the target remains fixed.

Critical strengths

The reported improvements are substantial on several long-context benchmarks. The fixed-target Sleep-EDFx experiment is particularly valuable. The authors also compare aligned joint fusion against late fusion using matched 512-dimensional interfaces, finding a smaller but positive advantage. These experiments provide reasonable evidence that broader context can improve downstream decoding rather than merely increasing model size.

Critical weaknesses

W1. The experimental design does not adequately isolate recording-scale pretraining from architectural advantages and downstream contextualization.

The central claim is that incorporating long-range EEG context during self-supervised pretraining produces more transferable representations than approaches that learn context only during downstream adaptation.

However, the principal comparisons evaluate frozen M²-EEG representations against conventional EEG foundation models whose pretrained representations are primarily local. The competing models do not receive comparable recording-level contextual processing within their pretrained backbones.

Although this comparison is meaningful for assessing the representations available immediately after pretraining, it does not determine whether the gains are attributable to the proposed pretraining paradigm, hierarchical architecture, longer effective receptive field, or additional representational capacity.

The paper includes a downstream temporal-aggregation comparison on NMT, but this is insufficient to establish the general advantage of recording-scale pretraining across the six evaluated tasks.

A particularly important missing comparison is an architecture-matched model with the same Macro–Micro structure trained without the proposed long-context pretraining objectives. Similarly, stronger downstream contextualization baselines—such as pretrained local EEG encoders followed by appropriately trained temporal Transformers or state-space models—would help establish the incremental value of learning context during pretraining.

The paper acknowledges that comparisons against separately trained recording-level contextualizers remain open. Nevertheless, this is central to its methodological claim rather than a peripheral future extension.

W2. The evaluation does not fully establish the robustness, efficiency, or independence of the reported improvements.

The manuscript reports results from a single pretrained checkpoint per backbone, with variation measured across independently initialized downstream runs. Consequently, the reported standard deviations do not capture uncertainty in pretraining or sensitivity to corpus composition and checkpoint selection.

This distinction is particularly important because the framework involves several sequential training stages, different sampling budgets, a learned routing mechanism, and a final contextual refinement stage. Without independent pretraining replications, it is difficult to determine how sensitive the reported advantages are to this relatively complex optimization procedure.

There are also limitations in the independence of the evaluation datasets. Several downstream datasets belong to domains represented in the pretraining corpus. Although the authors audit exact recording overlap and identifiable subject overlap, they explicitly acknowledge that cross-repository participant identities cannot be exhaustively resolved.

Some long-context results therefore support recording-disjoint transfer, but not necessarily fully subject-independent or unseen-domain generalization.

Finally, although the architecture is designed for efficient long-context processing, a complete quantitative comparison of pretraining cost, inference time, memory usage, and scaling behavior against relevant alternatives is missing. The asymptotic attention complexity is useful, but actual efficiency matters because the system still processes all Micro regions during full-coverage inference.

Together, these issues make it difficult to evaluate the reproducibility and practical cost-effectiveness of the proposed pretraining paradigm.

Questions for authors
  1. Can the authors compare M²-EEG against an architecture-matched Macro–Micro model trained without long-context self-supervised objectives? This would clarify whether the improvement comes from recording-scale pretraining rather than the architecture's ability to process long recordings.

Q2. Can the authors provide stronger comparisons with existing pretrained EEG encoders followed by task-specific long-context sequence models, such as temporal Transformers or state-space models, under matched computational and parameter budgets? How much advantage remains when those alternatives receive comparable recording-level context?

Q3. How sensitive are the results to independent pretraining seeds and the progressive training schedule? In particular, does the complete four-stage procedure consistently outperform simpler training strategies with matched optimization budgets?

Q4. Can the authors report actual inference latency, peak memory usage, and throughput as recording duration increases? How does the full-coverage Macro–Micro model compare with local-encoder-plus-sequence-model baselines on recordings spanning several hours?

4 comments
AuthorsOpenReview ↗

Response to Reviewer vMoR

Thank you for the thoughtful review and constructive suggestions. We conducted additional architecture-matched controls, contextualized external-backbone comparisons, independent pretraining replications, domain-exposure analyses, and inference benchmarks to address the main concerns.

W1/Q1. Separating long-context pretraining from architectural advantages.

We trained two models with the same Macro–Micro architecture, Stage-I checkpoint, 8,000 updates and downstream heads, differing only in pretraining supervision: one predicts 10/60-s targets over the whole recording, the other 1-s targets within isolated 10-s regions. Both are evaluated using the same full-record inference interface, holding evaluation-time contextual access constant. Long-context pretraining improves NMT balanced accuracy by +8.15 pp [+4.33, +11.97] and Sleep participant-macro F1 by +9.00 pp [+6.32, +11.68].

A separate control held the 1-s targets, masks and spectral support fixed and varied only cross-region communication. Enabling cross-region communication improves NMT by +5.43 pp [+1.89, +8.97] and Sleep by +4.68 pp [+2.27, +7.09].

Together these show a training benefit under a fixed architecture and inference interface, plus a separate communication benefit at matched objectives.

W1/Q2. Stronger downstream contextualization baselines.

We gave frozen REVE-Large and CBraMod a learned full-record Transformer under the same search and evaluation procedure as M²-EEG, with widths sized to each feature dimension. Results use a common rebuttal protocol, mean ± SD across three refits, and are not comparable to the main-paper frozen-probe values:

Task / metricM²-EEGREVE-Large + TransformerCBraMod + Transformer
NMT / BAcc0.8623 ± 0.00830.8152 ± 0.01090.7772 ± 0.0218
Sleep / participant-macro F10.6383 ± 0.00270.5959 ± 0.00590.5885 ± 0.0063

On NMT, bidirectional Mamba, bidirectional SSM and sparse local–global contextualizers were also tried on both backbones; none exceeded its Transformer (best 0.7935 vs 0.8152 on REVE-Large, 0.7554 vs 0.7772 on CBraMod; single fits).

M²-EEG therefore retains the highest observed means even when the external backbones receive learned recording-level contextualization. Procedures are aligned; backbone size, corpora and compute are not. M²-EEG's complete system is in fact about one-thirtieth the parameters of REVE-Large, which rules out the simplest reading that the gain comes from fielding a larger model; architecture, effective capacity and compute still differ, and the architecture-matched experiment above carries the pretraining attribution.

W2/Q3. Controlling training budget and assessing pretraining variability.

To distinguish joint adaptation from additional training, we continued the same checkpoint for 8,000 updates under identical long- and short-input exposure. The two branches differ only in whether the long-context prediction loss updates Micro: J0 allows these gradients, whereas J1 blocks them.

J0 − J1 (pp)NMT BAccSleep pF1
Direct continuation+4.89 [−0.77, +9.48]+2.97 [−0.96, +5.89]
After shared 2,000-step final refinement+7.61 [+3.95, +11.27]+6.20 [+3.75, +8.65]

The direct differences remain statistically inconclusive, whereas both post-refinement confidence intervals exclude zero.

We additionally ran three full pretrainings from random initialization, sharing no checkpoint. The post-refinement J0−J1 improvements hold in all three: +5.98, +6.52, and +7.07 pp on NMT, and +4.19, +3.90, and +3.50 pp on Sleep. All six paired intervals exclude zero.

At the same 20% Micro-access budget, learned routing beats random selection by +5.43 pp [+1.99, +8.87] on NMT and +4.71 pp [+2.37, +7.05] on Sleep. Against uniform selection it stays inconclusive on NMT.

These controls support the contributions of joint adaptation and of learned routing under matched update counts and access budgets respectively.

AuthorsOpenReview ↗

Response to Reviewer vMoR

W2. Dataset exposure, independence, and robustness.

The overlap audit finds no exact test-record matches; Sleep separation holds within its repository, NMT identities stay unresolved, and CHB-MIT and SEED-VIG are absent from pretraining.

We then removed all recordings of the target dataset from the SSL context pool, checkpoint selection and the fitting pool, retraining each time; downstream task-training sets were unchanged. The control removes an equal number of recordings from other datasets instead. These scores use a native-head evaluation protocol rather than the contextualized-Transformer protocol above (full data 0.8750 NMT / 0.6600 Sleep; after removal 0.8478 / 0.6248).

Performance change (pp)NMT BAccSleep pF1
Target-dataset removal−2.72 [−7.21, +1.77]−3.52 [−6.41, −0.63]
Matched-record-count removal elsewhere−6.52 [−9.77, −2.91]−7.17 [−9.46, −4.91]
Target removal − matched removal+3.80 [+0.15, +7.45]+3.65 [+1.33, +5.97]

The target removal costs less than removing the same count elsewhere; the NMT change is itself inconclusive while Sleep declines measurably. That shows different sensitivity to the two removals, not how much of the external-system margin survives. Earlier pretraining exposure and unresolved cross-repository identities remain limitations.

M²-EEG also holds its lead over the contextualized REVE-Large baseline under 10% channel dropout (0.8478 vs. 0.7880), 10 dB noise (0.8424 vs. 0.7772) and halved amplitude (0.8533 vs. 0.7935). The advantage is not uniform across recording length: among the 50 test recordings of 30 minutes or more, REVE-Large reaches 0.8344 against 0.8213.

W2/Q4. Measured inference efficiency.

Full-coverage inference was timed with actual weights and synthetic 16-channel waveforms on one GPU (FP32, batch 1), from waveform processing through encoding, contextualization, fusion and readout, excluding I/O; baselines include their Transformer contextualizers.

Model1-h latency (s)4-h latency (s)4-h peak memory (MiB)
M²-EEG (complete system)0.67482.6924491
REVE-Large + Transformer3.979316.16721,685
CBraMod + Transformer2.691810.7659363

M²-EEG has the lowest measured latency at both durations; CBraMod needs less peak allocated memory.

Overall, these results support a pretraining contribution under a fixed architecture, higher observed means than the tested contextualized baselines on NMT and Sleep, and a post-refinement joint-adaptation benefit that recurs across three independent pretraining seeds. We hope they address the main concerns.

Dear Reviewer,

Thank you again for your detailed and constructive feedback. We have conducted additional experiments addressing the key concerns, including architecture-matched comparisons, contextualized baseline evaluations, and independent pretraining replications. We hope these additions help clarify the contributions and empirical evidence of M²-EEG.

If any further clarification would be helpful, we would be glad to continue the discussion.

Thank you for your time and consideration !

Dear Reviewer,

Thank you again for your detailed review. As the discussion period is approaching its deadline, we wanted to briefly follow up on our responses and the additional experiments addressing your concerns, particularly the architecture-matched controls, independent pretraining replications, contextualized baseline comparisons, and inference efficiency analyses.

We would greatly appreciate knowing whether these additional results and clarifications address your main concerns. If so, we kindly invite you to reconsider your assessment and update your rating accordingly before the discussion period closes.

If any concerns remain, we would be happy to provide further clarification.

Thank you for your time and consideration!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Reply to all reviewers

We thank all three reviewers for their careful reading and constructive comments. We reply to each point under the corresponding review, and summarise here what the new analyses show and how the paper has changed (new text is marked in blue in the revised PDF).

The main findings hold, with one attribution narrowed. Long-context pretraining beats local-only training under a fixed architecture, a fixed inference interface and an equal update budget. M²-EEG also retains the highest mean on every row when REVE-Large and CBraMod are given the same full-record contextualizer. The equal-update J0−J1 difference, however, is established only after the shared full-coverage refinement; its before-refinement intervals include zero in all four lineages.

FindingNMTSleep-EDFx
Long-context − local training, fixed architecture, inference and 8,000 updates+8.15 [+4.33, +11.97]+9.00 [+6.32, +11.68]
Cross-region communication only, targets and masks fixed+5.43 [+1.89, +8.97]+4.68 [+2.27, +7.09]
Joint adaptation after shared refinement, three independent pretrainings+6.52 ± 0.54 (SD)+3.86 ± 0.35 (SD)
Target-dataset removal − record-count-matched removal+3.80 [+0.15, +7.45]+3.65 [+1.33, +5.97]

Difference in balanced accuracy (NMT) and subject-macro F1 (Sleep-EDFx), percentage points.

New analyses in this reply

  • An architecture-matched 2 × 2 control that trains the same Macro–Micro architecture from the same Stage-I checkpoint for equal updates, and a separate control that varies only cross-region communication (vMoR Q1, tbxD W1).
  • REVE-Large and CBraMod given full-record Transformer readouts under a shared downstream search (vMoR W1/Q2, tbxD W1/W2, eoXQ).
  • An equal-update J0/J1 gradient-path control replicated across three independent pretrainings from random initialization (vMoR Q3, tbxD W3).
  • Target-dataset removal from the later pretraining stages, paired with a record-count-matched removal control (vMoR W2, tbxD W4).
  • Learned downsampled Micro summaries as Macro inputs, each with its own local-only reference; they retain approximately 41% of the measured long-context gain on both tasks while remaining below the original descriptors in absolute score (eoXQ). These are single-fit point estimates, and both gains also change inference context, so the fraction does not isolate the Macro-input contribution.
  • Input perturbations and a recording-length stratification, where the advantage does not hold on recordings of 30 minutes or more (vMoR W2).
  • Measured inference latency, peak allocated memory and parameter counts (vMoR Q4).

The revision also reports findings that appear only in the paper, not in the individual replies: the before-refinement intervals for all four lineages (F.4), the local-versus-full inference contrast within the long-trained model (F.3), the native-probe rows and the extension of the contextualized comparison to TUSZ and CHB-MIT (F.2), and the Bridge and Micro-freezing ablations together with the cross-task fixed-target extension (F.4, F.7).

Changes to the paper

New Appendix F. Sections F.1–F.8 report the analyses above, with their evaluation settings, replication units and interval construction stated in F.1.

Sections 2.2, 3.2 and 4. Five added sentences point to the corresponding appendix section: the Macro-input comparison; the contextualized-baseline comparison; the removal controls; the equal-update gradient controls; and the cross-task fixed-target extension.

Section 4, stage progression. The sentence on the refinement gain now reads that "the additional refinement gain and the direct-stage difference under equal updates (Appendix F.4) remain uncertain", so the stage trajectory is no longer attributed to joint adaptation alone.

Appendix D.1, cost accounting. The statement that peak-memory measurements were unavailable is removed; measured latency and peak allocated memory are reported in Appendix F.8.

Conclusion. The closing sentence records that contextualized baselines and independent pretraining replications are supplied in Appendix F, while matched-resource schedule comparisons remain open.

Unchanged. The main results, tables and figures are as submitted. The new comparisons use evaluation settings distinct from the frozen-probe protocol of the main tables; the caption of Table 17 states that their absolute values come from different protocols and are not directly comparable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.