acceptodds
Under review as a conference paper at ICLR 2027

NeurDuo-EEG: A Long-Sequence EEG Foundation Model with Persistent State and Explicit Memory

16 views

Abstract

Electroencephalography (EEG) is recorded continuously over hours, with relevant dynamics spanning timescales from milliseconds to hours. Most EEG foundation models nevertheless process fixed windows independently, limiting their ability to capture information encoded in long-timescale dynamics. State-space architectures enable persistent recurrent processing, but long-range information remains implicitly compressed in recurrent states. We present NeurDuo-EEG, a causal EEG foundation model with channel-resolved persistent memory. NeurDuo-EEG introduces multi-timescale memory management with learned consolidation and selective retrieval, enabling persistent modelling of continuous EEG with fixed-size state. It is pre-trained on 3,955 hours of EEG from 17 public datasets using multichannel autoregressive prediction of discrete spectral codes. Across three short-window and two long-sequence downstream tasks, NeurDuo-EEG achieves the best performance on four of five benchmarks, including all three short-window tasks and seizure detection, where AUC-PR improves from to over the strongest non-NeurDuo baseline. NeurDuo-EEG also remains competitive on sleep staging and supports efficient streaming inference, with nearly constant per-chunk latency as the available history grows to one hour. Notably, the Small variant achieves this with only 4.7M backbone parameters. These results demonstrate the value of persistent, multi-timescale modelling for both long-sequence and short-window EEG analysis.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Electroencephalography (EEG), EEG Foundation Models, Long-Sequence Modeling, State-Space Models (Mamba), Causal Streaming Inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vZMzOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

NeurDuo-EEG combines a fast per-channel state-space stream, periodic cross-channel attention, a slower memory-writing stream, and bounded explicit memory. It is pretrained on approximately 3,955 hours of EEG and evaluated on short-window tasks as well as sleep staging and seizure detection with extended causal context. The paper compares model sizes, context lengths, architectural components, and carrying versus resetting recurrent state. The sequence evaluation reports AUC-PR, sleep-staging metrics, and false alarms per hour, alongside inference latency and memory measurements.

Critical strengths
  1. Direct evidence that persistent backbone state is useful. With the same evaluation framework, carrying rather than resetting state improves CHB-MIT 15-channel AUC-PR from 0.307 to 0.413 and Sleep-EDF macro-F1 from 0.745 to 0.755 (Table 6). The context-length sweep is also informative because it reports non-monotonic behavior rather than implying that longer context always helps. The Large model reaches 0.471 AUC-PR on the 15-channel seizure task, compared with 0.285 for the strongest reported classical pipeline (Table 3).

  2. Unusually thorough protocol analysis and disclosure. Architectural variants are re-pretrained, the common readout is capacity controlled, and both short-window and sequence tasks are considered. The appendix identifies relevant baseline pretraining overlaps and the shared chb01/chb21 patient, and quantifies the effect of removing that patient's test units. This sensitivity analysis preserves the AUC-PR advantage. These details make the experimental evidence substantially more interpretable.

Critical weaknesses
  1. The streaming-cost comparison does not measure the baseline pipelines used for the accuracy results. Section 5.3 and Appendix F.2 compare advancing NeurDuo by 0.5 seconds with re-encoding the entire history for each baseline. However, the sequence-task accuracy protocol encodes baseline inputs in separate 30-second units and uses a causal Mamba context head. Such a pipeline can cache unit embeddings and recurrent head state; it need not repeatedly encode an hour of raw EEG. The current experiment demonstrates the advantage over an expanding-window implementation, but does not establish the deployment advantage over those practical baselines. In addition, the 4.7M Small-model figure is a backbone count: the 15-channel CHB-MIT readout contains another 24.81–24.95M parameters (Table 16). Please report complete-pipeline latency, memory, and parameter counts at a common prediction cadence, including the readout and required signal accumulation time. This would preserve the valid bounded-state result while making the comparison relevant to the reported task performance.

  2. The false-alarm statistic has a boundary artifact and is not a deployment estimate. Appendix C explicitly sets the threshold from the evaluated fold's positive-unit scores and merges consecutive positive predictions across recording boundaries (p. 24, lines 1279–1288). The threshold choice is transparently described as a common-sensitivity comparison, which is acceptable as an exploratory operating-point analysis. Merging across separate recordings, however, can join unrelated alarms or attach a false alarm to a seizure in another record. Please recompute alarms with strict recording and participant boundaries. A validation-selected threshold, held fixed on test data, together with event-level sensitivity and detection delay, is needed if the paper retains practical alarm-performance claims. This concern does not invalidate the threshold-independent AUC-PR comparison, which is the stronger part of the evidence.

Questions for authors

1.How does NeurDuo compare with cached 30-second baseline encoders plus the existing causal context head when measuring the entire inference pipeline at the same output cadence? Please include the task-head parameters and accumulation latency.

2.How much do FP/h results change after preventing merges across recordings? Can the authors additionally report test results using thresholds selected only on validation subjects, with event sensitivity and delay?

3.Table 5 gives CHB-MIT AUC-PR of 0.413 for the full model, 0.389 without retrieval, and 0.406 for the Mamba-only variant. What are the paired fold-level differences? The evidence for persistent state is stronger than the evidence that the explicit-memory construction itself explains the long-sequence gain. Clarifying this distinction would materially improve the contribution claim.

5 comments

[Q1] How does NeurDuo compare with cached 30-second baseline encoders plus the existing causal context head when measuring the entire inference pipeline at the same output cadence? Please include the task-head parameters and accumulation latency.

We measured this directly. Every model runs its complete CHB-MIT inference pipeline on the same GPU (one NVIDIA GH200, fp32) and emits one decision per 30 s unit, the cadence of Table 3.

  • Window baselines. Each 30 s unit is encoded once, when it closes. Its readout embedding is cached, and the same causal context head (two Mamba layers) runs over the last 20 cached embeddings. Nothing is re-encoded.
  • NeurDuo. The backbone advances one 0.5 s chunk at a time as the signal arrives, and its state is never reset. When the unit closes, the same readout and context head produce the decision.
  • Models. The table includes CodeBrain (ICLR 2026), a recent EEG foundation model we added as a baseline in this revision.
  • Task heads. All models use the same two heads: the shared readout, whose size is set by the readout budget (K = 90,000 at 15 channels), and the context head (1.75M).
  • Two deployment settings. We time a single stream, and 64 concurrent streams (for example, 64 monitored patients) whose units close at the same time.

Table R1 · Whole-pipeline cost at one decision per 30 s unit, CHB-MIT, 15 channels. Median over the timed repetitions after warm-up; all models on the same GPU.

ModelBackboneReadout + context headTotalAccumulation (s)Decision latency, 1 stream (ms)Decision latency, 64 streams (ms)GPU time per decision per stream, 64 streams (ms)
BIOT3.2M23.1M + 1.8M28.1M308.323.30.36
CBraMod4.9M23.1M + 1.8M29.7M3010.417.50.27
LaBraM5.8M23.1M + 1.8M30.7M3016.435.50.55
EEGPT25.3M23.2M + 1.8M50.2M304.8205.03.20
ST-EEGFormer-S25.4M23.1M + 1.8M50.2M3014.4734.911.48
REVE-Base69.2M23.2M + 1.8M94.1M3013.9149.92.34
CodeBrain13.5M23.1M + 1.8M38.4M3038.874.41.16
NeurDuo-Small4.7M23.1M + 1.8M29.5M306.67.75.83
NeurDuo-Base24.3M23.1M + 1.8M49.2M309.010.58.56
NeurDuo-Large67.8M23.2M + 1.8M92.7M3012.313.711.62
  • "Decision latency" is the time from the last sample of a unit to the decision, for all streams together.
  • "GPU time per decision" is all the work attributable to one decision of one stream. For NeurDuo, this includes the 60 chunk steps taken while the unit was arriving.

Results:

  1. Accumulation latency is the same for every model. At one decision per 30 s unit, every pipeline waits for the unit to fill: 30 s.
  2. With many streams, NeurDuo decides fastest once a unit closes, and its latency hardly grows with the number of streams.
    • NeurDuo has already processed 59 of the unit's 60 chunks while the signal was arriving. Only one chunk step, the readout and the context head remain.
    • With 64 concurrent streams, NeurDuo decides in 7.7–13.7 ms. A cached window encoder must encode all 64 windows at that moment and needs 17.5–735 ms.
    • Even for a single stream, NeurDuo-Small (6.6 ms) decides faster than six of the seven baselines.
  3. The compute of every pipeline is small.
    • With 64 streams, no pipeline needs more than 11.6 ms of GPU time per 30 s decision per stream, that is, under 0.04 % of real time.
    • NeurDuo-Small needs 5.8 ms, less than ST-EEGFormer-S (11.5 ms).
    • One GPU can therefore follow many patients in real time with any of these models.
  4. The pipelines are of comparable size.
    • The task heads are identical for every model: the readout (23.1–23.2M) and the context head (1.75M). The size of a pipeline is therefore set by its backbone.
    • NeurDuo-Small's complete pipeline (29.5M) is the same size as those of BIOT, CBraMod and LaBraM (28.1–30.7M).
    • NeurDuo's state has a fixed size (5.1 MiB for Small) that does not grow with the length of the recording.
    • At 2 channels, the readout shrinks to 3.0–3.2M (K = 12,000).

At this same cadence, NeurDuo reaches an AUC-PR of 0.392–0.471, against at most 0.208 for the window baselines of Table R1, and it raises the fewest false alarms (Table 3; for CodeBrain, 0.174 AUC-PR and 6.90 false alarms per hour).

[Q2] How much do FP/h results change after preventing merges across recordings? Can the authors additionally report test results using thresholds selected only on validation subjects, with event sensitivity and delay?

Merges across recordings. Stopping every alarm at the boundary of its recording changes FP·h⁻¹ very little:

  • At the operating point of Table 3, the change is at most 1.5 % for any model at either montage, and at most 0.2 % for the NeurDuo models.
  • At the validation-selected operating point below, the change is at most 1.3 % (Tables R2 and R3 report both counts).
  • No ranking changes. In the revision, every FP·h⁻¹ is counted within recordings.

Validation-selected thresholds. Tables R2 and R3 report the operating point chosen without test labels:

  • For each fold, the threshold that reaches 50 % unit sensitivity on the validation subjects, at the selected epoch, is applied unchanged to the test subjects.
  • Within each recording, consecutive above-threshold 30 s units form one alarm. Consecutive seizure units form one seizure, which counts as detected if any alarm overlaps it.
  • Detection delay runs from seizure onset to the end of the first above-threshold unit inside the seizure. The shortest possible delay is therefore one unit, 30 s.
  • Counts and hours are pooled over the five folds.
  • The submitted runs did not store their validation predictions, so every model was re-trained once with the same folds and seed.
  • The tables include CodeBrain (ICLR 2026), a recent EEG foundation model we added as a baseline in this revision, evaluated under the same protocol.

Table R2 · CHB-MIT, 15 channels, threshold selected on the validation subjects.

MethodPre-trainedFP·h⁻¹FP·h⁻¹, alarms merged across recordingsUnit sensitivityEvent sensitivityMedian delay (s)
BP-GBDT—1.821.810.420.55 (101/185)30
FBCov-TS-Lin—3.583.540.430.46 (86/185)30
ERP-Lin—15.4915.310.500.67 (124/185)30
BIOTyes1.451.450.370.44 (82/185)30
CBraModyes4.454.430.340.33 (61/185)30
LaBraMyes9.819.780.320.41 (75/185)30
EEGPTyes6.706.680.310.35 (65/185)30
ST-EEGFormer-Syes23.2923.200.410.66 (122/185)30
REVE-Baseyes6.546.520.370.55 (102/185)30
CodeBrainyes2.392.380.370.41 (75/185)30
NeurDuo-Smallyes0.280.280.390.43 (79/185)60
NeurDuo-Baseyes1.000.990.430.47 (87/185)30
NeurDuo-Largeyes1.251.250.390.52 (97/185)30

Table R3 · CHB-MIT, 2 channels (T7, P8), same protocol.

MethodPre-trainedFP·h⁻¹FP·h⁻¹, alarms merged across recordingsUnit sensitivityEvent sensitivityMedian delay (s)
BP-GBDT—1.201.200.340.42 (78/185)30
FBCov-TS-Lin—2.612.610.390.52 (96/185)30
ERP-Lin—12.8912.730.550.70 (130/185)30
BIOTyes5.875.850.400.52 (97/185)30
CBraModyes1.691.690.360.44 (81/185)30
LaBraMyes6.566.490.370.50 (92/185)30
EEGPTyes2.992.960.360.44 (82/185)30
ST-EEGFormer-Syes14.2414.200.440.64 (118/185)30
REVE-Baseyes12.1012.050.480.69 (127/185)30
CodeBrainyes3.203.190.400.46 (85/185)30
NeurDuo-Smallyes0.270.270.460.48 (88/185)30
NeurDuo-Baseyes1.111.110.490.55 (101/185)30
NeurDuo-Largeyes0.540.540.460.45 (83/185)30

Results:

  1. NeurDuo raises the fewest false alarms of all models, at both montages.
    • At 15 channels, the three sizes raise 0.28–1.25 false alarms per hour, against at least 1.45 for every baseline.
    • At 2 channels, they raise 0.27–1.11, against at least 1.20. Their unit sensitivity (0.46–0.49) is higher than that of every baseline except ERP-Lin and REVE-Base, which raise 12–13 false alarms per hour.
  2. This holds at useful event sensitivity. NeurDuo detects 0.43–0.55 of the seizures. The pre-trained models that detect more, ST-EEGFormer-S and REVE-Base (0.55–0.69), pay for it with 6.5–23.3 false alarms per hour, 5 to 80 times as many.
  3. Delay. The median delay is one 30 s unit, the shortest possible, for every model except NeurDuo-Small at 15 channels (two units). That model runs at the lowest false-alarm rate of all, 0.28 per hour.

We will make corresponding revisions to the claim in the main text:

  • All FP·h⁻¹ values are counted within recordings.
  • Tables R2 and R3 are added to the appendix, and Section 5.2 reports the validation-selected operating point with event sensitivity and delay alongside the test-selected values of Table 3.

[Q3] Table 5 gives CHB-MIT AUC-PR of 0.413 for the full model, 0.389 without retrieval, and 0.406 for the Mamba-only variant. What are the paired fold-level differences? The evidence for persistent state is stronger than the evidence that the explicit-memory construction itself explains the long-sequence gain. Clarifying this distinction would materially improve the contribution claim.

We agree with this reading, and it reflects how the model is designed. The long-sequence gain comes from the recurrent state carried across units. The memory bank is a short-range store, and information older than the bank is held implicitly in the recurrent state, in particular the slow stream.

Table R4 gives the paired fold-level differences: CHB-MIT, 15 channels, NeurDuo-Small, five cross-subject folds, same seed. Each difference is the full model minus the variant on the same test fold.

Table R4 · Paired fold-level AUC-PR differences on CHB-MIT (full model minus variant). The full model reaches an AUC-PR of 0.584 / 0.506 / 0.512 / 0.372 / 0.093 on folds 0–4 (mean 0.413). Each fold entry is the full model minus the variant on that fold, so a positive entry means the full model is higher. The interval is a 95 % paired-t interval over the five folds.

GroupVariantMean AUC-PRFold 0Fold 1Fold 2Fold 3Fold 4Mean difference [95% CI]t
Persistent stateState reset at every unit (Table 19)0.307+0.125+0.267−0.080+0.207+0.015+0.107 [−0.068, +0.282]+1.69
Persistent stateShuffled unit order0.371+0.029+0.101+0.101−0.043+0.024+0.042 [−0.033, +0.118]+1.56
Explicit memoryw/o memory retrieval0.389+0.105+0.179+0.048−0.108−0.104+0.024 [−0.134, +0.182]+0.43
Explicit memoryw/o slow stream & memory0.394+0.167−0.114+0.067−0.048+0.024+0.019 [−0.115, +0.153]+0.40
Explicit memoryMamba-only0.406+0.099+0.094−0.087−0.069−0.002+0.007 [−0.102, +0.116]+0.18

Results:

  • Persistence is what drives the long-sequence gain.
    • Resetting the state at every unit lowers AUC-PR on four of the five folds, by 0.107 on average, and raises false alarms more than fivefold, from 0.96 to 5.15 per hour.
    • Shuffling the order of the units keeps every input but breaks their temporal order. It lowers AUC-PR on four of the five folds and raises false alarms to 3.79 per hour.
  • The explicit-memory variants stay close to the full model on CHB-MIT. Their differences are small (0.007–0.024) and change sign across folds.

Why this follows from the design.

  1. The memory bank is a short-range store, matched to the time scale of EEG events.
    • It holds eight memory tokens written every 2 s, that is, the last 16 s at 2 s resolution.
    • This is where the evidence about an event concentrates. On CHB-MIT, the 185 test seizures last a median of two 30 s units (IQR 2–3), and 88 % last at most four.
    • A perturbation analysis confirms that the model draws on past events mostly within this range. It uses NeurDuo-Small models re-trained with our current code, with the same folds and seed.
      • In each 10-min test window, we overwrote a 30 s span earlier in the window with a seizure from another recording of the same patient. The scored unit's own input was kept unchanged, and the context head received the original embeddings of all earlier units, so any change in the scored unit's prediction can only come through the backbone's state.
      • When the planted seizure fills the 30 s just before the scored unit, it raises the scored unit's seizure probability by 5.07 percentage points.
      • Moved 16 s further back, it raises the probability by 0.75 points, 15 % of the immediate effect. Moved 32 s back, by 0.23 points, 5 % of it.
      • Most of what the model uses about past events therefore lies in the last 30 s.
  2. Older information is held implicitly in the recurrent state, in particular the slow stream.
    • The planted seizure still raises the prediction when it lies wholly outside the bank: by +0.23 percentage points [+0.05, +0.45] at 32 s, and by +0.07 [+0.01, +0.15] at 60 s (paired 95 % intervals from a subject-level bootstrap).
    • Background EEG spliced in the same way has no effect at any lag.
    • The slow-stream output keeps a trace of the planted seizure for minutes. After 9 min, it still differs from the original by 0.69 % (median over windows), against 0.09 % for background EEG.
    • On CHB-MIT, this time scale is within reach of the recurrent state that every ablation variant keeps. That is why removing retrieval or the slow stream barely moves AUC-PR there, while removing persistence does.
Reviewer_ReVgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NeurDuo is a causal EEG foundation model. A fast per-channel Mamba stream runs on 0.5-s chunks with channel attention. Every 2 s, a memory writer feeds a slower Mamba stream and an 8-slot bank that the fast state queries (Eqs. 5-11). It is pretrained on 3,955 h from 17 datasets, on 32-s sequences, to predict the next per-channel spectral code. Downstream, all models share a capacity-matched readout. Sequence tasks add a causal context head over 20 units of 30 s, and only NeurDuo carries backbone state across units. The paper reports the best result on four of five tasks, a CHB-MIT AUC-PR of 0.471 against 0.285 for the best non-NeurDuo model, and per-chunk latency that stays flat up to one hour of history.

Critical strengths
  1. The evaluation work is careful, and others can reuse it without the model. The shared readout fixes head capacity per dataset (Table 16), and the pretraining-overlap audit flags LaBraM on KaggleERN and BIOT on CHB-MIT (Table 15). Appendix C writes down the CHB-MIT parsing traps, a recording-level oracle (AUROC 0.905 but AUC-PR 0.024) that supports using AUC-PR, and the SEED-VIG split mismatch (resolved from published numbers).

  2. The streaming property is real and measured honestly. Up to one hour of history, per-chunk latency stays at 4.83-8.94 ms and the Small model's state at 349 KB per channel (Tables 20-22). The carry-versus-reset comparison separates backbone state with everything else fixed (Table 6), and Table 19 gives the paired per-fold differences on Sleep-EDF.

Critical weaknesses
  1. Long-timescale memory is claimed but not tested. The abstract motivates the model by dynamics "from milliseconds to hours" and says the results show "the value of persistent, multi-timescale modelling". But the explicit memory only covers 16 s (8 slots written every 2 s, Table 10). Pretraining uses 32-s sequences, with the state reset at each start (L735-736). The paper's own probe finds that history beyond 4 s adds at most 0.017 nats, and 128-s pretraining did not beat 32-s (Appendix B.2, L1034-1036 and L1050-1054). In the component ablation (Table 5), memory retrieval matters most on FACED, a 10-s window task (κ 0.534 to 0.442). On CHB-MIT, removing retrieval or the slow stream with memory moves AUC-PR from 0.413 to 0.389 and 0.394, while fold SDs are 0.11-0.18. The context-length study (Table 4) is not monotone on CHB-MIT: AUC-PR goes 0.324, 0.322, 0.404, 0.331, 0.413, 0.406, 0.379, with SDs of 0.07-0.17, so "peaks at 10 min" reads more into it than five folds support. What is shown is that a recurrent backbone state carried across 30-s units helps under this protocol (Table 6). I could not find evidence, apart from nominal decay time constants read off the weights (Figure 5a), that the slow stream or the bank keeps information over minutes or hours. I'd want a retention test (for example, how far back a planted or annotated event can still be decoded from the state), ablations of the slow stream and bank at long context with paired per-fold differences, and title and abstract claims that match what is shown.

  2. Headline comparisons are within noise or depend on choices that favour NeurDuo. "Best on four of five" takes the best of three NeurDuo sizes in each column, but each baseline has one size. Small and Base each lead on only three tasks. Small trails REVE-Base on FACED (0.589 vs 0.605), and Base trails ERP-Lin on KaggleERN ROC-AUC (0.621 vs 0.623). Large leads on SEED-VIG by Pearson's r but not by R² (0.294 vs 0.297). Several leads are inside one SD: FACED 0.610 vs 0.605 (SD 0.007), and KaggleERN 0.636 vs 0.623 on a fixed set of ten test subjects, where the folds only change the training subset (L1134-1137). I was surprised that on CHB-MIT, a randomly initialised NeurDuo-Small already gets 0.309 (Figure 4). That is above every pretrained baseline (best 0.208) and the best classical pipeline (0.285). Much of the CHB-MIT margin over the pretrained baselines comes from the stateful architecture under this protocol, not from pretraining. The window baselines, all trained with one shared recipe, also fall below a classical pipeline there. FP·h⁻¹ uses thresholds picked on the test folds (L300, L1285-1287), and the CHB-MIT margins come with no paired per-fold comparison or interval. I'd ask for per-size results in the abstract, paired per-fold differences with intervals for CHB-MIT and Sleep-EDF, a causal from-scratch baseline in Table 3 (randomly initialised NeurDuo or a plain per-channel Mamba), and thresholds picked on validation for FP·h⁻¹.

Two smaller points. The two-channel explanation for the Sleep-EDF gap (0.755 vs LaBraM 0.766) is not tested. Also, CaMBRAIN, the closest persistent-state model, is left out because its weights are not available (L288-289).

Questions for authors
  1. Can you show that information older than the 16-s bank still affects predictions? For example, you could perturb an earlier segment and measure the effect on later outputs at growing lags.
  2. What are the per-fold paired differences, with intervals, between NeurDuo and the best baseline on CHB-MIT and Sleep-EDF, and for the retrieval and slow-stream ablations on CHB-MIT?
  3. How do a randomly initialised NeurDuo and a plain causal per-channel Mamba trained from scratch compare with the pretrained baselines in Table 3?
  4. What are the FP·h⁻¹ values when thresholds are picked on validation folds?
6 comments

[Q1] Can you show that information older than the 16-s bank still affects predictions? For example, you could perturb an earlier segment and measure the effect on later outputs at growing lags.

We conducted the suggested perturbation test, asking whether changing past EEG affects a later prediction through the backbone state. We used NeurDuo-Small models re-trained with our current code, with the same folds and seed as Table 3 (CHB-MIT, 15 channels; one model per fold, no further training).

For each 10 min test window, we evaluated the final 30 s unit. We created a perturbed copy by replacing an earlier 30 s span, leaving the evaluated unit’s EEG unchanged. The replacement span ended lag seconds before the start of the evaluated unit. We tested two replacement conditions:

  • Planted seizure: in 2,851 seizure-free windows from 20 patients, the span was replaced with a seizure segment from another recording of the same patient.
  • Background EEG control: in 3,311 windows from 24 patients, the span was replaced with non-seizure EEG.

After replacement, we reran the backbone through the remainder of the window. The downstream context head received the original embeddings of all earlier units and the recomputed embedding of the final unit. Because the final unit’s EEG was unchanged, any prediction difference could arise only through the altered backbone state. We measured the paired change in seizure probability and the relative change in the slow-stream output, with confidence intervals obtained from 2,000 patient-level bootstrap resamples.

Table R1 · Effects of historical EEG perturbations on later predictions and slow-stream outputs. “Before” denotes unperturbed predictions; “Rise” is the mean paired change in seizure probability, in percentage points. Slow-stream change is the median relative output change. Replaced-span times are relative to the start of each 10 min window. Brackets indicate 95% confidence intervals.

LagReplaced spanOutside the bankSeizure probability: planted seizureRise: planted seizure (pp)Seizure probability: background EEGRise: background EEG (pp)Slow-stream change: planted seizureSlow-stream change: background EEG
Before——4.47% [4.34, 4.60]—4.72% [4.53, 4.91]———
0 s9:00–9:30no9.54% [7.28, 12.20]+5.07 [2.79, 7.73]4.70% [4.52, 4.91]−0.02 [−0.10, 0.06]43.5% [23.2, 78.7]3.5% [2.5, 4.9]
4 s8:56–9:26no7.55% [6.11, 9.18]+3.08 [1.62, 4.72]4.71% [4.53, 4.91]−0.01 [−0.09, 0.06]39.4% [18.6, 73.1]3.2% [2.3, 4.5]
16 s8:44–9:14partly5.22% [4.78, 5.74]+0.75 [0.34, 1.26]4.71% [4.52, 4.92]−0.01 [−0.09, 0.06]24.0% [11.5, 54.9]2.4% [1.7, 3.6]
32 s8:28–8:58yes4.69% [4.47, 4.94]+0.23 [0.05, 0.45]4.70% [4.51, 4.89]−0.02 [−0.06, 0.01]13.7% [8.5, 29.0]1.8% [1.2, 2.6]
60 s8:00–8:30yes4.54% [4.39, 4.69]+0.07 [0.01, 0.15]4.71% [4.53, 4.90]−0.01 [−0.03, 0.02]7.1% [5.2, 12.1]1.1% [0.8, 1.7]
120 s7:00–7:30yes4.49% [4.36, 4.63]+0.02 [−0.00, 0.06]4.72% [4.53, 4.91]0.00 [−0.01, 0.01]3.7% [2.9, 7.3]0.60% [0.43, 0.97]
300 s4:00–4:30yes4.47% [4.34, 4.60]0.00 [−0.01, 0.01]4.72% [4.54, 4.91]0.00 [−0.00, 0.01]1.3% [0.9, 2.9]0.16% [0.11, 0.29]
540 s0:00–0:30yes4.47% [4.34, 4.60]0.00 [−0.01, 0.00]4.72% [4.53, 4.91]0.00 [−0.00, 0.00]0.69% [0.36, 1.30]0.09% [0.07, 0.14]

At 16 s the replaced span has left the bank, but the oldest stored token still overlaps it by 2 s; from 32 s on, no stored token has seen it.

Results:

  1. Information older than the bank reaches later predictions. At lags of 32 s and 60 s, a planted seizure increases the evaluated unit’s seizure probability by 0.23 and 0.07 percentage points, respectively, with 95% confidence intervals excluding zero. Background replacements show no detectable mean effect, arguing against a generic splice effect.
  2. There is no cliff at the bank boundary. The prediction effect decreases with lag (+0.75 → +0.23 → +0.07 pp from 16 s to 60 s), alongside the slow-stream change (24.0% → 13.7% → 7.1%). The influence therefore persists beyond the eviction of tokens written during the perturbed span.
  3. The trace stays in the state for minutes. At a lag of 540 s, the slow-stream output still differs from the original by 0.69% [0.36, 1.30] after a planted seizure, compared with 0.09% [0.07, 0.14] for background replacements. The effect on the prediction itself is concentrated in the first minute, while the internal trace persists for several minutes.

These results show that information beyond the bank’s 16 s span can influence later predictions, while an internal trace persists over longer timescales.

[Q2] What are the per-fold paired differences, with intervals, between NeurDuo and the best baseline on CHB-MIT and Sleep-EDF, and for the retrieval and slow-stream ablations on CHB-MIT?

We compare NeurDuo with the strongest baseline on CHB-MIT and Sleep-EDF, and with the retrieval and slow-stream ablations on CHB-MIT. Under matched evaluation settings, we report the paired difference within each of the five folds and a 95% paired-t confidence interval for the mean difference.

1. NeurDuo versus the strongest baseline.

Table R2 · Per-fold paired differences from FBCov-TS-Lin on CHB-MIT (15 channels). Values are NeurDuo minus the fixed baseline; positive ΔAUC-PR and negative ΔFP·h⁻¹ favour NeurDuo. Confidence intervals are for the mean difference across five folds.

MetricNeurDuoFold 0Fold 1Fold 2Fold 3Fold 4Mean difference [95% CI]
ΔAUC-PRSmall+0.176+0.283+0.167−0.030+0.045+0.128 [−0.024, +0.280]
ΔAUC-PRBase+0.139+0.259+0.001−0.114+0.249+0.107 [−0.094, +0.308]
ΔAUC-PRLarge+0.148+0.288+0.219+0.028+0.245+0.185 [+0.059, +0.311]
ΔFP·h⁻¹Small−0.09−1.93−0.88−2.12−1.06−1.21 [−2.24, −0.19]
ΔFP·h⁻¹Base−0.05−1.80−0.47−2.21+0.09−0.89 [−2.19, +0.42]
ΔFP·h⁻¹Large−0.03−1.95−0.86−2.12−0.88−1.17 [−2.24, −0.10]

Table R3 · Per-fold paired differences from LaBraM on Sleep-EDF. Values are NeurDuo minus LaBraM; positive differences favour NeurDuo. Confidence intervals are for the mean difference across five folds. Runs of Table 3.

MetricNeurDuoFold 0Fold 1Fold 2Fold 3Fold 4Mean difference [95% CI]
ΔMacro-F1Small−0.005−0.013−0.016−0.017−0.002−0.011 [−0.019, −0.002]
ΔMacro-F1Base−0.005−0.029−0.027−0.035−0.003−0.020 [−0.038, −0.001]
ΔMacro-F1Large−0.000−0.010−0.029−0.017−0.007−0.013 [−0.026, +0.001]
ΔκSmall−0.001−0.022−0.014−0.001−0.011−0.010 [−0.021, +0.001]
ΔκBase+0.002−0.036−0.014−0.019−0.010−0.016 [−0.032, +0.001]
ΔκLarge+0.003−0.012−0.020−0.005+0.003−0.006 [−0.019, +0.006]

On CHB-MIT, NeurDuo-Large achieves higher AUC-PR than the strongest baseline in all five folds, with a mean-difference interval excluding zero; Small and Base have positive mean differences but intervals spanning zero. Small and Large also show consistently lower false-alarm rates, with intervals excluding zero. On Sleep-EDF, NeurDuo-Small has lower macro-F1 than LaBraM, while the κ difference is not statistically significant.

2. The retrieval and slow-stream ablations on CHB-MIT To assess whether the ablation gains reported in the paper are robust to training randomness, we extended the evaluation to three seeds on the same five cross-subject folds. Table R4 reports the full-minus-ablated AUC-PR differences within each fold, averaged over seeds, together with confidence intervals for the mean differences.

Table R4 · Per-fold paired AUC-PR differences for NeurDuo-Small ablations on CHB-MIT (15 channels), averaged over three seeds. Differences are computed as full minus ablated model within each fold and seed. Each fold column reports the mean difference over three seeds. The 95% confidence interval is calculated across the five fold-level mean differences. Positive values favour the full model.

AblationFold 0Fold 1Fold 2Fold 3Fold 4Mean difference [95% CI]
w/o memory retrieval−0.088−0.016−0.030+0.089+0.004−0.009 [−0.088, +0.071]
w/o slow stream−0.046+0.012−0.049+0.144−0.054+0.001 [−0.103, +0.106]

Removing memory retrieval or the slow stream did not consistently reduce performance on CHB-MIT in the multi-seed evaluation. We will therefore revise the corresponding claims in the paper. Their contributions are task-dependent: the FACED and Sleep-EDF ablations in Table 5, together with the additional motor-imagery ablations discussed below, provide evidence of benefits from specific components on other tasks. We therefore retain these components in the architecture while qualifying the claims to reflect the lack of consistent gains on CHB-MIT. To further examine variability across training seeds, we analysed the per-seed performance of NeurDuo and all pre-trained baselines, as shown in Table R5.

Table R5 · CHB-MIT (15 channels), performance across three seeds. All models are re-trained with the same folds and three seeds. Each seed column reports mean AUC-PR over five test folds. The final column reports the mean and sample standard deviation of the three seed-level means.

ModelSeed 42Seed 0Seed 1Across-seed mean ± s.d.
BIOT0.2130.1590.2090.193 ± 0.030
CBraMod0.1810.1680.1480.166 ± 0.016
LaBraM0.1490.1010.1490.133 ± 0.027
EEGPT0.2440.1920.2100.215 ± 0.026
ST-EEGFormer-S0.0110.0200.0190.017 ± 0.005
REVE-Base0.1260.0990.1280.118 ± 0.016
CodeBrain0.1640.1370.1680.156 ± 0.017
NeurDuo-Small, full0.4140.3050.3290.349 ± 0.057
NeurDuo-Small, w/o memory retrieval0.3450.3410.3880.358 ± 0.026
NeurDuo-Small, w/o slow stream0.3720.2990.3740.348 ± 0.043

Several baselines, as well as NeurDuo, show variation across seeds, although its magnitude differs across models. Two dataset characteristics may contribute to the uncertainty in component comparisons. First, each test fold contains only 4–5 patients with seizures, with the largest contributor accounting for 31–57% of seizure-positive units and the top two for 54–73%. This concentration suggests that performance on a few patients may substantially affect fold-level scores. Second, seizures are generally short: the median duration is two 30 s units (IQR 2–3), and 88% last at most four units. Such short events may limit opportunities to distinguish the benefits of components operating at different timescales.

These observations suggest that the current CHB-MIT evaluation does not clearly isolate the long-range contribution of the memory components. Nevertheless, NeurDuo-Small achieves higher five-fold mean AUC-PR than every pre-trained baseline under each of the three seeds. To examine whether component contributions vary across EEG paradigms, we additionally evaluated BCI Competition IV-2a with a full set of ablations, as reported below.

3. An additional benchmark: BCI Competition IV-2a To examine component contributions in another EEG paradigm, we added BCI Competition IV-2a, a four-class motor-imagery dataset with 22 channels and 4 s trials. We follow the cross-subject split used by CBraMod: subjects 1–5 for training, 6–7 for validation, and 8–9 for testing. All neural models use the shared-readout training protocol of our other short-window tasks, with five seeds. As trials are classified independently, this evaluation tests component contributions within short windows rather than long-range information retention.

Table R6 · BCI IV-2a, four-class cross-subject motor imagery. Mean ± s.d. over five seeds; classical decoders are deterministic.

ModelParamsBal-Acc ↑Cohen's κ ↑Weighted F1 ↑
BP-GBDT–0.3050.0730.190
FBCov-TS-Lin–0.3170.0890.199
ERP-Lin–0.3320.1090.331
BIOT3.2M0.428 ± 0.0260.237 ± 0.0350.404 ± 0.034
CBraMod4.9M0.443 ± 0.0360.257 ± 0.0480.389 ± 0.061
LaBraM5.8M0.560 ± 0.0080.413 ± 0.0100.547 ± 0.010
EEGPT25.3M0.499 ± 0.0140.331 ± 0.0190.474 ± 0.018
ST-EEGFormer-S25.4M0.561 ± 0.0130.415 ± 0.0170.555 ± 0.013
REVE-Base69.2M0.592 ± 0.0180.456 ± 0.0240.582 ± 0.020
CodeBrain13.5M0.399 ± 0.0180.199 ± 0.0250.342 ± 0.024
NeurDuo (Small)4.7M0.559 ± 0.0080.412 ± 0.0110.552 ± 0.010
NeurDuo (Base)24.3M0.565 ± 0.0140.420 ± 0.0190.557 ± 0.016
NeurDuo (Large)67.8M0.583 ± 0.0230.444 ± 0.0300.574 ± 0.027

NeurDuo does not outperform the strongest baseline on this task, but achieves comparable mean performance. NeurDuo-Large reaches a balanced accuracy of 0.583 ± 0.023, compared with 0.592 ± 0.018 for REVE-Base. Notably, we further conducted component ablations on NeurDuo-Small under the same evaluation protocol, as reported in Table R7.

Table R7 · NeurDuo-Small ablations on BCI IV-2a. Mean ± s.d. over five seeds, using the same training and evaluation protocol for all variants.

VariantBal-Acc ↑Cohen's κ ↑
Full model0.559 ± 0.0080.412 ± 0.011
w/o memory retrieval0.574 ± 0.0210.432 ± 0.028
w/o slow stream & memory0.544 ± 0.0240.392 ± 0.032
w/o slow stream0.545 ± 0.0140.393 ± 0.019
w/o channel attention0.544 ± 0.0460.392 ± 0.061
Mamba-only0.533 ± 0.0490.378 ± 0.065
Reviewer_dfx5OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NeurDuo-EEG, a causal EEG foundation model designed for continuous long-sequence modeling. It combines channel-specific Mamba streams, multi-timescale recurrent states, and explicit memory retrieval to capture EEG dynamics across different timescales. Pre-trained on 3,955 hours of EEG from 17 datasets, NeurDuo achieves the best performance on four of five downstream benchmarks while supporting efficient streaming inference with bounded memory.

Critical strengths
  1. The paper aims to address an important limitation of EEG foundation models, namely their reliance on independently processed short windows, and explores a promising direction for continuous long-sequence EEG modeling.

  2. NeurDuo integrates channel-specific processing, fast and slow recurrent streams, and explicit memory retrieval into a unified architecture, enabling multi-timescale modeling with efficient streaming inference.

  3. The experiments demonstrate competitive performance across several EEG tasks, particularly seizure detection. The authors also provide extensive architectural ablations, pre-training analyses, and streaming efficiency evaluations.

Critical weaknesses
  1. Although the paper separately demonstrates the benefits of pre-training and persistent state, it remains unclear how much of the long-sequence improvement comes from transferable pre-trained representations versus downstream sequence modeling and fine-tuning.

  2. The evidence for reliable long-term modeling remains insufficient. While the paper evaluates context lengths up to 60 minutes, it does not thoroughly investigate prediction stability or recurrent-state drift during uninterrupted streaming.

  3. The contribution of explicit memory to long-range dependency modeling is not fully established. Pre-training uses only 32-second sequences, and the explicit memory bank covers approximately 16 seconds, leaving longer-term information primarily dependent on implicit recurrent states.

  4. The baseline comparison omits relevant recent EEG foundation models, such as CodeBrain (ICLR 2026), which also investigates multi-scale and long-range EEG modeling. This limits the completeness of the comparison with recent architectures.

  5. The downstream evaluation covers only five datasets, with limited representation of EEG paradigms such as motor imagery.

Questions for authors

See weaknesses.

10 comments

We thank the reviewer for the careful reading and constructive comments. We respond to each point below.

[W1] How much of the long-sequence gain comes from pre-training, and how much from downstream fine-tuning?

We thank the reviewer for raising this question. To assess the contribution of pre-training to the long-sequence gain, we use a 2 × 2 factorial design on CHB-MIT (15 channels, NeurDuo-Small):

  • Backbone initialisation: pre-trained vs. random.
  • Backbone state across windows: carried along the recording vs. reset before each window. Resetting removes the backbone’s cross-window memory while preserving the downstream sequence head’s access to neighbouring windows.

We keep the downstream sequence head (20 consecutive 30 s windows), fine-tuning recipe, random seed, and five cross-subject folds the same. For reference, we also include a pre-trained single-window model that classifies each window independently.

Table R1 · Decomposition on CHB-MIT, 15 channels (NeurDuo-Small, mean ± s.d. over 5 folds).

Backbone initialisationBackbone state across windowsDownstream sequence headAUC-PR ↑FP·h⁻¹ ↓
RandomReset20 windows0.252 ± 0.1365.81
RandomCarried20 windows0.264 ± 0.1136.65
Pre-trained–Single window0.324 ± 0.0952.02
Pre-trainedReset20 windows0.307 ± 0.1915.15
Pre-trainedCarried20 windows0.413 ± 0.1740.96

The table supports three observations:

  • Pre-training provides a substantial benefit under the same downstream training setup. With carried backbone states, pre-training improves AUC-PR from 0.264 to 0.413 (+0.149) and reduces false alarms from 6.65 to 0.96 per hour. The AUC-PR improvement holds across all five folds.
  • The downstream sequence head alone does not improve on pre-trained window features. The pre-trained single-window model achieves an AUC-PR of 0.324. Adding the sequence head with reset backbone states yields 0.307, whereas carrying the backbone state raises AUC-PR to 0.413.
  • The benefit of carrying state is much larger with pre-training. Carrying the state improves AUC-PR by 0.106 with pre-training, compared with only 0.012 with random initialisation.

Together, these results show that pre-training contributes substantially beyond downstream sequence modelling and fine-tuning alone, and suggest that it enables more effective use of persistent backbone state. We will add Table R1 to the ablation section of the revised paper.

[W2] Is prediction stable over long, uninterrupted streaming?

We thank the reviewer for raising this point. The concern is that a recurrent state advanced without re-initialisation could drift, so that predictions degrade the longer the model runs. We tested this directly on CHB-MIT (15 channels, NeurDuo-Small, five cross-subject folds).

Setup: We use NeurDuo-Small models re-trained with our current code, with the same folds and seed as Table 3 (one model per fold). For each of the 24 test patients, we streamed all of the patient's recordings back to back through a single state that is never reset. This gives 980 h of signal in total, and up to 156 h without interruption for one patient. We compare it with an evaluation that restarts the state every 10 min, as in the paper; with the re-trained models, this gives an AUC-PR of 0.417 ± 0.108, close to the 0.413 ± 0.174 of Table 3. The input of every 30 s unit is identical in the two runs; only the state differs. We measure three things:

  1. prediction quality as a function of elapsed stream time;
  2. the magnitude of the recurrent state over time;
  3. how quickly the model forgets its initial state.

Table R2 · Uninterrupted streaming vs. state reset every 10 min, by elapsed stream time. The false-alarm ratio uses the same threshold for both runs: the 50 % sensitivity point of the reset run in each fold.

Elapsed stream timeSignal (h)AUC-PR, reset every 10 minAUC-PR, uninterruptedFalse alarms, uninterrupted / reset
0–1 h240.3200.3191.05
1–2 h240.4910.5701.25
2–4 h480.2940.3111.03
4–8 h960.2320.1990.96
8–16 h1920.3250.3301.00
16–32 h3070.4970.4961.01
32–64 h1870.4520.4861.10
64–156 h1020.5430.5561.02
All (980 h)9800.417 ± 0.1080.426 ± 0.1061.03

Table R3 · Magnitude of the recurrent state during uninterrupted streaming. Each state norm is divided by the patient's first-hour mean. Entries are the median over the 24 patients, with the range over patients in brackets.

Elapsed stream timeFast stateSlow stateMemory bank
0–1 h1.0001.0001.000
1–2 h0.998 [0.983, 1.007]1.012 [0.983, 1.040]1.001 [0.985, 1.037]
2–4 h0.999 [0.989, 1.011]1.010 [0.965, 1.033]1.000 [0.977, 1.028]
4–8 h0.999 [0.986, 1.021]1.009 [0.998, 1.045]1.003 [0.994, 1.029]
8–16 h1.000 [0.978, 1.014]1.011 [0.997, 1.044]1.001 [0.983, 1.031]
16–32 h1.001 [0.985, 1.014]1.012 [0.991, 1.044]1.003 [0.985, 1.016]
32–64 h1.003 [0.984, 1.012]1.012 [0.965, 1.022]1.000 [0.991, 1.014]
64–156 h1.003 [0.980, 1.006]1.012 [1.006, 1.014]0.999 [0.993, 1.019]

Results:

  1. Prediction quality does not degrade with stream length. Across all 980 h, the uninterrupted stream matches the reset evaluation: AUC-PR 0.426 vs 0.417, and false alarms within 3 % at the same threshold. The difference between the two runs does not grow with elapsed time: it is as small after 64–156 h as in the first hour. The variation across rows comes from which recordings fall into each time bin, and it is shared by both runs. Per unit, the two runs give nearly identical seizure probabilities (median absolute difference 1.6 × 10⁻⁴).
  2. The state remains bounded (Table R3). The norms of the fast state, the slow state and the memory bank stay within 1.2 % (median over patients) of their first-hour level at every elapsed time up to 156 h. The largest transient excursion of any patient is 1.23× the first-hour level, and there is no upward trend.
  3. The initial state is forgotten within minutes. We streamed 1 h of a different patient before each test patient and compared the result with a stream started from a zero state.
    • The relative gap between the two representations falls below 1 % after 7.5 min (median over the 24 patients; at most 18.5 min) and below 0.1 % after 30 min (at most 48 min).
    • After 30 min, the seizure probabilities of the two runs differ by less than 0.01.
    • The state therefore does not accumulate the effect of past inputs indefinitely. It retains information on the scale of minutes.

Together, these results show that NeurDuo can run without re-initialisation for days: we observe no evidence of progressive state growth or systematic prediction degradation, and its predictions remain as accurate as with periodic resets. We will add this analysis, with the state-norm and forgetting curves, to the streaming appendix.

[W3] What supports information retention beyond the memory-bank and pre-training horizons?

We thank the reviewer for this point. We clarify that the 16 s memory bank provides short-term retrieval, while longer-term information is carried by persistent recurrent states, including the slow stream driven by memory tokens. Below, we explain this design and show that such information is retained and used.

1. How the memory is designed to work. The design prioritises recent EEG context while retaining a compressed influence of more distant history. Motivated by the expectation that nearby activity is generally more relevant to the current prediction, we combine direct retrieval of recent information with recurrent integration of older summaries, keeping storage and per-step computation bounded. The memory has two roles:

  • A bounded bank. Every 2 s, a learned writer compresses recent fast states into a memory token. The bank retains the latest 8 tokens (16 s) for content-addressable retrieval.
  • Longer-term integration through the slow state. The same tokens are the only input to the slow stream, whose recurrent state integrates these summaries over time, allowing information to persist after the corresponding tokens are evicted.

The 16 s limit therefore applies to the bank, rather than the slow state. During pre-training, each 32 s sequence produces 16 writes into an 8-token bank, so the slow state continues to update as older tokens are evicted. During downstream fine-tuning, this integration extends over longer sequences, for example, 10 min on CHB-MIT. We will clarify this distinction in the revision.

2. Evidence that information beyond the bank influences predictions. We tested whether changing past EEG affects a later prediction through the backbone state. We used the NeurDuo-Small models of our response to W2 on CHB-MIT (15 channels): re-trained with our current code, with the same folds and seed as Table 3, one model per fold, and no further training.

For each 10 min test window, we evaluated the final 30 s unit. We created a perturbed copy by replacing an earlier 30 s span, leaving the evaluated unit’s EEG unchanged. The replacement span ended lag seconds before the start of the evaluated unit. We tested two replacement conditions:

  • Planted seizure: in 2,851 seizure-free windows from 20 patients, the span was replaced with a seizure segment from another recording of the same patient.
  • Background EEG control: in 3,311 windows from 24 patients, the span was replaced with non-seizure EEG.

After replacement, we reran the backbone through the remainder of the window. The downstream context head received the original embeddings of all earlier units and the recomputed embedding of the final unit. Because the final unit’s EEG was unchanged, any prediction difference could arise only through the altered backbone state. We measured the paired change in seizure probability and the relative change in the slow-stream output, with confidence intervals obtained from 2,000 patient-level bootstrap resamples.

Table R4 · Effect of a perturbation lag seconds before the scored unit (CHB-MIT, 15 channels). "Rise" is the paired change in the scored unit's seizure probability, in percentage points; the mean before the change is 4.47 % (planted-seizure windows) and 4.72 % (background windows). The slow-stream change is the relative change of the slow-stream output at the end of the window (median over windows).

LagSpan outside the bankRise, planted seizure (pp) [95% CI]Rise, background EEG (pp) [95% CI]Slow-stream change, planted seizureSlow-stream change, background EEG
0 sno+5.07 [2.79, 7.73]−0.02 [−0.10, 0.06]43.5 %3.5 %
4 sno+3.08 [1.62, 4.72]−0.01 [−0.09, 0.06]39.4 %3.2 %
16 spartly+0.75 [0.34, 1.26]−0.01 [−0.09, 0.06]24.0 %2.4 %
32 syes+0.23 [0.05, 0.45]−0.02 [−0.06, 0.01]13.7 %1.8 %
60 syes+0.07 [0.01, 0.15]−0.01 [−0.03, 0.02]7.1 %1.1 %
120 syes+0.02 [−0.00, 0.06]0.00 [−0.01, 0.01]3.7 %0.60 %
300 syes0.00 [−0.01, 0.01]0.00 [−0.00, 0.01]1.3 %0.16 %
540 syes0.00 [−0.01, 0.00]0.00 [−0.00, 0.00]0.69 %0.09 %

At 16 s the span has left the bank, but the oldest token in the bank was written while the span was still being read, so it overlaps the span by 2 s. From 32 s on, no stored token has seen the span.

Results:

  1. Information older than the bank, and older than the pre-training sequences, reaches later predictions. At lags of 32 s and 60 s the span lies wholly outside the bank, and it ends 62 s and 90 s before the end of the scored unit, beyond the 32 s pre-training length. A planted seizure still raises the scored unit's seizure probability, with confidence intervals that exclude zero. Background EEG spliced in the same way has no effect at any lag, so the effect is not caused by the splice.
  2. There is no cliff at the bank boundary. The effect decays smoothly with lag (+0.75 → +0.23 → +0.07 pp from 16 s to 60 s; slow-stream change 24 % → 14 % → 7 %). Information is not lost when its token leaves the bank; it is carried on by the recurrent states.
  3. The trace stays in the state for minutes. The slow-stream output still differs from the original at 540 s: 0.69 % [0.36, 1.30] after a planted seizure against 0.09 % [0.07, 0.14] for background EEG. Its effect on the prediction falls below detection from 120 s on. This agrees with the forgetting analysis in our response to W2, where the influence of an initial state fades over minutes.

The results show that information older than both the bank’s span and the pre-training sequence length is retained in NeurDuo’s persistent state and influences later predictions.

3. Revision.

  • We will reframe the contribution: the persistent, causal state is the principal mechanism behind the long-sequence gain. The explicit memory is presented as bounded, content-addressable retrieval of the last 16 s that also drives the slow stream; we no longer claim that it alone explains the long-sequence gain.
  • We will add Table R4 and its protocol to the appendix.
  • We will state that the 32 s pre-training length does not cap retention: information from 62–90 s before the end of the scored unit still reaches the prediction (Table R4). Longer pre-training sequences and a bank whose span adapts to the task remain directions in the Outlook.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for the Area Chair

Dear Area Chair,

Thank you for overseeing the review process. As the discussion period draws to a close, we would like to briefly summarise our paper’s main contribution and the additional evidence we provided during the rebuttal.

Our main contribution is NeurDuo-EEG, a causal foundation model for long-sequence EEG with persistent state and explicit memory. NeurDuo carries a fixed-size, channel-resolved recurrent state across windows as new EEG arrives. An explicit memory bank enables selective retrieval of recent EEG summaries, and a slower recurrent stream integrates those summaries over longer timescales. We pre-trained NeurDuo on 3,955 hours of EEG using autoregressive spectral-code prediction and evaluated it across both short-window and long-sequence tasks.

During the rebuttal, we conducted additional experiments addressing three questions raised by the reviewers:

  • How do the memory components affect predictions? We ran historical EEG perturbation tests on CHB-MIT and Sleep-EDF and found that explicit retrieval contributed at short lags on both datasets. On CHB-MIT, a positive effect on seizure predictions was detectable at a 16-second lag, and the estimated effect was close to zero from 32 seconds onward. On Sleep-EDF, a small effect on sleep-stage predictions remained detectable at a nine-minute lag and depended on the slow recurrent stream. These results suggest that NeurDuo’s memory system can integrate past EEG over different timescales depending on the downstream task and dataset.
  • How does the model perform during long streams? We replayed 980 hours of CHB-MIT recordings, including up to 156 hours of signal in a single stream, and observed stable prediction performance and bounded recurrent-state norms. We also measured full-pipeline latency against cached window encoders at the same 30-second decision cadence. For seizure detection, we selected thresholds exclusively on validation data and reported false alarms alongside event sensitivity and detection delay.
  • How robust are the benchmark results, and what does pre-training add? We added CodeBrain, size-matched comparisons, and paired fold-level analyses. NeurDuo-Large led the strongest baseline on every CHB-MIT fold at both montages, and NeurDuo-Small led all evaluated pre-trained baselines on 15-channel CHB-MIT in each of three runs. LaBraM led on Sleep-EDF macro-F1. On the new cross-subject motor-imagery benchmark, NeurDuo-Large was close to REVE-Base (0.583 vs 0.592 balanced accuracy) using about one-fifteenth as much pre-training EEG. With all models trained from scratch under the same protocol, NeurDuo-Base and NeurDuo-Small ranked first and second, providing evidence of an architectural advantage. With the architecture and downstream training protocol fixed, pre-training improved NeurDuo-Small’s mean performance on all six tasks.

We appreciate the reviewers’ questions and follow-up discussion. We will incorporate these analyses into the revision and clarify the roles of the memory components and the timescales supported by each task. Thank you again for your consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.