acceptodds
Under review as a conference paper at ICLR 2027

Selective Coarse-Layer Discrete Distillation for Continual EEG Pretraining

Abstract

Pretrained EEG models may be adapted continually as data arrive from heterogeneous domains. For models that use discrete tokenizers, this adaptation risks both catastrophic forgetting and drift in the token assignments consumed by downstream models. We propose CAP-CL (Coarse Assignment Preservation for Continual Learning), which exploits the hierarchy of residual vector quantization to regularize coarse assignments while allowing finer representations to adapt. CAP-CL distills coarse-layer (L0) assignments from the previous-stage tokenizer on current and replay data, with replay capped separately for each domain and finer layers exempt from the distillation loss. On a stream of 21 EEG datasets, using an autoregressive readout trained with replay on hard token indices, CAP-CL achieves a mean final balanced accuracy of 0.421. It outperforms the tested replay and distillation controls in all five paired seeds, across three fixed stage orders, and under matched coarse-layer loss weights. Preserving assignments between successive stages has different effects from anchoring them to the initial tokenizer: a fixed initial teacher reduces cumulative coarse-layer flips from 0.876 to 0.568 but lowers final accuracy to 0.402. These results support coarse-only discrete distillation as an effective regularizer for continual EEG pretraining with an adaptive readout and show that stronger long-term index preservation alone does not explain downstream performance.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Continual pretraining, EEG representation learning, Vector quantization, Motor Imagery BCI, Assignment drift

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_ypoFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Some EEG foundation models first convert brain signals into discrete codes with a tokenizer, then train a larger model on those codes. This paper studies how to update such a model as new EEG datasets arrive over time, without retraining from scratch. The challenge is that updating the tokenizer can change the code assigned to the same signal, which shifts the “vocabulary” the downstream model depends on. The proposed method, CAP-CL, keeps a small memory of past data and encourages only the coarsest level of codes to stay consistent with the previous version of the tokenizer, while the finer levels remain free to adapt. On a sequence of 21 public EEG datasets, CAP-CL reaches about 0.42 balanced accuracy, versus roughly 0.40–0.41 for replay and the other continual-learning baselines. The ranking holds across seeds and dataset orderings. The authors also find that forcing the codes to match the original tokenizer throughout the sequence preserves more of the vocabulary but lowers accuracy. They conclude that stability between consecutive updates is helpful, but strict long-term preservation is not.

Critical strengths
  • Well-defined problem. It identifies a failure mode specific to continual learning with discrete tokenizers: the same signal can receive a different code after an update. Standard continual-learning work, which focuses on continuous representations or parameters, does not address this.

  • Simple, cheap method. CAP-CL adds only a distillation term on the coarse codes, with no new trainable modules, and works on top of standard replay. It would be easy to adopt in other residual-VQ pipelines.

  • Careful ablations. The paper separates which layers are constrained (coarse only vs. all layers vs. fine only) from which teacher is used (previous stage vs. initial tokenizer). It also tests the layer choice with the total loss weight held fixed, which rules out a simple "more regularization" explanation.

  • Useful negative finding. Showing that stronger long-term code preservation lowers accuracy is informative. It challenges the assumption that interface stability is what matters.

  • Consistent results. The method wins in all five paired seeds against every baseline, with paired confidence intervals reported. The ranking also holds across three dataset orderings, two of which were built deliberately to be smooth and abrupt.

  • Heterogeneous benchmark. The stream of 21 public datasets varies widely in montage, sampling rate, number of classes and paradigm. Combined with the planned release of the registry, preprocessing pipeline and stage orders, it could be a useful resource for the community.

  • Transparent about limitations. The authors openly state the within-file split, the single architecture, and the fact that they cannot fully explain why the regularizer helps. Held-out datasets and hyperparameter-selection data are kept carefully separate from the evaluation data.

Critical weaknesses
  • Small gains. CAP-CL improves Final accuracy by about 2 points over replay alone and by only 1–2 points over the other baselines (1.1 over Feat-KD@Θ0, 1.8 over DER++). The confidence intervals capture only seed variance under a fixed split and order, not variability across subjects, splits or datasets, which in EEG is typically much larger.

  • Unexplained mechanism. The method is motivated by preserving code assignments, yet the paper shows that stronger preservation lowers accuracy. The authors admit they do not know why the regularizer helps, so the main claim rests on an empirical gain without a convincing explanation.

  • Motivation and protocol are at odds. The stated goal is a stable interface for downstream consumers, but the main evaluation retrains the readout after every update, which largely removes the drift problem. The frozen-consumer experiment, which directly tests the motivation, is presented only as a secondary diagnostic. In addition, most codes change anyway (L0 flip ≈ 0.88, L2 ≈ 0.99 relative to the initial tokenizer).

  • Little forgetting to fight. With replay and head rehearsal in a task-incremental setting, BWT is close to zero or positive for all replay-based methods. The benchmark therefore barely exercises the continual-learning problem the method is meant to solve.

  • Weak case for continual training. Joint retraining is very cheap at this scale (8,400 steps on a single GPU) and still performs best. The replay buffer grows to roughly 20k trials, a large fraction of the training data. The argument that sequential updates avoid costly retraining is not supported in this setting.

  • Very small update budget. Each stage uses 400 steps with batch size 8 (about 3,200 samples), so the tokenizer is only lightly perturbed. It is unclear whether the findings hold under realistic continual pretraining.

  • Missing baselines. There is no model trained from scratch per task (e.g., EEGNet or Riemannian/CSP methods) and no general-purpose time-series foundation model used as a frozen feature extractor. Without these, the value of continual tokenizer pretraining cannot be judged.

  • Unfair or mislabeled comparisons. The EEG foundation models (BIOT, LaBraM, CBraMod) show strongly negative BWT, which suggests they were fine-tuned without replay, yet they appear in the main table. "DER++" is a feature-MSE variant that does not replay logits. Only three methods (CAP-CL, Feat-KD@Θt−1, DER++) received hyperparameter tuning, while LwF, EWC and Feat-KD@Θ0 used fixed coefficients.

  • Limited generality. The study uses a single tokenizer architecture (the authors' reimplementation of BrainRVQ) and a single 64-channel preprocessing pipeline. The within-file splits share subjects and recordings between train and test.

  • Limited novelty. The method combines replay with distillation of the previous model's code assignments, applied to one layer. The closest related work on drift-aware continual tokenization (Feng et al., 2026) is cited but not compared against.

  • Clarity issues. The paper is dense with acronyms and nested controls, which obscures the main message. The method's acronym is expanded differently in the abstract and the introduction, and Appendix I has a title but no content.

Questions for authors
  1. Mechanism. If stronger code preservation lowers accuracy, what do you think CAP-CL actually contributes? Could you provide an analysis that isolates the source of the gain, for example its effect on codebook utilization, feature geometry, or reconstruction quality on past domains?

  2. Interface stability as the primary metric. Since the paper is motivated by stable codes for downstream consumers, why is the main evaluation based on a readout that is retrained after every stage? Could you report the frozen-consumer setting (Table 6) for all baselines and treat it as a primary result?

  3. Absolute drift. Even with CAP-CL, L0 flip relative to the initial tokenizer is about 0.88 and L2 is about 0.99. In what sense is the interface preserved for a consumer that is not retrained?

  4. Forgetting. BWT is near zero or positive for all replay-based methods. Can you show results in a regime where forgetting is substantial, such as a smaller buffer, no head rehearsal, or a class- or domain-incremental setting, to test whether CAP-CL helps where continual learning is actually difficult?

  5. Baselines without pretraining. How does the pipeline compare with models trained from scratch per task (e.g., EEGNet, Riemannian or CSP-based methods) under the same splits? This is needed to judge whether continual tokenizer pretraining adds value.

  6. General-purpose time-series models. How would a frozen general-purpose time-series foundation model (e.g., Chronos, MOMENT) with a linear probe perform on the same tasks?

  7. Foundation-model baselines. How exactly were BIOT, LaBraM and CBraMod trained across the stream? Did they use replay and head rehearsal? If not, could you report them with the same continual-learning protocol so the comparison is fair?

  8. Tuning parity. Why were LwF, EWC and Feat-KD@Θ0 not tuned with the same OFAT protocol as CAP-CL, Feat-KD@Θt−1 and DER++? Given margins of 1–2 points, would equal tuning change the ranking?

  9. DER++ adaptation. The adapted DER++ uses feature MSE rather than logit replay. Could you include a version closer to the original, for example replaying stored code distributions or soft assignments?

  10. Cost argument. Joint-Offline is cheap at this scale and still performs best, while the replay buffer grows to roughly 20k trials. Under what data or compute regime do you expect sequential updates to be clearly preferable? Can you show results with a fixed global memory budget?

  11. Update budget. With 400 steps of batch size 8 per stage, the tokenizer is only lightly updated. Do the conclusions hold with substantially longer updates per stage?

  12. Evaluation variance. The confidence intervals reflect seed variance only. Can you report variability across splits or, ideally, a subject-disjoint evaluation, since within-file splits share subjects and recordings?

  13. Generality. Have you tested CAP-CL on other residual tokenizers (e.g., LaBraM-style VQ or audio codecs) to check that the coarse-only finding is not specific to your BrainRVQ reimplementation?

  14. Related work. How does CAP-CL compare empirically with drift-aware continual tokenization (Feng et al., 2026) or codebook-refresh approaches adapted to this setting?

  15. Missing content. Appendix I ("Seed-level robustness") is empty. Was content omitted? Also, which expansion of the CAP-CL acronym is correct?

2 comments

Response to Reviewer ypoF

Q1 · What does CAP-CL contribute? Table 2 reports Final BAcc of 0.421 for CAP-CL and 0.401 for ER. In the documented objective, setting the index-loss coefficient to zero recovers ER with the same replay and readout schedule. Table 4 further compares coarse-only and all-layer distillation at a common total coefficient, with Final BAcc of 0.421 and 0.405. These comparisons test the addition and layer allocation of the discrete loss. The design rationale is that the coarse assignment defines the partition on which residual codes are organized. Constraining that partition leaves fine layers trainable through reconstruction and quantization.

Table 3 compares teacher references with distillation on current and replay batches. The initial teacher preserves initial coarse assignments better than the previous-stage teacher (flip rates 0.568 and 0.876), while their Final BAcc values are 0.412 and 0.421. These values favor previous-stage supervision for adapted accuracy at the reported coefficient. Figure 3 shows a related pattern for replay-only L0L_0 distillation, with lower adjacent-stage flips on capped replay (c,d) alongside substantial cumulative drift on held-out first-domain trials (a,b).

Q2–3 · Readout and token compatibility. The primary evaluation adapts the readout after each stage to measure the updated pipeline's Task-IL performance under a shared supervised budget. The previous-stage teacher is refreshed at stage boundaries, so the constraint follows the evolving model. Successive local constraints can accumulate changes relative to initialization, as Figure 3 illustrates. The revised manuscript omits the quantitative frozen-consumer results from submitted Table 6.

Q4 · Forgetting. In Task-IL with replay and head rehearsal, Table 2 reports near-zero or positive BWT. BWT measures each method's change from its own score immediately after learning a task. Final BAcc measures the accuracy reached across the stream. CAP-CL and ER report 0.421 and 0.401 Final BAcc with BWT of 0.011 and 0.004. These results describe accuracy under a retention-oriented protocol. Naive-CPT omits replay and head rehearsal and reports BWT of −0.041.

Q7 · EEG foundation-model references. Table 2 presents BIOT, LaBraM, and CBraMod as references with native Task-IL stacks. The available runner uses current-domain labels without replay or head rehearsal. The reference rows are interpreted in light of these protocol differences. The matched continual-pretraining comparisons share LexVQ and its documented replay and readout budgets.

Q8–9 · Coefficients and DER++ adaptation. Appendix F reports the recorded coefficients. LwF, EWC, and initial-teacher Feat-KD use fixed coefficients. The total-weight-matched block fixes the index-loss coefficient budget when varying constrained layers. Adapted DER++ stores waveforms and matches previous-stage pre-quantization features on replay. This instantiates continuous-feature retention during unlabeled tokenizer updates. The comparison with CAP-CL tests the specified feature and discrete assignment objectives at their reported settings.

Q10–11 · Update and data-access setting. Joint-Offline reaches 0.434 Final BAcc compared with 0.421 for CAP-CL. Both use 8,400 tokenizer updates overall, with 400 updates per streaming stage at batch size 8. Joint-Offline pools all 21 capped training subsets. Streaming uses the current domain and stored replay as each domain arrives. This addresses sequential data access. The buffer grows by domain and holds up to 20,480 prior trials by stage 21.

Q12–13 · Evaluation scope. Table 14 reports intervals computed from the five submitted seed-level differences for the primary order. The documented pairing protocol uses the same evaluation split within each seed. Main-21 uses trial-disjoint within-file splits, so subjects and recordings may be shared across training and evaluation. Yang2025 is the corpus-unseen probe, and Jeong2020MI is a same-study-family paradigm-transfer probe. The evaluation uses one residual tokenizer architecture and a shared 64-channel mapping.

Q14 · Tokenizer adaptation. DACT aligns soft first-layer assignments across stages and conditionally reassigns finer codes when the coarse code changes. CAP-CL distills hard coarse indices on current and replay EEG while training fine layers with the base tokenizer objective. The manuscript compares layer scope and teacher reference on a BrainRVQ-derived tokenizer. Feng et al., 2026, §3

Q15 · Appendix and naming. The revised PDF places Table 14 under Appendix I. We use Coarse Assignment Preservation for Continual Learning consistently.

I thank the authors for the response. However, none of the additional analyses I requested were provided: no mechanism analysis (Q1), no regime with substantial forgetting (Q4), no non-pretrained or general time-series baselines (Q5–6), no re-run of the foundation models under the same CL protocol (Q7), no tuning parity (Q8), no fixed global memory budget or longer updates (Q10–11), no subject/session-disjoint evaluation (Q12), no additional tokenizer (Q13), and no empirical comparison with DACT (Q14).

More importantly, the response states that the quantitative frozen-consumer results (submitted Table 6) were removed because the diagnostic configuration fed waveform patches directly to the AR backbone, bypassing the tokenizer. This was the only experiment directly testing the paper’s motivation (a stable interface for consumers that are not retrained), and it was cited as a strength by several reviewers. Without it, the motivation rests only on a protocol where the readout is retrained after every stage, and the remaining evidence (initial-teacher control) indicates that stronger assignment preservation does not improve accuracy.

The paired improvement over ER is consistent across seeds, but it remains a ~2-point gain under a single order, a within-file split, and a single tokenizer, with an unexplained mechanism. I therefore maintain my score.

Reviewer_Srf1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual adaptation of discrete EEG tokenizers, where updating the tokenizer can change the integer assignments consumed by downstream models. The proposed CAP-CL method distills only the coarse residual-quantization layer from a frozen previous-stage teacher, using both current and replay data while leaving finer layers free of direct index-distillation constraints. Evaluation uses a 21-dataset EEG stream with an adapted autoregressive readout. CAP-CL improves final balanced accuracy over replay and the tested distillation controls. The paper also investigates layer selection, teacher reference, lookup tables, and frozen-consumer compatibility, showing that stronger preservation of token identities does not necessarily translate into better downstream performance.

Critical strengths
  1. The paper addresses a well-defined problem in continual learning with discrete representations. The distinction between preserving codebook parameters and preserving actual assignments is useful, and the proposed coarse-only constraint is a simple, interpretable intervention within an otherwise shared architecture.

  2. The main comparisons provide convincing evidence for the regularizer's usefulness within the evaluated setting. In Table 2, CAP-CL reaches 0.421 final balanced accuracy, compared with 0.401 for replay, 0.410 for the strongest reported continuous-feature distillation baseline, and 0.405 for all-layer discrete distillation. Table 14 reports positive paired differences against all seven continual-pretraining baselines across all five seeds in the primary order. Its mean final-accuracy advantage also persists under the two additional stage orders.

  3. The ablations examine the central design choices rather than only reporting an aggregate improvement. In particular, Table 4 compares layer subsets under both a fixed total loss coefficient and a fixed L0 coefficient, strengthening the evidence that the benefit is associated with selective coarse-layer supervision. The teacher-reference and shared-lookup controls further help distinguish alternative explanations.

  4. The analysis of assignment stability is appropriately nuanced. The initial-teacher experiment and the frozen-consumer diagnostic show that minimizing assignment flips alone is not a sufficient objective. Table 6 is particularly helpful: CAP-CL has a smaller immediate frozen-consumer accuracy drop than replay or all-layer distillation, even though all-layer distillation preserves more assignments. This makes the empirical findings more informative than a simple stability-versus-accuracy correlation.

Critical weaknesses
  1. The generalization evidence remains limited by the evaluation protocol. Main-21 uses within-file trial splits, allowing training and evaluation to share subjects and recordings. The authors explicitly acknowledge this, and Yang2025 provides a useful corpus-unseen transfer probe. Nevertheless, the main results do not establish robustness to unseen subjects or sessions, while Jeong2020MI is a same-family paradigm-transfer probe. A subject- or session-disjoint evaluation on a representative subset would substantially strengthen the practical relevance. Architectural generality is also untested beyond the single residual tokenizer and common 64-channel preprocessing contract.

  2. Some baseline comparisons require more careful interpretation. The adapted LwF and DER++ objectives use feature MSE rather than their original logit-based formulations, and some baseline coefficients are fixed rather than selected through the shared tuning procedure. These choices are disclosed, but the results should be interpreted as comparisons with the specified adaptations. There is also an ambiguity concerning Freeze-Lex: Table 8 marks it as having no head rehearsal, whereas Table 7 says it follows the same AR readout and head protocol as the other rows. A replay-matched frozen-tokenizer reference is important for isolating the benefit of tokenizer adaptation. This concern does not invalidate the direct CAP-CL-versus-ER comparison.

  3. Practical scalability is not fully characterized. Replay is capped at 1,024 trials per prior domain, so total storage grows with the stream. The main pipeline also performs supervised readout adaptation and rehearsal after every tokenizer update. These are reasonable experimental choices, but performance under a fixed global memory budget and the additional runtime and memory cost of teacher-based distillation remain unclear.

  4. A few reporting inconsistencies should be corrected. The abstract gives 0.402 for the initial-teacher variant, whereas Section 4.3 and Table 3 report 0.412. Figure 2 labels the cross-entropy as involving q2 and k2 while describing L0 alignment. These appear to be presentation issues, but they concern central experimental and methodological details.

Questions for authors
  1. Does Freeze-Lex use the same prior-task head rehearsal and labeled-data budget as CAP-CL? Please reconcile Tables 7 and 8 and, if necessary, provide a replay-matched frozen-tokenizer result.

  2. Do the improvements over replay and all-layer distillation persist under subject- or session-disjoint splits on a representative subset of the datasets?

  3. How sensitive is CAP-CL to replay capacity, particularly under a fixed global memory budget? Reporting per-stage runtime and peak GPU memory relative to ER would also help assess its practical cost.

  4. Please confirm whether the initial-teacher result is 0.402 or 0.412 and clarify the L0 index notation in Figure 2.

1 comment

Response to Reviewer Srf1

Q1 · Freeze-Lex and baseline protocols. The documented readout protocol specifies 800 supervised updates per stage and batch size 8 for Freeze-Lex, CAP-CL, and ER. Freeze-Lex fits its readout on current-task labels only. CAP-CL and ER sample prior-task batches with probability 0.5 once prior tasks are available. Thus, the shared update count accompanies different sources of labeled examples. Table 7 now clarifies that the AR backbone and heads are marked fixed for Freeze-Lex during the tokenizer phase. They are subsequently adapted during readout training. Table 8 identifies the methods using head rehearsal.

For the CAP-CL versus ER comparison, the documented replay and readout schedules coincide. Setting λcap=0\lambda_{\mathrm{cap}}=0 in the CAP-CL objective recovers ER, providing a direct loss-removal control. Table 2 reports 0.421 Final BAcc for CAP-CL and 0.401 for ER. The Freeze-Lex comparison additionally changes head rehearsal, so its interpretation includes both tokenizer adaptation and rehearsal. LwF and adapted DER++ use the feature-MSE objectives specified in the manuscript, and their results apply to those adaptations.

Q2 · Evaluation split and transfer. Main-21 uses trial-disjoint within-file train and evaluation sets. Evaluation trials are excluded from tokenizer updates, replay, and readout fitting. Subjects and recordings can occur on both sides of a split, so the main comparison assesses held-out trials under the represented recording conditions.

The transfer columns examine two specific settings using fresh linear probes on frozen features. Yang2025 evaluates transfer to an unseen corpus. Jeong2020MI evaluates paradigm transfer within the study family associated with the Jeong2020ME stream stage. These probes remain outside stream updates and tuning. The reported results therefore have a defined scope comprising the trial-split protocol, these transfer settings, one residual tokenizer, and a common 64-channel mapping.

Q3 · Replay and update schedule. The replay buffer stores up to 1,024 trials per prior domain, giving a maximum of 20,480 prior trials entering stage 21. Total replay storage consequently grows with the number of observed domains. CAP-CL and ER use the same replay storage and supervised readout schedule.

CAP-CL additionally retains a frozen previous-stage tokenizer and evaluates it on current and replay batches to generate coarse-index targets. The teacher is refreshed once per stage and receives no gradient updates. This specifies the additional model storage and forward-pass work associated with distillation.

Q4 · Initial teacher and Figure 2. The initial-teacher Final BAcc is 0.412 in Table 3. The revised abstract corrects the submitted value of 0.402. Figure 2 now labels the cross-entropy inputs q(0)q^{(0)} and i(0)i^{(0)}. The superscript denotes the coarse residual layer. The caption identifies the student's L0L_0 logits and the previous-stage teacher's hard L0L_0 index for the same waveform. The teacher index is a fixed target during the student update, consistent with the objective.

Reviewer_kT5UOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies continual pretraining of a residual-VQ EEG tokenizer whose hard token indices feed a downstream autoregressive model. The authors worry about “assignment drift,” where the same waveform gets a different integer after a tokenizer update. They propose CAP-CL, which distills the previous-stage tokenizer’s coarse (L0) assignments on current and replay data, leaving finer layers unconstrained. On a 21-dataset motor imagery/execution stream, CAP-CL reaches 0.421 final balanced accuracy versus 0.401 for replay alone and 0.398–0.410 for the continuous-distillation baselines. It also beats an all-layer discrete control (0.405). A fixed initial teacher cuts cumulative L0 flips but lowers accuracy (0.412), which the authors take to mean long-term index preservation doesn’t explain downstream performance.

Critical strengths
  1. A well-controlled layer ablation. L0-only distillation is compared against L0+L1, all-layer, and fine-only variants under both a total-weight-matched rule (Eq. 4) and an unnormalized rule (Table 4). Adding index loss on finer layers lowers Final accuracy in both (0.421 vs. 0.405 for all-layer), which rules out the simple “L0-only just regularizes less” explanation.
  2. A useful negative finding on the “stable interface” assumption. Freezing codebooks still leaves L0 flip at 0.854, and the initial-teacher variant cuts cumulative flip (0.876 → 0.568) without improving accuracy. Together with the frozen-consumer diagnostic (Table 6), this suggests that minimizing index drift should not be treated as a goal in itself.
Critical weaknesses
  1. The headline gains and the main conceptual claim rest on small margins and uneven tuning. CAP-CL beats replay by about 2 points (0.421 vs. 0.401), while Freeze-Lex already reaches 0.394 and seed std is about 0.01. CAP-CL got an OFAT screen, but several baselines use fixed coefficients (or unshown screens), and the all-layer and initial-teacher variants reuse a λcap tuned for the delayed L0 teacher. The previous-stage vs. initial-teacher gap (0.421 vs. 0.412) is about one std with no paired analysis.
  2. The paper does not establish why L0-only helps, and the evaluation is prone to leakage. In Table 6, CAP-CL has very high L1/L2 flips yet barely degrades the frozen consumer, suggesting the AR readout may ignore fine-layer indices, so the L0-only result could be specific to this readout. All results use within-file trial splits sharing subjects and sessions, and the “held-out” Jeong2020MI is the same cohort as a stream stage. Without a mechanism or a subject/session-disjoint evaluation, generality is unclear.
Questions for authors
  • Were LwF, EWC, Feat-KD@Θ0, the all-layer control, and the Θ0-teacher variant tuned at a budget comparable to CAP-CL’s OFAT screen? Please show the Feat-KD@Θt−1 and DER++ screens.
  • Can you provide paired per-seed differences with 95% CIs for Table 3 (previous-stage vs. initial teacher, and mem+cur vs. mem-only), and seed-level CIs for the progressive and adversarial orders (Tables 12-13)? Also, does the previous-stage advantage hold when λcap is tuned separately for the Θ0 teacher?
  • In Table 6, CAP-CL has very high L1/L2 flips yet barely degrades the frozen consumer. Does the AR readout actually depend on L1/L2 indices (e.g., feed only L0 indices, or randomize L1/L2)?
  • Do you have any subject-disjoint or session-disjoint results, even on a subset of the stream? If not, how do you rule out that the gains come from within-session trial similarity?
  • Why does CAP-CL beat Joint-Offline on both held-out probes while trailing it on Final (0.421 vs. 0.434)? Is this noise in the held-out columns, or does it point to an issue with the joint baseline?
  • The abstract reports 0.402 for the initial-teacher variant, but Table 3 and Section 4.3 report 0.412.
1 comment

Response to Reviewer kT5U

Q1 · Tuning settings and layer controls. The fixed coefficients are 1 for LwF, 10 for EWC, and 20 for Feat-KD@Θ0\Theta_0. Previous-stage Feat-KD and adapted DER++ use recorded coefficients of 1. Appendix F describes the five-stage CAP-CL screen and its old-domain probe criterion. The all-layer and initial-teacher controls use CAP-CL's total index-loss weight of 0.75. Dividing that weight across the selected layers fixes the nominal coefficient budget while testing its allocation. Table 4 reports 0.421 Final BAcc for coarse-only and 0.405 for all-layer distillation under this rule. A second block fixes the coarse-layer coefficient while adding fine-layer terms. Together these controls examine both loss allocation and the effect of adding fine-layer penalties. The comparisons apply at the recorded settings.

Q2 · Teacher, batch scope, and seed-level comparisons. At the common index-loss weight of 0.75, Table 3 reports 0.421 and 0.412 mean Final BAcc for the previous-stage and initial teachers on current and replay batches. Their cumulative coarse flip rates relative to initialization are 0.876 and 0.568. Thus, the configuration with better preservation of initial assignments has lower mean downstream accuracy at this setting. The previous-stage teacher constrains each transition using the preceding model as its reference.

For that teacher, the mean is 0.421 with both batch types and 0.414 with replay only. The total CE coefficient is held fixed by splitting it equally between current and replay batches. This tests which examples receive the constraint at a common coefficient budget. Tables 12 and 13 summarize the alternative orders. For the separate primary-order CAP-CL versus ER comparison, the five seed values listed in submitted Table 14 give a mean difference of 0.020 BAcc and a paired 95% interval of [0.015, 0.025]. The differences are positive in all five listed seeds. Pairing evaluates variation in the method difference, which can differ from the standard deviation of either method alone.

Q3 · Fine-layer indices. Flip rates count changed indices equally, regardless of the change in their embeddings or their influence on predictions. Table 4 measures the effect of applying distillation to different layers during training. Fine layers continue to receive reconstruction and quantization losses throughout. This supports an interpretation in terms of regularizer placement within the evaluated pipeline. The revised manuscript omits the quantitative frozen-consumer results from submitted Table 6.

Q4 · Subject and session overlap. Main-21 holds out evaluation trials from tokenizer updates, replay, and readout fitting. The split is within file, so both sets can include trials from the same subject and recording. The resulting comparison evaluates held-out trials under this protocol. Yang2025 separately probes transfer to an unseen corpus. Jeong2020MI probes paradigm transfer within the study family associated with the Jeong2020ME stream stage. These two probes address different forms of transfer.

Q5 · Joint-Offline and transfer. Final BAcc averages the adapted task-head scores across the 21 stream domains. Joint-Offline scores 0.434 and CAP-CL scores 0.421. Transfer uses fresh linear probes on frozen post-stream features. The representations and task-specific heads are therefore assessed through different outcomes, whose rankings can differ. CAP-CL and Joint-Offline score 0.633 and 0.619 on Yang2025, and 0.224 and 0.193 on Jeong2020MI, respectively. We interpret these as mean differences for the specific probes.

Q6 · Initial-teacher value. The initial-teacher configuration with memory and current data yields 0.412 Final BAcc in Table 3. The revised abstract uses 0.412.

Reviewer_pue3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies tokenizer adaptation for continual pretraining of EEG models built on a residual VQ tokenizer. It proposes CAP-CL, which distills only the coarse-layer (L0) assignments from the previous-stage tokenizer on current and replay data, and leaves L1/L2 free of the index cross-entropy loss. On a 21-dataset EEG stream it reports consistent gains over replay and distillation baselines.

Critical strengths

S1. Index drift when updating a discrete tokenizer is a relevant and under-studied problem for EEG models. S2. Constraining only the coarse layer (with a previous-stage teacher) is a simple and well-motivated stability–plasticity design. S3. The results are generally positive. The gain over replay holds across seeds and stage orders.

Critical weaknesses

W1. The paper does not clearly show that tokenizer adaptation is necessary. A frozen tokenizer is only about 0.007 behind replay alone, within seed noise. W2. The method is hard-label distillation from a snapshot teacher, restricted to one layer. The tokenizer is borrowed from BrainRVQ, and the main contribution is the empirical analysis of scope and temporal reference. Hence, the technical novelty is limited. W3. Lower cumulative drift does not improve accuracy, and CAP-CL barely reduces cumulative L0 flips. This weakens the drift motivation. W4. No existing tokenizer-adaptation method is evaluated. The related work cites some, but the baselines are generic continual-learning methods, mostly as feature-MSE variants. W5. The choice of the datasets needs to be justified. Are the selected datasets challenging? Notably, the performance improvement of the proposed solution on these datasets is marginal.

Questions for authors

Q1. Please give direct evidence for the motivation. Where does a frozen tokenizer fail on new domains, for example in reconstruction or codebook utilization?

Q2. Why does reducing drift not help downstream performance when the readout is not retrained?

Q3. How does CAP-CL differ technically from existing continual-learning and tokenizer-adaptation methods? Please compare against at least one tokenizer-adaptation baseline.

1 comment

W1/Q1 · Frozen tokenizer. Table 2 reports final balanced accuracy of 0.394 for Freeze-Lex and 0.401 for experience replay (ER). In the documented protocol, Freeze-Lex trains on current-task labels only. ER also rehearses prior-task labels when fitting the task heads. Table 7 now states this difference explicitly, since the observed gap combines tokenizer adaptation and head rehearsal. The CAP-CL versus ER comparison has a more focused interpretation. In Eq. 3, setting λcap=0\lambda_{\mathrm{cap}}=0 recovers ER with the same replay and downstream update schedule. Their reported scores of 0.421 and 0.401 assess the additional assignment regularizer within that protocol.

W3/Q2 · Assignment drift and the readout. Figure 3(a,b) measures flips relative to the initial tokenizer on held-out first-domain trials. Figure 3(c,d) measures adjacent-stage flips on capped replay. For the replay-only L0L_0 variant, lower adjacent-stage coarse flip rates on replay coexist with substantial first-domain drift from initialization. Each constraint concerns one transition, so successive updates can accumulate changes over the stream. The teacher control in Table 3 also separates preservation from adapted-pipeline accuracy. With current and replay batches, the initial teacher has lower cumulative L0 flip than the previous-stage teacher (0.568 versus 0.876), while their Final BAcc values are 0.412 and 0.421. Thus, these reported means associate stronger preservation of the initial assignments with lower adapted accuracy. The revised manuscript omits the quantitative results of submitted Table 6. The available diagnostic configuration feeds waveform patches directly to the AR backbone, bypassing the tokenizer output.

W2/W4/Q3 · Method and related work. CAP-CL applies hard cross-entropy targets from the previous-stage teacher to coarse-layer assignments on current and replayed EEG. The design motivation is to anchor the coarse partition on which residual coding is organized while allowing finer layers to adapt through reconstruction and quantization. Refreshing the teacher once per stage makes the reference follow the evolving tokenizer. The empirical contribution examines the scope of this constraint and its temporal reference. Table 4 reports 0.421 for L0-only and 0.405 for all-layer distillation at the same total index-loss weight of 0.75. Its second block holds the coarse coefficient fixed while adding fine-layer penalties. The two rules examine weight allocation across layers and the effect of adding fine-layer constraints at the stated coefficients. DACT aligns soft first-layer assignment distributions across stages and conditionally reassigns finer codes when the first-layer code changes. CAP-CL uses hard targets on current and replay EEG and trains finer layers through the base objective. This specifies its methodological relationship to DACT and the EEG setting in which we evaluate the layer and teacher choices. Feng et al., 2026, §3

W5 · Dataset selection and effect size. Main-21 includes releases with usable event timing and auditable recording metadata. It spans 14–128 native electrodes, 160–2,500 Hz, and 2–11 classes. These acquisition and task differences motivate testing sequential adaptation across domains under a common preprocessing contract. In the locked order, Table 2 reports 0.421 for CAP-CL and 0.401 for ER, a 2.0 percentage-point difference in final balanced accuracy. We interpret this magnitude within the shared Task-IL and within-file trial-split setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.