- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how pretrained EEG models can learn from new datasets over time in a federated setting, without revisiting earlier training data. BrainNEST compares summaries of multiscale channel and frequency-band structure across clients to decide which updates to share. Updates with broad support are merged into the shared backbone, while others are stored as parameter offsets for later reuse. The authors evaluate BrainNEST on 14 datasets from seven BCI task families over four stages, using CBraMod and CodeBrain. Experiments include federated and continual learning baselines, ablations, and transfer by fine-tuning the final backbone on two held-out datasets.
Critical strengths
-
The paper addresses a practical challenge: adapting pretrained EEG models across heterogeneous datasets without revisiting earlier training data. Local prediction heads accommodate different label spaces, while the federated continual setting combines collaboration across datasets with retention across stages.
-
BrainNEST consistently improves adaptation and reduces forgetting compared with ReFed under matched communication and training budgets. These gains hold across both backbones, and the ablations support the contribution of the main components.
Critical weaknesses
-
The analysis bounds aggregation weights, but larger clients take more optimizer steps under the one-epoch schedule with a fixed batch size. The resulting differences in local updates are neither covered by the bound nor isolated experimentally, leaving their effect on consolidation unclear.
-
The removals in Table 4 also change regularization, update filtering, or parameter reuse. The performance drops show that these modules help, but do not distinguish the benefit of structural matching from these other effects.
Questions for authors
1.How was Re-Fed’s replay mechanism adapted after clients left and were no longer retrained? Which historical samples, if any, remained available for replay?
2.When computing , was each historical dataset’s head paired with the current shared backbone, a personalized model retrieved from memory, or a saved checkpoint? Which memory state was used, and how was this handled across methods?
3.Were the comparisons and ablations in Tables 3–4 repeated with independent seeds beyond Seed 42? If so, how much did the AAA and forgetting gaps between BrainNEST and the strongest baselines vary?
4 comments
Response to Reviewer tbNw (2/3)
W2. Removal ablations confound structural matching with other mechanisms.
The last two rows of Table R3 address your concern by retaining equal-capacity parameter memory, personalization, and the Eq. (14) objective form. Evidence-guided consolidation and routing yield +0.57 BA points, +1.87 AAA points, and −0.73 Fgt points.
This comparison measures the joint contribution of consolidation and routing, including how the policy shapes memory contents and alignment references. Table R13 quantifies the role of each component. Removing Family reduces mean AAA by 5.42/5.43 points, while removing Routed reduces it by 3.19/3.21 points across the two backbones. The matched controls and component ablations together show that evidence-guided knowledge organization contributes gains beyond memory capacity, personalization, and objective form.
Table R3. Memory, personalization, and local-objective controls.
| Method | Matched memory | Personalization | Eq. (14) | BA ↑ | AAA ↑ | Fgt ↓ |
|---|---|---|---|---|---|---|
| FedAvg | No | Yes | No | 0.5737 | 0.5169 | 0.0594 |
| FedAvg without personalization | No | No | No | 0.5518 | 0.5014 | 0.0697 |
| FedAvg + matched memory, without personalization | Yes | No | No | 0.5846 | 0.5279 | 0.0523 |
| FedAvg + matched memory | Yes | Yes | No | 0.5931 | 0.5412 | 0.0468 |
| FedAvg + matched memory + local objective | Yes | Yes | Yes | 0.5972 | 0.5524 | 0.0426 |
| BrainNEST | Yes | Yes | Yes | 0.6029 | 0.5711 | 0.0353 |
BA is the equal-dataset macro-average across the four independent stages. AAA/Fgt use CBraMod. The last two rows retain equal-capacity parameter memory, stage-end personalization, and the Eq. (14) objective form. Their consolidation and routing policies differ.
Table R13. Repeated component ablations.
| Method | CBraMod AAA ↑ | CBraMod Fgt ↓ | CodeBrain AAA ↑ | CodeBrain Fgt ↓ |
|---|---|---|---|---|
| BrainNEST (Full) | 0.5738 ± 0.0068 | 0.0329 ± 0.0039 | 0.5867 ± 0.0074 | 0.0351 ± 0.0042 |
| w/o Family | 0.5196 ± 0.0124 | 0.0457 ± 0.0076 | 0.5324 ± 0.0131 | 0.0483 ± 0.0081 |
| w/o Sparse | 0.5372 ± 0.0103 | 0.0412 ± 0.0065 | 0.5498 ± 0.0112 | 0.0436 ± 0.0069 |
| w/o Gate | 0.5485 ± 0.0091 | 0.0478 ± 0.0052 | 0.5613 ± 0.0097 | 0.0374 ± 0.0056 |
| w/o Routed | 0.5419 ± 0.0117 | 0.0435 ± 0.0071 | 0.5546 ± 0.0123 | 0.0459 ± 0.0075 |
| w/o Feedback | 0.5593 ± 0.0096 | 0.0394 ± 0.0059 | 0.5721 ± 0.0104 | 0.0418 ± 0.0063 |
Mean ± sample SD over training seeds 42, 52, 2025, 2222, and 99. Full reuses the same runs as Table R6. These are component-removal experiments.
Q1. Historical-sample access in the ReFed comparison.
For the client-exit case you ask about, the ReFed* reference retains selected historical training samples for replay after their source clients leave. Only active clients optimize, with access to these retained samples. Re-Fed [1] uses sample caching and replay, so this comparison includes additional historical training-data access.
BrainNEST follows strict client exit and retains knowledge through parameter offsets and retrieval information. Historical test partitions are used exclusively for evaluation. ReFed* therefore serves as a replay-enabled reference. FedWeIT and the matched-memory control provide comparisons under the historical-training-data restriction.
[1] Towards Efficient Replay in Federated Incremental Learning.
Response to Reviewer tbNw (3/3)
Q2. Backbone, memory, and prediction-head state for historical evaluation.
For the historical evaluation you ask about, uses the current stage-end shared backbone and current routed memory, paired with dataset ’s frozen prediction head:
\sum_{k\in\mathcal A_{t,j}}\alpha_{t,jk}\Xi_k^t, \qquad R_{t,j}=\operatorname{BA}\!\left(\mathcal D_j^{\mathrm{test}}, h_j^{\mathrm{frz}}\circ f_{\Theta_{t,j}}\right).$$ Evaluation reads the state available at stage $t$, using the frozen training-time query and acquisition descriptor. Predictions are forward-only, and the current bank determines which memory entries are available. Consensus memory serves training-time alignment. All methods use the same stage-end timing, historical test splits, and task identity. Shared-model baselines use their current backbone with frozen historical heads. FedWeIT uses its retained task-specific parameters, and BrainNEST uses its current routed readout. This evaluates retention in each complete system. The unseen fine-tuning in Table 5 starts from BrainNEST’s final shared backbone alone and evaluates the quality of that initialization. --- > #### **Q3. Variation across independent training runs and ablations.** ${\color{green}\mathbf{Response.}}$ Yes. Tables R6 and R13 report **five independent training seeds**, including seed 42 and four additional seeds. BrainNEST’s mean AAA gains over FedWeIT are **3.89/4.93 points** on CBraMod/CodeBrain, and its mean Fgt reductions are **1.67/1.52 points**. Against ReFed*, the corresponding gains are **2.51/2.55 AAA points** and **0.96/1.10 Fgt points**. For BrainNEST, AAA SDs are **0.0068/0.0074**, compared with **0.0132/0.0141** for FedWeIT and **0.0108/0.0107** for ReFed*. The full ablation statistics are repeated below so you can compare each component’s effect with the observed variation. All ± values denote per-method sample SDs. Reported gains are differences between method means. **Table R6. Repeated continual evaluation.** | Method | CBraMod AAA ↑ | CBraMod Fgt ↓ | CodeBrain AAA ↑ | CodeBrain Fgt ↓ | | --- | --- | --- | --- | --- | | FedWeIT | 0.5349 ± 0.0132 | 0.0496 ± 0.0071 | 0.5374 ± 0.0141 | 0.0503 ± 0.0079 | | ReFed* | 0.5487 ± 0.0108 | 0.0425 ± 0.0052 | 0.5612 ± 0.0107 | 0.0461 ± 0.0064 | | BrainNEST (Ours) | 0.5738 ± 0.0068 | 0.0329 ± 0.0039 | 0.5867 ± 0.0074 | 0.0351 ± 0.0042 | Mean ± sample SD over training seeds 42, 52, 2025, 2222, and 99. ReFed* permits historical training-sample replay and is a separate reference. **Table R13. Repeated component ablations.** | Method | CBraMod AAA ↑ | CBraMod Fgt ↓ | CodeBrain AAA ↑ | CodeBrain Fgt ↓ | | --- | --- | --- | --- | --- | | BrainNEST (Full) | 0.5738 ± 0.0068 | 0.0329 ± 0.0039 | 0.5867 ± 0.0074 | 0.0351 ± 0.0042 | | w/o Family | 0.5196 ± 0.0124 | 0.0457 ± 0.0076 | 0.5324 ± 0.0131 | 0.0483 ± 0.0081 | | w/o Sparse | 0.5372 ± 0.0103 | 0.0412 ± 0.0065 | 0.5498 ± 0.0112 | 0.0436 ± 0.0069 | | w/o Gate | 0.5485 ± 0.0091 | 0.0478 ± 0.0052 | 0.5613 ± 0.0097 | 0.0374 ± 0.0056 | | w/o Routed | 0.5419 ± 0.0117 | 0.0435 ± 0.0071 | 0.5546 ± 0.0123 | 0.0459 ± 0.0075 | | w/o Feedback | 0.5593 ± 0.0096 | 0.0394 ± 0.0059 | 0.5721 ± 0.0104 | 0.0418 ± 0.0063 | Mean ± sample SD over training seeds 42, 52, 2025, 2222, and 99. Full reuses the same runs as Table R6. These are component-removal experiments. ---Dear Reviewer tbNw,
Thank you for reading our paper so carefully and for your thoughtful questions. We sincerely apologize for the delay in posting our response. The additional experiments on client dataset sizes, the contribution of structural matching, and variability across training runs took longer than we anticipated. We also clarify which historical samples were available for replay and how we evaluated earlier tasks after their clients left. We appreciate your patience and would welcome any further questions or discussion.
Best regards,
The authors
Response to Reviewer tbNw (1/3)
Thank you for your time and detailed suggestions on the analysis and experimental protocol. Below, we address your concerns about data-size imbalance, mechanism attribution, and evaluation with controlled results and precise definitions.
Response. Your question separates server-side weighting from local optimization. Proposition C.3 bounds αiG, while the full contribution also contains the exposed local update ϱi→G,ℓΔi(ℓ) (main-text Eq. (8)). Local data and optimization determine that update’s magnitude and direction.
Tables R11–R12 directly control the number of local updates. Every client receives 45 steps per round in Stage 1 or 47 in Stage 3. Evidence-balanced weighting achieves higher macro BA than both alternatives: 0.5560/0.4609, compared with 0.5418/0.4452 for uniform weighting and 0.5310/0.4334 for sample weighting. The advantage therefore persists with equal local update counts. These controls complement the coefficient bound by testing the optimization effect you identify.
Table R11. Stage 1 aggregation controls.
Every client receives the same 45-step budget in the fixed-step rows. One step denotes one optimizer update. This control fixes update counts per client. All dataset entries are BA.
Table R12. Stage 3 aggregation controls.
Every client receives the same 47-step budget in the fixed-step rows. One step denotes one optimizer update. This control fixes update counts per client. All dataset entries are BA.