- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MCD-EEG, a benchmark for EEG-based brainprint recognition across cognitive tasks, acquisition conditions, and populations. The dataset includes EEG recordings from 45 hearing and prelingually deaf participants, covering multiple cognitive tasks and EEG cap removal and reattachment (Re-cap). The authors establish within-task, cross-task, and cross-Re-cap evaluation protocols and assess several existing EEG models through performance comparisons and feature analyses.
Critical strengths
The paper considers multiple sources of distribution shifts in EEG-based identity recognition, including different cognitive tasks, populations, and acquisition changes caused by EEG cap removal and reattachment. Compared with datasets focusing on a single task or acquisition session, this benchmark provides more diverse evaluation scenarios and facilitates research on the robustness and generalization of EEG representations.
Critical weaknesses
1、The paper treats Re-cap as a core evaluation scenario. However, the Re-cap stage involves not only removing and reattaching the EEG cap but also changing the cognitive task from memory retrieval to an IQ assessment. Consequently, the observed performance degradation may result from both electrode repositioning and cognitive task changes, making it difficult to attribute the degradation specifically to Re-cap. I suggest adding a control experiment in which participants perform the same cognitive task before and after EEG cap reattachment to more accurately assess the distribution shift introduced by Re-cap.
2、Some experimental details need further clarification and consistency. For example, the main text represents EEG samples as 60 × 1600, whereas the Re-cap stage is described as extracting 2000-ms EEG segments. These descriptions are inconsistent in terms of temporal length. In addition, some important details, such as model training hyperparameters and specific data splitting procedures, are insufficiently documented. I suggest clarifying these details, ensuring consistency throughout the paper, and providing complete experimental configurations to improve the reproducibility of the benchmark.
3、1. The main contributions of this paper lie in data collection and benchmark construction, without proposing a new algorithm or representation learning method. The cognitive experimental paradigms and baseline models are also largely based on existing research. Although the work involves substantial effort, it primarily represents an integration of existing experimental designs and evaluation methods, with limited additional methodological innovation.
Questions for authors
In the within-task and cross-task identity recognition experiments, models such as EEGNet and FBCNet achieve 100% accuracy under multiple settings. Could these results potentially be affected by data leakage? I would appreciate further clarification on the specific data preprocessing and train-test splitting procedures, particularly whether there are overlapping or highly correlated EEG segments between the training and test sets, and whether any preprocessing steps utilize information from the test set.
7 comments
Response to Reviewer ijX7(Part 1/3: Comments 1)
Thank you for recognizing the value of evaluating multiple sources of EEG distribution change and for recommending matched-condition controls. Following our November 8 interim response, we have completed the additional ten-participant experiments. The new results address the task confound directly, and we clarify the segment definitions, training procedures, and evidence for train–test separation below.
1. Same-condition controls for the Re-cap evaluation
The completed controls show that substantial cross-recording degradation persists when the cognitive condition is held constant. The original Memory-to-IQ/Re-cap comparison changes both task and recording visit: IQ/Re-cap was recorded three days after Reading, Interference, and Memory Retrieval. We therefore added same-condition measurements to address this confound.
Between November 6 and 11, 2026, ten additional participants completed two recordings separated by 12 hours, with cap removal and reapplication between recordings. Section 4.4 and Table 6 evaluate resting state and the same 0-back, 1-back, or 2-back condition across recordings. FBCNet achieves resting-state accuracy of 79.66% ± 5.83%, while EEGNet achieves 68.23% ± 1.67%, 70.77% ± 3.05%, and 71.55% ± 2.41% for the three N-back conditions, respectively.
Table 9 further separates load and recording changes using the same participants' 0-back and 2-back data. For all eight models, task-specific training uses only first-recording 0-back, followed by four tests:
- First-recording 0-back: the reference condition.
- First-recording 2-back: load changes.
- Second-recording 0-back: recording changes while load is retained.
- Second-recording 2-back: both change.
Across all eight models, changing load within the first recording reduces accuracy by 0.00–6.42 percentage points, whereas changing the recording at the same load reduces it by 29.42–63.33 percentage points. Within each fold, the four tests share the same ten identities, selected checkpoint, and test-set size. These results directly support recording-associated changes as the larger source of generalization loss under this protocol.
The controls use resting state and N-back in an additional cohort. They do not separately estimate the contributions of cap reapplication, elapsed time, and physiological variation. We have aligned the manuscript's interpretation with the recording-associated effect established by these comparisons.
Response to Reviewer ijX7(Part 2/3: Comments 2)
2. Consistent segment lengths and complete experimental configurations
The identity-recognition tensor dimensions should be interpreted using the analysis-specific sampling rate and window:
| Analysis | Input or analysis window | Sampling rate | Duration or role |
|---|---|---|---|
| Original Reading, Memory Retrieval, and IQ/Re-cap identity recognition | 60 channels × 1,600 samples | 400 Hz | 4-second identity-recognition segments |
| Supplementary resting-state and N-back identity recognition | 60 channels × 800 samples | 400 Hz | 2-second identity-recognition segments |
| ERP analysis | 1,500 ms after stimulus onset, with a baseline from −200 to 0 ms | 400 Hz after downsampling | Separate event-related analysis |
We clarify that the previously mentioned 2,000-ms duration is a timing parameter of the experimental paradigm, distinct from the ERP analysis window. Specifically, the paradigm uses a 2,000-ms inter-trial interval, while ERP analysis is conducted over a 1,500-ms post-stimulus window with a 200-ms prestimulus baseline. We have clarified this distinction in the revised manuscript and appendix.
Preprocessing and training. The documented acquisition preprocessing includes downsampling to 400 Hz, 0.1–45 Hz filtering, bad-channel processing, ICA-based artifact removal, and re-referencing. For the original Reading/Memory/IQ evaluation, data-dependent preprocessing decisions and ICA parameters are fitted using training data within each fold and then held fixed for validation and testing. Classifier-input normalization is also fitted only on the corresponding source training partition.
The local baselines use AdamW and cross-entropy with label smoothing of 0.1. Learning rate is 0.001, except for EEGViT (0.0003) and the local EEGPT-named implementation (0.0005). Appendix E.3 and Tables 13–14 report architecture- and task-specific weight decay, batch size, and epoch budgets. Supplementary experiments use weight decay 0.01, batch size 32 except for BrainprintNet (8), gradient-norm clipping at 5, and at most 80 epochs, with a minimum of 20 epochs and early-stopping patience of 15. Checkpoint selection uses the lowest source-validation cross-entropy.
The original EEGPT-named baseline is a local implementation trained from scratch. We distinguish it from the additional evaluation using publicly released EEGPT weights, with a frozen encoder and a source-trained linear readout, reported in Table 11 and used for the EEGPT entry in Table 9.
Data partitioning. Within-task identity evaluation holds out complete epochs for testing, with validation confined to the remaining source data. Reading-to-Memory and Memory-to-IQ transfer use five source-side training/validation folds; each selected model is evaluated on the complete target recording. Their mean and standard deviation summarize five source-trained models on a fixed target set. Table 9 uses five blocked outer test folds, each with 720 source training segments, 240 separate source validation segments, and 240 test segments per condition. Validation and test blocks are disjoint. We have made these evaluation roles explicit in the captions and methods.
Re-cap Control Experiment with 10 Scheduled Participants (Expected Completion: November 11) and Preliminary Responses to Other Concerns
We sincerely thank the reviewer for the constructive comments and suggestions. We address the concerns as follows.