- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents EviFormer for subject-level consciousness assessment from long-duration EEG, combining temporal/spectral embeddings, multi-scale attention, gated causal convolutions (EviMemory), and weakly supervised subject-level learning. Results are reported on two linked clinical cohorts and four public Alzheimer’s disease EEG datasets.
The clinical motivation and reported results are promising. However, the current evidence does not adequately separate long-range temporal modeling from the accumulation of local EEG information, and the fairness of key baseline comparisons remains uncertain. Stronger mechanism controls, MIL comparisons, and more transparent external validation would substantially strengthen the submission.
Critical strengths
1、Clinically meaningful task with a plausible motivation for accumulating intermittent EEG evidence. 2、A valuable self-collected clinical cohort and broad comparisons across EEG and time-series models. 3、Ablations, context-length analysis, synthetic sequence tests, and physiological interpretation are provided.
Critical weaknesses
- Limited Receptive Field of EviMemory Figure 2 indicates that MMSE-140 performance is strongest with roughly 30–60 minutes of EEG history, while EviMemory uses two causal convolutions with a local receptive field of seven 30-second chunks (3.5 minutes). Why was this setting chosen? Would longer receptive fields or larger dilations improve performance? Note that the model still aggregates the full history through cumulative averaging.
- Temporal Modeling versus Evidence Aggregation Table 2 shows a benefit from EviMemory over chunk-level mean pooling, but this could arise from improved local temporal features rather than long-range dependencies. Likewise, the history-length gains in Figure 2 might reflect the availability of more EEG evidence. The synthetic ordering task in Appendix D can potentially be solved using local transition patterns. Please compare history-length curves with and without EviMemory and consider controlled temporal-order perturbations.
- Baseline Reproduction and Evaluation Fairness ADformer, LEAD, and other established EEG models perform substantially below values reported in previous studies on the same underlying public datasets. The original protocols are not directly comparable, but the manuscript does not adequately explain these gaps. Please document official implementations, checkpoints, input adaptation, training budgets, and hyperparameter tuning, and clarify the reasons for the discrepancies.
- Missing External DoC Validation Despite Available Public Data Although publicly accessible UWS/MCS EEG datasets have been reported (e.g., Bai et al., NeuroImage 2021), the DoC assessment is evaluated only on the authors’ cohort, while the public-data experiments focus on Alzheimer’s disease. Why were existing public DoC datasets not considered for external validation? The authors should clarify whether recording duration, electrode montage, acquisition protocols, or access restrictions prevent their use. Even if these datasets are unsuitable for evaluating long-duration accumulation, shorter-context external validation could still test cross-cohort generalization.
Questions for authors
Q1. EviMemory's effective receptive field is only ~3.5 minutes, much shorter than the 30–60 minute history window where performance peaks, and full history is aggregated by cumulative averaging. This makes it unclear whether the gains in Figure 2 come from long-range temporal modeling or simply from accumulating more local EEG evidence. Could you provide history-length curves with and without EviMemory, plus a controlled test (e.g., shuffling chunk order) to show whether global temporal order actually matters? Q2. ADformer, LEAD, and other baselines score well below previously reported results on the same public datasets, and the paper does not explain this gap. Please clarify whether official implementations/checkpoints were used, how inputs were adapted, and whether tuning budgets were comparable — do the relative rankings still hold under fair re-tuning?
2 comments
Response to Reviewer ssDu
R3. Baseline implementation, tuning, and comparability (Weakness 3, Question 2).
Thanks for your helpful comments. Differences in experimental settings can explain the performance gaps. The original LEAD study pretrained on multiple external datasets before downstream fine-tuning. Except for APAVA, which used a fixed split, it used 5 random seeds and assigned 80%, 10%, and 10% of subjects to training, validation, and testing, respectively.
ADformer's subject-independent experiments on ADSZ and ADFD assigned 60%, 20%, and 20% of subjects to these sets.
We trained LEAD and ADformer from scratch within each task dataset, evaluated them on held-out subjects, and reported mean BAcc and AUROC across 10 repetitions of subject-wise five-fold cross-validation.
R4. External validation on shorter DoC recordings (Weakness 4).
Thanks for your helpful comments. We did not include this dataset because its acquisition setting differs from long-term bedside monitoring. Each recording contains 10 minutes of resting-state EEG, which supports evaluation within that duration. Assessing evidence accumulation beyond 10 minutes requires longer recordings. In addition, the data from Bai et al. (2021) are no longer publicly accessible, and we are contacting the authors to obtain them.
Response to Reviewer ssDu
R1. Choice of local receptive field (Weakness 1).
Thanks for your helpful comments. We use a local receptive field of 3.5 minutes to extract patterns across adjacent chunks. Increasing the receptive field produced no significant improvement in our experiment.
The model can still aggregate the 30–60 minutes of history shown in Figure 2 with this 3.5-minute local receptive field. The write gate controls how local information updates the current representation, while the running mean accumulates all updated representations up to the current time. The recording can therefore continue to grow while each step retains a bounded historical state.
We added an experiment on MMSE-140 that progressively increased the number of causal convolutional layers with dilation 2. Expanding the local receptive field to 11.5 minutes produced no significant improvement, supporting the two-layer configuration.
R2. Observation length and sensitivity to chunk order (Weakness 2, Question 1).
Thanks for your helpful comments. The results indicate that accumulating local EEG evidence can explain the benefit of longer observation periods.
To examine the respective roles of observation duration and EviMemory, we compared mean BAcc and AUROC with and without EviMemory at ten durations on MMSE-140. M denotes EviMemory.
Extending the observation period from 0.5 to 60 minutes increased AUROC from .548 to .698 with EviMemory and from .520 to .618 without it, demonstrating the benefit of using a longer EEG history. EviMemory achieved higher mean values for both metrics at all ten durations, supporting the contribution of the memory module.