- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Bidirectional Exponential-memory Mixture (BEM) to aggregate CNN-generated class evidence across an EEG trial. The main finding is that emphasizing trial beginnings and ends improves imagined-speech classification on BCI2020. The authors investigate this pattern through kernel analysis, fixed pooling controls, and external experiments. Although the primary BCI2020 dataset shows substantial improvements, I have serious concerns about the scientific significance, interpretability, generalizability, and presentation of the work.
Critical strengths
- The authors did honestly report negative results and acknowledge that the observed boundary preference may not generalize across datasets.
- The authors analyzed and evaluated different temporal aggregations by comparing different readouts.
Critical weaknesses
-
I find it extremely hard to read. The paper has an unnecessarily complicated presentation and unclear figures. There are extensive mathematical terminology for what is fundamentally a temporal weighted-average operation, making the contribution difficult to follow. In Figure 1, there are unexplained graphical elements and terms such as "diagnose," "distill," and "same-evidence swap" without sufficiently intuitive explanations. In the Simplex mixture box, it’s unclear what it means that different lines (dashed and solid) connect dots. Figure 2C contains nearly overlapping curves. Figure 4 does not clearly communicate the statistical evidence, while Figure 5 combines multiple experimental settings without sufficiently clarifying their respective purposes. The complete model-performance comparison is relegated to the appendix rather than presented alongside the main findings.
-
Limited methodological novelty. The proposed BEM is essentially a constrained temporal pooling method. The model is therefore structurally biased toward boundary emphasis. More importantly, based on the reported initialization, it seems that approximately 91.92% of the initial kernel mass already lies in the outer fifths, compared with the reported 89.67% after training. This substantially weakens the interpretation that the model discovers a new temporal pattern. Furthermore, a simple fixed endpoint-pooling rule performs comparably to BEM, questioning the necessity of the proposed parameterization.
-
The most serious issue, in my opinion, is its unclear scientific significance and physiological validity. The central finding is that weighting the beginning and end of an EEG trial improves classification on BCI2020. However, the authors do not establish whether the discriminative information reflects imagined-speech activity or experimental confounds, such as cue-related responses, eye movements, muscular artifacts, or trial transitions. This is particularly concerning given the unusually high classification accuracy and the much weaker results reported in recent controlled imagined-speech benchmarks (e.g., Van Dyck and Van Hulle, 2026). Moreover, the study has limited itself to only one main dataset and encoder configuration. The external online experiment includes only three participants, showing no statistically significant raw-accuracy improvement. On ASU, BEM performs substantially worse than mean pooling, while the advantage disappears with an alternative encoder. These results provide little evidence that the proposed temporal weighting strategy generalizes.
References
Van Dyck, Bob. "The limits of imagined-speech decoding from EEG: A cross-dataset comparison and critical evaluation." (2026).
Questions for authors
- What evidence shows that the discriminative information near trial boundaries reflects imagined speech rather than cue-related activity or ocular/muscular artifacts? Can the authors provide time-resolved, spatial, and spectral temporal analyses, including experiments restricted to the actual imagery interval? More interestingly, would this finding only holds to imagined speech dataset or also to cue-based motor imagery dataset, visual ERP datasets: SSVEP, P300, etc.
- How does the learned temporal kernel differ from its initialization? What genuinely new temporal structure is learned during training?
- Why is BEM necessary when fixed endpoint pooling achieves comparable performance? Do attention (e.g., EEG FM can have attention pooling) and freely learned temporal pooling (e.g., CNN based pooling) baselines also learn to emphasize both boundaries? Given the negative or inconclusive external results, what evidence supports the practical usefulness of BEM beyond the original BCI2020 pipeline?
- Could the authors clarify the degree of independence of the historical audit and provide the complete model-comparison results in the main text?