- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
SpikeMIND is a method using differential entropy (DE) concepts based on physiology (electrode placement and spectral bands) to improve emotion classification across subjects and generate rationale for it. The model is tested across 2 datasets, SEED and SEED-IV, with 2 settings: subject-dependent and leave-one-subject-out (LOSO), where it outperforms baselines based on diversity of model (graph, transformer spiking net, …).
Critical strengths
- The system does not only focus on performance, from its design it uses physiological inductive bias and provides explainability with natural language rationale when classifying emotion.
Critical weaknesses
- The evaluation is based only on 2 datasets spanning 30 participants in total while focussing on probing cross-subject generalization.
- The system pipeline is complex, hard to understand in the current version of the manuscript, and seems specialized for emotion recognition.
Note: The code is not shared, even if the supplement material provides details for reproducibility.
Questions for authors
I would lean for a rejection of the paper. The paper method is specialized in EEG emotion recognition, which limits its significance especially with an evaluation limited to 2 datasets spanning 30 participants. The method evaluation would benefit from more dataset diversity, larger scale, and adaptation to other EEG tasks.
Questions ordered by importance:
- Q1: Could you explain the motivation to select SEED and SEED-IV as the datasets to evaluate SpikeMIND? Are there some of their characteristics that make them the best for probing properties, such as cross-subject generalization or rationale generation? To strengthen the evaluation, additional datasets in the EEG emotion recognition task family could be considered.
- SEED-V and SEED-VII provide harder classification tasks (more class) and more subjects that might leave more room for a model to outperform other methods in the LOSO setting.
- Outside the SEED dataset family, many other datasets: DEAP, DREAMER, FACED, MPED exist. Would those datasets fit the SpikeMIND framework or would they need adaptation outside the current method's scope (e.g., different electrode geometry). If possible, within the rebuttal time, could SpikeMIND be tested on at least 2 additional datasets?
- Q2: Could you explain if the SpikeMIND framework could be extended to other tasks like BCI or if the method is specialized in EEG emotion recognition? Testing the framework on another EEG task where the cross-subject problem is imporant would make the method more significant.
- Q3: Could you clarify if:
- the results reported are the ones from the baseline' paper or are they from rerunning the methods?
- the evaluation protocols match between SpikeMIND and the other original evaluation baselines?
- Q4: Could you clarify some points about the language generation rationale and its evaluation?
- Is the LLM only used to generate a rationale and provide explainability, or is it also a component improving emotion classification?
- What is the baseline in the language rating (Fig 3.c)?
- Could you provide in the supplementary materials some qualitative example of the rationale generated from the LLM and the baseline?
- Q5: Why are the weighted average (WA) and unweighted average (UA) metrics used? It seems that results are most of the time within 1 point. Are the SEED datasets unbalanced?
- Q6: Could you add some references and contrast SpikeMIND to the EEG foundation model approaches that try to tackle the cross-subject problem with a different philosophy?
Additional feedback not affecting the decision assessment:
- Adding to the supplementary material some context on the DE framework and concept methods (definition of the notions such as concept, prototype, evidence) would help readers non-familiar with them.
- To improve the paper's clarity, figures 1 and 2 would benefit from more detailed captions.
- The language ratings would benefit from more blind raters even if they would grade fewer paired records.
- In the text some notations are not introduced, for example:
- line 154
- , , and line 158
- line 194
- line 198
- line 199