- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes typicality-aware EEG counterfactual explanations using a subject-invariant variational classifier-decoder with latent optimization, physiological constraints, and class-conditional density regularization, evaluated on the DREAMER dataset.
Critical strengths
The LOSO protocol clearly separates held-out participants from model training and source-based typicality calibration. The paper addresses an important interpretability challenge by requiring counterfactuals to alter predictions while remaining distributionally typical, minimally modified, and physiologically plausible. It also appropriately positions the generated patterns as model-level explanations rather than causal neuroscientific evidence.
Critical weaknesses
The underlying classifier achieves only 56.04% and 57.50% balanced accuracy, barely above chance level, raising concerns about whether its decision boundaries are sufficiently reliable for meaningful counterfactual interpretation. Moreover, typicality optimization and success evaluation rely on the same diagonal Gaussian model, making entry into the calibrated 95% region partly an expected consequence of the optimization rather than independent evidence of typicality. The analysis considers only successfully flipped class-zero trials, excluding unsuccessful cases, reverse-direction counterfactuals, and comprehensive evaluation across all samples. Evaluation is further limited to a single dataset, without comparisons against established counterfactual methods, independent neuroscientific assessment, or downstream user studies. Consequently, the validity and practical usefulness of the generated explanations remain insufficiently demonstrated.
Questions for authors
-
How meaningful and reliable are counterfactual explanations when the underlying classifier performs only slightly above chance?
-
Can typicality be independently assessed using a separate density estimator or held-out calibration data?
-
How does the proposed method compare with established EEG or time-series counterfactual explanation methods?
-
Can independent neuroscientists or downstream users assess the physiological plausibility and practical usefulness of the generated explanations?
Our whole-trial balanced accuracies on DREAMER were 56.04% for valence and 57.50% for arousal. We interpret the counterfactuals as exploratory explanations of this classifier’s decisions. Our evaluation is limited to DREAMER because repeating the LOSO training and counterfactual evaluation on a second dataset exceeds our compute budget for this revision.
Trials were eligible when the original prediction matched the true class. In the archived class-0-to-class-1 analysis, 57/114 arousal trials and 92/161 valence trials were eligible. We have implemented the reverse direction and will report all eligible attempts, including failures, with counts and reasons for exclusions.
The 10th-percentile Gaussian cutoff selects a central region of the source target-class embeddings. To assess typicality separately from the optimized score, we measure distance to the fifth-nearest real target-class trial in the frozen embedding, using disjoint source participants for reference and calibration. Results from this evaluation and the other counterfactual method comparison using the same frozen classifier and held-out trials are pending.
Blinded ratings by EEG experts could assess the physiological plausibility of the decoded signals. A separate user study is underway to assess their practical usefulness. As we note in the Introduction, the counterfactuals explain the classifier’s predictions. Establishing a causal link between the highlighted EEG features and emotional states would require intervention studies.