acceptodds
Under review as a conference paper at ICLR 2027

Explaining Typical Subject-Invariant EEG Patterns of Emotion through Deep Learning Counterfactuals

Abstract

Counterfactual explanations of electroencephalography (EEG) deep learning models may provide actionable insights for downstream applications such as neuroadaptive brain-computer interfaces, user experimentation, and psychological assessment. However, counterfactual changes in EEG affective state classification are often incompatible with target-class distributions and may produce unrealistic EEG patterns. This complicates the interpretation of EEG-based affect models, particularly when they must generalize across participants. We propose a framework for generating counterfactual explanations under distributional and physiological constraints. We start by introducing a joint variational classifier-decoder architecture that combines affect prediction and EEG feature reconstruction with adversarial regularization to reduce subject-unique information. Then, we propose a counterfactual objective that combines a target-class prediction loss with class-conditional density regularization, proximity penalties in latent and reconstructed feature spaces, and physiological constraints on decoded EEG features. This objective explicitly favors target-class typicality while expressing changes in interpretable EEG frequency-band features. In a strict subject-independent setting, our classifier achieves highly competitive 56.04% valence and 57.50% arousal balanced accuracies on the DREAMER dataset. Typicality-constrained optimization improves counterfactual alignment with the learned target-class distribution relative to an objective without this constraint. All counterfactual EEG patterns generated for held-out subjects had distances to the target class within the 95th percentile of those observed for real target-class samples across subjects, supporting subject-independent generation of counterfactual patterns. These results suggest that typicality optimization alongside prediction change can produce more interpretable EEG affect counterfactuals that generalize across participants.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Explainable AI, Counterfactual Explainability, EEG, Deep Learning, Affective Computing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_GhnLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes typicality-aware EEG counterfactual explanations using a subject-invariant variational classifier-decoder with latent optimization, physiological constraints, and class-conditional density regularization, evaluated on the DREAMER dataset.

Critical strengths

The LOSO protocol clearly separates held-out participants from model training and source-based typicality calibration. The paper addresses an important interpretability challenge by requiring counterfactuals to alter predictions while remaining distributionally typical, minimally modified, and physiologically plausible. It also appropriately positions the generated patterns as model-level explanations rather than causal neuroscientific evidence.

Critical weaknesses

The underlying classifier achieves only 56.04% and 57.50% balanced accuracy, barely above chance level, raising concerns about whether its decision boundaries are sufficiently reliable for meaningful counterfactual interpretation. Moreover, typicality optimization and success evaluation rely on the same diagonal Gaussian model, making entry into the calibrated 95% region partly an expected consequence of the optimization rather than independent evidence of typicality. The analysis considers only successfully flipped class-zero trials, excluding unsuccessful cases, reverse-direction counterfactuals, and comprehensive evaluation across all samples. Evaluation is further limited to a single dataset, without comparisons against established counterfactual methods, independent neuroscientific assessment, or downstream user studies. Consequently, the validity and practical usefulness of the generated explanations remain insufficiently demonstrated.

Questions for authors
  • How meaningful and reliable are counterfactual explanations when the underlying classifier performs only slightly above chance?

  • Can typicality be independently assessed using a separate density estimator or held-out calibration data?

  • How does the proposed method compare with established EEG or time-series counterfactual explanation methods?

  • Can independent neuroscientists or downstream users assess the physiological plausibility and practical usefulness of the generated explanations?

1 comment

Our whole-trial balanced accuracies on DREAMER were 56.04% for valence and 57.50% for arousal. We interpret the counterfactuals as exploratory explanations of this classifier’s decisions. Our evaluation is limited to DREAMER because repeating the LOSO training and counterfactual evaluation on a second dataset exceeds our compute budget for this revision.

Trials were eligible when the original prediction matched the true class. In the archived class-0-to-class-1 analysis, 57/114 arousal trials and 92/161 valence trials were eligible. We have implemented the reverse direction and will report all eligible attempts, including failures, with counts and reasons for exclusions.

The 10th-percentile Gaussian cutoff selects a central region of the source target-class embeddings. To assess typicality separately from the optimized score, we measure distance to the fifth-nearest real target-class trial in the frozen embedding, using disjoint source participants for reference and calibration. Results from this evaluation and the other counterfactual method comparison using the same frozen classifier and held-out trials are pending.

Blinded ratings by EEG experts could assess the physiological plausibility of the decoded signals. A separate user study is underway to assess their practical usefulness. As we note in the Introduction, the counterfactuals explain the classifier’s predictions. Establishing a causal link between the highlighted EEG features and emotional states would require intervention studies.

Reviewer_D4rWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes counterfactual explanations for subject-independent EEG emotion classification on DREAMER. A joint variational model encodes 1-s windows of theta-, alpha- and beta-filtered EEG with a BiLSTM and a GCN-GRU. It classifies whole trials with a BiGRU and reconstructs the band signals with a shared decoder. The model is trained with first-order MLDG episodes, a Gaussian variant of variational classification, focal loss, a cross-subject supervised contrastive loss and a gradient-reversal subject adversary. Counterfactuals are found by gradient descent on the latent trial sequence. The objective combines a target-class loss, latent and reconstruction proximity terms, and a physiological penalty on the coherence and debiased wPLI of the decoded signal. Once the target confidence is reached, a typicality penalty is added, given by a variance-normalized distance to the learned class-1 Gaussian in the classifier embedding. Thresholding this distance at the 95 percentile of source-subject class-1 trials defines the typicality region.

Critical strengths

The first strength is that the classifier evaluation is careful. Under a strict zero-shot LOSO protocol, the authors report balanced accuracy, per-class recall, AUROC and per-subject results instead of plain accuracy. They also explain why per-subject scores become unstable when a subject has one or two minority-class trials and point out that one prior DREAMER result relies on a transductive protocol. This kind of reporting makes the results easier to interpret and harder to inflate than accuracy-only reports. The numbers also show that the model is only modestly above chance. The physiological penalty is a useful EEG-specific addition. Requiring the counterfactuals to keep plausible short-range coherence and wPLI, in addition to flipping the prediction, makes sense. The penalty is specified in detail in Appendix B.2. The authors only claim that coherence is evidence of plausibility, and they present the counterfactuals as model-level explanations for generating hypotheses. They also report negative results, such as the implausible reconstruction and the unchanged arousal median, and several of the problems below are based on these results.

Critical weaknesses

The first weakness is that the decoded EEG change, which is the explanation the user actually sees, is not shown to reflect the classifier's decision. Validity is only reported for the latent counterfactuals, and the decoded ones are never re-encoded and classified. In Table 2 the prediction flips with essentially no decoded change when the physiological penalty is removed, and the typicality penalty moves the latent about five times further for valence without changing the median decoded displacement. The decoded change only becomes noticeable when the physiological penalty is on, and in the case study the starting reconstruction already scores far worse on that penalty than the real trial. Much of the decoded difference may therefore come from the penalty repairing a poor reconstruction instead of from what flips the classifier. Since the stated contribution is to express class changes in interpretable EEG frequency-band features, this is the main reason for my soundness score. It also weakens the arousal reading in Section 4.2, which rests on one trial and on ERD work about task- and attention-related desynchronization.

Also, the evaluation does not support the claims about typicality, subject invariance and a highly competitive classifier. Typicality is scored with the same discrepancy that is optimized, so the 100% pass rate only shows that the optimization converged. If many class-0 trials already lie inside the region, as the case-study trial does, the constraint may be close to vacuous, and for arousal the typicality penalty does not change the median discrepancy. Plausibility has the same problem, since VCSC is a tolerance on the optimized penalty, and in the case study the two checks that are not optimized (amplitude and spectral power) fail. For subject invariance, the only direct evidence is the adversary's training accuracy. A region calibrated to hold 95% of source-subject class-1 trials also holds only about two thirds of held-out class-1 trials, which, if both are measured in the encoded space, points to a shift between subjects. The classifier itself is close to chance, so the counterfactuals mostly describe this model. No baseline is run under the same protocol, and the one other zero-shot DREAMER result in the paper with balanced accuracy (Zhang et al., 2026) is higher, though with a different label threshold. Finally, thresholded class-conditional Gaussian constraints (Artelt & Hammer, 2020) and latent counterfactual search with decoding (Joshi et al., 2019) already exist, and the paper neither cites nor compares with them.

Questions for authors
  1. For each arm in Table 2, what fraction of the decoded counterfactuals, when re-encoded and classified, are assigned to the target class and fall inside the typicality region? Is the alignment analysis computed on these re-encoded counterfactuals or on the latent counterfactuals directly?
  2. In a physiology-only control (lambda_o = 0) run for the same number of iterations, how large are the decoded change and the frontal alpha change from Section 4.2 across the arousal trials, compared with the full objective?
  3. How many of the original class-0 trials already lie inside the typicality region, and how many counterfactuals are inside it before the typicality penalty is switched on?
2 comments

Decoded validity and frontal alpha—In the submitted analysis, we measured how closely the optimized counterfactual embeddings matched the learned target-class distribution before decoding them into EEG waveforms. We observed prediction collapse when we passed the decoded waveforms back through the encoder and classifier. To address this, we are adding and testing decoding strategies that reintroduce subject-specific information. We will use the frozen encoder and classifier to check whether the resulting waveforms preserve the target-class flip. Per-arm rates for decoded target-class validity, typicality, and joint success are pending. A matched physiology-only comparison (lambda_o = 0) is also pending and will test whether the physiological objective alone produces similar waveform and frontal-alpha changes.

Typicality and subject invariance—At a retrospective 10th-percentile cutoff, 4/114 original arousal and 14/161 original valence class-0 trials met the criterion. Immediately before penalty activation, 26/28 arousal and 37/75 valence trials that reached that stage met it. Among archived endpoints, adding the penalty increased inclusion from 55/57 to 57/57 for arousal and from 49/92 to 87/92 for valence. After encoding real held-out class-1 EEG trials, we found that 27/300 arousal trials (9.0%) and 29/253 valence trials (11.5%) fell within the source-calibrated 10th-percentile region in embedding space. The earlier rates of 68.1% for arousal and 62.1% for valence came from a smaller sample of real trials scored directly in waveform space against a 95th-percentile cutoff. We will qualify the subject-invariance claim pending an independent subject-identification test.

Classifier and prior work—We retain the comparison with Zhang et al. to contextualize cross-subject performance on DREAMER. Our pooled whole-trial balanced accuracies are 57.50% for arousal and 56.04% for valence, indicating modest discrimination on unseen participants. Zhang et al. report balanced accuracies of 58.73% and 60.05% and ordinary accuracies of 75.15% and 65.55%, respectively. Their LOSO results provide aggregate metrics. Given the class imbalance, especially for arousal, we report recall for each class in Table 1 so readers can assess how reliably the classifier recognizes each class when interpreting its counterfactuals. As discussed in Appendix A.4.1, Zhang et al. average window-level metrics across held-out subjects and report a high-class cutoff of >3, compared with our >=3.

Artelt and Hammer use separately fitted Gaussian mixtures to constrain counterfactual search, and Joshi et al. search a learned latent space and decode the resulting counterfactuals. Our variational classifier jointly learns the EEG embedding and a diagonal Gaussian for each class. We use the learned target-class Gaussian directly in the counterfactual objective to guide the search toward typical target-class embeddings. We will cite both papers in the revised manuscript and clarify this distinction.

Thank you for the answer. The prediction collapse confirms my main concern. The decoded EEG is what a reader sees and it does not carry the class flip, so the frontal alpha interpretation in Section 4 is not supported. A decoder that reintroduces subject-specific information is a different model, so I cannot credit it or the other pending results here.

On typicality, the new counts use a stricter cutoff than the criterion in the paper, so they do not answer my question directly. From what you show, the penalty does almost nothing for arousal and the counterfactuals sit much closer to the class mean than real target-class trials, which is not the same as matching the target distribution. The alignment comparison between counterfactuals and real trials was not computed in the same space. My rating stays at weak reject.

Reviewer_qZpQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper trains a subject-adversarial variational EEG classifier–decoder on DREAMER and generates latent counterfactuals from predicted low to high valence or arousal. The counterfactual objective combines target-class prediction, class-conditional typicality, latent and reconstructed-signal proximity, and physiological constraints; the optimized representations are decoded into electrode–frequency-band waveforms for interpretation.

Critical strengths

The evaluation follows strict subject separation. The authors use 23-fold leave-one-subject-out cross-validation, and the typicality threshold in each fold is calibrated only on source-subject trials.

The counterfactual evaluation goes beyond prediction validity. It reports latent displacement, decoded-signal change, target-class discrepancy, and physiological diagnostics rather than only whether the prediction flips.

Critical weaknesses

The decoded signals have insufficient physiological fidelity for the central interpretation claim. In the reported case study, the physiological penalty is 0.191 for the original EEG but 1102.31 for its initial reconstruction, and the counterfactual passes only two of four physiological checks. Across eligible trials, the VCSC pass rates are only 49.12% for arousal and 81.52% for valence.

Typicality is directly optimized and lacks independent comparative validation. All optimized counterfactuals falling within the source-calibrated 95th-percentile region is not an independent plausibility test because this criterion is included in the objective. There is also no controlled comparison with existing time-series or EEG counterfactual methods. This is particularly relevant because the underlying classifier achieves only 56.04% valence and 57.50% arousal balanced accuracy, so the explanations may reflect a weak and threshold-sensitive decision boundary.

Questions for authors

Assessment-critical experiment: Can the authors compare against at least one existing time-series or EEG counterfactual method using the same classifier, reporting validity, proximity, target-class discrepancy, and physiological pass rates?

Clarification: If the generated counterfactuals are described as physiologically constrained, how should the 49.12% arousal VCSC pass rate be interpreted? Are physiologically failing samples still included in the claim that all counterfactuals satisfy the typicality criterion?

Assessment-critical experiment: Can the authors validate decoded counterfactuals using an independent physiological discriminator or real target-class nearest neighbours, and report reconstruction and counterfactual quality both with and without gradient reversal? This is important because the case study’s physiological penalty increases from 0.191 for the original EEG to 1102.31 for its reconstruction before counterfactual optimization.

2 comments

Thank you for raising the concern about physiological fidelity. The submitted typicality analysis included trials that failed the physiological checks. All four evaluable checks passed for 22/57 original arousal trials, 0/57 baseline reconstructions, and 3/57 decoded counterfactuals. For valence, the corresponding counts were 44/92, 0/92, and 0/92. A reconstruction can move closer to the reference ranges and still fail the binary checks. We are evaluating a complementary 0–1 score that captures how far each signal falls outside those ranges.

We have implemented a paired comparison with another established counterfactual time-series method on the same held-out trials and frozen classifier, assessing waveform-level target validity, proximity to the original trial, target-class typicality, and physiological fidelity. Results from this comparison and the matched evaluation with and without gradient reversal are pending.

Reviewer_qZpQOpenReview ↗

Feedback to the authors

Many thanks for your answering. You can furthe improve your works according to the reviewer's comments.

Comments on the paper

1 comment

Thank you for reading our paper so carefully. Your comments were very helpful, and we addressed every concern we could in this revision. Your suggestions substantially strengthened the experiments, particularly how we test whether a counterfactual explanation remains faithful to the classifier after decoding.

We now check whether the decoded EEG produces the intended prediction, apply a stricter test of target-class typicality, and describe an independent comparison with real target-class EEG. We also evaluate prediction validity, typicality, and physiological plausibility together. We added a physiology-only control to examine which changes arise from physiological constraints alone. We compared our method with CoMTE in both directions for valence and arousal, included unsuccessful attempts in the evaluation, and clarified what the gradient-reversal experiment tells us about subject identity in the learned representation.

We accepted some loss in classification accuracy to improve EEG reconstruction. Our main contribution is the counterfactual explanation method, and a more faithful decoder helps us test whether the EEG changes we interpret actually support the model’s prediction.

We have kept this comment brief out of respect for your time, since the revised manuscript describes these changes in detail.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.