acceptodds
Under review as a conference paper at ICLR 2027

Context-aware tokenization for Cross-subject Emotion Decoding from EEG

Abstract

In cross-subject EEG emotion decoding, neighboring windows can inform the representation of a target window, but they contain both background variation and sustained task signals. This creates a challenge for contextual representation learning because subtracting shared activity can also remove useful information. We introduce the Morlet Spectral Transformer (MST), which conditions target-window tokens on a structured spectral summary before attention. MST averages the Morlet log-amplitude spectra of unlabeled same-trial neighbors and uses frequency-specific electrode projections to combine the target spectrum with its reference-relative residual. This retains access to absolute activity while incorporating context into a fixed attended target-token grid. Without external pretraining, MST achieves 66.5%, 40.7%, 36.1%, and 27.9% accuracy on SEED, SEED-IV, SEED-V, and SEED-VII under leave-one-subject-out evaluation, averaged over three random seeds, outperforming the evaluated pretrained and from-scratch baselines. On FACED, MST achieves 17.52% accuracy in nine-class cross-subject evaluation, exceeding the strongest evaluated baseline by 1.15 percentage points. We also performed the ablation studies across all 15 SEED subjects to evaluate the contribution of different components of MST.

Primary area:
applications to neuroscience & cognitive science
Keywords:
brain decoding, affective computing, neural coding, brain-computer interfaces

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_poDYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes the Morlet Spectral Transformer (MST) for cross-subject EEG emotion recognition. It uses Morlet wavelets to extract time–frequency representations and constructs a local spectral reference from neighboring windows within the same trial. The target spectrum and its reference-relative residual are combined through frequency-specific spatial projections before Transformer-based classification. This design incorporates temporal context while maintaining a fixed attention token count.

The method is evaluated on SEED, SEED-IV, SEED-V, SEED-VII, and FACED against six general-purpose EEG representation models. The authors report improvements over the evaluated baselines and provide component ablations, context-span-matched comparisons, and past-only context experiments.

Critical strengths

1. The motivation for incorporating neighboring spectral information is reasonable, and the design is structurally simple.

Combining absolute and reference-relative representations provides a straightforward way to utilize contextual information while retaining potentially useful sustained emotion-related activity. Incorporating the neighborhood summary into target tokens before attention keeps the Transformer's token count independent of the number of neighbors.

2. The evaluation covers multiple datasets and includes relevant controls.

The paper evaluates five datasets with different numbers of emotion classes, reports results across multiple random seeds, and includes all-subject component ablations, context-span-matched comparisons, and past-only context experiments.

Critical weaknesses

1. The baseline selection is inadequate, and the reported results do not establish competitiveness on the target task.

1.1 Representative cross-subject emotion-recognition methods are missing.

The six evaluated baselines primarily target general-purpose EEG representation learning. Established emotion-recognition methods such as DGCNN [1], RGNN [2], IAG [3], and EmT [4] are absent from the quantitative comparisons. EmT is particularly relevant because it explicitly models temporal context for cross-subject EEG emotion recognition.

The paper discusses some of these methods in the related work but provides no experimental comparison against them. The current baseline coverage does not establish whether MST improves over mature solutions to the target task. Consequently, improvements over the selected general-purpose models are insufficient to establish an empirical contribution to cross-subject EEG emotion recognition.

1.2 Absolute discriminative performance is limited, and the substantial gap from prior work remains unexplained.

MST achieves only 17.52% accuracy and a Cohen's kappa of 0.0625 on nine-class FACED, indicating limited overall discriminative performance. Under its own evaluation protocol, the original FACED dataset paper [6] reported cross-subject nine-class accuracies of 35.2% for DE + SVM and 42.4% for CLISA [5].

The submission neither explains this gap through comparisons under compatible protocols nor establishes an advantage over a simple spectral-feature classifier. Outperforming the selected baselines, which themselves perform poorly in this evaluation, does not establish practical competitiveness.

2. The methodological contribution is incremental, and the specific fusion design lacks direct empirical support.

2.1 The method primarily combines established techniques without establishing a substantive incremental advantage.

The proposed method combines Morlet time–frequency analysis, local spectral referencing, frequency-specific spatial projections, and Transformer-based classification. DMNet [7] and EmT [4] have already explored contextual spectral differences for seizure detection and temporal contextual modeling for emotion recognition, respectively. Moreover, Equation (1) shows that the linear projection of the absolute spectrum and its residual is mathematically equivalent to jointly projecting the target and reference spectra.

The paper does not sufficiently demonstrate the advantage of this particular combination over simpler fusion strategies or establish a substantive advance over closely related approaches. The existing evidence therefore provides insufficient support for the methodological novelty of the proposed contextual representation.

2.2 The ablations conflate the benefits of neighboring information with those of the fusion mechanism.

Removing the reference branch also removes neighboring information, so this ablation only tests the effect of the entire context pathway. The context-span-matched comparison with CSBrain [8] still differs in architecture and training recipe and therefore does not isolate the proposed fusion mechanism.

Direct alternatives, such as reference-only classification and simple spectral aggregation, are missing. The experiments support the usefulness of access to neighboring information but do not establish that the gains depend on the proposed target–reference fusion. This gap directly concerns the empirical basis of the paper's central contribution.

3. Differences in training recipes prevent attribution of the reported gains to the proposed method.

3.1 Data augmentation is substantially asymmetric and materially affects performance.

Appendices B.3–B.4 state that the baselines use no input augmentation, whereas MST uses phase perturbation, band-specific noise, channel dropout, and wavelet time roll. In Table 3, removing augmentation reduces MST's SEED accuracy from 66.5% to 60.1%, a 6.4-percentage-point decrease that exceeds its 4.1-point improvement over pretrained LaBraM [9] at 62.4%.

The effect of augmentation is therefore comparable to, and numerically larger than, the reported improvement, yet no corresponding augmentation controls are provided for the baselines. Table 1 consequently does not separate the contribution of the model design from that of augmentation.

3.2 Training budgets differ without evidence that the baselines were sufficiently optimized.

MST is trained for 50,000 steps, whereas EEGPT [10], CBraMod [11], and CSBrain [8] are trained for 10,000 steps, with additional differences in batch size. Even the context-span-matched comparison retains different training configurations. The paper provides neither convergence evidence nor a budget-sensitivity analysis.

The experiments demonstrate the effectiveness of the complete MST training recipe relative to the selected baseline configurations, but do not convincingly isolate the benefit of context-aware tokenization.

4. Unresolved discrepancies between the code and the stated evaluation protocol limit the verifiability of the results.

4.1 The normalization pathway for held-out subjects does not match the paper's description.

Appendix B.1 specifies normalization of held-out subjects using statistics pooled from the training subjects. However, morlet/datasets/seed.py retrieves the mean and standard deviation using the current subject's ID and falls back to the current trial's statistics when the corresponding entry is unavailable. The released main-experiment pipeline does not provide an implementation for computing and applying source-pooled statistics within each LOSO fold. The code therefore does not establish that the experiments followed the stated normalization protocol.

4.2 The correspondence between the reported results and the checkpoints is unclear.

The paper states that evaluation uses the final training step without selecting models based on held-out-subject performance. However, the main configuration specifies best_metric = val_balanced_accuracy, and the script evaluates every 200 steps and saves the best-performing checkpoint as best, while separately saving final. The mapping between the reported results and these checkpoints is not documented, preventing verification that the reported scores follow the stated final-step evaluation rule.

References

[1] Song, T., Zheng, W., Song, P., and Cui, Z. EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks. IEEE Transactions on Affective Computing, 11(3):532–541, 2020. https://doi.org/10.1109/TAFFC.2018.2817622

[2] Zhong, P., Wang, D., and Miao, C. EEG-Based Emotion Recognition Using Regularized Graph Neural Networks. IEEE Transactions on Affective Computing, 13(3):1290–1301, 2022. https://doi.org/10.1109/TAFFC.2020.2994159

[3] Song, T., Liu, S., Zheng, W., Zong, Y., and Cui, Z. Instance-Adaptive Graph for EEG Emotion Recognition. AAAI, 2020. https://doi.org/10.1609/aaai.v34i03.5656

[4] Ding, Y., et al. EmT: A Novel Transformer for Generalized Cross-Subject EEG Emotion Recognition. IEEE Transactions on Neural Networks and Learning Systems, 2025. https://doi.org/10.1109/TNNLS.2025.3552603

[5] Shen, X., Liu, X., Hu, X., Zhang, D., and Song, S. Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition. IEEE Transactions on Affective Computing, 14(3):2496–2511, 2023. https://doi.org/10.1109/TAFFC.2022.3164516

[6] Chen, J., et al. A Large Finer-grained Affective Computing EEG Dataset. Scientific Data, 10:740, 2023. https://doi.org/10.1038/s41597-023-02650-w

[7] Tu, S., et al. DMNet: Self-comparison Driven Model for Subject-independent Seizure Detection. NeurIPS, 2024. https://proceedings.neurips.cc/paper_files/paper/2024/hash/3192aa37a2227627d7f8cbf582c075f3-Abstract-Conference.html

[8] Zhou, Y., et al. CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding. NeurIPS, 2025. https://proceedings.neurips.cc/paper_files/paper/2025/hash/7e199ad8ae40eb19b2980f61f659cb07-Abstract-Conference.html

[9] Jiang, W.-B., Zhao, L.-M., and Lu, B.-L. Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI. ICLR, 2024. https://openreview.net/forum?id=QzTpTRVtrP

[10] Wang, G., et al. EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals. NeurIPS, 2024.

[11] Wang, J., et al. CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding. ICLR, 2025. https://proceedings.iclr.cc/paper_files/paper/2025/hash/bbbd6d915cb90be21c1254a82d45cedd-Abstract-Conference.html

Questions for authors

Why were representative task-specific methods excluded from the quantitative evaluation, and how do the current results establish MST's competitiveness in cross-subject EEG emotion recognition? Beyond access to additional neighboring information, which experiments support the independent contribution of the proposed fusion mechanism? Given the differences in augmentation and training budgets, on what basis are the main performance gains attributed to context-aware tokenization? Which data were used to compute normalization statistics for held-out subjects, and do the reported results correspond to the best or final checkpoints?

3 comments

We thank the reviewer for the detailed comments.

Concern — W1.1 and the baseline-selection question. Why are DGCNN, RGNN, IAG, and EmT absent, and what competitiveness claim follows from the evaluated baselines?

Response. The current selection of baselines aim to test whether MST, built around context-aware spectral tokenization and trained without external pretraining, can compete with general-purpose EEG encoders. The current baseline includes five latest encoders evaluated with pretrained and scratch initialization, together with TFM-Tokenizer, which are complete for our purpose. Therefore, we did not include the results of DGCNN [1], RGNN [2], IAG [3], and EmT [4] in the manuscript.

Nonetheless, we include the results with DGCNN, RGNN, IAG, and EmT as follows.

MethodSEEDSEED-IVSEED-VSEED-VII
DGCNN52.67 ± 10.1630.65 ± 4.7125.16 ± 5.5618.92 ± 2.81
RGNN46.43 ± 7.1130.55 ± 3.9125.75 ± 4.4420.47 ± 3.32
IAG52.28 ± 8.8330.04 ± 5.1125.25 ± 6.8321.63 ± 3.56
EmT57.39 ± 8.1538.50 ± 6.6934.14 ± 7.6525.26 ± 4.45
MST (Table 1)66.5 ± 8.140.7 ± 5.136.1 ± 4.527.9 ± 3.9

The four added baselines use the same training budget per fold. The standard deviation across subjects are reported. For implementation of these four models, we adapt their design to our data shape and evaluation protocol. Implementation details will be included in the appedix of our manuscript.

Concern — W1.2. How should 17.52% accuracy and κ = 0.0625 be interpreted relative to the original FACED results of 35.2% for DE + SVM and 42.4% for CLISA cited in the review?

Response. We take a strict leave-one-subject-out evaluation protocol. Especially, we compute each channel's mean and variance using only the current one-second EEG window. However, the original CLISA method [5] takes a different data processing protocol. CLISA initializes normalization with training statistics, then updates feature means and variances using incoming test-subject data (§4.4). Thus, CLISA adjusts feature scaling to the test subject's accumulated data, while our window-wise normalization does not accumulate statistics across windows.

Concern — W2.1. Given the precedents in DMNet and EmT, and the equivalence in Equation (1), what does the design contribute?

Response. MST differs from DMNet [6] in its spectral representation and spatial modeling, and from EmT in how context enters the Transformer. DMNet encodes channel-wise spectral difference matrices with a CNN. MST instead retains both absolute and locally referenced Morlet spectra across electrodes, then mixes electrodes separately at each frequency. The local reference is designed to reduce temporal redundancy, while the absolute branch preserves access to sustained spectral activity.

EmT [4] represents successive EEG feature graphs as a token sequence and learns their temporal relationships with a contextual Transformer. MST averages neighboring spectra into a reference and uses it to form the target's time–frequency tokens before attention. Context therefore changes the target tokens without adding neighboring windows to the attention sequence (Sections 3.2–3.4 and Appendix E). These differences in representation and context construction remain under either parameterization in Equation (1).

Concern — W2.2 and the fusion-contribution question. Which experiment isolates fusion from access to neighbors? What about reference-only classification or simple spectral aggregation?

Response. The ablation aims to test the effectiveness of the complete neighbor-reference pathway. This design is motivated by removing the temporal correlation while preserving the original spectral information, thus using the neighboring EEG and a fusion technique. Our ablation alreadly shows that removing the pathway reduces mean accuracy by 5.6 percentage points (Table 3), directly showing the effectiveness of this design. Figure 2 also shows that the temporal correlation is reduced.

Therefore, isolating the fusion from access to neighbors would not be necessary and out-of-the-scope of our ablation, where we only aim to justify the effectiveness and motivation of the proposed design.

Concern — W3.1 and the attribution question. Does the augmentation imbalance explain the main performance difference?

Response. All our experiments share the same data processing pipeline, including the same data augmentation. The wording in Appendix B.3 was meant to say that the augmentation was designed specific for MST and may not be compatible with some of the the baselines. We will correct the wording in Appendix B.3 to ensure there is no confusion.

Concern — W3.2. Were the baselines sufficiently optimized, given the unequal budgets and lack of convergence analysis?

Response. The training budgets follow the same loss-based stopping criterion for all models. For each model, we use one seed to determine when training loss stops decreasing or begins to rise, then fix that number of steps for the remaining seeds. We also give every model the same hyperparameter-tuning budget and select its learning rate and batch size based on training loss. Different step counts therefore reflect when each model meets the shared stopping criterion.

On SEED, this procedure gives 50,000 steps for MST and LaBraM, and 10,000 steps for CSBrain, EEGPT, and CBraMod (Table 6 and Appendix B.4). We will clarify how training loss determines both the hyperparameters and the training duration.

Concern — W4.1 and the normalization-data question. The loader retrieves statistics by the current subject's ID and can fall back to trial statistics. How does this match the source-pooled normalization described in Appendix B.1?

Response. The reported SEED runs using this loader followed the source-pooled normalization specified in Appendix B.1. For each LOSO fold, we precomputed the mean and standard deviation using only the training subjects and saved them locally under the held-out subject's ID. The ID therefore identifies the fold's stored training statistics. The loader retrieves these precomputed values, so it does not need to recompute them during loading. All reported runs using this loader loaded the corresponding statistics successfully, and the trial-statistics fallback was never triggered. We will document the precomputation step and the subject-ID mapping in the released pipeline.

Concern — W4.2 and the checkpoint question. Do the reported results use the final or the best-performing checkpoint?

Response. We used the final checkpoint as reported in the manuscript. Our codes also save the intermediate checkpoints and the best checkpoint for analysis purpose.

References

[1] Tengfei Song, Wenming Zheng, Peng Song, and Zhen Cui. EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks. IEEE Transactions on Affective Computing, 11(3):532–541, 2020.

[2] Peixiang Zhong, Di Wang, and Chunyan Miao. EEG-Based Emotion Recognition Using Regularized Graph Neural Networks. arXiv:1907.07835, 2019.

[3] Tengfei Song, Suyuan Liu, Wenming Zheng, Yuan Zong, and Zhen Cui. Instance-Adaptive Graph for EEG Emotion Recognition. AAAI, 34(3):2701–2708, 2020.

[4] Yi Ding et al. EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition. IEEE Transactions on Neural Networks and Learning Systems, 2025.

[5] Xinke Shen et al. Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition. IEEE Transactions on Affective Computing, 2022 (online publication).

[6] Shihao Tu et al. DMNet: Self-comparison Driven Model for Subject-independent Seizure Detection. NeurIPS, 2024.

I appreciate the authors' response, including the additional emotion-recognition baselines and code clarifications. However, the response does not resolve my core concerns:

  1. Unexplained low performance on FACED. MST achieves 17.52% accuracy (κ=0.0625\kappa = 0.0625), compared with 35.2% for DE + SVM and 42.4% for CLISA in the original FACED paper, as well as 45.7% for MSHCL and 67.9% for DAEST. These published results are approximately 18 to 50 percentage points higher than MST. The response discusses CLISA's test-time normalization but neither quantifies its effect nor addresses the discrepancy with DE + SVM. The substantial performance gap therefore remains unexplained.

  2. Unvalidated fusion mechanism. Removing the neighbor-reference branch removes both the neighboring information and its fusion with the target. The reported 5.6-point drop therefore supports the entire context pathway, but not the proposed fusion specifically. Declaring this distinction unnecessary and out of scope leaves the claimed contribution of the proposed fusion unsupported.

  3. Inconsistent training and augmentation protocols. Selecting training budgets and hyperparameters from training loss does not establish that the baselines were adequately optimized for cross-subject generalization. The claim that all experiments use the same augmentation contradicts Appendix B.3, which states that the baselines use no input augmentation, and the response itself states that the augmentations were designed for MST and may be incompatible with some baselines. Removing augmentation reduces MST's SEED accuracy by 6.4 points, exceeding its 4.1-point gain over pretrained LaBraM.

These issues directly affect the paper's empirical conclusions. I maintain my rating of 1 (Clear rejection).

Reviewer_C8qEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MST, a Morlet Spectral Transformer for cross-subject EEG emotion decoding. A fixed Morlet bank maps a target window and its K=5 neighboring windows to a joint time-frequency grid; the neighbors' averaged log-amplitude spectrum forms a local reference, and absolute/residual views are projected per-frequency into a fixed grid of 320 tokens for a 2D-RoPE Transformer classifier. Without external pretraining, MST reaches 66.5/40.7/36.1/27.9% accuracy on SEED/IV/V/VII under strict LOSO (paired gains of 3.15–4.15 pp, Holm-corrected, over the strongest pretrained baselines) and 17.52% on FACED 9-class (+1.15 pp over BIOT; κ≈0.06). Validation includes all-15-subject component ablations, cross-architecture context controls, a past-only variant, and a statistical apparatus (subject-paired bootstrap, sign-flip tests, Holm correction, oracle envelope) that exceeds the subfield norm.

Critical strengths
  1. Evaluation rigor well above the community norm. Strict LOSO without calibration or validation-based selection, final-checkpoint scoring, 66 subject-dataset folds, subject-paired bootstrap CIs with exact sign-flip tests and Holm correction, plus an oracle-envelope analysis (App. C.2–C.3). This matters because it makes the paper's negative findings (non-significant controls) as trustworthy as its positive ones, and demonstrates a protocol the subfield should adopt.

  2. Transparent, well-scoped claims. Comparisons are explicitly framed as complete training recipes (§4.1); non-significant results are reported in full (context added to CSBrain: +0.83 pp, CI [0.00, 1.69], p=0.1738); diagnostics are backed by full-cohort statistics. This honesty materially raises my confidence in the authors' numbers even where I dispute their attribution.

Critical weaknesses
  1. The headline comparison is confounded, so the titular mechanism is not isolated. The augmentation stack is MST-only (App. B.3) while its self-reported contribution (−6.4 pp, Table 3) exceeds the headline margin over the strongest pretrained baseline (+4.07 pp on SEED, Table 10); inference cost is roughly an order of magnitude higher than most baselines (31.7 vs 1.2–2.7 GFLOPs, Table 16) and the pivotal "matched-context" control confounds a 5× training-step difference (CSBrain 10k vs MST 50k, App. C.4). Consequently the main-table gains cannot be attributed to context-aware tokenization rather than to augmentation and budget. This is consequential because it targets the paper's central contribution claim.

  2. The central mechanism lacks support outside MST. The only controlled test of context per se — adding the same context to CSBrain — is non-significant (+0.83 pp, p=0.1738, Table 12), and the ablation drops (Table 3, where the Morlet front-end at −11.8 outcontributes the context branch at −5.6) carry no paired CIs or tests. The paper's title and its own evidence therefore point at different components, and the mechanistic claim currently rests on an untested within-recipe ablation.

Questions for authors
  1. If LaBraM and CSBrain are trained with the same four-augmentation stack (or MST is compared without augmentation), how much of the 4.07 pp SEED margin survives?
  2. Does the context-matched comparison (MST 66.5 vs CSBrain 60.5, Table 12) hold when CSBrain is trained for 50,000 steps, or MST for 10,000?
  3. Can you provide paired CIs/sign-flip tests for the Table 3 ablations, in particular the −5.6 pp neighbor-reference effect (subject SD ≈ 8 in the background)?
  4. Given Proposition 1 equates Morlet tokens with (a shift of) differential entropy, why is there no five-band DE + shallow-classifier baseline, and how does 66.5% relate to published DE-based LOSO results on SEED?
  5. How were the model-specific budgets in Table 6 (e.g., LaBraM 50k steps on SEED vs 10k on SEED-IV/V) "fixed in advance"?
  6. On FACED (κ=0.0625 best, near chance), can confusion matrices show which emotion pairs drive the +1.15 pp gain and rule out class-prior effects?
5 comments

We thank the reviewer for the detailed questions.

Concern — W1 and Q1. How much of the 4.07-point margin remains with matched augmentation, or with augmentation removed from MST?

Response. All our experiments share the same data processing pipeline, including the same data augmentation. The wording in Appendix B.3 was meant to say that the augmentation was designed specific for MST and may not be compatible with some of the the baselines. We will correct the wording in Appendix B.3 to ensure there is no confusion.

Concern — W1 and Q2. Does the matched-context comparison survive equal training steps, and how should MST's higher inference cost be interpreted?

Response. The training budgets follow the same loss-based stopping criterion for all models. For each model, we use one seed to determine when training loss stops decreasing or begins to rise, then fix that number of steps for the remaining seeds. We also give every model the same hyperparameter-tuning budget and select its learning rate and batch size based on training loss. Different step counts therefore reflect when each model meets the shared stopping criterion.

On SEED, this procedure gives 50,000 steps for MST and LaBraM, and 10,000 steps for CSBrain, EEGPT, and CBraMod (Table 6 and Appendix B.4). We will clarify how training loss determines both the hyperparameters and the training duration.

Morlet wavelet convolutions are the main computational cost of MST. Excluding these convolutions, the remaining model requires approximately 7.329 GFLOPs per prediction. In proctical deployments, these fixed convolution can be largely accelerated though specialized hardware or optimzied convolution libraries.

Concern — W2 and Q3. Are the Table 3 component effects statistically supported, and does the non-significant CSBrain result undermine a general context claim?

Response. The following two-sided paired t-tests compare full MST with each ablation in all subjects(n = 15, df = 3).

AblationMean paired accuracy drop (pp)Unadjusted p-value
STFT front-end11.4500.0081
w/o neighbor-reference pathway5.3250.0243
Shared spatial projection2.8750.0418
w/o all augmentation5.8750.0574
w/o phase perturbation2.9250.0383
w/o band-specific noise3.1750.0310
w/o channel dropout3.6750.0490
w/o wavelet time roll5.0000.0147

The experiment that simply adding context to CSBrain in Table 12 leads to an unsignificant increase in accuracy is expected. The key point of this experiment is meant to show that simply adding context to a general EEG encoder will not work. We also need other components that we proposed for MST. Context aware is not meant to be the only contribution of this work, and we also need Morlet spectral tokenization and frequency-specific spatial projection to make effective use of the neighboring EEG.

Concern — Q4. If Morlet tokens are related to differential entropy, why omit five-band DE with a shallow classifier, and how does 66.5% compare with published DE results?

Response. The DE connection discussed in the paper is meant to support the motivation and interpretability of our spectral features. Five-band DE summarizes each EEG window into a small set of band-level features, collapsing within-band spectral differences and within-window temporal variation. Instead, our approach preserves a finer time–frequency representation and combines local spectral referencing with learned frequency-specific spatial projections.

Published DE-based evaluations can use a different adaptation protocol. RGNN [1] uses unlabeled held-out-subject data for domain-adversarial training (§IV-B), and the DE+MLP baseline in CLISA [2] updates normalization statistics using incoming test-subject data (§§4.4 and 4.6). MST uses only training subjects for learning and for computing held-out normalization statistics on SEED. These published scores therefore do not provide a matched comparison with our results.

For reference, [3] with a similar protocol as ours reports the following SEED LOSO accuracies. The foundation-model results use full-parameter fine-tuning.

Model[3] accuracy (%)Our Table 1 accuracy (%)
LaBraM52.2362.4
EEGPT48.7456.2
CBraMod53.6157.0
BIOT49.0459.0
CSP + LDA48.91—
MST (ours)—66.5

Concern — Q5. What determined the model-specific budgets before evaluation?

Response. For each model, we use one seed to determine when training loss stops decreasing or begins to rise, then fix that number of steps for the remaining seeds. We also select learning rate and batch size using training loss, with the same tuning budget for every model. The step counts differ because the models reach the shared stopping criterion at different times. We will clarify this procedure alongside Table 6.

Concern — Q6. Which emotion pairs explain the FACED gain, and can class-prior effects be ruled out?

Response. FACED [4] is not class-balanced in our evaluation. Neutral accounts for 14.29% of windows, compared with 10.71% for each other emotion, and both MST and BIOT predict Neutral most often.

Our confusion-matrix analysis shows that MST improves balanced accuracy from 15.28% to 16.37%. Excluding Neutral, mean recall across the other eight emotions still increases from 11.49% to 12.35%. The largest recall gains are for Amusement, Disgust, and Neutral. For example, Amusement misclassified as Anger or Fear decreases by 2.1 and 1.9 percentage points, respectively. The improvement therefore extends beyond the majority class and persists when classes receive equal weight.

[1] Peixiang Zhong, Di Wang, and Chunyan Miao. EEG-Based Emotion Recognition Using Regularized Graph Neural Networks. arXiv:1907.07835, 2019.

[2] Xinke Shen et al. Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition. IEEE Transactions on Affective Computing, 2022 (online publication).

[3] Dingkun Liu et al. EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models. arXiv:2601.17883, 2026.

[4] Jingjing Chen et al. A Large Finer-grained Affective Computing EEG Dataset. Scientific Data, 10:740, 2023.

Thank you for the response. However, the rebuttal does not answer my core question: whether the baselines received data augmentation, and if so, how it was implemented. The paper only describes the augmentation scheme for MST, and the rebuttal itself concedes that the augmentation "was designed specific for MST and may not be compatible with some of the baselines." How augmentation was actually implemented for the comparison baselines therefore remains unspecified. Moreover, I suspect that the 5× budget difference for the pretrained baselines may itself stem from this augmentation asymmetry: the "same loss-based stopping criterion for all models" mentioned in the rebuttal would, in that case, systematically favor the augmented, higher-capacity model (MST).

Reviewer_B6AfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the Morlet Spectral Transformer for cross-subject EEG emotion recognition. MST summarizes neighboring same-trial windows using averaged Morlet log-amplitude spectra and conditions each target-window token on both its absolute spectrum and its reference-relative residual. Frequency-specific electrode projections combine these views before Transformer attention, allowing contextual information to enter a fixed target-token grid. The method is evaluated under subject-disjoint protocols on four SEED-family datasets and FACED, with comparisons against pretrained and randomly initialized EEG encoders, component ablations, and context controls.

Critical strengths
  1. The paper studies a clear representation-learning question and proposes a simple, interpretable solution: combining absolute spectral activity with deviations from a local spectral reference before attention.
  2. The empirical evaluation is broad and generally careful. It includes five datasets, subject-disjoint protocols, three training seeds, all-subject SEED ablations, past-only context, and matched-span comparisons.
Critical weaknesses
  1. Neighboring windows from the same trial may share the stimulus, recording background, and emotion label. The current evaluation does not fully establish whether the gains arise from improved cross-subject representation or from access to trial-specific contextual information that strongly correlates with the target label.
  2. MST combines several established ingredients—Morlet features, reference subtraction, frequency-specific spatial filtering, contextual aggregation, and a Transformer. The paper needs stronger evidence that their particular tokenization principle generalizes beyond the SEED-style trial structure.
Questions for authors

1.Please report a control in which context windows are drawn from another trial of the same subject or are shuffled among trials while preserving context size. This would test whether gains require temporally relevant context rather than merely additional same-distribution EEG. 2.Can the authors provide the complete past-only results on all evaluated datasets? Since the symmetric context may include future windows, a consistent past-only evaluation is important for the claimed applicability to online decoding. 3.Please compare MST with a parameter- and augmentation-matched baseline that receives the same target and neighboring windows through simple concatenation or late feature fusion. This would isolate the contribution of the proposed pre-attention token conditioning. 4.Are the improvements statistically significant across held-out subjects after accounting for the multiple datasets and baseline comparisons? Please report paired confidence intervals or corrected tests for the principal comparisons. 5.Can the authors evaluate at least one cross-session or cross-dataset transfer setting, or explain why the learned improvement should extend beyond the same dataset and stimulus collection? Such evidence would materially strengthen the representation-learning contribution.

1 comment

We thank the reviewer for the careful assessment.

Concern — W1 and Q1. Same-trial neighbors share stimulus, background, and emotion labels. Would different-trial or shuffled context reveal whether temporally relevant context is necessary?

Response. Distinguishing the gain is from "improved cross-subject representation or from access to trial-specific contextual information" is out-of-the-scope of this paper, and having access to contextual information is an intended part of MST. Our cross-subject evaluation aims to test whether the model can use this information to decode emotions in unseen subjects.

This design is motivated by removing the temporal correlation while preserving the original spectral information, thus using the neighboring EEG as a natural choice. Thus, different-trial or cross-trial shuffled context is not a resonable local reference to remove the temporal correlation. Additionally, our ablation alreadly shows that removing this design reduces mean accuracy by 5.6 percentage points (Table 3), directly showing the effectiveness of the complete neighbor-reference design. Figure 2 also shows that the temporal correlation is reduced.

Concern — Q2. Please report past-only results on every dataset and clarify the use of future windows.

Response. Past-only results are very close to the centered results, as shown in the Table 4. It is reasonable because the key is to remove temporal correlation of EEG, thus both centered and past-only should have worked. The requested full past-only evaluation is as follows.

DatasetMST, centeredMST, past-only
SEED66.5 ± 8.166.1 ± 8.2
SEED-IV40.7 ± 5.139.8 ± 5.0
SEED-V36.1 ± 4.535.5 ± 4.3
SEED-VII27.9 ± 3.927.2 ± 3.5
FACED17.5217.23

Concern — Q3. Is pre-attention conditioning better than simpler fusion when context, parameters, and augmentation are matched?

Response. This control experiement is performed with results shown in Table 4. We compare CSBrain with MST with a similar parameter count. This comparison gives both models the same 22-second EEG context and uses the same augmentation. CSBrain concatenates the target and ten neighboring windows and achieves 60.5% accuracy, compared with 66.5% for MST. The paired gain is 5.99 percentage points [3.90, 8.14] (Tables 4 and 12).

The training budgets follow the same loss-based stopping criterion for all models. For each model, we use one seed to determine when training loss stops decreasing or begins to rise, then fix that number of steps for the remaining seeds. We also give every model the same hyperparameter-tuning budget and select its learning rate and batch size based on training loss. Different step counts therefore reflect when each model meets the shared stopping criterion.

Concern — Q4. Are the principal improvements significant across subjects after multiple-comparison correction?

Response. Yes. The gains over the strongest evaluated pretrained baseline on each SEED dataset remain significant after Holm correction (Table 10).

DatasetReferenceAccuracy gain, pp [95% CI]Holm-adjusted p
SEEDLaBraM4.07 [1.85, 6.19]0.00458
SEED-IVCSBrain4.15 [2.35, 6.11]0.00244
SEED-VLaBraM3.58 [1.78, 5.65]0.00275
SEED-VIILaBraM3.15 [1.64, 4.70]0.00252

We use paired-subject bootstrap intervals and exact two-sided sign-flip tests. The four pretrained comparisons, four from-scratch comparisons, and five FACED comparisons form separate Holm correction groups (Appendices C.2 and C.6). All four from-scratch gains remain significant, with adjusted p ≤ 0.000916. On FACED, the gain over BIOT is 1.15 points [0.66, 1.64], with adjusted p = 1.08 × 10⁻⁵ (Table 15).

Concern — W2 and Q5. Does the principle extend beyond the SEED trial structure, including cross-session or cross-dataset transfer?

Response. Our evaulation alreadly extends beyond the SEED-family datasets. Our method is tested with the FACED, and the results are reported in Table 2. FACED has a different acquisition setup, with 123 subjects and nine emotion classes. MST achieves 17.52% accuracy versus 16.37% for the baseline under subject-disjoint evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.