- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper explores how to improve the generalization of EEG representations, thereby enhancing their performance in cross-subject, cross-dataset, and cross-elicitation settings. The authors propose a two-stage post-training method called "Progressive Affective Contrastive Enhancement (PACE)," which first applies cross-subject contrastive supervision within a dataset, and subsequently performs contrastive supervision on cross-dataset and cross-paradigm samples. Experiments are conducted based on a "leave-one-dataset-out" evaluation task and four EEG foundation model architectures, with PACE evaluated through both full fine-tuning and linear probing. However, this work suffers from limited novelty, a less than compelling methodological contribution, and insufficient experimental evidence regarding cross-elicitation settings.
Critical strengths
- A perspective for improving generalization performance across elicitation methods is proposed;
- Sufficient experiments are conducted on the basic evaluation tasks.
Critical weaknesses
- The claimed originality at the level of knowledge discovery is limited. The authors claim that "this study reveals a gap between subject-specific affective information and transferable organizational structures." However, since EEG signals are inherently highly individualized affective signals, this conclusion appears to be a community consensus rather than an original finding.
- The methodological innovation is limited. The proposed method consists of two stages of positive-negative pair selection and contrastive learning, and it is difficult to distinguish the proposed method from existing contrastive approaches.
Questions for authors
- PACE first performs two-stage contrastive learning on M source datasets, and then carries out fine-tuning and linear probing optimization using the training and validation sets of the target dataset. If there is only one target dataset, why not directly fine-tune on the target dataset instead of first introducing M datasets for contrastive learning? In particular, given that these source datasets may differ in their elicitation methods, why is post-training with heterogeneous datasets or datasets with different elicitation methods beneficial? Although the ablation experiments help to alleviate this concern, a more reasonable explanation at the principled level is still needed.
- There are M source datasets available for training, and these datasets may contain different emotion categories. How, then, is the pretraining of the backbone model carried out?
- This paper investigates generalization across elicitation settings. Does "cross-elicitation" specifically refer to "differences between EEG samples of the same emotion category obtained through different emotion elicitation methods (video or music)"? If so, how did the authors control the variables? The datasets employed appear to contain only a single elicitation method, so "cross-elicitation" would then mean cross-dataset. However, subjects may differ across datasets; how can the authors determine that the differences between EEG samples are caused by the elicitation method rather than by subject differences?
- There appears to be a lack of sensitivity analysis experiments on hyperparameters, such as loss weights and representation dimensions.
- During cross-dataset training, if two datasets contain different emotion categories, a predefined emotion category mapping is required. How sensitive is the proposed method to the predefined emotion category mapping?
4 comments
Response to Question 2: Handling Different Emotion Categories across Datasets
PACE starts from the official pretrained weights of each EEG foundation model; it does not pretrain the backbone from scratch on the source emotion datasets. Differences in emotion categories are handled during our affective post-training.
Stage 1 performs cross-subject contrastive learning within each dataset using dataset-specific emotion labels, so a shared label space across datasets is not required. In Stage 2, we introduce a partial label-matching rule: categories representing the same emotion concept form positive pairs, clearly distinct categories form negative pairs, and ambiguous relations are excluded from the cross-dataset objective (Table 11, detailed in appendix B.4.1 and B.4.2). Thus, PACE aligns compatible emotion relations without requiring all datasets to share an identical emotion taxonomy.
Response to Question 3: Cross-Elicitation Generalization and Subject Variability
Cross-elicitation refers to the transferability of emotion-related EEG representations across elicitation paradigms. We agree that our main evaluation combines differences in elicitation paradigm, dataset, and subject identity, and therefore cannot isolate the effect of elicitation paradigm alone. Nevertheless, this evaluation is practically relevant, as real-world deployment often requires generalization across these jointly varying conditions.
To address subject variability, we conducted an additional within-subject cross-elicitation retrieval experiment on EmoEEG-MC, which includes video-viewing and emotion-imagery recordings from the same participants. For each participant, we test whether an imagery EEG trial can retrieve video-viewing trials of the same emotion, and vice versa. All queries and candidates belong to the same participant, holding subject identity fixed.
We rank trials from the other condition by cosine similarity between frozen EEG representations, treating trials with the same emotion label as relevant matches. Both EmoEEG-MC conditions are excluded from PACE post-training, and no downstream classifier is trained. We report AUC, which measures how well same-emotion candidates rank above different-emotion candidates, and Hit@1, the proportion of queries whose top-ranked candidate has the same emotion label.
Within-subject cross-elicitation EEG-to-EEG retrieval results (%,CBraMod). I→V: imagery to video; V→I: video to imagery.
| Model | I→V AUC | I→V Hit@1 | V→I AUC | V→I Hit@1 |
|---|---|---|---|---|
| Chance | 50.00 | 14.29 | 50.00 | 14.29 |
| Backbone | 51.91 | 15.53 | 52.03 | 16.07 |
| S1 | 51.78 | 17.61 | 52.37 | 17.61 |
| PACE | 52.61 | 18.75 | 52.48 | 18.60 |
PACE achieves the highest scores for both metrics in both directions. Compared with the pretrained backbone, Hit@1 improves by 3.22 percentage points for imagery→video and 2.53 percentage points for video→imagery. Although absolute performance remains modest, these results support improved affective correspondence across elicitation conditions with subject identity held fixed, complementing our broader cross-dataset evaluation.
Response to Question 4: Sensitivity to Loss Weights and Projection Dimension
We thank the reviewer for this suggestion. We conducted additional hyperparameter sensitivity analyses on CBraMod under full fine-tuning (FT), examining Stage 1 loss weights, Stage 2 loss weights, and projection dimensions.
Stage 1 loss weights. We varied the balance between stimulus/temporal correspondence (T) and native emotion-label supervision (E), evaluating all twelve held-out targets. Avg12 denotes the mean over all twelve target datasets.
| T/E weights | Avg12 BAcc (%) |
|---|---|
| 0.25 / 0.75 | 38.92 |
| 0.50 / 0.50 | 38.10 |
| 0.75 / 0.25 | 38.73 |
The average scores span 0.82 percentage points (pp). Subsequent sensitivity experiments use T/E = 0.25/0.75. The main setting uses 0.50/0.50 by default.
Stage 2 loss weights. We express the objective as
where in this sensitivity analysis, and and denote the average same-paradigm and cross-paradigm alignment losses, respectively. Thus, controls the total weight of retained within-dataset supervision, while controls how the cross-dataset weight is divided.
We tested to examine lower, default, and higher within-dataset supervision shares, and to vary the balance between cross-paradigm and same-paradigm alignment. The main setting corresponds to and , but uses equal T/E weights.
To limit computational cost, we evaluated the nine configurations on three held-out targets, one for each elicitation paradigm: FACED (video), MEEG (listening), and ImaginedEmotion (imagery). Avg3 denotes their mean BAcc.
| Avg3 BAcc (%) | |||
|---|---|---|---|
| 42.59 | 42.70 | 42.58 | |
| 42.45 | 42.05 | 42.39 | |
| 42.41 | 42.49 | 42.91 |
Across this grid, Avg3 BAcc ranges from 42.05% to 42.91%, a spread of only 0.86 pp.
Projection dimension. We evaluated output dimensions of 64, 128, and 256 across all twelve targets, fixing T/E = 0.25/0.75 and for this analysis.
| Dimension | Avg12 BAcc (%) |
|---|---|
| 64 | 39.82 |
| 128 | 40.31 |
| 256 | 40.46 |
The average scores span 0.64 pp, suggesting limited sensitivity to projection dimension under this weight configuration.
Overall, these analyses show modest variation in average performance over the tested ranges. Detailed per-dataset results and configurations are provided in Appendix G4.
Response to Question 5: Sensitivity to Predefined Emotion Relations
We thank the reviewer for this question, which is also addressed in our response to Reviewer bmQd (Weakness 4). We conducted an additional experiment comparing PACE's Positive–Negative–Ambiguous–Unmapped (PNAU) relation modeling with a simplified Positive–Negative (PN) strategy that treats identical label names as positives and differently named labels as negatives. Across twelve held-out datasets under CBraMod FT, PNAU improves average BAcc from 39.14% to 40.03% (+0.89 pp), outperforming PN on 10/12 targets, with notable gains on FACED (+2.68 pp) and MC-V (+1.18 pp). These results demonstrate the value of selective semantic supervision that excludes ambiguous and unmapped emotion relations when aligning heterogeneous emotion taxonomies. Please refer to our response to Reviewer bmQd (Weakness 4) for the complete experimental results and analysis.
Follow-up on our response to Reviewer Ho7L
Thank you again for your valuable feedback. As the discussion period nears its end, we wanted to follow up on whether our previous responses and revised manuscript have addressed your concerns. Please let us know if you have any further questions or need additional clarification. We would be glad to discuss any remaining points.
Response to Weakness 1: Characterizing the Affective Generalization Boundaries of EEG Foundation Models
Our contribution is to characterize the generalization boundaries of EEG foundation models rather than simply denoting the subject varaibiity in EEG. EEG foundation models are expected to learn generalizable representations, but their genelizability on emotion recognition has rarely been explored. Here, we systematically investigate whether EEG foundation models preserve emotion-related structures within subjects, across subjects, across datasets, and across elicitation paradigms. This is not trivial, as we have little knowledge about whether foundation models learn reserved emotion structures across these settings.
We evaluate frozen representations from four EEG foundation models using a unified emotion-relation ranking protocol. As shown in Figure 1(a), average performance decreases from 57.23% within subjects to 51.85% across subjects remaining above th 50% chance level, and approach chance across datasets (50.32%) and elicitation paradigms (49.87%). Figure 1(a) reports statistical comparisons against chance with Holm correction. Appendix E.1 describes uncertainty estimation using 1,000 query-subject cluster-bootstrap replicates.This consistent pattern across backbones indicates that emotion-related structure accessible within subjects does not necessarily remain comparable across broader transfer settings.
Our contribution is therefore to quantify the limitation in pretrained foundation models for emotion representation. These findings motivate affective alignment beyond individual subjects, beginning with more reliable within-dataset supervision before extending to heterogeneous datasets and elicitation contexts. PACE follows this progression, with stage ablations providing empirical support for the design.
Response to Weakness 2: The innovation of Progressive Affective Post-Training for EEG Foundation Models
Affective post-training of EEG foundation models is important yet underexplored. PACE contributes a post-training framework for progressively learning from affective supervision across subjects and datasets. Its central design principle is to expand alignment according to the reliability of the available relations: first establishing cross-subject affective structure within datasets, and then extending this structure across heterogeneous datasets and elicitation contexts while retaining within-dataset supervision.
This distinguishes PACE from directly applying contrastive learning to pooled heterogeneous datasets: Within-dataset alignment provides a starting point for alignment across datasets and elicitation paradigms, and its supervision is retained to preserve the learned affective structure. Our CBraMod FT ablation makes this progression explicit: S2-only achieves 32.40% Avg12 BAcc, below the pretrained backbone with direct target fine-tuning (32.98%). S1 reaches 38.10%, while S1→S2 further improves to 40.03% (+1.93 pp over S1, improving 11/12 targets). These results suggest that how affective supervision is organized and introduced matters beyond simply adding more datasets or applying a contrastive objective. These findings may also inform EEG post-training on heterougeneous datasets beyond emotion recognition.
Response to Question 1: Why Multi-Source Affective Post-Training?
Our aim is to learn transferable emotion representations across datasets and elicitation paradigms. Fine-tuning on a single target dataset may exploit dataset-specific associations—such as recording setup and experimental protocol—that may not hold in other contexts. Even when only one target dataset is of interest, multi-source post-training provides additional supervision for learning emotion-related relations beyond those observable in the target training set. Improved linear-probing performance on held-out datasets, with the post-trained backbone frozen, supports the transferability of the resulting representations.
Although source datasets are heterogeneous, their diversity in subjects, stimuli, and elicitation paradigms provides an opportunity to learn emotion representations that are less dependent on any single context. At the same time, differences in recording conditions and label taxonomies can complicate alignment. To tackle the problem, PACE progressively aligns representations across subjects within datasets and then across datasets. Compared with directly adapting pretrained EEG FMs to each target, PACE improves Avg12 BAcc by 2.66–7.05 pp under FT and 3.52–5.77 pp under LP across four backbones, with gains in 43/48 backbone–target comparisons in each setting. These improvements demonstrate the substantial benefit of multi-source affective post-training beyond target-only adaptation.