acceptodds
Under review as a conference paper at ICLR 2027

PACE: PROGRESSIVE AFFECTIVE ALIGNMENT FOR EEG FOUNDATION MODELS

Abstract

EEG foundation models can encode useful affective information, but their representations are not necessarily organized for transfer across heterogeneous subjects, datasets, and elicitation contexts. We diagnose frozen representations from four EEG foundation models and find that emotion relations are substantially more stable within subjects than across heterogeneous datasets and elicitation contexts, revealing a gap between recoverable affective information and transferable affective organization. We introduce Progressive Affective Contrastive Enhancement (PACE), an affective post-training framework that progressively expands affective alignment from reliable within-dataset relations to heterogeneous datasets and elicitation contexts. PACE first learns cross-subject affective structure within individual datasets, then extends alignment across datasets and elicitation contexts while retaining within-dataset supervision. We evaluate PACE on twelve completely held-out target datasets and four foundation backbones. PACE improves average full-fine-tuning balanced accuracy by 2.66–7.05 percentage points and linear-probing performance by 3.52–5.77 points. Moreover, PACE outperforms CE, SupCon, and VREx post-training approaches on all twelve target datasets. Stage ablations support the progressive strategy, and frozen-representation analyses show improved emotion-relation geometry across all four backbones. Together, these results establish affective post-training as an effective bridge between generic EEG pretraining and transferable affective representations across heterogeneous datasets and elicitation contexts.Code is available at https://anonymous.4open.science/r/ICLR2027_PACE/

Primary area:
applications to neuroscience & cognitive science
Keywords:
Electroencephalography (EEG), EEG Emotion Recognition, EEG Foundation Models, Cross-Context EEG Decoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ho7LOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores how to improve the generalization of EEG representations, thereby enhancing their performance in cross-subject, cross-dataset, and cross-elicitation settings. The authors propose a two-stage post-training method called "Progressive Affective Contrastive Enhancement (PACE)," which first applies cross-subject contrastive supervision within a dataset, and subsequently performs contrastive supervision on cross-dataset and cross-paradigm samples. Experiments are conducted based on a "leave-one-dataset-out" evaluation task and four EEG foundation model architectures, with PACE evaluated through both full fine-tuning and linear probing. However, this work suffers from limited novelty, a less than compelling methodological contribution, and insufficient experimental evidence regarding cross-elicitation settings.

Critical strengths
  1. A perspective for improving generalization performance across elicitation methods is proposed;
  2. Sufficient experiments are conducted on the basic evaluation tasks.
Critical weaknesses
  1. The claimed originality at the level of knowledge discovery is limited. The authors claim that "this study reveals a gap between subject-specific affective information and transferable organizational structures." However, since EEG signals are inherently highly individualized affective signals, this conclusion appears to be a community consensus rather than an original finding.
  2. The methodological innovation is limited. The proposed method consists of two stages of positive-negative pair selection and contrastive learning, and it is difficult to distinguish the proposed method from existing contrastive approaches.
Questions for authors
  1. PACE first performs two-stage contrastive learning on M source datasets, and then carries out fine-tuning and linear probing optimization using the training and validation sets of the target dataset. If there is only one target dataset, why not directly fine-tune on the target dataset instead of first introducing M datasets for contrastive learning? In particular, given that these source datasets may differ in their elicitation methods, why is post-training with heterogeneous datasets or datasets with different elicitation methods beneficial? Although the ablation experiments help to alleviate this concern, a more reasonable explanation at the principled level is still needed.
  2. There are M source datasets available for training, and these datasets may contain different emotion categories. How, then, is the pretraining of the backbone model carried out?
  3. This paper investigates generalization across elicitation settings. Does "cross-elicitation" specifically refer to "differences between EEG samples of the same emotion category obtained through different emotion elicitation methods (video or music)"? If so, how did the authors control the variables? The datasets employed appear to contain only a single elicitation method, so "cross-elicitation" would then mean cross-dataset. However, subjects may differ across datasets; how can the authors determine that the differences between EEG samples are caused by the elicitation method rather than by subject differences?
  4. There appears to be a lack of sensitivity analysis experiments on hyperparameters, such as loss weights and representation dimensions.
  5. During cross-dataset training, if two datasets contain different emotion categories, a predefined emotion category mapping is required. How sensitive is the proposed method to the predefined emotion category mapping?
4 comments

Response to Weakness 1: Characterizing the Affective Generalization Boundaries of EEG Foundation Models

Our contribution is to characterize the generalization boundaries of EEG foundation models rather than simply denoting the subject varaibiity in EEG. EEG foundation models are expected to learn generalizable representations, but their genelizability on emotion recognition has rarely been explored. Here, we systematically investigate whether EEG foundation models preserve emotion-related structures within subjects, across subjects, across datasets, and across elicitation paradigms. This is not trivial, as we have little knowledge about whether foundation models learn reserved emotion structures across these settings.

We evaluate frozen representations from four EEG foundation models using a unified emotion-relation ranking protocol. As shown in Figure 1(a), average performance decreases from 57.23% within subjects to 51.85% across subjects remaining above th 50% chance level, and approach chance across datasets (50.32%) and elicitation paradigms (49.87%). Figure 1(a) reports statistical comparisons against chance with Holm correction. Appendix E.1 describes uncertainty estimation using 1,000 query-subject cluster-bootstrap replicates.This consistent pattern across backbones indicates that emotion-related structure accessible within subjects does not necessarily remain comparable across broader transfer settings.

Our contribution is therefore to quantify the limitation in pretrained foundation models for emotion representation. These findings motivate affective alignment beyond individual subjects, beginning with more reliable within-dataset supervision before extending to heterogeneous datasets and elicitation contexts. PACE follows this progression, with stage ablations providing empirical support for the design.

Response to Weakness 2: The innovation of Progressive Affective Post-Training for EEG Foundation Models

Affective post-training of EEG foundation models is important yet underexplored. PACE contributes a post-training framework for progressively learning from affective supervision across subjects and datasets. Its central design principle is to expand alignment according to the reliability of the available relations: first establishing cross-subject affective structure within datasets, and then extending this structure across heterogeneous datasets and elicitation contexts while retaining within-dataset supervision.

This distinguishes PACE from directly applying contrastive learning to pooled heterogeneous datasets: Within-dataset alignment provides a starting point for alignment across datasets and elicitation paradigms, and its supervision is retained to preserve the learned affective structure. Our CBraMod FT ablation makes this progression explicit: S2-only achieves 32.40% Avg12 BAcc, below the pretrained backbone with direct target fine-tuning (32.98%). S1 reaches 38.10%, while S1→S2 further improves to 40.03% (+1.93 pp over S1, improving 11/12 targets). These results suggest that how affective supervision is organized and introduced matters beyond simply adding more datasets or applying a contrastive objective. These findings may also inform EEG post-training on heterougeneous datasets beyond emotion recognition.

Response to Question 1: Why Multi-Source Affective Post-Training?

Our aim is to learn transferable emotion representations across datasets and elicitation paradigms. Fine-tuning on a single target dataset may exploit dataset-specific associations—such as recording setup and experimental protocol—that may not hold in other contexts. Even when only one target dataset is of interest, multi-source post-training provides additional supervision for learning emotion-related relations beyond those observable in the target training set. Improved linear-probing performance on held-out datasets, with the post-trained backbone frozen, supports the transferability of the resulting representations.

Although source datasets are heterogeneous, their diversity in subjects, stimuli, and elicitation paradigms provides an opportunity to learn emotion representations that are less dependent on any single context. At the same time, differences in recording conditions and label taxonomies can complicate alignment. To tackle the problem, PACE progressively aligns representations across subjects within datasets and then across datasets. Compared with directly adapting pretrained EEG FMs to each target, PACE improves Avg12 BAcc by 2.66–7.05 pp under FT and 3.52–5.77 pp under LP across four backbones, with gains in 43/48 backbone–target comparisons in each setting. These improvements demonstrate the substantial benefit of multi-source affective post-training beyond target-only adaptation.

Response to Question 2: Handling Different Emotion Categories across Datasets

PACE starts from the official pretrained weights of each EEG foundation model; it does not pretrain the backbone from scratch on the source emotion datasets. Differences in emotion categories are handled during our affective post-training.

Stage 1 performs cross-subject contrastive learning within each dataset using dataset-specific emotion labels, so a shared label space across datasets is not required. In Stage 2, we introduce a partial label-matching rule: categories representing the same emotion concept form positive pairs, clearly distinct categories form negative pairs, and ambiguous relations are excluded from the cross-dataset objective (Table 11, detailed in appendix B.4.1 and B.4.2). Thus, PACE aligns compatible emotion relations without requiring all datasets to share an identical emotion taxonomy.

Response to Question 3: Cross-Elicitation Generalization and Subject Variability

Cross-elicitation refers to the transferability of emotion-related EEG representations across elicitation paradigms. We agree that our main evaluation combines differences in elicitation paradigm, dataset, and subject identity, and therefore cannot isolate the effect of elicitation paradigm alone. Nevertheless, this evaluation is practically relevant, as real-world deployment often requires generalization across these jointly varying conditions.

To address subject variability, we conducted an additional within-subject cross-elicitation retrieval experiment on EmoEEG-MC, which includes video-viewing and emotion-imagery recordings from the same participants. For each participant, we test whether an imagery EEG trial can retrieve video-viewing trials of the same emotion, and vice versa. All queries and candidates belong to the same participant, holding subject identity fixed.

We rank trials from the other condition by cosine similarity between frozen EEG representations, treating trials with the same emotion label as relevant matches. Both EmoEEG-MC conditions are excluded from PACE post-training, and no downstream classifier is trained. We report AUC, which measures how well same-emotion candidates rank above different-emotion candidates, and Hit@1, the proportion of queries whose top-ranked candidate has the same emotion label.

Within-subject cross-elicitation EEG-to-EEG retrieval results (%,CBraMod). I→V: imagery to video; V→I: video to imagery.

ModelI→V AUCI→V Hit@1V→I AUCV→I Hit@1
Chance50.0014.2950.0014.29
Backbone51.9115.5352.0316.07
S151.7817.6152.3717.61
PACE52.6118.7552.4818.60

PACE achieves the highest scores for both metrics in both directions. Compared with the pretrained backbone, Hit@1 improves by 3.22 percentage points for imagery→video and 2.53 percentage points for video→imagery. Although absolute performance remains modest, these results support improved affective correspondence across elicitation conditions with subject identity held fixed, complementing our broader cross-dataset evaluation.

Response to Question 4: Sensitivity to Loss Weights and Projection Dimension

We thank the reviewer for this suggestion. We conducted additional hyperparameter sensitivity analyses on CBraMod under full fine-tuning (FT), examining Stage 1 loss weights, Stage 2 loss weights, and projection dimensions.

Stage 1 loss weights. We varied the balance between stimulus/temporal correspondence (T) and native emotion-label supervision (E), evaluating all twelve held-out targets. Avg12 denotes the mean over all twelve target datasets.

T/E weightsAvg12 BAcc (%)
0.25 / 0.7538.92
0.50 / 0.5038.10
0.75 / 0.2538.73

The average scores span 0.82 percentage points (pp). Subsequent sensitivity experiments use T/E = 0.25/0.75. The main setting uses 0.50/0.50 by default.

Stage 2 loss weights. We express the objective as

L2=βLTE+(1−β)[ρL‾s+(1−ρ)L‾c]\mathcal{L} _ 2 = \beta\mathcal{L} _ {TE} + (1-\beta)[\rho\overline{\mathcal{L}} _ s + (1-\rho)\overline{\mathcal{L}} _ c]

where LTE=0.25LT+0.75LE\mathcal{L} _ {TE}=0.25\mathcal{L} _ T+0.75\mathcal{L} _ E in this sensitivity analysis, and L‾s\overline{\mathcal{L}} _ s and L‾c\overline{\mathcal{L}} _ c denote the average same-paradigm and cross-paradigm alignment losses, respectively. Thus, β\beta controls the total weight of retained within-dataset supervision, while ρ\rho controls how the cross-dataset weight is divided.

We tested β∈{1/6,1/3,1/2}\beta\in\{1/6,1/3,1/2\} to examine lower, default, and higher within-dataset supervision shares, and ρ∈{1/4,1/2,3/4}\rho\in\{1/4,1/2,3/4\} to vary the balance between cross-paradigm and same-paradigm alignment. The main setting corresponds to β=1/3\beta=1/3 and ρ=1/2\rho=1/2, but uses equal T/E weights.

To limit computational cost, we evaluated the nine configurations on three held-out targets, one for each elicitation paradigm: FACED (video), MEEG (listening), and ImaginedEmotion (imagery). Avg3 denotes their mean BAcc.

Avg3 BAcc (%)ρ=1/4\rho=1/4ρ=1/2\rho=1/2ρ=3/4\rho=3/4
β=1/6\beta=1/642.5942.7042.58
β=1/3\beta=1/342.4542.0542.39
β=1/2\beta=1/242.4142.4942.91

Across this grid, Avg3 BAcc ranges from 42.05% to 42.91%, a spread of only 0.86 pp.

Projection dimension. We evaluated output dimensions of 64, 128, and 256 across all twelve targets, fixing T/E = 0.25/0.75 and (β,ρ)=(1/2,3/4)(\beta,\rho)=(1/2,3/4) for this analysis.

DimensionAvg12 BAcc (%)
6439.82
12840.31
25640.46

The average scores span 0.64 pp, suggesting limited sensitivity to projection dimension under this weight configuration.

Overall, these analyses show modest variation in average performance over the tested ranges. Detailed per-dataset results and configurations are provided in Appendix G4.

Response to Question 5: Sensitivity to Predefined Emotion Relations

We thank the reviewer for this question, which is also addressed in our response to Reviewer bmQd (Weakness 4). We conducted an additional experiment comparing PACE's Positive–Negative–Ambiguous–Unmapped (PNAU) relation modeling with a simplified Positive–Negative (PN) strategy that treats identical label names as positives and differently named labels as negatives. Across twelve held-out datasets under CBraMod FT, PNAU improves average BAcc from 39.14% to 40.03% (+0.89 pp), outperforming PN on 10/12 targets, with notable gains on FACED (+2.68 pp) and MC-V (+1.18 pp). These results demonstrate the value of selective semantic supervision that excludes ambiguous and unmapped emotion relations when aligning heterogeneous emotion taxonomies. Please refer to our response to Reviewer bmQd (Weakness 4) for the complete experimental results and analysis.

AuthorsOpenReview ↗

Follow-up on our response to Reviewer Ho7L

Thank you again for your valuable feedback. As the discussion period nears its end, we wanted to follow up on whether our previous responses and revised manuscript have addressed your concerns. Please let us know if you have any further questions or need additional clarification. We would be glad to discuss any remaining points.

Reviewer_wagHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PACE, a two-stage contrastive post-training strategy to adapt EEG foundation models for cross-dataset emotion recognition. In Stage 1, each source dataset is assigned an independent cross-subject supervised contrastive loss with two types of positive pairs: signal segments from the identical stimulus or time position (T), and segments carrying the same original affective label (E). Stage 1 losses are retained in Stage 2, which further incorporates cross-dataset contrastive terms. For these cross-dataset terms, raw labels are mapped onto a manually defined vocabulary of nine concepts. A predefined matrix then determines which concept pairs count as positives, negatives, or excluded samples. These pairs are split into same-paradigm and cross-paradigm groups. Evaluations adopt a leave-one-dataset-out protocol covering twelve datasets spanning video viewing, audio listening, and mental imagery. Four backbone models are tested: CBraMod, REVE, CSBrain and LaBraM. Experiments include both fine-tuning and linear probing, repeated across six subject-split random seeds. Additional analyses contain relation-AUC computed on frozen features, alongside kNN classification and t-SNE visualizations.

Critical strengths

The evaluation suite is comprehensive and generally carefully designed. Twelve held-out target datasets, four backbones, six subject-level splits with reported standard deviations, plus both LP and FT protocols, exceed the scope of most existing studies in this field. The authors also jointly excluded MC-V and MC-I due to overlapping subjects, demonstrating awareness of data leakage risks.

On video-based datasets, PACE delivers large and consistent performance improvements across backbones. For instance, CBraMod FT accuracy rises from 53.35 to 64.09 on SEED, and from 33.45 to 54.40 on FACED. The ablation using Stage 2 alone yields an informative negative finding: directly aligning datasets without Stage 1 performs worse than the unmodified backbone. This outcome is valuable for researchers developing multi-dataset emotion pretraining pipelines.

Critical weaknesses
  1. Nearly all performance gains originate from Stage 1, while the cross-dataset component contributes only marginal benefits. The paper centers its narrative on cross-dataset and cross-context alignment. However, Table 16 shows Stage-1-only already reaches an average FT score of 38.10 across the 12 targets, compared with 40.03 for full PACE (+1.93). The advantage shrinks further to +0.87 under LP. Table 17 reports nearly identical AUROC between Stage-1-only and full PACE: FT (68.61 vs 68.53), LP (64.71 vs 64.79). Stage 1 resembles CLISA-style cross-subject alignment combined with per-dataset SupCon training. Hence the experimental evidence strongly supports the claim that within-dataset cross-subject contrastive post-training benefits foundation models, rather than the progressive cross-context alignment highlighted in the title and abstract.

  2. The core claim about learning across elicitation contexts lacks downstream empirical backing. Performance gains concentrate heavily on video datasets. Results on the two imagery targets hover near random chance. MC-I contains seven classes (chance level 14.29), with reported values ranging from 13.98 to 15.67. ImaginedEmotion has fifteen classes (chance 6.67), with results between 6.84 and 10.94. PACE changes these scores by −1.20 to +1.41, well within the listed standard deviations. Improvements on listening datasets remain modest. In my view, stating that PACE learns transferable affective structure across elicitation contexts overstates the observations in Tables 1 and 14.

  3. The relation-AUC diagnostic analysis is partially circular with small effect sizes. Appendix E states that this diagnostic reuses the same label mapping and positive/negative/ambiguous/unknown relation matrix used during training. As a result, CD and CP scores directly measure the exact relationships optimized in Stage 2. Numerically, PACE lifts cross-paradigm AUC from roughly 50 to 52.7–54.5 (50 corresponds to random guessing). While statistically detectable, this magnitude is small. Clarification is needed regarding whether gallery datasets for CD/CP are the post-training source sets.

4.Several manuscript claims overreach beyond tabular results. The abstract states PACE surpasses CE, SupCon and VREx post-training across all twelve target datasets. This conclusion only holds for balanced accuracy under CBraMod FT. For LP (Table 18), PACE underperforms the strongest baseline on MEEG (−1.41) and ImaginedEmotion (−0.56). For FT macro-F1 (Table 19), it falls short on EAV (−0.16) and MC-I (−0.46).

  1. Potential overlaps exist between source and target datasets. Multiple datasets originate from the same research groups, including SEED/SEED-IV/SEED-V/SEED-VII. FACED and EmoEEG-MC may also share origins. The largest performance jumps appear on target datasets with closely related counterparts in the source pool.
Questions for authors
  1. Could you add an ablation comparing T-only versus E-only objectives inside Stage 1, alongside relation-AUC numbers for the S1-only model? This would verify whether CD/CP gains truly come from Stage 2.

  2. Beyond the MC subsets, do any source-target dataset pairs share subjects or stimuli (SEED family; FACED versus EmoEEG-MC)?

  3. Were task-specific and emotion-pretrained baselines executed under exactly the same pipeline as foundation-model entries? Why were MC-V, MC-I and ImaginedEmotion excluded when calculating their Avg9 metric?

4 comments

Response to Weakness 1: Contributions of Stage 1 and Stage 2

We thank the reviewer for highlighting this distinction. We agree that Stage 1 accounts for most downstream classification gains. Below, we clarify the additional contribution of Stage 2 through downstream results and new stage-wise representation analyses.

Additional downstream gains. Stage 2 improves FT balanced accuracy over S1-only on 11 of 12 targets on CBraMod, raising the average from 38.10% to 40.03% (+1.93 pp). This accounts for approximately 27% of the total gain over the pretrained backbone (32.98% → 40.03%). The downstream AUROC remains essentially unchanged. AUROC measures score-ranking performance, whereas balanced accuracy reflects the correctness of final classification decisions across classes. Gains in balanced accuracy remain practically relevant despite the limited change in AUROC.

Stage-wise evidence for cross-dataset and cross-paradigm alignment. Beyond the stage-wise gains in downstream classification, cross-paradigm emotion relations are also a central focus of this work. Specifically, we conducted a stage-wise analysis of frozen representations using the relation-ranking protocol in Appendix E. Each target dataset was excluded from its corresponding model’s post-training sources, and no target-side fitting was performed. As shown below, Stage 2 increases average cross-paradigm Relation-AUC by approximately 1.59 pp over S1-only, comparable to the approximately 1.74 pp gain from Stage 1 over the backbone. The gains are positive across all four backbones.

Cross-paradigm Relation-AUC (%) and stage-wise gains (percentage points).

Method / Stage-wise gainCBraModLaBraMREVECSBrainAverage
Backbone49.9449.7550.7449.0549.87
S151.5052.0750.9551.9551.61
PACE52.6853.4354.4652.2153.20
Stage 1 gain: S1 − Backbone+1.56+2.32+0.20+2.90+1.74
Stage 2 gain: PACE − S1+1.18+1.36+3.51+0.26+1.59

We additionally evaluated cross-dataset relation ranking within the same elicitation paradigm. Stage 1 provides most of the improvement in this setting, and Stage 2 yields further gains across all four backbones, averaging +0.54 pp.

Cross-dataset Relation-AUC (%) and stage-wise gains (percentage points).

Method / stage-wise gainCBraModLaBraMREVECSBrainAverage
Backbone50.5950.0850.3250.2950.32
S153.3254.2054.9551.9753.61
PACE53.6854.5455.8352.5454.15
Stage 1 gain: S1 − Backbone+2.73+4.12+4.63+1.68+3.29
Stage 2 gain: PACE − S1+0.36+0.34+0.88+0.57+0.54

Taken together, these results suggest that Stage 1 provides a foundation for broader alignment: cross-subject supervision within datasets may reduce subject-specific variation and emphasize affective features that also transfer across datasets. Stage 2 then introduces explicit cross-dataset alignment, with improved cross-paradigm relation ranking supporting better alignment of shared affective structure. Nevertheless, the absolute Relation-AUC values remain modest, motivating further improvement and deeper analysis of which affective relations transfer across paradigms. We will include this stage-wise analysis and discussion in the revised manuscript.

Response to Question 1: T/E ablation and S1 relation-AUC

We added Stage 1 ablations on CBraMod. Both T-only and E-only improved average full-finetuning BAcc across the twelve held-out targets. Combining T+E outperformed them by 1.14 and 3.01 pp, respectively, supporting their complementary contributions. Complete ablation results per-dataset have been included in Appendix G.5.

Full-finetuning balanced accuracy averaged across twelve targets (%, CBraMod).

BackboneS1: T-onlyS1: E-onlyS1: T+E
32.9836.9635.0938.10

We also added S1-only relation-AUC results for all four backbones, reported alongside the backbone and PACE results in our response to Weakness 1. Stage 2 improved CD/CP relation-AUC across all four backbones, averaging +0.54/+1.59 pp.

Response to Weakness 2: Cross-elicitation downstream evidence

We thank the reviewer for highlighting the uneven gains across elicitation contexts. Across all four EEG foundation backbones, PACE achieves positive average BAcc gains on Video datasets (+3.96 to +9.45 pp). It also improves performance on both Listening datasets (EAV and MEEG) across all four backbones, with average gains ranging from +0.73 to +3.41 pp. These results demonstrate that the benefits of PACE extend across different externally elicited emotional contexts, with the largest improvements observed in Video.

Imagery presents a more challenging setting for emotion decoding. Imagery relies on internally generated emotional experiences, with greater variability in duration and inter-subject responses. To better understand the limitation of PACE on imagery paradigm, we explored two possible factors: whether decodable affective information varies across the imagery period, and whether cross-elicitation correspondence is more evident when subject identity is held constant. We conducted two complementary analyses.

Temporal variation in imagery decoding. On the MC-I dataset, We divided the imagery period into three consecutive 10-second segments following the end of the instruction cue: Early (0–10 s), Middle (10–20 s), and Late (20–30 s). We evaluated each segment separately under full fine-tuning (FT).

FT BAcc (%): Backbone → PACE (Δ in pp)

SegmentCBraModCSBrainLaBraMREVE
Early14.93→16.60 (+1.67)14.96→16.40 (+1.44)14.99→16.76 (+1.77)14.77→17.25 (+2.48)
Middle13.80→14.98 (+1.18)14.55→14.61 (+0.06)15.33→15.77 (+0.43)15.65→15.43 (−0.22)
Late13.83→14.25 (+0.42)14.94→13.57 (−1.37)14.30→13.68 (−0.62)13.51→13.61 (+0.10)

Notably, during the Early segment, PACE improves FT BAcc across all four backbones, with gains ranging from +1.44 to +2.48 pp. These improvements become smaller and less consistent in the subsequent segments. This suggests that the decodability of internally generated emotional information varies over time, potentially reflecting the temporal dynamics of affective experiences during imagery.

Cross-elicitation retrieval within the same subject. To examine cross-paradigm affective correspondence while controlling for subject identity and dataset differences, we evaluated video–imagery retrieval within the same subjects and dataset, using native emotion labels to define relevance. PACE improves Hit@1 over the backbone by +3.22 pp for imagery-to-video and +2.53 pp for video-to-imagery retrieval. These gains provide complementary evidence of improved cross-paradigm correspondence. Full details are provided in our response to Reviewer Ho7L (Question 3).

Overall, PACE improves downstream transfer to Video and Listening datasets and strengthens cross-paradigm relation ranking, as reported in our response to Weakness 1. The additional imagery analyses show gains in the Early segment and improvements in within-subject cross-elicitation retrieval. These results provide complementary evidence of transferable affective information, although robust downstream transfer to Imagery remains a challenge. We will revise the manuscript to denote the uneven gains in different paradigms explicit.

Response to Weakness 3: Relation-AUC circularity and gallery sources

Although Stage 2 and Relation-AUC share the same semantic rules, optimizing the Stage 2 objective alone does not guarantee improved relation ranking on held-out data. Without Stage 1 initialization, S2-only improves average CP Relation-AUC by just 0.07 pp over the backbone, whereas full PACE achieves a gain of 3.33 pp under the same evaluation protocol. Importantly, Stage 2 adds 1.59 pp when applied after Stage 1. These results support the value of progressive alignment: the structure learned in Stage 1 enables more effective subsequent cross-dataset alignment.

Cross-paradigm Relation-AUC (%), AUC (Δ vs. Baseline in pp).

MethodCBraModLaBraMREVECSBrainAverage
Baseline49.9449.7550.7449.0549.87
S1-only51.50 (+1.56)52.07 (+2.32)50.95 (+0.20)51.95 (+2.90)51.61 (+1.74)
S2-only49.70 (-0.24)49.77 (+0.02)50.59 (-0.15)49.69 (+0.64)49.94 (+0.07)
PACE52.68 (+2.74)53.43 (+3.68)54.46 (+3.72)52.21 (+3.16)53.20 (+3.33)

In each fold, queries come from the held-out target dataset, while galleries contain samples from validation subjects per eligible source dataset. The diagnostic therefore evaluates whether the predefined emotion relations generalize to held-out target queries, rather than measuring their fit on training samples.

Response to Weakness 4: Clarification of manuscript claims

We thank the reviewer for pointing out that our wording did not specify the results precisely. The statement that PACE outperforms CE, SupCon, and VREx on all twelve targets applies specifically to balanced accuracy under CBraMod full fine-tuning. We have revised the abstract and corresponding statements to explicitly include these qualifications. We have also clarified that PACE performs best in 41 of the 48 dataset–protocol–metric comparisons, with exceptions under LP and macro-F1 evaluation, rather than consistently outperforming all baselines in every setting.

Response to Weakness 5 and Question 2: Source–target overlap

We identified shared stimuli within the SEED family and masked source trials overlapping with each held-out target based on stimulus names and time intervals. Same-title stimuli with uncertain timing were also conservatively excluded. The complete dataset-specific stimulus exclusion rules are provided in Appendix G.3. After masking, PACE retained an average FT BAcc of 41.56%, exceeding CBraMod by 5.71 pp across the four SEED targets. We also randomly masked the same amount of data within each source dataset, keeping all other settings identical. Rule-based and random masking differed by only 0.09 pp, showing that PACE retained its average performance advantage after removing potentially shared stimuli.

For the SEED/SEED-IV holdouts, we also excluded all source training and validation recordings from the two shared participants, mhw and zjy.

FT BAcc (%, CBraMod). Avg4 is the unweighted mean across the four SEED targets.

DatasetCBraModPACE (original)PACE (rule masking)PACE (random masking)
SEED53.3564.0963.5363.77
SEED-IV36.3844.0441.8741.24
SEED-V29.6838.5635.8236.09
SEED-VII23.9926.5025.0224.80
Avg435.8543.3041.5641.47

We checked the EmoEEG-MC stimulus list on OpenNeuro and found no overlap with the stimuli used in FACED. We further confirmed with the EmoEEG-MC authors that FACED materials were included in the initial candidate pool but were not selected for the final EEG experiment following ratings by an independent group of volunteers.

Response to Question 3: Baseline evaluation and Avg9

All task-specific baselines, emotion-pretrained models, and foundation models were evaluated under a common downstream protocol, including preprocessing, subject-disjoint train/validation/test splits (25%/25%/50%), six random seeds (42–47), and the same learning-rate candidates and model-selection strategy within each evaluation mode. Each implementation retains its respective architecture, with input adaptations to accommodate the unified 60-channel, 10-second EEG format. Task-specific models are trained end-to-end with supervised learning, whereas emotion-pretrained models are initialized from their respective pretrained weights. These settings enable comparisons of different architectures and pretrained encoders under common downstream tasks and evaluation conditions.

Avg9 was an earlier aggregate mistakenly retained during supplementary-material preparation, not an intentional exclusion of the three datasets. We have replaced it with Avg12 covering all twelve targets. Per-dataset results remain unchanged. We thank the reviewer for identifying this oversight.

Thanks for the detailed response and the extra experiments. You have honestly addressed most of my concerns. Since the downstream gains from Stage 2 over Stage 1 are still relatively small compared to Stage 1 itself, I will maintain my original score, though I now have a much more positive view of the paper's technical soundness.

Reviewer_bmQdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines limitations in the transfer of affective representations from general-purpose EEG foundation models across subjects, datasets, and emotion-elicitation paradigms, and proposes PACE, a two-stage affective post-training framework. The first stage establishes representation structure within source datasets using cross-subject stimulus/temporal correspondences and native emotion labels. The second stage aligns representations across datasets and paradigms through partially shared emotion concepts while retaining the within-dataset supervision from the first stage. The framework jointly updates the pretrained encoder and a shared projection head, incorporating feature alignment within blocks of windows from the same subject. Experiments across multiple datasets and backbones, together with additional analyses, assess the framework’s effectiveness. The main contributions are a progressive training protocol for heterogeneous affective EEG data and relatively broad empirical evidence of transfer performance.

Critical strengths
  1. The research question is well defined and addresses key challenges in transferring EEG emotion representations across domains. The paper focuses on whether affective representations from general-purpose EEG foundation models can transfer across domains, considering difficulties arising from subject variability, dataset heterogeneity, and changes in elicitation contexts. The motivating diagnostics, methodological design, and experimental evaluation are reasonably well aligned, giving the research question clear relevance.
  2. The layered representation diagnostics provide complementary value. Through emotion-relation ranking, the paper examines frozen representations at four levels: within subjects, across subjects, across datasets, and across paradigms. This provides a perspective on the gap between affective information within a domain and representation organization across domains, complementing evaluation based solely on downstream classification performance.
  3. The progressive alignment and handling of heterogeneous labels are reasonable. The method first establishes cross-subject structure within datasets and then extends alignment across datasets and paradigms, supporting progressive learning across domains. The use of partially shared emotion concepts, together with rules excluding ambiguous and unmapped relations, allows the method to accommodate both native-label supervision and semantic correspondence across domains.
  4. The experiments cover a broad range of settings and include several levels of analysis. Twelve target datasets, four foundation models, and three categories of elicitation paradigms provide a broad evaluation scope. Linear probing, full fine-tuning, stage ablations, comparisons with alternative post-training methods, and relation-ranking and neighborhood analyses collectively provide evidence of the framework’s effectiveness from multiple perspectives.
Critical weaknesses
  1. The coverage and representativeness of the post-training baselines are limited. Direct comparisons with alternative post-training methods are conducted only on CBraMod, leaving insufficient evidence to determine whether PACE’s relative advantages generalize to other backbones. The comparisons primarily involve established objectives such as CE, SupCon, and VREx, with limited discussion of their representativeness and limited comparisons with recent post-training strategies.
  2. The specific training configurations of the post-training baselines are insufficiently specified. The appendix describes the shared encoder–projector architecture, the EA32 procedure, and the common downstream optimization protocol. However, the loss combinations, sampling and pairing rules, method-specific hyperparameters, and tuning ranges used by CE, SupCon, and VREx during post-training remain insufficiently detailed. This limits the assessment of whether these baselines were adequately optimized and evaluated under comparable training budgets.
  3. The resource costs associated with the performance gains are insufficiently quantified. The paper does not adequately compare actual sample exposure, training time, peak GPU memory usage, and inference costs across the post-training methods. Consequently, it remains difficult to determine the extent to which PACE’s gains are accompanied by additional data exposure or computational expenditure, or to assess its performance advantages and practical efficiency under comparable resource budgets.
  4. Sensitivity to predefined emotion relations and their impact on generalization remain unclear. PACE’s cross-dataset alignment relies on predefined shared emotion concepts and fixed positive and negative relations. Although the method excludes some ambiguous and unmapped relations, the analysis does not sufficiently characterize performance sensitivity to the choice of semantic mappings and relation rules. It therefore remains unclear how strongly the gains depend on the current rules and whether these rules support robust generalization across different emotion taxonomies and elicitation contexts.
Questions for authors

Why does the SupCon baseline in Table 3 cite the master’s thesis by Pasinato (2026), whereas Appendix Table 9 lists the published work by Khosla et al. as its source?

3 comments

Response to Weakness 1: Coverage and Representativeness of Post-training Baselines

We thank the reviewer for this valuable suggestion. Our original comparison includes three representative and complementary post-training objectives: CE, SupCon, and VREx, covering distinct strategies for improving affective transfer. Specifically, CE represents direct supervised adaptation using emotion labels, SupCon represents label-based contrastive representation learning (Khosla et al., 2020), and VREx represents domain generalization through risk regularization (Krueger et al., 2021), whose effectiveness in cross-subject EEG emotion recognition has also been demonstrated by Kim et al. (2025). Together, these methods provide complementary comparisons across supervised adaptation, representation alignment, and domain generalization, allowing us to assess whether PACE's improvements extend beyond these established learning strategies.

Meanwhile, studies specifically focusing on affective post-training of general-purpose EEG foundation models—particularly for transfer across heterogeneous emotion datasets—remain scarce. To further strengthen the comparison, we have incorporated two additional baselines adapted from recent relevant work:

  • JADE-adapted (CE + SupCon): Pasinato (2026) investigated affective adaptation of the pretrained REVE model on FACED by jointly optimizing cross-entropy and supervised contrastive learning. We extend this joint objective to our multi-source post-training setting, examining whether combining direct emotion supervision with contrastive alignment can achieve comparable generalization.
  • V–A alignment (EMOD-adapted): Chen et al. (2026) proposed valence–arousal-guided contrastive learning to align heterogeneous emotion representations through affective similarity. We adapt this semantic alignment objective to the same EEG foundation backbones as PACE.

We further extended the original post-training comparisons from CBraMod to all four EEG foundation backbones, resulting in an evaluation of five alternative post-training methods across four backbones and twelve held-out target datasets. The expanded full-fine-tuning results are summarized below as average balanced accuracy (%) across the twelve target datasets (Avg12). Full per-dataset results, expressed as mean ± standard deviation across six random seeds, will be included in Appendix G1.

MethodCBraModREVECSBrainLaBraM
CE34.3637.1132.9534.36
SupCon34.7535.9332.7033.43
VREx34.5438.1732.1133.76
JADE-adapted36.0038.1533.4936.36
V–A35.6033.7233.6234.18
PACE40.0340.9436.6538.28

PACE achieves the highest average balanced accuracy on all four backbones, outperforming the strongest alternative post-training method by 1.92–4.03 percentage points. Moreover, PACE achieves the highest performance in 43 of 48 backbone–target comparisons. These results demonstrate that PACE's advantages generalize across different foundation architectures and remain consistent when compared with both established post-training objectives and recent approaches to affective adaptation and semantic alignment.

Response to Question 1: Clarification of SupCon References

We thank the reviewer for pointing out this citation inconsistency. Khosla et al. (2020) introduced the original supervised contrastive learning (SupCon) method, while Pasinato (2026) applied SupCon to EEG foundation models for cross-subject emotion recognition. Our SupCon baseline draws on both works, using the original contrastive learning formulation and its adaptation to EEG emotion recognition.

Notably, Pasinato (2026) also investigated JADE, which combines SupCon with cross-entropy supervision. Following the reviewer's suggestion to include more recent post-training strategies, we have additionally incorporated a JADE-adapted baseline in our expanded experiments, as detailed in our response to Weakness 1.

We have standardized the SupCon citations to Khosla et al. (2020) in the revised manuscript, while citing Pasinato (2026) separately when discussing its application to EEG emotion recognition and the JADE-based comparison.

Response to Weakness 2: Training Fairness and Baseline Configuration

Our comparison follows a controlled protocol that isolates the effect of the post-training objective. CE, SupCon, VREx, Jade ,V-A and PACE share the same pretrained backbone initialization, encoder–EA32–projector architecture, and eligible source-dataset pool. Targets are held out during post-training, and downstream LP/FT evaluations use identical subject-disjoint splits and optimization procedures. All methods use Adam, the same reference learning rates, and four-GPU FP32 training.

The baselines employ complementary objectives under matched training budgets. CE minimizes cross-entropy using native-label dataset-specific heads. SupCon performs supervised contrastive learning with within-dataset same-label positives and different-label negatives. VREx combines mean classification risk with subject-level risk-variance regularization. Their learning rates and batch sizes are fixed, while method-specific settings follow the original formulations (Khosla et al., 2020; Krueger et al., 2021), including a SupCon temperature of 0.07 and a VREx penalty coefficient of 10. Each baseline receives 100 anchor-exposure epochs, while PACE allocates 50 epochs to Stage 1 and 50 to Stage 2, maintaining the same total exposure budget. Complete loss definitions, pairing/sampling rules, and hyperparameters are provided in the revised Appendix G2.

Response to Weakness 3: Computational Efficiency

We further quantified the computational costs of different post-training methods. Using CBraMod with SEED held out, we measured training time and peak GPU memory across CE, SupCon, VREx, and PACE on four NVIDIA H20 GPUs. All four methods process the same 7.72 million encoded EEG views and complete their prescribed training budgets. The results are summarized below.

MethodTraining time (min)Peak GPU memory (GiB/card)
CE84.8531.06
SupCon84.4430.87
VREx84.6730.94
PACE98.5630.98

PACE maintains comparable peak GPU memory, with approximately 14 additional minutes of one-time offline post-training. Importantly, PACE uses the same inference pipeline as the baselines, introducing no additional modules or forward passes and therefore maintaining the same inference computational cost and expected latency. These results demonstrate that PACE's transfer gains are achieved without increased data exposure, model capacity, or inference overhead.

Response to Weakness 4: Effectiveness of PNAU-Based Emotion Relation Modeling

We thank the reviewer for recognizing the value of our approach to handling heterogeneous emotion labels and for suggesting further analysis of its sensitivity. PNAU enables selective semantic supervision across heterogeneous emotion taxonomies. It aligns labels mapped to shared emotion concepts (Positive), separates pairs supported as distinct concepts (Negative), and excludes Ambiguous and Unmapped relations from the contrastive objective. This strategy preserves native-label supervision while establishing reliable cross-dataset emotion relations.

We tested PNAU through an additional Stage 2 ablation. The simplified PN variant treats identical cross-dataset label names as positives and every differently named label as a negative, without ambiguity or unmapped-label handling. Other settings are unchanged. The table reports CBraMod full-fine-tuning balanced accuracy (%) on twelve held-out datasets; Δ = PNAU − PN (pp).

MethodSEEDSEED-IVSEED-VSEED-VIIFACEDMC-VREFEDDEAPEAVMEEGMC-IImag.Avg12
PN62.8242.9837.9326.7851.7233.0130.8957.2538.5363.4414.759.6339.14
PNAU64.0944.0438.5626.5054.4034.1931.5458.1039.6864.3515.399.5340.03
Δ (pp)+1.27+1.06+0.63−0.28+2.68+1.18+0.65+0.85+1.15+0.91+0.64−0.10+0.89

PNAU achieves consistent gains over PN across 10 of the 12 held-out datasets, raising average balanced accuracy from 39.14% to 40.03% (+0.89 pp), with notable improvements on FACED (+2.68 pp), MC-V (+1.18 pp), and EAV (+1.15 pp). These results indicate the utility of distinguishing reliable negative relations from ambiguous or unmapped pairs when transferring affective representations across heterogeneous emotion taxonomies, supporting the role of selective semantic supervision in PACE's progressive cross-context alignment.

AuthorsOpenReview ↗

Follow-up on our response to Reviewer bmQd

Thank you again for your valuable feedback. As the discussion period nears its end, we wanted to follow up on whether our previous responses and revised manuscript have addressed your concerns. Please let us know if you have any further questions or need additional clarification. We would be glad to discuss any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.