acceptodds
Under review as a conference paper at ICLR 2027

UniPER: Towards a Unified Predictive Model of Affective EEG

Abstract

EEG emotion recognition requires transferable representations, but general EEG pretraining often draws on clinical and other non-affective recordings. Learning transferable emotion representations from multiple affective datasets remains challenging because stimuli, participants, sensor layouts, and annotation schemes differ across sources. We introduce UniPER, a latent predictive framework that learns shared affective EEG representations without requiring source emotion labels or cross-dataset label alignment. UniPER trains a montage-aware online encoder to predict masked latent content from a full-view target encoder updated by an exponential moving average (EMA). Position-matched content subtracts each encoder's zero-waveform response at the same sensor and time location; a matched zero reference also centers the predictor output. Masked prediction and context identity learn from waveform content and source-local recording events, while within and cross-source relations organize latent similarities. We pretrain UniPER on nine affective EEG datasets and evaluate it on six held-out target datasets under subject-dependent, subject-independent, and zero-shot protocols, retaining each target's emotion labels. Under subject-dependent adaptation, UniPER achieves the highest balanced accuracy on five of six targets, reaching 65.60% on FACED, 5.97 percentage points above the strongest evaluated baseline. Component ablations examine zero references, local prediction, context identity, source relations, and representation regularization; their effects vary by target and metric. These results support multi-source affective pretraining through latent prediction as a route to reusable emotion representations across heterogeneous EEG datasets.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, Emotion Recognition, Self Supervised Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_MTUBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The problem this paper aims to solve is very practical: subjects, devices, numbers of channels, and emotion labels all differ across affective EEG datasets. Can these data be used together to pretrain a model that transfers more easily? The authors propose UniPER, which combines masked prediction, zero-input reference, and event relationship learning, and test it on six target datasets. I appreciate this research direction, and the improvements reported in the paper are worth noting. However, after reading it, I am still not entirely sure how much of these gains come from the proposed new designs and how much come from the pretraining data and training approach themselves. In addition, the specific procedure of the "zero-shot" experiments and the performance of some baselines also need further explanation.

Critical strengths

First, the paper does not require all datasets to use the same set of emotion labels, which is of great practical value. Different datasets inherently use inconsistent emotion classification schemes, and being able to directly leverage these data is more convenient than forcibly aligning labels first. Second, the experimental coverage is relatively broad, and some of the improvements are not small. For example, BACC on FACED reaches 65.60%, nearly 6 percentage points higher than the strongest baseline; on SEED-V there is also an improvement of about 5.5 percentage points. Gains are also observed in subject-independent experiments, so these results deserve serious consideration. Another point worth acknowledging: the authors explain the data splits, with SD split by complete trials and SI split by subjects. The paper also reports the case where it does not outperform on SEED-IV and acknowledges that FACED used only some subjects, allowing readers to understand more clearly the scope in which the results apply.

Critical weaknesses
  1. I still cannot clearly see where the most critical advance of UniPER over existing methods lies. Masked latent prediction, multi-dataset pretraining, and exploiting cross-subject relationships under the same stimulus all have related prior work. The potentially valuable distinction of this paper may be the zero-input reference and the combination of these objectives. However, a very critical control is missing: what would happen with the same backbone, the same nine source datasets, and the same training budget, but using only conventional EMA masked prediction? Without this experiment, it is hard for me to judge where the advantage of the full model actually comes from. The paper also discusses closely related work such as MGCRL but does not compare directly with it, which makes it harder to judge the contribution.

  2. The "zero-shot" experiments are easily misunderstood, and the details are not fully explained. The appendix states that at test time a labeled gallery from the target dataset was used. That is, although the model does not update its parameters, labeled samples from the target domain are still needed for classification. This setting itself is feasible, but the main text should state this more directly. It is currently unclear how large the gallery is, how many samples per class are included, whether the gallery and query come from the same subjects or trials, and how the final class prediction is obtained. These details could all significantly affect the results in Table 3 and need to be supplemented.

  3. The results of some baselines make me question whether the comparison is sufficiently thorough. For example, on FACED, DGCNN and EEG Conformer perform close to chance level. This does not necessarily mean there is a problem with the implementation, but at least it needs explanation: did the models converge, is the input processing suitable for these methods, and was reasonable hyperparameter tuning performed? In addition, the paper does not explain clearly enough what weights and what source data each pretraining baseline specifically used, and whether they were retrained. If these conditions differ greatly, the comparison in the table can hardly directly show that the method itself is better.

  4. The ablation shows that some components are useful, but it does not yet explain clearly why they are useful. For example, performance drops markedly after removing the zero reference. This is an interesting result. But this experiment removes the reference in both the encoder and the predictor at the same time, so it is impossible to know which one is actually responsible, or whether the two must be used together. Similarly, does the improvement brought by the cross-source relationship loss really indicate that the model has learned emotion information that transfers more easily across datasets? The current evidence is relatively indirect. I would like to see more direct analyses, for example, how much data-source or subject-identity information is still retained in the representation, and whether performance is more stable when encountering unseen stimuli or channel changes.

  5. Small improvements still need more sufficient statistical support. The improvement on MPED is very small, and the BACC on SD is only 0.45 percentage points ahead. Although the paper reports means and standard deviations, it still needs to distinguish fluctuations from the target data split from fluctuations from pretraining itself. For such close results, I would not consider a stable advantage to have been established based on ranking by mean alone. I hope the authors will focus on answering the following in their response

Questions for authors
  1. Under the same data, backbone, and training budget, what level can conventional EMA masked prediction achieve?

  2. Why was no direct comparison with the closest methods such as MGCRL included?

  3. What exactly are the gallery size, split, and classification rule? Can it be guaranteed that the gallery and query are isolated at the trial level?

  4. Were the several baselines on FACED that are close to chance level sufficiently trained and tuned? Can training curves or related explanations be provided?

  5. How much do the encoder zero reference and the predictor zero reference each contribute? Does increasing the number of pretraining data sources still yield gains under a fixed data volume?

Reviewer_peBaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

UniPER learns EEG representations from nine affective datasets without using their emotion labels. A montage-aware encoder predicts masked representations from an EMA teacher. The method subtracts matched zero-input responses from both encoder and predictor outputs, and combines masked prediction with event-identity learning, within- and cross-source relations, and SIGReg on pooled window features. Evaluation covers six datasets excluded from UniPER’s source pretraining, using each target’s native labels. The experiments include subject-dependent and subject-independent adaptation, a frozen-encoder gallery protocol, and reduced-label settings. UniPER achieves the highest mean subject-dependent balanced accuracy on five of the six targets.

Critical strengths
  1. Learning across affective EEG datasets with different montages and label vocabularies is a relevant problem. Holding six target datasets out of UniPER’s source pretraining provides a useful setting for studying transfer.

  2. The reported improvements on some targets are appreciable. FACED subject-dependent BACC increases from 59.63% for the strongest evaluated baseline to 65.60%, and SEED-V increases from 65.96% to 71.46% (Table 2). Subject-independent results are also reported, including a FACED improvement from 39.04% to 44.06% (Table F.5).

Critical weaknesses
  1. Most of the framework follows existing lines of work acknowledged in Appendix A: multi-dataset affective pretraining, masked EEG latent prediction, event-based contrastive learning, and SIGReg regularization. The main distinguishing construction is the matched zero-input reference. Table 4 shows that removing this construction hurts performance, but it removes the encoder and predictor references together. This leaves their individual roles and interaction unresolved. The experiment establishes that the bundled change matters within this particular system, without identifying what new representation-learning principle it contributes.

There is also a gap between the proposed objectives and the intended shared affective structure. The cross-source objective in Eqs. 7–8 matches similarities produced by the model’s own EMA teacher, which supplies no independent cross-source semantic correspondences. In Eq. 9, source means are subtracted before SIGReg is applied, so this regularizer does not directly penalize differences between source means in the exported representation.

These observations make empirical validation of the proposed explanation particularly important. Figure 1 is explicitly conceptual, and Figure 5 shows each target separately; neither tests the cross-source organization illustrated in the motivation. The paper leaves unclear whether its distinctive components reduce the relevant dataset biases or mainly improve optimization and initialization. That distinction is central to assessing the claimed advance.

  1. The component-removal experiments in Table 4 compare variants of the full system. They do not show how much is gained over ordinary masked latent prediction with the same encoder, source corpus, event sampling, regularization, and training budget. EEG2Rep, an existing predictive EEG method discussed in the paper, is also absent from the comparisons. The random-initialization result establishes the value of pretraining under the documented adaptation procedure; it does not identify the value of UniPER’s particular pretraining design.

The comparisons with existing pretrained models are difficult to interpret for the same reason. Appendix D identifies the baseline families but does not specify whether their results come from released checkpoints, common-source retraining, or additional adaptation. Their source-data exposure and training budgets are therefore unclear. A larger comparison table cannot resolve this attribution problem.

The frozen-encoder evaluation does not close the gap. Appendix E.1 states that it uses a labeled target gallery, so Table 3 measures transfer with target-label support. The gallery size, trial/participant separation, prediction rule, and class scores used for AUROC are unspecified. Its difficulty and supervision budget therefore cannot be assessed. Although the subject-independent adaptation results provide useful evidence of transfer, they still do not isolate the contribution of the proposed learning scheme.

Questions for authors

With the encoder, source corpus, event sampling, regularization, and update budget fixed, how does UniPER compare with a basic masked-latent-prediction model? Can the encoder-only and predictor-only zero-reference variants be reported to identify the source of the improvement?

What empirical evidence supports the proposed explanation that the reference construction and cross-source objectives address dataset-specific biases? In particular, how is the effect of the cross-source loss distinguished from preserving the EMA teacher’s existing similarity structure?

For the leading pretrained baselines, which checkpoints, source datasets, and training budgets were used? For Table 3, what are the gallery size, gallery/query separation at the trial and participant levels, prediction rule, and AUROC scoring procedure?

Reviewer_P9oAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents UniPER, a pretrained 6.5M-parameter EEG encoder on nine affective datasets without the use of their emotion labels. The objective is masked latent prediction against an EMA teacher, in the style of I-JEPA and data2vec, with three additions: a contrastive loss that matches windows sharing a recording event, within- and cross-source relation losses with weight at most 0.05, and SIGReg from LeJEPA on the pooled window vector. The main claimed novelty is subtracting each encoder’s response to a zero waveform at the same sensor and time position before pooling. The encoder is then fine-tuned on six held-out targets (SEED, SEED-IV, SEED-V, DEAP, FACED, MPED) under subject-dependent, subject-independent and gallery-based transfer, against fourteen re-run baselines. The paper reports the best subject-dependent balanced accuracy on five of six targets and ablates each component on three of them.

Critical strengths

The evaluation design is designed to answer the paper's premise, which is an intereseting one. All six targets are excluded from pretraining, each keeps its own label space, and results are given under subject-dependent, subject-independent and gallery transfer with shuffled-pair and random-initialisation controls.

Critical weaknesses
  1. The baseline results are not credible, with every claim in the paper resting on them. RGNN reaches 40.15% balanced accuracy on three-class SEED under subject-dependent training. Its own paper reports 94.24% under a within-subject split, using DE features and plain accuracy, which does not account for a gap of this size. On nine-class FACED, DGCNN and EEG Conformer score 10.53% and 11.95% with macro-F1 of 2.6 and 3.5, which is what predicting a single class produces. RGNN on FACED subject-independent is 10.55 with zero variance over three seeds. TSception scores higher subject-independent than subject-dependent on DEAP (62.14 versus 52.83). Appendix D lists the fourteen baselines and gives no epochs, learning rates, sampling rate or checkpoint source. No published number is used as a check. All entries in Tables 2, 3 and F are the authors’ re-runs, so the claim of best results on five of six targets cannot be assessed.
  2. The ablations credit large gains to components the paper never analyses, and the protocol has gaps that could explain the same gains. Removing the zero-waveform subtraction, a term that does not depend on the input, costs 10.5 points on SEED-IV and 13.5 on DEAP, placing UniPER below twelve of the fourteen baselines on both. The within-source relation has weight at most 0.05. Removing it costs about two points, but shuffling its pairs costs seven to nine. SIGReg is computed on 24 window vectors per update, yet removing it costs five to seven points. Several ablation rows report F1 standard deviations of 4 to 8 points, against 1 to 2 in the main table. The representation-health diagnostics mentioned at line 739 are never shown, so model collapse cannot be ruled out. On the data side, the SEED-IV cache holds 4,946 windows for 1,080 trials of about two minutes, roughly 4.6 windows per trial where about 24 would be expected. FACED is cut to 51 of 123 participants with no reason given. The zero-shot gallery is not described well enough to tell whether gallery and query windows come from the same participants or trials.
Questions for authors
  1. For each baseline, state epochs, learning rates, sampling rate, preprocessing and whether released checkpoints were used. Explain why DGCNN, EEG Conformer and RGNN fall to chance on FACED and RGNN to 40% on SEED, when their own papers report 90% or more.
  2. Were the mdJPT or EMOD checkpoints pretrained on any of the six targets? Does SEED-VII share participants or stimuli with SEED, SEED-IV or SEED-V?
  3. Why does the SEED-IV cache hold 4,946 windows for 1,080 trials, and why is FACED restricted to 51 of 123 participants?
  4. For the zero-shot gallery: how many windows, how were they chosen, and do gallery and query windows share participants, sessions or trials? Were the Table C.2 hyperparameters selected on a validation split or on the test partitions?
  5. Report the seed count and representation-health diagnostics (line 739) for the ablations, separating the encoder-side and predictor-side zero references, and release code and caches.
Reviewer_JtXEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UniPER, a self-supervised framework for learning transferable representations from heterogeneous affective EEG datasets without requiring emotion labels or cross-dataset label alignment during pretraining. The method employs a montage-aware encoder and a JEPA-style masked latent prediction framework with an EMA teacher. It introduces position-matched zero-waveform references to reduce input-independent responses and combines masked prediction with context identity learning, within-source ranking, cross-source relation alignment, and SIGReg regularization. UniPER is pretrained on nine affective EEG datasets and evaluated on six held-out datasets under subject-dependent, subject-independent, and zero-shot protocols. Experimental results show that UniPER achieves the highest balanced accuracy on five of the six datasets in the subject-dependent setting, including 65.60% on FACED, outperforming the strongest compared baseline by 5.97 percentage points. Ablation studies further investigate the contributions of the proposed training components.

Critical strengths
  1. The paper addresses an important challenge in EEG emotion recognition: learning transferable representations from heterogeneous datasets with different electrode configurations, stimuli, and emotion label spaces. Pretraining without requiring emotion labels or cross-dataset label alignment is particularly valuable for integrating diverse affective EEG datasets.

  2. The authors pretrain on nine affective EEG datasets and evaluate on six held-out datasets under subject-dependent, subject-independent, and zero-shot settings. The inclusion of multiple baselines and evaluation metrics provides a relatively comprehensive assessment of the model's transferability. UniPER achieves competitive performance across different datasets, with particularly notable improvements on FACED.

Critical weaknesses

W1. The context identity and within-source relation objectives explicitly encourage alignment between EEG recordings corresponding to the same stimulus and temporal position. However, such shared representations may reflect stimulus-locked sensory or attentional responses rather than emotion-related neural activity. The current evaluations primarily demonstrate improvements in downstream classification performance, without adequately disentangling these factors. This limits the evidence supporting the central claim that UniPER learns transferable affective representations.

W2. The proposed cross-source relation objective distills relative similarity distributions from an EMA teacher to the online encoder. However, preserving the teacher's similarity structure does not necessarily encourage semantic alignment across datasets or eliminate source-specific information. Consequently, it remains unclear whether this objective effectively promotes a unified representation space or mainly preserves existing similarity relationships. This distinction is important given that cross-dataset representation learning is a central contribution of the paper.

Questions for authors

Q1. The spatial and temporal masking ratios are both fixed at 28%, resulting in approximately 47% masked tokens. What motivates this particular choice? Could the authors provide a sensitivity analysis with different spatial and temporal masking ratios, including higher overall masking levels, to evaluate the robustness of this design?

Q2. Could the authors provide comparisons with single-source JEPA pretraining and a simple pooled multi-source JEPA baseline, using the same encoder architecture and matched pretraining budgets? These comparisons would help clarify whether the reported improvements arise from heterogeneous data exposure or the proposed learning objectives.

Q3. Given that the Context Identity and Within-source Relation objectives explicitly align EEG responses to the same stimulus, how do the authors distinguish emotion-related representations from stimulus-locked sensory or attentional responses? Could the authors provide additional evidence to demonstrate that the learned representations capture emotion-related information beyond stimulus identity?

1 comment

We thank the reviewer for asking what transfers across stimuli and what drives the gains.

Q1: Why mask 28% along each axis?

A1: We chose 0.28 for both axes by preliminary validation on held-out source data. The ideal grid union is 0.28 + 0.28 − 0.28² = 48.16%, and 47.20% of valid tokens were masked in practice. The SD sweep varies both axes together and retains the Table 4 aggregate at 0.28.

Spatial / temporal maskRealized hidden fractionSEED-IV BACC / F1 / AUROCFACED BACC / F1 / AUROC
0.15 / 0.1527.84%71.82 ± 1.53 / 71.18 ± 1.72 / 86.37 ± 0.9163.24 ± 1.28 / 63.47 ± 1.38 / 85.93 ± 1.48
0.28 / 0.2847.20%74.34 ± 1.33 / 74.09 ± 1.45 / 88.08 ± 0.5365.60 ± 1.14 / 65.82 ± 1.19 / 87.77 ± 1.31
0.40 / 0.4064.02%73.58 ± 1.42 / 73.37 ± 1.58 / 87.46 ± 0.8365.13 ± 1.24 / 65.28 ± 1.32 / 87.24 ± 1.37
0.55 / 0.5579.82%70.87 ± 1.76 / 70.34 ± 1.97 / 85.73 ± 1.1262.84 ± 1.53 / 62.91 ± 1.58 / 85.27 ± 1.62

The 0.28 and 0.40 settings give similar scores. Performance is lower at 0.15 and 0.55. We will clarify the mask accounting and the scope of this sweep in the camera-ready appendix.

Q2: Do pooled data or the objectives drive the gains?

A2: Pooling improves the EMA baseline with SIGReg. UniPER scores higher than the pooled EMA+SIGReg control on both targets. The predictive controls use the same backbone and 7,200 updates.

Pretraining variantSEED-IV BACC / macro-F1FACED BACC / macro-F1
Single-source EMA prediction with SIGReg58.17% / 56.83%52.28% / 51.16%
Pooled nine-source EMA prediction65.84% / 64.23%58.67% / 57.92%
Pooled EMA prediction with SIGReg69.42% / 68.13%61.48% / 60.27%
UniPER (manuscript SD aggregate)74.34% / 74.09%65.60% / 65.82%

Q3: Does performance transfer across stimulus identities?

A3: With both participants and clips held out of target training, UniPER reaches 71.93% BACC on SEED-IV and 63.38% on FACED. The matched predictive baseline reaches 66.82% and 59.07%. FACED holds out 9 of 28 clips, one per class, and uses 19 for training. SEED-VII reuses stimuli from the SEED series. These results support transfer across target clips and leave the contribution of correlated sensory and attentional cues open.

Q4: What does cross-source supervision change?

A4: Cross-source KL matches EMA teacher similarities. Its supervision comes from the teacher, with no independent emotion correspondence labels. SIGReg regularizes source-centered features.

MeasureUniPERWithout cross-source loss
SEED-IV BACC (Table 4, SD)74.34%71.75%
FACED BACC (Table 4, SD)65.60%64.72%
Source-dataset accuracy64.18%69.73%
SEED-IV subject accuracy58.94%64.27%
FACED subject accuracy56.42%61.76%

The 9-way source probe uses held-out participants (uniform chance 11.11%). Subject probes operate within each target. Source and subject accuracy are lower with the loss enabled, and Table 4 shows higher SD BACC. Source identity remains recoverable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.