acceptodds
Under review as a conference paper at ICLR 2027

Auditing Individual Specificity in EEG Foundation Models with Occupancy-Preserving Temporal Nulls

Abstract

EEG foundation models are typically evaluated through downstream predictive performance, providing limited insight into what individual-specific information their frozen representations preserve when transferred to an unseen dataset. In this study, we introduce a representation-audit framework that distinguishes fingerprintability from temporal fingerprintability. A temporal representation supports an individual-specific dynamical interpretation only if its cross-session fingerprintability exceeds an occupancy-preserving temporal null. We formalize this distinction through temporal excess, . On the Temple University Epilepsy Corpus (TUEP), we evaluate two independently pretrained frozen EEG encoders, CNN–Transformer and LUNA-Base, and discretize their latent representations into states. State occupancy is reproducible across separate recording sessions (), with for CNN–Transformer and for LUNA-Base. Empirical lag-1 transition representations also appear individually specific under conventional metrics, reaching Top-1 retrieval rates of and , respectively, compared with chance. In contrast, the apparent temporal fingerprintability does not exceed the occupancy-preserving temporal null: for CNN–Transformer (real vs. shuffled : vs. ) and for LUNA-Base ( vs. ). Residual-transition and state-persistence probes yield the same conclusion at the primary resolution. These results show that a temporal representation can be individually fingerprintable without its fingerprintability depending on temporal order. More generally, does not imply , motivating occupancy-preserving temporal nulls when interpreting individual-specific dynamics in learned representations.

Primary area:
interpretability and explainable AI
Keywords:
EEG foundation models, representation learning, individual specificity, latent-state dynamics, temporal fingerprintability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DwKGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks if frozen EEG encoders keep individual-specific information on a new clinical dataset, and if that information depends on temporal order. TUEP windows go through two frozen encoders (a 1.46M-parameter CNN-Transformer and LUNA-Base), and global k-means puts them into k=4k=4 states. State occupancy π\pi and the lag-1 transition vector WW are compared within and between subjects across two sessions (N=112N=112) using a fingerprintability gap ΔFP\Delta_{FP}. A temporal-excess statistic Δtemp\Delta_{temp} (Eq. 8) compares ΔFP\Delta_{FP} of the original sequences with that of occupancy-preserving shuffles. Both occupancy and WW reach ΔFP\Delta_{FP} of about 0.10, but Δtemp(W)\Delta_{temp}(W) is −0.002-0.002 and −0.010-0.010. The authors conclude that ΔFP>0\Delta_{FP}>0 does not imply Δtemp>0\Delta_{temp}>0.

Critical strengths
  1. Section 4 separates reproducible individual information (ΔFP\Delta_{FP}) from information that depends on temporal order (Δtemp\Delta_{temp}). The toy sequence in Section 4.2 makes the shuffle null easy to follow. The point that a transition matrix inherits subject-specific occupancy is correct and matters for any latent-state analysis.

  2. The paper reports the one setting that goes against its conclusion (LUNA-Base at k=6k=6, pperm=0.03p_{perm}=0.03, Table 5 and Figure 6), low Top-1 retrieval and the dependence on kk. The limitations paragraph (Section 7, L437-447) names the single dataset, short recordings and two encoders. I recomputed Δtemp\Delta_{temp} as original minus permuted from Tables 1, 4 and 5; it matches the ΔFP\Delta_{FP} columns, so the numbers are internally consistent.

Critical weaknesses
  1. I can't interpret the central negative result as it stands. The claim (abstract; L103-107; Section 8) rests on Δtemp(W)\Delta_{temp}(W) being about zero or negative (Table 1, Section 6.3). The only sanity check I found is Δtemp(π)=0\Delta_{temp}(\pi)=0 "by construction" (Section 4.2), and that says nothing about whether the test can detect temporal individuality. A session needs only 20 windows (Section 5). So a session at that minimum gives 19 transitions for the 16 entries of WW. Also, uncentered cosine on non-negative vectors starts from a high baseline (between-subject 0.45, Table 1). A negative Δtemp\Delta_{temp} may just be noise in WW. The paper has no positive control (for example, synthetic sequences where subjects share occupancy but differ in transition matrices, run at the real session lengths) and gives no detectable effect size. There is also no uncertainty over subjects. The Mann-Whitney tests compare within- and between-subject similarities, and as far as I can tell the between-subject pairs reuse the same signatures, so they are not independent. The shuffle standard deviations in Table 1 (0.008 and 0.006) only cover permutation noise; one subject at N=112N=112 moves a Top-1 rate by about 0.9 points. Without an equivalence margin and subject-level intervals, failing to reject a difference does not support "no temporal excess". The long-recording check (D.1, N=35N=35) reports residual dwell, not Δtemp(W)\Delta_{temp}(W).

  2. The residual and persistence nulls still carry identity, and ΔFP\Delta_{FP} may partly reflect group or session similarity. In Table 4 the shuffled dwell-perp signature reaches ΔFP\Delta_{FP} of 0.157 and 0.194, higher than the original (0.106 and 0.118). W⊥W_\perp acts the same way: 0.113 and 0.122 after shuffling, against 0.089 and 0.098. A residual near zero under shuffling should carry little identity. So I suspect finite-sample effects that depend on π\pi and session length make it fingerprintable; Appendix D.1 explains residual dwell in a similar way. If that is right, the negative Δtemp\Delta_{temp} values can't be read as "no temporal information". Appendix D also reports that a dwell-preserving block shuffle lowers WW fingerprintability, which is the opposite direction, and only describes it as preserving run lengths. Separately, Eqs. 5-6 average over all between-subject pairs, including pairs from different diagnostic groups, and 75 of the 112 subjects have epilepsy. In Table 6 (CNN-Transformer, all 198 labeled subjects) π\pi similarity is higher within epilepsy (0.605) than within non-epilepsy (0.492). So part of ΔFP\Delta_{FP} could come from diagnosis, age, medication or recording setup. This is the part that bothers me most. The paper also doesn't give the interval between sessions A and B.

A few smaller points. The title and abstract talk about EEG foundation models, but the audit covers two encoders on a 4-state discretisation. There is no raw-feature or randomly initialised baseline to show whether pretraining matters for the 0.10 gap. "demonstrates" (L105-107) is stronger than what a null at k=4k=4 on one dataset supports. Appendix G (L845) calls k=4k=4 pre-specified, even though ΔFP\Delta_{FP} peaks there in Table 5, and Appendix F only says that k=4k=4 is not a unique optimum. "mLTD" (Appendix E, L775) is never defined, and the y-axis of Figure 4 reads "OUR".

Questions for authors
  1. What does the pipeline give on synthetic sequences with matched occupancy and subject-specific transitions, at your real session lengths?
  2. What are the subject-level bootstrap intervals for ΔFP(W)\Delta_{FP}(W), ΔFP(π)\Delta_{FP}(\pi) and Δtemp(W)\Delta_{temp}(W), with an equivalence margin for the claim of no temporal excess?
  3. How exactly is the dwell-preserving block shuffle defined? Why does it lower ΔFP(W)\Delta_{FP}(W) when the occupancy-preserving shuffle doesn't?
  4. If you keep only between-subject pairs from the same diagnostic group, does ΔFP(π)\Delta_{FP}(\pi) stay near 0.10? How does it change with the interval between sessions?
  5. Are any TUEP subjects or recordings in the pretraining data of either encoder? I'm not sure that TUEP and the TUEG pretraining corpus are disjoint, and Appendix A doesn't say what LUNA was pretrained on.
Reviewer_Jiv5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a representation audit for frozen EEG foundation models to identify individual specificity beyond latent state occupancy via an occupancy-preserving temporal null protocol. The paper shows auditing results on TUEP datasets with two models, which are CNN-Transformer and LUNA-Base. The experiments show no extra individual specificity information carried by temporal ordering compared to just latent space occupancy.

Critical strengths
  • The paper addresses a critical gap in evaluating EEG foundation models by questioning whether latent representations capture true temporal dynamics or merely static state occupancy, offering valuable diagnostic insights for the community.
  • The proposed method is simple and reasonable.
Critical weaknesses
  • The method can only evaluate the temporal information between encoded windows from EEG fms and cannot identify the temporal information encoded within the window on which the EEG fms are pretrained.
  • The evaluation scope is limited to just one dataset and 2 models.
  • The reliance on coarse, fixed-state clustering (kk-means) risks obscuring subtle temporal dynamics, as changing cluster resolution in sensitivity analysis produces varying temporal fingerprintability outcomes.
Questions for authors
  • Why look at recordings-level temporal information for EEG FMs while they are pretrained with small (<30s) windows?
  • Could the authors provide results on more datasets of all tasks (usually used in EEG FM papers) with the latest models?
  • Could the main finding (Δtemp≤0\Delta_{temp} \le 0) be an artifact of coarse kk-means discretization (k=4k=4), given that temporal fingerprintability became significant at k=6k=6?
Reviewer_NWZXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks what information about a person survives when an EEG model converts a recording into a set of features. The authors compare two recordings from each of 112 people. They measure how often each person occupies each learned state and how often one state follows another. With four states, both models show some consistency across recordings from the same person. Shuffling the order of the states preserves much of this similarity.

Critical strengths

Shuffling the order of the states while keeping their counts fixed gives a simple way to test whether timing contributes to the similarity between recordings.

Critical weaknesses
  1. The motivation is unclear. What scientific or clinical question can we answer by finding a fingerprint across two EEG recordings from the same person?
  2. The paper finds that state proportions are similar across recordings, but it does not identify which features of the EEG signal produce this similarity.
  3. The main fingerprint analysis uses only one dataset, TUEP. It is unclear whether the finding holds in other cohorts or recording settings.
Questions for authors
  1. Could you give one concrete example where knowing that two recordings are similar because of state frequencies would change how a researcher interprets an EEG model?
  2. Do your results link any recurring state pattern to a specific property of the recorded EEG? If the evidence only identifies state proportions, how narrowly should readers interpret the term “fingerprint” here?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.