acceptodds
Under review as a conference paper at ICLR 2027

How Does Pre-Trained Face Familiarity Affect the Decodability of Identity-Specific EEG Representations?

Abstract

While EEG-based visual decoding has attracted enormous interests and illustrated strong potentials over recent years, fine-grained facial identity decoding directly from brain-perceived EEGs has proved to be extremely difficult and challenging. In this paper, we investigate whether a pre-trained face familiarity can improve the discriminability and decodability of identity-related EEG representations. To address the challenge of identity decoding with fine-grained representations and sharing of highly similar visual features, we propose an EEG Spatio-Temporal Adaptive Fourier Fusion Network (EEG-STAFF) to effectively model complementary spatiotemporal and spectral information, which is supported by our establishment of a controlled and pre-trained familiarization paradigm for facial identity decoding. Experiments consistently show the improved identity decoding performances with the pre-trained familiarization, where the strongest gains emerge at the duration of 100-300ms, whist representation analysis reveals greater within-identity compactness and inter-identity separation. Compared with the existing representational state of the arts in relevant areas, the proposed EEG-STAFF achieves the best overall decoding performances, suggesting that the pre-trained face familiarization enhances the identity related neural representations and improves the EEG-based identity decoding.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Face Familiarization, Brain-Computer Interface, Neural Representation, Face Familiarity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eoddOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates whether experimentally acquired face familiarity is associated with improved 40-way identity decoding from EEG. Five participants complete Pre-Fam recording, face–name familiarization using a separate image set, and Post-Fam recording using the same 2,000 EEG stimuli. EEG-STAFF combines CBAM-adapted channel/spatiotemporal attention, multiscale temporal convolutions, a learnable Fourier-mode gain branch, and trial-dependent feature fusion. The manuscript reports Top-1 accuracy increasing from 5.11% to 7.81% and Top-5 from 18.60% to 24.76%, with the largest temporal gains at 100–300 ms. Additional evidence includes ERP plots, held-out learned-feature geometry, component ablations, a label-shuffle sanity check, and public-dataset evaluations.

Critical strengths

1. A relevant empirical question and several useful experimental provisions. Studying image-generalizable facial identity decoding is worthwhile, and the intended image-level folds are more informative than evaluating previously trained images. The authors screen for prior familiarity, use separate images for familiarization, assess recognition on new images, and report consistent Pre/Post trends across participants and multiple decoders. The appendix contains a shuffled-label sanity check and evaluations on EEG-Face and THINGS-EEG; these should be acknowledged rather than described as missing controls.

2. The core decoder is understandable and largely corresponds to the described operations. The supplied model implements residual attention, multiscale temporal convolutions, bounded learnable spectral gains, and normalized trial-specific fusion. Limited checks show that the weighted spectral gains can receive gradients, Top-k evaluation counts samples correctly including an uneven final batch, and a reduced-dimension model containing all of the proposed components can be saved and reloaded consistently. The fold helper correctly generates disjoint, class-balanced folds for an ideal 40-by-50 label vector. These checks support basic implementation plausibility, not the independence or validity of the reported evaluation. Table 2 is compatible with averaging Table 1 after rounding, and the reported t(4) values in Table 6 do correspond to unadjusted p<0.001.

Critical weaknesses

1. The supplied evaluation uses outer-test performance to select checkpoints, and the reported results cannot yet be reconciled with a complete, auditable cross-validation pipeline.

Outer-test selection. Section 5.1 defines five held-out test folds, each containing 400 trials. In train_staff.py, these same test folds are evaluated after every training epoch (lines 226–244). The script retains the epoch with the highest test Top-1 (lines 256–259), then reports its Top-1 and corresponding Top-5 in the fold averages (lines 261–285). There is no separate validation subset in this entry point. Thus its output is an average of test-selected maxima, not an independent estimate from held-out outer folds. The fact that test images are absent from gradient updates does not remove this selection bias. Its magnitude, or its differential effect on Pre versus Post, cannot be determined without result-matched outputs. A nested training/validation split or a prospectively fixed epoch rule is required, followed by evaluation on the outer fold. The baselines and label-shuffle procedure must use the same valid selection policy. If a different pipeline produced the tables, that pipeline and its selection records should be supplied.

Incomplete data/provenance path. The documented command fails at from data import BrainFaceDataset, EEGTensorDataset, load_brainface_dataset because data.py is absent from the supplied archive (train_staff.py:18). The README says review-time snippets are provided and uses ./sample_data in its command, but neither those snippets nor their loader is present. This is a specific missing executable dependency, not a demand for unconditional public release of human recordings. It also prevents checking whether Pre/Post folds contain the same image IDs: the visible helper stratifies label indices, and matching random seeds alone cannot ensure identical image-level splits if row order or artifact rejection differs between sessions. Appendix A.5 lists ICA, artifact rejection, and MVNN without enough implementation detail to establish which data fit their parameters. I have not confirmed preprocessing leakage; the split manifests, rejection counts, and train/test fitting scope are needed. The release also lacks the baseline, permutation, temporal, ERP, and geometry result-generation paths needed to connect the claimed analyses to the reported outputs.

A conditional numerical-protocol inconsistency. Under the stated three equally weighted seeds, five equal folds, and 400 hard Top-k decisions per fold, each participant mean must equal

Acc(\mathrm{Acc}(\\%)=100\frac{K}{3\times5\times400}=\frac{K}{60},\qquad K\in\mathbb{Z}.

Several displayed Table 1 means cannot arise by rounding this quantity to two decimals. For example, EEG-STAFF Post-Fam Subject 2 Top-1 is 7.76%, while the adjacent attainable values round to 7.75% and 7.77%; its Subject 1 Top-5 is 25.61%, between values rounding to 25.60% and 25.62%. The supplied evaluator uses integer hit counts and sample-weighted aggregation, so it does not explain these entries. Different retained trial counts, additional averaging, intermediate rounding, truncation rather than rounding, or another reporting policy could explain the discrepancy, but would need to be disclosed and reconciled with Section 5.1. This is a reporting/protocol check, not an allegation that the data were fabricated. Correct-count numerators and denominators per participant, seed, and fold would resolve it directly.

2. The controls do not isolate either familiarity-specific neural improvement or the distinctive spectral mechanism used to support the model contribution.

Familiarity and session order are confounded. All five participants undergo the same Pre-Fam → familiarization → Post-Fam order and see the identical 2,000 EEG images in both recording sessions (Sections 3.1–3.2). A separate familiarization image set is a useful control against memorizing those particular training images, but does not separate the intervention from repeated exposure to the EEG stimuli, time/session effects, or changes in task engagement. Neither another classifier nor an ERP difference on the same recordings resolves that confounding. The results could support an appropriately qualified pre/post association, but do not isolate acquired identity familiarity as the cause of increased neural discriminability. An exposure-matched control or another suitable design separating familiarity from order is needed for the stronger interpretation. I am not suggesting that familiarization can simply be undone to reverse the within-person order.

The temporal/representational evidence has narrower scope than the conclusion. Table 6 correctly uses participants rather than seeds as the statistical unit; a large t-value with n=5 is not inherently impossible. However, positive Pre/Post effects in individual windows do not establish that the 100–300-ms effect is larger than effects elsewhere; that requires a participant-level between-window contrast or condition-by-time analysis and a stated multiplicity policy. Figure 3b shows a representative participant, and Appendix A.7 specifies ROI averaging but not the sampling unit, participant-selection rule, SD definition, or timewise correction behind the shaded significance intervals. These details are necessary before using the plot as population-level temporal corroboration. Geometry from separately trained, test-selected encoders is not independent confirmation of a physiological mechanism; its fold-specific extraction/aggregation must also be specified so embeddings from independently fitted models are not silently pooled as one common coordinate system. The five-person homogeneous sample further limits population-level generalization.

The Fourier ablation is not a distinct intervention as currently implemented. models.py labels the configuration switches as corresponding to Table 4 (lines 26–30). In the WFSE class, mode="none" sets enhanced=v, while mode="vanilla" applies an orthonormal real FFT, multiplies by unit weights, and immediately applies the inverse FFT (lines 181–212). Both then use the identical pointwise nonlinear, residual, and normalization layers. Consequently,

irFFT⁡(1⊙rFFT⁡(v))=v,\operatorname{irFFT}(1\odot\operatorname{rFFT}(v))=v,

so these modes implement the same mapping, apart from numerical round-off, at matched weights and stochastic operations. They also both retain the spectral branch; none does not remove it. Limited double-precision checks found output differences below 1e-14 for the WFSE module and for matched reduced-dimension models containing all branches. This does not imply that finite-precision training trajectories must remain bitwise identical. It does mean that Table 4's 6.23% versus 7.17% Post-Fam comparison cannot, using these modes alone, establish a benefit from adding Fourier-domain information. If “w/o WFSE” instead removes other layers or the entire branch, the actual ablation implementation and capacity change must be provided. The weighted mode is genuinely different once its gains depart from one, and its gradient path works; the concern is the control and interpretation, not that the main spectral module is inactive.

The architecture otherwise combines explicitly acknowledged CBAM/Inception-style components with learned frequency gains and a gate. Reusing components is not itself a rejection reason, particularly for a study with an empirical neuroscience question. But removal ablations that also change capacity, and an identity FFT/IFFT control, do not establish a substantial independent algorithmic contribution. The reported Post-Fam advantage over DAEEGViT is 0.47 percentage points (Table 2), which could be useful but needs paired uncertainty after correcting evaluation. The low absolute accuracy is not by itself my objection.

Overall assessment. I recommend rejection in the current form. The main obstacles are the test-selected evaluation, the unresolved result/protocol correspondence, the pre/post confounding of the scientific interpretation, and the non-distinct Fourier control. These require corrected analyses, traceable implementations, and either stronger experimental controls or narrower claims—not just language changes. No conclusion of intentional fabrication is warranted from the current evidence. A valid held-out reanalysis and a reconciled account of the actual experimental/control procedures could materially change the assessment.

Questions for authors

1. What generated the reported cross-validation scores? Was the highest outer-test accuracy used to choose each epoch, as in the supplied entry point? Please identify the table-producing version and provide validation-selected or prospectively fixed-epoch outer-fold results for all methods and the shuffled-label control. Include the selection rule and participant/seed/fold correct-count denominators, particularly for the Table 1 entries inconsistent with the stated averaging grid.

2. How can the effect of familiarity be separated from the recording sequence? Was an exposure-matched comparison, untreated identity subset, or another suitable session/order control collected but not reported? Please specify EEG presentation order and randomization, session timing, training exposure, and retention/rejection counts. If no separating control exists, which conclusions are intended only as descriptive pre/post associations?

3. What exactly differs between the Fourier ablations? How do the Table 4 “w/o WFSE” and “Vanilla Fourier Transform” models differ, given that the supplied none and vanilla modes are mathematically equivalent? Please provide their result-matched implementations, parameter counts, and paired runs under a valid selection policy, and distinguish removing a branch from bypassing only its spectral operator.

4. Can the executable data and preprocessing path be supplied? Please provide the missing data.py, the advertised review snippets or an equivalent privacy-preserving loader example, and explicit paired Pre/Post image-ID fold manifests. Which observations fit ICA/MVNN and any other estimated preprocessing, and are changes in retained trials reflected in the denominators? Please also identify the temporal-window, baseline, and public-benchmark configurations.

5. What supports the temporal and geometry inferences at participant level? Please provide participant-level window differences, a direct contrast testing the claimed 100–300-ms concentration, the multiplicity policy, and the unit and correction method for ERP significance. Specify whether geometry was computed within each independently trained fold and how results were aggregated. These clarifications target the existing claims rather than requiring a new unrelated task.

Reviewer_CmPLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper raises an interesting question: can face familiarity acquired during an experiment make identity information in EEG easier to decode? In the experiment, participants sequentially completed pre-familiarization EEG recordings, face–name learning, and post-familiarization EEG recordings, and the pre- and post-familiarization recordings were classified separately. Meanwhile, the authors propose EEG-STAFF, which combines spatiotemporal features with spectral features through adaptive fusion. The analysis covers several perspectives, including decoders, temporal windows, ERPs, and feature geometry, and the proposed method achieves strong decoding results on the collected dataset and two public benchmarks.

Critical strengths
  1. This paper raises an interesting question by linking face familiarity to fine-grained EEG decoding. The study finds post-familiarization performance improvements across all participants, multiple decoding models, and different temporal windows. ERP analyses help elucidate the temporal characteristics of the observed differences.

  2. On the collected datasets, EEG-Face and THINGS-EEG, EEG-STAFF performs best among the compared methods.

Critical weaknesses
  1. The familiarization effect is confounded with repeated viewing. In the main experiment, the Post-Fam phase repeatedly presented the same set of 2,000 stimulus images shown in the Pre-Fam phase. Therefore, the observed performance improvement cannot be fully attributed to the familiarization procedure.

  2. The ablation experiments in Table 4 confirm the effectiveness of each component, but fail to isolate the advantage of adaptive feature fusion. Comparing the learned sample-dependent weights with fixed-weight fusion or feature concatenation would help clarify the added value of this design.

Questions for authors
  1. What evidence can distinguish " familiarization" from "repeated viewing"?

  2. How does adaptive fusion perform compared with fixed-weight fusion or feature concatenation?

  3. Did EEG-STAFF and the compared baselines converge within the reported 200 training epochs?

  4. Can the authors provide more details on the proposed method and its implementation to facilitate replication?

Reviewer_2TPtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses an interesting problem: decoding fine-grained facial identity from EEG signals. The authors propose an EEG spatio-temporal adaptive Fourier fusion network and achieve good performance with pre-trained familiarization. The model reveals consistent post-fam advantage during 100-300 ms interval.

Critical strengths
  1. This work provides a clear research pipeline for facial identity and designs a neural network for decoding tasks.
  2. Spatial and temporal analysis are given to see the brain pattern under the paradigm.
Critical weaknesses
  1. The overall decoding accuracy is relatively low. It may be useful to check the overall settings, such as data preprocessing, modeling, and classification, to improve performance.
  2. The work needs further comparison with other related works, not only in terms of decoding performance.
Questions for authors
  1. Are there some similar face paradigms that could be used for comparison? Their data could also be used to test the performance of the model.
  2. How did you choose the 40 identities? Do they have some clear differences in character?
  3. Have you performed temporal analysis for all subjects? The time scale could be more specific, and further spatial and spectral analysis would strengthen the results.
  4. The face recognition is not new, it's would be benefical to give a clearer statement of the novelty.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.