How Does Pre-Trained Face Familiarity Affect the Decodability of Identity-Specific EEG Representations?

While EEG-based visual decoding has attracted enormous interests and illustrated strong potentials over recent years, fine-grained facial identity decoding directly from brain-perceived EEGs has proved to be extremely difficult and challenging. In this paper, we investigate whether a pre-trained face familiarity can improve the discriminability and decodability of identity-related EEG representations. To address the challenge of identity decoding with fine-grained representations and sharing of highly similar visual features, we propose an EEG Spatio-Temporal Adaptive Fourier Fusion Network (EEG-STAFF) to effectively model complementary spatiotemporal and spectral information, which is supported by our establishment of a controlled and pre-trained familiarization paradigm for facial identity decoding. Experiments consistently show the improved identity decoding performances with the pre-trained familiarization, where the strongest gains emerge at the duration of 100-300ms, whist representation analysis reveals greater within-identity compactness and inter-identity separation. Compared with the existing representational state of the arts in relevant areas, the proposed EEG-STAFF achieves the best overall decoding performances, suggesting that the pre-trained face familiarization enhances the identity related neural representations and improves the EEG-based identity decoding.