Beyond Aggregate Gains: Evaluating Robust Cross-Subject EEG-to-Text Retrieval

Cross-subject EEG-to-text retrieval aims to recover linguistic content from in- dividuals unseen during model development, yet improvements in aggregate re- trieval performance do not necessarily indicate consistent benefits across unseen subjects. This distinction is especially important when architectural effects are comparable in magnitude to variability across subjects and stochastic training runs. We conduct a controlled evaluation under a leave-one-subject-out setting with training-only normalization, subject-disjoint model selection, canonical retrieval over 400 unique sentence contents, and three fixed training seeds. On a cleaned five-subject ZuCo sentence-reading setting, we compare a Graph-Free EEG Trans- former with prototype-based subject interaction, multi-head low-rank adaptation (MHLR), a Controlled BPR-style dense EEG–text baseline, and a deterministic Ridge control. The Graph-Free reference achieves 0.544 ± 0.407% Unique-Sentence Top-1 and 2.549 ± 0.855% Unique-Sentence Top-5 accuracy. Prototype-Only SIG attains the highest descriptive means (0.576 ± 0.293% and 2.765 ± 0.828%), but its paired gains over Graph-Free are only +0.032 and +0.216 percentage points and vary in both magnitude and direction across held-out subjects and training seeds. MHLR exhibits similarly heterogeneous effects, while the Controlled BPR-style and Ridge baselines do not outperform the Graph-Free reference under the same retrieval task. These experiments provide a concrete case in which a positive aggregate cross- subject retrieval gain does not translate into a consistent improvement across un- seen subjects and stochastic training trajectories. The results motivate evaluat- ing cross-subject EEG–language retrieval with subject-level effects and stochastic replication alongside aggregate performance.