NEAR-Bench: Benchmarking Predictive Performance and EEG Input Dependence in EEG-to-Text Decoding

Advances in electroencephalography (EEG) representation learning and pretrained language models have driven research on decoding text from noninvasive brain signals. However, differences in data splits and evaluation tasks hinder direct comparison across studies; even under identical evaluation conditions, higher text scores do not necessarily indicate greater use of EEG corresponding to the target text. We introduce NEAR-Bench, an evaluation and diagnostic benchmark that jointly assesses text prediction performance and dependence on EEG input. Built on ZuCo 1.0 and 2.0, NEAR-Bench standardizes data splits, candidate text identities, and reporting rules, jointly holding out participants and texts during downstream training. For sequence decoders, we fix the trained model, replace only its EEG input, and compare predictions on the same observations. We also evaluate candidate identification separately from text generation without candidate sentences or reference prefixes. Results show that better text prediction does not necessarily accompany stronger EEG input dependence: some sequence models outperform an independent language-model reference, yet their prediction scores and generation behavior change little when EEG values are zeroed or replaced with other real recordings. NEAR-Bench provides a common basis for comparing model performance under explicit conditions, testing the contribution of correctly paired EEG input, and interpreting progress in EEG-to-text.