NEST: A Large-Scale Benchmark for Predicting fMRI in Native Anatomy from EEG and Structural MRI

Predicting functional MRI (fMRI) from non-invasive electrophysiology (EEG) could enable cost-effective, high-resolution brain imaging, yet existing benchmarks omit subject-specific anatomies and rely on small cohorts that include fewer than $40$ subjects. We propose NEST (Native-anatomy EEG–sMRI–fMRI Tri-modal benchmark), a large-scale benchmark for predicting 4D fMRI in native anatomy from EEG and structural MRI during naturalistic viewing. By aligning separately recorded EEG and fMRI through a shared film timeline, rather than requiring simultaneous recording, NEST scales to $904$ and $963$ subjects watching two films, over $40\times$ more subjects than the largest public EEG–fMRI dataset. Models are evaluated on unseen subjects by voxel-level reconstruction, regional temporal correlation, and functional connectivity. We also propose NESTOR, which uses modality-specific conditioning, in which the subject's sMRI serves as a static anatomical condition and the EEG as a time-varying condition. NESTOR outperforms existing EEG-to-fMRI models by a clear margin, highlighting the importance of the subject's native anatomy. We demonstrate that NESTOR transfers to unseen movies in a zero-shot setting, indicating that it truly predicts fMRI from EEG signals rather than merely memorizing stimulus-specific fMRI patterns.