Shadow Brain: Biomimetic Learning for Decoding Visual Neural Signals

Visual neural decoding aims to infer the visual information perceived by an individual from brain activity and is key to connecting biological perception and artificial intelligence in brain–computer interface research. Visual information undergoes parallel processing, selective convergence, and hierarchical integration along biological visual pathways, giving rise to neural responses that vary across individuals. This variability, together with the low signal-to-noise ratio of electroencephalography (EEG), makes accurate alignment between visual information and EEG responses challenging. We propose Shadow Brain, a biomimetic visual–EEG learning framework built on the information-processing principles of the visual pathway from the retina through the lateral geniculate nucleus to the visual cortex. The visual branch draws on the functional sequence from retinal photoreception and signal processing to ganglion cell output. It constructs feature representations under five visual conditions and selectively integrates them using subject-specific learnable weights shared across stimuli within each subject. Inspired by population-level information integration in the visual cortex, the EEG branch employs NeuroWeft as a unified module to jointly model spatial relationships and temporal structure in multichannel EEG. Through contrastive learning, visual fusion, NeuroWeft, and cross-modal projectors are jointly optimized in a shared semantic space, adapting visual information integration to individual EEG signals. Within-subject 200-way image retrieval experiments using trial-averaged EEG from ten subjects in THINGS-EEG achieve mean Top-1 and Top-5 accuracies of 68.4% and 90.9%, respectively. Shadow Brain incorporates biological visual processing principles into visual–EEG correspondence learning, shifting from direct visual–EEG alignment to personalized visual–neural alignment constrained by biological visual processing principles.