SANE: Stimulus-Adaptive Neural Evidence for fMRI Visual Reconstruction

What part of an fMRI response actually supports a decoded visual prediction? Modern fMRI-to-image systems can reconstruct increasingly rich visual content, yet reconstruction accuracy alone does not reveal which neural measurements support a particular prediction. We introduce Stimulus-Adaptive Neural Evidence (SANE), which scores anatomically defined cortical parcels, selects a fixed-budget stimulus-adaptive subset before cross-parcel contextualization, and tests the selected evidence through retention, removal, and cross-stimulus substitution for sufficiency, functional importance, and content specificity, respectively. On the official 1,000-image NSD shared test set across four subjects, a K=32 evidence budget retains only 39.15% of available cortical parcels while preserving 97.89% of dense DINOv3 decoding performance, reducing neural-side computation from 1.073 to 0.445 GFLOPs. Yet this strong compressibility does not imply interchangeability: removing SANE-selected evidence reduces target cosine similarity by 0.1428, compared with 0.0354 for uniform-random removal and 0.0843 for voxel-count-matched removal, while cross-stimulus substitution produces a donor-directed shift of 0.4523, compared with 0.2226 and 0.3869 for the corresponding controls. The selected evidence also shows a reproducible stimulus-dependent component: repeated measurements of the same image yield more similar selected parcel sets than measurements of different images. Finally, the same frozen evidence mechanism remains usable across five heterogeneous pretrained reconstruction systems through lightweight backbone-specific adapters. These results reveal a striking asymmetry in fMRI visual decoding: dense cortical responses are highly redundant for sufficiency, yet the compact evidence identified by SANE remains functionally structured and stimulus-dependent. SANE thus shifts the question from how much neural data can be discarded to which measured neural signals actually support a decoded visual prediction.