Mind-BIRTH: Learning Hierarchical Canonical Cortical Representations for Zero-Shot Cross-Subject Visual Decoding
Abstract
Reconstructing perceived images from human fMRI offers a way to study neural representations and visual processing, but most existing approaches remain strongly subject-specific. Cross-subject decoding is particularly challenging because individuals differ in voxel coverage, spatial organization, and response distributions, making it difficult to learn representations that transfer reliably to unseen subjects. We propose Mind-BIRTH, a framework for learning hierarchical canonical cortical representations for cross-subject visual decoding. Mind-BIRTH organizes visual cortical responses into Early, Middle, and Late stages, converts heterogeneous ROI responses into coordinate-anchored tokens, and maps them into stage-wise canonical spaces using visual-semantic alignment. The learned representations are then decoded through hierarchy-matched diffusion conditioning, preserving the correspondence between cortical hierarchy and generator depth. This design supports zero-shot reconstruction of held-out subjects without target-subject training and lightweight few-shot personalization when limited target data are available. Experiments on NSD show that Mind-BIRTH achieves strong semantic reconstruction performance among cross-subject methods under both zero-shot and few-shot settings while remaining competitive in low-level fidelity. Controlled ablations further validate the importance of stage-wise cortical representations, canonical alignment, and hierarchy-matched routing. These results suggest that learning transferable, hierarchy-aware cortical representations provides an effective inductive bias for cross-subject visual decoding.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.