Mind the Parameters: Lightweight and Efficient Brain Visual Decoding with Shared Tensor Cores

Linking brain activity to computational representations of visual perception is a central goal at the intersection of neuroscience and machine learning. However, cross-subject decoding from fMRI remains challenging because neural responses vary substantially across individuals, while existing methods often rely on subject-specific fine-tuning or parameter-heavy decoders. This limits generalization to unseen subjects and hinders efficient deployment. We propose BrainTC–Brain Tensor Cores, a lightweight framework for cross-subject brain visual decoding based on shared tensor decomposition. BrainTC separates shared functional structure from subject-specific variation through a shared tensor-core alignment module that maps ROI-wise responses into a common functional space with shared tensor bases and residual subject factors. This supports both zero-shot decoding for unseen subjects and few-shot adaptation by updating only residual parameters. We then develop a Brain Tensor-Transformer with tensorized attention to compactly model inter-ROI dependencies, together with a hierarchical neural-to-visual mapping module for multi-level visual prediction. Experiments show that BrainTC achieves competitive cross-subject decoding performance with substantially improved parameter efficiency, particularly in zero-shot generalization to unseen subjects.