Connecting Brain Activity to Language through Mixed-Resolution Visual Representations

Brain-to-text decoding aims to describe viewed scenes from fMRI recordings. Pretrained vision-language models provide rich visual and linguistic priors, and their visual representations determine the spatial detail that brain decoders are trained to predict. Inspired by the non-uniform allocation of visual information in human perception, we propose MindMRQ, which uses adaptive visual granularity as a bridge between cortical representations and language models. Its mixed-resolution visual representations retain finer detail in regions sensitive to spatial compression and encode surrounding context more coarsely while maintaining full image coverage. We then connect brain activity to these visual targets through dual-level visual-semantic alignment, combining region-level matching with query-level semantic supervision to support language generation from brain-predicted representations. Extensive experiments demonstrate competitive decoding performance, with controlled comparisons and ablations supporting the effectiveness of the proposed representations and alignment strategy.