Context-Aware Open-Vocabulary MEG-to-Text Decoding with Large Language Model

Decoding magnetoencephalography (MEG) signals into text holds immense potential for restoring communication through non-invasive brain-computer interfaces, but remains highly challenging. The scarcity of paired MEG and text data limits models from learning robust signal to text mappings, causing prior approaches to fall short in the open-vocabulary setting. We address this challenge by reframing open-vocabulary decoding as a context-guided inference task, where MEG signals carry the linguistic content and preceding context guides the language model toward decoding it. From this point of view, we present a novel Context-aware MEG-to-Text decoding framework (CoMET) that leverages preceding context as a semantic anchor, narrowing the textual search space. CoMET first aligns MEG signals with corresponding text at the word-level, and subsequently maps these representations into the latent space of a large language model under contextual conditioning. Extensive evaluations on two MEG-text datasets demonstrate that while prior open-vocabulary decoders collapse to near random performance, CoMET achieves state-of-the-art open-vocabulary decoding performance, highlighting its potential for practical brain-to-text decoding. Code will be made available upon publication.