Instruct, Reason, Decode: Fine-Grained Semantic Instruction and Reasoning for EEG-to-Text Decoding

Decoding open-vocabulary text from electroencephalography (EEG) remains challenging because of the low signal-to-noise ratio of EEG and the complexity of natural language. Although large language models can generate coherent text, limited understanding of EEG semantics can lead to excessive reliance on linguistic priors. During autoregressive generation, early prediction errors can also affect subsequent predictions. Existing approaches improve EEG–text alignment or introduce auxiliary semantic tasks, but their sentence-level learning objectives and coarse-grained prompts provide limited guidance about the specific content to express. To address these challenges, we propose SIRE (Semantic Instruction and Reasoning from EEG), an EEG-to-text framework that strengthens fine-grained semantic understanding and provides semantic guidance for generation. Task-specific question–answer instruction provides explicit learning targets for decoding evaluation aspects, judgments, factual categories, and semantic relations from EEG. Reasoning from EEG then adaptively selects semantic questions and integrates the predicted answers through chain-of-thought reasoning before sentence generation. These answers remain available throughout autoregressive decoding, allowing the model to use the predicted semantics alongside EEG representations and previously generated text. Experiments on ZuCo 1.0 and ZuCo 2.0 show that SIRE outperforms the compared baselines across multiple semantic decoding metrics under fully autoregressive generation. Ablations support the contributions of semantic instruction and reasoning, while input controls show that generation benefits from correctly paired EEG.