MAS-BIT: A Multi-Agent System for End-to-End BraIn-to-TexT Decoding

Speech brain-computer interfaces aim to restore communication by translat- ing neural activity into text, yet end-to-end decoders continue to trail cascaded systems in accuracy. We introduce a Multi-Agent System for BraIn-to-Text (MAS-BIT), the first multi-agent framework for end-to-end brain-to-text decod- ing from intracortical recordings. MAS-BIT coordinates three specialized agents: a neural agent that interprets continuous neural representations, a phonetic agent that decodes soft phoneme posteriors, and an arbitration agent that integrates their complementary evidence with direct access to neural representations to generate the final transcription. The expert agents communicate with the arbiter through token-embedding-based latent communication, constructing continuous messages as probability-weighted combinations of token embeddings. These messages re- tain alternative token hypotheses and provide a differentiable interface through which sentence-level supervision jointly trains the neural encoder and specialized agents. MAS-BIT achieves state-of-the-art end-to-end decoding performance on the Brain-to-Text ’24 and ’25 benchmarks, with word error rates of 8.418% on ’24 and 4.878% on the ’25 public test set (5.288% on the private test set). These results narrow the gap to cascaded systems without ensembling independently trained models or using external language-model rescoring.