- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors present an approach for open-vocabulary continuous non-invasive brain-to-text decoding called CoMET. The main innovation in CoMET is to provide the preceding sentence as context to the model before autoregressively predicting the current sentence from the corresponding brain activity. CoMET is trained by aligning corresponding embedding outputs to word embeddings from a frozen text encoder. At inference-time, the model sees sentence-level brain activity. An adapter is also trained to map the output of the encoder to the input of a frozen LLM, which produces the final autoregressive text predictions. The authors test their approach on two datasets and provide ablations against random context and random brain activity controls.
Critical strengths
- The paper is well-written and generally easy to follow
- The idea of using prior context is simple and well-motivated
- Training with a word-aligned contrastive loss over frozen word embeddings, but operating on sentence-level MEG signals at test time is a unique and interesting strategy
- Table 2 provides a thorough and convincing set of ablations
Critical weaknesses
W1: The main issue with this work is that I do not agree that providing the previous sentence as context is practical or typically available at inference. Providing the previous sentence(s) as context is equivalent to autoregressively predicting the next sentence with teacher-forced tokens for the previous sentence. While I agree that this is better than teacher-forcing every token as some prior work has done, this still does not represent a practical open-vocabulary decoding setting in my view. The authors state that “conditioning on preceding context is practically realizable at inference time” unlike teacher-forcing and that the “preceding context is available at inference”. This is an odd claim to make as teacher-forcing can be thought of as assuming all prior tokens are correctly predicted, and CoMET makes only the slightly weaker assumption that the prior sentence is correctly predicted. I would not say this is “practically realizable” as the decoder needs to have predicted the previous sentence correctly. This makes the comparisons in Table 1 particularly unfair as CoMET is provided information from an oracle for its context.
To overcome this limitation, the authors should test what happens when the previous sentence context comes from the decoder’s own predictions of the last sentence, instead of assuming perfect reconstruction of the previous sentence. How does CoMET perform in this setting?
W2: In section 2.2, the authors introduce their strategy for aligning MEG to text with a contrastive loss between encoded word-aligned MEG and word embeddings from a frozen text encoder. This is very reminiscent of the strategy used in [A] and the authors should cite this work here. At the moment, the paper presents this approach as if it is novel.
W3: LibriBrain contains stories from the canon of Sherlock Holmes and these stories are available in popular public domain text repositories such as Project Gutenberg. As a result, these stories are highly likely to have appeared in the pre-training data of most modern LLMs, leading to a potential data leak. The LLM could use the prior sentence context to continue the story it has already memorised in pre-training. The “Random MEG + Context” ablation in Table 2 somewhat mitigates this concern, so this is only a minor weakness, but I would be keen to understand the authors’ perspective on this problem.
W4: The authors do not provide any uncertainties via training seeds / bootstrap error. It is very difficult to determine which of the results in their tables are significant without some indication of error.
[A] d’Ascoli, Stéphane, et al. "Towards decoding individual words from non-invasive brain recordings." Nature Communications 16.1 (2025): 10521.
Questions for authors
Q1: The authors say in the abstract that the “scarcity of paired MEG and text” is the main limitation for learning signal-to-text mappings. What makes the authors believe MEG provides sufficient information to recover text if MEG-text data were in abundance?
Q2: In 2.2, the authors say a “fixed-size window captures surrounding signals rather than restricting each embedding to the exact word interval”. Does this mean that windows overlap? If this is the case, the authors should be careful about the potential for learning timing shortcuts in training as this can leak the interval between words [B].
Q3: Have the authors considered using LibriBrain100 [C] instead of LibriBrain? It provides almost twice the amount of data and could lead to better results.
Q4: Why do the authors think the length-related effect occurs where increasing M tends to produce longer generated outputs? Did the authors try mitigating this through their prompt? It would otherwise be natural to expect more context to improve decoding performance further.
[B] Jayalath, Dulhan, and Oiwi Parker Jones. "Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text." arXiv preprint arXiv:2609.40359 (2026).
[C] Mantegna, Francesco, et al. "LibriBrain100: One hundred hours of broad and deep MEG data for neural speech decoding at scale." arXiv preprint arXiv:2608.25204 (2026).
13 comments
Response to Reviewer jspo (2/3)
W3. Possible memorization of Sherlock Holmes
This is a valid concern. We cannot rule out the possibility that the Sherlock Holmes texts were included in the pretraining data of the LLM. However, such memorization does not explain the gains of CoMET. If the LLM were continuing a memorized story from the preceding sentence, the target should be recoverable from context alone without MEG.
(1) Context alone does not recover the target. Given only context, the pretrained LLM achieves a BLEU-1 of 4.08 and a BERT† of −0.07. Even when generation is stopped at the end of the first sentence to match output length, it reaches only 9.21 and 0.11, far below the 16.75 and 0.26 achieved by CoMET under the same condition.
(2) The same holds after training on the training data. The LLM fine-tuned with LoRA on the training split also achieves a BLEU-1 of 3.82 and a BERT† of −0.07, and only 7.22 and 0.12 when stopped at the end of the first sentence. A model that trains an adapter with a dummy prefix using the same LLM and context is also lower than CoMET on all metrics (see our response to Reviewer pBR8, W1 & Q2).
(3) Performance collapses when the MEG comes from another sentence. As in the Random MEG + Context condition mentioned by the reviewer, replacing only the MEG with the signal of another sentence while keeping the ground-truth context drops BLEU-1 from 16.58 to 5.47 and BERT† from 0.24 to 0.05. Even when the MEG is replaced with one that matches the prefix length, the results are a BLEU-1 of 4.17 and a BERT† of 0.06.
(4) Qualitative example. In Table 4(c), CoMET exactly recovers "I have no complaint to make.", whereas Context only, given the same context, generates a completely different sentence. If memorization were responsible, Context only should also have generated the target.
(5) Reproduction on another dataset. GWilliams uses four short fictional stories selected from the Manually Annotated Sub-Corpus (MASC) of the Open American National Corpus as stimuli, which are not as widely known as Sherlock Holmes. On this dataset as well, adding context increases BLEU-1 from 6.08 with MEG only to 10.33 with CoMET, reproducing the effect of context. We added this point to Section 4.3.
In summary, context alone does not recover the target, performance collapses when the MEG is replaced with that of another sentence, and the same effect appears on stories that are not widely known. Therefore, the gains of CoMET come from the combination of sentence information in the MEG and the context, rather than from memorization by the LLM.
Q1. Why MEG would provide sufficient information with more data
Thank you for the question. The following evidence shows that the information that can be decoded from MEG increases with more training data and observations.
(1) The original LibriBrain paper (Özdogan et al., 2025) showed that performance on speech detection, phoneme classification, and word classification improves substantially as the amount of training data increases.
(2) [A] reported that word decoding accuracy increases with the amount of training data and with the number of observations averaged at test time.
(3) In our results as well, although the two datasets differ in subject composition and stimuli, which makes it difficult to attribute the difference to data quantity alone, LibriBrain, with about 52 hours collected from a single subject, achieves higher performance than GWilliams, with about 2 hours per subject.
Therefore, we expect that more linguistic information can be extracted from MEG as data increase. Moreover, CoMET proposes a direction that, even in the current data-limited regime, combines the information in MEG with linguistic context to exploit as much prior information as possible.
[A] Stéphane d'Ascoli, Corentin Bel, Jérémy Rapin, Hubert Banville, Yohann Benchetrit, Christophe Pallier, and Jean-Rémi King. Towards decoding individual words from non-invasive brain recordings. Nature Communications, 16:10521, 2025.
Response to Reviewer jspo (3/3)
Q2. Overlapping windows and timing shortcuts
Thank you for the comment. Yes, the windows of adjacent words overlap. However, timing shortcuts do not occur in CoMET.
The timing shortcut in [B] arises when the model jointly receives overlapping windows cropped from the raw signal at each word onset, as in [A]. In this case, the model can infer the interval between words, i.e., word length, by comparing how far the same signal is shifted across windows.
In contrast, CoMET first encodes the MEG of the whole sentence without word-boundary information, then crops word-level windows from the output, pools each of them independently, and compares each only with the embedding of its own word. Although the windows overlap, they are never compared with each other, so timing shortcuts structurally cannot occur. Moreover, Stage 2 decodes sentences from the whole sentence-level MEG without word-level segmentation, so it uses no word timing at all and is unrelated to timing shortcuts. [B] also notes that such leakage is not an issue in settings where word onsets are unknown. We added this explanation to Section 2.2.
Q3. LibriBrain100
Thank you for the suggestion. LibriBrain100 was released on arXiv in August 2026, about a month before the submission deadline, after our experiments were already underway. We believe LibriBrain100 is particularly valuable for this work. With about 80 hours from a single subject, it allows us to examine how performance changes with data quantity, and since it includes stimuli other than Sherlock Holmes, such as podcasts and random sentences, it also allows a direct test of the memorization concern raised in W3. Although we could not use it in this work, we will use LibriBrain100 to examine these questions in future work. Thank you for pointing us to this valuable dataset.
Q4. Effect of M on output length
Thank you for the question.
Why outputs become longer. The average output length increases from 12.6 words for M=1 to 14.6 words for M=2 and 15.0 words for M=3, and the maximum length increases from 60 to 107 words (Section 4.6). Since the MEG prefix is placed at the beginning of the template, a larger M lengthens the text between the MEG and the generation position. We presume that this weakens the influence of the MEG during generation and lets the language prior of the LLM drive the output, producing longer and more elaborate outputs. This is consistent with the trend in Section 4.7, where larger LLMs generate longer outputs.
Effect of more context. Moving from no context (M=0) to M=1 greatly improves performance, and beyond M=1 the performance remains stable regardless of M. On LibriBrain, M=2 and M=3 achieve higher BLEU-1/2/3 than M=1, but the differences are small compared with the difference between M=0 and M=1. That is, a single preceding sentence is sufficient to anchor decoding, and further increasing M does not keep improving performance. We presume the following reasons. First, the preceding sentence is the most directly related to the content of the next sentence, and earlier sentences provide relatively little additional information for narrowing the candidates. Second, as explained above, the relative influence of the MEG weakens as M grows, partly offsetting the benefit of more context. Third, the longer outputs at larger M slightly worsen WER and increase ROUGE-1 recall, so these length effects in the metrics partly mask the improvement in content recovery.
W4. Uncertainty estimates
Thank you for the comment. We agree that uncertainty estimates are necessary to determine which results are significant. We are currently training CoMET and the word-level and sentence-level alignment variants with three seeds, retraining both Stage 1 and Stage 2 for each seed, and computing paired bootstrap 95% confidence intervals of the differences between methods (see also our response to Reviewer pBR8, Q6). We will share the results as soon as they are available.
[A] Stéphane d'Ascoli, Corentin Bel, Jérémy Rapin, Hubert Banville, Yohann Benchetrit, Christophe Pallier, and Jean-Rémi King. Towards decoding individual words from non-invasive brain recordings. Nature Communications, 16:10521, 2025.
[B] Dulhan Jayalath and Oiwi Parker Jones. Removing timing shortcuts improves non-invasive brain-to-text. arXiv preprint arXiv:2609.40359, 2026.
Response to Reviewer jspo (1/3)
We thank the reviewer for the constructive comments. We address each point below.
W1. Practicality of the preceding-sentence context and evaluation with generated history
Thank you for the comment. Teacher forcing provides the ground-truth tokens of the sentence being decoded as input. In contrast, CoMET provides only the complete sentences preceding the sentence being decoded as context, and generates the sentence being decoded from MEG from beginning to end. There are also practical situations in which the preceding sentence is available as text. For example, when reading, the preceding sentence is already given as text, and in dialogue, the interlocutor's previous utterance can be obtained by speech-to-text (STT). Moreover, as the reviewer suggested, when the preceding sentence decoded by the model (generated history) is used as the context for the next sentence, CoMET is lower than with the ground-truth context, but higher on most metrics than the baselines that process each sentence independently, the setting without context (MEG only), and the setting with an irrelevant context (MEG + Random Context). Some baselines achieve higher R-R because of a bias related to output length. Thus, even when the ground-truth preceding sentence is unknown, the approach of CoMET that uses the preceding sentence as context is effective, and its advantage in Table 1 comes not only from access to the ground-truth context but also from the way it decodes by leveraging context. We added this result to Appendix E of the revised manuscript.
W2. Relation to [A]
Thank you for the comment. The word-level alignment of CoMET is similar to [A] in that both contrastively align word-aligned MEG responses with frozen word embeddings. However, the two methods differ in how windows are processed. [A] segments the raw signal at word onsets and jointly encodes the overlapping windows of a sentence, whereas CoMET first encodes the whole sentence without word-boundary information and then extracts word-level windows from the encoder output and pools each of them independently. Therefore, the timing shortcut raised in Q2 occurs in [A] but structurally does not occur in CoMET (see Q2). In addition, [A] predicts word-level embeddings given word-onset timing and retrieves words from a fixed vocabulary, leaving the combination with language models to future work. In contrast, CoMET uses this alignment only as a training signal for the encoder, and at inference it generates sentences by feeding the whole sentence-level MEG to the LLM as a prefix without word timing. Moreover, the core contribution of CoMET lies not in word-level alignment itself but in sentential context, i.e., leveraging complete preceding sentences as context. To correct the missing citation, we cite [A] in Section 2.2 and state the difference in window processing between the two methods.
[A] Stéphane d'Ascoli, Corentin Bel, Jérémy Rapin, Hubert Banville, Yohann Benchetrit, Christophe Pallier, and Jean-Rémi King. Towards decoding individual words from non-invasive brain recordings. Nature Communications, 16:10521, 2025.