acceptodds
Under review as a conference paper at ICLR 2027

EEG Does Not Spell What You Read: Semantic Neighbourhood Synthesis for Open-Vocabulary EEG-to-Text

Abstract

Open-vocabulary EEG-to-text asks a model to reproduce, word for word, the sentence a person silently read, given only their scalp EEG. We argue that this target is mis-specified for reading and that the field's metrics have concealed the problem. Verbatim decoding partly succeeds when the task is typing, where every character is time-locked to a motor event, but silent reading offers no such clock, and on our evidence the exact word sequence does not survive the skull, though its meaning does. Indeed, on our benchmark the best BLEU-1 and ROUGE-1 belong to a system whose real-EEG performance never rises above its own information-free controls. We recast the problem as semantic reconstruction and introduce Semantic Neighbourhood Synthesis, which treats a trial as a region of meaning rather than a word sequence. An EEGCLIP encoder retrieves the nearest training sentences, and a language model writes a new sentence from them. Under a uniform protocol with every baseline retrained from scratch, our method is best on nine of the ten metrics that score meaning, improving 24-way retrieval accuracy by relative, and is the only method whose information-free controls fall to chance. We then ask what limits performance, and the answer is not the decoder. Interventions spanning the front-end, the decoder and the signal itself all shift the score by less than run-to-run noise, while the same readout separates subjects and, above all, sentences, with effects and that of the entire representation. On this evidence, reading EEG carries the gist of a sentence and very little of its wording.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG-to-text, Semantic Reconstruction, Neural Decoding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_c2xsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

None

Critical strengths

None

Critical weaknesses
  1. The abstract is not readable.
  2. Figure creation confusion.
  3. EXPERIMENTAL issues.
Questions for authors
  1. The abstract is not readable. Hard to read and understand.
  2. The AI use statement does not include figure creation; was Figure 1 produced using AI tools?
  3. The EXPERIMENTAL SETUP is limited to support the claimed contribution.
  4. The RESULTS analysis, the description is very poor.
  5. Table 1 does not support the claimed statement.
  6. No github, the implementation details description is very bad in EXPERIMENTAL SETUP.
4 comments

Thank you for your review. The three Weaknesses correspond to the six Questions, so we answer them through the Questions:

  • Weakness 1 (The abstract is not readable): Q1.
  • Weakness 2 (Figure creation confusion): Q2.
  • Weakness 3 (EXPERIMENTAL issues): Q3–Q6 (setup, results analysis, Table 1, code and implementation details).

Q1–Q3 are answered here and Q4–Q6 in part 2. We mark the changes in the revised PDF in blue.

Q1. The abstract is hard to read.

We acknowledge that the abstract can and should be improved. The OpenReview abstract field lost several numbers and symbols in rendering (e.g. "34%" and the arrow in "EEG→CLIP"). That field is now fixed. We also rewrote the abstract itself in clear and structured language, one idea per sentence. We thank the reviewer for raising this issue and hope this version of abstract reads better:

Open-vocabulary EEG-to-text decoding asks a model to reproduce, word for word, the sentence a person silently read. We argue that this verbatim target is poorly matched to reading EEG and that n-gram metrics hide the problem: on our benchmark, the best BLEU-1 and ROUGE-1 belong to a system that never beats its own information-free controls. We therefore treat the task as semantic reconstruction, recovering what a sentence means rather than its exact words, and propose Semantic Neighbourhood Synthesis (SNS). SNS maps the EEG into CLIP's text-embedding space, retrieves the nearest training sentences, and has a language model write a new sentence from them. We retrain every baseline from scratch on a split in which no sentence appears in both training and test, and score all methods with one evaluation harness and the same information-free controls. SNS raises 24-way retrieval accuracy from 13.60% for the strongest baseline to 18.22%, a 34% relative gain. The gain holds when outputs are scored with independent text encoders that SNS never uses, against distractors from the same reading task, and for nearly every subject, while SNS stays at chance when the EEG is replaced by information-free input. Most of the gain comes from EEG-guided retrieval, and the synthesis step lets SNS write new sentences instead of copying training sentences. The model is subject-specific and does not yet transfer to unseen readers without calibration. Within the features and decoders we tested, reading EEG lets us recover what a sentence means, but little of its exact wording beyond what a language model already predicts.

Q2. Was Figure 1 produced with AI tools?

Yes. Figure 1 was generated with an AI tool, and the front-end schematic (Figure 5) was composed with an AI tool from panels our scripts generate from real EEG. Thank you for catching this, and we apologize: the AI-use statement in the submission did not describe how these figures were made. The revised statement now does so. Both figures are schematics that report no results; the numbers drawn in Figure 1 are illustrative, and the channel and dimension labels in Figure 5 are the actual specifications. Every data figure, table and number in the paper is produced by our code from stored results. The revised statement also lists each other use of AI tools: aiding and polishing the writing, drafting sections, finding related work, and research ideation and execution with coding assistants, all under author direction and review.

Q3. The experimental setup is too limited to support the claims.

We have strengthened it and made it explicit in a new setup table (Table 14, referenced from §4):

revised setup
DataZuCo 1.0 and 2.0, 30 readers; test sentences held out, with duplicate sentence texts removed from training
BaselinesEEG2Text, GLIM and SemKey; all retrained from scratch with official code
Fairnessevery method on the same test trials, scored by one harness
Controlsshuffled EEG and two noise inputs per method, to catch outputs that ignore the EEG
Robustnessmultiple random splits; four independent evaluation encoders (new); per-subject results (new, Figure 2b); unseen readers (new)
Statisticspaired, sentence-clustered bootstrap confidence intervals; three training seeds

The new experiments directly test the claims: the margin holds under four encoders our pipeline never uses, on every converged random split and for 28 of 30 readers. They also expose a limitation we now state plainly: the model does not yet transfer to readers unseen in training (§8).

Continued in part 2.

Q4. The results analysis is poorly described.

We agree it needed a clearer structure. §5 now leads with the main result and then breaks it down:

  1. SNS leads on meaning-level retrieval (Table 1). The margin holds under four independent evaluation encoders, with distractors from the same reading task, and across random splits (Figure 2a; Tables 6 and 9).
  2. A new per-subject figure (Figure 2b) shows that SNS is ahead of SemKey for nearly every subject.

§5 then explains why the word-overlap metrics behave differently, and §6 (Figure 3) measures how much of each metric survives when the EEG is destroyed. We also added example outputs (best, typical and worst cases) beside those of SemKey, GLIM and our own system on shuffled EEG (Tables 20–21).

Q5. Table 1 does not support the claimed statement.

We have rewritten the paper to state exactly what Table 1 supports. Together with the controls in Figure 3a, it supports two claims:

method24-way, real EEGinformation-free controlBLEU-1
EEG2Text4.324.4033.14
GLIM12.585.1721.89
SemKey13.606.9820.68
SNS18.224.2720.35
  1. SNS leads meaning-level retrieval, 18.22 against 13.60 (95% CI of the difference +4.01 [+2.50, +5.58]).
  2. Word-overlap metrics can be won without using the EEG: EEG2Text has the best BLEU-1, yet its real-EEG retrieval never beats its own control.

We agree the submission's claim to be "best on nine of the ten metrics that score meaning" was loose, since it counted several closely related retrieval scores separately. We removed that count from the abstract, text and Table 1, and state only the two claims above; their confidence intervals are in Table 15. If you had a different statement in mind, we would be glad to address it.

Q6. No code; the implementation details are poorly described.

The anonymous code is at https://anonymous.4open.science/r/eeg2text_sns-78BD. It contains the evaluation harness, all five methods, the frozen splits, every stored model output (so every table can be re-scored on a CPU), and step-by-step reproduction instructions, including how to obtain ZuCo. In the paper, the details are now grouped by topic: §4 and Table 14 (setup, baselines, metrics, statistics, compute), Appendix A (EEG front-end), Appendix B (model, training and the exact synthesis prompt), and Appendix C (how each method's test trials are selected and the fairness checks).

We hope these revisions address your concerns, and would be glad to clarify anything further.

I will maintain my original rating.

Thank you for your response. Your comments help us improve the quality of the paper, including our analysis and experimental setup, readability of abstract, AI use statement, and the release of repository.

Reviewer_gUKoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper argues that verbatim transcription is a poorly specified objective for silent-reading EEG and reframes open-vocabulary EEG-to-text as semantic reconstruction. It proposes Semantic Neighbourhood Synthesis, which maps EEG into a semantic embedding space, retrieves a neighbourhood of related training sentences, extracts consensus semantic cues, synthesizes new candidate sentences with an instruction-tuned language model, and selects among them using EEG-predicted embeddings. The paper further evaluates all methods with information-free shuffled/noise controls and argues that retrieval-based semantic metrics reflect EEG information more reliably than conventional n-gram metrics such as BLEU.

Critical strengths
  1. The paper provides a careful validity analysis of EEG-to-text evaluation. Information-free controls show that some conventional text-overlap metrics can remain high even when the EEG signal is destroyed, highlighting an important evaluation problem in this literature.

  2. The semantic reconstruction results are consistently stronger than the listed baselines. The proposed method is best on nine of ten semantic metrics, with additional multi-split and bootstrap evidence supporting the gain.

  3. The experimental protocol is unusually controlled and fair. All baselines are retrained from scratch, evaluated on the same 3,166 held-out trials, and scored with one locked evaluation harness, substantially reducing comparison confounds.

Critical weaknesses
  1. Most of the performance gain appears to come from semantic retrieval rather than the full synthesis pipeline. Pure retrieval already achieves 16.32 24-way accuracy, while reranking reduces performance to 15.52 and the full synthesis pipeline reaches 17.81. This suggests that the main contribution may lie in EEG-to-neighbourhood retrieval, while the additional generation and selection stages provide comparatively limited gains.

  2. The headline retrieval metric is partially coupled to the model’s embedding choices. MPNet is involved in the model pipeline and is also used for the 24-way retrieval evaluation, making the main metric not fully independent of the representation spaces used during decoding. Although the paper reports additional metrics and information-free controls, an evaluation using a completely independent semantic encoder would strengthen the evidence.

  3. The comparison against prior semantic-reconstruction methods is not yet complete. The paper does not include SID [1], which shares the central idea of decoding higher-level semantics from EEG before text reconstruction rather than directly modeling verbatim output. A direct comparison would better position the contribution of semantic-neighbourhood synthesis.

Questions for authors
  1. How robust are the semantic gains to using a completely independent evaluation embedding? In particular, can the authors report retrieval accuracy with an encoder that is not used anywhere in Stage 1 or Stage 4?

  2. Could the authors further clarify the contribution of each stage to the final reconstruction quality?

4 comments

Thank you for your thoughtful and generous review. We agree that EEG-guided retrieval carries most of the gain, and the revision now says so directly. We answer the Weaknesses first (W1–W2 here, W3 in part 2), then your two Questions in part 2. Changes in the revised PDF are marked in blue.

W1. Most of the gain comes from retrieval; generation and selection add comparatively little.

We agree, and the paper now says so in the abstract, §1 and §7. The numbers you cite came from a single seed of the submission's ablation setting (original split before text deduplication, one prompt sampled at temperature). We apologize for the rush of experiments and now provide a full set of experiments. We re-measured every stage on the deployed system over three seeds. Each row is a different output of the same trained model:

output of the same trained modelMPNetGTR-T5E5-large-v2
top-1 retrieved neighbour16.1115.4517.22
synthesis, one candidate picked at random17.2817.1917.91
synthesis + Stage-4 selection (SNS)18.2217.9819.08
side check: selector on the neighbours, no synthesis16.5215.5117.62
SemKey, strongest baseline13.6013.6714.40

Retrieval carries most of the gain: the top-1 neighbour alone already beats SemKey by +2.51. Synthesis plus selection add +2.11. Reranking the neighbours alone, which looked harmful in that single seed, has no reliable effect either way. We still see two reasons to keep generation: it is what makes every output a newly written sentence, and an oracle choosing the best of the 16 candidates would reach 29.57, so the candidates hold far more than the current selector recovers. We present this as the clearest direction for future work.

Revision in the paper: §1 (Contributions), §7, Figure 4, Table 2 and Table 17.

W2. The headline metric is coupled to MPNet, which the model also uses.

We agree this needed a direct test rather than mitigation. Our pipeline uses CLIP, Gemma-3, Qwen3, MPNet and BGE, and BERTScore uses RoBERTa. We re-scored every method, including under the information-free controls, with four encoders from other model families:

encoderSNSSemKeyGLIMSNS − SemKey [95% CI]SNS control
MPNet (paper metric)18.2213.6012.58+4.01 [+2.50, +5.58]4.27
GTR-T5-large17.9813.6713.12+3.92 [+2.47, +5.41]4.40
Sentence-T5-XL16.6012.7412.41+3.62 [+2.30, +5.02]4.51
E5-large-v219.0814.4012.69+4.02 [+2.56, +5.54]5.22
E5-Mistral-7B19.5514.3113.29+4.80 [+3.32, +6.35]4.65

The ranking and the margin hold under every encoder, with every confidence interval above zero, and SNS's controls stay near chance (4.17). We also removed the coupling from the method itself: the MPNet-free system reaches 17.85, and a CLIP+BGE selector matches the original three-space selector, so we now recommend it.

Revision in the paper: §4, §5 (Figure 2a), §7 (Table 2), §8 and Table 15.

Continued in part 2.

W3. No comparison with SID.

Thank you for the pointer. SID (Li et al., 2026) shares our premise of decoding semantics before text: it predicts a set of semantic units and has a language model assemble them into a sentence. SNS differs in the unit of decoding: it retrieves whole sentences and has a language model rewrite their shared meaning. The revised paper now discusses SID in the related work.

We also began porting SID's public code into our protocol (same split, controls and scorer; two proprietary parts replaced with open ones: BGE-large for its embedding space and Qwen3-4B as its sentence writer). We share the preliminary numbers for transparency:

method24-way, real EEGinformation-free controlgap
SID port (released code)5.154.22+0.93
SID port (encoder trained end-to-end)5.504.21+1.29
SNS18.224.27+13.96

We do not consider these numbers confirmed. They are close to chance and far below what SID reports in its own setting. SID's published results come from per-subject models and use different metrics in a proprietary embedding space, so they cannot be compared with ours, and we cannot yet tell how much of the gap comes from our harder protocol and how much from our port. A faithful comparison needs more time: we plan to first reproduce SID's published results under its own protocol, then move it to ours. Until then we do not report SID's performance in the paper. We are afraid that this may not be completed before the rebuttal period.

Revision in the paper: §2 (new paragraph on SID).

Q1. Retrieval accuracy with an encoder used nowhere in Stage 1 or Stage 4.

The table under W2 (part 1) answers this directly. GTR-T5, Sentence-T5, E5 and E5-Mistral are used nowhere in our pipeline. Under each of them SNS stays ahead of SemKey by 3.6 to 4.8 points, with every confidence interval above zero, and its information-free controls stay near chance.

Revision in the paper: §5 (Figure 2a) and Table 15.

Q2. The contribution of each stage.

The stage table under W1 (part 1) gives the full breakdown, over three seeds and three encoders. In short:

  • Retrieval is the main contribution: the top-1 neighbour alone already beats SemKey by +2.51. Replacing the EEG-retrieved neighbourhood with random sentences drops the system to near chance, so the EEG enters at this stage.
  • Synthesis adds +1.17 even with a random pick, and turns every output into a new sentence rather than a corpus copy.
  • Selection adds a further +0.94 over a random pick from the same candidates, reliably across seeds and encoders.
  • Reranking the retrieved neighbours without synthesis has no reliable effect: the difference from the top-1 neighbour changes sign across seeds, so it is not part of the deployed pipeline.

Revision in the paper: §7, Figure 4, Table 2 and Table 17.

We hope these additions strengthen your assessment, and would welcome any further suggestions.

Reviewer_gUKoOpenReview ↗

Official Comment

Thank you for the detailed response and clarifications. I will maintain my original rating.

Thank you for your response and for your positive assessment. Your questions about independent encoders and the contribution of each stage shaped our revision. If there is a specific point that would further strengthen the paper in your view, we would be glad to address it.

Reviewer_J2w8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that verbatim EEG-to-text decoding is poorly suited to natural reading and proposes Semantic Neighbourhood Synthesis (SNS) for semantic reconstruction. The method maps EEG signals into CLIP space through multiple encoders, retrieves semantically related sentences, and uses a language model to synthesize and select a final prediction.

Critical strengths
  • The proposed retrieval-and-synthesis framework aligns with the goal of semantic reconstruction. It combines EEG-grounded semantic retrieval with language-model generation, enabling flexible reconstruction beyond fixed corpus sentences without requiring explicit EEG-to-word alignment.
  • The paper provides fair baseline comparisons, information-free controls, and extensive ablations. These experiments support the reported improvements and offer useful insights into the reliability of EEG-to-text evaluation.
Critical weaknesses
  • The experiments show that the evaluated models struggle with verbatim reconstruction, but do not establish that exact lexical information is absent from reading EEG. The limited gains from architectural changes do not establish that EEG decoding has reached its intrinsic limit.
  • The evaluation does not test generalization to unseen subjects. Moreover, the use of MPNet in both the method and the primary evaluation metric raises concerns about metric alignment.
  • Several components offer limited benefits: reranking retrieved neighbours reduces accuracy (16.32% to 15.52%), while removing content cues barely affects performance (17.81% to 17.75%). These findings leave the complexity of the nine-encoder pipeline insufficiently justified.
Questions for authors
  • See the weakness point 2: How well does the method generalize to unseen subjects?
  • Could you compare Stage-4 reranking against random candidate selection using the same 16 generated candidates to better isolate its contribution?
4 comments

Thank you for your careful and fair review. We answer the three Weaknesses here and the two Questions in part 2. We marked the changes in the revised paper in blue.

W1. The evidence does not show that lexical information is absent from EEG, or that decoding has reached an intrinsic limit.

We agree on both counts. Our experiments bound the features and decoders we tested, not the information in the signal, and our manuscript should not claimed more than that. We have rewritten both claims:

  • Lexical information. We no longer say the wording "does not survive the skull". We now say that, with the methods we tested, sentence meaning is recoverable (24-way retrieval is +14.0 above the information-free controls), but little exact wording is recoverable beyond a language prior (BLEU-1 is only +4.0 above them). We also state that this bounds our methods, not the information in the signal (§8).
  • Intrinsic limit. We no longer say the decoder is not the constraint. We now report only what we measured: most interventions we tried have small effects, with clear exceptions such as the multi-window front-end. §8 adds that better front-ends, subject-adaptive models or more data may recover more, including words.

Revision in the paper: abstract, §1, §7, §8, and Appendix A (front-end ablation).

W2. Unseen subjects; MPNet used in both the method and the main metric.

  • Unseen subjects: see Q1 (part 2).
  • MPNet coupling: we agree this needed a direct test. We re-scored every method with four encoders from model families our pipeline never uses:
encoderSNSSemKeySNS − SemKey [95% CI]
MPNet (paper metric)18.2213.60+4.01 [+2.50, +5.58]
GTR-T5-large17.9813.67+3.92 [+2.47, +5.41]
Sentence-T5-XL16.6012.74+3.62 [+2.30, +5.02]
E5-large-v219.0814.40+4.02 [+2.56, +5.54]
E5-Mistral-7B19.5514.31+4.80 [+3.32, +6.35]

The margin holds under every encoder. We also removed MPNet from the method altogether; this MPNet-free system reaches 17.85.

Revision in the paper: §5 (Figure 2a), §7 and Table 15.

W3. Several components add little (reranking neighbours, content cues), so the pipeline's complexity is not justified.

Our original ablation study came from a single seed of the ablation setting: the original split before text deduplication, and one prompt sampled at temperature instead of 16 reordered prompts. The -0.80 you cite was therefore one noisy draw. We apologize for the rush of experiments and provide a full set of experiments. We now re-measured every stage on the deployed system over three seeds. Each row is a different output of the same trained model:

output of the same trained modelMPNetGTR-T5E5-large-v2
top-1 retrieved neighbour16.1115.4517.22
synthesis, one candidate picked at random17.2817.1917.91
synthesis + Stage-4 selection (SNS)18.2217.9819.08
side check: selector on the neighbours, no synthesis16.5215.5117.62
SemKey, strongest baseline13.6013.6714.40

What this shows, and what we changed:

  • Why reranking looked harmful before and not now. Reranking the neighbours alone has no reliable effect in either direction. In the submission, the same model gave -0.80 on the test fold but +0.69 on the validation fold. On the deployed system the difference ranges from -0.29 to +1.20 across seeds and encoders, and its 95% interval contains zero in eight of nine cases. The -0.80 in the original submission was therefore one noisy draw.
  • Synthesis helps in both settings: +1.49 over the top-1 neighbour in the submission, and +2.11 with selection on the deployed system, positive in every seed.
  • Retrieval carries most of the gain: the top-1 neighbour alone already beats SemKey by +2.51. The paper now credits the stages in this order.
  • Content cues help only slightly (removing them costs -0.67). The paper now says they are not load-bearing.
  • Simplification: we now recommend the two-space selector (Q2, part 2) and report the MPNet-free system (W2), and we state which components carry the result (front-end, retrieval, synthesis, selection) and which are marginal (cues, and fusing the seven Stage-1 pathways, which adds about 0.3).

Revision in the paper: §7, Figure 4 (stage comparison, three seeds, two encoders) and Table 2. The submission's single-seed decomposition stays in Appendix H (Table 22), now labelled with its setting and with a caption explaining why its reranking result was noise.

Continued in part 2.

Q1. How well does the method generalize to unseen subjects?

It does not yet, and we now state this as the main limitation. We held seven of the 30 readers out of all training and evaluated seen and unseen readers on the same sentences:

readers24-wayinformation-free control
seen in training17.414.60
unseen3.924.54
unseen, second seed4.894.52
unseen + label-free per-reader centering4.704.30

Unseen readers are at chance: the EEG-to-CLIP map is subject-specific, and a simple calibration that uses no labels does not fix it. Cross-subject transfer will need subject-adaptive training or alignment, which we name as the main open problem. Within the standard pooled protocol, SNS is ahead of SemKey for 28 of 30 readers individually. In the paper: abstract, §5 (Figure 2b), §8, and Appendix F (Tables 18 and 19).

Q2. Stage-4 selection vs a random pick from the same 16 candidates.

We appreciate this question since this is the right control. Because each trial is scored on its own output alone, any rule over the same 16 candidates can be scored exactly, with a random pick being the average over the 16. We also score with two encoders our pipeline never uses (GTR-T5 and E5-large-v2):

rule (same 16 candidates)MPNetGTR-T5E5-large-v2
random pick17.2817.1917.91
Stage-4 selector, three spaces (submitted)18.2217.9819.08
Stage-4 selector, CLIP+BGE only18.0417.8918.96
oracle best-of-1629.5729.6832.06

Selection gives a modest but reliable gain over a random pick (+0.94 under MPNet, +0.79 under GTR-T5, +1.16 under E5; the confidence interval excludes zero in every seed under every encoder). The two-space CLIP+BGE selector is as good as the three-space one, so we now recommend it: it is simpler and removes MPNet from selection. The oracle row shows the candidates hold far more than any selector recovers, which makes selection the clearest target for future work.

Revision in the paper: §7, Table 2 and Table 17.

We hope these results resolve your concerns, and would be grateful if you would consider revisiting your score.

Thanks for the reply and detailed explanation. The rebuttal addresses most of my concerns and I will raise my score to 3.

Thank you for engaging with our response and for raising your score. Your questions about selection against a random pick, the MPNet coupling and unseen readers led directly to the new controls and the stated limitation, and the paper is now more comprehensive because of them. If any concern remains, we would be glad to address it.

Comments on the paper

1 comment

We thank the reviewers for careful reviews. Our claim in brief: from scalp EEG recorded during silent reading, the meaning of a sentence is far more recoverable than its exact wording, and word-overlap metrics hide this, since a system that ignores the EEG can win them. Semantic Neighbourhood Synthesis (SNS), built for meaning-level reconstruction, leads on meaning-level retrieval while its information-free controls stay at chance.

A correction we found ourselves. While preparing the rebuttal we checked the split at the level of sentence text, not just sentence ID. ZuCo reuses some sentences across its tasks, so a few held-out sentences also appeared, under a different ID, in our method's training data and retrieval corpus. The baselines' official pipelines already exclude such duplicates; ours did not. Our outputs never reproduced a held-out sentence verbatim, but we retrained our method on a split with these duplicates removed. The test set is unchanged, so the baseline numbers stand, and the result holds:

24-way retrievalsubmittedafter the fix
SNS18.2818.22
SemKey, strongest baseline13.6013.60
relative gain+34%+34%
SNS − SemKey, 95% CI+4.60 [+3.15, +6.16]+4.01 [+2.50, +5.58]
SNS information-free control4.314.27

All results for our method in the main comparisons and in these replies come from the corrected models; the few submission analyses kept in the paper (front-end and design sweeps) are labelled with the split they used.

What the revision adds:

questionanswer
Is the margin an artifact of MPNet?No. It holds under four encoders our pipeline never uses, and an MPNet-free version of SNS reaches 17.85
Which stages matter?Retrieval carries most of the gain (top-1 neighbour 16.11); synthesis and selection add +2.11 (full system 18.22)
Does selection beat a random pick?Yes, modestly: +0.79 to +1.16 points over a random pick from the same candidates, reliably across seeds and encoders
Unseen subjects?A real limitation: readers held out of training are at chance (3.92), and we now say so
Per subjectSNS beats SemKey for 28 of 30 readers (new Figure 2b)
SIDDiscussed in the related work; a preliminary port is shared in the reply to gUKo but not reported in the paper until it is verified

Other changes. Anonymous code, with every stored model output so all tables can be re-scored on a CPU: https://anonymous.4open.science/r/eeg2text_sns-78BD. The OpenReview abstract field lost several numbers and symbols in rendering; it is now fixed. The AI-use statement now also describes how the two schematics (Figures 1 and 5) were made (see the reply to c2xs). Claims are narrowed throughout, and every change is marked in blue in the revised PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.