acceptodds
Under review as a conference paper at ICLR 2027

SpectraLoom: Spectral-Attentive Multi-Scale Neural Decoding for Open-Vocabulary EEG-to-Text Translation

Abstract

Open-vocabulary EEG-to-text decoding generates the sentence a participant reads from the electroencephalogram recorded at each fixated word, and is a step towards non-invasive brain–computer interfaces for communication. Existing decoders flatten the eight frequency bands of every word event into one vector, model temporal context at a single scale, and connect the EEG encoder to a pretrained language model through a fixed linear projection. We propose SpectraLoom, in which each component answers one of these limitations: (1) Spectral Band Attention (SBA) computes a word-specific gate over the eight band slices with 315 parameters and retains all 840 input features; (2) a multi-scale Conv1D–BiLSTM encoder combines parallel convolutions over three, five and seven neighbouring word events with sentence-wide bidirectional recurrence; and (3) a Cross-Attention Bridge (CAB) refines every word-aligned EEG state by attending over the whole sentence before it enters the BART encoder. We evaluate five systems trained under one protocol, with a sentence-grouped split, identical decoding and three seeds, on ZuCo 1.0 and zero-shot on ZuCo 2.0. SpectraLoom attains the highest ROUGE-L and the lowest word error rate on both corpora (16.80 and 97.55 in domain, 17.65 and 92.70 zero-shot) with 44% fewer parameters than T5-large, and it attains the highest teacher-forced BLEU-1 to BLEU-4. Over three seeds, removing or simplifying any component lowers BLEU-1 to BLEU-4 and ROUGE-L on both corpora, and a word-independent band gate costs 4.87 BLEU-1. Across three seeds, its BLEU-1 varies with a standard deviation of 0.74, against 3.35 for BrainTranslator with the same BART-large backbone. Code is available at https://github.com/anonymoususer98700/spectraloom.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG-to-text decoding, brain-computer interfaces, spectral band attention, multi-scale encoding, cross-attention, open-vocabulary generation, ZuCo, neuroscience-informed architectures

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_7Wp4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SpectraLoom, an open-vocabulary EEG-to-text architecture operating on word-level EEG features extracted during natural reading. The model introduces three components: (1) Spectral Band Attention, which produces a word-specific softmax gate over eight EEG frequency-band slices while retaining the full 840-dimensional input; (2) a multi-scale encoder combining parallel one-dimensional convolutions with kernel sizes 3, 5, and 7 and a three-layer bidirectional LSTM; and (3) a cross-attention bridge that allows each word-aligned EEG representation to attend over the full EEG sequence before being supplied as input embeddings to BART-large.

Critical strengths
  1. Careful architectural specification and matched experimental implementation

The paper describes the architecture, training procedure, tensor shapes, trainable parameter subsets, decoding settings, split construction, and random seeds in substantial detail. Four baselines are retrained using the same split, epoch budget, early stopping rule, decoding configuration, and three seeds. This is consequential because EEG-to-text results are sensitive to evaluation protocol and decoding choices, and locally matched comparisons are more interpretable than comparisons copied from papers using incompatible splits.

  1. Appropriate attention to ablations, free generation, and stability

The paper does not rely exclusively on teacher-forced metrics. It reports free beam-search generation, output uniqueness, WER, multiple overlap metrics, component ablations, per-seed variability, and computational cost. Each proposed component is replaced with an alternative of the same output width, and the full model is reported as more stable than the two BART-based front ends across the three selected seeds. These are useful experimental practices for a benchmark where generation can otherwise be dominated by language-model behaviour.

Critical weaknesses
  1. The input controls fail to establish that the model uses EEG values

The zero-input and noise-input models match or outperform the real-EEG model on several central metrics, particularly in zero-shot evaluation. On ZuCo 2.0, zero input with the mask retained exceeds real EEG in BLEU-1, BLEU-4, ROUGE-L, and output uniqueness; the model without the mask also obtains higher ROUGE-L and lower WER. Thus, the experiments do not demonstrate that the claimed performance comes from neural information rather than the pretrained language model, sequence mask, sentence length, dataset distribution, or training-set language regularities.

  1. The evaluation does not isolate generalisation across readers, and output degeneracy remains severe

The split groups repeated presentations of the same normalised sentence, which appropriately prevents the same sentence from appearing in multiple partitions. However, the paper does not report a reader-held-out ZuCo 1.0 evaluation. Therefore, the in-domain experiment does not establish generalisation to unseen individuals, and may permit the model to exploit participant- or session-specific regularities. The ZuCo 2.0 evaluation uses different participants and sessions, but also introduces a corpus/session/task shift, making it difficult to attribute performance to subject generalisation specifically.

Questions for authors
  1. Input controls: Can the authors run the real-EEG, Gaussian-noise, zero-input, and zero-input-without-mask experiments for all three seeds and report paired differences with uncertainty? The present single-seed controls appear to show that zero input outperforms real EEG on several headline metrics, especially on ZuCo 2.0. If that result persists, what evidence remains that SpectraLoom decodes information from EEG values?

  2. Correspondence controls: Can the authors evaluate a training-time and test-time permutation control in which complete EEG sequences are reassigned across sentences while preserving sequence length, participant, task, masks, and the marginal EEG distribution? Gaussian noise and zero input alter the input distribution, whereas sentence-level or word-level permutation would directly test whether the EEG-text correspondence contributes.

  3. Length and mask leakage: How much of performance can be obtained from sequence length alone? Please include a mask-only or length-only baseline with learned constant embeddings, and group results by sentence length. The strong control performance suggests that the number of fixated words or valid positions may provide substantial information.

  4. Reader-held-out evaluation: Does the sentence-grouped ZuCo 1.0 split share readers across training, development, and test sets? If so, can the authors report a split that holds out readers, ideally crossed with sentence grouping? This would help distinguish linguistic decoding from reader- or session-specific fitting.

  5. Statistical support: Are the improvements over T5-large or other baselines statistically reliable under free generation? For example, the in-domain ROUGE-L difference between SpectraLoom and T5-large is 0.38 points, while SpectraLoom’s reported seed standard deviation is 0.99. Please report paired seed-level results or confidence intervals and avoid interpreting small mean differences without uncertainty-aware comparison.

  6. Generated-output analysis: What are the most frequent generated sentences, and what fraction of outputs belongs to the top 1, 5, and 10 generated strings? Given the very low uniqueness rates, qualitative examples and frequency statistics are necessary to determine whether metric gains reflect improved conditioning or a better corpus-level default prediction.

  7. Biological interpretation of SBA: Are the learned band weights consistent across seeds, subjects, tasks, or words, and do they correspond to reproducible spectral patterns? The current ablation supports the utility of input-dependent gating, but does not establish that the gate discovers meaningful neurophysiological frequency selectivity rather than functioning as generic feature attention.

Reviewer_pkvVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

SpectraLoom combines a word-specific eight-band spectral gate, parallel convolutions with window sizes of three, five, and seven, followed by a BiLSTM, and a cross-attention bridge to BART-large. It compares five systems using a sentence-grouped ZuCo 1.0 split, three training seeds, and zero-shot evaluation of the ZuCo 1.0 checkpoints on ZuCo 2.0. The proposed architecture leads in ROUGE-L and word error rate (WER) among the locally evaluated systems, with component ablations and additional remove-and-retrain EEG input controls in Appendix B.

Critical strengths
  1. The baseline comparisons are more controlled than comparisons assembled from published numbers. BrainTranslator, EEG2Text-FA and the alternative sequence decoders are evaluated with the same decoding settings and three seeds. The paper appropriately distinguishes free generation from teacher-forced next-token prediction (Tables 1–2).

  2. The architecture is examined with actual component substitutions. Table 3 contrasts the full system against uniform and static band gates, removal of multi-scale encoding, and replacement of CAB by a linear mapping on both ZuCo corpora. Releasing the unfavorable input-control results also makes the central claim testable.

Critical weaknesses
  • The paper's own no-EEG controls directly undercut the claimed EEG-specific contribution. Appendix B, Table 4 retrains the proposed model after replacing every EEG feature value by zero or Gaussian noise. On ZuCo 2.0, the zero-valued model with the valid-position mask retained improves BLEU-1 from 16.29 (real EEG) to 18.64, BLEU-4 from 0.73 to 1.04, and ROUGE-L from 16.74 to 18.35. Even Gaussian-noise training surpasses the real-EEG model in several ZuCo 2.0 metrics; removing both values and masks yields ROUGE-L 19.33 versus 16.74 and a lower WER. These are single-seed remove-and-retrain comparisons, not same-checkpoint causal interventions, so they do not prove that EEG never carries useful linguistic information. They nevertheless create a direct and unresolved contradiction for a paper whose three claimed innovations are designed to extract more language-relevant information from EEG. Crucially, the usual zero-input condition still retains the valid-position mask derived from gaze-defined word events, so the language model can exploit sequence structure without EEG values. The authors have not established a benefit from the neural measurements themselves, let alone from physiologically meaningful spectral selection. This is a fundamental failure of evidence for the central contribution, not a routine missing ablation.

  • The autonomous outputs do not substantiate open-vocabulary sentence decoding, and the reported gains are not consistently stronger than the controlled baselines. In Table 1, SpectraLoom's free-generation BLEU-4 is only 0.38 on ZuCo 1.0 and 0.69 on ZuCo 2.0; WER remains 97.55% and 92.70%, respectively. Only 6.68% and 1.16% of generated sentences are distinct across observations, indicating substantial output collapse. T5-large has higher in-domain BLEU-1, BLEU-3 and BLEU-4; BrainTranslator has higher zero-shot BLEU scores. SpectraLoom's strongest claims are therefore limited to ROUGE-L and WER differences in a regime with poor sentence-specific recovery. The additional teacher-forced BLEU advantage largely reflects next-token prediction with the reference prefix and cannot establish autonomous semantic reconstruction. Finally, the combination of SBA, the Conv1D–BiLSTM encoder, and CAB adapts established attention and sequence-modeling modules; Table 3 shows metric changes when components are removed, but, given Table 4, provides no evidence that these changes correspond to exploiting the distinctive physiological information in the EEG bands.

Questions for authors
  1. How do the authors reconcile Table 4's higher zero-EEG and noise-input scores with the main claim that spectral attention and multi-scale neural encoding improve EEG-specific language decoding? Can they report same-checkpoint experiments that replace paired EEG with real donor recordings over all three seeds, with fixation sequence length and masks held fixed?
  2. What sentence-specific content, rather than corpus templates, is actually recovered? Please quantify accuracy in recovering content words and named entities and paired-input semantic gains, especially given 1.16% distinct generated sentences on ZuCo 2.0.
Reviewer_gcC7OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SpectraLoom, an EEG front end for open-vocabulary EEG-to-text decoding on ZuCo. Each fixated word is represented by 840 band-power features (8 frequency bands x 105 electrodes). The features pass through (1) Spectral Band Attention (SBA), a word-specific softmax gate over the 8 band slices with 315 parameters; (2) parallel Conv1D branches (kernels 3/5/7) followed by a 3-layer BiLSTM; and (3) a "Cross-Attention Bridge" (CAB). The output replaces the input embeddings of a partially fine-tuned BART-large. Under a sentence-grouped split, the authors retrain four baselines with identical training budget and decoding over three seeds. They report the highest ROUGE-L and lowest WER on ZuCo 1.0 and zero-shot on ZuCo 2.0, ablations in which every component helps, and lower seed variance than BrainTranslator. Input controls following Jo et al. (2024) are reported in Appendix B.

Critical strengths
  1. Careful evaluation protocol. The split groups all readings of a sentence into one partition and removes ZuCo 2.0 overlap from training. All baselines are retrained with the same budget and decoding, free-running generation is the primary metric, results cover three seeds, and code is released. This matters because earlier ZuCo results are hard to trust under sentence leakage. It is the main reason the paper's numbers can be examined at all.
  2. Input controls are reported (Appendix B). Following Jo et al. (2024), the authors retrain the model on noise and zero inputs. This is good practice and makes the central claim checkable, although, as explained below, the controls do not support the claim.
Critical weaknesses
  1. The paper's own input controls give no evidence that the model uses the EEG. In Table 4, a model retrained on all-zero input scores about the same as, or higher than, the model trained on real EEG on both corpora. Gaussian-noise input also scores close to real EEG. Without the mask, the zero-input model outputs essentially one sentence for every trial, yet it achieves the highest ZuCo 2.0 ROUGE-L in the paper (19.33, vs. 17.65 for SpectraLoom). The paper reports these controls but does not interpret them. The headline scores can therefore be matched without any brain signal, most likely through BART's language prior plus sentence length. This undermines the central claim, and with it the ablation and stability claims. Removing a single module lowers BLEU-1 by several points (Table 3), while removing the EEG entirely does not (Table 4, single seed), so the ablations reflect optimisation effects rather than information extracted from EEG. Likewise, a front end that converges more reliably to a language-prior solution is not a better EEG decoder.

  2. The differences are close to trivial-output levels and are not statistically supported. SpectraLoom's WER is barely below the 100% that an empty output would get, and several baselines are above it. In domain, Pegasus-XSum, which outputs only one or two distinct sentences, reaches almost the same WER as SpectraLoom. Outputs are highly repetitive (Uniq. column of Table 1). On ZuCo 2.0, BrainTranslator wins every BLEU metric while producing only a handful of distinct sentences. BLEU and ROUGE on this benchmark therefore reward template-like output. Against this background, the headline margins are within seed variance; for example, the in-domain ROUGE-L lead over T5-large is smaller than SpectraLoom's own standard deviation. The ablation gaps also come with large standard deviations over only three seeds. The paper reports no significance tests and no generated examples.

  3. (Minor) Claims not supported by Table 3. Section 3.3 cites a BLEU-3 drop, and the abstract and contribution list claim drops on BLEU-1 to BLEU-4 on both corpora. Table 3, however, reports no BLEU-3 at all and no BLEU-2 for ZuCo 2.0.

  4. (Minor) Teacher-forced results presented as a contribution. The paper itself cites Jo et al. (2024) on why teacher-forced scores overstate decoding quality, yet Table 2 is listed as a contribution. The teacher-forced margins are also within noise.

Questions for authors
  1. How do you interpret Table 4? Over all three seeds and with a significance test, does real-EEG training outperform zero and noise training on any metric?

  2. Please evaluate the real-EEG-trained checkpoints at inference time with (a) Gaussian noise and (b) EEG taken from a different sentence of the same length. If the scores do not drop, the model is not using the content of the EEG.

  3. Please run the same input controls for at least one baseline, for example BrainTranslator: zero/noise input at training time, and mismatched-sentence EEG at inference time. If the baselines also score the same without real EEG, the comparisons in Table 1 measure how well each system fits the language prior, not how well it decodes EEG, and the claimed advantage of SpectraLoom's front end would need to be restated. Please also show randomly sampled generations with their references.

Reviewer_LzctOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SpectraLoom decodes the sentence a person reads from ZuCo word-level EEG. It adds three modules before a partly fine-tuned BART-large: a 315-parameter word-specific band gate (SBA), a multi-scale Conv1D–BiLSTM encoder, and a Cross-Attention Bridge. Five systems are trained on ZuCo 1.0 with a sentence-grouped split and compared over three seeds, in domain and zero-shot on ZuCo 2.0.

Critical strengths
  1. A careful protocol. The paper uses a sentence-grouped split, makes free generation the primary evaluation, gives all systems the same budget and decoding, and reports input controls following Jo et al. (2024). This is what makes the central claim testable.

  2. A compact, well-specified idea. SBA is a 315-parameter gate that becomes an identity map under a uniform distribution. It merits testing on a task with a demonstrated EEG signal.

Critical weaknesses

1. The evidence does not show that the scores come from sentence-specific EEG

The paper's own controls undercut its central claim. In Table 4, on ZuCo 2.0, real EEG has the lowest ROUGE-L (16.74) and the highest WER (94.92) of the four training inputs. Zero input without a mask produces essentially one sentence (Uniq. 0.02%), yet reaches ROUGE-L 19.33 and WER 91.37, better than every system in Table 1. In domain, zero input with the mask beats real EEG on BLEU-1 (18.45 against 17.80) and is level on ROUGE-L (16.53 against 16.59). The paper reports these results but does not interpret them.

In fairness, the model does respond to its input: in domain, real EEG beats Gaussian input on ROUGE-L (16.59 against 15.46). Table 4 also uses a single seed, so it is not conclusive. However, the paper offers no other evidence that the scores depend on EEG content, and it reports no test with EEG from a different sentence, which would be the direct check. The outputs are also highly repetitive, with only 6.68% and 1.16% of generated sentences distinct (Table 1).

Table 1 therefore cannot be read as a ranking of decoding quality.

2. The component and ranking claims rest on three-seed means

The paper reports no significance test, and the margins are small relative to seed variation. The in-domain ROUGE-L lead over T5-large is 0.38, while SpectraLoom's own standard deviation is 0.99. On ZuCo 2.0, every BLEU-1 ablation gap in Table 3 is smaller than the standard deviation of the ablated variant; for example, the uniform gate scores 14.50 ± 3.72 against 16.33 ± 0.60.

The ablation also sits badly with the controls. A static gate is reported to cost 4.87 BLEU-1, while zero input does not lower BLEU-1 at all in Table 4. A module can hardly matter more than the signal it processes, so the ablation gaps may reflect training variation rather than information from EEG. A more stable front end may likewise converge more reliably to the same language-prior outputs.

The metrics also reward template output. Pegasus-XSum produces one or two sentences per seed, yet its in-domain WER (97.95) is close to SpectraLoom's (97.55).

Questions for authors
  1. Do the scores drop if the trained checkpoints receive EEG from a different sentence of the same length?
  2. Do the baselines also keep their scores without real EEG? If so, Table 1 cannot show an advantage for SpectraLoom's front end.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.