acceptodds
Under review as a conference paper at ICLR 2027

Whose Brain Is It? Reader Identity Accounts for Almost All of EEG-to-Text Retrieval, and a Per-Reader Adapter Recovers It

Abstract

EEG-to-text systems are evaluated on splits that hold out sentences but keep every reader in training, so the reported numbers describe a new sentence read by a known person. We evaluate the setting a deployment would face, a reader absent from training. Tiling all 30 ZuCo readers into five disjoint folds under duration-matched candidate pools, retrieval on a held-out reader is 0.0458, 1.10× chance and not distinguishable from it, against 0.1090, 2.62× chance, for readers present in training on identical pools: 94% of the above-chance margin is reader identity rather than language. The same protocol on ChineseEEG, a different language, paradigm and laboratory, loses 96%, and no label-free normalisation moves it. We then close the gap. Freezing the reader-agnostic encoder and giving each new reader a residual linear remap of the channel axis, at most 16,384 parameters and exactly the identity at initialisation, reaches the seen-reader ceiling on ZuCo from 400 labelled sentences and beats full fine-tuning at matched labels on ZuCo and on clause-level ChineseEEG. Four controls isolate the mechanism: matching the learning rate does not reproduce it, a shared readout with no per-reader parameters is the worst arm, the adapter on a random frozen encoder stays at chance, and a transplanted adapter loses the entire gain. Pre-training and calibration are separately near-worthless and jointly super-additive. Both halves survive a 4 Hz high-pass; the advantage of ZuCo's best electrode subset, which is entirely frontal, does not, a confound we report against our own recommendation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG-to-text, brain decoding, cross-subject generalization, subject adaptation, evaluation protocol, contrastive retrieval, low-rank adaptation, ZuCo, ChineseEEG

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ShurOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper audits sentence-retrieval evaluation in EEG-to-text systems. It argues that sentence-held-out splits overstate generalization because every reader remains represented in training. Under five leave-six-reader-out ZuCo folds and 24-way duration-matched retrieval pools, the submission reports top-1 accuracy falling from 0.1090 for reader-seen evaluation to 0.0458 for unseen readers, close to the 0.0417 chance level. It then introduces an identity-initialized per-reader residual channel adapter fitted with labelled target-reader trials and reports recovery toward the seen-reader ceiling, outperforming full fine-tuning at matched label counts.

Overall, the paper contributes: (i) a reader-held-out evaluation protocol for EEG-to-text retrieval; (ii) evidence that most performance under the conventional seen-reader protocol does not transfer to unseen readers; (iii) a compact per-reader calibration mechanism; and (iv) a careful analysis of calibration cost, architecture dependence, cross-corpus behaviour, and ocular or montage-related confounds.

Critical strengths
  1. A consequential evaluation correction: the reader-disjoint protocol directly targets the generalization regime relevant to deployment and controls candidate-pool difficulty.

  2. Strong diagnostic experimentation: the adapter controls, alternate architectures, high-pass analysis, and second-corpus replication are unusually thorough and transparent.

Critical weaknesses
  1. Causal overinterpretation plus uncertain inference unit: the experiments show failure of cross-reader transfer, not an information decomposition into reader identity versus language; pool-level bootstrapping may understate uncertainty across 30 readers.

  2. Adapter mechanism and practical value remain unresolved: the adapter may learn session/cap/artifact mappings rather than stable reader neurophysiology, and 400 labelled trials is a substantial calibration requirement.

  3. “Reader identity” is stronger and more specific than the experiments establish. Holding out a reader shows that the model depends on reader-correlated information and that this dependence is not repaired by the tested label-free normalisations. It does not by itself separate neural idiosyncrasy from head geometry, electrode placement, impedance, recording-session effects, ocular behaviour, or other reader-correlated acquisition factors.

Questions for authors
  1. Can you report per-reader effects and a reader-clustered or hierarchical bootstrap/permutation CI for the seen–unseen difference and the 94%/96% lost-margin statistic, including seed uncertainty?

  2. Are adapter calibration, validation/model selection, and final retrieval pools strictly stimulus-disjoint? Additionally, can ChineseEEG support training an adapter on one novel/session and testing it on the other for the same reader? This would distinguish stable reader calibration from same-session acquisition mapping.

  3. What exactly is included in the “94%” and “96%” attribution to reader identity? Do the authors intend “identity” to mean any stable reader-correlated signal, including recording and behavioural nuisance factors, or specifically a physiological reader signature?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Shur(Q1&Q2)

Thank you for the careful review. All three of your questions led to new analyses, and the answer to Q2 changes how we interpret the adapter. The shared tables are in the general response.

Q1. Per-reader effects and reader-clustered / hierarchical CIs, including seed uncertainty.

We reassigned every retrieval decision to the reader whose trial was the query and redid the inference with readers as the unit (20,000 bootstrap resamples), plus a two-level reader × seed bootstrap.

ZuCo (30 readers)ChineseEEG (10 readers)
unseen − seen, reader-level mean−0.0669−0.0613
reader × seed 95% CI[−0.0858, −0.0485][−0.0672, −0.0544]
readers with unseen > seen1/300/10
Wilcoxon signed-rankp=2.9e-6p=0.0020
margin lost, reader × seed CI95% [80%, 109%]95% [88%, 102%]
zero-shot vs chance (t-test over readers)p=0.29p=0.059
TOST, within ±25% of the seen marginp=1.4e-4p=2.9e-6

The submission's 94% (ZuCo) and 96% (ChineseEEG) were pool-weighted. Weighting readers equally gives 95% on both. With non-overlapping pools (stride 24) the figures are 94% [78%, 109%] and 95% [87%, 102%]. The revision reports the reader × seed intervals as the main ones and adds per-reader plots for every arm in the appendix. You were right that the pool-level CIs understated uncertainty. The reader-level intervals are wider, and the conclusion holds under them.

Q2a. Are calibration, model selection and test pools stimulus-disjoint?

Yes for the data, and we checked it directly. Sentence IDs are disjoint across train / val / test on both corpora (ZuCo 1,167 / 180 / 182, ChineseEEG 7,982 / 998 / 998; zero overlap in every pair). A held-out reader's calibration trials are drawn only from that reader's training-split sentences, and scoring uses only test-split pools. Calibration runs use a fixed 20 epochs, so validation trials play no role in stopping or checkpoint choice.

One point needs disclosure. The adapter learning rate was chosen from {1e-3, 3e-3} on ZuCo by comparing test-pool scores on folds 0–2. At N=400 the adapter scored 0.1031 at 1e-3 and 0.1094 at 3e-3 on those folds. Full fine-tuning scores 0.0907 over all five folds, so the adapter stays ahead at either learning rate. The revision states this selection step in full. All ChineseEEG runs reuse the ZuCo value (3e-3) without further tuning, so the second corpus works as an untuned replication.

Q2b. Train the adapter on one novel and test on the other.

We ran this for the 9 ChineseEEG readers who read both novels (30 runs, N=400, three seeds):

test novelzero-shotcalibrated on same novelcalibrated on other novel
GarnettDream0.04430.0827 (+0.0383, 9/9, p=0.0039)0.0425 (−0.0018, p=0.50)
LittlePrince0.04080.0819 (+0.0412, 9/9, p=0.0039)0.0380 (−0.0028, p=0.50)

The adapter does not transfer across sessions for the same reader. Your question separates the two readings, and the data support the second: the adapter learns a mapping specific to the session (or the session's stimulus), not a stable reader calibration. This also bears on your Weakness 2. Because the calibration has to be repeated per session, its practical value as a calibrate-once BCI step is weaker than the submission implied. The revision says so directly (Sec. 6.2, Sec. 7, Sec. 9) and presents the adapter as a diagnostic: it shows that the missing signal is recoverable by a linear per-session map in front of a frozen encoder.

Two caveats from the submission still hold, and the revision states them alongside this result (Sec. 6, "ocular" paragraph; Appendix E, Table 11). The reader effect survives a 4 Hz high-pass on ZuCo (92% of the margin lost), so it is not mainly the low-frequency frontal component that we attribute to ocular activity. After the filter, the adapter at N=400 still matches the filtered seen-reader score (0.0936 vs 0.0960, n.s.). That result comes from joint calibration as in the submission, and the revision labels it so.

On the 400-label cost: on ZuCo the calibration curve is at 0.0588 (1.41× chance) with N=25 and recovers about half the margin by N=100 (0.0780). Those budgets were run with joint calibration. The independent runs cover N=400 only, where the adapter recovers 87% on ZuCo and 40% on ChineseEEG. The revision reports the curve with that caveat (Fig. 1(b), Table 3 and Table 5 captions, Sec. 8).

AuthorsOpenReview ↗

Response to Reviewer Shur(Q3)

Q3. What does "reader identity" include?

We mean it operationally: any signal that a model trained on other readers cannot use for a new reader. That includes recording and behavioural nuisance factors (head geometry, electrode placement, impedance, session state, ocular behaviour) as well as any physiological signature. Our experiments do not separate these, and the cross-session result shows that at least the recoverable part is session-bound. The revision:

  • replaces "reader identity" with "reader- and session-correlated factors" in the title, abstract and claims, and defines the term operationally where it is first used;
  • states that the 95% is the share of the seen-reader margin above chance that is lost when the reader is unseen, and not an information decomposition into reader versus language;
  • lists the candidate sources, notes which ones our controls address (data volume, label-free normalisation, 4 Hz high-pass, cross-session transfer) and which remain open (cap placement, impedance, behavioural differences).

Thank you again. Q2 in particular produced the most informative new result in the revision.

Reviewer_LdhkOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2(+1)
Read review
Paper summary

Authors are exploring what accounts for the small above-chance level performance in EEG-to-text retrieval. They correctly cite the relevant literature pointing out that previous works on ZuCO dataset were affected by errors and then explore what account for the left-over retrieval performance. This is an interesting question but on data that are not really informative. Their hypothesis is that what could explain the performance is within-subject training and validation set, so they retrain models keeping some subjects held-out and show that performance drop to chance level, re-finding the well known issue of poor cross-subject generalization of EEG decoding models. Their solution is a channel re-mapping or subject recalibration on new subject that recover most of the lost performance, with an approach similar to Defossez et al, just done after training the model.

Critical strengths

The analysis is well conducted, the question is well posed and the answer is clear. Unfortunately I feel the findings are not really novel.

Critical weaknesses

I have two main concerns about this work, one major and one minor.

The major one is the task itself has little to say. Authors correctly cite the two main relevant works that show that previous EEG-to-text works (like DeWave and follow-ups published at NeurIPS) are mostly driven not by the signal, but to artifacts or improper training (such as teacher forcing -accounting for language priors and previous high metric and duration mismatch). They do cite the work of Jo et al and Zhang et al that proved this points very well and elegantly for the ZuCo dataset. Therefore, there is little to no signal in the ZuCo dataset and to me, this further analysis is only partially interesting. The big question of whether decoding text from EEG is feasible probably requires a completely new dataset.

Given that, the authors start from the point that even solving the duration match issue some retrival task is doable above chance level and therefore investigate what accounts for it. This may be interesting and their main finding is that - one more time, the answer is not in any language related representation in the EEG data, while in the readers identity, and by training the model and testing it on an held-out set, performance is at chance level, then proceeding to suggest a re-mapping of channels (similar to Defossez et al) with a frozen encoder backbone recover the little retrieval performance. Finding the root case of this left-over above chance level performance might be interesting, but honestly I think adds little to the fact that is known that this EEG/text dataset are mostly noise at the moment.

The second minor reason is that the paper doesn't read well at the moment. Authors correctly disclose the use of generative AI for code and writing of certain sections, but the tone and writing style of Claude here is extremely recognizable. It's not a problem per se, it's within the rules, but I would encourage the authors to rewrite (or if they prefer, to use a different prompt for LLMs) most the paper. Right now it really seem to read the flow of reasoning and summarizing the results (including the style of figures) that you can get from a Claude session. Some details are redudant and buries the imporant things in a sea of little things that worked or didn't. The prose feels very dense and rethorical (for example - "we lost a sweep to this" - why is this information in the main text of the paper?)

Questions for authors

I don't really have specific questions, the main claims are clear from the text. Please see weaknesses

[EDIT] After rebuttal, authors addressed some of my comments with proper experiments. While I'm still not convinced by the novelty and clarity of the paper, the work done improved it, so I edited some of my scores to give authors better chances if strongly supported by other reviewers.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Ldhk

Thank you for the review. You say the analysis is well conducted and the answer is clear, and your objections are to how much the answer adds and to the writing. We address both. Some of the new results (see the general response) weaken our own claims, and we report them as they came out.

1. "ZuCo is mostly noise, so this analysis adds little."

We agree that Jo et al. and Zhang et al. showed most published ZuCo EEG-to-text results come from teacher forcing, language priors and duration leakage. Our paper starts after those corrections. What it adds is a specific measurement of what is left:

  • With duration-matched pools, a model trained and tested on the same readers retrieves at 2.6× chance (0.1122 reader-level). This residual is real, and in our experience it is the number a new method reports as progress.
  • That residual is almost entirely absent for readers the model has not seen: 95% of the above-chance margin is lost (reader × seed CI [80%, 109%]), and the zero-shot arm is statistically equivalent to chance (TOST p=1.4e-4).
  • So a sentence-held-out split overstates generalisation even after the known leaks are fixed. This is a protocol problem that the earlier audits did not cover. Zhang et al. fix the duration shortcut but keep every reader in training.

Your point that the field needs a new dataset is well taken, and it is why the second corpus matters. ChineseEEG is a newer recording with about 10,900 trials per reader, against about 745 in ZuCo, so its results do not depend on ZuCo's preprocessing or noise floor. It reproduces the result: 95% of the margin lost ([88%, 102%]), TOST p=2.9e-6. The pattern holds on a dataset much larger per reader than ZuCo. Any future dataset can be tested with the same protocol, and the revised paper commits to releasing the split definitions and pool indices for that purpose (Reproducibility statement).

2. "Poor cross-subject generalisation is well known."

We agree that it is known in general, and the paper says so (Sec. 2). For this task, nobody had reported its size or established that it accounts for nearly all of the post-correction signal. The new cross-session experiment adds a result that is not known. For 9 ChineseEEG readers who read both novels, an adapter fit on one novel lifts retrieval on that novel (+0.038 and +0.041, 9/9 readers each). It has no effect on the other novel by the same reader (−0.002 and −0.003, p=0.50). The usable signal is specific to the recording session, not just to the reader. This means per-reader calibration as practised in BCI (calibrate once, reuse) does not work for this task, and it moves the paper's claim toward the acquisition-factor reading you and the other reviewers raised.

3. Relationship to Défossez et al.

You are right that the adapter is architecturally the same as their subject layer, and the revision now says that first, not as a qualifier. We no longer present the module as a contribution (Sec. 1, contributions; Sec. 2). What we keep is the evidence about where the per-reader parameters have to go, which their joint-training setting does not test:

  • A shared head with no per-reader parameters is the weakest arm at every N.
  • The same adapter on a random frozen encoder stays near chance (1.19–1.25× at the largest budgets, against 2.4–2.6× with the trained encoder), so the trained encoder does most of the decoding.
  • An adapter transplanted to another reader loses the entire gain on both corpora.
  • With independent single-reader calibration, the adapter beats full fine-tuning on ChineseEEG (+0.0107, reader × seed CI [+0.0044, +0.0170], 9/10 readers). On ZuCo the gap is +0.0095 (21/30 readers, Wilcoxon p=0.022), but the two-level CI includes zero, so we now call it consistent, not conclusive.

4. Writing and tone.

We accept this criticism. The revision makes these edits:

  • Process narration such as "we lost a sweep to this" is removed (Sec. 5).
  • The architecture-variant table moves to Appendix C, so the space goes to the new results that bear on the main claim: independent calibration (Table 2) and the cross-session test (Sec. 6.2, Table 6).
  • Figure labels are larger (Figs. 1 and 3), and the Fig. 1(b) parameter label is fixed to 16,384.

We hope the clarified scope (a protocol finding on two corpora, plus a session-specificity result that is new) and these edits address your concerns. We would value your view on whether the revised framing changes your assessment.

Reviewer_LdhkOpenReview ↗

Response to Rebuttal

Thank you for the detailed response and additional analyses. The replication on a second corpus and the calibration controls strengthen the empirical findings. I have updated my methodological soundness assessment accordingly.

I still have reservations about the novelty and interpretation of the results. In particular, the cross-session experiment also changes the novel, so it does not isolate session specificity. More generally, limited cross-reader generalization does not establish whether the recoverable information reflects task-related signals or acquisition confounds.

Nevertheless, I appreciate the clarified scope, additional evidence, and presentation improvements, and appreciated the hard work put into this work. I'm raising my overall score to 2, still heading towards rejection but less strongly so if other reviewers support you, your acceptance odds are better.

Thank you for re-reading the paper and for updating your assessment.

You are right about the cross-session experiment. Changing the session also changes the novel, so the result shows that the calibration does not transfer across session-and-stimulus, not that it is specific to the session alone. We will state it that way in Sec. 6.2 and the conclusion, and list stimulus specificity as an alternative alongside session effects in the Limitations.

We also agree that the reader-holdout result does not separate task-related signal from acquisition confounds. The revision describes the lost signal operationally and lists cap placement, impedance and behavioural differences as open sources (Sec. 8). Separating them would need recordings designed for it, such as repeated sessions of the same text, and we note this as future work.

Reviewer_n5ssOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates reader dependence in EEG-to-text retrieval. The authors evaluate 30 ZuCo readers using five disjoint reader-holdout folds and duration-matched candidate pools. They report that Top-1 accuracy drops from 0.1090 to 0.0458 when readers are held out, corresponding to a 94% loss of the above-chance performance margin. The paper then introduces a per-reader spatial adapter on a frozen EEG encoder to recover performance with limited labelled data. The method is further evaluated across different encoder architectures, ChineseEEG, and several control experiments. The investigation of reader dependence and low-cost calibration has practical value, and the paper presents extensive experiments with positive results. However, substantial concerns remain regarding its scientific novelty, interpretation of the main findings, and the validity of some experimental comparisons. These issues significantly weaken the scientific contribution and the evidence supporting the paper's main conclusions.

Critical strengths
  1. Systematic reader-holdout evaluation. The paper evaluates all 30 ZuCo readers using five disjoint folds and identical duration-matched candidate pools (Sec. 3–4, Table 1). The data-volume control in Sec. 4 is also useful for distinguishing the reader-holdout effect from simply reducing training data.

  2. Extensive calibration experiments. The paper evaluates multiple calibration budgets and compares the adapter against full fine-tuning (Table 2). The four controls in Table 3 also provide useful evidence regarding learning rate, shared parameters, random encoders, and adapter transplantation.

  3. Evaluation across different settings. The experiments cover three encoder architectures (Table 4) and ChineseEEG (Sec. 6.1, Table 5). The authors also investigate trial segmentation (Appendix E.2), ocular-related confounds (Appendix D), and electrode placement (Sec. 7, Appendix F). Reporting these limitations and negative findings is a positive aspect of the study.

Critical weaknesses
  1. Unclear scientific novelty and insufficient discussion of direct prior work. The distinction from existing EEG-to-text cross-subject research is not sufficiently established. Yin et al. (EMNLP 2025, Rethinking Cross-Subject Data Splitting for Brain-to-Text Decoding) already investigated subject-disjoint and text-disjoint evaluation on ZuCo. More directly, McGuire and Moshfeghi (SIGIR 2025, Towards Brain Passage Retrieval) studied EEG-to-passage retrieval with LOSO evaluation across 30 participants. Neither work is discussed, while the paper makes broad claims that previous evaluations keep all readers in training (Introduction, lines 43–46; Sec. 2, lines 119–120). Although the settings differ, the additional scientific insight beyond existing cross-subject findings needs to be more clearly established. The duration-only retrieval claim attributed to Zhang et al. (2026) in the Introduction (lines 39–40) also needs a clearer source.

  2. Insufficient evidence for the main reader-identity interpretation. The reported 94% measures the loss of above-chance retrieval performance when readers are held out. However, this does not directly establish that the performance comes from reader identity rather than language. Similar degradation could occur if linguistic representations differ across subjects. The failure of three label-free normalization methods (Sec. 4, Appendix A) does not distinguish these explanations. The 4 Hz high-pass control also cannot fully exclude ocular or other physiological confounds (Appendix D). The observed reader dependence is meaningful, but the stronger identity-versus-language interpretation requires more direct evidence.

  3. Concerns about the calibration protocol and full fine-tuning comparison. The paper claims that jointly calibrating six readers is exactly equivalent to independent calibration (Sec. 5, lines 208–210). However, the submitted implementation jointly optimizes multiple readers under a contrastive objective, so the optimization is not strictly independent. Moreover, the adapter has reader-specific parameters, while full fine-tuning updates a shared encoder. Thus, the current comparison does not establish superiority over independent per-reader full fine-tuning, or sufficiently isolate why the adapter performs better. The controls in Table 3 do not fully resolve this issue. The adapter advantage also becomes statistically non-significant for sentence-level ChineseEEG (Appendix E.2, Table 11).

  4. Limitations in statistical inference and evaluation controls. The submitted code constructs overlapping duration-matched candidate pools, but the bootstrap treats pool-level results as independent after averaging training seeds. This may underestimate dependencies between repeated trials, readers, and training runs, affecting the reported confidence intervals. Also, a non-significant difference from the seen-reader reference at N = 400 does not establish equivalence or a true performance ceiling (Table 2). Further clarification is needed regarding duration matching, text-identity separation, and repeated-text negatives in the contrastive objective. The borderline GLIM results (Appendix C, Table 8) and segmentation-dependent ChineseEEG results (Appendix E.2, Table 11) further limit the generality of the conclusions.

The figures would benefit from further polishing. Some labels and annotations are small and crowded, particularly in Figures 2 and 3. Figure 1(b) also appears to label the full-rank adapter curve as using 8,192 parameters, while Table 2 reports 16,384. Improving the readability and consistency of these figures would help.

Questions for authors
  1. Could the authors clarify the scientific contribution beyond Yin et al. (EMNLP 2025) and McGuire and Moshfeghi (SIGIR 2025)? Why were these directly relevant studies not discussed? In particular, what new scientific insight is established beyond existing cross-subject retrieval results? Could the authors also clarify the duration-only retrieval claim attributed to Zhang et al. (2026)?

  2. What evidence supports interpreting the 94% performance loss as reader identity rather than subject-dependent linguistic representations? Which experiments distinguish these explanations?

  3. Why is joint calibration considered exactly equivalent to independent per-reader calibration? Could the authors provide a matched comparison between independent per-reader adapters and independent per-reader full fine-tuning? Does the advantage remain?

  4. Could the authors re-evaluate the main confidence intervals while accounting for overlapping candidate pools and reader/seed variability? Could they also clarify text-identity separation, repeated-text negatives, and duration matching? How would these checks affect the main results and the claimed recovery of the seen-reader ceiling?

3 comments
AuthorsOpenReview ↗

Response to Reviewer n5ss (Q1&Q2)

Thank you for the detailed review. Q3 and Q4 identified real problems, and the new experiments change two of our claims. The shared numbers are in the general response; here we answer each question.

Q1. Novelty beyond Yin et al. (EMNLP 2025) and McGuire & Moshfeghi (SIGIR 2025); the Zhang et al. claim.

We should have discussed both papers. Our statement that no EEG-to-text result reports unseen-subject numbers is wrong, and the revision removes it (Sec. 1, Sec. 2). Yin et al. show that subject-disjoint and text-disjoint splits on ZuCo change generation metrics. McGuire & Moshfeghi report LOSO EEG-to-passage retrieval on 30 participants. What we add beyond them:

  1. The measurement is made under a protocol that controls trial duration. As far as we can tell, neither prior work matches candidates on trial duration. On ZuCo, predicting each sentence from its mean reading duration across other readers, with no EEG, gives 2.39× chance in random 24-way pools and 1.07× in our duration-matched pools. Only after removing that shortcut can one ask how much of the remaining signal transfers across readers. Our answer is that almost none does: 95% of the margin is lost, reader × seed CI [80%, 109%], and the zero-shot arm is TOST-equivalent to chance.
  2. Quantification with reader-level uncertainty on two corpora. That cross-subject transfer is poor is expected. That it is essentially zero under a duration-controlled protocol, in two languages and with three architectures, is a specific and testable finding.
  3. The calibration analysis. We measure how many labels an unseen reader must supply, where the parameters should go, and (new) that the calibration does not transfer across sessions.

The Zhang et al. (ACL 2026) citation refers to their analysis showing that non-duration-matched candidate pools can be solved without the EEG. The sentence in our Introduction overstated this as "duration alone retrieves the right sentence". The revision cites their finding as they state it (Sec. 1, Sec. 2) and reports our own duration-only baseline separately in Sec. 2 (2.39× chance unmatched, 1.07× matched, ZuCo test set).

Q2. Why "reader identity" rather than subject-dependent linguistic representations?

We agree that our experiments do not separate these, and the revision no longer claims that they do. A held-out-reader drop shows that the model relies on information that does not transfer across readers. That information may be neural (including reader-specific linguistic representations), anatomical, related to electrode placement or impedance, session-specific, or ocular. The new cross-session experiment narrows this down. For the same reader, an adapter fit on one novel improves that novel (+0.038 and +0.041, 9/9 readers) and does nothing for the other novel (−0.002 and −0.003, p=0.50). A stable reader-level representation, linguistic or physiological, should transfer within a reader across sessions. It does not, so at least the part the adapter recovers is tied to the session or stimulus. The revision renames the quantity "reader- and session-correlated" signal (Sec. 4; title changed to "does not transfer to unseen readers"), defines it operationally, and lists the candidate sources as open (Sec. 8). The cross-session result is in Sec. 6.2, Table 6.

AuthorsOpenReview ↗

Response to Reviewer n5ss (Q3&Q4)

Q3. Joint vs. independent calibration; matched independent comparison against full fine-tuning.

You are right, and the claim in Sec. 5 is wrong. The encoder is frozen, but InfoNCE uses the other readers' trials in the batch as negatives, and the temperature is shared across readers, so the six readers are coupled. We ran the matched comparison you asked for. Each held-out reader is calibrated in its own run, on the same 400 trials it received in the joint run, for both arms, with 3 seeds. Test trials of reader i are scored by the run that calibrated reader i.

N=400, independentZuCo (30 readers)ChineseEEG (10 readers)
adapter0.1032 [0.0887, 0.1186]0.0693 [0.0653, 0.0730]
full fine-tuning0.0937 [0.0774, 0.1110]0.0587 [0.0550, 0.0621]
difference, reader × seed CI+0.0095 [−0.0037, +0.0224]+0.0107 [+0.0044, +0.0170]
readers improved; Wilcoxon21/30; p=0.0229/10; p=0.0039
same, stride-24 pools+0.0128 [−0.0028, +0.0276]; 23/30; p=0.013+0.0109 [+0.0041, +0.0185]; 9/10; p=0.0039

The direction holds on both corpora. On ChineseEEG the advantage is clear under every test. On ZuCo the per-reader tests agree with it, but the two-level CI includes zero, so the revision describes it as consistent rather than established (Sec. 6.1, Sec. 9). Joint calibration helped the adapter (+0.0094 on ZuCo, p=0.0055; +0.0077 on ChineseEEG, 10/10 readers) and did not help full fine-tuning (+0.0021, p=0.60; −0.0005, p=0.85). Part of the advantage reported in the submission therefore came from coupling between readers, and the revision reports the independent numbers as the headline (Table 2, Sec. 6).

On why the adapter does better: the comparison varies where the parameters go (16K reader-specific input parameters vs. 1.0M shared ones) at a fixed label budget. Table 3 rules out the learning rate, extra capacity on the head, and a random encoder. We do not claim to have isolated the mechanism beyond this. The sentence-level ChineseEEG result (Table 13, n.s.) stays in the paper as a limitation.

Q4. Inference unit, overlapping pools, equivalence, and protocol details.

  • CIs. See the table in the general response. Readers are the unit, with a two-level reader × seed bootstrap and non-overlapping pools (stride 24). The main conclusions hold. Under these tests the zero-shot arm is equivalent to chance (TOST, ±25% of the seen margin: p=1.4e-4 on ZuCo, p=2.9e-6 on ChineseEEG), so we no longer rely on a non-significant difference.
  • Ceiling. You are right that non-significance does not establish equivalence, and with the independent runs the adapter is significantly below the seen reference on ZuCo (−0.0090, Wilcoxon p=0.033). We withdraw "reaches the ceiling" and report recovery instead: 87% of the lost margin for the adapter and 72% for full fine-tuning (ZuCo), 40% and 23% (ChineseEEG).
  • Text-identity separation. Train, validation and test splits are disjoint at the sentence level on both corpora (0 shared text IDs across splits, verified on the caches). Calibration trials are drawn only from training-split sentences, so calibration and test are stimulus-disjoint. Calibration runs use no validation-based model selection. The validation set belongs to other readers, so we train for a fixed 20 epochs and keep the final weights.
  • Repeated-text negatives. Within a pool, each sentence appears at most once. In training, the standard one-hot InfoNCE does not mask duplicates. A batch of 64 on ZuCo contains on average 2.1 same-sentence pairs (from different readers), which are treated as negatives. This false-negative noise affects every arm equally. The revision states it in Sec. 3 and adds a duplicate-masked control in Appendix G.
  • Duration matching. Within each task (ZuCo) or novel (ChineseEEG), trials are sorted by duration and pools are formed from consecutive windows of 24 distinct sentences. Duration is otherwise visible only through the padding mask.

Figures. Fig. 1(b) mislabels the full-rank adapter as 8,192 parameters. It has 16,384 (C×C, C=128), and 8,192 is the rank-32 variant. The revision fixes this and enlarges the labels in Figs. 1 and 3.

AuthorsOpenReview ↗

A direct test of what duration control adds to a cross-subject evaluation (Q1)

To make our answer to Q1 concrete, we ran one further analysis. It involves no training: we re-scored our existing reader-holdout models (5 folds × 3 seeds per corpus) on the held-out readers' test trials, once with random 24-way candidate pools and once with our duration-matched pools. CIs are bootstrapped over readers.

CorpusPoolsDuration only (no EEG)Unseen readerSeen reader
ZuCorandom2.41×1.24× [1.14, 1.35]2.80×
ZuCoduration-matched1.20×1.09× [0.93, 1.25]2.70×
ChineseEEGrandom18.3×3.31× [3.25, 3.37]5.49×
ChineseEEGduration-matched3.26×1.07× [1.01, 1.14]2.54×

(values are top-1 as multiples of chance, 1/24)

With random pools, the zero-shot model on ChineseEEG reaches 3.3× chance, all 10 held-out readers are above chance, and about half of the seen-reader margin appears to transfer. When the same model is scored on duration-matched pools, it drops to 1.07×. On ZuCo the inflation is smaller but still significant (+0.0064 top-1, CI [+0.0017, +0.0112]). The seen-reader reference barely changes on ZuCo (2.80× vs 2.70×). So the inflation affects the cross-reader number specifically.

We do not claim this applies to McGuire & Moshfeghi's results: their task (passage retrieval) and their pools are different from ours. The claim is narrower. Under a protocol that does not control duration, cross-reader retrieval can look like substantial transfer when it is mostly a duration shortcut. Our contribution is to measure transfer after removing that shortcut, and the answer is close to zero. We will add this table to Appendix H in the final version.

We hope this clarifies the novelty question. If it addresses your concern, we would be grateful if you would consider revising your score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank all three reviewers. The reviews agree that the reader-held-out protocol and the controls are sound, and they raise four shared concerns: (a) novelty relative to prior cross-subject work, (b) "reader identity" is stronger than what we measured, (c) pool-level bootstrap CIs may be too narrow, and (d) joint calibration of six readers is not equivalent to independent calibration. We ran new experiments for (c) and (d) and for Shur's cross-session question. Some of the new results weaken claims in the submission. We report them as they came out and revise the paper to match.

New experiments (all three seeds, five folds, all readers):

  1. Reader-level inference. Every retrieval decision is assigned to the reader whose trial was the query. We bootstrap over readers (20,000 resamples) and run a two-level reader × seed bootstrap. We also repeat everything with non-overlapping pools (stride 24).
  2. Independent single-reader calibration. Each held-out reader is calibrated in its own run (one reader, 400 trials, no other readers in the batch), for both the adapter and full fine-tuning. That is 180 runs on ZuCo and 60 on ChineseEEG.
  3. Cross-session calibration (ChineseEEG). The adapter is fit on 400 trials from one novel only and scored separately on each novel. That is 30 runs.

Findings and how they change the paper

ZuCo (30 readers)ChineseEEG (10 readers)
seen-reader reference0.1122 [0.0983, 0.1265]0.1059 [0.1008, 0.1105]
unseen reader, zero-shot0.0453 [0.0389, 0.0519], p=0.29 vs chance0.0447 [0.0421, 0.0473], p=0.059
margin lost, reader × seed CI95% [80%, 109%]95% [88%, 102%]
TOST: zero-shot within ±25% of seen margin of chancep=1.4e-4p=2.9e-6
independent adapter, N=4000.1032 [0.0887, 0.1186]0.0693 [0.0653, 0.0730]
independent full fine-tuning, N=4000.0937 [0.0774, 0.1110]0.0587 [0.0550, 0.0621]
adapter − full FT (reader × seed CI; readers improved; Wilcoxon)+0.0095 [−0.0037, +0.0224]; 21/30; p=0.022+0.0107 [+0.0044, +0.0170]; 9/10; p=0.0039

(Brackets are reader-clustered 95% CIs unless marked. Chance is 0.0417. Reader-level means weight readers equally, so they differ slightly from the pool-level means in the submission: 0.1090 / 0.0458.)

  • The collapse holds up under reader-level inference. With readers as the unit, the zero-shot arm is statistically equivalent to chance on both corpora (TOST), and the 95% lost-margin figure has a reader × seed CI of [80%, 109%] on ZuCo. With non-overlapping pools the numbers barely move: 94% [78%, 109%] on ZuCo and 95% [87%, 102%] on ChineseEEG.
  • Joint calibration was not equivalent to independent calibration (n5ss is right). On ZuCo, the independent adapter scores 0.0094 below the jointly calibrated one (higher for only 9/30 readers, Wilcoxon p=0.0055); on ChineseEEG it is 0.0077 lower (0/10 higher). We withdraw the "exactly as six separate runs" sentence, and the independent runs become the headline numbers.
  • The adapter still beats full fine-tuning when both are independent, but by less. On ChineseEEG the gap is clear: 9/10 readers, and the reader × seed CI excludes zero. On ZuCo the advantage is +0.0095 (21/30 readers, Wilcoxon p=0.022), but the two-level CI includes zero. We now describe the ZuCo comparison as "consistent with, not conclusive".
  • We withdraw "reaches the seen-reader ceiling". The independent ZuCo adapter recovers 87% of the lost margin and full fine-tuning recovers 72%. The adapter remains 0.0090 below the reference (Wilcoxon p=0.033). On ChineseEEG it recovers 40%.
  • The calibration is specific to the session (Shur Q2). For the 9 ChineseEEG readers who read both novels, an adapter fit on one novel lifts retrieval on that novel (+0.038 and +0.041 over zero-shot, 9/9 readers each). It has no effect on the other novel (−0.002 and −0.003, p=0.50). What the adapter learns is therefore a mapping specific to the recording session or stimulus, not a stable reader signature. This is direct evidence for the reviewers' point (b), and the revision states it in the abstract, Sec. 6.2 and the conclusion.
AuthorsOpenReview ↗

Where each change is in the revised PDF

Only substantive additions and corrections are in blue; the rest of the text is as submitted. Section, table and appendix numbers refer to the revised PDF.

changelocationraised by
Title: "reader identity" → "does not transfer to unseen readers"; "recovers it" → "recovers most of it"Title, Abstractn5ss Q2, Shur W3/Q3
Lost signal defined operationally as reader- and session-correlatedSec. 4, last paragraphn5ss Q2, Shur W3/Q3
Yin et al. and McGuire & Moshfeghi discussed; "no prior work" claim removed; duration-only baseline (2.39× unmatched, 1.07× matched)Sec. 2, first paragraphn5ss Q1
Adapter stated to be architecturally the Défossez et al. subject layerSec. 1 (contributions), Sec. 2Ldhk (novelty)
Reader-level inference added under the submitted numbers: reader-clustered CIs, Wilcoxon, TOSTTable 1n5ss Q4, Shur Q1
Fixed 20 calibration epochs, no early stopping; learning-rate selection disclosed (0.1031 vs 0.1094 on folds 0–2)Sec. 3; Sec. 8Shur Q2
Duplicate same-sentence negatives stated; masked controlSec. 3; Appendix Gn5ss Q4
Joint vs independent calibration: coupling explained, "exactly as six separate runs" withdrawnSec. 5 "Joint and independent calibration"n5ss Q3
Independent single-reader calibration, N=400, both corpora (headline numbers)Table 2 (new)n5ss Q3
"Reaches the seen-reader ceiling" withdrawn; recovery reported as 87% / 40%Abstract, Sec. 1, Table 3 caption, Sec. 9n5ss Q4
Joint curves labelled as upper boundsFig. 1, Table 3 and Table 5 captions, Appendix Cn5ss Q3
ZuCo adapter vs full FT described as "consistent, not conclusive"Sec. 6.1, Sec. 9n5ss Q3
Cross-session calibration (9 readers, two novels)Sec. 6.2, Table 6 (new)Shur Q2
Calibration must be repeated per sessionSec. 6.2, Sec. 7, Sec. 9Shur W2/Q2
4 Hz high-pass caveat (does not exclude ocular activity above 4 Hz)Abstract; Sec. 6 "ocular" paragraph; Appendix EShur Q2
Non-overlapping pools (stride 24)Appendix H, Table 14 (new)n5ss Q4, Shur Q1
Three-architecture table moved to the appendix (text unchanged) to make roomAppendix C, Table 9—
Limitations: joint-only curves, session-bound calibration, open nuisance sources, LR selectionSec. 8 (four items added)all
Split definitions and pool indices releasedReproducibility statementLdhk
Fig. 1(b) label fixed to 16,384; larger fonts in Figs. 1 and 3Fig. 1, Fig. 3n5ss W3, Ldhk (writing)

A version of the PDF without colour is also uploaded as supplementary material.

Summary of what changed in the claims

  • Title and claims: "reader identity" is replaced by a statement of what was measured (no transfer to unseen readers), and the lost signal is defined operationally as reader- and session-correlated: any signal that a model trained on other readers cannot use. New title: Whose Brain Is It? EEG-to-Text Retrieval Does Not Transfer to Unseen Readers, and a Per-Reader Adapter Recovers Most of It.
  • Withdrawn: "exactly as six separate runs", "reaches the seen-reader ceiling", "no prior work reports unseen-subject numbers", and calibrate-once use of the adapter.
  • Per-reader effects are reported as reader counts and Wilcoxon tests in Tables 1, 2 and 6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.