acceptodds
Under review as a conference paper at ICLR 2027

Who Did What to Whom? Relational Structure in EEG Semantic Decoding

Abstract

Recovering structured meaning from neural activity is a central goal of brain-to-language decoding and a key step toward natural communication through brain-computer interfaces. Recent semantic-unit decoding frameworks provide an interpretable route to recovering semantic content from EEG, yet content alone does not specify its relational organization. This leaves a fundamental boundary between recovering what is represented and identifying who did what to whom. To study this boundary, we develop a three-stage evaluation framework that separates role-neutral semantic content from relational organization. (1) For functional relevance, text-side content-controlled structural interventions show that correct predicate-argument binding improves fidelity to the intended proposition beyond semantic content alone. (2) For structural readout, contextual representations support reproducible role-related readout beyond retrieved semantic-unit identity, while larger relation-predictability effects are also reproduced by non-neural controls, limiting their interpretation as EEG-specific structure. (3) For occurrence-specific attribution, EEG conditioning yields small gains beyond semantic content, but matched-record specificity remains weak and heterogeneous. Together, these results identify relational structure as a distinct target for EEG semantic decoding and provide a principled basis for separating functional relevance, structural readout, and occurrence-specific neural attribution.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG semantic decoding, neural language decoding, relational structure, predicate-argument structure, semantic role decoding, EEG-to-text

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VHLNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether EEG semantic decoding can recover not only semantic content but also predicate–argument structure, i.e., “who did what to whom.” The authors distinguish role-neutral semantic units from relational organization and propose a three-stage evaluation framework covering functional relevance, structural readout, and occurrence-specific neural attribution. Experiments on ChineseEEG-2 and Chisco show that explicit binding improves proposition-level reconstruction, contextual EEG-derived representations support modest role-related readout, but stronger relation-predictability effects are also reproduced by non-neural controls, and matched-record specificity is generally weak.

Critical strengths
  1. The paper identifies a meaningful limitation of semantic-unit decoding. Recovering semantic concepts alone does not determine predicate–argument organization, so the paper targets a real representational gap in EEG semantic decoding.

  2. The three-stage evaluation framework is conceptually clear. Separating functional relevance, structural readout, and neural attribution helps avoid conflating probe decodability with EEG-specific evidence.

  3. The control analyses are careful and informative. Retrieved, gold-lexical, fixed-memory, and mismatched-EEG controls help reveal that some apparent relational predictability can arise from non-neural or decoder-related priors.

Critical weaknesses
  1. The paper diagnoses the relational limitation of current EEG semantic decoding but does not propose a method that actually solves it. The work stops at analysis and probing rather than developing a decoder that explicitly predicts relational structure from EEG. Gold predicate–argument binding clearly improves reconstruction, but this only shows that relational structure is useful, not that it can be reliably decoded from EEG.

  2. The paper lacks a direct end-to-end comparison with SID on relational reconstruction. It does not show whether explicitly modeling relational structure from EEG improves final sentence or proposition recovery over the original semantic-unit decoding pipeline.

  3. The evaluation is limited to Chinese-language EEG datasets. Both ChineseEEG-2 and Chisco use Chinese linguistic stimuli, so it remains unclear whether the observed relational readout and strong lexical priors generalize across languages; widely used English EEG-language resources such as ZuCo are not evaluated.

Questions for authors
  1. How does the proposed framework compare directly with SID on relation-sensitive reconstruction metrics, such as predicate, ARG0, ARG1, or predicate–argument exact match?

  2. Given that matched-record specificity is weak in most settings, what evidence supports the claim that the observed structural readout is genuinely derived from the current EEG occurrence rather than lexical or decoder priors?

  3. How well do the reported effects generalize beyond within-subject settings and the relatively small number of subjects used here?

Reviewer_5o4uOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates whether EEG semantic decoding recovers predicate–argument relations beyond semantic-unit identity, addressing “who did what to whom.” It proposes a three-stage evaluation framework separating the functional relevance of relational structure, structural readout from EEG-derived representations, and dependence on the correctly matched neural record. Experiments on ChineseEEG-2 and Chisco combine text-side structural interventions, representation probes, and content-only, matched-EEG, and mismatched-EEG controls. Correct binding improves text-side proposition fidelity, and contextual representations provide modest role-readout gains over retrieved semantic units. However, additional predictive gains from EEG are small, and matched-record specificity is weak and heterogeneous. The main contribution is an evaluation and interpretation framework for relational decoding rather than a new high-performing EEG-to-text architecture.

Critical strengths

1.The paper proposes an important question and distinguishes frequently conflated levels of evidence. Predicting semantic roles does not establish that the information comes from the current EEG record. Separating functional relevance, representational readout, and matched-record specificity helps prevent language priors or model biases from being mistaken for successful neural decoding.

2.The study includes substantial controls and generally interprets weak effects cautiously. The analyses consider lexical features, fixed memory, label permutations, mismatched EEG, and an alternative encoder, and report findings that do not support general neural attribution. This analysis is useful for understanding the limitations of EEG-to-text evaluation.

Critical weaknesses

1.The findings remain dependent on the chosen decoding pipelines, with insufficient calibration connecting their performance to the recoverability of information from EEG. The alternative-encoder experiments provide some robustness evidence, but retain the same typed-query decoder, training objectives, and main evaluation procedures. They therefore do not fully address sensitivity to shared modeling choices. In particular, weak or near-zero matched-record specificity could reflect limited information in the data, limitations of representation learning or readout, or insufficient assay sensitivity.

I appreciate that the authors do not interpret this result as demonstrating the absence of relational information in EEG. Nevertheless, appropriately cautious wording does not replace validation of the assay’s detection capability. The paper should more clearly establish basic semantic-content decoding performance on both datasets and calibrate the stricter relational assays using directly supervised relation-decoding baselines or positive controls with known relational signals. Without such evidence, the findings primarily characterize the tested pipelines rather than support broader scientific conclusions about EEG decoding.

2.The measurement validity and connection between the three stages require stronger evidence, and the presentation makes these issues difficult to assess. The first stage is a text-side experiment: it establishes the relevance of binding to proposition expression but does not validate the sensitivity of the subsequent neural assays. The second-stage AP differences measure readout gains relative to retrieved representations without directly identifying the information source. In the third stage, M1 and M2 are trained separately, while some effects are only on the order of 10⁻⁴ to 10⁻³, making it necessary to distinguish correspondence effects from optimization variability.

Moreover, the mismatched-EEG controls preserve different nuisance variables across ChineseEEG-2 and Chisco, limiting comparability. Appendix B also states that the structural targets are not independently human-annotated gold labels; independent human validation and error analysis would strengthen the conclusions. The main text densely presents contrasts, endpoints, and confidence intervals without sufficiently emphasizing what each measurement can and cannot establish. Figure 1 is crowded, and important protocol details are distributed across the appendices. The notation should also distinguish target-side U from predicted semantic units, probabilistic model notation from implemented scoring functions, and dataset-specific permutation procedures more explicitly. The concern is measurement justification and reporting clarity, rather than the value of confidence intervals themselves.

Questions for authors

Clarification and key evidence: What is the basic semantic-content decoding performance of the representation-learning pipelines on both datasets, relative to mismatched-EEG or EEG-free controls? Are directly supervised EEG relation-decoding results available to distinguish failure of the learned representation to retain relational information from difficulty decoding it from the available data?

1.Is there a positive control with a known relational signal demonstrating that the second and third stages can detect it? Such calibration should address neural readout and matched-record specificity, rather than only whether a text generator follows supplied role assignments.

2.Given separate training of M1 and M2, how stable is matched-record specificity across paired initializations, repeated training runs, and alternative donor assignments? Could matched-versus-mismatched input evaluation using a fixed model complement the existing retrained controls?

3.Is there an independently human-validated subset of structural labels, with agreement statistics? Does the Chisco ARG0 specificity gain persist when its donor control also matches stimulus source and recording duration?

Reviewer_kdmyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work attempts to establish whether role-filler bindings can be decoded from EEG-derived embeddings. The work first provides a simple sanity check establishing that role information is important for the semantics of a sentence. Next, the work establishes that EEG-derived vectors provide some benefit to predicting role information, over and above the word identities themselves. Finally, they establish that an irrelevant EEG-derived vector provides the same benefit, calling into question the results of their second study.

Critical strengths

The core question is interesting and important. Filler-role binding is a core aspect of symbolic linguistic processing, and decoding how the brain encodes these aspects of language would represent a large step forward in understanding biological language processing.

Critical weaknesses
  1. The paper is extremely difficult to understand. In all experimental sections, the precise design is unclear, and appears spread out over several sections that are laden with unnecessary jargon. For example, it appears that 3.2 amounts to an appendix-worthy sanity check that quantifies whether role information is relevant to reconstructing the semantic content of sentences, however it is not immediately obvious how this procedure is implemented. This is, of course, true, and is the central premise motivating investigations of role-filler bindings in the first place. Establishing this is not a novel contribution.
  2. The core novel finding, in section 3.3 (EEG-derived vectors increase role decodability), is immediately undermined by the result in section 3.4 (ANY EEG-derived vector increases role decodability).
Questions for authors

What exactly is the process for performing the sanity check in section 3.2? How is the role-conditioned reconstruction performed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.