acceptodds
Withdrawn from ICLR 2027

Predictive Information Attribution with Regret Accounting in Sequential Neural Decoding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural decoding performance can reflect task structure and behavioral history as well as the current neural observation. We ask how much the observation improves prediction beyond the information available to the decoder. We compare a context predictor with a joint predictor using their log score difference on held-out events. The expected gain satisfies , relating conditional information to the difference in predictive regret between the fitted models. Controlled experiments recover three regimes with known ground truth. Persistence produces strong marginal decoding with zero conditional gain. Noisy XOR yields 0.531 conditional bits per sample despite zero marginal information. An underfit context model yields 0.363 fitted bits per sample under a conditional null; capacity matching removes the gain. In auditory attention EEG from 18 participants, supervised previous-trial label feedback reduces gain relative to a prevalence baseline, while a positive versus negative lag contrast remains under interleaved and contiguous splits. For spectral decoding of motor imagery EEG from 40 unseen participants, cue history reduces gain over schedule context by 0.0098 bits per trial. These results show how context and predictor capacity change the interpretation of neural decoding performance.

Primary area:
applications to neuroscience & cognitive science
Keywords:
information theory, multimodal learning, predictive modeling, sequential learning, neural decoding, computational neuroscience

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.60 over 5 reviews, above 10% of the others.
Reviewer_Er1FOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This study evaluates how much information can be read from EEG signals by using a sequential decoder. Specifically, the authors trained the predictors with and without observed neural data to evaluate the information gain.

Critical strengths

The idea is sound, and some of the results may interest the researchers who analyze EEGs.

Critical weaknesses
  • The authors did not clearly discuss how their study differs from the earlier neural decoding studies. Some of the earlier studies are listed, but they do not explain how and why their study is relevant or connected. The entire section reads like a random list of early neural decoding studies without careful consideration. Instead of merely listing earlier studies, the authors should clearly state the significance of their own study in relation to other studies.

  • More importantly, some of the details are missing, which are crucial to understand their study. A few questions come into mind: Which predictors have the authors trained? Are they linear models or neural networks? How did they preprocess neural signals?

  • The main idea is simple and based on well-established method, but it is difficult to read the manuscript due to the lack of clear explanations and undefined jargons.

  • The results of their study appear reasonable but also generic, and the authors do not clearly state what could be learned from their analysis.

-The lack of clarity and proper explanations suggest that this manuscript needs to be revised. Also, due to the narrow scope of their study, it may be more suitable for more focused venue.

Questions for authors

Please explain the terms so that the readers can understand the context of your study. For instance, the authors should clarify the meaning of “folder map” in line 141, “innovation” and “redundancy” in line 190 and “R04/08/12” in line 265. These are just a few examples out of many that should have been explained.

1 comment

Thank you for the review.

Relation to earlier studies

The related-work section is revised so that each paragraph answers one question and ends with what our analysis adds:

  1. How is predictive information measured? (conditional V\mathcal V-information, Granger causality and transfer entropy, information decomposition)
  2. How do decoders use behavioral dynamics, and how can data splits and confounds inflate decoding? (Li et al., 2021; Yan et al., 2025; Rotaru et al., 2024; Xu et al., 2026)
  3. How do recent studies attribute decoding performance? This paragraph includes a direct comparison with Du et al. (2026a).

Predictors and preprocessing

Section 5 names each model, and Appendix D lists every model and search grid in Table D.1, "Model and selection specifications".

DTU (auditory attention, 18 participants).

  • Preprocessing: the released preprocessed 64-channel EEG, averaged to 32 Hz.
  • EEG window: the decoder uses EEG 125–375 ms after the speech. A control uses EEG 125–375 ms before the speech.
  • Models: three decoders reconstruct the attended speech envelope:
    • linear ridge regression (577 parameters);
    • EEGNet-lite (1,465 parameters);
    • a causal temporal convolutional network (8,449 parameters).

EEGMMI (motor imagery, 40 participants).

  • Features: log band power from 8 to 30 Hz over motor-cortex channels. Each participant's features are normalized by their own unlabeled run 4.
  • Model: an ℓ2\ell_2-regularized logistic regression, fit with the context prediction as a fixed offset.
  • Additional analyses: EEGNet variants, with and without calibration.

Context models. Logistic regressions on trial-history features, and a smoothed lookup table over the cue protocol.

Terms

All terms are now defined at first use. The ones you list mean:

  • Fold map: which outer fold's model scores each trial.
  • Innovation: the part of the target that its own past cannot predict (ϵt\epsilon_t in Section 4); γ\gamma sets how strongly the neural observation measures it.
  • Redundancy: information about the target that the observation shares with the context.
  • R04, R08, R12: EEGMMI recording runs 4, 8 and 12.

What can be learned

Accuracy can miss what the log score measures. In auditory-attention EEG with contiguous folds, the previous label alone gives 0.945 balanced accuracy. Adding EEG raises it only to 0.948, yet the log-score gain is 0.041 [0.025, 0.057] bits per trial.

How much EEG adds depends on what the decoder already knows.

  • Auditory attention: knowing the previous label lowers the lag contrast in gain from 0.179 to 0.042 bits per trial.
  • Motor imagery: cue history absorbs 59% of the neural gain in EEGMMI. In a dataset with random cue order, it absorbs 1.6%.
Reviewer_WAgAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies an information decomposition with the goal of quantifying information about a target (Y) contained in a neural signal (X) above and beyond information in a context (C). The primary audience of this paper is computational neuroscientists, but the information decomposition that is introduced used may be of broader interest.

Critical strengths

A point in favor of this work is that it combines information decomposition with a notion of usable information in a restricted family (as originally introduced by Xu et al 2020); the key identity in equation 3 relates conditional Shannon information to three other quantities (GG, R0R_0, and R1R_1) that depend only on the limited predictive families q0q_0 and q1q_1. As far as I can tell, this is a novel way of performing an information decomposition (which is a somewhat crowded space; see refs below) while taking into account a limited predictive family.

Critical weaknesses

I think the work here is interesting, but I do not think this paper is ready for publication. The primary issue is a lack of comparison to existing methods and lack of discussion of prior work. For instance, this paper introduces an information decomposition to account for past history but does not discuss the various other ways that neuroscientists have thought about alternative information decompositions or alternative ways of regressing out trial history ("serial dependencies") to isolate different components of information. What specifically is novel or interesting about this work compared to previous approaches to information decomposition is unclear. There are many existing proposals for information decomposition out there; I sampled just a few in the references below, none of which appear to be cited or discussed here. Not all of these are directly relevant, but I hope the authors will do their own careful literature review and add a section comparing and contrasting their method to existing information decompositions, especially those that have been developed for a neuroscience audience.

  • Throughout, terminology relating to Partial Information Decomposition (1) is used (e.g. "synergy" of information) but the origin of these terms is not cited, as far as I could tell.
  • Ref (2) below discuss "intersection information" (II) to quantify the information in a neural population that is used to drive behavior. A point in favor of the present work is that (2) admits a weakness of II is that it does not account for context/history.
  • Ref (3) below is a good source for information theory in neuroscience, and includes a discussion of partial information decompositions
  • Ref (4) below discusses "unique" information, yet another information decomposition.

My second critique is related; the present study looks at decoding performance on a variety of existing neural datasets (DTU, EEGMMI). While this is an effective exercise in illustrating the mechanics of how the present information decomposition can be used to compute various bits of information, I was left wondering "so what?". There seems to be an unstated implication that prior attempts at looking at these datasets made some methodological or interpretive error, and that the present work corrects those past mistakes. If that's the case, this paper would be far stronger if the authors could show that past interpretations of the same data were missing something. Note: I am not simply saying that this is a benchmarking problem and that the authors need to "outperform" others on some benchmark. I am saying that the authors need to interpret the results and show us that their information decomposition tells us somehting about these data that was not previously known using other methods.

My third substantical critique is that the paper seems to imply that I[Y;X]I[Y;X] can contain spurious information, and that accounting for CC as in I[Y;X∣C]I[Y;X|C] can better "attribute" information to XX. This interpretation is not stated outright, but terms like "attribute" certainly suggest it, and it is not outright discussed or denied. The issue is one of correlation and causation. If one is interested, say, in the causal information in XX that drives behavior YY, then it is unclear whether accounting for CC is the right thing to do. After all, it could be the case that past trial history (CC) is represented in the brain and measured by XX, and this information has a causal role in determining YY. All this to say: "controlling for" context seems like a good idea at first, but what one should or should not control for in a given analysis depends on various causal assumptions (see, e.g. Pearl's "Book of Why"). The authors might not agree with my particular example here, which is fine. But I think there ought to be some discussion of how any measure of 'information' cannot answer causal questions, and therefore the application of these new information-decomposition tools might or might not be appropriate controls depending on the modeler's assumptions, the experimental setup, the particulars of X,Y, and C, etc.

Minor comments:

  • formulae and numbers should be removed from the abstract; there is no context to make sense of them.
  • Lines 43- are redundant with lines 106-
  • many acronyms are used without definitions ("DTU", "EEGMMI")
  • many terms are used before or without definitions ("calibration", "G", "fold map", the "L" and "H" subscripts in (8), "marginal correlation" and "conditional gain" in Fig 2, "R04" R08" and "R12", ...)
  • The numeric bracket notation wasn't defined. Are the confidence intervals? Credible intervals? What percentile? How computed?
  • In section 6.2, why account for just previous trial labels but not previous behaviors? It's common in the analysis of serial dependencies to consider the tuple (past label, past choice) or (past reward, past choice) to capture behaviors like 'win-stay/lose-shift'.
Questions for authors

Sorry for improper format; my questions are sprinkled throughout the above comments.

2 comments

Thank you for a constructive review and for the literature pointers.

1. Relation to information decompositions and serial dependence

Section 2 now covers these topics. It cites the origin of the redundancy and synergy terminology (Williams & Beer, 2010) and relates our analysis to your references (1)–(4).

Partial information decomposition. In a PID that satisfies the standard consistency equations, I(Y;X∣C)I(Y;X\mid C) equals the unique information of XX plus the synergistic information of XX and CC about YY. Two of our controlled cases are textbook examples:

  • in the target-persistence case, XX contributes only redundant information, so the conditional information is zero;
  • noisy XOR is pure synergy.

What our analysis adds. PID, intersection information and unique information decompose the information in the true distribution. Our analysis concerns the next step: how far the scores of fitted predictors on held-out data depart from those quantities.

Serial dependence. Our context models are serial-dependence regressions on trial history. The gain measures what the neural observation adds beyond such a regression.

2. "So what?"

Accuracy can miss what the log score measures (DTU). With the previous label as context and contiguous folds, adding EEG raises balanced accuracy only from 0.945 to 0.948. The log-score gain is 0.041 [0.025, 0.057] bits per trial. Its lag contrast is 0.032 [0.019, 0.046], positive in 16 of 18 participants.

Context absorbs neural gain to the extent that it predicts the target.

  • EEGMMI: cue history predicts the cue with 0.761 balanced accuracy and absorbs 59% of the spectral gain (0.0166 → 0.0068 bits per trial).
  • BCI Competition IV-2a, where cue order is random: cue history reaches 0.559 and absorbs 1.6% (0.0980 → 0.0964).

DTU history.

  • Adding the previous label lowers the lag contrast from 0.179 to 0.042 bits per trial. This is consistent with history carrying much of the predictive information, since the attended speaker changes only 1–3 times in 60 trials.
  • The positive-lag marginalized gain that remains is 0.011–0.031 bits per trial across model families.

EEGMMI neural model. Beyond cue history, the spectral model finds no reliable evidence. An exploratory EEGNet finds a positive marginalized gain of 0.077 [0.047, 0.106] (10 seeds). Because the marginalized gain cannot exceed conditional information in expectation, this indicates EEG evidence beyond cue history that the spectral model does not recover, given replacement strata that here omit trial position.

In the submission (Appendix F.6; 3 seeds; driven mainly by one participant), two EEGNet variants have nearly equal accuracy (0.779 and 0.783) but opposite mean gains (−0.162 and +0.082).

3. Correlation and causation

We agree, and Section 7 now includes a paragraph on this.

  • All of these quantities are predictive.
  • Conditioning on CC answers a decoding question: what does the current observation add for a decoder that already has CC?
  • Your example marks where this differs from a causal question. If history is represented in the brain, measured by XX, and drives YY, then conditioning on CC removes that pathway. This is appropriate for the decoding question; causal claims about XX require causal assumptions.
  • In both EEG datasets, the target is an instructed cue set by the protocol. These data therefore answer the decoding question.

Minor comments

  • Abstract and redundancy. The abstract now contains no formulas or numbers. The introduction keeps only a brief statement of the comparison and the identity; the formal definitions and the estimator appear once, in Section 3.
  • Defined at first use:
    • DTU: the Technical University of Denmark auditory-attention dataset.
    • EEGMMI: the PhysioNet EEG Motor Movement/Imagery dataset.
    • R04, R08, R12: recording runs 4, 8 and 12. Run 4 is used for normalization; runs 8 and 12 contain the imagined left- and right-fist trials.
    • Fold map f(i)f(i): the outer fold whose model scores trial ii.
    • L and H: the lower and higher context in a ladder.
    • Also defined: calibration, GG, marginal correlation, and conditional gain.
  • Intervals.
    • DTU and EEGMMI (Section 3.4): 95% percentile intervals from 20,000 bootstrap resamples of participant means, given the cross-fitted predictions.
    • Synthetic: percentiles over simulation replicates.
  • Previous choice. Neither dataset records a choice on each trial: DTU instructs the attended speaker, and EEGMMI instructs the imagined hand. The closest analogue is the decoder's own previous output. The revised PDF adds it as a context, alone and in an HMM filter (Section 6.2, Appendix I), and the lag contrast stays positive in all three model families.

I thank the authors for engaging with all five reviewers and making an effort to address all concerns. However, the edits did not fully address my concerns, and I will also reinforce some concerns raised by other reviewers.

First, I pointed out connections to PID and other reviewers pointed out connections to more recent relevant work that hadn't been cited in the original submission. The updated related work section now mentions these connections, which is good, but it reinforces (to quote Reviewer G898) that the "main offering [of this paper] is a new evaluation protocol, but the methodology is not new." My questions about serial dependence were only partially addressed; I do not see a reply from the authors to address the fact that context includes stimulus history but not behavior history (but I may have misunderstood this point in the original paper).

Second, I find the response to my concerns about correlation-vs-causation to be weak. Here is a direct quote from Pearl & Mackenzie's "Book of Why" (page 139):

"Oddly, statisticians both over- and underrate the importance of adjusting for possible confounders. They overrate it in the sense that they often control for many more variables than they need to and even for variables that they should not control for. I recently came across a quote from a politcal blogger named Ezra Klein who expresses this phenomenon of "overcontrolling" very clearly: "You see it all the time in studies. 'We controlled for...' and then the list starts. The longer the better. Income. Age. Race. Religion. Height. Hair color. Sexual preference. Crossfit attendance. Love of parents. Coke or Pepsi. The more things you can control for, the stronger your study is --- or, at least, the stronger your study seems. Controls give the feeling of specificity, of precision... But sometimes, you can control for too much. Sometimes you end up controlling for the thing you're trying to measure."

I'd like to contrast this idea of "overcontrolling" with the reply by the authors of the present manuscript under review:

If history represented in X drives Y, conditioning on C removes that pathway. This is appropriate for decoding; causal claims about X require causal assumptions (Pearl, 2009; Pearl & Mackenzie, 2018).

Simply stating that controlling is "appropriate for decoding" seems to me to miss the point. Whether or not controlling is appropriate depends on the situation, and while it clearly is a matter of making "causal assumptions," it's simply incorrect to assert that in the absence of causal assumptions, controlling for everything is the right approach. Clearly it depends on why a decoder is being built in the first place (e.g. as a BCI vs as a scientific model of processes going on inside the subjects' head).

The reply to my "so what?" concern (and the concern from other reviewers about modest or difficult-to-interpret gains on real data) also misses the point. My concern wasn't that adding neural data fails to improve the decoder. Sure, X adds a few bits of performance above and beyond context alone. Conversely, one could say that context adds only a few bits of performance above and beyond neural data alone. The issue is that merely demonstrating additional bits doesn't seem to provide fundamental insight on what is going on in those studies, nor does it seem to lead directly to better context-independent decoders. For a reader like me, the authors will need to clearly articulate a more complete answer to the causality question and why controlling for C is the right thing to do in the first place and explain how existing methods fall short of providing the right scientific insights on these data (not just how they differ numerically in performance).

Finally, I'd like to echo concerns raised by other reviewers on difficult (likely AI-generated) phrasing throughout. This seems to have been exacerbated during the rebuttal period; the new text added to the paper has all the same issues (noun stacking, using keywords without defining them, over-emphasis on numeric results without interpretation, etc).

Reviewer_Cxa6OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper examines whether in EEG settings neural data provides additional predictive utility for a decoder model beyond behavior in previous trials and the structure of the task. They contrast a decoder that is trained using only the previous context with a model trained on the previous context and the current trial's neural data. The authors examine synthetic settings, an auditory setting, and a motor imagery setting, and find that in many cases neural observations provide little new information for decoding beyond behavioral and task variables. Therefore, they urge contrasting neural decoders with a context model when examining the utility of neural data.

Critical strengths
  • The paper highlights a potentially important (if straightforward) control for neural decoding studies, where decoder predictivity may be equated with neural predictivity despite the neural data not adding significant information.
  • The paper uses two distinct EEG settings with human participants to establish its conclusions.
Critical weaknesses
  • Unfortunately, I do not think that the contribution of this paper is very novel or significant. It describes a simple statistical control and finds that indeed it detects lack of utility from neural data. However, I do not see this as a very surprising finding given that trial history would likely contain much of the information required to decode task-relevant variables, and the fact that the entire paper revolves around a rather standard control means its results do not offer a sufficient contribution in my opinion.
  • The paper is written in an unnecessarily dense and hard-to-read way, defining many terms regarding information theory to convey a rather simple point. As a result, it is quite difficult to understand the empirical results and the authors' main points. Finally, the paper' was very likely heavily drafted by AI, which, while allowed, made the experience of reading the paper quite difficult and frustrating.
Questions for authors

None.

2 comments

Thank you for the review. We agree that trial history is informative. The revised PDF (changes in blue) adds two points beyond that observation.

1. Accuracy can miss what the log score measures.

  • Auditory attention. With the previous label as context and contiguous folds, adding EEG raises balanced accuracy only from 0.945 to 0.948, while the log-score gain is 0.041 [0.025, 0.057] bits per trial.
  • Synthetic data. An underfit context model lets accuracy rise from 0.500 to 0.793 although the neural signal carries no information beyond the context. A new marginalized gain, bounded above by conditional information, gives −0.077 in that case.

2. Adding history does not mechanically lower neural gain.

  • Motor imagery (EEGMMI). Cue history predicts the cue with 0.761 balanced accuracy and absorbs 59% of the neural gain (0.0166 → 0.0068 bits per trial).
  • Random cue order (BCI Competition IV-2a). Cue history reaches 0.559 and absorbs 1.6% of the neural gain (0.0980 → 0.0964).

Writing. We accept that the submission was hard to read, and the revision addresses this directly:

  • no formulas in the abstract;
  • one term per concept;
  • every acronym and symbol defined at first use;
  • each result stated as question, finding, and implication.

I thank the authors for their response, and for their work to improve the paper. However, as mentioned in my review, I unfortunately do not see the core contribution of this paper as novel or significant, and the authors' responses have not changed my perspective in that regard. Hence, I will unfortunately maintain my score.

Reviewer_dtsYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigate how much a neural observation helps predict a target beyond other contexts. This is achieved by comparing a context-only predictor with a joint predictor that also sees the neural signal, using their paired log-score difference on held-out events (in bits), which they relate to conditional mutual information plus a difference in model regret. They illustrate this decomposition on synthetic systems with known ground truth, then apply it to auditory attention (DTU) and motor imagery (EEGMMI) EEG, showing that previous-trial labels and cue history substantially reduce the gain attributed to EEG.

Critical strengths

Recent studies show that task structure can confound the performance of EEG decoding models, which this submission examines in depth. The evaluations are carefully controlled, and the reporting is honest; e.g., outlier participants are kept.

Critical weaknesses

The proposed method seems fairly standard. The authors measure the held-out log-loss difference between predictors, which is conditional V-information or conditional probing (Xu et al., 2020; Hewitt et al., 2021). And the key equation (3) comes from cross-entropy = entropy + KL. It is unclear to me what is new beyond applying the standard attribution analysis to sequential neural decoding.

The regret decomposition can only be computed on synthetic data, where the true distribution is known. On EEG, the authors acknowledge (in Section 7) that the information and regret terms cannot be separated. Moreover, the main finding, that the gain drops when the context becomes more predictive, is somewhat expected from the definitions even for a fully valid neural signal, and it does not necessarily indicate that the decoder was exploiting a shortcut. So it still does not fully resolve the attribution problem for the domain of interest in real data.

Questions for authors

Could the authors report the key differences between the proposed analysis and recent work, e.g., Du et al. (2026a), in more depth?

A small note on presentation: the abstract introduces the key equation without explaining the notation at all (e.g., R_0 and R_1). The same is the case for the introduction.

1 comment

Thank you for the review. Your second point identifies the main gap in the submission, so we start there.

Separating information from model error on real data

The new marginalized gain addresses part of this problem. It removes the direct effect of the context model, and under stated sampling conditions it is a lower bound on conditional information. It does not identify I(Y;X∣C)I(Y;X\mid C) exactly.

Construction. Average the joint model's predicted probabilities over replacement observations that share the context. This gives a context-only predictor qˉ1\bar q_1, derived from the joint model. Then

G=(R0−Rˉ1)+Gmarg,Gmarg=I(Y;X∣C)−(R1−Rˉ1)≤I(Y;X∣C).G = (R_0 - \bar R_1) + G_{\rm marg}, \qquad G_{\rm marg} = I(Y;X\mid C) - (R_1 - \bar R_1) \le I(Y;X\mid C).

The inequality follows from joint convexity of the KL divergence (proof sketch in General Response, part 2; full proofs in Appendix H).

What it guarantees:

  • The context model enters only the first term, so an underfit context model cannot produce a positive GmargG_{\rm marg}.
  • With replacements drawn from P(X∣C)P(X\mid C), a clearly positive GmargG_{\rm marg} indicates conditional information.

What it leaves: joint-model error, which still enters through R1−Rˉ1R_1 - \bar R_1.

On real data, replacements come from the same participant with matched context features, so the bound holds approximately.

  • The match is closest in DTU, where the context is discrete.
  • It is weaker in EEGMMI, where the match omits trial position.
  • One DTU configuration violates the sampling conditions, and we report it as a diagnostic (General Response, part 2).

Results:

  • DTU (previous-label context, contiguous split, 5 seeds): the positive-lag marginalized gain is 0.031 [0.018, 0.043] bits per trial for ridge and 0.012 [0.004, 0.019] for EEGNet.
  • Synthetic conditional null: an underfit context model gives a fitted gain of 0.363 bits, while the marginalized gain is −0.077.

"The gain drop is expected from the definitions"

Adding context can lower conditional information, and it can also raise it: in our XOR case it rises from 0 to 0.531 bits. The context-ladder identity (Eq. 9) writes the change as an interaction-information term plus a change in model error. The size of the change is therefore an empirical result.

We agree that a drop alone does not show the decoder used a shortcut. It also does not by itself measure redundancy, because model error changes too. The revision therefore reports the drop directly:

  • the DTU lag contrast falls from 0.179 to 0.042 bits per trial;
  • cue history absorbs 59% of the EEGMMI gain (0.0166 → 0.0068).

It interprets these only as consistent with history carrying much of the predictive information.

The drop is not automatic. In BCI Competition IV-2a, where cue order is random, cue history predicts the cue with 0.559 balanced accuracy and absorbs 1.6% of the neural gain (0.0980 → 0.0964).

What is new, and Du et al. (2026a)

The score and the identity are standard. The introduction now states three contributions:

  • the failure mode of an underfit context model under correct splits, shown with ground truth;
  • the marginalized-gain bound and its real-data approximation;
  • the dataset analyses.

Du et al. (2026a) audit intracortical macaque decoding of continuous velocity using squared-error risk. We share three elements with their work: the declared context, the context ladder, and trial replacement. Section 2 credits them for the protocol. Our work differs in four ways:

  • it uses log scores in bits on human EEG classification;
  • it states the model dependence as an exact identity;
  • it adds synthetic cases with known information;
  • it adds the marginalized-gain bound.

A full comparison table is in our response to Reviewer G898 (Q3).

Notation

The abstract now contains no formulas. Before stating the identity, the introduction defines R0R_0 and R1R_1 in words: each is a fitted model's excess expected log loss over the best predictor that uses the same inputs.

Reviewer_G898OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks how much additional information the current neural signal provides beyond what the decoder already knows at prediction time (e.g., the previous trial's label). The authors compare a context-only predictor with a context-plus-neural predictor by the difference of their log scores on the same held-out events (NEGain, in bits per trial), and show that the expected gain equals the conditional mutual information plus the difference between the two models' predictive regrets, G = I(Y; X | C) + R0 − R1. The approach is evaluated in two parts: three synthetic settings with known ground truth (redundancy, XOR synergy, and an underfit context model under a conditional null), and two EEG datasets (DTU auditory attention and EEGMMI motor imagery), with a descriptive ECoG analysis in the appendix.

Critical strengths
  1. The question has practical value for the BCI community. in sequential decoding, when much of the answer can be inferred from non-neural information (e.g., the previous trial's label in the DTU dataset), how much does the current neural signal still contribute? This matters because accuracy-based evaluation can credit the neural signal with information that task structure alone already provides.
  2. The experimental design is careful and the set of controls is comprehensive: temporal controls (negative vs. positive lags in DTU, pre- vs. post-cue windows in EEGMMI), trial swap/replacement, capacity matching, probability calibration, contiguous splits, multiple model families, and multiple seeds.
  3. The paper clearly articulates a failure mode in which a weak baseline manufactures spurious neural gain, and it provides operational tests for it. Prior critiques of EEG/AAD evaluation (Li et al., 2021; Xu et al., 2026; Rotaru et al., 2024; Yan et al., 2025) mainly concern leakage and confounds: improper data splits or experimental designs that let models score through trial fingerprints, slow drift, or eye movements. This paper identifies a different case: even with correct splits and no leakage, if the context model lacks capacity and the neural signal is correlated with the structure it misses, the joint model can use the neural signal to recover that structure and obtain gain without any genuine neural information (0.363 bits/sample under a true conditional null in the synthetic experiment).

References: Li, R., et al. The perils and pitfalls of block design for EEG classification experiments. IEEE TPAMI, 2021. Yan, Y., et al. Overestimated performance of auditory attention decoding caused by experimental design in EEG recordings. Interspeech, 2025.

Critical weaknesses
  1. Limited novelty. The core motivation is a known problem: that blocked designs and temporal autocorrelation inflate EEG/AAD decoding results has been documented by Li et al. (2021, "The perils and pitfalls of block design for EEG classification experiments", TPAMI), Xu et al. (2026), Rotaru et al. (2024), and Yan et al. (2025, "Overestimated performance of auditory attention decoding caused by experimental design in EEG recordings", Interspeech). The paper cites Xu et al. and Rotaru et al. but does not position its contribution against this literature, and it does not cite Li et al. or Yan et al. Its main offering is a new evaluation protocol, but the methodology is not new either. The paired log-score difference is essentially a conditional log-likelihood comparison between nested models, the same class of quantity as conditional V-information and conditional probing, which the paper itself cites. Quantifying how much a signal predicts beyond the target's own history is the idea behind Granger causality (Granger, 1969) and transfer entropy (Schreiber, 2000), which the paper does not mention. The regret identity follows directly from the decomposition cross-entropy = entropy + KL divergence. Because the contribution is primarily methodological, this weighs heavily in my assessment.
  2. Presentation. Some writing problems slow down reading: heavy noun stacking (e.g., "supervised previous-trial label feedback"), inconsistent terminology for the same concept ("information set" vs. "information boundary"; "conditional replacement gap" vs. "swap gap"), and key terms left undefined in the abstract (e.g., "prevalence baseline", "positive versus negative lag contrast"). Unifying the terminology and defining key terms where they first appear would make the paper much easier to follow.
  3. Weak empirical support for the EEGMMI claims. The effects are small and their intervals sit close to zero: the headline EEGMMI result (the reduction in neural gain when cue history is added to the schedule context) is 0.0098 [0.0009, 0.0178] bits/trial, and the residual gain with cue history as context is 0.0068 [−0.0017, 0.0162].

References: Granger, C. W. J. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 1969. Li, R., et al. The perils and pitfalls of block design for EEG classification experiments. IEEE TPAMI, 2021. Schreiber, T. Measuring information transfer. Physical Review Letters, 2000. Yan, Y., et al. Overestimated performance of auditory attention decoding caused by experimental design in EEG recordings. Interspeech, 2025.

Questions for authors
  1. The title refers to sequential neural decoding, but the evidence consists of two EEG datasets whose label structure is created by the experimental design, plus a descriptive ECoG analysis of three participants in the appendix. Have the authors considered settings where behavioral dynamics matter most, such as intracortical BCIs?
  2. The paper does not show what the new metric reveals beyond a simpler approach. The motivating example itself uses balanced accuracy: cue history alone reaches 0.761, the joint model 0.763, and EEG alone 0.565 (Introduction and Section 6.3). Reporting a context-only baseline together with a proper data split would therefore reveal the same conclusion. Is there a real case in which conventional practice leads to a wrong conclusion that NEGain corrects?
  3. How does this work differ from Du et al. (2026a), which the paper cites? The framework appears very similar: a declared context, a context ladder, trial replacement as a pairing control, explicitly model-relative estimates, and even the term "information boundary". From my reading, the main differences are the log-score scale, the explicit regret decomposition, the synthetic redundancy/synergy/underfitting regimes, and the application to EEG classification, but the paper does not state the relationship directly.

Reference: Du, Z., et al. (2026). A context-conditional audit of trial-pairing-dependent neural gain in motor-cortex decoding. bioRxiv preprint, posted August 10, 2026.

2 comments

Thank you for the detailed review. Your strengths section separates our failure mode from split leakage. The revised introduction states that distinction as the first contribution, and the related-work section places it next to the block-design literature.

Q2. What does the log score show beyond a simpler approach?

You are right about the EEGMMI spectral result: a context-only accuracy baseline gives the same qualitative conclusion. Cue history alone reaches 0.761, and the joint model reaches 0.763.

The log score adds three things.

  1. Sensitivity where accuracy saturates. In DTU with contiguous folds, the previous label alone reaches 0.945 balanced accuracy, and the joint model reaches 0.948. Yet the log-score gain from EEG is 0.041 [0.025, 0.057] bits per trial. Its lag contrast is 0.032 [0.019, 0.046], positive in 16 of 18 participants (0.042 [0.024, 0.058] with the original interleaved folds).
  2. A common scale across contexts and datasets. Cue history absorbs 59% of the neural gain in EEGMMI (0.0166 → 0.0068 bits per trial). In BCI Competition IV-2a, where cue order is random, it absorbs 1.6% (0.0980 → 0.0964).
  3. A check on the context model. In the synthetic conditional null, accuracy (0.500 → 0.793) and fitted gain (0.363 bits) both credit the neural signal, although it has no information beyond the context. The marginalized gain is −0.077 [−0.090, −0.063]. The rise in accuracy is real; the error lies in reading it as information carried by the neural signal.

The submission contains one further example (Appendix F.6; 3 seeds; driven mainly by one participant). Two EEGNet variants have nearly equal accuracy (0.779 and 0.783) but opposite mean gains (−0.162 and +0.082 bits per trial).

Q1. Intracortical BCIs

We agree that this is the most consequential setting.

  • Related work on intracortical data. Du et al. (2026a) study intracortical macaque data with a related protocol (see Q3).
  • Continuous outcomes. Our estimator also covers continuous kinematics through Gaussian scores, as the ECoG appendix shows.
  • What we did during the rebuttal. We did not add an intracortical dataset. We spent the time on deployment contexts for DTU and on the randomized-cue control.

Q3. Relation to Du et al. (2026a)

The two works share a declared context, a ladder of context models, trial replacement, and model-relative estimates. Section 2 now says this directly and credits Du et al. for the protocol. The text now uses "information set" throughout, in place of "information boundary"; one panel title in Figure 3(a) still carries the old term and will be updated in the final version.

Du et al. (2026a)This work
DataMacaque intracortical arrays (NLB MC_Maze, MC_RTT, LINK); continuous velocityHuman EEG classification (auditory attention, motor imagery); ECoG appendix
ScoreHeld-out squared-error risk and R2R^2Held-out log score in bits
Model dependenceCompared across context rungs and model familiesExact identity G=I+R0−R1G = I + R_0 - R_1, with controlled cases where each term dominates
Synthetic dataCalibration over neural embedding strengthKnown-information regimes: redundancy, synergy, context underfitting
Pairing controlSingle nearest-context replacement; within-condition permutationMarginalized gain: averages over replacements, with an information bound under stated sampling conditions

Novelty

We agree with both points.

  • The score belongs to the family of log-likelihood comparisons between nested models, which includes conditional V\mathcal V-information and conditional probing.
  • The identity follows from cross-entropy = entropy + KL.

The introduction (Section 1) now states three contributions:

  1. The failure mode you identify in your strengths section. It is distinct from split leakage. Even with correct splits and no leakage, an underfit context model can still produce neural gain: 0.363 bits under a true conditional null. Controlled cases with known ground truth show when each term of the identity dominates.
  2. The marginalized gain. It is bounded above by conditional information whatever the joint model, and the context model cannot inflate it. On real data, it is approximated with matched replacements.
  3. The dataset analyses (General Response, part 1).

Related literature

  • Li et al. (2021) and Yan et al. (2025). They show that block designs inflate decoding when training and test data share blocks. Our analysis addresses what remains once splits are correct: the test-time gain can still come from context the evaluation omits, or from an underfit context model.
  • Granger (1969) and Schreiber (2000). Transfer entropy is a conditional mutual information given the target's own past. Our target quantity has the same form, with a declared information set that may contain decoded history. Granger and transfer-entropy estimates depend on the fitted model in the same way; for held-out log scores, the identity states that dependence exactly.

Both topics are now in Section 2.

EEGMMI evidence

The pre-specified spectral results stand as reported.

  • Adding cue history absorbs 0.0098 [0.0009, 0.0178] bits per trial, i.e., 59% of the gain over schedule context (0.0166 → 0.0068).
  • The remaining gain is 0.0068 [−0.0017, 0.0162].

The intervals are close to zero, but the comparison is informative. In BCI Competition IV-2a, where cue order is random, cue history absorbs 1.6% of the neural gain. Its context predicts the cue with 0.559 balanced accuracy, compared with 0.761 in EEGMMI.

New analyses:

  • Spectral model. The post-cue marginalized gain under cue history is 0.006 [−0.003, 0.015].
  • Bounded-input EEGNet. The submission reported it with 3 seeds: 0.082 [0.040, 0.122] bits per trial. With 10 seeds, the gain beyond cue history is 0.088 [0.045, 0.128]. The marginalized gain is 0.077 [0.047, 0.106], and all 10 seed means are positive.

Because the marginalized gain cannot exceed conditional information in expectation, this indicates EEG evidence beyond cue history that the spectral model does not recover. The bound relies on replacement strata, which here omit trial position. The bounded input was chosen after diagnosing one participant's amplitude drift, so the spectral analysis remains the primary result.

Presentation

The revised PDF:

  • removes the noun stacks you point out;
  • defines the prevalence baseline (Section 5) and the lag contrast (Sections 1 and 3.4) where they first appear;
  • uses one term per concept, including "information set" and "replacement contrast".

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response (1/2): main findings and changes in the revised PDF

We thank all five reviewers. We agree that the log-score gain and the identity G=I(Y;X∣C)+R0−R1G = I(Y;X\mid C) + R_0 - R_1 are standard, and that the submission did not state its contribution clearly.

Reviewers also raised two further questions:

  • Can information be separated from model error on real data?
  • What does the analysis reveal about the datasets?

This post summarizes the findings and the changes. Part 2 gives the new estimator and the full results. The revised PDF is now uploaded, with changes marked in blue. Section and appendix numbers below refer to it.

Main findings

1. Accuracy can miss what the log score measures (DTU).

  • Context is the previous trial's label (ridge model).
  • The lag contrast is 0.042 [0.024, 0.058] bits per trial with the original interleaved folds, and 0.032 [0.019, 0.046] with contiguous folds (positive in 16 of 18 participants). It is the gain at positive EEG lags minus the gain at negative lags. So the evidence has the expected temporal direction.
  • With contiguous folds, adding EEG raises balanced accuracy only from 0.945 to 0.948, while the log-score gain from EEG is 0.041 [0.025, 0.057] bits per trial.

2. Context absorbs neural gain to the extent that it predicts the target.

  • EEGMMI. Cue history predicts the cue with 0.761 balanced accuracy. It absorbs 59% of the spectral model's gain (0.0166 → 0.0068 bits per trial).
  • BCI Competition IV-2a. Cue order is random. Cue history reaches 0.559 balanced accuracy and absorbs 1.6% of the neural gain (0.0980 → 0.0964).
  • The two datasets also differ in recording and protocol. The comparison shows that adding history does not mechanically lower neural gain.

3. An underfit context model can create neural gain, and the marginalized gain detects it.

  • In a synthetic conditional null, the neural signal carries no information beyond the context.
  • Even so, accuracy rises from 0.500 to 0.793, and the fitted gain is 0.363 bits.
  • The accuracy gain is real; the error lies in reading it as information carried by the neural signal.
  • The new marginalized gain is −0.077 [−0.090, −0.063].

4. Some neural evidence remains beyond history.

  • DTU, previous-label context. The positive-lag marginalized gain is 0.011 to 0.031 bits per trial across three model families. The TCN interval touches zero.
  • DTU, context built from the decoder's own past outputs. No true labels are used. The lag contrast is positive in all families.
  • EEGMMI. Beyond cue history, the spectral model finds no reliable evidence. The exploratory bounded-input EEGNet has a marginalized gain of 0.077 [0.047, 0.106] bits per trial (10 seeds). Because the marginalized gain cannot exceed conditional information in expectation, this indicates EEG evidence beyond cue history that the spectral model does not recover. The bound relies on replacement strata, which here omit trial position.

Changes in the revised PDF

  1. Contribution (Section 1). The introduction states that the score and the identity are standard. The contributions are:
    • (a) a failure mode distinct from split leakage: even with correct splits and no leakage, an underfit context model can still produce neural gain (0.363 bits under a true conditional null);
    • (b) the marginalized-gain bound and its real-data approximation;
    • (c) the dataset analyses above.
  2. Related work (Section 2). Granger causality and transfer entropy; partial, intersection and unique information; serial-dependence regression; block-design studies (Li et al., 2021; Yan et al., 2025); conditional predictive impact; and a direct comparison with Du et al. (2026a).
  3. New analyses. The marginalized gain with its conditions (Section 3.4, Appendix H); decoded-history contexts for DTU (Section 6.2, Appendix I); the IV-2a control (Section 6.3, Appendix J); ten seeds for the bounded-input EEGNet (Appendix F.6). Table 2 marks the new rows in blue.
  4. Causality (Section 7). A paragraph on predictive versus causal interpretation.
  5. Readability.
    • No formulas in the abstract.
    • Acronyms and symbols defined at first use, with one term per concept.
    • Synthetic and participant-bootstrap intervals described separately (Section 3.4).
    • Eq. 2 now written as the participant-level average used in all results.
AuthorsOpenReview ↗

General response (2/2): the marginalized gain and new results

Which analyses were specified before they were run:

  • Specified in advance: the synthetic sweep, the DTU decoded-history and marginalized-gain analyses, and the BCI IV-2a control.
  • Applied to saved predictions: the EEGMMI analyses use the same estimator on predictions already computed.
  • Exploratory: the bounded-input EEGNet, as in the submission.

From the replacement contrast to the marginalized gain

The replacement contrast in the submission (Section 3.4) has no information-theoretic guarantee. The marginalized gain changes one step.

  • The replacement contrast averages log scores over replacement observations X′X' that share the context.
  • The marginalized gain averages the joint model's probabilities over those replacements. This gives a context-only predictor qˉ1(y∣c)\bar q_1(y\mid c). GmargG_{\rm marg} is the log-score gain of q1q_1 over qˉ1\bar q_1.

By Jensen's inequality, GmargG_{\rm marg} is never larger than the replacement contrast. It is bounded above by conditional information.

What holds in theory (replacements drawn from P(X∣C)P(X\mid C)):

  • Exact split. G=(R0−Rˉ1)+GmargG = (R_0 - \bar R_1) + G_{\rm marg}. The context model q0q_0 enters only the first term.
  • Upper bound. Gmarg=I(Y;X∣C)−(R1−Rˉ1)≤I(Y;X∣C)G_{\rm marg} = I(Y;X\mid C) - (R_1 - \bar R_1) \le I(Y;X\mid C) for every joint model.
    • Proof sketch. Adding and subtracting the Bayes log scores gives Gmarg=I(Y;X∣C)−R1+Rˉ1G_{\rm marg} = I(Y;X\mid C) - R_1 + \bar R_1. Both P(y∣c)P(y\mid c) and qˉ1(y∣c)\bar q_1(y\mid c) are averages over the same X′∼P(X∣C)X' \sim P(X\mid C). Joint convexity of the KL divergence therefore gives Rˉ1≤R1\bar R_1 \le R_1.
    • This is the conditional form of the InfoNCE bound (van den Oord et al., 2018; Poole et al., 2019).
  • Null case. With finitely many replacements that include the observed trial, the expected value is at most zero when YY is independent of XX given CC.

What it does not remove. Joint-model error still enters through R1−Rˉ1R_1 - \bar R_1. The marginalized gain is therefore a lower bound on conditional information.

Conditions on real data. Replacements come from the same participant and run.

  • They must capture P(X∣C)P(X\mid C).
    • In DTU, they are matched on test fold and the discrete context value.
    • In EEGMMI, they are matched on the previous cues, but not on trial position, which the cue-history context also uses.
  • They must be independent of the current target given the stratum. This fails for the fold-prior context in contiguous DTU blocks. There, replacements share the block's label, and the marginalized gain is negative in all three model families (−0.10 to −0.21 bits per trial). We report that case only as a diagnostic.

New results

Units are bits per trial.

  • Intervals. DTU and EEGMMI: 95% intervals from 20,000 participant bootstrap resamples. Synthetic: percentiles over 128 simulations.
  • Window. DTU has no cue, so its windows are positive or negative EEG lags. EEGMMI uses post-cue windows.
AnalysisResult
Synthetic conditional null, underfit context modelAccuracy rises from 0.500 to 0.793; fitted gain is 0.363. The marginalized gain is −0.077 [−0.090, −0.063]. Across four signal strengths, its mean never exceeds the true conditional information.
DTU without labels: context from the decoder's own past outputs or an HMM filter of them; contiguous split; 3 families × 5 seedsThe lag contrast is positive in every family: from 0.049 [0.026, 0.071] to 0.127 [0.076, 0.170]. Ridge gain at negative lags is also positive (about 0.03), consistent with a slowly changing block-level state. The contrast isolates the lag-specific part.
DTU, previous-label context, contiguous split, 5 seeds: positive-lag marginalized gainRidge 0.031 [0.018, 0.043]; EEGNet 0.012 [0.004, 0.019]; TCN 0.011 [−0.000, 0.020].
EEGMMI, cue-history context: post-cue marginalized gainSpectral model 0.006 [−0.003, 0.015]. Bounded EEGNet, 10 seeds (3 in the submission): 0.077 [0.047, 0.106].
BCI IV-2a, random cue order, 9 participantsCue history predicts the cue with 0.559 balanced accuracy (0.761 in EEGMMI). It absorbs 1.6% of the neural gain (0.0980 → 0.0964), compared with 59% in EEGMMI (0.0166 → 0.0068).

Full tables: Table H.1 (synthetic), Table I.1 (DTU decoded history), Table J.1 (IV-2a). Code for all new analyses will be added to the supplementary material.

Dear Reviewers,

The authors have provided their rebuttal, but except for Reviewer Cxa6, no follow-up comments have been posted yet. Reviewer dtsY, WAgA, and Er1F, as the author-reviewer discussion period is ending soon, please review and acknowledge the rebuttal, and engage in further discussion if any concerns remain unresolved.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.