acceptodds
Under review as a conference paper at ICLR 2027

Predictability Is Not Provenance: Rethinking Next-Word Prediction in LLM-Based Neural Encoding

Abstract

Large language model (LLM)-based semantic encoding studies often interpret pre-onset performance in target-aligned lag– curves as evidence that the brain predicts upcoming words. We show that this inference has two limitations. First, contextual representations of successive words are statistically dependent, while responses elicited by those words overlap in the measured signal. Conventional target-word encoding can therefore conflate the target contribution with covariance-weighted contributions from historical sources, making predictive success ambiguous with respect to neural provenance. Second, held-out correlation quantifies aggregate predictive adequacy but does not identify which word generated predictable activity or how its contribution evolves from onset. We formalize this ambiguity with a constrained-availability source-time finite impulse response model (CAST-FIR) and derive Target Isolation from it as a source-adjustment procedure. A strictly causal simulation and a passive acoustic benchmark produce prospective-looking performance without anticipatory computation. Across English ECoG and Chinese SEEG, Target Isolation contracts the cross-onset temporal-generalization area by % and %, respectively. Fitted source-relative kernels peak at ms in English and ms in Chinese after source onset. These results do not rule out neural prediction. They show that target-aligned predictability alone does not establish target-word provenance: performance should establish predictive adequacy, whereas temporal interpretation requires explicit source-time structure.

Primary area:
applications to neuroscience & cognitive science
Keywords:
neural encoding, large language models, neuroscience, temporal attribution, brain decoding, ECoG, SEEG, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hqoHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposed a constrained-availability source-time finite impulse response model. Target Isolation is derived from it as a source-adjustment procedure. The simulation presents the performance without anticipatory computation.

Critical strengths

None

Critical weaknesses
  1. experimental issues.
  2. implementation issues.
Questions for authors
  1. The proposed correction is not validated against real signals.
  2. No identifiability analysis is presented.
  3. Implementation details are not given.
  4. No source code.
Reviewer_wbDqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that pre-onset encoding performance in target-aligned LLM curves can come from cross-word covariance of contextual embeddings combined with overlapping neural responses ("temporal source mixing"). It formalizes this with an OLS decomposition, proposes a one-sided source-time FIR (CAST-FIR) whose fitted non-target contributions are subtracted from the response ("Target Isolation"), and reports that this removes nearly all cross-onset temporal generalization in English ECoG and Chinese SEEG.

Critical strengths
  • The question matters for the field: pre-onset encoding (Goldstein et al., 2022) is widely cited as evidence of neural next-word prediction. The authors postulate that the quantity that matters is the cross-covariance rather than pairwise cosine similarity, and show that centered window averaging alone can leak post-onset signal into pre-onset lags (Eq. 9).
  • The authors make their point with a clean synthetic experiment, which shows convincingly that a strictly causal generator reproduces the empirical lag-r shape. The positive control (App. B.6, Fig. 10) usefully shows that raw cross-onset temporal-generalization quadrants look the same with and without true prediction.
  • Adjusting the response rather than the predictor is a principled choice. The representational audit (Table 6) shows that the projection and residualization controls degrade semantic benchmarks in 16 of 18 comparisons, which supports keeping the embedding intact.
  • The empirical validation is extensive: the authors collect a new Chinese SEEG dataset with slowed speech to lower overlap on top of the public English ECoG data, and include a cross-LLM robustness check (Table 7, 6 model families x 2 layers).
Critical weaknesses
  • Target Isolation is biased toward the null, and this is never tested. The CAST-FIR kernel is one-sided by construction (h(a)=0 for a<0; Eq. 5 uses only bins in [0,L)). Suppose the brain really does represent x_i before O_i. The FIR fit cannot assign that activity to x_i, so it will partly absorb it into the history kernels, and subtracting those kernels then removes real predictive signal. The obvious check is to run Target Isolation on the App. B.6 simulation with the predictive signal switched on, and show that the second- and fourth-quadrant blocks survive while the prediction-free blocks disappear. This is missing. B.6 only compares against the oracle "target-only" condition, so the near-total collapse in Fig. 5A cannot yet be told apart from a method that cannot detect prediction at all.

  • The passive and neural results do not fit together. On acoustics, Target Isolation leaves about 40–50% of pre-onset r. On neural data, by contrast, it drives pre-onset r to roughly 0 and removes 98–100% of the thresholded generalization area. If the method cannot fully remove mixing in a simple passive system, near-complete removal in the brain needs explaining.

  • The headline numbers have no statistical support. The 98.1%/100% contraction is the change in area above an arbitrary threshold (r>0.075) with no permutation or null distribution, and no sensitivity analysis over the threshold. Isolation probably lowers SNR across the whole matrix, including post-onset, so a thresholded area can shrink without any change in the shape of generalization.

  • The kernel-peak interpretation is weak. The norm of a contextual-embedding coefficient block peaking at 75 ms (English) is implausibly early for semantic processing. It more likely reflects word-onset or acoustic and phonetic responses that the embedding encodes indirectly.

Questions for authors
  • The authors claim that with target isolation there is not pre-onset stimulation, but in Figure 5A, the target-isolated temporal generalization area seems to begin before the onset in english
  • Does Target Isolation remove the difference between correctly and incorrectly predicted words, or between high- and low-surprisal words, in pre-onset encoding?
3 comments
AuthorsOpenReview ↗

Response to Reviewer wbDq (1/3)

We thank the reviewer for a careful and constructive review. Every point led to new analyses, several of which make the paper more rigorous and complete; we report these analyses below. Figures R1–R11 are available at anonymous repository. The updated supplementary material (version 3) contains the code and numerical results for the new analyses.

W1. One-sided kernel: is Target Isolation biased toward the null?

Kernel recovery, and the temporal-generalization check the reviewer proposed (Table R1; full visualization in Figs. R1–R2). We ran the App. B.6 generator with the predictive signal switched off (R1) and on (R2), at ρ = 0 and .72, with and without response overlap. We fit both the one-sided CAST-FIR and a two-sided version in which each word can also contribute before its own onset. We also use a cleaner and stricter cross-validation scheme, in which no model that produces an isolated response has seen the held-out test data, so that TI is not biased toward zero.

Table R1 summarizes the condition in which both mechanisms are present (ρ = .72 with response overlap; bottom-right panels of R1–R2) as the mean r over the second and fourth quadrants within ±600 ms of onset (the span of the predictive kernel):

RawOne-sided TITwo-sided TI
Without prediction.43.04.04
With prediction.52.23.19

Fig. R1. App. B.6 generator, predictive signal off.

Fig. R2. App. B.6 generator, predictive signal on.

  • Without prediction, the one- and two-sided fits are indistinguishable, and both recover the causal kernel.
  • With prediction, the reviewer's concern is confirmed. When successive embeddings are correlated (ρ = .72), the one-sided model attributes the upcoming word's pre-onset response to the preceding word, which distorts the early causal kernel (R2, top middle). This defeats the purpose of separating each word's contribution in time. The two-sided model recovers both the predictive and the causal kernel (R2, top right).
  • Temporal generalization after TI (Table R1; bottom rows of Figs. R1–R2) answers the reviewer's question. The raw cross-onset generalization is large whether or not the signal contains prediction, so it cannot separate the two cases. After one- or two-sided TI, it is near zero without prediction and clearly positive with prediction.

Systematic comparison of one- and two-sided fits (Figs. R3–R4). To test whether the difference between the two models in R1–R2 holds beyond a single simulated dataset, we added a statistical comparison over many ground truths. We repeated the fits on 3,600 simulated datasets: 600 ground-truth settings (6 fixed kernel shapes × 50 seeds, plus 300 randomly drawn pairs of predictive and causal kernels), each at ρ ∈ {0, .358, .72} and two noise levels. R3 shows 12 of the random kernels.

Fig. R3. Ground truth vs one- and two-sided fits for 12 random kernel pairs (ρ = .72, prediction on).

Fig. R4. Kernel error of the one- vs the two-sided fit on every simulated dataset.

  • With a predictive signal, the two-sided model has the lower kernel error on every dataset (3,600/3,600; Wilcoxon signed-rank p < 10−30010^{-300}). The median error falls from 19.9% to 1.2% of the kernel energy.
  • Without a predictive signal, the two-sided model is slightly worse on every dataset (median error 1.27% vs 1.52%; p < 10−30010^{-300}). Its 24 extra pre-onset bins have no signal to fit and absorb only noise. The cost is about 0.25 percentage points.

We are grateful for this suggestion, which greatly inspired our revision: in this comparison, the two-sided CAST-FIR is clearly the more appropriate choice. We therefore use it instead of the one-sided model as the main model in the remaining analyses (R5–R11), and we will revise the corresponding parts of the paper accordingly.

AuthorsOpenReview ↗

Response to Reviewer wbDq (2/3)

W2. The passive and neural results do not fit together

We apologize for the unclear presentation. The 40–50% for the acoustic benchmark referred to the pre-onset r on the temporal-generalization diagonal, whereas the 98–100% for the neural data referred to the thresholded temporal-generalization area. We now compute both measures for both systems with the same procedure. Table R2 summarizes the temporal-generalization diagonal at pre-onset lags (full curves in Fig. R5), and Fig. R6 shows the full temporal generalization.

Table R2. Reduction of the pre-onset r by TI (one-sided / two-sided):

Mean over pre-onset lags, ChineseMean over pre-onset lags, EnglishAt −25 ms, ChineseAt −25 ms, English
Acoustic benchmark60% / 67%48% / 61%42% / 42%38% / 49%
Neural data44% / 53%49% / 59%30% / 32%27% / 33%

Fig. R5. Pre-onset r on the temporal-generalization diagonal: raw, one-sided TI and two-sided TI, for the acoustic benchmark (top) and the neural data (bottom).

Fig. R6. Full temporal generalization for the acoustic benchmark (left) and the neural data (right).

With matched metrics, the neural data no longer show near-complete removal. In line with the reviewer's intuition, the passive system is cleaned at least as thoroughly as the neural data, and clearly more so just before onset. Both systems keep a pre-onset residual.

W3. No statistical support for the headline numbers

Threshold sensitivity (Table R3; full curves in Fig. R7). We swept the threshold c from .06 to .15 under the stricter cross-validation described in W1. The contraction of the cross-onset area depends on c (Table R3):

Contraction (one- / two-sided)c = .06c = .075c = .09c = .11
English75% / 83%76% / 85%80% / 88%87% / 96%
Chinese69% / 81%93% / 100%100% / 100%100% / 100%

Fig. R7. Contraction of the r > c cross-onset area as a function of the threshold c.

We thank the reviewer for this point, which prompted us to change how the paper reports the reduction of the generalization area: we will replace the thresholded area with these threshold curves and with signed, continuous measures such as those in Table R4.

Does TI do more than subtract a signal of a given size? A semantic sham (Table R4; full visualization in Fig. R8). We kept every part of TI unchanged: kernels, timing, cross-fitting and the temporal-generalization pipeline. Only the subtracted non-target contribution was replaced by one computed from circularly shifted word identities, and it was subtracted at the same amplitude (middle column; average of 4 shifts).

  • This sham has the same size and temporal structure as true TI but the wrong word identities, and it leaves the temporal generalization almost unchanged. Within ±600 ms of onset, the cross-onset mean changes only from +.059 to +.057 in English and stays at +.067 in Chinese (Table R4).
  • Cross-onset generalization across scales (Fig. R8, right). We averaged r over the second and fourth quadrants within |train lag|, |test lag| ≤ L, for every L from the full range down to 25 ms. At every scale, the sham curve is nearly identical to the raw curve, whereas true TI lies clearly below both. The difference from the sham is significant at every L in English (8/8 participants; paired t-test, FDR q < .05) and at every L ≤ 1425 ms in Chinese (5/5 participants). Table R4 reports the mean r within ±600 ms of onset in both languages, as in Table R1.
Cross-onset mean r, ±600 msRawShamTITI < sham
English+.059+.057+.0058/8 (exact p = .0078)
Chinese+.067+.067+.0345/5 (paired t-test p = .003)

Because the sham subtracts a signal of the same size and timing, a mere loss of SNR would lower both curves equally. The reduction under true TI therefore depends on the identity, that is, the semantic content, of the preceding words, and cannot be explained by a lower SNR alone.

Fig. R8. Raw temporal generalization, semantic sham (same amplitude, wrong word identity), and Target Isolation (two-sided CAST-FIR). Right: mean r over the second and fourth quadrants within |train|, |test| ≤ L, as L shrinks from the full range to 25 ms (mean ± SEM across participants); grey background marks L where TI < sham (paired t-test, FDR q < .05).

AuthorsOpenReview ↗

Response to Reviewer wbDq (3/3)

W4. The kernel-peak interpretation is weak

Fig. R9. Real data. Left: coefficient norm of the one- and two-sided CAST-FIR. Right: temporal generalization, raw and after one- and two-sided TI. Rows: Chinese SEEG (200-ms window), Chinese SEEG ([lag, lag + 25) ms window), English ECoG (200-ms window).

We agree. Three points:

  • The 75-ms peak is limited by the measurement window. The neural responses are 200-ms centered window averages, so the value at "75 ms" is the mean over −25 to 175 ms, and any latency read from the kernels is smeared by about ±100 ms. The English data are released only in this windowed form; for the Chinese data, we re-extracted the responses in causal [lag, lag + 25) ms windows (Fig. R9, middle row).
  • The one-sided kernel itself distorts the early peak. As Figs. R1–R4 show, when successive embeddings are correlated, the one-sided CAST-FIR absorbs the pre-onset contributions of upcoming words into its early causal bins, which reshapes the kernel. The 75-ms peak may therefore be partly an artifact of this absorption and is likely too early to be read as a latency of semantic processing.
  • Only the overall shape is robust. The coefficient-norm profile peaks after onset and then falls, in both languages, with both window sizes and for both the one- and two-sided models (Fig. R9, left column).

Given these limitations, and because the coefficient norm also contains ridge noise, we will reduce this part from a quantitative latency claim to a qualitative description and remove the semantic interpretation of the early peak. We also agree that early components may reflect word-onset, acoustic and phonetic information that contextual embeddings encode indirectly, and we will say so in the paper.

Q1. The isolated English temporal generalization seems to begin before onset

The reviewer is right, and the cause is the same window average as in W4. A 200-ms centered window allows a strictly causal response to appear up to 100 ms before its true onset.

Fig. R10 shows this in the B.6 simulation with no predictive signal:

  • with point samples (left), the isolated temporal generalization starts at 0 ms;
  • with the 200-ms window used for the neural data (right), the same generator produces generalization around 0 ms in the second and fourth quadrants, although the ground truth contains no prediction.

Fig. R10. B.6 generator without prediction: 25-ms point samples (left) vs a 200-ms centered window average (right).

Q2. Does TI remove the difference between correctly and incorrectly predicted words?

No. We split words into two surprisal classes: correct (in the language model's top-5 predictions; low surprisal) and incorrect (high surprisal); 58% of English (GPT-2 XL) and 62% of Chinese (Qwen2.5-1.5B) words are correct. We then compared high-gamma activity of the two groups before and after two-sided TI (Table R5; full time courses in Fig. R11, where grey shading marks lags with FDR q < .05).

  • English. Correctly predicted words show higher activity before onset and lower activity after onset; incorrectly predicted words show the opposite pattern. TI removes neither difference.
  • Chinese. The post-onset pattern is the same and also survives TI. Before onset, there is no reliable difference in this window.

Table R5. High-gamma activity, correctly minus incorrectly predicted words.

Correct − incorrect (SD)Before TIAfter TIParticipants (before / after)Paired t-test p (before / after)
English, −1000 to −100 ms+.022+.0128/8 / 7/8 positive< .001 / .001
English, 100 to 1000 ms−.035−.0298/8 / 8/8 negative< .001 / < .001
Chinese, −1000 to −100 ms−.019−.0115/5 / 4/5 negative.07 / .18
Chinese, 100 to 1000 ms−.085−.0655/5 / 5/5 negative< .001 / < .001

We cannot tell whether the difference between the languages before onset reflects the recording modality (ECoG vs SEEG), the language, or the participants. In answer to the reviewer's question, TI does not remove the difference between correctly and incorrectly predicted words.

Fig. R11. High-gamma activity of correctly vs incorrectly predicted words (top-5), raw (left) and after two-sided TI (right).

We thank the reviewer again. These analyses made Target Isolation cleaner, changed our main model to the two-sided CAST-FIR, and replaced the thresholded headline with matched, statistically supported measures.

Reviewer_38tEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses decoding of perceived naturalistic speech from fMRI, ie. retrieving correct speech segment from about 2,000 candidates. The authors propose NOEMA, a subject-specific decoder that applies a shared per-frame spatial projection and two dilated temporal convolution blocks to a (sometimes) seven-frame fMRI window, predicting an embedding for the central frame. Models use a symmetric CLIP-style contrastive loss against frozen multimodal embeddings that concatenate text,audio and speech representations.

Evaluated on two naturalistic listening datasets (Cephalonauts One and LeBel) with session-based splits, NOEMA reaches approx 43% accuracy, outperforming a ridge control and single-frame MLP, MindEye-style MLP and a seven-frame Context MLP baseline despite having fewer parameters than the latter. Ablations indicated that author-proposed methnod benefits more than the Context MLP from additional frames, depends on frame order, and is improved when adding audio and speech targets to text. The advantage persists when the candidate pool is extended with thousands of segments from unseen data.

Critical strengths
  • Baselines all go through same tuning procedure (but see below for some confusion on the baseline comparisons in other places)
  • Provided code for reproducibility
  • Good use of stats and providing erorr bars
  • Key comparison is well controlled, showing authors didn't just take a single positive result and try to massage it but there seems to be an attempt to show it holds under a variety of potential confounding factors, which I liked
  • admission on the fact it's retrieval and not open-ended generation (lots of similar work tries to pass this off as that, so candid nature of explanation helps to understand the authors aren't trying to over-extend their findings (similar story for admission on difficulties with the HRF and what's happening with temporal structure in the task)
  • Biggest issue avoided by using session-wise splits, a huge concern and this being taken into account points to more of a rigid set of experience with correct data analysis in the field
Critical weaknesses
  • References and citations seem sparse - a lot has been done in this area and while some major works are cited, it feels like a very thin layer that reflects maybe an incomplete / non-comprehensive familiarity of related works in the area on the part of the authors
  • The related work feels like a patchwork of a bundle of related papers with sometimes only a thin thread linking them together, not really telling the story as emergent over time in a form of a developmental progression as I would have expected
  • There are citations to important references but no acknowledgement of the main processes of analysis. For example, the way that Huth and friends would deal with this is to resample the stimuli and have FIR delays incorporated into the analysis, while this boxcar-style motivation is more similar to the early fMRI alignment work in Bingel et al. (2017), which was fine for 2017 but not so much for 2026. I’d be happy to accept the motivation for this analysis if the more recent and standard one was recognised and argued against, but that doesn’t happen and it’s a gap that I must assume is due to the authors’ familiarity with how this is done in a state of the art way
  • The MLP and Ridge baseline only takes a central frame, right? While the other models see 7 frames? But the abstract and other parts of the paper often talk about “outperforming the baselines” - isn’t this unfair as they have seen fewer data points and have an impoverished signal to work with? I guess the Context MLP is contrasted, which does have the same input, but e.g. in Appendix C NOEMA is contrasted against just the MLP, and for conclusions to be drawn there, we’d expect the same inputs not 1-frame versus multiple, so I feel there is an issue here.
  • Separate linear projections in the Context MLP exist, while there’s a strong element of weight sharing in the NOEMA model, which we know can serve as a good regulariser. Therefore, conclusions made on the fact that temporal convolutions were used can’t be easily disentangled from the multiple differences between the two models under comparison. Applying shared-weight version without temporal convolution would disentangle the contributions and be a stronger result to present
Questions for authors

My only questions really come from the weaknesses I raised above, so would like to hear from the authors about anything they felt I missed or might have misunderstood and how the authors would choose to strengthen their paper because of it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.