acceptodds
Under review as a conference paper at ICLR 2027

A Low Rank Space Captures All Linear Alignment between LLM and Human Neural Representations during Naturalistic Language Listening

Abstract

Large language models (LLMs) are increasingly used as computational models of human language processing, yet it remains unclear which components of their representations drive alignment with neural activity. Here, we use ridge reduced-rank regression to identify the brain-predictive subspace of LLM representations: the representational directions that support linear prediction of neural responses. Across three naturalistic language-listening datasets spanning intracranial electrophysiology and fMRI, and six model conditions spanning autoregressive, bidirectional masked, and randomly initialized language models, we find that full linear encoding performance can be recovered using only  1–3% of the original embedding dimensionality. These low-dimensional subspaces generalize across held-out participants and, to a lesser extent, across datasets, suggesting that neural predictivity is concentrated in a compact and partially shared representational space. We next compare these brain-predictive directions with the principal axes of variation in the embeddings. Principal component analysis requires substantially more dimensions to recover neural prediction, demonstrating that the directions most useful for predicting neural activity are distinct from those capturing the most overall variance within LLM representations themselves. Finally, characterizing the learned subspaces allows us to surface interpretable features that are overrepresented in neural activity above and beyond the LLM representations. Together, these results show that LLM–brain alignment is supported by a small, structured subset of model representations and provides a framework for better understanding exactly why these representations align.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neuroscience, Encoding Models, LLMs, Language Listening

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XQxbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work uses ridge reduced-rank regression to identify the representational directions that support linear prediction of neural responses. Testing results show that LLM–brain alignment is supported by a small, structured subset of model representations.

Critical strengths

None

Critical weaknesses
  1. No source code.
  2. Experimental issues.
Questions for authors
  1. This work proposes a low-rank space that captures all linear alignment between LLM and human neural representations during naturalistic language listening. However, the authors have not provided the source code for the proposed method. Without access to the code, it is difficult for readers to verify the experimental results.
  2. Go on with CMT 1, the implementation details are not presented. The presentation is missing hyperparameter settings and training and testing configurations. This lack reduces a thorough understanding in practice.
  3. The work lacks a comprehensive error analysis and bad case studies, such as selection leakage. Analyzing failure cases is crucial to analyzing LLM and human neural representations during naturalistic language listening.
  4. Why SATURATION-RANK COMPARISONS are introduced. The motivation for introducing SATURATION-RANK COMPARISONS is not given in detail.
  5. The experimental evaluation is weakened, and the conclusion is not convincing due to an insufficient number of compared baselines. Recent SOTA methods should be added in the evaluation part.
  6. The established dataset coverage does not support the proposed claims. Evaluation on a limited benchmark can not proof its generalizability to real scenarios.
2 comments
AuthorsOpenReview ↗

Response Part 1

===== COMMENT 1/2 ===== Response to Reviewer 3 (1/2)

We thank the reviewer for their feedback. Changes made in response are shown in blue and tagged [R3] in the revised PDF; our general comment summarizes the changes for all reviewers. We address the code in Q1 and the experimental concerns in Q2–Q6.

Q1. Source code

The full code was included with the original submission as a zip file in the supplementary material. As this is easy to miss, the Reproducibility statement now points to it explicitly. It includes a guide that walks through every step needed to reproduce our results from the publicly available datasets. The final version will link to a public repository.

Changes: Reproducibility statement.

Q2. Implementation details

We agree these were hard to find. The appendix describes them in full:

  • Preprocessing and embeddings (A.1): word alignment, epochs, high-gamma extraction for Cars 2, and layers sampled at 0/25/50/75/100% depth.
  • Train/test configuration (Section 3.4, A.4): five contiguous held-out word folds for iEEG; training stories vs. a held-out, repeated test story for fMRI.
  • RRR fitting and ranks (A.2.1): basis construction and orthonormalization; ranks doubling from 1 to the full dimensionality.
  • Ridge penalties (A.2.2): a 25-value, scale-aware grid selected by inner cross-validation per target.
  • Target selection (A.4): circular-shift nulls with Benjamini–Hochberg correction.
  • Feature decoding (A.2.4–A.2.5): decoders, features, and metrics.
  • Statistics (A.3): sign-flip tests and bootstrap procedures, including resample counts and seeds.

Changes: Appendices A.1–A.4.

Q3. Error analysis and failure cases

Selection leakage. We deliberately select targets broadly, keeping every site the full-embedding ridge model predicts significantly better than chance, because we want an upper bound on the dimensionality of the aligned subspace (Section 3.3). In iEEG, selection and evaluation share word folds; in fMRI, the test story is excluded from selection. Selection can therefore raise the absolute performance of selected iEEG targets, but our conclusions never rely on absolute performance:

  • Selection uses only the full-embedding model, so it is independent of method and rank and cannot favor RRR over PCA.
  • RRR and PCA are evaluated on exactly the same targets.
  • Each saturation rank is relative to the maximum of its own curve, so scaling absolute performance does not by itself change it.
  • With shuffled embeddings, the same targets show no RRR advantage (Appendix A.14).
  • fMRI, where evaluation is fully separate from selection, shows the same low-rank effect.

We added this reasoning to Appendix A.4.

Failure cases. The paper reports several conditions where the effect weakens or disappears:

  • Shuffled embeddings (A.14): RRR no longer saturates below PCA in any dataset.
  • Random GPT-2 (Section 4.1, A.11): mixed; an RRR advantage in Cars 2 and fMRI but not Podcast.
  • New participants (Section 4.2): in Cars 2, transferred subspaces need more dimensions than subspaces fit with that participant's data.
  • New datasets (Section 4.3, A.9): cross-dataset transfer is less efficient than within-dataset RRR, partly due to stimulus-dependent shifts in embedding geometry.
  • Subspace geometry (Section 4.3, A.12): RRR subspaces from different datasets are no more similar to each other than to PCA.

Changes: New paragraph in Appendix A.4; Appendices A.11, A.12, A.14.

AuthorsOpenReview ↗

Response Part 2

===== COMMENT 2/2 ===== Response to Reviewer 3 (2/2)

Q4. Motivation for the saturation rank

The saturation rank answers our central question: how many dimensions of an LLM representation are needed to recover essentially all of its linear neural predictivity. We expected this number to be large: hundreds of principal components are needed to capture most of the variance of the embeddings, and thousands for the neural responses. If LLMs are good models of human language processing, the correspondence between the two might be similarly high-dimensional. Instead, roughly 10–20 dimensions suffice.

We compare RRR with PCA because PCA keeps the embeddings' directions of greatest variance without reference to the brain. If RRR saturated at the same rank as PCA, the brain-predictive subspace would simply be the dominant directions. That PCA needs 8–18 times as many dimensions shows that neural prediction depends on specific, non-dominant directions. Appendix A.3 tests this difference across models.

Changes: Introduction, paragraphs 3–4; Section 3.6.

Q5. Additional baselines

We do not propose a method to benchmark against state-of-the-art neural predictors; we ask how much of an LLM representation supports linear alignment with the brain. Linear encoding models are the standard way this alignment is measured e.g., Schrimpf et al., 2021; Goldstein et al., 2022; Caucheteux & King, 2022; Antonello et al., 2023, and we follow their procedures. Nonlinear models can predict neural activity more accurately but do not measure representational alignment in the same sense. Within this framing, the relevant comparisons are:

  • PCA, the standard unsupervised reduction to k dimensions, which tests whether the brain-predictive directions are simply the dominant ones;
  • controls: a randomly initialized GPT-2, static GloVe embeddings, and shuffled embeddings;
  • six trained models, including recent large models such as Llama 3 8B, spanning autoregressive and masked objectives. The effect holds for every one.

Q6. Dataset coverage

We agree that three datasets cannot cover all of human language comprehension. Within publicly available naturalistic datasets, however, our evaluation is broad:

  • about 20 participants across three independent datasets (9 Podcast, 3 Cars 2, 8 fMRI);
  • three recording techniques (ECoG, sEEG, fMRI) differing greatly in temporal and spatial resolution and SNR;
  • three kinds of stimuli: a podcast, a movie, and narrative stories;
  • six trained models, each at five layers.

The low-rank effect holds in every dataset, model, recording technique, and anatomical region we examined (Section 4.1). Subspaces also transfer to participants not used to estimate them (Section 4.2) and partially to entirely different datasets (Section 4.3).

Summary

We pointed explicitly to the code, pointed to the full implementation details, explained why target selection cannot bias our conclusions, described the failure cases the paper reports, and clarified the motivation for the saturation rank. Together with the controls added for other reviewers (see general comment), we believe these revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their assessment.

Reviewer_d9TrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper provides a clean methodology, ridge reduced-rank regression, to analyze the subspaces of LLM with respect to LLM-brain alignment, crossing three modalities, ECoG, sEEG, and fMRI.

Critical strengths

The methodology is well defined, and the major insight is clear. The methodology is well defined. The RRR outperforms PCA is something simple but elegant. The discovery that RRR can use much less components to outperform PCA, and the reference ridge regression model with proper ranks, and how they address, explain, and analyze it is also the strength of this paper.

Critical weaknesses

However, there are some crucial issues that must be resolved in the rebuttal: (1) the lack of the regions of interest in the brain (2) the lack of the nature of brain modality

The most important issue is the regions of interest in the brain. For the ECoG and fMRI datasets, the stimulus is audio, while for the sEEG dataset, the stimuli are visual and audio. So, what regions of interest are we talking about, for each modality? For sEEG and ECoG, are all the electrodes used in the analysis? For fMRI, all the voxels across the brain, or cortex, are used? Different ROI exhibit very different activations, especially when aligned with LLM, or predicted by LLM embeddings. To make the whole paper stand, only the language network, and the audio cortex, or the visual cortex (if in sEEG dataset with visual stimuli ) can be considered. The analysis must also done on different ROI and show the different are they and provide insight and observation. Moreover, the sEEG dataset also contains the visual input. Such mismatch provides more uncontrollable confounder in the analysis, not to mention the difference in the stimuli themselves are confounders.

The other important issue is the lack of the nature of the three modalities chosen in the paper. They choose sEEG, ECoG, and fMRI, which is a good combination to provide in-depth analysis. However, in the whole paper, the analysis and insight does not link the nature of these three modalities with the observed result.

Questions for authors
  1. The sEEG dataset also contains the visual input, but only the transcript is input to LLM, no other prompts, like the description of the scene written in text for LLM, or using vision model to take input from the scene? For sEEG dataset, are both the audio and visual cortex considered for activation prediction?
  2. In line 245, inside 3.8 (Generalization analysis), how is “For cross-dataset transfer, we learned an RRR or PCA subspace from a source dataset and projected the target dataset’s embeddings into that space.” done exactly? There is no appendix embedded in this subsection. But how cross-dataset transfer is done should be clearly stated.
  3. Please explain in the rebuttal, why figure 2c looks so different with respect to figure 3c. The saturation rank for RRR in Podcast is 20 in 2c, but 62.7 in 3c. PCA’s saturation rank also differ by a lot. What cause such difference?
  4. The description of figure 3c does not match figure 3c. In section 4.2, line 316, it says “In the Podcast and fMRI datasets, the transferred subspaces required a comparable number of dimensions—and in some cases fewer—to reach saturation, whereas the Cars 2 dataset showed a larger difference between transferred and participant-inclusive RRR.” This does not match what I see in figure 3c. Also, what is shown in figure 3c is not properly mentioned. What I see in figure 3c is that saturation rank for transferred subspaces is substantially higher for Podcast and Cars, but substantially lower for fMRI. How to explain this? As an analysis paper, such explanation is a must in this section to properly address the explainability and insight.
  5. Please mention in section 4.4 how to obtain the enrichment profile in figure 6b, and why comparing the relative AUC between datasets, and what the correlation can convey.
  6. This paper mentioned "six model conditions spanning autoregressive, bidirectional masked, and randomly initialized language models" in abstract and introduction, but no related RRR within and transfer analysis is provided in the main context. Is there major insight here? What can I learn for RRR among the six deployed models?

Minor on format or notation:

  1. Lewis et al. citation is not complete. (No year, no source, etc.)
  2. Figure 1c does not match its description. What does the red dashed line mean? Why the blue and green dashed line are dropped there? They don’t seem to align with what is described in 3.7 (Saturation rank), in which the saturation rank is defined by all the selected quantiles reach 99% of the reference performance. With full annotation as in Figure 3a, I can understand this plot. But in Figure 1c, a reader cannot get it provided the information given before Figure 1a and Section 3.7.
  3. The label is not aligned. In the current notation, like in figure 2 and 3, the three datasets are labeled by Podcast, Cars 2, and fMRI. What should be labeled should be something like ECoG, sEEG, and fMRI, or Podcast (ECoG), Cars 2 (sEEG), and the dataset’s name (fMRI).
3 comments
AuthorsOpenReview ↗

Response Part 1

===== COMMENT 1/3 ===== Response to Reviewer 2 (1/3)

We thank the reviewer for their detailed feedback, which has made the paper considerably clearer. Changes made in response are shown in blue and tagged [R2] in the revised PDF; our general comment summarizes the changes for all reviewers.

W1. Regions of interest

Which targets we use. In all three datasets we used every recording site whose responses were significantly predicted by full-embedding ridge models: electrodes for Podcast (ECoG) and Cars 2 (sEEG), and voxels for fMRI. Significance was assessed against circular-shift nulls with Benjamini–Hochberg correction (Section 3.3, Appendix A.4). Selection is not restricted anatomically, but every retained site carries reliable signal that the embeddings predict.

Why we did not restrict the main analysis to ROIs. We want an upper bound on the dimensionality of the subspace that supports neural prediction as it is typically measured. Restricting to a priori regions would build in assumptions about where language information is represented, and a smaller, more homogeneous target set would lower the rank we recover. Including every predicted site is therefore the conservative choice: if the subspace is small here, it should be no larger within any subset. Whole-brain encoding without a language-network restriction is also common in this literature e.g., Huth et al., 2016; Jain & Huth, 2018; Antonello et al., 2023; Goldstein et al., 2022. Section 3.3 now states this rationale.

ROI analysis. We agree that regions differ, and that a low pooled rank could hide higher-dimensional alignment within particular regions. We therefore repeated the main analysis within eight anatomical regions commonly implicated in language comprehension: superior, middle, and inferior temporal gyri, temporal pole, inferior frontal gyrus, temporoparietal junction, and precentral and postcentral cortex. Because functional localizers are not available, the regions are defined anatomically (Destrieux atlas) and only approximate the language network. In every region and dataset, RRR saturated at a substantially lower rank than PCA (all p ≤ 0.031, one-sided model-level sign-flip tests), so the low rank is not an artifact of pooling heterogeneous regions.

Saturation-rank definition. We replaced the original quantile-based criterion with the smallest rank at which explained variance across all selected targets reaches 99% of its maximum. This is easier to interpret and maps directly onto what we want to bound: the number of dimensions needed to recover essentially all linearly explainable neural variance. Conclusions are unchanged under the original definition (Appendix A.16).

Changes: Section 3.3; new "Robustness across anatomical regions" paragraph in Section 4.1; new Appendix A.6; Section 3.6; Appendix A.16.

AuthorsOpenReview ↗

Response Part 2

===== COMMENT 2/3 ===== Response to Reviewer 2 (2/3)

W2. Linking results to each recording modality

We agree that the paper does not offer a complete account of how recording modality shapes the subspace, and we have strengthened this aspect. Several results bear on it:

  • Size. RRR saturation ranks are similar for fMRI (9.5) and sEEG (9.1), despite very different temporal resolution and SNR, so the low rank is not a property of one measurement type.
  • Generality. Cross-dataset transfer (Section 4.3) tests how well a subspace learned in one modality transfers to another.
  • Content. The two iEEG datasets enrich similar linguistic features (r = 0.70), while neither resembles fMRI (r = −0.10, −0.11). Section 4.4 now relates this to resolution: iEEG samples fast, word-locked activity from local populations, whereas fMRI integrates hemodynamic responses over several seconds, and so several words, across larger volumes of tissue.
  • Modality-specific analyses. The per-lag analysis (Appendix A.7) uses the temporal resolution of iEEG, and the noise ceiling (Section 3.7) uses the repeated story available only for fMRI.

A full account would require datasets that hold the stimulus fixed while varying the recording method, which is beyond our scope. A new future-directions paragraph in the Discussion names this as a next step.

Changes: Section 4.4, paragraph 3; final paragraph of the Discussion.

Q1. Visual input in Cars 2

Participants watched the movie, while the models received only the transcript. As in the other datasets, we included every electrode significantly predicted by the embeddings, wherever it was located. Our goal is to bound the size of the language-model subspace, not to explain neural responses fully, and the visual input can only make this bound more conservative: retained targets include any signal the transcript embeddings predict, whether linguistic or visual but correlated with the language, and any purely language-driven subset would require at most as many dimensions. Consistent with this, Cars 2 has the lowest mean RRR saturation rank (9.1, vs. 21.2 for Podcast and 9.5 for fMRI). Section 4.3 notes the visual component as a possible contributor to cross-dataset differences.

Q2. Details of cross-dataset transfer

We expanded Section 3.8. For a given model and layer, the orthonormal basis Q_k (embedding dimensionality × k) is estimated from the source dataset's embeddings (and, for RRR, its neural responses) and applied unchanged to the target dataset's embeddings from the same model and layer. Target embeddings are centered with the source statistics, because the basis is defined in that space. A new ridge model is then fit and evaluated using only the target dataset's neural responses, so only the embedding subspace is transferred.

Changes: Section 3.8, paragraph 2.

Q3. Saturation ranks in Figures 2c vs. 3c

The panels compute different quantities. Figure 2c uses one explained-variance curve averaged over all participants' targets. In Figure 3c, all three methods are scored only on the held-out participant's targets, and per-participant saturation ranks are averaged. The all-participant RRR subspace is the same in both; only the evaluation differs. The two summaries can differ because (i) the pooled curve is dominated by participants with many well-predicted targets, whereas averaging per-participant ranks weights all participants equally, and (ii) single-participant curves are noisier, and a small rise at a high rank can push one participant's rank up substantially. Neither affects the comparison within Figure 3c, where all methods are scored identically.

The large gap the reviewer noted was also partly due to the original quantile-based criterion, which was very noisy for single participants. Under the revised definition, the summaries are much closer: Podcast 21.2 (Fig. 2c) vs. 23.6 (Fig. 3c), Cars 2 9.1 vs. 12.1, fMRI 9.5 vs. 22.9.

Changes: Section 3.8 and the Figure 3 caption.

AuthorsOpenReview ↗

Response Part 3

===== COMMENT 3/3 ===== Response to Reviewer 2 (3/3)

Q4. Section 4.2 text vs. Figure 3c

The reviewer is right, and we have rewritten this part of Section 4.2. Transferred RRR requires a comparable number of dimensions to all-participant RRR in Podcast (24.6 vs. 23.6), more in Cars 2 (19.8 vs. 12.1), and fewer in fMRI (12.4 vs. 22.9). In Cars 2, part of each participant's predictable signal likely lies in directions the remaining participants do not emphasize. The lower transferred rank in fMRI is harder to interpret; one untested possibility is that including noisier participants leads some leading RRR directions to capture noise. In every dataset, the transferred subspace remains low-dimensional without access to the evaluated participant's responses.

Changes: Section 4.2, paragraph 2.

Q5. Enrichment profiles

Section 4.4 now describes the method before the result. For each feature and model layer, we compute the relative decoding advantage of RRR over PCA at each rank and summarize it by its area under the curve. The set of these values for a dataset is its enrichment profile. Correlating two datasets' profiles asks whether neural supervision in each enriches the same features relative to PCA: a high correlation means they favor the same information, a correlation near zero that they favor different information. The scatter plot is now Appendix Figure A8.

Changes: Section 4.4, paragraph 3; Appendix Figure A8.

Q6. Insights across model conditions

We used multiple models mainly to test whether the low-rank effect generalizes across architectures, objectives, and scales, and it does: every trained model saturates at a lower rank with RRR than with PCA in every dataset. Figures 2c and 3c now show each model's mean and bootstrap CI, and all tests use the model as the unit of analysis.

The question also prompted a closer look at the randomly initialized GPT-2. Its results were mixed: like the trained models, its PCA rank lay well above its RRR rank in fMRI and Cars 2, but fell below it in Podcast. As a single, weakly predictive model, its ranks are noisy and untestable, so whether training is necessary remains unclear; untrained representations still carry word identity and position, which may partly produce the effect. Section 4.1 now discusses this, with full results in a new appendix.

We also corrected the abstract: we analyze six trained models (four autoregressive models, RoBERTa, and static GloVe embeddings) plus the random control. Comparing the subspaces themselves across architectures is an interesting question that we now name as a future direction in the Discussion.

Changes: Abstract; Section 3.2; new paragraph in Section 4.1; new Appendix A.11; final Discussion paragraph.

Minor points

  • M1. The Lewis et al. citation is now complete.
  • M2. Figure 1c is redrawn to match the revised saturation-rank definition. The caption now explains that vertical dashed lines mark each method's saturation rank and the red dashed line marks full-embedding ridge performance.
  • M3. All figures now label datasets with their modality: Podcast (ECoG), Cars 2 (sEEG), Moth (fMRI).

Summary

We added an ROI analysis showing the low-rank effect within every region examined, explained why the main analysis uses all predicted sites, linked the results more closely to recording modality, specified the transfer procedure, explained the difference between Figures 2c and 3c, and corrected Section 4.2. We believe these revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their assessment.

Reviewer_5MNLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses ridge reduced-rank regression (RRR) to find a "brain-predictive subspace" of LLM word embeddings. It covers three naturalistic listening datasets (Podcast ECoG, Brain Treebank Cars 2 sEEG, Deep fMRI), five LMs plus random-GPT-2/GloVe controls, and five layers each. Full ridge encoding performance saturates at roughly tens of dimensions, compared with hundreds for PCA. The subspace transfers across held-out participants and partially across datasets, and feature-decoding contrasts (RRR vs. PCA) show different linguistic "enrichment" profiles for iEEG and fMRI.

Critical strengths
  • The question is clearly posed. It turns a scalar encoding score into a representational object (a subspace) whose size, transferability, and content can be studied. This is a useful reframing for the LLM-brain alignment literature.
  • Breadth: three datasets covering two recording modalities (iEEG, fMRI), several model families (causal, masked, random, static), and a layer sweep. The low-rank result looks consistent across this grid (Fig. 2c, Fig. 10).
  • The generalization tests are well designed. Only the embedding subspace is transferred and the readout is refit on the target participant or dataset (Sec. 3.8), which separates subspace generality from mapping generality.
  • The PCA-transfer gap is a neat, neural-data-free control for stimulus distribution shift, and it covaries with the RRR transfer gap (Fig. 5). This is a useful way to split up why cross-dataset transfer fails.
  • Methodological care in the appendix: threshold and floor sensitivity analyses (Figs. 11-12), quantile-based saturation instead of the mean alone, and RoBERTa context windows that do not see future words (A.1.1). Code is provided.
Critical weaknesses
  • Missing control: check what happens when RRR fitted on shuffled Y, to check whether the subspace is related to actual alignment or simply distributional properties
  • "Brain-predictive directions differ from principal axes”: to show this in more detail, the authors could investigate more deeply the structual properties of the low-dimensional RRR subspace: for example, how aligned/misaligned are the top k right singular vectors with the top k PCA components?
  • The subspace size is probably limited by the neural data, not the LLM. The saturation rank is likely set by the effective dimensionality and SNR of Y: noisy high-gamma at a handful of lags, or heavily smoothed BOLD. The paper never reports the effective rank of Y to test this. Without that, "1-3% of embedding dimensionality" cannot be interpreted
  • Vague (if not overselling) statements: for example, the abstract says the work surfaces "features that are overrepresented in neural activity above and beyond the LLM representations.”, whereas the analysis only decodes features from a projection of the LLM embeddings

Minor

  • The sign-flip tests and the Fig. 5 correlation (r=0.58, n=184) treat model-layer combinations as independent. Layers of the same model, and models evaluated on the same stimuli, are strongly dependent, so the tests are pseudo-replicated and the p-values anti-conservative.
    • No error bars or CIs on saturation ranks are described.
Questions for authors
  • How stable are the subspaces themselves? Please report principal angles or CKA between RRR subspaces across folds, layers, and models, and between LOSO and all-subject fits. If transfer and saturation hold but the subspaces themselves are unstable, the interpretation changes.
  • What happens if we replace PCA/RRR by other unsupervised/supervised dimensionality reduction methods (PLS, CCA, autoencoders…)
3 comments
AuthorsOpenReview ↗

Response Part 1

===== COMMENT 1/3 ===== Response to Reviewer 1 (1/3)

We thank the reviewer for their careful reading and helpful comments, which have substantially strengthened the paper. Changes made in response are shown in blue and tagged [R1] in the revised PDF; our general comment summarizes the changes for all reviewers.

W1. Shuffled-Y control

We agree that this control was missing. We repeated the full within-dataset analysis after randomly permuting the word embeddings relative to the neural responses. This breaks the word–response correspondence while leaving both distributions unchanged; everything else (target selection, fitting, rank sweep, saturation rank) was identical.

If the low RRR rank reflected the statistics of either space rather than their alignment, the RRR–PCA gap should persist. It does not:

  • Explained variance falls sharply in every dataset (e.g., peak mean r² for GPT-J 6B layer 21 drops roughly tenfold in both iEEG datasets).
  • RRR no longer saturates below PCA in any dataset (mean saturation rank, RRR vs. PCA: Podcast 89.9 vs. 67.2, p = 0.83; Cars 2 112.3 vs. 102.3, p = 0.78; fMRI 1.0 vs. 1.1, p = 0.25; one-sided model-level sign-flip tests).
  • The normalized curves are nearly flat, so saturation ranks under shuffling are dominated by noise.

The low-rank structure thus depends on the correspondence between embeddings and neural responses, not on RRR or on the marginal statistics of either space.

Changes: New Appendix A.14 and Figure A19; summary sentence in Section 4.1.

W2. Alignment between RRR and PCA directions

We added a principal-angle analysis of the subspaces (new Appendix A.12). At each rank r, we measure the overlap of two subspaces as the mean squared cosine of their principal angles (1 = identical, 0 = orthogonal), relative to the chance level r/d.

Cross-participant. For each held-out participant, we compared the RRR subspace fit on that participant alone with (i) the RRR subspace fit on the remaining participants, which shares no neural data with it, and (ii) the PCA subspace. Comparison (ii) answers the reviewer's question directly; comparison (i) asks whether independently estimated RRR subspaces agree with each other more than with PCA.

The result depends on rank as expected if neural signal determines the RRR directions. Below each model's saturation rank, RRR subspaces from disjoint participants overlapped more with each other than with PCA in every dataset (p = 0.016 in each, the minimum attainable with six models; one-sided model-level sign-flip tests). Above it, the advantage reversed in both iEEG datasets (p = 0.016 each) and shrank to near zero in fMRI: once the predictive signal is exhausted, the remaining RRR dimensions are no longer constrained by the neural data.

Cross-dataset. RRR subspaces learned in different datasets did not overlap with each other more than with PCA. This is consistent with our PCA transfer analysis (Section 4.3), which shows that the stimulus sets occupy different regions of embedding space. Transferred RRR subspaces nevertheless remain more predictive than transferred PCA subspaces for every dataset pair (Figure 4b), so brain-predictive structure carries over across datasets without appearing as direct geometric overlap. Section 4.3 now states this.

Changes: "Subspace consistency" paragraph in Section 4.2; new sentence in Section 4.3; new Appendix A.12 with Figures A13–A14.

AuthorsOpenReview ↗

Response Part 2

===== COMMENT 2/3 ===== Response to Reviewer 1 (2/3)

W3. Saturation rank may be limited by the neural data

We agree this is an important alternative and added three analyses.

  1. Effective dimensionality of Y (new Appendix A.5). Several thousand principal components are needed to explain 80% of the variance of the selected neural targets (Podcast 3,343; Cars 2 5,311; fMRI 4,818), two orders of magnitude above the RRR saturation ranks of about 10–20. Because this measures total rather than reliable variance, we added the next analysis.
  2. Reliable signal in fMRI (new Section 3.7, Appendix A.8). Using the test story repeated five times, we estimated a per-voxel noise ceiling (CC_max; Schoppe et al., 2016). At each voxel's saturation rank, models captured a median of only 7–18% of its explainable variance (12–18% for trained contextual models beyond the input layer). Most reliable signal is still unexplained when RRR saturates, so the rank appears limited by what is linearly available from the embeddings, not by the reliable signal in the recordings. This is not possible for iEEG, which lacks repeated stimuli.
  3. Temporal sampling in iEEG. Each electrode contributes 128 lags (−2 to 2 s around word onset at 32 Hz). Computing saturation ranks at each lag separately, RRR saturates below PCA at 123/128 (Podcast) and 124/128 (Cars 2) lags (Appendix A.7).

In addition, the RRR–PCA comparison is internally controlled: both methods predict the same Y, yet PCA needs 8–18 times as many dimensions. And saturation ranks are similar for fMRI (9.5) and sEEG (9.1), despite very different temporal resolution and SNR.

We agree that the noise ceiling shows how much reliable signal remains, not how many dimensions would be needed to capture it, and the Discussion states this caveat.

Changes: Section 3.7; new paragraph in Section 4.1; Discussion paragraph 2; Introduction paragraph 3; new Appendices A.5 and A.8.

W4. Overstated claims ("above and beyond the LLM representations")

We agree that this wording was inaccurate. The decoding analysis compares two projections of the same embeddings, so it cannot reveal information absent from them; every feature we decode is present in the full embedding space.

What the analysis does show is a mismatch in representational geometry. PCA keeps the directions along which the embeddings vary most; RRR keeps those most useful for predicting neural responses. A feature decoded better from RRR than from an equally sized PCA subspace is carried by directions that matter for neural prediction but are not the embeddings' dominant axes of variance. This complements our finding that PCA needs 8–18 times as many dimensions. The abstract now reads:

"Finally, comparing which linguistic features can be decoded from the brain-predictive and principal-component subspaces reveals a mismatch in representational geometry: interpretable features that are present but not dominant in the embeddings are preferentially carried by the directions that best predict neural activity."

Section 4.4 states the same interpretation and keeps the caveat that greater decodability does not show that a feature itself drives neural prediction.

Changes: Abstract; Section 4.4, paragraph 1.

AuthorsOpenReview ↗

Response Part 3

===== COMMENT 3/3 ===== Response to Reviewer 1 (3/3)

M1. Pseudo-replication

We agree. All sign-flip tests now use the model as the unit of analysis: we average each quantity (e.g., log₂ saturation rank) across layers within each model, then apply an exact one-sided sign-flip test across the six trained models (all 2⁶ = 64 sign assignments; minimum p = 1/64 ≈ 0.016). Every effect significant in the original submission remains significant (p < 0.05).

For the correlation between RRR and PCA transfer gaps (now Appendix Figure A7), we no longer report a p-value, since the observations share models, layers, and stimuli. We report it descriptively (r = 0.59, n = 156 model-layer × dataset-pair combinations, trained models only), and treat the cross-dataset enrichment correlations the same way (Appendix A.10).

Changes: Appendix A.3; Appendices A.9 and A.10; captions of Figures 2–4.

M2. Uncertainty on saturation ranks

We added 95% cluster-bootstrap confidence intervals for every model's saturation rank (Figures 2c, 3c) and for AUC differences (Figure 4b). Because targets are not independent, we resampled participants (Podcast, fMRI) or depth electrodes (Cars 2, which has only three participants), recomputed curves and saturation ranks in each of 2,000 resamples, and averaged across layers within each model. For the leave-one-subject-out analysis we resampled held-out participants (not computed for Cars 2, where three participants allow only ten distinct resamples). The estimates are stable, though some models have wider intervals than others.

Changes: New Appendix A.3.2; Figures 2–4.

Q1. Stability of the subspaces

Please see W2 for the principal-angle analysis.

Changes: Section 4.2; Appendix A.12.

Q2. Alternative dimensionality-reduction methods

This is an interesting question. PLS, CCA, or nonlinear autoencoders optimize different objectives from RRR and could surface different, complementary structure. Our aim, however, is to characterize the part of the embedding space that supports the standard linear encoding model used in this literature, and RRR suits this aim because it finds the rank-k linear map that best reproduces the fitted neural responses. We see a systematic comparison of these methods as valuable future work that our framework makes straightforward.

Changes: None.

Summary

We added a shuffled-embedding control, principal-angle comparisons of the subspaces, three analyses showing that the saturation rank is not set by the neural data, model-level statistics, and bootstrap confidence intervals, and we restated the decoding claim precisely. We believe these revisions resolve the soundness concerns raised, and we would be grateful if the reviewer would consider updating their assessment.

Comments on the paper

1 comment

We thank all three reviewers for their time and constructive feedback. Reviewers noted that the paper turns a scalar encoding score into a subspace whose size, transferability, and content can be studied (R1). They found the central result, that RRR recovers neural predictivity with far fewer dimensions than PCA, simple but elegant (R2). They also pointed to the breadth of datasets, models, and layers, and to the design of the generalization tests (R1). The reviews also raised important concerns, which we have addressed with new controls, new analyses, and clarifications. Below we summarize the main changes, and each reviewer's thread provides more details.

All changes are shown in blue in the revised PDF and tagged by reviewer ([R1], [R2], [R3]).

Main changes

1. Is the low-rank effect genuine alignment?

  • Shuffled-embedding control (R1): we repeated the full analysis after shuffling word embeddings relative to the neural data. Explained variance collapses, and RRR no longer saturates below PCA in any dataset (Section 4.1, Appendix A.14).
  • Randomly initialized GPT-2 (R2): unlike every trained model, the untrained control does not show a consistent RRR advantage (Section 4.1, Appendix A.11).

2. Is the saturation rank limited by the neural data rather than the embeddings? (R1)

  • Neural dimensionality: thousands of principal components are needed to explain 80% of the neural variance, far above RRR saturation ranks of about 10–20 (Appendix A.5).
  • fMRI noise ceiling: at their saturation ranks, models capture only a minority of each voxel's explainable variance, so most reliable signal remains unexplained when RRR saturates (Sections 3.7 and 4.1, Appendix A.8).
  • Per-lag analysis in iEEG: RRR saturates below PCA at nearly every lag (Appendix A.7).

3. Brain regions and recording modality (R2)

  • Region-of-interest analysis: the low-rank effect holds within each of eight anatomical language-related regions (Section 4.1, Appendix A.6).
  • Recording modality: we now relate the differences in decoded features between fMRI and iEEG to the temporal and spatial resolution of each modality (Section 4.4). We also added a future-directions paragraph to the Discussion.

4. Stability of the subspaces (R1)

  • Principal-angle analyses compare RRR subspaces with each other and with PCA, across participants and across datasets (Sections 4.2 and 4.3, Appendix A.12).

5. Statistics (R1)

  • Model-level exact sign-flip tests now treat the model as the unit of analysis, avoiding pseudo-replication across layers. Every originally significant effect remains significant.
  • Cluster-bootstrap 95% confidence intervals are shown on all saturation ranks and AUC differences (Figures 2–4, Appendix A.3).

6. Clarity and implementation details (R2, R3)

  • Saturation rank: a simpler definition, the smallest rank reaching 99% of maximum explained variance (Section 3.6).
  • Generalization analyses: a fuller description of cross-dataset transfer (Section 3.8) and a corrected description of the leave-one-subject-out results (Section 4.2).
  • Target selection: an explanation of why selection cannot favor RRR over PCA or set the saturation rank (Appendix A.4).
  • Implementation details: pointers to the hyperparameters and training and testing configurations (Appendices A.1–A.4).
  • Figures: Figure 1c redrawn, and recording-modality labels added to all figures.

7. Claims (R1, R2)

  • Abstract, decoding claim: restated as a mismatch in representational geometry. The features best decoded from brain-predictive directions are present in the embeddings but are not their dominant axes of variance.
  • Abstract and Introduction, models: corrected to describe six trained models plus a randomly initialized control.

8. Reproducibility (R3)

  • Code: the full code, with a step-by-step reproduction guide, is in the supplementary material. The Reproducibility statement now points to it, and the final version will link to a public repository.

We believe these revisions substantially strengthen the paper and address the reviewers' main concerns and thank the reviewers and chairs for taking the time to look over our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.