- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work uses ridge reduced-rank regression to identify the representational directions that support linear prediction of neural responses. Testing results show that LLM–brain alignment is supported by a small, structured subset of model representations.
Critical strengths
None
Critical weaknesses
- No source code.
- Experimental issues.
Questions for authors
- This work proposes a low-rank space that captures all linear alignment between LLM and human neural representations during naturalistic language listening. However, the authors have not provided the source code for the proposed method. Without access to the code, it is difficult for readers to verify the experimental results.
- Go on with CMT 1, the implementation details are not presented. The presentation is missing hyperparameter settings and training and testing configurations. This lack reduces a thorough understanding in practice.
- The work lacks a comprehensive error analysis and bad case studies, such as selection leakage. Analyzing failure cases is crucial to analyzing LLM and human neural representations during naturalistic language listening.
- Why SATURATION-RANK COMPARISONS are introduced. The motivation for introducing SATURATION-RANK COMPARISONS is not given in detail.
- The experimental evaluation is weakened, and the conclusion is not convincing due to an insufficient number of compared baselines. Recent SOTA methods should be added in the evaluation part.
- The established dataset coverage does not support the proposed claims. Evaluation on a limited benchmark can not proof its generalizability to real scenarios.
2 comments
Response Part 2
===== COMMENT 2/2 ===== Response to Reviewer 3 (2/2)
Q4. Motivation for the saturation rank
The saturation rank answers our central question: how many dimensions of an LLM representation are needed to recover essentially all of its linear neural predictivity. We expected this number to be large: hundreds of principal components are needed to capture most of the variance of the embeddings, and thousands for the neural responses. If LLMs are good models of human language processing, the correspondence between the two might be similarly high-dimensional. Instead, roughly 10–20 dimensions suffice.
We compare RRR with PCA because PCA keeps the embeddings' directions of greatest variance without reference to the brain. If RRR saturated at the same rank as PCA, the brain-predictive subspace would simply be the dominant directions. That PCA needs 8–18 times as many dimensions shows that neural prediction depends on specific, non-dominant directions. Appendix A.3 tests this difference across models.
Changes: Introduction, paragraphs 3–4; Section 3.6.
Q5. Additional baselines
We do not propose a method to benchmark against state-of-the-art neural predictors; we ask how much of an LLM representation supports linear alignment with the brain. Linear encoding models are the standard way this alignment is measured e.g., Schrimpf et al., 2021; Goldstein et al., 2022; Caucheteux & King, 2022; Antonello et al., 2023, and we follow their procedures. Nonlinear models can predict neural activity more accurately but do not measure representational alignment in the same sense. Within this framing, the relevant comparisons are:
- PCA, the standard unsupervised reduction to k dimensions, which tests whether the brain-predictive directions are simply the dominant ones;
- controls: a randomly initialized GPT-2, static GloVe embeddings, and shuffled embeddings;
- six trained models, including recent large models such as Llama 3 8B, spanning autoregressive and masked objectives. The effect holds for every one.
Q6. Dataset coverage
We agree that three datasets cannot cover all of human language comprehension. Within publicly available naturalistic datasets, however, our evaluation is broad:
- about 20 participants across three independent datasets (9 Podcast, 3 Cars 2, 8 fMRI);
- three recording techniques (ECoG, sEEG, fMRI) differing greatly in temporal and spatial resolution and SNR;
- three kinds of stimuli: a podcast, a movie, and narrative stories;
- six trained models, each at five layers.
The low-rank effect holds in every dataset, model, recording technique, and anatomical region we examined (Section 4.1). Subspaces also transfer to participants not used to estimate them (Section 4.2) and partially to entirely different datasets (Section 4.3).
Summary
We pointed explicitly to the code, pointed to the full implementation details, explained why target selection cannot bias our conclusions, described the failure cases the paper reports, and clarified the motivation for the saturation rank. Together with the controls added for other reviewers (see general comment), we believe these revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their assessment.
Response Part 1
===== COMMENT 1/2 ===== Response to Reviewer 3 (1/2)
We thank the reviewer for their feedback. Changes made in response are shown in blue and tagged [R3] in the revised PDF; our general comment summarizes the changes for all reviewers. We address the code in Q1 and the experimental concerns in Q2–Q6.
Q1. Source code
The full code was included with the original submission as a zip file in the supplementary material. As this is easy to miss, the Reproducibility statement now points to it explicitly. It includes a guide that walks through every step needed to reproduce our results from the publicly available datasets. The final version will link to a public repository.
Changes: Reproducibility statement.
Q2. Implementation details
We agree these were hard to find. The appendix describes them in full:
Changes: Appendices A.1–A.4.
Q3. Error analysis and failure cases
Selection leakage. We deliberately select targets broadly, keeping every site the full-embedding ridge model predicts significantly better than chance, because we want an upper bound on the dimensionality of the aligned subspace (Section 3.3). In iEEG, selection and evaluation share word folds; in fMRI, the test story is excluded from selection. Selection can therefore raise the absolute performance of selected iEEG targets, but our conclusions never rely on absolute performance:
We added this reasoning to Appendix A.4.
Failure cases. The paper reports several conditions where the effect weakens or disappears:
Changes: New paragraph in Appendix A.4; Appendices A.11, A.12, A.14.