- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper analyzes the relationship between LLM hidden layer activations and the brain fMRI stimulus. The authors first uses regression to test whether linear combinations of LLM activations can predict brain stimulus, and then evaluated subspace orientation similarity between LLM and brain representations. Experiments show that while there exists some linear relation between LLM hidden layer activations and brain stimulus, the principal subspaces are not aligned.
Critical strengths
1.The paper presents a new perspective for the alignment of the LLM and brain representation. 2.Preliminary results support that some linear relation exists between LLM activation and brain stimulus patterns.
Critical weaknesses
1.Significance of conclusion we can derive from the experiments. The experiments show that the LLM representation can be a weak linear predictor of the brain stimulus, but the two subspaces are hardly aligned. However, a general linear mapping need not preserve leading principal-component subspaces. The paper does not sufficiently explain what additional scientific insight the observed dissociation provides beyond this mathematical fact, or how it advances our understanding of the relationship between LLM and cortical representations. 2.Method validity. The study tries to draw correlation between the brain fMRI stimulus from visual input and LLM activation representation from textual input. Since these two modalities hold different information, weak alignment may partly reflect differences in the information available to the two systems, rather than differences in representational organization. 3.Insufficient experiment results. The study includes only one LLM, one text encoder, and one fMRI dataset. Whether the finding generalizes to other models and datasets is not tested. 4.Writing. The authors could enhance the writing by clearly stating the problem, the hypothesis, main findings and conclusions. The current introduction and discussion part is not well-organized and hard to read.
Questions for authors
1.What scientific insight does the observed dissociation provide beyond the fact that linear mappings need not preserve dominant principal-component subspaces? How does this finding refine existing claims about brain–LLM correspondence? 2.Could the authors provide methods to distinguish representational differences from differences between image and caption information? 3.Please show results on more models and fMRI datasets to demonstrate that the claimed relationship consistently exists across models and datasets.
3 comments
W3.3/Q3.3: We thank the reviewer for this suggestion. We have extended our analysis to Qwen2.5-7B and two additional fMRI datasets, BOLD5000 (four participants) and THINGS-fMRI (three participants), with the results included in Appendices S5--S7 of the revised manuscript.
On NSD, Qwen reproduces the regional alignment pattern observed with Llama: EBA shows the strongest correspondence across all four subjects, followed by FBA and early visual cortex (mean chordal distances of 7.054, 7.174, and 7.269, respectively; random baseline 7.354). Unlike Llama, however, Qwen exhibits depth-dependent alignment, with EBA chordal distance decreasing from approximately 7.25 at layer 4 to 6.95 at layer 24.
The results also extend to BOLD5000 and THINGS-fMRI. On BOLD5000, Llama achieves positive held-out encoding accuracy (–), with LOC outperforming EarlyVis in all four participants. On THINGS-fMRI, encoding remains positive (–), and EBA shows stronger geometric alignment than the average early visual region across all three participants. In both datasets, brain-Llama chordal distances remain below the random-subspace baseline, although the regional ordering is less consistent than in NSD.
One relevant difference is that BOLD5000 and THINGS-fMRI were acquired at 3T, whereas NSD was acquired at 7T. Differences in acquisition resolution and signal reliability may contribute to the weaker correspondence observed in these datasets. However, stimulus sets, caption sources, and ROI definitions also differ, so we cannot isolate the contribution of scanner field strength.
W3.4: We thank our reviewer for highlighting areas where our presentation could be improved. We have revised the Introduction to clarify the central question of whether successful cortical prediction implies shared local representational geometry, motivate tangent-space alignment as a complement to linear encoding, and summarize our principal findings. We have also revised the Discussion to specify the dissociation between cortical encoding and geometric alignment, characterize within-system organization and cross-system correspondence, and evaluate robustness and generalizability using additional Appendix analyses.
Thank you for the response and the additional experiments. The newly added results address my concern on the experimental coverage. The revised presentation also makes the central findings clearer. However, two concerns remain.
-
The additional scientific insight relative to prior work remains insufficiently established. Doerig et al. (2025) supported brain–model correspondence using linear encoding and other strategies. The observed disassociation between linear predictability and local principal-subspace alignment does not clarify how the present findings refine that work.
-
The contribution of input-modality differences remains unresolved. Comparing Llama and CLIP-text on identical captions shows that model differences affect cortical correspondence. It does not determine how much of the weak brain–model alignment arises because the brain receives images while the models receive incomplete textual descriptions.
We appreciate our reviewer’s recognition that the paper presents a “new perspective for the alignment of the LLM and brain representation.” We address the weaknesses and questions in order below.
W3.1/Q3.1 We agree with our reviewer’s premise that a general linear map need not preserve principal-component subspaces. The question we address is empirical: given the mathematical freedom of linear encoding to use any direction in LLM feature space, how much do their dominant subspaces overlap? Our contribution with this paper is to evaluate the extent of this dissociation in brain-model alignment and propose geometric alignment as a complementary measure of representational correspondence. Previous work has demonstrated correspondence between language-model representations and cortical responses using linear encoding models, including Caucheteux and King (2022) for natural language comprehension and Doerig et al. (2025) for visually evoked cortical responses predicted from image-caption embeddings. While successful prediction indicates that model features are linearly informative about cortical activity, it does not establish that the two systems share similar patterns of variation across stimuli. Our approach addresses this distinction by comparing local tangent spaces in a shared stimulus-coordinate space (defined by the leading left singular vectors), so their alignment is invariant to orthogonal rotations of the original feature coordinates. This allows us to compare the dominant patterns of stimulus variation without requiring direct correspondence between individual model features and cortical voxels. Our framework also allows us to examine how local tangent-space orientations vary across model layers and along the cortical hierarchy, and we show systematic changes in subspace orientation with increasing separation in both systems.
We agree with the reviewer’s premise that a general linear map need not preserve principal-component subspaces. The question we address is empirical: given the mathematical freedom of linear encoding to use any direction in LLM feature space, do the brain and LLM in fact share dominant subspaces when the field’s standard ridge-encoding pipeline is applied? The literature has largely assumed yes, interpreting encoding scores geometrically (e.g., as evidence of ‘shared representations’). Conwell et al. (2024) have already shown this interpretation is problematic: 126 vision models reach OTC encoding scores within r=0.1 despite radical architectural differences. Our tangent-space diagnostic is a specifically stronger criterion that does distinguish brain-model pairs with and without shared principal subspaces. The observation that EBA (the strongest effect) as well as the other regions V1-hV4 sit below the random-subspace baseline is a non-trivial empirical claim: This graded dependence is the new scientific insight: the dissociation is not a universal fact of linear mappings but an ROI-specific finding about which cortical regions share geometry with caption-LLM representations.
W3.2/Q3.2 We thank our reviewer for highlighting this important methodological concern: differences in input modality may contribute to the observed weak brain-model alignment, as the brain processes visual scenes while the LLM receives textual descriptions. Our cross-modal comparison of brain and LLM representations is motivated by prior work, notably Doerig et al. (2025), which demonstrated that language-model representations of scene captions can predict cortical responses to the corresponding viewed images. However, we agree that even when images and their corresponding textual descriptions capture similar semantic content, weak geometric alignment may reflect differences in the information available to each system rather than differences in how they organize shared stimulus information.
The CLIP-text encoder comparison in Appendix S5 directly addresses this concern. Both Llama-3-8B and CLIP-text receive identical caption inputs for the same stimuli, so any difference between their cortical correspondence patterns cannot be attributed to the caption-vs-image modality asymmetry. We observe that CLIP-text shows depth-dependent alignment (monotone encoding, stronger cross-system chordal at deeper layers) while Llama shows layer-flat alignment. Because the two models process identical text inputs, the difference isolates representational-organization effects from modality content. To further strengthen this disentanglement, we have added a Qwen2.5-7B replication on S7. Qwen shows a similar layer-flat encoding pattern as Llama (although its layer dependence is more pronounced ) and a similar cross-system chordal signature, confirming the finding is a property of text-only autoregressive LLMs rather than Llama-specific.