acceptodds
Under review as a conference paper at ICLR 2027

SAIL: Sparse Autoencoders for Interpretable Alignment of Human Vision and Multimodal Large Language Models

Abstract

Alignment between human high-level visual representations and those of multimodal large language models (MLLMs) offers a quantitative framework for understanding information processing in human vision. However, similarities between model and brain representations do not directly reveal the semantic content underlying their correspondence, as this content is difficult to isolate in entangled MLLM representations. Here, we introduce SAIL, an interpretable framework that formulates voxel interpretation as a multi-unit sparse semantic correspondence problem between MLLMs and human vision. First, we train unified cross-layer sparse autoencoders (SAEs) on MLLM representations of Natural Scenes Dataset images and captions, and align each brain voxel with multiple SAE units. Second, we quantify semantic similarity between their top-response image sets to map semantic alignment across the cortex. We project multiple semantic labels back onto the cortex to interpret the content of this alignment. Finally, we assess these correspondences using category responses measured in independent fLoc experiments. Compared with SAEs trained independently at each layer, SAIL exhibits lower redundancy in SAE unit response profiles and higher semantic alignment. We find that semantic alignment is organized along the human ventral visual pathway across the models and subjects examined. The projected labels reveal semantic preferences within canonical visual regions, while fLoc experiments show agreement between predicted and measured category response patterns. Exploratory analyses further reveal cortical distributions of action-related semantics consistent with established functional accounts of lateral occipitotemporal and posterior parietal cortex. Overall, SAIL provides an interpretable account of the semantic correspondences between MLLM representations and human visual responses.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Sparse Autoencoders, Multimodal Large Language Models, Interpretability, Brain-Model Alignment, Semantic Alignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AukBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

SAIL studies interpretable correspondences between multimodal language models and human visual cortex. It pools and reduces features from multiple language layers, trains one sparse autoencoder on their concatenation, and matches each voxel to several units using training-response cosine similarity. It evaluates preferred-image collections with SEED and optimal transport, assigns unit labels through caption-derived candidate prompts, and visualizes cortical semantics. Experiments cover four models and eight NSD subjects, with layerwise SAE comparisons, searchlight RSA, independent fLoc category responses, and an auxiliary CLIP-based encoding-model evaluation.

Critical strengths
  1. The evaluation covers four models and eight subjects, fits preprocessing and SAE training within training folds, and includes controls for candidate-unit counts in the redundancy analysis and supplementary RSA (Appendices A, D, and F.5). These choices provide meaningful support for the reported response-profile differences.

  2. The fLoc analysis tests fixed NSD-derived correspondences against category responses from an independent experiment (Section 6 and Appendix I). This is useful evidence that the correspondence is informative beyond the NSD stimulus set, although it does not isolate the benefit of the proposed components.

Critical weaknesses

1.The evidence does not establish that the additional matched units provide more informative semantic explanations of individual voxels.

Section 3 measures redundancy across the candidate dictionary using the maximum similarity between unit response profiles. It does not evaluate semantic redundancy or coverage among the units actually selected for each voxel. Equation (3) selects these units by separate similarities to the voxel, without testing whether they contribute distinct information.

The labeling procedure also leaves a substantial interpretive gap. Appendix H extracts candidates from the captions of one maximally activating image, keeps that image fixed, and selects the candidate producing the largest target-unit activation for the complete image-text input. This tests candidate prompts on one selected stimulus; it does not establish that the assigned label predicts the unit's or voxel's preference across other images, independently of co-occurring content. The authors acknowledge this ambiguity in the Discussion.

Neither the SEED/OT image-collection score nor the auxiliary predicted ROI responses in Appendix J directly evaluate label validity or the additional information supplied by multiple labels. The fLoc measurements are valuable, but Table 1 and Appendix I report only SAIL, without layerwise SAE or single-unit comparisons. Their category-wise cosine similarities also lack a cross-category comparison or a control for category-shared spatial response components. Figure 5a supplies a body/place contrast visualization, but does not establish a quantitative component advantage across subjects. Thus, the results support response correspondence and broad category organization, while the central claim of better voxel-level multi-semantic interpretation remains unverified. Direct validation of distinct label preferences and their incremental information is necessary for that claim.

2.The comparisons do not sufficiently isolate the contribution of the proposed representation relative to existing interpretable mapping approaches.

Multi-semantic voxel mapping is already addressed by CLIP-MSM [1] and BrainLMM [3], while SAEs-BrainMap [2] establishes SAE-based brain alignment. SAIL therefore needs to demonstrate what its unified dictionary and multiple matches add to these capabilities. Figures 2 and 3 provide evidence in favor of the proposed integration, but the remaining controls do not resolve this issue.

Appendix F constructs RAW/PCA/ICA RDMs directly from image features, whereas SAIL and SAEs-BrainMap RDMs are constructed from predicted voxel responses using brain-derived correspondences. This changes the brain-response readout as well as the representation. Matching candidate dimensionality in Appendix F.5 does not remove that difference; its RAW control also uses the final language layer, whereas SAIL integrates multiple layers. Consequently, those results do not isolate the benefit of sparse decomposition over the same input information with the same fitted readout.

Similarly, the vision-tower and projector comparisons in Section 4 change the feature-extraction stage as well as access to captions. They do not substitute for image-only, caption-only, or shuffled-caption controls at the same language layers. The benefit attributed to multimodal representations consequently remains confounded with these changes.

The auxiliary validation in Appendix J relies on one CLIP ViT-based voxelwise encoding model. Appendix J does not specify its brain-response readout architecture, and no alternative encoding model is reported. It therefore remains unclear whether the preferred-image ranking is robust to the choice of response predictor. This limits the robustness of that model-based validation, although the independent fLoc measurements remain valid evidence.

Finally, candidate-count matching does not establish comparable reconstruction fidelity and activation sparsity, neither of which is reported alongside the redundancy comparison. There is also no direct comparison with the cited multi-semantic mapping methods on interpretation quality. These omissions limit the conclusion to improvement over the particular reported configurations, rather than establishing a substantive advance in reliable multi-semantic explanation.

References

[1] Yang, G., Xue, M., Mao, Z., Zheng, H., Xu, J., Sheng, D., … Li, X. (2025). CLIP-MSM: A multi-semantic mapping brain representation for human high-level visual cortex. Proceedings of the AAAI Conference on Artificial Intelligence, 39(9), 9184-9192. https://doi.org/10.1609/aaai.v39i9.32994

[2] Mao, Z., Xu, J., Pan, W., Xue, M., Jin, Y., & Yang, G. (2026). SAEs-BrainMap: Unveiling the emergence of specialized concepts in deep models via brain alignment. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 306, pp. 86540-86565). https://proceedings.mlr.press/v306/mao26h.html

[3] Gao, T., Xue, M., Zheng, H., Lv, S., Xu, J., Sheng, D., … Yang, G. (2026). BrainLMM: A label-free framework for mapping multi-semantic representation in the human visual cortex. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4176-4184. https://doi.org/10.1609/aaai.v40i6.42413

Questions for authors

Please refer to Critical Weaknesses

4 comments

We sincerely thank the reviewer for the constructive feedback.

W1:

(1) We evaluated voxel-level label semantic redundancy. As our labels primarily consist of individual words or short phrases, we used pretrained fastText word vectors and the same redundancy metric as in the main text. With matched label counts per voxel, SAIL consistently achieves lower semantic redundancy than SAEs-BrainMap (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/1.pdf).

(2) We tested whether the labels predict brain responses to other images and whether additional labels improve prediction across four models and eight subjects. We fixed the voxel–unit matches established on the training set and split the held-out images into separate sets for label generation and evaluation. For each evaluation image, we checked whether each label appeared in its corresponding COCO captions, assigning a binary value of 1 if the label was mentioned and 0 otherwise. We used the first one to five labels to predict measured voxel responses through ridge regression with five-fold cross-validation. For the five-label comparison, we retained the first label and replaced the other four with frequency-matched random labels, repeating this control 20 times. Prediction R² generally increased with more labels, and five labels outperformed the random control (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/2.pdf). These results support the ability of the label sets to predict brain responses to other images and suggest that additional labels provide predictive information beyond a single label.

(3) Figure 4 projects the text embeddings of the labels into a UMAP semantic space and maps the resulting multi-label representations onto the cortical surface using color. The spatial distributions of person, scene, and food related semantics correspond to established category selective regions, providing evidence for label validity at the regional level.

(4) We added SAIL-Single and layerwise SAE controls across four MLLMs and eight subjects. Both controls used the same fLoc stimuli and cosine similarity metric (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/3.pdf).

(5) We conducted two additional analyses using Qwen across 8 subjects. First, we compared the predicted response for each category with the measured responses for all eight categories, constructing a cross-category similarity matrix (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/4.pdf). Second, we subtracted the mean response across all eight categories at each voxel, separately for predicted and measured responses, before recomputing their similarity. This removes the shared spatial component represented by the mean response map across categories. Positive correspondence persisted across categories after removing the shared spatial component (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/5.pdf).

(6) Using Qwen, we assessed how well each method reproduced the voxelwise body–place response contrast. For both predicted and measured responses, we calculated each voxel’s mean response to body images minus its mean response to place images, then computed the cosine similarity between the two contrast vectors across voxels. SAIL achieved higher similarity than both baselines in all eight subjects (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/6.pdf).

W2:

(1) We compared SAIL, BrainLMM, and CLIP-MSM on Subject S5 using Qwen. We identified predicted regions based on the semantic similarity between voxel labels and the corresponding categories, then measured their spatial overlap with the reference ROIs. SAIL achieved higher overlap than BrainLMM and CLIP-MSM across all five ROIs, indicating greater spatial consistency between its label descriptions and the semantic functions of these regions (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/7.pdf).

(2) RSA compares brain–model correspondence across the complete pipelines of the evaluated methods, rather than isolating the contribution of sparse decomposition.

(3) We conducted additional image-only (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/1.pdf) and fixed-text (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/2.pdf) control experiments using SAIL at the same language layers across four models and eight subjects.

(4) Appendix J tests whether highly activating images selected by different methods elicit higher predicted responses in the target ROIs, rather than evaluating ranking stability across encoding models.

(5) We added reconstruction fidelity and activation sparsity results for Qwen across 8 subjects. Compared with the layerwise SAE baseline, SAIL achieved higher reconstruction quality with a lower proportion of active units (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/8.pdf).

Once again, we appreciate the reviewer's thoughtful comments.

Thanks for the additional experiments. The new fLoc comparisons and label controls are useful, but the label-based prediction R² remains very low, and I am still not convinced that the additional labels provide distinct and reliable voxel-level semantic explanations rather than capturing correlated concepts. I also share MVa4's broader concern about the semantic validity of voxel–unit matching, while the current comparisons still do not clearly isolate the contribution of the unified cross-layer SAE. These are my main concerns about the paper's novelty and interpretability, so I will keep my score at 2.

Thank you for further explaining your remaining concerns. We apologize for not sufficiently explaining the design and scope of the previous label prediction experiment. To address this, we conducted a new experiment (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/3/10.pdf), described below .

1. Label prediction and additional semantic information

The original experiment represented each concept only by its presence or absence, without capturing its prominence within an image. When two concepts co-occurred, both were encoded as 1, even if one was central and the other appeared only in the background. Although the readout could learn different weights for different labels, these fixed weights could not recover changes in concept prominence across images. The readout also relied on linear combinations of at most five label features to predict continuous voxel responses. These limitations may contribute to the relatively low R² values.

To further assess label validity and the incremental information provided by multiple labels, we conducted a label-based image retrieval experiment across four models and eight subjects, without fitting a brain response readout. We retained the voxel–unit matches established on the training set and their associated labels, and evaluated images not used for label generation. For each voxel, we used the first kk labels, where k=1,…,5k = 1, \ldots, 5, and normalized the original matching scores of their corresponding units within the selected set so that the weights summed to one. We then computed cosine similarities between the CLIP text embeddings of the labels and the CLIP image embeddings of the test images, combined these similarities using the normalized weights, and retrieved the ten highest-scoring images.

We evaluated the measured voxel responses elicited by these images. Each response was standardized by subtracting the voxel’s mean response across all test images and dividing by its response standard deviation. We averaged these standardized responses over the ten retrieved images and then across the evaluated voxels. As a control, we retained the first actual label and replaced the remaining labels with frequency-matched random labels, applying the same evaluation procedure and averaging over 20 repetitions.

This experiment directly tests whether multiple labels more effectively retrieve other images that elicit higher voxel responses than a single label or random label extensions, thereby assessing whether additional labels provide useful semantic information beyond a single label.

2. Meaning and semantic validity of voxel–unit matching

Equation (3) compares the responses of one SAE unit and one voxel across the same images, rather than directly comparing two independently learned feature spaces. Under our encoder formulation, the -th row of contains the encoding weights for unit . Each image–caption representation produces one scalar activation for this unit, while the voxel provides one response value for the corresponding image [1].

For example, an SAE unit might have activations [0.2,0.8,0.1][0.2, 0.8, 0.1], while a voxel has responses [0.3,0.9,0.2][0.3, 0.9, 0.2] across the same three images. Equation (3) computes cosine similarity between these vectors, whose corresponding entries refer to the same image. Their correspondence is determined by image identity, not by the ordering or semantic meanings of SAE units and voxels.

This explains why the response profiles can be compared without prior feature-space alignment. We recognize, however, that response similarity alone does not establish semantic validity. The conditional label analysis and independent fLoc experiments provide complementary evidence concerning semantic interpretation and functional response correspondence.

3. Contribution of the unified cross-layer SAE

We will distinguish the roles of our comparisons more clearly. Comparing SAIL-Single with the layerwise single-unit baseline examines dictionary construction while retaining single-unit matching. Comparing SAIL with SAIL-Single examines multiple matching within the same unified dictionary. These comparisons provide evidence concerning the separate design choices, but we will not present them as eliminating every difference in capacity or training configuration.

We will incorporate these clarifications, additional experiments, and their scope into the paper so that all reviewers can assess the updated evidence. We appreciate the reviewer’s careful consideration and constructive feedback.

Reference

[1] Allen, E. J., St-Yves, G., Wu, Y., Breedlove, J. L., Prince, J. S., Dowdle, L. T., Nau, M., Caron, B., Pestilli, F., Charest, I., et al. (2022). A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25(1), 116–126.

Thank you again for your thoughtful feedback. We have further addressed your concerns and updated the paper accordingly. We would greatly appreciate it if you could take another look at the revised results. Thank you for your time and consideration!

Reviewer_MVa4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper introduces SAIL, an interpretable framework for identifying and analyzing semantic correspondences between multimodal large language models (MLLMs) and human visual cortex. By aligning brain voxels with sparse autoencoder units and mapping their shared semantic content, SAIL reveals interpretable semantic organization along the ventral visual pathway. Its findings are further supported by independent fLoc experiments, demonstrating agreement between predicted and measured category responses.

Critical strengths
  1. This paper investigates the alignment between MLLM representations and human brain representations, which is an interesting research problem.
  2. The paper leverages SAEs to facilitate cross-modal alignment between MLLM representations and human visual responses.
  3. The paper conducts experiments across multiple MLLMs and provides informative visualizations to analyze the resulting semantic correspondences.
Critical weaknesses
  1. The paper lacks sufficient discussion and comparisons with closely related methods for multi-modal SAE, such as VL-SAE [1] and U-SAE [2]. It is unclear what advantages the proposed method offers over these approaches in processing cross-modal representations.

  2. As shown in Figure 1(b), the authors directly compute cosine similarity between SAE hidden activations and voxel responses. Why is this comparison meaningful? The SAE is not explicitly trained to align its hidden activations with voxel responses, so it is unclear whether the two representations can be meaningfully compared in this way.

  3. Why do the authors choose MLLMs rather than vision-only models? The role of the language modality remains unclear, as the visualizations and analyses appear to rely entirely on visual inputs. A comparison with vision-only models would help clarify the benefits of using MLLMs.

  4. Although aligning MLLM representations with human visual representations is an interesting research direction, the potential applications remain unclear. For example, could such alignment enable mutual improvements between MLLMs and human vision research? The paper would benefit from a discussion of potential future applications and research directions.

[1] VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set. Advances in Neural Information Processing Systems (NeurIPS) 2025.

[2] Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment. International Conference on Machine Learning (ICML) 2025.

Questions for authors

Overall, although the paper investigates an interesting question, the lack of comparisons with existing methods and the implicit assumption that SAE hidden activations and voxel representations can be directly compared raise concerns about the novelty and reliability of the proposed approach.

5 comments

We sincerely thank the reviewer for the thoughtful and constructive feedback.

W1:Comparison with Existing SAE Methods

We thank the reviewer for suggesting these additional baselines to strengthen the comparative evaluation. We have added VL-SAE and U-SAE baselines across four MLLMs and eight subjects. For VL-SAE, we matched voxels to SAE units in its shared latent concept space. For U-SAE, we adapted the original setting across models to different layers within the same MLLM, learning shared concepts through reconstruction across layers. All three methods used the same number of matched units and semantic evaluation procedure. Across the models, SAIL achieved higher mean semantic similarity than both baselines (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/2/1.pdf). Furthermore, SAIL achieved higher score in Selective ROIs than in the whole cortex, while U-SAE and VL-SAE showed similar scores across both regions. This suggests that SAIL not only improves overall alignment but also captures stronger semantic correspondences in category-selective regions, consistent with the functional organization of the human visual cortex.

W2:Rationale for voxel–unit Matching

Although the SAE is trained without brain supervision, both SAE activations and voxel responses vary with image content, providing a basis for matching their responses to shared stimuli. Related work has also explored correspondences between SAE units and brain voxels directly using cosine similarity, without retraining the model to predict brain responses [1]. Previous research has suggested that fitting model features to voxel responses may reduce representational differences between models [2]. Directly aligning model activations with voxel responses may therefore better preserve distinctions between models. Accordingly, we compared responses to shared visual stimuli without fitting an additional voxel-wise linear readout, aiming to minimize the possibility that flexible fitting could obscure differences in semantic alignment across MLLMs. We will clarify this motivation in the paper.

W3:Motivation for Selecting MLLMs and the Role of Language

We chose MLLMs because they integrate visual information with linguistic semantics, providing a way to interpret correspondences between model representations and responses in higher visual cortex at the semantic level. SAIL-Vision uses the visual encoder of each MLLM as a control. Both LLaVA models use CLIP ViT-L/14; DeepSeek-VL-7B uses a hybrid visual encoder combining SigLIP-L and SAM-B; and Qwen2.5-VL-7B uses a ViT supporting dynamic resolution. These encoders receive only images, and we extract representations from their individual layers rather than from the language layers. Figure 3b shows that SAIL achieves higher semantic alignment than this visual encoder control. To examine the role of language modality, we added a SAIL (image-only) control across four MLLMs and eight subjects, removing captions and text prompts while keeping other settings unchanged. Mean semantic similarity changed by less than 0.0019 in the whole cortex and 0.0013 in Selective ROIs (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/1.pdf). These results suggest that alignment does not primarily depend on inference-time text. Consistent with prior work [3] showing that vision-language pretrained image encoders can better predict responses in high-level visual regions using images alone, language may instead contribute through semantic knowledge acquired during pretraining. Captions also provide candidate labels for matched units. We will include these results and clarify these roles in the paper.

W4: Potential Applications and Future Directions

For human vision research, the semantic labels and associated images provided by SAIL could help formulate hypotheses about voxel preferences and guide controlled stimulus experiments, building on prior work on model-guided visual experiments. For model development, these correspondences could reveal similarities and differences between models and the brain, informing subsequent improvements. We will discuss these potential applications and future directions in the paper.

Once again, we appreciate the reviewer's thoughtful comments.

[1] Ziming Mao, Jia Xu, Wenxuan Pan, Mufan Xue, Yaochu Jin, and Guoyuan Yang. SAEs-BrainMap: Unveiling the Emergence of Specialized Concepts in Deep Models via Brain Alignment. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026.

[2] Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. A large-scale examination of inductive biases shaping high-level visual representation in brains and machines. Nature Communications, 15, 9383, 2024.

[3] Wang, A. Y., Kay, K., Naselaris, T., Tarr, M. J., & Wehbe, L. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nature Machine Intelligence, 5(12), 1415–1426, 2023.

Thank you to the authors for their response. First, I encourage the authors to incorporate the rebuttal clarifications and promised improvements into the revised manuscript. One advantage of the ICLR review process is that authors can update their submissions, allowing all reviewers to assess the actual improvements to the paper.

Second, regarding the response to W2, is it valid to directly compute cosine similarity between two independently learned representation spaces? For example, an image of a car might activate the first 10 dimensions of the SAE representation, while the corresponding voxel representation might have no activation in those dimensions, despite encoding the same semantic concept. Given that the semantic meaning and ordering of SAE hidden neurons are not explicitly controlled, how do the authors ensure that cosine similarity across these independently learned spaces reflects meaningful semantic similarity?

Thank you for the suggestion. We will incorporate the clarifications and improvements from our rebuttal into the revised manuscript so that all reviewers can evaluate the updated paper.

Thank you for the follow-up question. We compare the responses of a single SAE unit and a single voxel across the same images, rather than directly comparing representations in two independently learned feature spaces. Under the encoder formulation in the paper, the jj-th row of WencW_{\mathrm{enc}} contains the encoding weights for SAE unit jj. For each image–caption pair, its input representation x\mathbf{x} is processed by this unit to produce a single scalar activation. Likewise, a voxel provides a single response value for the corresponding image [1].

For example, for three images and their associated captions, an SAE unit might have activations [0.2,0.8,0.1][0.2, 0.8, 0.1], while a voxel has responses [0.3,0.9,0.2][0.3, 0.9, 0.2] to those same three images. Equation (3) computes cosine similarity between these two vectors, with each corresponding entry referring to the same image. Thus, correspondence between vector entries is determined by the images, not by the indices, ordering, or semantic meanings of SAE units and voxels. This computation measures agreement between their response profiles across the same images without requiring the two feature spaces to be aligned beforehand. We will clarify this computation in the Methods.

We thank the reviewer again for the constructive feedback and helpful suggestions, which have helped us improve the clarity of our presentation.

[1] Allen, E. J., St-Yves, G., Wu, Y., Breedlove, J. L., Prince, J. S., Dowdle, L. T., Nau, M., Caron, B., Pestilli, F., Charest, I., et al. (2022). A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25(1), 116–126.

Reviewer_BwZDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose SAIL, a sparse autoencoder (SAE)-based framework that aims to make the alignment between human vision and multimodal large language models (MLLMs) semantically interpretable at the voxel level. The framework trains a unified cross-layer SAE on MLLM representations of NSD images and captions, matches each voxel to multiple SAE units, and computes a semantic alignment score by comparing the voxel's preferred images with those of its matched units. The method is evaluated with several metrics and analyses, including unit redundancy, semantic alignment, searchlight RSA, cortical label maps, and fLoc category response reconstruction, across eight subjects and four MLLMs.

Critical strengths
  1. The paper is generally well written and easy to follow. The multi-unit sparse semantic correspondence is a sensible improvement over single-unit voxel labelling: describing each voxel with several matched sparse units, together with a redundancy-aware unified dictionary, is a nice idea. The qualitative results are well presented.

  2. The paper conducts a careful and broad neuroscience evaluation, with held-out images, an independent fLoc experiment, eight subjects, and multiple MLLMs. The results are consistent across these settings, and it is good to see that the resulting cortical maps reproduce known visual regions (EBA/FFA for bodies and faces, RSC/PPA/OPA for scenes, and the food-selective region).

Critical weaknesses
  1. My main concern is that the multimodal interpretation is confounded by the human-written captions, which act as a text-based "middle step" between the visual input and the corresponding cortical responses. The MLLM receives the COCO captions as input, the unit labels are derived from the same captions, and the caption term of the SEED evaluator uses GIT captions, which are strongly correlated with the COCO captions. The advantage over SAIL-Vision / SAIL-Projector may therefore reflect injected human annotation rather than better MLLM representations. Since the subjects never saw these captions, I am not convinced that the paper's main claim, i.e., MLLM–human-vision alignment, is sufficiently supported.

  2. Although the idea is interesting, the method itself is not novel: a single SAE over concatenated multi-layer activations is closely related to existing cross-layer approaches (e.g., crosscoders), and the remaining components (cosine top-K matching, SEED, optimal transport, etc.) are existing tools. The method is therefore largely a combination of existing tools, and the main contributions lie in the neuroscientific descriptions and results. For this reason, I think this work may be a better fit for a computational neuroscience venue than for ICLR.

Questions for authors

Please respond to the weaknesses above. In addition, I have the following questions:

  1. For SAIL-Single and SAEs-BrainMap, how is the K-image model-side set constructed: the top-K images of a single unit, or the top-1 images of K units? Does SAEs-BrainMap also use K matched units from the pooled layer-wise dictionary?

  2. What text input accompanies the MLLM when processing the fLoc stimuli (which have no captions), and when computing alignment for SAIL-Vision / SAIL-Projector? How do the results change if the COCO caption is replaced by a fixed generic prompt for all images?

  3. How sensitive are the results to K and λ?

4 comments

We thank the reviewer for the thoughtful and constructive feedback.

W1: Image-only semantic alignment

We thank the reviewer for raising this important point. To address it, we added a SAIL (image-only) control across four MLLMs and eight subjects, removing captions and additional text prompts while keeping the experimental setup consistent. Without captions, mean semantic similarity changed by less than 0.0019 across the whole cortex and 0.0013 in selective ROIs for all four models (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/1.pdf). These results suggest that semantic alignment does not primarily depend on textual descriptions provided at inference. Prior work [1] showed that vision-language pretrained image encoders can better predict responses in certain higher visual regions using images alone. Thus, language may contribute through semantic knowledge learned during pretraining, consistent with our findings. Additionally, captions provide candidate labels for matched units. We will include these results and clarify the role of captions in representation learning and semantic labeling.

W2: Methodological contribution and relevance

We thank the reviewer for this thoughtful comment. Recent findings of correspondence between large language model representations and human visual cortex offer an important perspective on semantic information in models and its relationship to human visual processing [2].However, representational similarity alone does not directly reveal the semantic content underlying these correspondences. SAIL addresses this gap by formulating voxel interpretation as a multi-unit sparse semantic correspondence problem between MLLMs and human vision. It learns sparse features through unified cross-layer SAE training, characterizes voxel preferences through multiple unit matches, and maps the resulting semantic descriptions onto the cortex. Compared with layerwise SAEs, SAIL exhibits lower redundancy in unit response profiles and higher semantic alignment, extending the analysis from the degree of correspondence to its content and cortical organization. We acknowledge the connection to approaches such as crosscoders and do not claim each component as a new algorithm. The contribution lies in establishing interpretable semantic correspondences that provide concrete evidence for understanding MLLM representations and their relationship to human visual cortex. We will clarify this contribution and its connection to prior work in the paper.

Q1: Clarification of Method Configurations

For both SAIL-Single and SAEs-BrainMap, the model-side image collection comprises the top-K images of the single best-matching unit. Comparing SAIL with SAIL-Single evaluates the benefit of multiple versus single unit matching, while comparing SAIL with SAEs-BrainMap assesses the overall performance of our method against an established baseline. SAEs-BrainMap uses the units from the pooled layer-wise dictionary.

Q2: Limited influence of caption inputs

For fLoc stimuli, we use blank text without image descriptions or category prompts, retaining only the input formatting required by the model interface. For SAIL-Vision and SAIL-Projector, we retain the image and COCO caption inputs to the full model but extract outputs from the vision encoder and projection module, respectively. We also replaced the captions with the fixed prompt “Describe this image.” and reran the experiment across four models and eight subjects. Relative to the original setting, mean semantic similarity changed by less than 0.0015 across the whole cortex and 0.0005 in selective ROIs for every model (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/2.pdf). Thus, COCO captions had little effect on semantic alignment.

Q3: Sensitivity to K and λ

We evaluated sensitivity to K and λ using Qwen2.5-VL-7B-Instruct for S5. With λ fixed at 1.25, semantic similarity increased as K rose from 1 to 15, with progressively smaller gains at larger K. With K fixed at 15, varying λ from 0.3125 to 5 yielded score ranges of 0.00134 for the whole cortex and 0.00157 for Selective ROIs. Within this setting, semantic similarity initially increases with K and then gradually plateaus at larger K values, while remaining relatively stable across the tested λ values (https://anonymous.4open.science/r/iclr_reubttal-BAB0/r/1/3.pdf). We will include the experimental settings and results in the paper.

Once again, we appreciate the reviewer's thoughtful comments.

[1] Wang, A. Y., Kay, K., Naselaris, T., Tarr, M. J., & Wehbe, L. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nature Machine Intelligence, 5(12), 1415–1426, 2023.

[2] Doerig, A., Kietzmann, T. C., Allen, E., Wu, Y., Naselaris, T., Kay, K., & Charest, I. High-level visual representations in the human brain are aligned with large language models. Nature Machine Intelligence, 7(8), 1220–1234, 2025.

Dear Reviewer BwZD,

Can you please take a look at the authors' response and provide your further input? Thank you.

AC.

Thank you for the authors' response. Most of my questions and concerns have been adequately addressed in the rebuttal. Therefore, I would like to increase my rating to 3, but it is close to a borderline rating from my point of view.

Also, after reviewing the other reviewers' comments, I share many of their concerns. I believe the paper would benefit from addressing these issues, and I would encourage the authors to carefully consider and address all the outstanding points to further strengthen the submission.

Thank you for reviewing our responses and reconsidering your assessment! We have carefully considered and addressed all remaining concerns. We have also incorporated additional experiments and methodological clarifications into the paper. Your feedback has helped us improve our work and its presentation. Once again, we sincerely thank you for your thoughtful consideration and constructive feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.