- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This empirical study examines how visual representations change across the language decoders of 13 open-weight vision–language models (VLMs). The authors extract hidden states at image-token positions, apply mean pooling, and use representational similarity analysis (RSA) to compare the resulting representations with neural representational dissimilarity matrices (RDMs) from seven visual cortical streams in NSD. The study presents interesting findings through a carefully designed analysis informed by established practices in neural data analysis. However, several issues concerning effect-size interpretation, anatomical specificity, representation readout, and cross-dataset consistency require clarification.
Critical strengths
-
Insightful experimental design and careful statistical analysis. The authors use subjects as the unit of inference, avoiding pseudo-replication from treating model–subject pairs as independent observations. Reporting both checkpoint-weighted and lineage-balanced summaries also addresses unequal representation of model families. The bootstrap analyses are particularly valuable for an empirical study whose conclusions depend heavily on statistical reliability.
-
Convincing diagnosis of rogue dimensions. The energy share of the top three channels increases from 23% to 83% and falls to 34% after normalization. Complementary controls—including feature standardization, top-three-channel removal, and rank transformation—strengthen this analysis. Depth-matched, randomly initialized decoders further support the interpretation that the observed patterns depend on learned weights.
Critical weaknesses
-
The practical magnitude of the effects is unclear. The main effects are approximately 0.03–0.05 in units normalized by the lower noise ceiling, but the main text does not provide absolute noise ceilings, alignment at llm.00, or alignment at the visual encoder output. These references are necessary to judge whether the changes are substantial relative to baseline alignment. For example, Figure 2 appears to show limited alignment for most models, with intermediate improvements concentrated in two Qwen models before declining again. Please report raw RSA scores alongside normalized changes and relevant baselines.
-
The emphasis on the lateral stream exceeds the evidence for anatomical specificity. Section 4.1, Table 2, and Figure 11 show similar changes in other higher-order streams. Supporting a specifically lateral trajectory would require analyses such as partial RSA or variance partitioning across ROIs. Otherwise, the title and abstract should describe the broader early-to-higher-order contrast.
-
Mean pooling may obscure spatial information relevant to early visual cortex. Early visual cortex is retinotopically organized, whereas averaging image tokens discards their spatial arrangement. The late-layer decline in early-cortex alignment could therefore partly reflect a depth-dependent interaction between pooling and spatial information, rather than a general loss of alignment in the underlying representations. A spatially preserving readout, such as patch-position concatenation or spatially resolved RDMs, would help distinguish these explanations. Additionally, trimming the top 1% of tokens by norm does not disentangle channel-concentrated activations from activations concentrated at attention-sink positions; these mechanisms warrant separate controls.
-
Statistical resolution is limited. None of the 26 checkpoint-level tests survive Holm correction. The panel-level adjusted (p=0.0156) also appears to reach the minimum attainable value for a two-sided exact Wilcoxon signed-rank test with eight subjects and correction across two tests, highlighting the coarse resolution of the inference. The authors acknowledge that their joint bootstrap does not implement the variance correction of Schütt et al. (2023). Given its availability in rsatoolbox, please explain why this procedure was not used and, preferably, report it as a robustness check.
-
The NSD–NOD differences need targeted analysis. Wider intervals on NOD are primarily attributed to its smaller stimulus set (120 versus 515 images), but the effects of stimulus count, signal quality, and other dataset differences are not separated. More importantly, standardization improves directional consistency on NSD—early-cortex endpoint decreases rise from 9/13 to 13/13 models—but reduces it on NOD, where only 6/10 models show decreases. Cross-dataset directional agreement also falls from 19/20 to 14/20 comparisons. This complicates the interpretation that standardization better reveals the relevant representational geometry. The absence of three models from NOD—PaliGemma2-10B, Idefics2-8B, and Qwen2-VL-7B—may further affect the comparison, as these models generally show pronounced early-cortex declines on NSD. Three analyses would help: (a) repeatedly subsample NSD to 120 images and assess interval widths and directional consistency; (b) report absolute ROI-level noise ceilings for both datasets; and (c) recompute NSD panel summaries using only the ten models shared with NOD.
Questions for authors
Please refer to the weakness.
2 comments
Response to Reviewer nLyM (2/2)
Response (2/2)
W4. Statistical resolution and the Schütt et al. (2023) bootstrap
We agree on both points. The panel is indeed the floor for an exact two-sided signed-rank test with and two tests. It says only that all eight participants agree; magnitude comes from the bootstrap intervals. Checkpoint-level tests are now labeled descriptive and panel summaries confirmatory (Secs. 3.4, 4.1).
Why it was not used. The inference plan (signed-rank tests plus an ordinary crossed bootstrap) was frozen before the corrected analysis, and we listed the missing variance correction as a limitation rather than change the test after seeing results.
Result. We now report it (rsatoolbox eval_dual_bootstrap with its correction, participants and images resampled jointly, each checkpoint's readouts as fixed models, pairwise tests, 200 bootstrap samples; Appendix B.7, Table 14):
| step (11 checkpoints) | raw: claimed direction / / Holm | standardized |
|---|---|---|
| lateral llm.00→0.75 | 8 / 7 / 6 | 9 / 9 / 4 |
| early 0.75→ | 11 / 6 / 3 | 10 / 7 / 4 |
(Holm over 22 checkpoint × step tests per metric.) With this test, checkpoint-level effects become resolvable at . The early decline is never significant in the opposite direction. The lateral step is so in one checkpoint per metric: PaliGemma2-3B-mix (raw, Holm-significant) and Qwen2.5-VL-3B (standardized, not Holm-significant). The revised text reports this heterogeneity.
W5. NSD–NOD differences
(a) NSD subsampled to 120 images (1,000 random subsets, with brain RDMs, ceilings and model RDMs recomputed; Table 9):
- Interval width. The panel's early last-quarter decline ranges across subsets from to (2.5–97.5%, raw). This range is as wide as NOD's participant interval ( to ), but it never crosses zero.
- Directional consistency. Subsets reproduce the full-NSD checkpoint signs of this step in a median of 9/10 checkpoints (95%: 7–10). NOD matches NSD in 6/10 under standardization, below that range.
- Ceilings. NSD's lower ceiling is unchanged at 120 images (early 0.50 [0.46, 0.54]).
Image count therefore does not explain the NOD differences, and Sec. 4.2 no longer attributes them to the smaller stimulus set.
(b) Absolute ceilings (lower, NSD vs NOD; Table 3): early 0.50 vs 0.13, lateral 0.46 vs 0.16, ventral 0.71 vs 0.25, parietal 0.36 vs 0.03. NOD's reliability is 3–4× lower in every stream. Together with its acquisition differences (scanner, field of view, preprocessing), this is the likely source of the NSD–NOD differences. Two datasets cannot separate these factors, and the text says so.
(c) NSD restricted to the 10 shared checkpoints (Tables 6–7) gives the same picture as the full panel. Raw: early endpoint 7/10 negative, []; lateral 8/10 positive, ; last-quarter early decline 10/10, all 8 participants. Under standardization the early endpoint is negative in 10/10 on NSD but 6/10 on NOD for the same checkpoints. The three missing models therefore do not cause the divergence.
Standardization. We agree this complicates any claim that standardization "reveals" the right geometry, and we do not make it. Standardization is a control for channel scale. We claim only patterns that survive all four feature treatments, and recommend reporting a scale-invariant metric beside raw RSA, not instead of it. Under standardization the NOD last-quarter early step remains negative at the participant level ( [], 8/9), but its checkpoint signs agree with NSD in only 6/10. We report this as a genuine dataset difference, and Sec. 4.4 states that NOD does not replicate the checkpoint-level pattern.
We thank the reviewer again; these analyses sharpened the claims, especially on regional specificity, and we are happy to provide further detail.
Response to Reviewer nLyM (1/2)
Response (1/2)
We thank the reviewer for a detailed and constructive review. Every request led to a new analysis, computed from the released artifacts and caches; the revised PDF marks all changes in red (new Sec. 4.5 and Appendix B). Part 1 covers W1–W3, Part 2 covers W4–W5.
W1. Practical magnitude: raw RSA, noise ceilings and baselines
Sec. 4.1 now states the raw scale, and Appendix B.1 gives ceilings for all seven streams in both datasets (Table 3), raw ρ per stream (Table 4) and per checkpoint (Table 5). NSD panel means, 13 checkpoints:
In raw units the lateral rise to 0.75 depth is +0.019 and the early last-quarter decline −0.016, each about 16% of the llm.00 value. Peaks reach 22–41% of the lower ceiling. NOD's lower ceilings are 3–4× lower (early 0.13, lateral 0.16, parietal 0.03); Appendix B.1 now notes that ceiling-scaled NOD parietal values are therefore not interpretable.
Baselines.
On Fig. 2 and the Qwen models. The lateral rise is positive in 10/13 checkpoints; the largest rises are SmolVLM-2.2B, Qwen2-VL-2B, InternVL3-2B and SmolVLM-500M. Without the three Qwen checkpoints the panel still shows both patterns in 8/8 participants: lateral rise +0.035 raw / +0.052 standardized, early decline −0.024 / −0.014. The standardized early decline is about a third as large without Qwen, so the Qwen checkpoints do carry much of its size.
W2. Lateral specificity
We agree. Partial Spearman correlations per participant (model RDM vs one region's RDM, controlling for other regions' RDMs; 11 checkpoints; Sec. 4.5, Table 12):
The rise is shared with the ventral stream, so it is a higher-order occipitotemporal pattern, not a lateral one. The abstract, introduction and discussion now say so; the title does not name the lateral stream. The late decline, by contrast, is specific to early cortex.
W3. Mean pooling, retinotopy and attention sinks
Retinotopy. We agree that mean pooling discards spatial layout. Sec. 3.2 and the Limitations now state that our early-cortex scores concern image-level geometry, not retinotopy. A spatially resolved readout needs per-token activations, which our caches do not store; we will add it for the checkpoints with fixed token grids in the final version. The readout dependence the reviewer suspects is real in one form we could test: at the generation position the late early-cortex trend reverses (Sec. 4.5). The claim is now restricted to the mean over image positions.
Channels vs positions. These are now separated (NSD panel; raw mean pooling gives lateral rise +0.040, early decline −0.032):
Both patterns survive every control. Trimming selects positions by norm, not by attention mass; a sink definition based on attention mass is a better position control, and we will add it.
(continued in Part 2)