- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether speech foundation models and the human brain share similar representational hierarchies during the transformation from sound to meaning. The authors introduce a multi-definition-to-target paradigm that separates semantic concepts from their surface linguistic forms. They examine eleven speech model checkpoints across different training objectives, together with intracranial recordings from 29 patients. Through layer-wise probing, representational geometry, temporal generalization, and feature-based encoding analyses, the paper identifies both similarities and differences between artificial and biological speech processing. The results suggest that lower-level linguistic representations exhibit more similar dynamics across the two systems, whereas higher-level syntactic and semantic representations show substantial differences in their organization and temporal stability.
Critical strengths
-
Interesting research question and experimental paradigm. The paper goes beyond conventional brain–model alignment based on neural predictivity and instead investigates the linguistic properties and temporal dynamics of representations within each system. The multi-definition-to-target paradigm is particularly interesting because it evaluates semantic generalization across different surface forms.
-
Comprehensive empirical analysis. The study covers a diverse collection of speech foundation models, including self-supervised, ASR-supervised, and audio-LLM encoders. The combination of computational models and intracranial recordings from 29 patients provides a valuable empirical foundation for the analysis.
-
Interpretable findings about representational organization. The combination of probing, temporal generalization, and feature-based encoding offers complementary perspectives on speech representations. The observation that models and cortex show greater similarity for lower-level linguistic information than for higher-order compositional structure is particularly interesting.
Critical weaknesses
-
Does the semantic decoding task adequately isolate compositional meaning? Although the target word is excluded from the definitions, the task involves only 33 target concepts, each associated with four definitions. Successful cross-definition classification may still rely on lexical or topical associations rather than compositional semantic understanding. Could the authors include a simple lexical baseline, such as a bag-of-words classifier operating on transcripts, or an analysis controlling lexical overlap across definitions? This would help establish how much of the semantic decoding performance reflects abstraction beyond lexical cues.
-
Comparability of temporal dynamics between models and cortex. The temporal generalization analysis is an important contribution, but the interpretation of the differences between model and neural dynamics requires further clarification. In particular, the evaluated speech encoders generally have access to bidirectional context, whereas the brain processes incoming speech online. Could this difference in available context contribute to the observed temporal generalization patterns? It would be useful to discuss this limitation and, if feasible, examine the robustness of the findings under more closely matched temporal contexts or preprocessing choices.
-
Interpretation of the different representational hierarchies. The paper argues that models and cortex exhibit substantially different organizations of low- and high-level linguistic information. However, the comparison involves different representational axes (model depth versus cortical anatomy), and the variance-partitioning procedures are not identical across the two systems. How sensitive is the conclusion to these methodological differences? The authors should clarify whether the observed differences reflect distinct representational organizations or, at least in part, the different ways in which the two systems are analyzed.
-
Statistical robustness of temporal generalization results. Appendix C indicates that some temporal generalization contours use an uncorrected significance threshold. Given the large number of temporal comparisons, could the authors clarify whether the main conclusions remain robust under appropriate multiple-comparison correction?
Questions for authors
See weaknesses.