Pretraining Domain Shapes Auditory Neural Predictivity Differently Across Species

Deep neural networks have successfully captured aspects of neural representation in both humans and nonhuman primates, particularly in vision. However, whether such generalization extends to species-specific domains, such as speech, remains unclear. We tested self-supervised and supervised models spanning training corpora from human speech to animal vocalizations on both animal and human neural responses prediction tasks. Across all animal neural-prediction tasks, models achieved nearly identical best-layer encoding accuracy, however, on human conditions, diverged across domains. To examine whether this convergence versus divergence pattern is already present at the level of model representations, we applied unbiased Centered Kernel Alignment (uCKA) to the activations and observed a pretraining-corpora-dependent input geometry across all datasets. To investigate why this geometry is not fully reflected in encoding accuracy, we introduced a sensitive subspace analysis to the frozen predictor and found pretraining-corpora-dependence in speech models disappeared on animal datasets but preserved on human datasets. Our findings demonstrate a species-dependent effect of pretraining domain and, using speech as a case study, explain how brain filters out domain-specific model representations absent from that species.