acceptodds
Under review as a conference paper at ICLR 2027

What Multilingual Models Share: The Functional and Neural Significance of Representational Convergence

Abstract

Multilingual models transfer knowledge across languages, offering a way to study shared principles of language organization. Interpreting representational convergence requires an account of how shared structure relates to computation and human processing. We compare 60 languages across nine models from five families and examine the functional and neural significance of shared organization using grammatical interventions and Chinese intracranial recordings. Relations among aligned meanings recur more strongly across model families than activation organization or sequential response strength, while local reconstruction depends on the source language. Edits to grammatical-number information selectively weaken English agreement for forms excluded from estimation in every model tested, and later edits offset the loss in most models; confined to the units that carry the recurrent relations, the edit retains 3–41% of its effect, within or below the range of matched random units, so grammatical influence is distributed beyond those units. In Qwen2.5-7B, English- and Hebrew-derived edits produce similar rankings of affected Hebrew sentences, while strength and surrounding number information shape effect magnitude. Chinese intracranial recordings from 21 participants distinguish overall neural resemblance from the additional correspondence to the same sentences, and the shared and model-specific parts of a representation differ in that correspondence by less than one tenth of the neural reliability reference. These findings clarify the functional and neural significance of shared linguistic knowledge, informing theories of multilingual intelligence that accommodate common organization and computational diversity. Code, results, and experimental settings are available at https://anonymous.4open.science/r/Multilingual-69FC/

Primary area:
interpretability and explainable AI
Keywords:
multilingual language models, representation geometry, causal interventions, cross-lingual transfer, brain–model correspondence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UM1dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how shared patterns relate to grammar and brain responses. It finds that Concept (similarity of relations among aligned meanings across languages ) gives the most consistent cross-model language relations, but the selected units do not show stronger grammatical effects than random units. Number interventions ( using steering vectors to alter singular or plural information and affect subjectverb agreement) also transfer from English to Hebrew. Shared representations (the part predicted from other models’ representations) do not show clearly stronger correspondence with brain responses than model-specific representations.

Critical strengths
  1. Connecting intracranial brain recordings with LLM representations is interesting. Although the conclusions are limited, the released data and code could support further research in this direction.
  2. The paper goes beyond measuring representation similarity and tests whether the selected units have a causal role in grammar. Comparing their effects with matched random units is useful, since similar representations alone do not establish functional importance.
Critical weaknesses
  1. Many results are negative or inconclusive, and the findings seem scattered. The paper does not clearly bring them together into a coherent account of what multilingual models share.

  2. Interpreting Concept as meaning seems overstated. Both Concept and Subspace measure aspects of representational geometry, but use different features and similarity measures. It is therefore unclear whether their difference supports the proposed distinction between meaning relations and activation organization.

  3. The grammatical interventions only test subject–verb number agreement. Different forms of number marking provide useful variation, but they still test the same grammatical phenomenon, so broader claims about grammar seem overstated.

Questions for authors
  1. Line 162: both edits act at layer 14 . Why layer 14?

  2. Why is Concept interpreted as meaning, while Subspace is interpreted as activation organization? Both compare representation geometry. Could the difference come from the features and metrics used?

  3. Why should the selected units carry number agreement information? Could their weak effects simply mean that the selection method does not pick out units relevant to this task?

  4. What is new about the English-to-Hebrew steering experiment compared with earlier work on cross-language grammatical interventions?

  5. Do the claims about grammar only refer to subject–verb number agreement? If not, what supports the broader claims?

  6. The brain correspondence is small and the trial-reassignment test gives p = .072. How strong a conclusion can be drawn here? How can we interprete the results?

  7. What is the main finding that connects the three parts of the paper? I find the individual results hard to put together into one clear conclusion.

Reviewer_z1aFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates what multilingual language models share across languages and whether shared representations have functional and neural significance. The authors compare 60 languages across nine models from five families using three representational measures: Concept, Subspace, and Rhythm. They find that semantic relations are more consistent across models than activation patterns or sequential responses. Grammatical-number interventions reveal distributed grammatical influence and partial English-to-Hebrew transfer. Using Chinese intracranial recordings from 21 participants, the authors further examine the correspondence between model representations and neural responses. The results suggest that representational convergence, grammatical influence, and brain correspondence capture distinct properties of multilingual models.

Critical strengths
  1. An important research question beyond representational similarity.
  2. Broad multilingual and cross-model evaluation.
  3. Combining representational analysis with interventions and neural data.
Critical weaknesses
  1. Although Concept shows the highest cross-model recurrence, its correspondence with external linguistic properties remains weak. After controlling for genealogy, geography, and writing systems, its largest mean WALS coefficient is only 0.0457, with an incremental R2R^2 of 0.00314. Thus, while the paper demonstrates representational convergence, it remains unclear which linguistic properties drive this shared structure. Further analysis of specific semantic and grammatical factors would strengthen the interpretation.

  2. The English-to-Hebrew transfer experiments are conducted only on Qwen2.5-7B. Since models differ in training data, tokenization, and grammatical representations, results from a single model may not generalize across model families. Although seven models are evaluated on English grammatical interventions, extending the cross-language experiments to these models would strengthen the generality of the findings.

Questions for authors

see weakness.

Reviewer_7ehqOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The paper investigates shared representational organization across languages and models, and examines its functional and neural significance through grammatical interventions and comparisons with human brain recordings.

Critical strengths

The paper addresses an interesting and important question: to what extent do multilingual models share representations and computation across languages, and how does this sharing relate to human language processing? Investigating these connections could deepen our understanding of cross-lingual generalization and help clarify what language models can reveal about the organization of the human language system.

Critical weaknesses
  • The paper requires substantial revision in presentation and organization. I found it difficult to connect the individual experiments to the central arguments and determine how the evidence supports the main conclusions. In particular, the relationships among the 60-language representational comparison, the English/Hebrew grammatical interventions, and the Chinese brain-recording analysis need to be explained more explicitly, with the scope of each finding clearly stated. Several technical terms, including “overall neural resemblance” and “same-sentence correspondence,” are introduced with insufficient intuitive explanation. The intervention experiments would also benefit from a worked example in the main text showing what is edited, how the controls differ, and how the resulting measurements address the research question. Bringing essential methodological details from the appendices into the main text would substantially improve readability.
  • The models used in this paper are fairly old and outdated. Given a lot of these models are not multilingual, I wonder if the arguments made in this paper could generalize to other more "multilingual" LLMs.
Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.