acceptodds
Under review as a conference paper at ICLR 2027

MELD: Measuring Multilingual Capability of Language Models through Latent Distribution Geometry

Abstract

Multilingual large language models exhibit substantial variation in capability across languages, yet evaluating this variation typically requires costly downstream benchmarks. We introduce MELD, an intrinsic metric for estimating multilingual capability directly from the geometry of hidden states. MELD is motivated by growing evidence that multilingual LLMs develop a shared semantic and reasoning space in their intermediate layers, while language-specific structure is more prominent near the input and output boundaries. MELD measures distributional language neutrality by contrasting the separation between reference- and target-language representation distributions with their within-language dispersion. We evaluate MELD across 14 open-weight LLMs spanning multiple model families, parameter scales, depths, and training regimes, and validate it against five multilingual benchmark families covering more than 30 languages. We assess MELD from two complementary perspectives: predicting cross-language performance differences within a model and ranking models by their overall multilingual capability. In the within-model setting, MELD achieves average Pearson and Spearman correlations of about 0.94 and 0.93, respectively, with downstream multilingual performance. We further show that this predictive relationship remains strong when sentence-level correspondence is removed using non-parallel reference data, whereas prior methods degrade substantially. MELD also strongly correlates with multilingual capability across all models, achieving average Pearson and Spearman correlations of about 0.78 and 0.75, respectively, with overall downstream performance. Together, these results demonstrate that MELD provides a robust intrinsic signal of multilingual capability across both languages and models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multilingual LLMs, Multilingual Evaluation, Representation Geometry, Cross-Lingual Alignment, Intrinsic Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1ZY8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a new methodology called MELD (metric) to estimate multilingual capability from model's hidden states. Their metric achieves high correlation scores with downstream model capability, and also works across model families.

Critical strengths

The metric seems novel and performs better than their discussed baseline. The metric also has high-correlation with both in-model downstream multilingual benchmarks and across different model families as well.

Critical weaknesses
  1. All the downstream benchmarks target "discriminative" tasks where the answer space is defined via MCQs. The authors did not test their methodology on generative tasks. Any reasons behind not looking into those? They can look into trying XLSum, or even use NTREX as an evaluating task where they use FLORES to calculate the initial intrinsic score.

  2. Another problem regarding the downstream benchmarks: m-MMLU and MMMLU are the same benchmarks and only differ by the fact that m-MMLU is machine translated v/s MMMLU which is human-translated. Why did the authors choose both of these benchmarks? m-ARC and m-HellaSwag also come from the same pipeline or suite of translated benchmarks (Okapi) - not sure if it makes sense to agree with the high-correlation of these benchmarks. An ideal eval setup could have been MMMLU, Belebele and a couple of generative benchmarks like XLSum, Translation (WMT) etc.

  3. The main results in Table 2 and 3 takes into account all languages (low, medium, high-resource languages). Separating them out (like done in Figure 1) would actually help us know if the correlations actually explain the middle-language layer neutrality or is an artifact of the amount of training data that these models have seen.

Questions for authors
  1. I see in appendix that the authors did a sweep on the percentage of middle layer and come to conclusion that 10% works better. However, the graph (Figure 2) that the peak is hit between 30-40%. I don't understand how materially does the extra 20-30% affect this choice. Can the authors explain their rationale? How did you define the "middle" layer across various model architecture for your experiments in lines 354-357?

  2. Any reason why did the authors choose different eval setups for pre-trained v/s instruct and reasoning models? 0-shot v/s 5-shot eval? why not do 0-shot or 5-shot all the way for all the models?

  3. I am still not convinced with the motivation of this work. How does the MELD score help in future design of LLMs?

  4. Suggestion: The authors would greatly benefit from using simpler phrases / sentences in their draft to improve the readability of the paper.

4 comments

We sincerely thank the reviewer for their encouraging and positive assessment of MELD’s novelty and empirical performance, as well as for their thoughtful and constructive suggestions. Several of these comments motivated additional analyses and revisions to the manuscript (the revised manuscript has been uploaded). We address each concern below.

1. Evaluation on Generative Tasks. We appreciate the suggestion to include generative benchmarks in the analysis. We would like to clarify that MELD primarily aims to predict multilingual understanding and reasoning performance, rather than the full range of multilingual capabilities, including unrestricted generation.

This focus is motivated by prior studies suggesting that intermediate layers support shared semantic and reasoning processes, whereas language-specific processing and output realization are more prominent in the outer layers [1–3]. Multiple-choice benchmarks allow us to assess comprehension, knowledge access, and reasoning while reducing the influence of output formatting and language-specific answer realization. This choice is also consistent with prior intrinsic evaluation methods, including MEXA and Language Ranker [4,5].

Related studies further motivate distinguishing understanding from generation. For example, LinguaMap [6] shows that models can identify correct answers while responding in an unintended language, while Zhao et al. [7] demonstrate that reducing language-specific representations can improve reasoning while affecting output fidelity. Similar distinctions have been observed in multilingual factual recall [8,9].

To clarify this scope, we revised the manuscript as follows:

  • Title: Changed to “MELD: Measuring Multilingual Understanding in Language Models through Latent Distribution Geometry.”
  • Contributions: Clarified that MELD predicts multilingual understanding and reasoning performance across languages and models.
  • Section 3.1: Refined our central hypothesis to emphasize the relationship between middle-layer language neutrality and understanding-related performance.
  • Section 4.3: Added an explanation of our multiple-choice benchmark selection and its motivation.
  • Abstract and conclusion: Revised the terminology to reflect MELD's intended evaluation scope.

We agree that multiple-choice tasks do not completely isolate understanding from generation, and extending MELD to open-ended generative evaluation remains an important direction for future work.

2. Overlap Among Downstream Benchmarks. We agree that m-MMLU and MMMLU share the same underlying benchmark and should not be considered independent task families. We include both to examine whether the relationship between intrinsic scores and downstream performance remains consistent across different translation methodologies and language coverage: m-MMLU is machine-translated and covers 31 languages, whereas MMMLU is professionally translated and covers 14.

Similarly, although m-ARC and m-HellaSwag share the Okapi translation pipeline, they evaluate different underlying capabilities: science reasoning and commonsense inference, respectively. BELEBELE provides an additional reading-comprehension evaluation outside the Okapi suite. Importantly, MELD achieves strong correlations on the independently sourced benchmarks as well.

We agree that broader benchmark diversity would provide a more comprehensive assessment. However, multilingual benchmarks with broad language coverage remain relatively scarce.

3. Language Resource-Group Analysis. We appreciate this suggestion. To examine whether the correlations differ across language resource levels, we conducted an additional analysis dividing the 41 target languages into high-resource, medium-resource, and low-resource groups.

We recomputed within-model correlations separately for each group. MELD achieves Pearson/Spearman correlations of 0.894/0.844 for high-resource languages and 0.921/0.893 for medium-resource languages, outperforming or performing comparably to the baselines. For low-resource languages, correlations are weaker (0.419/0.385), and no method consistently dominates. As documented in Table 6, only four low-resource languages are covered by m-MMLU, m-ARC, and m-HellaSwag, while MMMLU covers just one. Such limited coverage makes subgroup correlations sensitive to individual languages.

We report the resource-group assignments, effective benchmark coverage, and correlations in Appendix A.9 (Tables 14–16). These findings provide a more detailed characterization of MELD's strengths and limitations, although they cannot fully disentangle the influence of training-data availability.

4. Selection of the Central 10% of Layers. Our selection is motivated by a trade-off between predictive performance and computational efficiency, rather than the claim that 10% always yields the highest correlation. As Figure 2 shows, correlations remain relatively stable across a broad range of middle-layer fractions, with only modest improvements near 30–40%. The central 10% achieves correlations close to the observed maximum while requiring representation extraction and scoring from substantially fewer layers.

We define the middle-layer region proportionally for each architecture, selecting approximately the central 10% of its transformer layers, centered at the network midpoint. Thus, the selected region adapts to models with different depths rather than relying on fixed layer indices.

We have clarified this rationale in Section 4.6 and Appendix A.2.2, where Figure 2 presents the layer-fraction ablation.

5. Zero-Shot vs. Five-Shot Evaluation. We agree that different evaluation protocols across models could affect the fairness of model comparisons. To address this, we revised our across-model evaluation to use a uniform five-shot protocol for all 14 models, regardless of training regime. Under this setting, MELD achieves average Pearson/Spearman correlations of 0.839/0.827, compared with 0.730/0.569 for MEXA and 0.487/0.541 for Language Ranker. For within-model language ranking, we retain the original settings (zero-shot for pretrained models and five-shot for instruction-tuned and reasoning models), since languages are compared under the same protocol within each model.

We have clarified this distinction in Section 4.5 (Evaluation Metrics) and updated the across-model results in Table 4.

6. Practical Motivation and Future Usefulness. We appreciate the opportunity to clarify MELD's intended practical value. MELD is designed as an intrinsic diagnostic and evaluation tool, rather than a direct method for modifying model architectures or training procedures.

By estimating multilingual understanding-related performance from internal representations, MELD could help compare multilingual model checkpoints, identify languages with weaker representation neutrality, and monitor the effects of multilingual training strategies without requiring extensive downstream benchmarks or aligned translation corpora.

Our results establish predictive associations rather than causal mechanisms. Therefore, we do not claim that directly maximizing MELD would necessarily improve multilingual understanding. Instead, MELD provides a potentially useful measurement tool for evaluating and investigating future multilingual model designs.

Finally, we appreciate the reviewer's constructive feedback, which helped us improve the consistency of our evaluation, better characterize resource-dependent performance, and clarify MELD's motivation and limitations.

References

[1] Wendler et al. (2024). Do Llamas Work in English? On the Latent Language of Multilingual Transformers. ACL.
[2] Bandarkar et al. (2025). Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models. ICLR.
[3] Harrasse et al. (2025). Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders. arXiv.
[4] Kargaran et al. (2025). MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment. Findings of ACL.
[5] Li et al. (2025). Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages. AAAI.
[6] Tamo et al. (2026). LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them? ICLR.
[7] Zhao et al. (2026). When Less Language Is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners. NeurIPS.
[8] Lu et al. (2025). Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline. EMNLP.
[9] Wang et al. (2025). Lost in Multilinguality: Dissecting Cross-Lingual Factual Inconsistency in Transformer Language Models. ACL.
Reviewer_1ZY8OpenReview ↗

Replying to the rebuttal

Thank you for running the experiments. Like I said earlier, and the other reviewer also stated: the current metric does not bring anything new to the table. It would have been interesting if your metric was robust and task-agnostic. My current score reflects the assessment of this work.

We thank you for the time and effort devoted to reviewing our paper. We appreciate your feedback and constructive suggestions. They helped us improve the manuscript.

We believe the novelty of our proposed approach lies in replacing sentence-pair alignment with a set-based, distribution-level measure that operates on independently sampled monolingual corpora, without requiring parallel datasets as prior methods do.

Furthermore, MELD is specifically designed to measure multilingual understanding and reasoning. As explained earlier, our choice of multiple-choice benchmarks is deliberate, as they assess comprehension, knowledge access, and reasoning while reducing confounding factors associated with unrestricted generation.

Once again, we sincerely appreciate your engagement and valuable feedback.

Reviewer_VBsXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MELD, a training-free intrinsic metric that estimates the multilingual capability of LLMs by comparing the distributions of English and target-language sentence representations in intermediate layers, treating stronger distributional language neutrality as a proxy for stronger capability. Because it relies only on set-level statistics of the two representation sets, MELD does not require parallel data. Across 14 open-weight LLMs and five multilingual benchmark families, MELD achieves an average within-model correlation of about 0.94 with downstream performance, remains stable when parallel data is replaced by non-parallel data.

Critical strengths
  • Only requires forward passes over roughly 10% of the layers and does not depend on sentence-level parallel data, which broadens its applicability.
  • Somehow robust in the non-parallel setting. When the English side is replaced with non-parallel sentences, MELD's correlations remain nearly unchanged while MEXA and LR degrade substantially (Table 3), which is practically relevant for languages that lack parallel data.
  • Reasonably thorough analysis. Ablations on the covariance approximation and the middle-layer fraction (Table 5, Fig. 2), significance testing, and released code.
Critical weaknesses
  • Limited novelty. The core hypothesis, that intermediate layers are more language-neutral and that this relates to multilingual capability, is inherited from prior work, and the paper mainly verifies this correlation on more models. The metric itself is close to a classical standardized centroid distance, and the paper offers neither new mechanistic insight nor causal evidence.
  • Confounded comparison with baselines. The methods extract representations differently. MELD mean-pools over all tokens and uses only the central ~10% of layers, LR uses the last-token representation sampled every fifth layer, and MEXA uses weighted mean pooling across all layers (Sec. 4.4, 4.6). Part of MELD's advantage may therefore come from pooling and layer selection rather than from the scoring function itself, so the comparison should be repeated under identical representation settings.
  • Insufficient non-parallel evaluation. Only the English side is replaced, while all target languages still come from the same parallel corpus with identical content. Robustness when each language uses independently sampled monolingual corpora remains unverified.
  • Narrow downstream evaluation. All benchmarks are log-likelihood-scored multiple-choice comprehension tasks, most of them machine-translated, with no generative tasks, which makes it hard to support a claim as broad as "multilingual capability".
Questions for authors

See weakness.

2 comments

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We appreciate the recognition of MELD's efficiency, robustness in non-parallel settings, and experimental analysis. We have addressed the concerns through additional experiments and revisions that clarify our contributions and evaluation scope.

1. Limited Novelty. We agree that the observation of greater language neutrality in intermediate layers builds on prior work, and we do not claim a new mechanistic explanation or causal evidence. Our contribution is to translate this observation into a practical intrinsic metric for predicting multilingual understanding and reasoning performance. We acknowledge its connection to standardized centroid distance; the contribution lies in its use as a multilingual evaluation metric and its systematic validation across diverse models, languages, and benchmarks.

We have clarified this positioning in the revised manuscript, including the contribution statements and Section 3.1, where we explicitly frame stronger middle-layer language neutrality as a hypothesis about better multilingual understanding performance rather than a causal mechanism.

2. Confounded Comparison with Baselines. We thank the reviewer for highlighting this important concern. Our original experiments followed the representation-extraction settings specified in each baseline's original paper. However, we agree that differences in pooling and layer selection could influence the comparison. To address this, we conducted an additional experiment in which all models use identical mean-pooled representations from the central 10% of transformer layers, while retaining their respective scoring functions.

Under these settings, MELD outperforms both baselines. Averaged across these settings, MELD achieves 0.942/0.931, compared with 0.860/0.875 for Language Ranker and 0.835/0.887 for MEXA. We have included the complete results in Appendix A.5 (Evaluation Under Matched Representation Settings) and added the following clarification at the end of Section 4.6:

Additionally, to further control for differences in representation extraction and layer selection, we evaluate all three methods using identical mean-pooling and central-layer settings in Section A.5 of the Appendix.

3. Insufficient Non-Parallel Evaluation. We agree that replacing only the English reference corpus does not fully evaluate a setting in which all languages are represented by independently sampled monolingual data. To address this, we conducted a fully non-parallel experiment using FineWeb for English and FineWeb2 for the remaining 41 languages. MELD maintains strong average Pearson/Spearman correlations of 0.888/0.899 for within-model language ranking and 0.883/0.829 for across-model ranking.

We have included the complete experimental setup and results in Appendix A.6 (Evaluation Under Fully Non-Parallel Settings). We also added the following paragraph at the end of Section 5.2:

To further evaluate robustness under fully non-parallel conditions, we conduct an additional experiment using independently sampled monolingual documents for both reference and target languages. Full experimental details and results are provided in Appendix A.6.

4. Narrow Downstream Evaluation. We appreciate the concern regarding the exclusive use of multiple-choice benchmarks. We would like to clarify that MELD primarily aims to predict multilingual understanding and reasoning performance, rather than the full range of multilingual capabilities, including open-ended generation. This focus is motivated by prior studies showing that intermediate layers are associated with shared semantic and reasoning processes, while language-specific processing and output realization are more prominent in the outer layers [1–3]. Accordingly, our choice of multiple-choice benchmarks is deliberate. They assess comprehension, knowledge access, and reasoning while reducing factors associated with unrestricted generation. This approach is also consistent with prior intrinsic evaluation methods, including MEXA and Language Ranker [4,5].

Importantly, previous studies further motivate distinguishing understanding-related performance from generation. LinguaMap [6] demonstrates that models can identify correct answers while responding in an unintended language. Zhao et al. [7] show that suppressing language-specific representations can improve reasoning while affecting output fidelity. Related studies [8,9] also identify discrepancies between internally accessible factual knowledge and its realization in the target language.

To clarify the intended scope, we made the following manuscript revisions:

  • Title: Revised to “MELD: Measuring Multilingual Understanding in Language Models through Latent Distribution Geometry.”

  • Contributions: Clarified that MELD predicts multilingual understanding and reasoning performance across languages within models and overall performance differences across models.

  • Section 3.1: Refined the central hypothesis to emphasize the relationship between middle-layer language neutrality and multilingual understanding performance.

  • Section 4.3: Added the following explanation:

    Our evaluation focuses on multilingual understanding and reasoning rather than unrestricted text generation. Accordingly, we select multiple-choice benchmarks that assess comprehension, knowledge access, and reasoning while reducing the influence of output formatting and language-specific answer realization.

  • Abstract and conclusion: Revised to consistently emphasize multilingual understanding and reasoning rather than general multilingual capability.

We acknowledge that multiple-choice evaluation does not completely isolate understanding from generation and that extending MELD to open-ended generative tasks remains an important direction for future work.

We sincerely appreciate the reviewer's suggestions, which helped us strengthen the experimental comparisons, extend the non-parallel evaluation, and clarify MELD's contributions and limitations.

References

[1] Wendler et al. (2024). Do Llamas Work in English? On the Latent Language of Multilingual Transformers. ACL.
[2] Bandarkar et al. (2025). Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models. ICLR.
[3] Harrasse et al. (2025). Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders. arXiv.
[4] Kargaran et al. (2025). MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment. Findings of ACL.
[5] Li et al. (2025). Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages. AAAI.
[6] Tamo et al. (2026). LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them? ICLR.
[7] Zhao et al. (2026). When Less Language Is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners. NeurIPS.
[8] Lu et al. (2025). Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline. EMNLP.
[9] Wang et al. (2025). Lost in Multilinguality: Dissecting Cross-Lingual Factual Inconsistency in Transformer Language Models. ACL.
Reviewer_bn4jOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MELD, an intrinsic metric that estimates multilingual capability by measuring language-conditioned distribution separation relative to within-language dispersion. Experiments across 14 open-weight LLMs and five multilingual benchmark families show strong correlations between MELD and downstream performance, including when parallel sentence correspondence is removed.

Critical strengths
  • The method is simple, intuitive, and does not require sentence-aligned parallel data.
  • The evaluation covers diverse model families, scales, training regimes, benchmarks, and languages.
  • MELD achieves consistently strong within-model correlations with downstream multilingual performance.
  • The non-parallel evaluation provides compelling evidence that MELD captures distribution-level properties rather than sentence-level alignment.
  • The paper is generally clear and includes useful covariance and layer-selection ablations.
Critical weaknesses
  1. Under the diagonal approximation, the within-language dispersion is nearly determined by hidden dimensionality, so MELD may largely reduce to a transformed variance-normalized centroid distance.
  2. The cross-model evaluation includes only 14 models and may be confounded by model size, hidden dimensionality, and different few-shot protocols.
  3. All main experiments use English as the reference language, limiting evidence for reference-language independence.
  4. Most downstream benchmarks are translated multiple-choice tasks, leaving generalization to generation, translation, and dialogue tasks unclear.
Questions for authors

No

2 comments

We thank the reviewer for the thoughtful assessment and for recognizing MELD's simplicity, broad evaluation, and robustness without sentence-aligned data. We particularly appreciate the technical observations, which helped us strengthen our evaluation (the revised manuscript has been uploaded). We try to address each concern below.

1. Interpretation of the Diagonal Approximation. We thank the reviewer for this insightful observation, which we agree with. MELD (Under the diagonal approximation and negligible regularization) can be interpreted as a bounded transformation of the variance-normalized centroid separation, further normalized by hidden dimensionality. This normalization helps make scores comparable across models with different hidden sizes. We make this relationship explicit in Appendix A.1 (Equations 15–16). MELD contribution is to operationalize this geometric quantity as an efficient, set-based intrinsic predictor of multilingual understanding, particularly through its middle-layer aggregation and independence from sentence-level alignment.

2. Model Scale and Architecture Confounding. We agree that correlations across 14 models require careful interpretation. To investigate potential confounding and sensitivity to the evaluated models, we conducted three additional analyses.

  • First, controlling for model scale, we compute partial correlations between intrinsic scores and downstream performance while controlling for log parameter count. MELD maintains strong partial Pearson/Spearman correlations of 0.828/0.861.

  • Second, testing individual-model sensitivity, we perform leave-one-model-out evaluation, recalculating correlations after excluding each model in turn. MELD maintains Pearson correlations of 0.810–0.902 and Spearman correlations of 0.783–0.912.

  • Third, examining within-family consistency, we compare whether intrinsic scores correctly order pairs of models belonging to the same family. MELD achieves 100% ranking agreement across 10 pairs per corpus.

We also addressed the few-shot protocol concern by adopting uniform five-shot evaluation across all 14 models for across-model comparisons. These changes are reflected in Section 4.5 and Table 4, with the three additional robustness analyses reported in Appendix A.7 (Tables 10–12). We have also added a reference to these analyses in Section 5.3.

While the results strengthen the evidence for robustness, we acknowledge that computational resource constraints limited our evaluation to 14 models, preventing us from fully disentangling the effects of model architecture, hidden dimensionality, and training differences.

3. Dependence on English as the Reference Language. To examine whether MELD's effectiveness depends on an English reference, we conducted additional experiments using Spanish, Mandarin Chinese, and Arabic as three different language families and alternative reference languages. In these settings, MELD maintains strong average Pearson/Spearman correlations of 0.925/0.919 for Spanish, 0.914/0.851 for Chinese, and 0.920/0.871 for Arabic. Averaged across the three references, MELD achieves 0.920/0.880. These findings indicate that MELD's predictive effectiveness is not restricted to English, although the correlation strength varies with the reference language. We report the complete results in Appendix A.8 (Table 13) and have added a corresponding reference in Section 5.1.

4. Generalization Beyond Multiple-Choice Benchmarks. We agree that our current evaluation does not establish MELD's effectiveness for open-ended generation, translation, or dialogue. Our benchmark selection is motivated by MELD's intended focus on multilingual understanding and reasoning performance. Prior studies associate intermediate layers with shared semantic and reasoning processes, while language-specific processing and output realization are more prominent near the model boundaries [1–3]. In this way, multiple-choice benchmarks provide a relatively controlled setting for evaluating comprehension, knowledge access, and reasoning while reducing the influence of unrestricted generation. This is also consistent with the evaluation approaches adopted by MEXA and Language Ranker [4,5]. Related studies further demonstrate that correct internal processing and successful target-language generation need not coincide [6–9].

To ensure our claims reflect this scope, we made several revisions:

  • Title: Revised to “MELD: Measuring Multilingual Understanding in Language Models through Latent Distribution Geometry.”
  • Contributions: Clarified that MELD predicts multilingual understanding and reasoning performance across languages and models.
  • Section 3.1: Refined the central hypothesis to focus on the relationship between middle-layer language neutrality and understanding-related performance.
  • Section 4.3: Added an explicit motivation for selecting multiple-choice benchmarks.
  • Abstract and conclusion: Revised the claims to emphasize understanding and reasoning rather than unrestricted multilingual capability.

We recognize that multiple-choice evaluation does not fully isolate understanding from generation. Extending MELD to generative tasks remains an important direction for future evaluation.

Ultimately, we appreciate the reviewer's constructive feedback. These clarifications and additional experiments have helped us provide a stronger evidence for the robustness and intended scope of MELD.

References

[1] Wendler et al. (2024). Do Llamas Work in English? On the Latent Language of Multilingual Transformers. ACL.
[2] Bandarkar et al. (2025). Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models. ICLR.
[3] Harrasse et al. (2025). Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders. arXiv.
[4] Kargaran et al. (2025). MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment. Findings of ACL.
[5] Li et al. (2025). Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages. AAAI.
[6] Tamo et al. (2026). LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them? ICLR.
[7] Zhao et al. (2026). When Less Language Is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners. NeurIPS.
[8] Lu et al. (2025). Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline. EMNLP.
[9] Wang et al. (2025). Lost in Multilinguality: Dissecting Cross-Lingual Factual Inconsistency in Transformer Language Models. ACL.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.