acceptodds
Under review as a conference paper at ICLR 2027

Multilingual GSM-Symbolic: What determines capability transfer across languages?

Abstract

We understand little about how capabilities acquired in one language carry over to another, or what governs this transfer: evaluations rely on incomparable, saturation-prone datasets and rarely examine its determinants jointly. Identifying what predicts transfer would let us avoid exhaustive evaluation across all language pairs and let developers target the factors that limit performance in low-resource languages. To evaluate cross-lingual capability transfer, we introduce MULTILINGUAL GSM-SYMBOLIC, an extensible multilingual mathematical dataset covering 30,000 item-matched question–answer pairs and spanning 15 languages. It utilises symbolic templates to prevent overfitting and ensure generalisation by allowing generation of millions of high-quality variations from a single sample. Using MULTILINGUAL GSM-SYMBOLIC, we quantify the largest determinants of capability as model size (β = 1.77), language resource level (β = 0.77), reasoning (β = 0.67) and typological distance (β=−0.25). This joint estimation allows these determinants to be expressed in terms of one another: a 32B model evaluated in Marathi performs like a 10B model in English. Our findings have important implications for model developers, showing that model size and reasoning narrow the performance gap between low- and high-resource languages (β=−0.27, β=−0.20, respectively), while similar levers have little or no effect on typologically distant languages. Overall, our analysis framework explains 92% of between-language variation, but only 23% of the model-by-language variation, and predicts a model’s performance on an unseen language within 6.0pp (r=.96). Incorporating measurements from just 10 templates in the target language reduces this to 4.19pp, enabling reasonable estimates of performance with little or no downstream dataset. We release the full dataset and code to support future research on, and more evaluation of, cross-lingual capability transfer.

Primary area:
datasets and benchmarks
Keywords:
multilingual evaluation, cross-lingual transfer, mathematical reasoning‚ benchmarking, symbolic templates, low-resource languages, performance prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WvuVOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces Multilingual GSM-Symbolic, a multilingual extension of GSM-Symbolic containing 100 underlying mathematical templates translated and localized into 15 languages. The authors render 20 instances from each translated template, yielding 30,000 question–answer instances, and state that the templates, generation code, validation tests, model outputs, and annotation provenance are released through an anonymous repository. The benchmark is evaluated with a broad collection of open-weight instruction-tuned models.

The paper then fits a generalized linear mixed model to estimate how model size, reasoning mode, language resource level, typological distance from English, tokenizer fertility, and their interactions relate to accuracy. It reports that model size, language resource level, and reasoning are the largest positive predictors, while typological distance is negatively associated with performance. The authors additionally perform leave-one-language-out prediction, symbolic-versus-original comparisons, and analyses of where errors occur within generated solutions.

The benchmark construction is useful, and the joint quantitative analysis is more systematic than reporting separate multilingual accuracy tables. However, the methodological novelty is limited, most headline conclusions are already expected from prior multilingual evaluation work, and the paper overstates both the independence of the benchmark instances and the causal or transferable meaning of its regression coefficients.

Critical strengths
  1. A controlled and reusable multilingual evaluation resource. Using the same underlying mathematical structures across 15 languages is preferable to comparing unrelated monolingual benchmarks. Releasing editable symbolic templates and generation code, rather than only a fixed set of rendered questions, may support future contamination-resistant evaluation and correction of dataset errors.

  2. Broad evaluation with a joint analysis of model and language factors. The paper evaluates a diverse set of model families and places model size, reasoning, resource level, typological distance, and tokenizer fertility in one statistical framework. The leave-one-language-out analysis is also more informative than reporting only aggregate multilingual accuracy.

Critical weaknesses
  1. The effective dataset novelty, structural diversity, and translation validation are overstated. The reported 30,000 instances are generated from only 100 underlying GSM-style templates. The paper's own variance analysis shows that approximately 93% of the remaining uncertainty comes from variation across templates and that generating additional instances from the same templates provides little benefit. Most headline findings are computed on a fixed set of rendered instances and could plausibly have been obtained with a sufficiently large static item-matched multilingual dataset. The paper does not demonstrate which central scientific conclusion specifically requires its symbolic, extensible infrastructure. Given the availability of MGSM, MGSM-Pro, mGSM-Symbolic, and related resources, the incremental contribution is closer to a useful engineering consolidation than a fundamentally new benchmark paradigm. In addition, all translators used LLM assistance, but the paper does not clearly report independent double review, inter-annotator agreement, or language-wise error rates for naturalness and semantic equivalence.

  2. The robustness, external validity, and operational usefulness of the regression analysis are limited. Common Crawl page count is a model-agnostic proxy rather than the actual language composition of each model's training corpus, while model size, model family, training compute, data mixture, tokenizer, and post-training recipe remain partly confounded. Only 15 languages support the language-level conclusions, and typological distance is highly correlated with tokenizer fertility. Moreover, the appendix reports a large template-by-language random effect that substantially improves AIC, yet this term is omitted from the primary model, and the shared difficulty of individual symbolic instances is not modeled. The paper reports only the standard deviations of the random effects, not the fitted effects required to reproduce a concrete prediction from the published formula. Finally, leave-one-language-out evaluation considers known models in an unseen language, but does not establish performance for a new model family or a jointly unseen model-language pair. The fitted model therefore functions more convincingly as a retrospective summary of the evaluated matrix than as a generally usable prediction tool.

Questions for authors
  1. Which principal result specifically depends on having symbolic and extensible templates rather than a static item-matched multilingual dataset? Can the authors demonstrate that template-based resampling materially improves model ranking reliability, contamination resistance, or estimation of the reported fixed effects? Given that the paper's own analysis finds that template-level variance dominates, why was structural coverage limited to only 100 underlying templates?

  2. How many templates in each language were independently reviewed by a second native speaker? Please report language-wise grammaticality, naturalness, semantic-equivalence, and answer-correctness error rates. All translators used LLM assistance, so a translator reviewing their own LLM-assisted output is not sufficient to establish independent validation.

  3. How do the primary coefficients, confidence intervals, leave-one-language-out errors, and explained-variance estimates change when the template-by-language effect and an instance-level random effect are included? The reported template-by-language standard deviation is large, indicating that translation or localization changes template difficulty across languages.

  4. The main text reports approximately 92% of between-language and 23% of model-by-language variation explained, whereas the appendix reports different values for the original and alternative specifications. Please reconcile these numbers and identify the exact model underlying each headline result.

  5. How well does the regression model predict performance when an entire model family is held out, or when both the model family and language are unseen? For a genuinely new model-language pair, how should a practitioner use the published formula when the family, model, and model-by-language random effects are unavailable? Please provide the complete prediction procedure and the information required to reproduce the reported forecasts from the released model fit.

3 comments

We thank the reviewer for the detailed review and are glad the item-matched design, released code and joint analysis were seen as useful.

[…] why was structural coverage limited to only 100 underlying templates? […] resampling […]?

We suspect the 93% comes from Appendix A.5: "At n = 20 the template-level term is already 93% of the total…". The argument is that 20 instances per template is a reasonable budget because more would add little, not that instances are worthless. Equivalently sized benchmarks (e.g. MGSM) are widely used for model evaluation; 100 templates gives a strong enough signal while balancing validator effort across 18 languages.

[…] static item-matched multilingual dataset. Which principal result […] depends on […] extensible templates?

A static dataset may give a similar signal on model competence, but ours adds two things it lacks: (1) human-validated templates that can be scaled to arbitrary size with full control over which problem aspects vary, and (2) evaluation under variability, which reduces the effect of benchmark leakage. Extensibility is not required for our findings, but the open pipeline lets practitioners vary aspects of their choice (number ranges, gender of names, …), and since submission it has grown from 15 to 18 languages (Punjabi, Swedish, Urdu).

[…] independent double review, inter-annotator agreement, or language-wise error rates […]

Most languages start with one human validator, who checks and improves the initial machine translation. The open repository lets further native speakers validate and extend it, which has happened for five languages; there, template edits were merged only once all validators were satisfied. As this is an open discussion on GitHub, inter-annotator agreement is hard to report. Naturalness is hard to measure objectively, but validators were asked to actively correct templates for grammaticality, naturalness and semantic equivalence. The public git history (Added analysis in Appendix K.3) documents this: reviewers edited 98.4% of templates but changed the answer rule in only 2.9%, and even such non-wording edits often only accommodate target-language grammar without changing the computation (e.g. Danish nouns are sampled with their article because of grammatical gender, template 0032).

Common Crawl page count is a model-agnostic proxy […]

Training-corpus composition is not consistently reported, and data quality would remain; we note this in the limitations. The model × language effect captures model-specific deviations, e.g. the English-centric OLMo models do better on English than their size and features imply (+1.11 log-odds).

[…] typological distance is highly correlated with tokenizer fertility.

We therefore use relative fertility (deviation from the language mean, §3.3).

[…] template-by-language […] omitted from the primary model […] How do the primary coefficients […] leave-one-language-out errors, and explained-variance estimates change?

We agree and have made it the primary model (see the general response); everything was refitted, with leave-one-language-out errors of 3.1 pp (language mean) and 6.0 pp (model in language). Under the previous specification, significant coefficients keep their sign and significance except distance × reasoning (p = .03 → .06), and size, resource and reasoning are about 10% smaller (Appendix A.1, Table 2).

[…] only the standard deviations of the random effects, not the fitted effects […]

The fitted models were already available through the released code and rendered analyses (transfer_analysis.html, model_ablations.html) and we now also release all fixed effects and every random-effect level as CSV in paper/artifacts/analysis/fitted_effects/.

The main text reports approximately 92% […] Please reconcile […]

The 92% / 23% came from the original specification (recomputed: 91.7% / 22.6%). The appendix values (91% / 16%, 89% / 20%) did not reproduce, an error on our side. Every headline number now comes from the primary model (93% / 19%), and Appendix A.1 reports the original specification separately.

Finally, leave-one-language-out evaluation considers known models in […]

This evaluation simulates the case where the model is given but data in the target language is not, so holding a model or family out would not match the real-world use case; the goal is to estimate a known model's performance in a new language. For this, the procedure (Appendix F) is to refit on the other languages and predict with the fixed effects and the family, model, variant and template effects, setting the language, model × language and template × language terms to zero. Evaluating about 10 templates in the target language reduces the error to 4.4 pp. Refitting takes minutes on a laptop.

Finally, we are surprised by the presentation score of 1 and would be glad to improve it if you can point to passages that were hard to follow.

Reviewer_WvuVOpenReview ↗

Response to the Authors

We thank the authors for the detailed response. Two concerns remain.

Presentation: the manuscript still falls short of publication standards — headline coefficients disagree (size β=1.77 in the abstract vs. 1.88 in Finding 1; −0.27 vs. −0.25 in the conclusion), typos persist ("eThis" §3.1; "shortcormings" §2.1), Fig. 12 lacks model labels on its x-axis, and several pages contain unexplained blank regions.

Contribution: the response acknowledges extensibility is not required for the findings, and Findings 1–3 are explicitly consistent with prior work. Beyond a localized translation of GSM-Symbolic and a regression fit, what precisely is the novel contribution?

the manuscript still falls short of publication standards — headline coefficients disagree (size β=1.77 in the abstract vs. 1.88 in Finding 1; −0.27 vs. −0.25 in the conclusion), typos persist ("eThis" §3.1; "shortcormings" §2.1),

We have corrected typos. While the inconsistencies were indeed true in the original version it is not true in the revised version.

Fig. 12 lacks model labels on its x-axis, and several pages contain unexplained blank regions.

Fig 12 has models on its x-axis and contains no unexplained blank regions. This is true in both the review version. I had sent the PDF to a third party to confirm that this is indeed the case. I have also opened it in multiple pdf readers (zotero, mozilla firefox and Preview). I would be happy to address this issue but cannot reproduce it.

the response acknowledges extensibility is not required for the findings

Indeed, but it argued that is valuable. Releasing expandable high-quality resources allows others to utilize, contribute and thus forward science. This in itself is valuable.

, and Findings 1–3 are explicitly consistent with prior work. Beyond a localized translation of GSM-Symbolic and a regression fit, what precisely is the novel contribution?

While 3 of our 5 findings extend those of previous work, previous work tackled these individually and not nec. in a multilingual and symbolic setting. As many of these attributes interact modelling them jointly establishes exact relative ordering which allow prioritization of resources. We also mention this in line 304-306. Reproduction especially under different conditions strengthens existing claims and reproduction of existing results is itself valuable to our science.

However, besides these contributions, we also provide an important distinction of resource level and typological distance (also noted by VT6k), which have important implications for how to target development in low-resource languages. Similarly, we also show that using just a few target language template can drastically improve downstream estimates which is of importance for benchmark developers. Besides these headline finding we examine several other dimensions including the difference imperial and metric, the effect of validation on scores and explore the positive effect of synthetic samples in appendix A as well as provide a detailed solution gap analysis of where the error happens ($4.4, appendix I)

We thank the authors for the detailed response. Two concerns remain.

We are happy to hear that we have addressed most of the reviewers concerns and hope these last comments can clarify the remaining. If these address the reviewer concerns we hope that the reviewer will consider updating their scores accordingly.

Reviewer_JimmOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

I think the paper makes two main contributions: (1) create this multilingual GSM-Symbolic data set with 100 templates. Different from the static benchmark data sets, this one is “dynamic” in that users can generate “equivalent” items in different languages from the templates. The current data set contains 30,000 item-matched question-answer pairs from 15 languages. One thing that is unclear is how item-level equivalency is established. As alluded to my comments above, this equivalency is assumed in the main model, but the empirical evidence seems to suggest otherwise. (2) A generalized linear mixed model (GLMM) to evaluate how model features (i.e., model size, language resource level, reasoning, etc) affect model performance while accounting for six different random effects: including model family, model, variant, language, template, and model × language. Again, as mentioned above, I am susceptible to some random effects, such as model family, as if there are only a few models within a family, such variance term cannot be estimated accurately, and how such estimation error propagate to impact fixed effects estimate can only be quantified through simulation studies.

Critical strengths
  1. Table 1 shows the strength of the new benchmark, Multilingual GSM-symbolic, that the author is contributing to the field. I found it to be very valuable.
  2. GLMM is an appropriate (assuming template discrimination is the same across all 100 templates) analytical approach, and I found the empirical validation to be thorough. Including template resampling and out-of-sample prediction is a strength.
Critical weaknesses

There are three main gaps that I mentioned earlier and repeated here, which I consider to be the critical weakness:

  1. One major modeling gap is the current model assumes the template difficulty is invariant across languages. The model includes a template random intercept, which allows the 100 templates’ difficulty to vary, but it neither includes template-by-language interaction nor the within template variance across 20 questions. So overall, although the current model is reasonable to include template random intercept, because the paper aims to explain differences in cross-lingual mathematical reasoning capability, the absence of a template-by-language interaction term, or a within-template residuals, is potentially consequential.

  2. The empirical analysis is fairly extensive. One thing I noted is that in Appendix A.1., the authors fit an alternative model with a template × language random effect and the results are quite revealing. The “template × language” SD is substantial, and the authors report an AIC improvement of approximately 10^5. This suggests that the same mathematical problem structure can have very different difficulties across languages, which is precisely the measurement non-invariance (or differential item functioning) in psychometric literature. I wonder why when the alternative specification appears substantially better fitting, the authors still retained the simpler model as their primary analysis.

  3. The paper does not include Monte Carlo simulation studies. Because of small number of models and small number of questions per template, I wonder if the model parameters (especially the variance parameters) can be properly recovered. A simulation study would be helpful to demonstrate the fidelity of the proposed model.

Questions for authors

Nothing else in addition to my comments above.

3 comments

We thank the reviewer for the careful and constructive review. Your three gaps led directly to the main changes in the revision (see the general response).

One major modeling gap is the current model assumes the template difficulty is invariant across languages. [...]

I wonder why when the alternative specification appears substantially better fitting, the authors still retained the simpler model as their primary analysis.

You are right that template difficulty varies by language, which is differential item functioning in psychometric terms. Our item-matched design matches the mathematical problem structure across languages, not its difficulty, and its parallel nature is meant to enable exactly this analysis: whether the same core logic is processed with more or less difficulty in different surface forms. Such variation should therefore be modelled. We initially treated the template × language model as a robustness check, since our goal is inference on the fixed effects and not the best fit, but we realise this could read as retaining a worse-fitting model. We have therefore moved it to the main analysis (Equation 1, §3.2) and moved the original specification to Appendix A.1. Everything was refitted. The new term is large (sd = 0.95, larger than model × language and language) and reduces AIC from 422,410 to 320,129. All significant effects keep their sign and significance with one exception, distance × reasoning (p = .03 → .06), which we had already flagged as fragile. The three largest effects increase by about 10% (size 1.77 → 1.97, resource 0.77 → 0.84, reasoning 0.67 → 0.77), as can expected when a notable variance component is omitted from a logistic mixed model.

I am susceptible to some random effects, such as model family, as if there are only a few models within a family, such variance term cannot be estimated accurately [...] how such estimation error propagate to impact fixed effects estimate can only be quantified through simulation studies.

The paper does not include Monte Carlo simulation studies. [...]

We added a simulation study (Appendix A.2): we simulate from the fitted model (fixed effects and variance components as truth, new random effects and binomial counts on the real design) and refit it, 100 replicates, plus variants with 10 or 5 instead of 20 instances per cell and with only five of the nine families per replicate.

  • Fixed effects are recovered with negligible bias, and 95% intervals cover the truth in 93% of replicates on average (87–97% across terms). Fewer instances per cell leaves this unchanged.
  • Variance components: template, model × language and template × language are recovered to within 1%. Family is underestimated by 9% (with a wide 90% interval, 0.66–1.96 against a truth of 1.48), language by 14%, and the model-level variance (sd 0.17) is weakly identified (estimated as zero in 15% of replicates), we do not interpret it individually anywhere in the manuscript. With only five families the reasoning coefficient's coverage falls to 80%, the one place where the family sample size matters.
  • Propagation to the fixed effects: refitting with no family intercept, no model intercept, or family as a fixed effect leaves the language-level effects (resource, distance, resource × size) identical to three decimals. Only terms identified across models move (size 1.68–2.01, reasoning 0.75–0.77 Appendix A.1).

It seems that the model-level variance is not recovered well, which, you pointed out that it was understandable given the few models involved. In this regard, do you think the model should be re-considered without such a variance term, i.e., estimating each model separately rather than assuming a distribution of them.

While it is the one that recovers the worst it still recovers fairly often. I also find that the model including the term to be the more reasonable model specification -- so from a purely theoretical point of view I would prefer to keep it especially since removing it doesn't meaningfully alter our conclusions. We also include the ablation in the appendix A1. Including it also leads to more conservative effect sizes.

We hope that this cover all of your concerns otherwise please let us know if something is missing.

Reviewer_VT6kOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Multilingual GSM-Symbolic, a template-based math benchmark across 15 languages. It evaluates 46 model settings and examines how model size, reasoning, language resources, and grammatical distance from English relate to performance. Its main analytical finding is that scaling and reasoning reduce the resource-related disadvantage more consistently than the disadvantage associated with grammatical distance. It also predicts performance in held-out languages.

Critical strengths

MGSM exists, but this new proposed benchmark is more easily expandable given the template nature, be it to more languages or more questions. I do like the thoroughness of the experiments this paper present. The headline findings is perhaps non-surprising, eg some languages are worse than the others. But I do like the discussion on distinction between resource-related and structural-distance gaps is interesting, provided it is presented as an observed pattern rather than a demonstrated casuality.

Critical weaknesses

I don't have a strong weakness on this paper. Common Crawl page counts measure available web content rather than the amount or quality of data each model actually received. The results support different statistical patterns for these measures, but the explanation of why those patterns differ remains less clear. This also limits how confidently the findings can guide model development. URIEL itself is also not perfect, but this is perhaps orthogonal question.

The reported improvement over predicting each model’s average accuracy across other languages is useful. However, comparison with a simple predictor combining model performance and language resource level would clarify how much the additional features and interactions contribute.

Questions for authors
2 comments

First of all we are happy to see that you found no strong weaknesses and that you found distinction between resource and structural-distance insightful. We are also especially happy that the importance of the dataset being expandable and open was recognized and are happy to announce that the dataset has since expanded from 15 to 18 languages, showcasing the importance of having expandable datasets.

For the specific weaknesses:

Common Crawl page counts measure available web content rather than the amount or quality of data each model actually received.

We agree, ideally the model would have been fitted using known training data distributions of the model, and note this in the limitations. Of course the model x language random effect does model language specific deviations, e.g. the OLMo models, which are predominantly trained on English, perform better on English than their size and features imply (+1.11 log-odds on average in the model × language effect), and similarly EuroLLM's documented coverage separates supported from unsupported languages (−2.29 vs +0.34 log-odds; Welch p < 0.001, though based on only two unsupported languages), though no effect was found for Apertus.

URIEL itself is also not perfect, but this is perhaps orthogonal question.

We again agree here, sadly many of these measures are highly correlated so including multiple is problematic we found URIEL both covers a reasonable spectrum and is well-known. Since many of these are highly correlated estimates doesn’t vary meaningfully using alternative formulations. The only exception is the distance x reasoning interaction term which under some variations is minor and significant or non-significant (we also see this in the model ablations in Appendix A.1, where it goes from p = .03 to p = .06 once the template × language effect is included).

However, comparison with a simple predictor combining model performance and language resource level would clarify how much the additional features and interactions contribute.

We agree this is a natural baseline to add. We added it to Appendix F (Table 8) and the main paper (§4.2): a binomial regression on model identity and log Common Crawl pages, refitted with each language held out. It recovers most of the gain over the model-mean baseline (language-level MAE 8.5 → 3.4 pp; model-in-language 9.8 → 6.5 pp), and the full model reduces it only slightly further (3.1 and 6.0 pp). The clear further gain comes from evaluating a few templates in the target language: with 10 templates the error falls to 4.4 pp (model-in-language) and 2.0 pp (language mean), 2.2 pp [0.8, 3.6] and 1.4 pp [0.5, 2.5] below the model + resource baseline. We have rephrased our conclusions accordingly.

This also limits how confidently the findings can guide model development. [and] …provided it is presented as an observed pattern rather than a demonstrated causality.

We agree that these are associations from observational data and not causal estimates. We would like to examines causal estimates further in future studies.

Dear authors

Thanks for the reply and discussion. The additions are nice to have for sure, I hope the best for this work!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all reviewers for their careful reading. The reviews led to three substantive changes to the analysis, all now in the revised manuscript; the dataset and the main conclusions are unchanged.

1. The template × language random effect is now the primary model. Reviewers Jimm and WvuV noted that template difficulty varies by language, which was previously explore in Appendix A1 and wondered why it was sitting in the appendix. We agree and have moved it into the main analysis (Equation 1, §3.2). The specification we initially fitted is now reported in Appendix A.1 (Table 2). All fixed effects, tables, figures, the leave-one-language-out and few-shot analyses were refitted. All of the conclusions hold and we see that every effect that was significant remains so, with one exception (distance × reasoning, p = .03 → .06, which was already flagged as fragile in §4.1.1). The main quantitative change is that the initial specification understated the three largest effects by about 10% (size 1.77 → 1.97, resource 0.77 → 0.84, reasoning 0.67 → 0.77) and its standard errors by about 5%. Variance explained is 93% (between-language) and 19% (model-by-language).

2. A simulation study of parameter recovery (Appendix A.2). Simulating from the fitted model and refitting (100 replicates, plus variants with fewer instances per cell and fewer families) shows that the fixed effects are recovered with negligible bias and close to nominal 95% coverage (mean 93%), and that the large variance components (template, template × language, model × language) are recovered to within 1%. The language and family variances are underestimated by 14% and 9%, and the model-level variance (sd 0.17) is weakly identified (we do not interpret it individually in the manuscript). We also refit with alternative family and model random-effect structures (no family, no model, family fixed); the language-level effects are unchanged.

3. A simpler baseline for the prediction analysis (§4.2, Appendix F). Following Reviewer VT6k, we compare the full model against a predictor of model identity and resource level alone. This baseline recovers most of the gain over the naive baseline, and the full model's further improvement is not distinguishable from zero with 15 languages. The clear further gain comes from evaluating about 10 templates in the target language. We have rephrased the abstract, §4.2 and the conclusion accordingly.

Corrections. In the process we audited the numbers in the paper against the code and corrected stale values: the variance-explained percentages quoted in §4.2 and Appendix A.1, the resource-cost figures in Appendix G, the fit statistics in Appendix C, and the pair count in Figure 6. All tables and figures in the revised manuscript are generated from the new primary model and marked in blue on the PDF. We will remove the blue color in the final submission.

Extensibility. Since submission the dataset has grown from 15 to 18 languages (Punjabi, Swedish and Urdu). We are evaluating the new languages and will include them in the final version, together with any further additions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.