acceptodds
Under review as a conference paper at ICLR 2027

Cross-lingual Multi-view On-Policy Self-Distillation:Many Voices, One Model, No New Knowledge

Abstract

Multilingual language models often exhibit weaker reasoning capabilities in low- resource languages. Cross-lingual reasoning can leverage high-resource-language renderings of the same problem at inference time, but doing so requires repeated multilingual decoding and answer aggregation for every query. We ask whether this inference-time benefit can instead be internalized into a model that reasons di- rectly in the target language. We introduce Cross-lingual Multi-view On-Policy Self-Distillation (CMOPD), which uses multiple language versions as privileged training-time views of the same problem. The views provide dense token-level feedback on one low-resource on-policy trajectory, and on-policy distillation con- solidates this feedback into a single target-language policy. After training, the model answers directly from the low-resource query without multilingual decod- ing or answer aggregation. Across Qwen3 1.7B, 4B, 8B backbones and four AfriMGSM target languages (Swahili, Igbo, Yoruba, Amharic), CMOPD with English and Chinese views improves mean pass@12 over our reproduced single- view baseline at every scale (1.7B: +0.2, 4B: +2.6, 8B: +1.8), while per-language effects remain heterogeneous. On Qwen3-8B Swahili, adding the Chinese view yields similar gains during training (+4.8) and test-time synthesis (+4.4).

Primary area:
foundation or frontier models, including LLMs
Keywords:
Cross-lingual reasoning, Multilingual language models, On-policy distillation, Multi-view learning, Low-resource languages, Knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GjFTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates whether inference-time benefit can be internalized into a model that reasons directly in the target language. The authors propose CMOPD an online policy distillation where the privilege information is multiple language versions of the same problem. Therefore, the logits of the high-resource language reasoning is distilled into the low-resource language logits. At inference time, we simply sample the low-resource language model. Experiments on Qwen3 models and two high-resource languages with four low-resource languages show the effectiveness of the method.

In terms of method, given a target-language problem, the student first samples a reasoning trajectory and then a frozen teacher evaluates the same trajectory under each privileged high-resource language information. The internalization is learnt with the average KL divergence for each token and languages.

The experiment section train different Qwen3 models with LoRAs on the COPSD-AfriMGSM dataset (500 problems per language). Each solution is translated into chinese, spanish, and french using Qwen3-4B. Trained models are evaluated on 250 problems in Swahili, Igbo, Yoruba, and Amharic. The baseline methods are the base model, GRPO and COPSD (1 language). Results show that CMOPD outperform the baseline and slightly COPSD (which uses only english instead of english and chinese used as privilege information). Authors ablate what is the best strategy to integrate two high-resource languages (mix batches) and leverage the teacher predictions (logit mean averages) - nothing surprisingly and pretty known strategies.

The method relies heavily on translation, but the translation quality is not evaluated (Qwen3-4B is when Qwen3-8B is included in the experiments). As a consequence, translation errors could propagate into the distilled student. Additionally, training sets are small (and LoRA is used) compared to larger datasets and full fine-tuning. Moreover, it is unclear whether the benefits transfer to other domain like coding.

Overall, this paper is well-written with an incremental method over OPSD (using two languages instead of 1), the gains are modest, and it is unclear whether the method generalizes to other domain. More importantly, the translation quality is not studied.

Critical strengths

Elegant method for test-time multilingual reasoning (especially low-resource languages). Well-written paper.

Critical weaknesses

novelty, limited gains, lack of translation evaluation.

Questions for authors

Could you experiment with another model family? Could you analyze the quality of the translation?

Reviewer_dNGnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper examines a clear performance gap: large language models perform far worse on mathematical reasoning in low-resource languages, such as Swahili and Igbo, than in high-resource languages like English and Chinese. An existing remedy translates each test problem into multiple high-resource languages at test time, conducts separate reasoning paths, and aggregates the answers. While effective, this test-time strategy requires multiple forward generations per problem, leading to high computational costs. This paper asks whether the benefits of such multilingual parallel reasoning can be transferred into the model during training, allowing it to generate the final answer in the target language through a single forward pass.

The authors propose CMOPD, an extension built upon prior work, COPSD. In COPSD, the model solves problems in the target language while the same model, conditioned on the English version of the problem and its reference solution, provides token-level scores to guide each generation step; this supervision signal then trains the target model. CMOPD extends this single-view setup by conditioning on both English and Chinese versions simultaneously (with room to scale to more languages), and averages the token-level predictions across these linguistic perspectives to form the training objective. These translated references appear only during training; at deployment, the model operates independently of them.

The stated contributions are threefold:

  1. Proposing the use of multiple language versions as auxiliary supervision signals during training;
  2. Developing a training framework that integrates these signals into a single model targeting the low-resource language;
  3. Evaluating performance across multiple model scales and languages, along with an analysis of token-level disagreements among different language views.

For empirical evaluation, experiments use the 1.7B, 4B, and 8B variants of Qwen3. The training set consists of 500 math problems per language, evaluated on 250 problems per language across four African languages from AfriMGSM using the pass@12 metric (the proportion of problems solved correctly in at least one of 12 samples). Compared with the authors' own re-implementation of COPSD, the dual-view setup (English plus Chinese) yields average improvements of 0.2, 2.6, and 1.8 percentage points across the three model sizes, outperforming the baseline in 8 out of 12 model-language configurations while underperforming in 4. Ablation studies examine different fusion strategies, the number of language views, and joint fusion versus alternating schedules. Further analysis indicates that predictions from different language views largely agree on numbers and mathematical operators, with disagreements concentrated on surface-level tokens such as punctuation and formatting. In addition, general instruction-following capability shows no noticeable degradation, and dual-view training introduces only about a 3% computational overhead. All reported results stem from a single training run with a single random seed, without reported variance.

Critical strengths

1.Practical problem formulation and economical method design. Moving multilingual reasoning benefits from inference time to the training stage so that the deployed model needs only a single forward pass in the target language serves a practical purpose for low-resource language users. The method introduces no additional teacher models and adds only gradient-free forward computation on the same frozen backbone, increasing dual-view training overhead by only about 2.8% (Table 5). This makes the overall research direction worth pursuing even if the current experiments remain incomplete.

2.Clean control experiments showing within-update parallel fusion outperforms sequential scheduling. Under identical settings, Table 2 compares parallel logit fusion (79.2), alternating teachers (75.6), and blocked curriculum scheduling (72.8). This shows that if multi-view supervision helps, the gain comes from token-level joint supervision rather than simple data mixing, providing the most convincing set of controls in the paper.

3.Informative analysis and restrained reporting. Figure 3 shows that different language views show near-perfect agreement on numbers and mathematical operators, while disagreements concentrate on punctuation and formatting tokens. Figure 5 shows that cross-view disagreement positively correlates with reasoning divergence points. The authors also point out that the normalization term in the geometric mean does not depend on individual tokens and cannot serve as evidence of beneficial correction (Section 5.3). Such honest reporting improves trust in the rest of the paper.

Critical weaknesses

1.Lack of evaluation metrics matching the core claim. The paper repeatedly claims that the model can answer questions in a single forward pass (Abstract, Figure 2, Section 3.4), yet all reported accuracy numbers use pass@12, which measures whether at least one out of 12 samples is correct. This metric is sensitive to output diversity, meaning multi-view fusion could raise pass@12 simply by increasing sample dispersion rather than improving single-pass correctness. Without pass@1 or avg@12, the primary claim of the paper remains unverified, which forms the main reason for my rejection recommendation.

  1. Effect sizes indistinguishable from experimental noise. All results come from a single checkpoint trained with a single random seed, without reported variance. In Figure 4a, logit averaging and hidden averaging, which the authors treat as implementation-level duplicates, differ by 1.2 percentage points. This self-exposed noise level is six times the average gain of the 1.7B model (+0.2, corresponding to only 2 more correct cases out of 1,000 test examples). Across the 12 comparisons in Table 1, the model wins 8 and loses 4, and the languages that gain and lose between 1.7B and 8B are largely reversed, which matches random fluctuation rather than systematic transfer.

  2. Ablation results contradicting the main narrative. In Figure 4b, increasing the number of views from 1 to 4 produces pass@12 results of 29.6, 29.2, 24.4, and 31.2. Adding Spanish causes a drop of 5.2 percentage points, which the text leaves unexplained while simply stating that four views perform best. Furthermore, all K=2 fusion methods in Figure 4a (27.2 to 29.2) score lower than the K=1 baseline in the same setting (29.6), meaning the configuration used for the ablation study fails to support the value of multi-view training.

4.Missing baselines and controls needed for proper attribution. The paper provides no Chinese-only K=1 baseline, making it impossible to separate benefits of fusion from the possibility that the Chinese teacher is simply stronger. It also lacks a K=2 control using two English paraphrases, leaving cross-lingual complementarity indistinguishable from general multi-context ensembling. The same problem appears on the inference side: the oracle union in Figure 1 uses 48 samples against 12 for English, and the EN+ZH composition in Table 3 uses three calls against one, with neither providing an equal-budget English-only baseline.

5.Training gains lower than simple test-time alternatives. In Table 3 on 8B Swahili, CMOPD reaches 79.2 after training, whereas translating the problem into English at test time and generating an answer once reaches 81.2. The core motivation is to avoid multilingual computation at test time, but single-pass English reasoning already outperforms the trained model, making the argument for transferring gains into training unconvincing. Additionally, the analysis of language pairs promised at the end of Section 4.3 is missing from the main text despite being claimed in the conclusion, and Table 5 lists an ES+FR setting without corresponding accuracy numbers, suggesting incomplete revisions.

Questions for authors
  1. What are the pass@1 (greedy decoding) and avg@12 (average accuracy over 12 samples) scores for each checkpoint in Table 1 and Table 2? Does the advantage of CMOPD over COPSD still hold under single-pass generation? This is the most critical question in determining whether I change my recommendation.
  2. Can the authors repeat the Table 1 runs for COPSD and CMOPD across at least three training seeds and report the mean and standard deviation? In Figure 4a, logit averaging and hidden averaging differ by 1.2 percentage points even though they should yield identical results. What is the source of this gap: non-determinism in vLLM sampling during training, evaluation sampling, or implementation details?
  3. What is the K=1 training performance using only the Chinese view? In addition, what are the results when using two different English paraphrases as K=2 views? Without these two controls, it is impossible to determine whether the gains stem from cross-lingual complementarity, general multi-context ensembling, or a stronger Chinese teacher.
  4. Table 5 indicates that the authors trained K=4 and ES+FR configurations on 8B Swahili; what are their pass@12 scores? Furthermore, how do the authors explain the 4.8 percentage point drop on the 1.7B model when Spanish is introduced in Figure 4b?
  5. Can the authors provide equal-budget English-only baselines for Figure 1b and Table 3, specifically an oracle union over 48 English samples, and an EN+EN synthesis combining two English reasoning paths with one synthesis call?
  6. Clarification: The end of Section 4.3 notes an upcoming analysis on language-pair identity, and the conclusion also claims to characterize the impact of language-pair choices, yet this content is absent from the main text. Was this analysis moved to the supplementary materials, or removed during revisions? If it exists, please clarify the main findings.
  7. Clarification: The eight token categories in Figure 3 cover a total of 3,456 positions, whereas the text mentions analyzing 15,360 positions. How were the remaining ~77% of positions (presumably standard Swahili words) handled? In addition, Figure 3 and Figure 5 use Qwen3-8B-Instruct, while the main experiments use Qwen3-8B. Is this a typo, or was a different model used intentionally?
  8. Clarification: What is the exact procedure for "English CoT" in Table 3? Does it involve translating the Swahili problem into English before reasoning, or prompting the model to reason in English directly from the Swahili prompt? Were the Chinese, Spanish, and French views translated from the original English problems, or from the low-resource language versions?
Reviewer_c8bvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether the benefit of multilingual test-time reasoning—generating and aggregating reasoning traces in several high-resource languages for every query—can be internalized into a single target-language policy that answers the low-resource query directly. It introduces Cross-lingual Multi-view On-Policy Self-Distillation (CMOPD): for each low-resource problem, machine-translated renderings of the problem and reference solution in K high-resource languages serve as privileged training-time views for a frozen self-teacher. The student samples one on-policy target-language trajectory; the teacher scores each next token under every view; the per-view logits are averaged into a single fused target, equivalent to a normalized geometric mean; and the student adapter is trained by KL divergence to this target. Inference uses one decoding path in the target language with no translation or answer aggregation. Validation on AfriMGSM in Swahili, Igbo, Yoruba, and Amharic across Qwen3-1.7B/4B/8B shows EN+ZH CMOPD improves mean pass@12 over the authors' reproduced single-view COPSD baseline by +0.2, +2.6, and +1.8 at the three scales, winning 8 of 12 model–language cells; the largest gain is 8B Swahili, improving from 74.4 to 79.2. Multilingual test-time synthesis on the same setup improves by +4.4, which the authors present as evidence that the inference-time benefit is largely internalized. Ablations favor parallel logit fusion over sequential view use, and the four-view set achieves the best results on SWA and IBO, though not monotonically: SWA dips to 24.4 at K=3, below the single-view 29.6. Token-level analyses localize cross-view disagreement to wrapper tokens such as punctuation, step markers, and meta phrases rather than mathematical content, and correlate disagreement with unstable continuations, with Spearman ρ = 0.289. IFEval shows no instruction-following degradation, and EN+ZH adds only 2.8% training cost. All results are obtained from a single trained checkpoint per configuration under the default seed 42, on 250-problem test sets, with teacher views translated by Qwen3-4B.

Critical strengths
  • S1. The paper asks a well-posed question and answers it with a clean matched comparison: Table 3 shows adding a Chinese view yields +4.8 in training-time distillation versus +4.4 from inference-time EN+ZH synthesis on 8B SWA, directly quantifying how much multilingual test-time benefit is internalized into the target-language policy. This is the paper's central claim, and the experimental design isolates it convincingly.

  • S2. The analysis goes beyond headline numbers: token-level cross-view disagreement concentrates on wrapper tokens (punctuation, step markers, meta phrases) rather than mathematical content, and higher cross-view KL marks prefixes where continuations fork in correctness (Fig. 3, Fig. 5). Combined with honest reporting of heterogeneous per-language effects and low overhead (+2.8% for two views), this characterizes both the mechanism and the cost of the method.

Critical weaknesses
  • W1. The central claims are not statistically supported. Every result derives from a single trained checkpoint under one seed (42), evaluated on 250 problems with pass@12, and the paper reports no seed variance or significance test. The average gains are +0.2 / +2.6 / +1.8 across scales, with 1.7B (+0.2) almost certainly within noise — at that scale CMOPD wins two cells (IBO +1.6, AMH +1.6) and loses two (SWA −0.4, YOR −2.0), so the positive average is a cancellation, not a consistent effect. CMOPD loses in four of twelve model–language cells (8B AMH −2.4, 1.7B YOR −2.0, 1.7B SWA −0.4, 8B IBO −0.4). "Improves at every scale" is therefore an assertion about a point estimate whose uncertainty is unmeasured.

  • W2. No experiment falsifies the stated mechanism. Every ablation varies the fusion operator or the view count while keeping the translations valid, so none distinguishes cross-lingual complementarity from the generic variance-reduction effect of averaging multiple logit streams into one teacher target. Coupled with a 500-problem training set per language and teacher views machine-translated by Qwen3-4B, whose errors the authors disclose but never ablate, the evidence supports "ensembling two logit streams helps on 8B Swahili," not the broader internalization claim.

Questions for authors
  • Q1. Please clarify the denominator in Fig. 3: the displayed token-role counts sum to 3,456, not 15,360. Are the remaining 11,904 positions omitted as an “other” category? If so, please state this explicitly. Also, please specify whether Fig. 1 uses the same model variant, prompt/template, decoding parameters, and evaluation seed as the Base row in Table 1; the IBO pass@12 differs from 3.6 to 5.2.

  • Q2. All training runs use the Hugging Face default seed 42, and §4.1 states that each result comes from a single trained checkpoint. On 250 test problems, what is the run-to-run variance of the Table 1 gaps, in particular the 8B SWA +4.8 and the 1.7B average +0.2? Without seed variance or a paired significance test, it is unclear whether "improves mean pass@12 at every scale" is distinguishable from noise, especially given the negative cells (8B AMH −2.4, 1.7B YOR −2.0).

  • Q3. The fused teacher is produced by averaging the logit streams of multiple views, which is itself an ensemble. Averaging multiple predictions can improve the target even when the predictions carry no cross-lingual information. Do you have any control showing that the gain comes specifically from the views being in different languages, rather than from ensembling as such? If not, the internalization claim should be stated with this caveat.

  • Q4. A direct test of the mechanism would be an arm in which the privileged views are corrupted while the pipeline is unchanged, for example by shuffling the translations across problems or replacing them with translations of unrelated problems. If the CMOPD gain over COPSD persists under corrupted views, then the improvement comes from ensembling or extra computation rather than from cross-lingual information. Do you have results from any such arm? If not, can you add one?

1 comment

Strengths

The paper asks a clear question: can the benefits of multilingual reasoning be built into training so that a model can answer directly in the target language? The approach is intuitive, and the experiments cover several model sizes and languages. The paper also looks at differences between language views and reports training costs. On 8B Swahili, CMOPD improves pass@12 by 4.8 points, with an additional training cost of about 2.8%.

Main Concerns

Each setting uses only one training run, and each language is tested on just 250 problems. The paper does not report uncertainty, so it is hard to know whether the gains are reliable. The 1.7B average gain is only 0.2 points and could be due to random variation; CMOPD also performs worse in 4 of the 12 model–language comparisons.

The paper compares a +4.8-point gain from training with a +4.4-point gain from multilingual reasoning at inference. But the two gains use different baselines, and inference-time synthesis requires multiple model calls. This comparison alone does not show that training has “internalized” the benefit of multilingual reasoning.

It is also unclear whether the gains come from cross-lingual information or simply from averaging multiple predictions. The translations used for training were generated by Qwen3-4B, but the paper does not separately test the effect of translation quality.

Questions for the Authors

  1. How much do results vary across random seeds? In particular, are the +0.2 points for 1.7B and +4.8 points for 8B Swahili consistent across runs?
  2. Do the gains remain if you use different paraphrases in the same language, or pair each problem with another problem’s translation? This would help show whether the gains really come from relevant cross-lingual information.
  3. The counts shown in Figure 3 add up to 3,456, while the text says 15,360 token positions. Also, Figure 1 reports 5.2% for Igbo, while Table 1 reports 3.6% for the base model. What explains these differences? Were the evaluation settings the same?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.