acceptodds
Under review as a conference paper at ICLR 2027

Teach Only What Is Missing: Frontier-Calibrated Counterfactual Self-Distillation for Multilingual Reasoning

Abstract

An instruction-tuned language model that solves a math question posed in English often fails the same question posed in a low-resource language, although its reasoning stays English-centered inside. Existing methods add a cross-lingual signal at the input, by translating the question or the training solutions, or at the answer, by preferring English-derived responses or rewarding correct ones. None of them locates where, inside the reasoning, the target-language question falls short, and none uses the model's own English-conditioned distribution as the target. We start from a simple test: a prefix of the model's own verified English reasoning, placed before the target-language question's reasoning, restores the correct answer, and the length needed differs across questions and languages. We propose Counterfactual English-Conditioned Ladder Training (CELT), a self-distillation method that trains the model toward its own English-conditioned distribution, only on the missing span. To locate the span, Frontier Measurement probes the model on the target-language question with a growing number of steps of the English trace supplied, and records the smallest sufficient count as the frontier. To close it, Counterfactual Distillation matches the next-token distributions of the adapted model under the target-language question to those of the frozen model under the English question, with full weight on the missing span, from several starting depths. To keep what the model already does and to help it read the question, we add Own-Solution Preservation and Question Reading. At inference nothing is added. Across three backbones and three benchmarks, CELT is the most accurate method in every language, with the largest margins on low-resource languages, and improves English as well, because the supervision comes from the model itself and lands where each question's reasoning is missing.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multilingual reasoning, self-distillation, chain-of-thought, low-resource languages, knowledge distillation, large language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hq4COpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CELT to address missing reasoning spans in multilingual reasoning. It uses counterfactual distillation to align target-language reasoning with English teacher distributions while preserving existing capabilities. Experiments across three backbones and benchmarks show consistent improvements over seven baselines, particularly in low-resource languages.

Critical strengths

The paper provides a clear and empirically grounded characterization of missing reasoning spans in multilingual reasoning. Its Frontier Measurement offers a useful fine-grained diagnostic beyond the proposed method. CELT also achieves strong and consistent improvements across models, benchmarks, and languages, particularly in low-resource settings.

Critical weaknesses

1.The paper lacks a comparison with directly fine-tuning on English questions and reasoning traces, making it unclear whether the gains stem from counterfactual distillation or additional English supervision.

2.The paper does not analyze unreachable cases, leaving it unclear whether these failures arise from language understanding, mathematical computation, or reasoning limitations.

3.Frontier Measurement requires substantial additional decoding, but the paper does not adequately report its computational cost or wall-clock time.

4.The reading loss is essentially a question-translation objective, but the paper does not establish whether its benefits come from improved question understanding or general cross-lingual representation alignment.

Questions for authors
  1. How does CELT compare with directly fine-tuning on English questions and reasoning traces under the same training budget?

  2. What causes the unreachable cases: language understanding, mathematical computation, or reasoning limitations?

  3. What are the computational cost and wall-clock time of Frontier Measurement?

  4. Does the reading loss improve question understanding or primarily enhance cross-lingual representation alignment?

4 comments
AuthorsOpenReview ↗

Fine-tuning on English questions and traces under the same training budget

  1. The paper lacks a comparison with directly fine-tuning on English questions and reasoning traces, making it unclear whether the gains stem from counterfactual distillation or additional English supervision.
  1. How does CELT compare with directly fine-tuning on English questions and reasoning traces under the same training budget?

Thank you, this is the natural alternative to rule out. We ran two controls on all three backbones of Table 1. Each uses the same optimizer updates and training tokens as CELT on that backbone, the same adapter and the same three seeds.

  • English SFT: token likelihood of the model's own verified English traces under the English questions. This is CELT's preservation stream, filled to the budget with further verified samples. We run it without and with CELT's reading rows (+ reading).
  • Cross-lingual SFT: token likelihood of the model's own English trace under the target-language question, for every failing cell with a reference trace. Preservation and reading stay as in CELT, so only the distillation stream is replaced.

AVG (%), mean of three seeds; 3B and 7B are the Qwen2.5 backbones, L8B is Llama3-8B:

3B MGSM3B MATH3B CSQA7B MGSM7B MATH7B CSQAL8B MGSML8B MATHL8B CSQA
Base52.4444.1144.5567.1657.8954.2752.2226.0045.12
English SFT55.3147.6747.1469.5861.3956.1454.4526.8547.48
English SFT + reading61.0949.9748.5873.3562.1157.6060.0727.5049.10
Cross-lingual SFT62.5550.6349.6874.2762.6558.3961.9526.7649.95
CELT72.9158.6055.9881.7568.1163.6571.5628.5156.38

On Qwen2.5-7B-Instruct MGSM, with kept and recovered as in Table 2:

ENURLkeptrecovered
Base89.2048.801000
English SFT94.8051.3096.7811.02
English SFT + reading94.5357.1098.4625.50
Cross-lingual SFT91.7361.1095.8940.96
CELT96.1371.6099.0464.45

What we see.

First, more English supervision raises English but barely reaches the target languages. English SFT reaches 94.80 on English MGSM and 78.47 on English MATH500 (Base 70.80, CELT 78.80), and recovers 11.02% of the frontier cells.

Second, the same English traces as hard targets under the target-language question recover 40.96%. CELT recovers 23.49 points more. It differs in its target and in its rows. The hard-token row of Appendix E keeps CELT's rows and recovers 56.27, so the rows account for 15.31 points and the soft target for 8.18. Both hard-target controls also lose some cells that the frozen model answers, while CELT keeps 99.04%. So cross-lingual SFT recovers more cells than the w/o LdistL_{dist} row of Table 2 (40.96 against 28.90) but keeps fewer (95.89 against 98.77). Both reach the same AVG (74.27 and 74.32).

Third, in the nine columns CELT is 1.66 to 17.60 points above English SFT and 1.01 to 11.82 above it with reading. It is 1.75 to 10.36 above cross-lingual SFT. The remaining cell, with preservation and reading kept and only distillation removed, is the w/o LdistL_{dist} row of Table 2: 74.32 AVG and 28.90 recovered. In CELT, English enters only through preservation, and no distillation row has an English question on the student side (Section 4.2). Under the same budget, more English supervision therefore does not explain the gain.

Revision. We will add both controls on all three backbones to Table 2 and Appendix B. We hope this answers the question, and we are happy to add further controls.

AuthorsOpenReview ↗

What causes the unreachable cells

  1. The paper does not analyze unreachable cases, leaving it unclear whether these failures arise from language understanding, mathematical computation, or reasoning limitations.
  1. What causes the unreachable cases: language understanding, mathematical computation, or reasoning limitations?

Thank you for this question. We annotated every unreachable MGSM test cell of the three backbones, 751 cells in total (Table C.1). Two annotators per language, fluent in it and in English, read the English and target-language questions and the model's own English solution. They also read the frozen model's output and its continuation from the whole English solution. The order of the causes was fixed before annotation, and each cell gets the first applicable cause:

  • Language understanding: a quantity, a relation or the asked quantity of the target-language question is misread.
  • Computation: the question is read correctly and an arithmetic step is wrong.
  • Reasoning: the question is read correctly and the arithmetic is right, but the plan is wrong.
  • Question divergence: the benchmark's target-language question states a different number or relation than the English one.
  • Answer form: the right value in another unit, rounding or form.

Each label describes the continuation from the whole English solution. In computation cells it recomputes a supplied step wrongly, and in reasoning cells it drops the supplied plan.

The annotators agreed on 89.21% of the cells (Cohen's kappa 0.84). Cells per cause:

BackboneGroupcellsunderstandingcomputationreasoningquestion divergenceanswer form
Qwen2.5-3B-InstructHRL1254917301613
Qwen2.5-3B-InstructURL1216412211410
Qwen2.5-7B-InstructHRL1024411221510
Qwen2.5-7B-InstructURL114638171610
Llama3-8B-InstructHRL1556322371815
Llama3-8B-InstructURL1346915241511
allall751352 (46.87%)85 (11.32%)151 (20.11%)94 (12.52%)69 (9.19%)

Which method answers which cause. On Qwen2.5-7B-Instruct MGSM, we scored the unreachable cells under the runs of Table 1. We also trained polyGRPO on the merged CELT model, with three seeds. Correct cell-runs (%), both groups pooled; the frozen model answers none of these cells:

understandingcomputationreasoningquestion divergenceanswer formall
CELT15.588.7710.262.1510.0011.57
polyGRPO12.4645.6138.466.4571.6724.69
CELT, then polyGRPO17.1340.3540.177.5365.0026.39

Findings.

First, language understanding is the largest cause on every backbone and in both groups, 39.20% to 55.26% of the cells and 46.87% overall. Computation accounts for 11.32% and reasoning for 20.11%.

Second, the share of understanding does not come from the order of the causes. In a free ranking, with the causes shown in random order, the main cause matches the ordered label in 732 of 751 cells (97.47%). Understanding stays the largest cause everywhere, at 44.34% overall.

Third, two automatic checks, which look at the same misreading in another output, are consistent with the label. The frozen model restated each target-language question in English. The restatement repeats the misreading in 77.27% to 81.25% of the understanding cells, against 6.58% to 8.77% of the other cells. In 85.71% to 87.50% of the understanding cells, the continuation from the whole English solution restates the misread quantity or relation and computes from it.

Fourth, this explains why no prefix helps the understanding cells. After the supplied steps, the model reads the question again and returns to its misreading, so even the whole English solution does not bring it to the answer. In a further 12.52% of the cells, the benchmark's own target-language question differs from the English one.

Fifth, each method does better on the causes it trains on. CELT answers more understanding cells than polyGRPO, 15.58% against 12.46%, the cause Question Reading targets. The answer reward of polyGRPO answers more computation, reasoning and answer-form cells, where the question is read correctly. CELT has no distillation row for unreachable cells, of which 19 are computation and 39 reasoning cells on Qwen2.5-7B-Instruct. The two compose: polyGRPO after CELT answers the most unreachable cell-runs and raises MGSM AVG from 81.75 to 83.83.

Revision. We will add this analysis to Appendix C, with the free ranking, the two checks, examples of each cause and the table by method. It will also list the MGSM items of the divergence and answer-form cells.

AuthorsOpenReview ↗

Computational cost and wall-clock time of Frontier Measurement

  1. Frontier Measurement requires substantial additional decoding, but the paper does not adequately report its computational cost or wall-clock time.
  1. What are the computational cost and wall-clock time of Frontier Measurement?

Thank you for asking for concrete numbers. We measured Frontier Measurement and training on one node with 8 A100 80GB GPUs. Decoding uses vLLM (greedy, prefix caching) and training the LoRA configuration of Appendix A.4, both data parallel over the 8 GPUs. GPU-hours are busy time summed over the GPUs, and wall-clock on the node includes about 5% for scheduling and loading. Frontier Measurement has two stages. The own traces decode every cell once (82203 decodings). The prefix probes decode only failing cells with a reference trace (Table A.1).

Backboneown traces: tokens (M)GPU-hwall-clock (min)prefix probes: decodingstokens (M)GPU-hwall-clock (min)training: GPU-hwall-clock (min)
Qwen2.5-3B-Instruct23.351.259.87930015.300.826.44.6236.4
Qwen2.5-7B-Instruct21.541.9315.26173810.990.987.88.1464.1
Llama3-8B-Instruct24.742.4519.38861718.171.8014.29.3873.9

Tokens are generated tokens.

What the numbers say.

First, Frontier Measurement is short. The probes take 6.4 to 14.2 minutes of wall-clock on one node, 8.9% to 13.2% of CELT's total GPU-hours (own traces, probes and training). The own traces take 9.8 to 19.3 minutes, and preservation uses them as well.

Second, CELT as a whole is cheaper than every trained baseline. On Qwen2.5-7B-Instruct the full pipeline takes 11.05 GPU-hours on the same node. The baselines take 19.64 (xRFT), 24.31 (LIDR), 21.73 (MAPO), 52.86 (GRPO) and 61.47 (polyGRPO). These include their own sampling, translation or rollouts.

Third, the measurement runs once before training and is shared by all seeds and design variants. Its size grows only with the failing cells, so the backbone that fails fewer cells needs fewer probes: 61738 on Qwen2.5-7B-Instruct against 79300 on Qwen2.5-3B-Instruct.

Revision. We will add this table to Appendix A.2 and the total cost of every method to Appendix A.7. We hope this gives the cost picture you asked for.

AuthorsOpenReview ↗

Question understanding and representation alignment under the reading loss

  1. The reading loss is essentially a question-translation objective, but the paper does not establish whether its benefits come from improved question understanding or general cross-lingual representation alignment.
  1. Does the reading loss improve question understanding or primarily enhance cross-lingual representation alignment?

Thank you, this is a good question, and the two explanations predict different things. We separated them with a control and with direct measures of each.

Control. CELT with LreadL_{read} replaced by an alignment loss LalignL_{align}. For every reading row, LalignL_{align} is one minus the cosine similarity of two mean-pooled hidden states, averaged over layers 10 to 18 of 28. The two states come from the target-language question and from the English question (stop-gradient). LalignL_{align} optimizes alignment directly and never produces a restatement. It has the unit row weight of LreadL_{read}, and the layers were fixed in advance. We also ran weights 0.5 and 2, and all layers instead of 10 to 18.

Measures (Qwen2.5-7B-Instruct, MGSM test set, three seeds):

  • Restatement fidelity: the share of target-language test questions whose English restatement keeps every quantity, every relation and the asked quantity. Qwen2.5-72B-Instruct judges it against the English question, and agreed with two annotators on 191 of 200 pairs.
  • Misread rate: the share of outputs on the 481 cells with a frontier that misread a quantity or relation, same judge.
  • Alignment: cosine similarity of mean-pooled hidden states at layer 14, and top-1 retrieval of the parallel question among the 250 English questions.
restatement fidelity, HRLrestatement fidelity, URLmisread ratecosine, URLretrieval, URLMGSM URLrecovered
Base88.4061.7071.100.61257.3048.800.00
CELT w/o LreadL_{read} (Table 2)88.9163.2033.680.64160.8065.0750.94
CELT with LalignL_{align}89.1164.0031.950.88795.6065.8352.39
CELT93.6082.3018.300.68968.9071.6064.45

Findings.

First, the gain of LreadL_{read} goes with better question understanding. Relative to CELT without it, LreadL_{read} raises URL restatement fidelity by 19.10 points. Restatement is close to what LreadL_{read} trains. The stronger evidence is the misread rate on the task outputs, which LreadL_{read} does not train directly: it falls by 15.38 points. The URL cosine rises by only 0.048.

Second, alignment alone does not deliver the gain. LalignL_{align} raises the URL cosine by 0.246, about five times as much, and URL retrieval to 95.60. Yet restatement fidelity rises by only 0.80 and URL accuracy by 0.76, against 6.53 for LreadL_{read}. Across the six weight and layer settings, LalignL_{align} raises them by at most 1.10 and 0.90, while the URL cosine reaches 0.851 to 0.926.

So the reading loss improves how the target-language question is understood, and the alignment it adds is small.

Revision. We will add the LalignL_{align} control, its sweep and these measures to Appendix E. We hope this separates the two explanations clearly.

Reviewer_MUkTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CELT to address multilingual reasoning gaps by identifying missing reasoning spans that models fail to elicit from low-resource language questions. It uses Frontier Measurement to locate these spans and selectively distills the model’s own English-conditioned reasoning into target languages, while preserving existing capabilities. Experiments across three backbones and three benchmarks demonstrate consistent improvements over baselines, particularly in low-resource languages, without additional inference-time overhead.

Critical strengths

1. Fine-Grained Approach to Multilingual Reasoning Gaps

Unlike existing methods that supervise entire reasoning traces or final answers, CELT identifies missing reasoning spans for each question and language, enabling more targeted supervision.

2. Self-Improvement Without Additional Reasoning Annotations

The proposed method improves reasoning performance in low-resource languages by leveraging the model's own English reasoning capabilities, without requiring externally annotated reasoning traces. This aligns well with the growing interest in self-improvement and self-distillation for LLMs.

3. Consistent Performance Improvements Without Inference Overhead

CELT consistently outperforms baselines across three backbones and three benchmarks, with particularly strong gains in low-resource languages. Moreover, it introduces no additional inference-time overhead.

Critical weaknesses

1. High Computational Cost of Frontier Measurement

CELT requires multiple inference passes for each question-language pair to identify the missing reasoning span. Although Appendix A reports approximately 61K–89K additional probe decodings per backbone, the paper does not provide a detailed analysis of the computational cost relative to the overall training process. Therefore, it remains unclear whether the performance gains justify the additional computational overhead, especially when scaling to larger datasets or more languages.

2. Limited Distinction from Translation-Based Approaches

The proposed method aims to make low-resource language questions elicit the same reasoning behavior as English questions, while the reasoning itself is performed in English. From this perspective, CELT appears conceptually similar to translating the input question into English and leveraging the model's existing reasoning capabilities. This raises a fundamental question about what the model actually learns beyond what can be achieved through input translation. Although the paper includes Translate-Test as a baseline and reports better performance with CELT, it does not sufficiently explain the fundamental difference between these approaches or what additional capabilities CELT provides. Given that this comparison is closely related to the paper's core motivation, a more detailed analysis and discussion in the main text, rather than relying primarily on results in the appendix, would help clarify the necessity and contribution of the proposed method.

3. Limited Novelty and Missing Comparisons with Related Self-Distillation Methods

The proposed approach shares substantial similarities with existing self-distillation methods, particularly those that improve reasoning by distilling a model's own predictions [1, 2]. While CELT introduces Frontier Measurement and span-weighted supervision, the underlying self-distillation framework is not fundamentally different from prior work. However, the paper neither discusses these closely related studies nor provides experimental comparisons with them. This makes it difficult to assess the novelty of the proposed method and whether its additional complexity offers meaningful advantages over existing self-distillation approaches.

References

[1] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models.

[2] Self-Supervised On-Policy Distillation for Reasoning Language Models.

Questions for authors

N/A

6 comments
AuthorsOpenReview ↗

Cost of Frontier Measurement relative to the whole training process

CELT requires multiple inference passes for each question-language pair to identify the missing reasoning span. Although Appendix A reports approximately 61K–89K additional probe decodings per backbone, the paper does not provide a detailed analysis of the computational cost relative to the overall training process. Therefore, it remains unclear whether the performance gains justify the additional computational overhead, especially when scaling to larger datasets or more languages.

Thank you for asking for the cost in full. In short, the probes are a small part of it, and CELT as a whole is the cheapest trained method. We measured every stage of CELT on one node with 8 A100 80GB GPUs. Decoding uses vLLM (greedy, prefix caching) and training the LoRA configuration of Appendix A.4, both data parallel over the 8 GPUs. GPU-hours are busy time summed over the GPUs. Wall-clock is on the node and includes about 5% for scheduling and loading.

Backboneown traces (GPU-h)prefix probes (decodings)prefix probes (GPU-h)prefix probes (wall-clock, min)training (GPU-h)CELT total (GPU-h)probes, share of total
Qwen2.5-3B-Instruct1.25793000.826.44.626.6812.2%
Qwen2.5-7B-Instruct1.93617380.987.88.1411.058.9%
Llama3-8B-Instruct2.45886171.8014.29.3813.6413.2%

Total GPU-hours of each trained method on Qwen2.5-7B-Instruct, same node, including its own sampling, translation or rollouts:

xRFTLIDRMAPOGRPOpolyGRPOOPSDSSOPDCELT
19.6424.3121.7352.8661.4715.9258.3911.05

OPSD and SSOPD are two self-distillation methods we now compare with as well. The configurations behind these numbers, which Appendix A.7 will list:

  • xRFT: 8 sampled paths per question and language.
  • LIDR: 2 iterations of preference optimisation, 4 responses per question and language.
  • MAPO: 4 responses per question and language, scored by NLLB-200 1.3B.
  • GRPO and polyGRPO: groups of 8 rollouts, 64 questions per step, 1000 steps.
  • OPSD and SSOPD: one and four student rollouts per question and language, two epochs.

What the numbers say.

First, the probes are a small part of the cost. They take 6.4 to 14.2 minutes on one node and 8.9% to 13.2% of CELT's total GPU-hours (own traces, probes and training). The own traces, one greedy pass per cell, also serve preservation. Frontier Measurement as a whole, own traces and probes, takes 26.4% to 31.2%.

Second, CELT as a whole is the cheapest trained method, at 11.05 GPU-hours on Qwen2.5-7B-Instruct against 15.92 to 61.47 for the baselines. It is also the most accurate one in Table 1. At equal compute the gap is larger. The checkpoints of their own runs at CELT's 11.05 GPU-hours reach 71.67 MGSM AVG for polyGRPO and 70.63 for GRPO, against 81.75. With the learning-rate schedule fitted to this budget instead, they reach 72.61 and 71.48. The reinforcement-learning baselines sample a group of rollouts for every question, while CELT decodes each cell once and probes only the failing ones.

Third, the cost scales well.

  • The probes run once, before training, and are shared by all seeds and design variants.
  • Their number grows with the failing cells, not with all cells, so it falls on a stronger backbone: 61738 on Qwen2.5-7B-Instruct against 79300 on Qwen2.5-3B-Instruct.
  • Among the ten target languages, the most expensive one adds 15096 probes and 0.241 GPU-hours on Qwen2.5-7B-Instruct. This is Swahili, the language with the deepest frontiers.
  • On a larger dataset the probes grow linearly with the failing cells, like the distillation rows they produce.
  • On long reasoning they grow with the steps of the trace. With thinking mode on Qwen3-8B, a reference trace has 31.4 steps, and the probes grow from 49312 to 182629 decodings. Frontier Measurement then takes 42.3% of CELT's 50.75 GPU-hours, below the 264.8 of polyGRPO in the same mode.
  • More languages add little. We translated the training families into Arabic, Hindi, Urdu, Vietnamese and Portuguese with NLLB-200, under the number check of Appendix A.1 used for Telugu. This adds 33381 probes, 0.53 GPU-hours of probing and 1.92 of training. It raises their X-CSQA accuracy by 1.16 to 5.17 points, while MGSM and MATH500 stay at 81.65 and 68.18.

Revision. We will add this accounting to Appendix A.2 and the five added languages to Appendix E.

AuthorsOpenReview ↗

What CELT learns beyond translating the question

The proposed method aims to make low-resource language questions elicit the same reasoning behavior as English questions, while the reasoning itself is performed in English. From this perspective, CELT appears conceptually similar to translating the input question into English and leveraging the model's existing reasoning capabilities. This raises a fundamental question about what the model actually learns beyond what can be achieved through input translation. Although the paper includes Translate-Test as a baseline and reports better performance with CELT, it does not sufficiently explain the fundamental difference between these approaches or what additional capabilities CELT provides. Given that this comparison is closely related to the paper's core motivation, a more detailed analysis and discussion in the main text, rather than relying primarily on results in the appendix, would help clarify the necessity and contribution of the proposed method.

Thank you, this is the central question, and we agree it deserves a direct answer in the main text. Translate-Test changes the input at inference: a translator rewrites the question, and the model reasons on the rewrite. CELT changes the model: the target-language question itself is trained to call out the model's own English-conditioned solution. Three measurements on Qwen2.5-7B-Instruct and MGSM separate the two.

1. Where the answers change. Correct cells (%) in the classes of Figure 3a, from the runs of Table 1:

GroupMethodself-sufficient cells (kept)cells with a frontierunreachable cellscells without a reference trace
HRLTranslate-Test93.9345.4024.516.06
HRLCELT99.3069.3312.0938.72
URLTranslate-Test89.9626.7318.423.75
URLCELT98.4361.9511.1132.50

On the translated question, the model fails more of the questions it already solves, and it recovers fewer missing spans. Translate-Test answers more of the unreachable cells, 21.30% against 11.57%. There even the whole English solution does not help, so CELT has no distillation row for them. A rewritten question removes some of the misread clauses the model returns to.

2. Distance to the English condition. We repeated the measurement of Figure 3b with the Translate-Test input as the student condition. The frozen model reads the translated question and is forced along its own English solution. Forward KL to the frozen model under the English question, in nats per token:

GroupStudent conditioninside the missing spanbeyond the frontier
HRLfrozen model, target-language question0.4350.095
HRLfrozen model, translated question0.1980.081
HRLCELT, target-language question0.1040.073
URLfrozen model, target-language question0.6880.129
URLfrozen model, translated question0.3520.108
URLCELT, target-language question0.1440.100

In the low-resource languages, the translated question closes 48.8% of the in-span divergence and CELT, without any translation, closes 79.1%. Inside the span and beyond it, the target-language condition of CELT is closer to the English condition than a translation of the question is. This measurement locates where the gap closes, and the accuracy above is the evidence that it matters.

Another translator does not change this. Translate-Test in Table 1 follows the Translate-EN setting of Shi et al. (Appendix A.5). With NLLB-200 3.3B as the translator instead, Translate-Test changes by 0.33 to 0.55 points on the three benchmarks of Qwen2.5-7B-Instruct and stays 6.24 to 10.73 points below CELT.

3. Translation as training. The w/o LdistL_{dist} row of Table 2 trains on the two ingredients of question translation training. These are reading the question into English and the model's own solutions. It recovers 28.90% of the frontier cells, against 64.45% for CELT.

So CELT does more than translate the question, at inference or in training. In many frontier cells the span begins at a clause the model reads differently (Appendix F). What CELT trains is the English-conditioned continuation from there, not a rewrite of the question. Nothing is added at inference, while Translate-Test needs a translation of every question.

Revision. We will add a short paragraph on this comparison to Section 4.3 and the translated-question condition to Figure 3b. The details go to Appendix D. Related Work will say that CELT needs no translation at inference. Its training uses the parallel questions of MGSM8KInstruct, and NLLB-200 for Telugu.

AuthorsOpenReview ↗

Relation to OPSD and SSOPD, with both run as baselines

The proposed approach shares substantial similarities with existing self-distillation methods, particularly those that improve reasoning by distilling a model's own predictions [1, 2]. While CELT introduces Frontier Measurement and span-weighted supervision, the underlying self-distillation framework is not fundamentally different from prior work. However, the paper neither discusses these closely related studies nor provides experimental comparisons with them. This makes it difficult to assess the novelty of the proposed method and whether its additional complexity offers meaningful advantages over existing self-distillation approaches.

Thank you for the two pointers.

  • [1] is OPSD (Zhao et al., arXiv 2601.18734). One model is teacher and student. The teacher sees privileged information such as a reference solution, the student only the question, and the per-token divergence is minimized on the student's own rollouts.
  • [2] is SSOPD (Tan and Hong, arXiv 2605.17497). It adds to GRPO a distillation term whose teacher sees the shortest correct completion of the group, applied to prefixes of the longest wrong completion.

Both work within one language and give the teacher a solution as context. CELT shares the self-distillation form and differs in three places:

  1. What the teacher conditions on. Teacher and student share the same solution prefix and differ only in the language of the question. Nothing is added to the teacher's context, so the target is the model's own English-conditioned distribution.
  2. Where the loss acts. Frontier Measurement locates, per question and language, the span the target-language question fails to call out, and the span weight puts the loss there.
  3. From which states. The scaffold ladder starts the student at every depth inside the span.

What the measurement adds. A prefix of the model's own English solution restores 71.18% to 76.09% of the failing low-resource cells with a reference trace. Inside that span the two conditions diverge 5.32 times more than beyond it. None of the methods above locates where in the reasoning the gap sits.

Both methods under our protocol. We ran both on all three backbones of Table 1. All rows share the 7473 families in eleven languages, the adapter, the optimiser and three seeds. OPSD is run as published, with one rollout of at most 1024 tokens and forward KL with pointwise clipping. Its teacher is the frozen initial model with the GSM8K solution in the published teacher prompt. SSOPD is run as published on top of our GRPO baseline. AVG (%), mean of three seeds (3B, 7B: Qwen2.5; L8B: Llama3-8B):

3B MGSM3B MATH3B CSQA7B MGSM7B MATH7B CSQAL8B MGSML8B MATHL8B CSQA
GRPO61.7150.5147.8873.0263.2057.1064.5824.4353.36
polyGRPO64.6552.4951.7875.9364.9460.4668.1126.7153.64
OPSD63.3950.2250.1674.3564.1658.7465.7125.3152.19
SSOPD62.4851.3847.6573.7663.3057.5165.1324.8453.51
CELT72.9158.6055.9881.7568.1163.6571.5628.5156.38

On Qwen2.5-7B-Instruct, we then add the parts of CELT to OPSD one at a time:

MGSMMATH500X-CSQArecovered
OPSD74.3564.1658.7436.38
OPSD, English-question teacher76.4165.0260.8542.41
+ span weight on the student rollout77.7165.5361.2347.68
+ frontier rows, ladder, preservation, reading80.9567.6062.9760.71
CELT (tails from the model's own English solution)81.7568.1163.6564.45

Results.

First, both methods improve over GRPO on MGSM on every backbone, as in their own papers, and in 15 of 18 cells. OPSD, whose teacher sees only GSM8K solutions, stays below GRPO on Qwen2.5-3B MATH500 and Llama3-8B X-CSQA, and SSOPD on Qwen2.5-3B X-CSQA. Both stay below polyGRPO. CELT is 3.20 to 9.52 points above OPSD and 2.87 to 10.43 above SSOPD, and is first in every language.

Second, every part of CELT adds accuracy. The English-question teacher alone adds 0.86 to 2.11 points to OPSD. The span weight, on the first j∗j^* steps of the student rollout, lifts recovered from 42.41 to 47.68. The frontier rows, ladder, preservation and reading, each removed alone in Table 2, lift it to 60.71, and the model's own English tails to 64.45.

Third, the last step shows why CELT continues its own English solution. Under the target-language question, a student rollout usually leaves the solution at the misread step, so the teacher scores a path that already contains the error.

Revision. We will discuss both papers in Section 2, add both methods to Table 1 and add the other results to Appendix B.

Reviewer_H9PrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CELT, a self-distillation method for improving multilingual reasoning in language models. For each question with a verified model-generated English solution, CELT measures the shortest English reasoning prefix that enables the frozen model to answer the corresponding target-language question correctly. It then matches the adapted model’s target-language-conditioned next-token distributions to the frozen model’s English-conditioned distributions. Experiments on three backbones and three benchmarks report consistent improvements, particularly for low-resource languages, with no additional inference module after merging the adapter.

Critical strengths

The method connects a measurable, question- and language-specific intervention to token-level distillation. The same-model teacher and student avoid requiring an external reasoning teacher.

Critical weaknesses

The measurement is dependent on one trace and one decoding path. A single greedy English solution, punctuation-based segmentation, and one greedy continuation per prefix define the frontier.

Questions for authors
  1. If reference traces ee are generated with nonzero-temperature sampling instead of greedy decoding, how would CELT’s performance change? Moreover, could you evaluate CELT on the benchmarks using nonzero-temperature sampling at inference time, in addition to greedy decoding, given that sampling-based decoding is commonly used in practical LLM deployment?
  2. Could you evaluate CELT on more recent backbones, such as Qwen3 and Qwen3.5, in addition to Qwen2.5 and Llama3?
4 comments
AuthorsOpenReview ↗

Frontier Measurement with sampled traces, other step boundaries and sampled probes

The measurement is dependent on one trace and one decoding path. A single greedy English solution, punctuation-based segmentation, and one greedy continuation per prefix define the frontier.

  1. If reference traces ... are generated with nonzero-temperature sampling instead of greedy decoding, how would CELT's performance change?

Thank you for pressing on this. A frontier read off one greedy trace could indeed be fragile, so we tested each of the three choices. We changed each one, rebuilt the training corpus from the new measurement and retrained CELT. Everything else is the configuration of the paper (Qwen2.5-7B-Instruct, seeds 17, 18 and 19).

Setup.

  • Sampled reference trace: one English sample per family at temperature 0.7 and top-p 0.95, kept if verified.
  • Clause steps: boundaries also at commas, semicolons and colons outside numbers and brackets.
  • 24-token steps: a boundary every 24 tokens, moved to the next whitespace and never inside a number.
  • Sampled probes: four continuations per prefix at temperature 0.7, and a prefix suffices if at least two reach the gold answer.

Kept and recovered are measured on the fixed classes of Figure 3a, as in Table 2. Mean of three seeds, with the standard deviation of the AVG:

MeasurementMGSM ENHRLURLAVGkeptrecoveredMATH500X-CSQA
deployed (Table 2)96.1386.1171.6081.75 ± 0.3199.0464.4568.1163.65
sampled reference trace95.8785.7871.1781.38 ± 0.4498.9563.6267.9063.27
clause steps96.0086.3871.4381.82 ± 0.4899.1064.5267.8363.59
24-token steps95.4786.0271.4781.59 ± 0.1898.9763.9667.7463.32
sampled probes96.2786.0771.3081.62 ± 0.2198.9164.2468.0463.45

On the training families, only failing cells with a reference trace can change class under other steps or probes, since V(0) comes from the cell's own trace. With a sampled reference trace, the cells without one can change as well. Qwen2.5-7B-Instruct:

Measurementcells that can change classclass unchanged among themamong all 74730 target-language cellsfrontier ends in the same deployed step
sampled reference trace2542774.4%91.3%86.9% at the same relative depth (within 0.15), since the steps differ
clause steps2052995.3%98.7%90.6%
24-token steps2052992.4%97.9%84.2%
sampled probes2052984.7%95.8%89.4%

Results.

First, none of these choices changes the result materially. MGSM AVG stays within 0.37 points of the deployed measurement, close to the seed standard deviation of Table 1 (0.31). MATH500 and X-CSQA stay within 0.37 and 0.38. With four sampled traces per family, each measured separately and the rows of a family sharing one unit of weight, MGSM AVG is 81.81. The lowest MGSM AVG is still 5.45 points above polyGRPO, the strongest baseline (75.93).

Second, CELT needs a frontier, not this exact one. With a sampled reference trace, a quarter of the cells that can change class do so, and MGSM AVG moves by 0.37 points. Without Frontier Measurement (Table 2), every failing cell with a reference trace trains from the bare question. CELT then loses 6.83 MGSM AVG points and recovers 44.70% of the frontier cells instead of 64.45%. Where both measurements find a frontier, it ends in the same deployed step in 84.2% to 90.6% of the cells.

Other backbones. The same four changes on the other two backbones of Table 1, MGSM AVG, mean of three seeds:

deployed (Table 1)sampled reference traceclause steps24-token stepssampled probes
Qwen2.5-3B-Instruct72.9172.6972.9572.5272.79
Llama3-8B-Instruct71.5671.0871.6471.0971.58

MGSM AVG stays within 0.39 and 0.48 points of the deployed measurement, and MATH500 and X-CSQA within 0.50 and 0.21. Among the cells that can change class, 79.1% to 95.0% keep it.

Revision. We will add this experiment, on all three backbones, to Appendix E. We hope this settles the concern about one trace and one decoding path, and we are glad to run any other variant you would find informative.

AuthorsOpenReview ↗

CELT under sampled decoding

Moreover, could you evaluate CELT on the benchmarks using nonzero-temperature sampling at inference time, in addition to greedy decoding, given that sampling-based decoding is commonly used in practical LLM deployment?

Thank you, this is a fair point, since deployed models usually sample. We evaluated every method of Table 1 with the sampling configuration shipped with each backbone:

  • Qwen2.5: temperature 0.7, top-p 0.8, top-k 20, repetition penalty 1.05.
  • Llama3: temperature 0.6, top-p 0.9.

Each question gets four samples, and accuracy is their mean. Every method has three runs. AVG accuracy (%), mean ± standard deviation:

3B MGSM3B MATH5003B X-CSQA7B MGSM7B MATH5007B X-CSQALlama MGSMLlama MATH500Llama X-CSQA
Base50.38 ± 0.3141.68 ± 0.2843.66 ± 0.1864.46 ± 0.2054.61 ± 0.2952.96 ± 0.2050.22 ± 0.3224.73 ± 0.2644.21 ± 0.24
English-CoT58.85 ± 0.4343.80 ± 0.1644.63 ± 0.1568.57 ± 0.2057.45 ± 0.4254.70 ± 0.1653.48 ± 0.3225.19 ± 0.3945.87 ± 0.16
Translate-Test56.71 ± 0.3043.65 ± 0.2145.93 ± 0.1967.48 ± 0.2856.87 ± 0.3055.69 ± 0.1254.12 ± 0.3724.98 ± 0.1848.60 ± 0.15
xRFT51.97 ± 0.4839.43 ± 0.1339.36 ± 0.1465.77 ± 0.4850.48 ± 0.2148.33 ± 0.1451.96 ± 0.2022.39 ± 0.3747.33 ± 0.22
LIDR52.77 ± 0.2746.06 ± 0.2842.49 ± 0.1666.62 ± 0.1659.55 ± 0.1452.23 ± 0.2053.61 ± 0.4323.18 ± 0.3251.31 ± 0.17
MAPO49.64 ± 0.1444.95 ± 0.4640.42 ± 0.1463.35 ± 0.3657.80 ± 0.3549.68 ± 0.1758.49 ± 0.4024.33 ± 0.4342.97 ± 0.32
GRPO59.85 ± 0.1248.80 ± 0.4347.32 ± 0.1271.18 ± 0.1560.87 ± 0.1356.47 ± 0.1563.02 ± 0.2123.59 ± 0.2752.75 ± 0.14
polyGRPO63.04 ± 0.3650.90 ± 0.1551.07 ± 0.1774.36 ± 0.2963.12 ± 0.3159.73 ± 0.1366.68 ± 0.2926.21 ± 0.3053.14 ± 0.15
CELT71.23 ± 0.4257.41 ± 0.4255.43 ± 0.1780.18 ± 0.2566.12 ± 0.1262.88 ± 0.1569.86 ± 0.7027.85 ± 0.3655.78 ± 0.16

What we see.

First, CELT stays first in all nine columns and polyGRPO second, as in Table 1. The margin over polyGRPO is 1.64 to 8.19 points, against 1.80 to 8.26 under greedy decoding.

Second, the margin is again largest in the low-resource languages, which hold most of the missing spans. On MGSM, CELT is 5.57 to 18.19 points above polyGRPO there under sampling.

Revision. We will add these results to Appendix B. In short, the ranking of Table 1 does not depend on greedy decoding.

AuthorsOpenReview ↗

CELT on Qwen3-8B and Qwen3.5-9B

  1. Could you evaluate CELT on more recent backbones, such as Qwen3 and Qwen3.5, in addition to Qwen2.5 and Llama3?

Thank you for suggesting the newer backbones. We ran the full pipeline on Qwen3-8B and Qwen3.5-9B with thinking mode off, within the 2048-token budget of Appendix A.4, together with every baseline of Table 1. Both use the deployment prompt of Appendix A.5. This prompt asks for the reasoning in English inside <think>...</think> and the answer in a box, the output format under which Wu et al. (2026) also report Qwen3-8B. CELT uses the same 7473 families, adapter and hyperparameters as in the paper. The baselines use the configurations of Appendix A.7, and all trained methods run three seeds.

The missing span on the new backbones. Frontier Measurement on the training families, and the recoverable share of the MGSM test cells (Table C.1):

Backboneverified English familiesprobe decodingsrecoverable share, HRLrecoverable share, URL
Qwen3-8B6948493120.62420.7537
Qwen3.5-9B7086389470.61540.7586

As on the paper's backbones, a prefix of the model's own English solution restores most failing cells that have a reference trace. It does so more often in the low-resource languages.

Accuracy. AVG (%), trained methods as mean ± standard deviation over seeds:

Qwen3-8B MGSMMATH500X-CSQAQwen3.5-9B MGSMMATH500X-CSQA
Base82.1571.8962.1985.1378.1466.53
English-CoT84.0473.4663.1086.7379.2967.23
Translate-Test84.0773.2063.9386.3679.4368.03
xRFT83.16 ± 0.3868.54 ± 0.2259.09 ± 0.1585.71 ± 0.4476.02 ± 0.4863.65 ± 0.21
LIDR83.05 ± 0.2374.32 ± 0.2461.71 ± 0.1785.94 ± 0.3379.98 ± 0.2065.78 ± 0.14
MAPO81.84 ± 0.3473.37 ± 0.2859.80 ± 0.1584.80 ± 0.6479.33 ± 0.2764.13 ± 0.24
GRPO84.97 ± 0.3974.43 ± 0.4864.30 ± 0.1887.52 ± 0.1580.39 ± 0.2868.17 ± 0.16
polyGRPO87.45 ± 0.2075.43 ± 0.3766.63 ± 0.1488.87 ± 0.5281.22 ± 0.4270.62 ± 0.13
CELT90.86 ± 0.3478.28 ± 0.3669.32 ± 0.2591.33 ± 0.5883.18 ± 0.1772.92 ± 0.20

On Qwen3-8B MGSM, our Base and polyGRPO (82.15 and 87.45) are close to the 81.45 and 87.27 that Wu et al. (2026) report with their own prompt.

Findings.

First, CELT is first in all six columns and in every language. It is 1.96 to 3.41 AVG points above polyGRPO, again the strongest baseline in every column.

Second, the margin is again largest in the four low-resource languages of MGSM. There CELT reaches 85.73 (Qwen3-8B) and 88.07 (Qwen3.5-9B), against 80.00 and 83.00 for polyGRPO and 74.30 and 78.60 for Base.

Third, a stronger backbone fails fewer cells. It needs fewer probes, 49312 and 38947 against 61738 to 88617 on the paper's backbones. It also leaves fewer missing spans to recover, which is why the margins are smaller than on Qwen2.5.

Thinking mode. We also ran Qwen3-8B with thinking mode on and an 8192-token budget, with Base, CELT and polyGRPO, the strongest baseline in every column. The reasoning in the thinking block is cut into steps with the rules of Appendix A.2, 31.4 steps per reference trace on average. AVG (%), three seeds:

MGSMMGSM URLMATH500X-CSQA
Base87.0980.6083.6663.40
polyGRPO89.81 ± 0.5383.8785.39 ± 0.2667.05 ± 0.18
CELT92.00 ± 0.2387.7386.98 ± 0.2069.64 ± 0.13

Every method gains from thinking. CELT stays first in every language of the three benchmarks, 2.19, 1.59 and 2.59 points above polyGRPO. Its margin is again largest in the low-resource languages of MGSM.

Revision. We will add both backbones with all methods to Appendix B, the thinking-mode runs to the same table, and their census to Appendix C.

Thanks for your comment. I will keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of new experiments and coverage

We thank the reviewers. All newly trained models use the protocol of Table 1 with three seeds.

New experiments

#experimentresult
1Frontier Measurement with a sampled trace, other step boundaries or sampled probes, on all three backbonesMGSM AVG within 0.37 of the deployed measurement and still 5.45 above polyGRPO on Qwen2.5-7B-Instruct, within 0.48 on all three backbones
2Sampled decoding with the configuration shipped with each backboneCELT first in all nine columns, polyGRPO second
3Qwen3-8B and Qwen3.5-9B with every method of Table 1, and Qwen3-8B with thinking mode onCELT first in all six columns and every language, 1.96 to 3.41 above polyGRPO, and 1.59 to 2.59 with thinking on
4Cost and wall-clock of every stage, the RL baselines at CELT's GPU-hours, and five more training languagesprobes 6.4 to 14.2 minutes on one node, CELT the cheapest trained method
5Translate-Test against CELT by class, divergence of the translated question, a dedicated translatorCELT closes 79.1% of the in-span divergence, the translation 48.8%
6OPSD and SSOPD under our protocol on all three backbones, and the parts of CELT added to OPSD one at a timeCELT 3.20 to 9.52 above OPSD and 2.87 to 10.43 above SSOPD
7English SFT, with and without reading, and cross-lingual SFT under the same budget, on all three backbonesCELT 1.66 to 17.60, 1.01 to 11.82 and 1.75 to 10.36 above them
8Causes of the 751 unreachable cells, two annotators, ordered and free ranking, and which method answers which causelanguage understanding is the largest cause, 46.87%; polyGRPO after CELT reaches 83.83 on MGSM
9LreadL_{read} against an alignment-only loss, six weight and layer settingsLreadL_{read} raises restatement fidelity by 19.10, alignment by at most 1.10

Coverage

reviewerpointanswered by
H9Prone trace and one decoding path, and Q1 on sampled reference tracesexperiment 1
H9PrQ1, sampling at inferenceexperiment 2
H9PrQ2, Qwen3 and Qwen3.5experiment 3
MUkTW1, cost of Frontier Measurementexperiment 4
MUkTW2, difference from translationexperiment 5 and Table 2
MUkTW3, self-distillation methodsexperiment 6 and three measured differences
hq4CW1 and Q1, English fine-tuningexperiment 7
hq4CW2 and Q2, unreachable casesexperiment 8
hq4CW3 and Q3, cost and wall-clockexperiment 4
hq4CW4 and Q4, reading lossexperiment 9

Revisions

  • In the main text, Section 2 adds one paragraph on OPSD and SSOPD, and Tables 1 and 2 two rows each. Section 4.3 adds one paragraph on Translate-Test, with the translated-question condition in Figure 3b. Related Work states that CELT needs no translation at inference.
  • All other results go to the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.