- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Crosslingual On-Policy Self-Distillation (COPSD) to improve mathematical reasoning in low-resource languages. The same model serves as both student and teacher: the student sees only the low-resource question, while the teacher additionally receives the English question and reference solution as privileged information. COPSD distills the teacher’s token-level distributions over student-generated trajectories. Experiments across Qwen3-1.7B/4B/8B, 17 African languages on AfriMGSM, and eight PolyMath languages show that COPSD consistently outperforms Base, GRPO, Trace-SFT, and Translation-SFT, with faster convergence and larger gains on harder tasks and lower-resource languages.
Critical strengths
The work addresses a well-motivated problem in low-resource language reasoning and the proposed method is simple and practical, requiring neither target-language rationales nor external teachers. The experiments are extensive across languages, model scales, and benchmarks, providing evidence of its effectiveness and generalization.
Critical weaknesses
1.The comparison with Translation-SFT may not be fully fair. Both methods rely on English reference solutions, while Translation-SFT additionally depends on translation quality and uses hard targets rather than distributional supervision. The authors should better control translation-related factors and further analyze whether COPSD’s gains come from distributional supervision or differences in target quality.
2.The test-time scaling claim should be stated more carefully. COPSD achieves larger absolute gains with increased generation budgets, but its relative gains are not consistently higher than the baselines. For example, Base shows a slightly larger relative improvement than COPSD on the 8B model. On the 1.7B model, COPSD continues to improve at 2,048 tokens while GRPO and Trace-SFT degrade, suggesting better scaling robustness, but the reason for this behavior remains unclear.
3.The paper lacks a direct evaluation of generation quality in low-resource languages. Although it attributes the challenge to weak low-resource generation and imperfect teacher signals, it does not evaluate generation fluency and grammaticality, how COPSD gains correlate with language-specific generation capability. Such analyses would provide stronger evidence for the proposed knowledge-transfer mechanism beyond final Pass@12 results.
Questions for authors
1.Why does COPSD plateau on the 1.7B model but exhibit performance degradation after early peaks on the 4B and 8B models?
4 comments
3.The paper lacks a direct evaluation of generation quality in low-resource languages...
Thank you for this suggestion. We agree that target-language generation capability may influence how much a model benefits from COPSD. To examine this, we evaluated models on English-to-target-language translation using FLORES+, with 200 examples per language across AfriMGSM languages, providing an assessment of conditioned target-language generation. We then computed correlations between per-language chrF++ scores and COPSD's absolute Pass@12 gains over Base, as shown below (p-values are from two-sided permutation tests with 9,999 shuffles).
| Model | Avg. chrF++ | Pearson r | p-value | Spearman ρ | p-value |
|---|---|---|---|---|---|
| Qwen3-1.7B | 2.36 | 0.419 | 0.1085 | 0.412 | 0.1148 |
| Qwen3-4B | 3.97 | −0.238 | 0.3493 | −0.197 | 0.4577 |
| Qwen3-8B | 5.92 | −0.457 | 0.0893 | −0.364 | 0.1677 |
The low translation scores suggest that generation in these languages remains challenging. In addition, the correlations are not consistent across model sizes, and none is statistically significant. These results indicate that there is no simple relationship between generation capability and COPSD gains and that other factors may also play a role.
We also want to mention that the gains may depend on how well the model can reason in English, as COPSD aims to transfer reasoning capability from English to other languages. As you can see in our PolyMath evaluation, we observe larger improvements for Swahili and Telugu, both low-resource languages. However, this does not mean that the model has better generation capability in these two languages than in Japanese or Spanish (with fewer improvements). In other words, COPSD gains may depend on both target-language generation capability and English reasoning capability, among other factors. We will include this discussion in our camera-ready version.
Response to questions:
1.Why does COPSD plateau on the 1.7B model but exhibit performance degradation after early peaks on the 4B and 8B models?
Indeed, training instability appears to be a broader challenge for OPSD-like methods. Related work reports degradation during continued self-distillation [1], and we observed similar issues when reproducing OPSD using its original codebase [2]. The difference across model sizes may reflect model-specific responses to continued updates. The 1.7B model may reach a point where further teacher guidance brings limited benefit, whereas larger models may adapt more strongly to both useful and imperfect teacher signals, producing early gains followed by degradation. Nevertheless, with validation-based checkpoint selection, we observe average improvements over the baselines at all three scales. We believe that further stabilizing training to sustain these gains is an important direction for future work.
[1] Lai and Nissim (2024). mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models.
[2] Hwang et al. (2025), Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning.
[3] Sutawika et al. (2026), Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning.
[4] Liu et al. (2025). Understanding R1-Zero-Like Training: A Critical Perspective.
[5] Kim et al. (2026). Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
[6] Zhao et al. (2026). Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models.
Your rebuttal has adequately addressed my concerns. Considering the scope of this work, I maintain my original score.
Thank you very much for engaging in the discussion and for confirming that our rebuttal has addressed your concerns. We appreciate your positive assessment and will incorporate the additional analyses and clarifications into the camera-ready version.
Dear Reviewer RL2h,
Thank you for your thoughtful review and positive feedback. We appreciate your recognition of the paper's motivation, clarity, novelty, and soundness. We want to address your concerns below.
Thank you for raising this point. Translation quality can indeed affect SFT performance, and this is precisely one of our motivations for proposing COPSD: avoiding the costly and potentially unreliable translation of reasoning steps, especially for extremely low-resource languages. Translation is therefore an integral part of the Translation-SFT setup. In other words, Translation-SFT is a baseline where we allow for imperfect automatic translations, following a common translate-training approach in the community [1, 2, 3]. And our comparison evaluates the effectiveness of these complete training approaches under practical conditions.
Additionally, Trace-SFT provides a complementary baseline that does not require translating reference solutions: it trains on target-language traces generated using the same privileged teacher context. COPSD also outperforms both Translation-SFT and Trace-SFT, supporting COPSD as a better option when a reliable translator cannot be guaranteed, especially for extremely low-resource languages as in our setup Nevertheless, it would be quite interesting (but also expensive and beyond the scope of this paper) to investigate how translation quality affects Translation-SFT when translations are produced by (1) humans, (2) dedicated translation systems, (3) open-weight LLMs, and (4) closed-source LLMs.
This is a good observation. We agree that COPSD's relative gains are not consistently higher than those of the baselines. However, relative improvement also depends on performance at the lower generation budget. For example, on the 8B model, COPSD achieves a larger absolute gain than Base (5.44 vs. 4.68 percentage points), despite a slightly smaller relative improvement (30.0% vs. 31.8%). We will revise the discussion in the test-time scaling part accordingly.
We also appreciate your observation regarding COPSD's better scaling robustness on the 1.7B model. One possible contributing factor is its more concise generation (see Table below): under the same 4,096-token budget, COPSD generates 1,680 tokens on average, compared with 2,991 for GRPO and 2,498 for Trace-SFT. Prior work has identified a length bias in the original GRPO objective that can encourage excessively long responses [4]. Trace-SFT may also retain verbosity from its model-generated training traces. These observations are consistent with our hypothesis that COPSD uses the available budget more effectively.