acceptodds
Under review as a conference paper at ICLR 2027

X-OPSD: Culture-Preserving On-Policy Self-Distillation for Cross-Lingual Capability Transfer

Abstract

Cross-lingual reasoning transfer can strengthen multilingual large language models (LLMs) while disrupting culturally grounded predictions. We identify this failure mode as transfer spillover and hypothesize that representation changes supporting stronger reasoning can be partially decoupled from those interfering with cultural specificity. To investigate this, we introduce X-OPSD, a culture-preserving on-policy self-distillation framework that constructs complementary implicit teachers from a single frozen backbone. Reasoning-transfer teachers attenuate language-related activation components to elicit stronger reasoning, while culture-preserving teachers restrict these updates to directions outside an identified culture-related subspace. Their task-dependent supervision is distilled into a single student that requires neither activation steering nor teacher routing at inference time. Across three state-of-the-art multilingual backbones, X-OPSD improves mathematical reasoning and culture-sensitive benchmark averages over the base models, while reducing cultural forgetting relative to English-privileged distillation. Representation analyses and teacher ablations support the role of selective intervention in mitigating spillover. These findings motivate making cultural preservation part of how reasoning supervision is constructed, rather than treating it as a separate objective after transfer.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multilingual, Representation Engineering, On-Policy Self-Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QArPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper identifies "transfer spillover" where cross-lingual reasoning transfer often overwrites culturally grounded answers the base model previously got right. The authors propose X-OPSD as a solution. X-OPSD builds two teachers from one frozen backbone using activation interventions. The first is a reasoning teacher that attenuates a language-related subspace, the second is a culture preserving teacher that intervenes only along directions orthogonal to a culture subspace. A single student is student is trained by on-policy self-distillation, with math inputs routed to the first teacher and the cultural inputs routed to the second. There is no need for steering or routing at inference. The contributions are the spillover framing with a strong paired forgetting metric. The validation covers three backbones on three culture-sensitive benchmarks against Base, OPD, OPSD, COPSD, and MRRE. X-OPSD improves both averages and reduces CulturALL forgetting relative to COPSD. However all results are single runs on small evaluation sets and the baselines are not trained on matching data.

Critical strengths
  1. The ablations in table 3 hold the training data fixed and show that cultural protection and task-specific teacher routing each matter. Full X-OPSD has the best joint averages and the lowest forgetting.

  2. Measuring "transfer spillover" and pairing it with forgetting gives a better failure than aggregate accuracy. This framing is useful even beyond this method.

Critical weaknesses
  1. X-OPSD trains and calibrates on INCLUDE data while OPSD and COPSD see only math. So the culture gains over those baselines can't be attributed to the method rather than to the data itself.

  2. The gains are often 1-2 points from single runs with no variance reported. The forgetting rates rest on 65-81 items and the per-language coefficients are hand picked (frequently to be zero). The Gemma culture teacher is inactive for six of eight languages.

  3. The teacher preference metric (eq. 7) has a drift problem away from base. For languages where the culture teacher is 0 (table 2) the culture teacher is identical to base. Any fine-tuning that lowers the student's overlap with Base inflates the Delta_route, whether or not the student moves toward the reasoning teacher. Table 4 shows that the largest preference scores are exactly in those languages. A controlled student distilled without reasoning intervention is needed.

Not critical, but observations, I think referencing limitations in the main body would be beneficial to the overall structure of the paper. I also noticed that two citations are showing ("?"). One for mAce and the other for MRRE in the appendix.

Questions for authors

How do OPSD and COPSD perform when trained on INCLUDE as well?

What is the variance across multiple runs and seeds?

How were the coefficients chosen and how sensitive are they to the results?

Why does language fidelity on the 35B model fall below base?

1 comment
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer QArP,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

Reviewer_9dc3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes X-OPSD, an on-policy self-distillation framework for improving cross-lingual reasoning while preserving culture-specific knowledge. It constructs reasoning-transfer and culture-preserving teachers through representation-space interventions and distills their predictions into a single student. Experiments across multilingual models show improvements in mathematical reasoning and culture-sensitive benchmarks, with reduced forgetting of previously correct cultural predictions. Component ablations and representation analyses provide further evidence for the proposed approach.

Critical strengths

The paper addresses cultural forgetting during cross-lingual reasoning transfer, where improvements in reasoning may come at the expense of previously acquired capabilities. The paired forgetting analysis provides a useful perspective beyond aggregate accuracy. The proposed framework improves reasoning and culture-sensitive benchmark averages across multiple backbones without requiring inference-time interventions. Component ablations and representation analyses provide additional empirical support for the approach.

Critical weaknesses

The interpretation of the culture-related subspace is not sufficiently justified. It is constructed from language-wise mean activations, which may reflect linguistic or dataset-specific differences rather than cultural knowledge. The paper does not adequately distinguish these factors. Moreover, several culture-preserving teacher configurations apply no intervention, making it unclear how much of the improvement comes from subspace protection rather than self-distillation.

The experimental evidence does not fully isolate the contribution of the proposed mechanism. The main distillation baselines use different training data and supervision, and component ablations are limited to one backbone. Furthermore, while the paper evaluates cultural forgetting and response-language fidelity, it does not examine whether reasoning trajectories preserve the target language. This limits the conclusions about the broader effects of cross-lingual reasoning transfer.

Questions for authors

Could the authors clarify how the benefits of culture-related subspace protection can be distinguished from those of self-distillation using the unmodified backbone? Please also address the concerns raised in the weaknesses above.

1 comment
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer 9dc3,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

Reviewer_wNcTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes a new On-policy self-distillation algorithm to preserve Culture while doing OPSD (x-OPSD). It builds 2 teachers from the same base model backbone using activation interventions: reasoning teacher and culture preserving teacher. The paper evaluates 3 model backbones on 3 reasoning and 3 culture-sensitive benchmarks. It also quantifies "forgetting" of cultural concepts by evaluating on CulturALL benchmark.

Critical strengths

The problem statement is well-motivated and the framing is good. I also liked the definition and measuring the phenomenon of "transfer spillover". This itself is a nice contribution by this work. I am also happy with their model sweeps and their downstream eval benchmark selection for this work.

Critical weaknesses
  1. Looking at Table 1 results, are the baselines (OPSD, COPSD, MRRE) calibrated well for Qwen 3.5 4B? The performance of these methods are below the Base model. This inflate the gains for their proposed method
  2. Also, the baseline gaps (Table 1) are quite small. I think the table would benefit from multiple seed runs / confidence intervals for their headline results. This would also help in proving their robustness on the CulturALL dataset which is used to show less forgetting by using their proposed method.
  3. I also have concerns about how the training data was selected for their baselines: I can see in Line 242 that they use INCLUDE and MGSM prompts for their x-OPSD methodology training. However, according to their appendix (Appendix D), plain OPD uses a multilingual mixture of 778 prompts (400 mathematical prompt - what dataset? and 378 INCLUDE prompts). OPSD uses the complete openthoughts dataset (~30k prompts) and COPSD (polymath dataset ~24k samples across 8 languages). The makes INCLUDE ood for these baselines (OPSD and COPSD) as they do evaluate these baselines on the INCLUDE dataset.
  4. Any reasons why forgetting was not reported in Figure 5 for OPD and OPSD?
Questions for authors
  1. Figure 3 is not helpful. Instead, I would prefer a table with language-wise results instead of a spider-plot.
  2. I did not find it in the main paper as to how you define and calculate language fidelity. Are you using an LID to detect if the language output is in the correct language or something else?
  3. broken citations in appendix b and c
  4. How did you sample / select the INCLUDE prompts for various baselines / ablations? Was it just uniform sampling or did you select culture specific domains from the benchmark?
1 comment
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer wNcT,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.