acceptodds
Under review as a conference paper at ICLR 2027

RA-MoE: Routing-Aligned Fine-Tuning for Multilingual Adaptation of Mixture-of-Experts Models

Abstract

Mixture-of-Experts (MoE) models enable efficient LLM scaling, yet adapting them to non-English downstream tasks remains challenging. Standard multilingual fine-tuning largely ignores their heterogeneous routing structure. Across multiple MoE models and tasks, we find strong cross-lingual routing alignment in middle layers, with routing divergence associated with target-language performance gaps. Motivated by this observation, we propose RA-MoE (Routing-Aligned MoE Fine-Tuning), a three-stage framework for multilingual MoE adaptation. RA-MoE categorizes parallel examples into four correctness groups (cc/ci/ic/ii) and identifies task-relevant experts in middle layers. It then selectively aligns target-language routing on ci examples toward successful English routing patterns, jointly matching the total routing mass assigned to task experts and its relative allocation among them. Experiments across three MoE models, three downstream tasks, and six target languages show that RA-MoE consistently outperforms standard SFT and strong routing-aware baselines. Further analyses confirm the intended routing changes and reveal that middle-layer task routing is largely shared and transferable across languages, providing mechanistic evidence for the cross-language transferability of task-specific routing. Code is available at https://anonymous.4open.science/r/ra-moe-CD06/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, multilingual adaptation, routing alignment, cross-lingual transfer, multilingual reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YYpqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • presents a training method to improve cross-lingual MoE agreement for target tasks & languages with strong results compared to baselines
  • using an in-domain training set, identifies questions the model gets correct in English but wrong after the question is translated to the target language
  • then, identifies experts that the model relies upon in English and trains the model using LoRA to call upon those experts using an auxiliary loss
  • Beyond demonstrating strong results on the target task/language, they also demonstrate that MoE routing agreement increases between English and target language. In addition, they show that it also leads to lower cross-entropy on the gold answers.
Critical strengths
  1. Writing is good and clear, at times quite concise (in a good way). The paper flows well. Numerous good figures (that I suggest making much larger with the extra page allotted for eventual camera-ready).
  2. Clever multi-stage pipeline with data curation step, expert selection step, and training step.
  3. Section 5 is really helpful, strong analysis of the impact of the loss
  4. careful language about claims made, soundness is reasonable
Critical weaknesses
  1. A concern is the dependence on the parallel data construction part and its scalability. Not only is this an expensive data generation process (requires many inference calls before training even starts) but also requires a funnel (only taking the ci samples) that, for languages&models that are better, would severely limit the amount of training data available. Note that it is also a bit of a shame that this method cannot make use of the tremendous quantity of translation/parallel data already available in open-source.
  2. Fine-tuning an LLM for a specific task and a specific language is narrow scope and not very common. This is also limited to task/QA eval-style data that already can be judged in a binary way, even if Section 5.5 provides a proxy way for tasks that can’t be easily judged. This limits the applicability of the method to a narrow set of tasks; rather those very close to evaluation benchmarks. This method would be difficult to expand to just language fine-tuning (e.g. improving LLM in Bengali generally) or task fine-tuning (e.g. improving LLM in tasks across many languages).
  3. Similar to above, Contrived experimental setting where training data is super in-distribution to eval data. This leads to overly clean environment where this method could simply be “better at overfitting” than vanilla SFT. Does not imply generalization to conditions where tasks are much broader (e.g. mathematical reasoning as a whole vs. specifically GSM8K-style questions).
  4. I had to dig through the appendix to find information about the training and found that it was LoRA ! While I can understand resource constraints, the use of LoRA needs to be made explicit in the main body (including even in the introduction), since this impacts the potential conclusions.
  5. Related Work is super weak & limited, it does not properly contextualize the paper in the broader landscape of MoE LLMs, multilinguality in LLMs, and language adaptation in LLMs. I think this affects the motivation behind the paper quite a bit. In particular, it’s not properly justified why such a method is not possible in dense models.
  6. Reproducibility: No recommended reproducibility statement & the link in the abstract leads to an anonymous.4open.science repository with no linked Github “Unavailable. This content can’t be shown. The repository was not found on GitHub. Check the URL and spelling, make sure you are signed in to the account that can see it, and confirm the repo isn't hidden under an org that restricts third-party app access.”
  7. Similarly, the AI Use Statement is lazy, undetailed, and overall lends very little confidence into the statement that “generative AI tools were used solely to assist with English writing” and at no other parts of the project. Note that the author’s are very clear about it in the main body so it’s not an issue if added, but “Generate synthetic data sets” is a category that requires mandatory disclosure in this statement.

Overall, I think the paper is well written, the method is clever, and the experiments are well-designed. The results are impressive as well. But these only convince me that it works “when targeting a specific eval set in a specific language where there is abundant training data super close to the eval and you’re willing to do inference over the whole training set before starting training”.

Questions for authors
  1. L50-52 citation ?
  2. Contribution 1 is just a re-hash of Bandarkar et al., 2026, Chen et al., 2026 ? Unless, I’m missing something, it should be removed (and it doesn’t seem to be particulary expanded upon later anyways)
  3. It took me a while to understand that “generated token position” G L161 meant the output tokens in the QA format. Why only use these tokens ? In theory, wouldn’t the model already be figuring out how to answer while processing the input. Given the tasks, there are likely very few output tokens to work with.
  4. Why use OLMoE-1B-7B-0924-SFT ? This is an oddly specific checkpoint.
  5. Assuming \tau stays fixed, this means the range changes language-to-language, task-to-task ? I would assume that lower-resource languages therefore has a much smaller range ? Would this not be undesired ? How was \tau chosen ?
  6. Would the use of FLORES English tasks as the “general English distribution” not cause weird bias ? Especially because that’s also unsupervised texts, not chat-formatted QA samples.
  7. I would suggest a different way of presenting Section 3.4, if I understood it correctly I think it’s a lot simpler than the section makes it out to be. One suggestion could be to ignore the concept of the mini-batch (leave that for the code implementation), unless it matters in a way I don’t understand.
  8. From practice, I am not concerned with the potential for translation errors in the data generation part because it’s likely small and rare enough to undermine the training operation. It would mostly just add a small bit of noise. However, I believe the potential for errors should be mentioned and argued why it wouldn’t be too much of an issue.
  9. L400-406, I don’t understand how the set of experts is different across target languages if they’re both defined on the combined cc and ci sets, which should be the same regardless of the target language since it only uses English.
  10. How are confidence intervals possibly calculated ? Just using binomial distributions ? I would advocate to remove these if so, because it conveys statistical intervals that only cover eval variance in a very narrow way; not training run variance, data variance, etc. and other forms that impact the experiments. As a result I feel like it gives a semblance of statistical soundness of experimental results that isn’t there (for the record, I understand that this isn’t possible and am not docking points for this). I also acknowledge that a different reviewer may have a different opinion.
7 comments
AuthorsOpenReview ↗

Response to Reviewer YYpq — Part 1: Data requirements, practical scope, and mathematical generalization (W1–W3)

Thank you for the detailed review and constructive questions. We clarify the use of training data, preparation costs, and the scope of our contributions, and provide additional evidence on independent mathematical generalization and training-seed robustness. We also address the implementation, reproducibility, and disclosure concerns below.


W1. Does the ci selection restrict training-data availability, and how scalable is parallel-data preparation?

  1. All target-language training examples participate in cross-entropy training.

    RA-MoE does not train exclusively on ci examples. As defined in §3.4, all target-language training examples contribute to the standard cross-entropy objective; only eligible ci examples additionally contribute to routing alignment. Consequently, a smaller ci pool reduces the amount of auxiliary alignment supervision, rather than reducing the supervised training dataset.

  2. Data-construction costs and RA-MoE-specific costs are distinct.

    In our experiments, SFT and RA-MoE use the same translated target-language training data and therefore share the translation cost. Given these parallel task data, RA-MoE additionally performs correctness annotation and routing profiling.

    Table 14 reports this additional preparation cost as up to 16.7% of fine-tuning time, reduced to up to 10.0% when reusing the routing configuration. These measurements begin after parallel-data construction and therefore do not include translation. Starting from native target-language data would additionally require constructing and validating reference-language counterparts.

  3. Existing parallel data can be used when they supply the required task information.

    RA-MoE does not require newly translated pairs specifically. Existing parallel corpora can supply language pairs and reduce translation work. Applying the current pipeline additionally requires task responses and a signal for selecting alignment examples. Parallel task datasets containing these elements can therefore be used directly; general parallel text alone does not supply all of them.


W2. Why focus on task–language adaptation, and is binary correctness required?

Task-specific fine-tuning has a practical role when a deployed model must perform a particular function reliably. In a multilingual service, unequal task performance across languages can also give users substantially different experiences. Improving a particular task–language setting is therefore a useful adaptation objective.

Our experiments investigate this setting across three task categories and six target languages. They evaluate task-conditioned multilingual adaptation; they do not claim to establish general-purpose language improvement or joint adaptation across all tasks and languages.

The method is also not restricted to tasks with exact binary verification. Section 5.5 uses response perplexity as a selection proxy for open-ended instruction-following data. This variant improves over SFT by 2.1 percentage points on average across six target languages on IFEval. It retains the requirement for paired instruction–response data, while relaxing the need for an exact correctness verifier.


W3. Do the improvements generalize beyond GSM8K-style questions?

We additionally evaluated the existing Qwen1.5-MoE-A2.7B Spanish math checkpoints on an independent 500-question Spanish PolyMath evaluation, containing 125 questions from each difficulty tier. No additional fine-tuning on PolyMath was performed, and all three checkpoints used the same evaluation protocol.

ModelLowMediumHighTopOverall
Base12.8%1.6%0.0%0.0%3.6%
SFT55.2%3.2%0.8%0.0%14.8%
RA-MoE59.2%4.0%1.6%0.8%16.4%

RA-MoE improves over SFT by 1.6 percentage points, with a 95% CI of [0.6, 2.6], corresponding to eight additional correct answers. The improvement is positive in every difficulty tier, with the largest absolute gain in the Low tier.

This independent evaluation supports transfer beyond the original GSM8K evaluation setting, rather than improvements confined to GSM8K-style test questions.

AuthorsOpenReview ↗

Response to Reviewer YYpq — Part 2: LoRA, related work, and Contribution 1 (W4–W5、Q1/Q2)

This part clarifies the adaptation setup, the architectural scope of RA-MoE, and the relationship between our empirical findings and prior work.


W4. Why was the use of LoRA specified mainly in the appendix?

Appendix C.2 specifies that we apply LoRA to the up/down projections of expert feed-forward layers while freezing the remaining parameters. The training-based comparisons use the same adaptation setup.

We agree that this choice should be visible in the main text. We will explicitly state the use of LoRA in the Experimental Setup. Our fine-tuning results establish the effectiveness of RA-MoE under this parameter-efficient adaptation setup.


W5. How does the related work contextualize RA-MoE, and why does the method require MoE structure?

Section 2 covers four relevant strands: multilingual downstream fine-tuning, MoE-based language expansion, multilingual routing analysis, and routing-aware MoE adaptation. This includes cross-lingual representation alignment in dense models and distinguishes expanding a model for additional languages from adapting an existing multilingual MoE.

Regarding dense models, cross-lingual alignment is certainly possible in both architectures. The architectural requirement concerns the specific alignment target used by RA-MoE:

  • RA-MoE identifies a subset of task-relevant experts.
  • Its objective aligns both the total routing probability assigned to those experts and the relative allocation within that subset.
  • A standard dense feed-forward layer has no corresponding token-dependent distribution over explicit experts.

Thus, the proposed expert-routing objective directly applies to MoE models. A dense-model method could instead align hidden representations or other internal quantities, but that would require a different alignment target.


Q1/Q2. What supports lines 50–52, and how does Contribution 1 differ from prior work?

  1. The U-shaped routing pattern is explicitly attributed to prior work.

    The Introduction already states: “Prior work has revealed a characteristic U-shaped layer-wise pattern in cross-lingual routing divergence.” We do not claim to have discovered this pattern.

    Prior work also includes downstream-task analyses: Bandarkar et al. relate routing divergence to BELEBELE performance, and Chen et al. analyze layer-wise cross-lingual similarity using BELEBELE examples.

  2. Figure 1 provides our task-specific evidence.

    Building on these findings, our analysis examines mathematical reasoning, logical reasoning, and knowledge understanding. Figure 1 validates the layer-wise pattern in these settings and shows that larger middle-layer divergence is associated with larger target-language performance gaps relative to English. The statements at lines 50–52 refer to these results.

  3. Contribution 1 concerns this empirical extension and its relevance to adaptation.

    The contribution is phrased as: “We validate the characteristic U-shaped cross-lingual routing divergence pattern in task-specific settings and show that middle-layer divergence is associated with target-language performance gaps.”

    This evidence motivates using middle-layer routing as an alignment target during task adaptation. We will make the connection to Figure 1 explicit at lines 50–52 and sharpen the distinction between the previously established pattern and our analyses in the studied task settings.

AuthorsOpenReview ↗

Response to Reviewer YYpq — Part 3: Response tokens, checkpoint choice, and language-specific configurations (Q3–Q4、Q5/Q9)

This part explains the routing positions we analyze, the OLMoE starting checkpoint, and how middle-layer configurations vary across target languages.


Q3. Why profile only generated-answer token positions?

In §3.3, G denotes positions occupied by the generated response, excluding the prompt. We replay the generated response with teacher forcing and summarize routing over these positions.

Our aim is to characterize routing associated with a particular task response, including the successful reference-language response used for alignment. Response positions provide a consistent scope for this analysis across examples.

We agree that task-relevant computation can already occur while processing the input. Restricting profiling to response positions does not imply that reasoning begins only during response generation. The number of positions also varies by task: mathematical responses can contain multi-token derivations, whereas short-answer tasks provide fewer positions per example, which are aggregated across task examples for expert identification.


Q4. Why use OLMoE-1B-7B-0924-SFT?

The date suffix is part of OLMoE’s official checkpoint naming: 0924 and 0125 distinguish the September 2024 and January 2025 releases. We use the original September release’s official SFT checkpoint.

The choice of the SFT version is intentional. It provides an instruction-tuned starting point for studying cross-lingual task adaptation, without introducing a separate general instruction-tuning stage from the pretrained base model. Our objective is to evaluate task-expert routing alignment, rather than the acquisition of general instruction-following ability. RA-MoE and its corresponding OLMoE baselines use the same starting checkpoint.


Q5/Q9. How is the middle-layer range chosen, and why can expert configurations differ across target languages?

  1. The selection rule is shared, but the numerical threshold is configuration-dependent.

    As specified in Appendix B.2, τ is the median of the layer-wise mean cross-lingual divergences for each model–task–target-language configuration. We select the longest contiguous interval whose divergence is at or below this median, breaking equal-length ties toward the earlier interval.

    Thus, τ is not a single fixed numerical threshold shared across all languages and tasks.

  2. Lower-resource languages do not necessarily receive shorter intervals.

    A language with higher absolute divergence can also have a higher median threshold. The interval depends on the distribution of relatively low-divergence layers across depth.

    For example, Table 9 selects 11 layers for Bengali and 10 for Spanish with Qwen1.5-MoE on GSM8K; the corresponding OLMoE intervals contain 6 and 8 layers. Resource level alone therefore does not determine interval length.

  3. Language-specific intervals explain differences in the overall expert configuration.

    For fixed English examples and predictions, the reviewer is correct that cc ∪ ci identifies the same English-successful examples, regardless of the target-language correctness labels.

    The target language nevertheless affects the selected middle-layer interval. Since experts are identified within that interval, the resulting overall layer–expert configuration can differ. At shared layers, expert selection is the same when the English task examples and FLORES reference are held fixed.

  4. The overlap between intervals motivates configuration reuse.

    These intervals are similar across languages in Table 9. For example, Qwen1.5-MoE on GSM8K selects layers 5–13 for Chinese and 5–14 for Arabic. This overlap motivates the configuration-reuse experiment in Table 4, which applies the Chinese middle-layer interval and associated expert sets to Arabic and Bengali.

Reviewer_2jP6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RA-MoE, which uses successful English routing patterns as auxiliary supervision for target-language adaptation of MoE models. It categorizes parallel examples by correctness in each language, identifies task-relevant experts in middle layers, and selectively aligns routing for examples answered correctly in English but incorrectly in the target language. The alignment objective accounts for both the total probability mass assigned to task experts and its relative allocation among them. The method is evaluated across three models, three tasks, and six target languages, with component ablations, routing analyses, and cross-language transfer experiments.

Critical strengths
  • The paper systematically examines the contributions of its main design choices. In the ablation setting, it separately removes the two alignment components and the expert, sample, and layer selection mechanisms, and tests sensitivity to alignment strength and expert count. These comparisons provide concrete evidence for the choices underlying the method.

  • The paper connects performance improvements with changes in internal routing. RA-MoE outperforms SFT in 53 of 54 settings, while further analyses show reduced cross-lingual routing divergence in middle layers and increased task-expert selection rates. These findings show that the gains are accompanied by routing changes consistent with the alignment objective.

  • I found the idea of reusing successful task routing across languages particularly interesting. Experiments using non-English reference languages and reusing expert configurations identified for another target language support the potential of this approach beyond an exclusively English-reference setting.

Critical weaknesses
  • English retention after target-language fine-tuning is not evaluated. Using successful English routing as a reference does not guarantee that English capabilities are preserved. A comparison of English performance before and after adaptation, including standard SFT, is needed to assess whether the target-language gains come at the expense of English performance.

  • The experiments lack a baseline that performs standard SFT only on ci examples. Removing ci filtering changes which examples receive the alignment loss; it does not restrict SFT itself to ci examples. A comparison under a controlled training budget is needed to assess the additional value of routing alignment over simply focusing training on examples answered correctly in English but incorrectly in the target language.

  • The method depends on parallel English–target-language task data. Although translation quality is assessed, applying the method to target-language-native data still requires constructing reliable English counterparts. This requirement and its implications for practical applicability warrant explicit discussion.

  • OOD evaluation for logical reasoning and knowledge understanding is missing. The current results do not establish whether the gains extend beyond the distributions represented in the adaptation data. Evaluation on independent benchmarks with different domains or problem types would strengthen the generalization evidence.

Questions for authors

Please see my comments in the Weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2jP6 — Part 1: English retention and budget-matched ci-only SFT (W1, W2)

Thank you for your constructive review and for recognizing the component ablations, routing analyses, and cross-language transfer results. We address your concerns with English-retention evaluation, a budget-matched ci-only SFT baseline, clarification of the parallel-data requirement, and evaluation on independent benchmarks.

W1. How much English performance is retained after target-language adaptation?

We evaluated the original OLMoE checkpoint and its SFT and RA-MoE variants on the corresponding English test sets after adaptation on Chinese MMLU or Spanish GSM8K. All checkpoints used the same evaluation protocol within each benchmark.

Target-language adaptationEnglish evaluationBase (%)SFT (%)RA-MoE (%)
Chinese MMLUEnglish MMLU49.749.649.6
Spanish GSM8KEnglish GSM8K47.046.646.1

On English MMLU, RA-MoE matched SFT and scored 0.1 percentage points below the original checkpoint. On English GSM8K, RA-MoE scored 0.5 points below SFT and 0.9 points below the original checkpoint. Thus, English performance changed only slightly in these settings, including a small additional decline relative to SFT on GSM8K.


W2. Can focusing SFT on ci examples reproduce the gains under a controlled training budget?

  1. Budget-matched baseline.

    We added ci-only SFT for OLMoE on Chinese LogicBench. This baseline resamples eligible ci examples for cross-entropy training and uses no routing alignment loss.

    Both SFT variants used:

    • Training seed 42.
    • 12,908 training rows after resampling.
    • Global batch size 256.
    • 51 optimizer steps.
    • 21,914 CE-supervised tokens.
  2. Results under the same evaluation protocol.

    MethodAccuracy, mean ± SD across five evaluation runs
    Full-data SFT44.5 ± 0.7%
    ci-only SFT43.8 ± 0.6%
    RA-MoE47.8 ± 0.8%

    The ci-only baseline scored below full-data SFT in all five evaluation runs. Its mean accuracy was 0.7 percentage points below full-data SFT at the matched training budget and 4.0 points below RA-MoE.

    These results show that concentrating supervised training on ci examples alone did not reproduce the RA-MoE improvement in this setting.

AuthorsOpenReview ↗

Response to Reviewer 2jP6 — Part 2: Parallel-data requirements and independent-benchmark evaluation (W3, W4)

W3. How does the parallel-data requirement affect applicability to native target-language task data?

RA-MoE requires paired task examples so that a successful reference-language response can provide routing supervision for its target-language counterpart.

  1. Data construction in our experiments.

    We start from English task data and translate it into the target languages. SFT and RA-MoE use the same translated target-language training data, sharing the translation and quality-checking costs.

  2. Starting from native target-language data.

    Applying RA-MoE requires constructing and validating reference-language counterparts. This includes checking that translation preserves the question’s meaning, numerical or symbolic constraints, option order, and answer validity. Questions involving language-specific or culturally specific knowledge may require additional review.

    This is an additional preparation requirement compared with standard SFT on existing native target-language data.

  3. Scope of the proxy extension.

    Section 5.5 uses response perplexity to select alignment examples when exact correctness labels are unavailable. Its IFEval results extend the approach to open-ended instruction-following data, while retaining the requirement for paired instruction–response examples. Applying the method to unpaired native corpora remains outside the current experimental scope.


W4. Do the gains extend to independent logical reasoning and knowledge benchmarks?

We evaluated the existing OLMoE checkpoints on two independent Chinese benchmarks, without further fine-tuning. All checkpoints used the same evaluation protocol within each benchmark.

Adaptation task → Independent test setBase (%)SFT (%)RA-MoE (%)RA-MoE − SFT
Chinese MMLU → INCLUDE Chinese31.432.133.6+1.5 pp
Chinese LogicBench → LogiQA 2.0 Chinese29.129.430.5+1.1 pp

RA-MoE outperformed target-language SFT on both independent test sets. These results show that its gains extended to a different benchmark in each of the two task categories tested.

Thank you for the clarifications and additional experiments. They address my main concerns and reinforce my positive assessment of this interesting and well-motivated work. I encourage the authors to incorporate these results and clarifications into the paper. I am keeping my score.

AuthorsOpenReview ↗

Thank You for Your Feedback and Support

Dear Reviewer 2jP6,

Thank you very much for revisiting our responses and confirming that the additional experiments and clarifications address your main concerns. We sincerely appreciate your positive assessment and continued support.

We will incorporate the additional results and clarifications into the paper, as you suggested. Your feedback has helped us strengthen the evaluation and make the method’s scope and requirements clearer.

Thank you again for your time and thoughtful engagement throughout the discussion.

Best regards,
The Authors

Reviewer_TPnsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces RA-MoE, motivated by the finding that middle layers exhibit more similar routing patterns for the same task across different languages than early and late layers. This method breaks the data into four categories based on correctness in English and the target language (cc, ci, ic, ii). It then uses the ci examples to guide the model into closer routing between the two languages on parallel data. The performance is compared with normal SFT and other MoE adaptation baselines, showing improved task performance.

Critical strengths

The paper takes an established finding in multilingual models (early and late layers are more language-related while middle layers are more task-related) and uses it to form a loss that better takes advantage of mixture-of-expert architectures compared to SFT. The method leverages the model's existing potential for cross-lingual transfer to route input in lower-resource languages to experts used in successful higher-resource language generation in the middle layers, thereby improving task performance for the lower-resource language.

Critical weaknesses
  1. The main motivation of this work is that correct task performance relies on task experts in middle layers, where routing patterns can be shared across languages. The process for identifying these experts involves comparing correct English task examples with general text from FLORES-200. However, these differences may reflect domain, input format, or generation characteristics rather than the task-specific reasoning ability the authors aim to identify. The authors present multiple experiments in Section 5, but they do not sufficiently address this issue. In Section 5.2, they show that a training objective designed to minimize routing divergence indeed reduces that divergence. In Section 5.4, they report higher expert overlap across languages (for the same task) than across tasks. This supports the assumption motivating RA-MoE, but does not establish that the identified experts are task-specific or responsible for the observed improvements. Interventions on the identified experts could provide stronger evidence for the proposed mechanism.

  2. There are certain limitations of this work that should be addressed. RA-MoE relies on ci examples (correct source, incorrect target), and Section 5.1 shows that the proportion of these examples positively correlates with performance gains. However, the paper does not establish whether the method would be effective when the performance gap is small or when both languages perform poorly. I believe this limitation should be addressed in the paper. Additionally, my understanding, based on the appendix, is that the reported standard deviations are computed across evaluation runs rather than independent training runs with different seeds. As a result, we cannot conclude that the training method is stable, especially when the reported improvements are modest.

Questions for authors
  1. Regarding the task experts, would interventions on the selected experts meaningfully affect task performance? Do correct and incorrect responses from the same task exhibit different usage of these experts?
  2. Are the reported standard deviations computed across independent training runs with different seeds, or only across evaluation runs?
  3. What happens when the number of ci examples is small? Is there a minimum proportion of ci examples required for the method to be effective?
  4. In Table 3, would sequential SFT on the reference language followed by the target language yield improvements similar to RA-MoE, given that non-English reference languages are adapted before RA-MoE training?
  5. Does perplexity remain an effective selection criterion for tasks requiring reasoning or factual correctness, where output likelihood may not reliably reflect performance?
4 comments
AuthorsOpenReview ↗

Response to Reviewer TPns — Part 1: Expert interventions and routing in correct versus incorrect responses (W1/Q1)

Thank you for your constructive review and for recognizing the potential of transferring successful task routing across languages. Your questions prompted us to add expert interventions, within-task routing comparisons, independent training runs, controlled reductions of alignment supervision, a sequential SFT baseline, and validation of the PPL selector.


W1/Q1. Do the identified experts affect task performance, and are they used differently in correct and incorrect responses?

  1. Controlled expert intervention.

    We evaluated Qwen1.5-MoE-A2.7B on Spanish GSM8K using the same task experts identified independently from MetaMathQA for both the SFT and RA-MoE checkpoints. Our main intervention excluded these experts from top-k routing and allowed subsequent eligible experts to fill their slots.

    We constructed five control sets with the same number of experts per selected layer. Using 256 independent Spanish calibration examples, we jointly matched top-k hits and normalized routing mass across both checkpoints. The mean absolute per-layer relative matching error was below 0.5% for each metric and checkpoint.

    CheckpointUnperturbed accuracyAfter excluding task expertsTask-expert dropMean control dropExcess drop over controls (95% CI)
    SFT58.5%55.3%3.2 pp1.5 pp1.7 pp [0.5, 2.9]
    RA-MoE61.3%51.0%10.3 pp0.7 pp9.6 pp [7.7, 11.4]

    Zeroing the selected experts’ outputs produced the same pattern, with excess drops over controls of 1.2 pp for SFT and 7.9 pp for RA-MoE.

    During rerouting, the controls had approximately 11% fewer top-k hits and 9% less routing mass than the task experts in both checkpoints. These statistics were collected on each intervention’s own generation trajectory and can reflect both differences from the calibration distribution and intervention-induced trajectory changes.

    The selected experts caused larger performance drops than the controls in both checkpoints. Moreover, excluding them reduced RA-MoE’s advantage over SFT from +2.8 pp to −4.3 pp, supporting their contribution to the observed improvement.

  2. Correct versus incorrect responses within each checkpoint.

    We replayed each model’s unperturbed answers to all 1,319 Spanish GSM8K questions with teacher forcing. We measured routing at answer-token positions across the 11 selected MoE layers, averaging first within each answer and then across layers. Expert sets were fixed independently of GSM8K correctness.

    For each response, we computed routing-mass specificity, defined as the routing mass assigned to task experts minus the mean mass assigned to the five matched control sets. Confidence intervals resample responses, and p-values use two-sided label permutation tests within each checkpoint.

    CheckpointCorrect responsesIncorrect responsesCorrect minus incorrect (95% CI)p-value
    SFT0.0008360.000464+0.000372 [0.000014, 0.000717]0.0375
    RA-MoE0.0009900.000429+0.000560 [0.000210, 0.000917]0.0016

    Correct responses assigned more routing mass to the selected task experts relative to the matched controls in both checkpoints, with confidence intervals above zero. Top-k-hit specificity showed the same direction, with clearer evidence in RA-MoE than in SFT.

    Correct answers were approximately 38–40 tokens shorter on average. The routing-mass difference remained positive within all four answer-length quartiles in both checkpoints.

These results provide complementary evidence of the experts’ functional contribution and their association with successful responses within the same task and language. Domain and format differences may still contribute to the task–FLORES selection score.

AuthorsOpenReview ↗

Response to Reviewer TPns — Part 2: Scarce alignment supervision and training variability (W2/Q3, W2/Q2)

W2/Q3. What happens when few ci examples are available? Is there a minimum proportion required?

  1. Controlled reduction of alignment supervision.

    We retained different fractions of the eligible ci pool for routing alignment in two OLMoE settings. All target-language training examples continued to contribute to the cross-entropy objective. The percentages below refer to retention within the original ci pool.

    Eligible ci pool retainedArabic GSM8K accuracy (%)Chinese MMLU accuracy (%)
    0% (SFT)9.432.1
    5%9.432.2
    10%9.432.3
    25%9.932.5
    100%10.833.6

    At 5–10% retention, performance was close to SFT. At 25%, the observed gains were 0.5 and 0.4 pp, compared with 1.4 and 1.5 pp using the full pool.

    Thus, 25% was the lowest tested retention rate with positive gains in both settings. The minimum useful amount of supervision may vary with the task, language pair, and training protocol.

  2. Small language gaps and weak reference performance.

    A small aggregate accuracy gap can still leave useful ci supervision, since the gap equals p(ci)−p(ic)p(ci)-p(ic). When reference-language accuracy is low, fewer successful reference responses are available for alignment. RA-MoE continues to train on all target-language examples through the cross-entropy objective.

    Our subsampling results show that the additional benefit decreases as alignment supervision becomes scarce, approaching standard SFT at 5–10% retention in the tested settings. Section 5.1 further suggests that target-language competence also matters. For example, OLMoE–Bengali exhibits little improvement despite a high ci proportion.


W2/Q2. Are the reported standard deviations computed across training runs or evaluation runs?

The standard deviations in Table 1 summarize five evaluations of the same trained checkpoint in each setting.

To assess training variability, we additionally trained OLMoE with paired seeds 42, 123, and 456 in three settings. In the following table, mean ± SD is computed across independent training seeds. Gains are calculated within each seed before aggregation.

SettingSFT (%)RA-MoE (%)Paired gain (pp)
Chinese LogicBench44.40 ± 0.5647.63 ± 0.21+3.23 ± 0.70
Spanish GSM8K28.20 ± 0.1729.60 ± 0.20+1.40 ± 0.26
French MMLU38.00 ± 0.3039.30 ± 0.20+1.30 ± 0.10

RA-MoE improved over SFT in all nine paired training runs, with gains ranging from 1.1 to 3.9 percentage points. These experiments provide training-variability evidence across three tasks with OLMoE.

AuthorsOpenReview ↗

Response to Reviewer TPns — Part 3: Sequential SFT and validation of PPL selection (Q4, Q5)

Q4. Can reference-language SFT followed by target-language SFT achieve gains similar to RA-MoE?

We continued supervised fine-tuning of Qwen1.5-MoE-A2.7B on Japanese from the Spanish SFT checkpoint and used the same Japanese GSM8K evaluation protocol as Table 3.

MethodJapanese GSM8K accuracy
Direct Japanese SFT42.3%
Spanish SFT → Japanese SFT43.2%
RA-MoE with Spanish reference routing44.5%

Sequential SFT recovered part of the improvement, gaining 0.9 pp over direct Japanese SFT. RA-MoE achieved 1.3 pp higher accuracy than sequential SFT in this setting.


Q5. Does perplexity remain an effective selection criterion for reasoning and factual questions?

  1. Validation protocol.

    We evaluated the PPL selector against correctness-defined ci labels using OLMoE on held-out English–Arabic MetaMathQA pairs used for GSM8K adaptation and English–Chinese MMLU pairs.

    • Thresholds were fitted on a separate calibration split and applied to the holdout.
    • PPL was computed over reference-answer tokens: full reference solutions for MetaMathQA and the correct option letter for MMLU.
    • We compared the proxy with the true ci base rate and an equal-size selection using the lowest English PPL.
  2. Selection results.

    Settingci base rateLow-English-PPL precisionProxy precision (95% bootstrap CI)Proxy recall
    MetaMathQA, English → Arabic77.54%86.07%81.04% [80.34%, 81.73%]26.13%
    MMLU, English → Chinese22.08%24.20%59.90% [59.00%, 60.80%]68.46%

    The proxy selected 25.00% and 25.24% of the respective filtered candidate pools.

    On MMLU, proxy precision exceeded random selection by 37.8 pp and the low-English-PPL control by 35.7 pp. This supports the usefulness of the cross-language PPL gap for selection in the tested multiple-choice knowledge setting.

    On the mathematical data, proxy precision exceeded random selection by 3.5 pp but fell 5.0 pp below the low-English-PPL control. Adding the cross-language PPL-gap condition therefore did not improve precision over the English-PPL baseline in this high-ci-base-rate setting.

These results show that the usefulness of PPL-based selection depends on the task. The MMLU result concerns correct-option likelihood. Section 5.5 separately provides evidence of downstream training benefits from proxy-based selection on IFEval.

AuthorsOpenReview ↗

Gentle Reminder to Revisit Our Rebuttal and Evaluation

Dear Reviewer TPns,

Thank you again for your positive assessment and thoughtful suggestions. Your questions helped us strengthen the evidence for the method’s mechanism and clarify its practical limitations.

We have added expert interventions, comparisons of routing in correct and incorrect responses, independent training-seed experiments, scarce-supervision analysis, a sequential-SFT baseline, and validation of PPL-based selection.

We would be grateful if you could revisit our responses and consider whether the additional evidence warrants an updated assessment. Please let us know if any concerns remain or if further clarification would be helpful.

Thank you very much for your time and continued support.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.