acceptodds
Under review as a conference paper at ICLR 2027

Mixture of Outputs: Dynamic Decoding-Time Expertise Integration Between Large Language Models

Abstract

Adapting large language models (LLMs) to new capabilities typically requires additional post-training, which can be costly in both supervision and computation. Existing approaches reduce this cost either through parameter-efficient fine-tuning or decoding-time adaptation, but the former still requires task-specific training of the target model, while the latter often relies on fixed combinations of auxiliary predictions or specially matched proxyxen when the desired capability is already available in another specialized models. Meanwhile, specialized capabilities can often be acquired more cheaply in smaller models. This motivates a question: can a smaller specialized model serve as a low-cost capability donor for a larger model? Therefore we study whether such capabilities can instead be transferred at decoding time, without updating the pretrained models themselves. To this end, we propose Mixture of Output (MoO), a lightweight framework that dynamically transfers capabilities from one or more frozen donor models to a frozen receiver. At each decoding step, MoO uses the receiver’s hidden state to determine how much each donor should contribute to the next-token prediction. Only a lightweight module is trained to predict the contribution of donors, while the receiver and donor models remain frozen. Our evaluation shows that MoO improves the receiver by up to 14.27 percentage points on average across nine benchmarks. For alignment transfer, MoO reduces ASR from 0.44 to 0.02 and from 1.00 to 0.17 on the receiver models. Combining multiple reasoning donors provides further gains of 2.01 and 1.92 points on the two tested receivers. Overall, these results demonstrate that MoO can effectively transfer complementary capabilities from specialized donor models to a frozen receiver across various tasks.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
large language models, decoding-time model fusion, capability transfer, frozen model collaboration, mixture of experts, context-dependent gating, LLM alignment, parameter-efficient adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BvgYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Mixture of Outputs (MoO), a decoding-time framework that combines a frozen receiver’s logits with those of frozen specialized donors. A lightweight gate uses the receiver’s hidden state to predict token-dependent donor strengths, with vocabulary alignment and logit-scale calibration supporting fusion.

Critical strengths
  1. Method is good, lightweight, compositional adaptation. MoO learns token-dependent donor weights while keeping the receiver and donors frozen. Independent gates allow multiple donors to contribute simultaneously, offering a practical way to reuse specialized models without updating the receiver.
  2. The visualization is done very well, making it relatively easy to understand the motivation and methods of the paper, and the result presentation is also relatively beautiful. If possible, could please introduce what tools were used to create these figures?
Critical weaknesses
  1. Insufficient reproducibility. The paper does not adequately specify the gate-training datasets, cross-vocabulary projection procedure, or safety-evaluation criteria, and further understanding of the specific implementation is hindered by a lack of supplementary materials.
  2. Limited evidence of robust superiority. Approximately 53% of the headline 14.27-point improvement comes from MBPP, where the receiver scores only 0.5%. This unusually low baseline requires investigation, and small reported gains need statistical validation. This is my main concern.
Questions for authors

See the Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer BvgY

We thank the reviewer for finding MoO a lightweight, compositional way to reuse specialized models while keeping both receiver and donors frozen, and for the kind words on the figures and result presentation. We address the two weaknesses below.

W1: Reproducibility.

Thank you. We address the three missing items in turn: (1) gate-training data, (2) the cross-vocabulary mapping and (3) the safety-evaluation criteria.

(1) Gate-training data. Donor-generated, correctness-filtered examples: 1,736 for the single AceMath donor (1,000 math, 236 code, 500 general), 3,472 for the two-donor setting, and 860 for alignment. Reasoning questions come from training splits; AdvBench and XSTest items 1–200 are reserved for evaluation.

(2) Cross-vocabulary mapping. We correct the submission: the Gemma/Qwen map is a deterministic sparse token-index mapping built from the two vocabularies, not a learned projection. Donor probabilities are summed into the receiver vocabulary through this index.

(3) Safety evaluation. The first 200 AdvBench behaviors in the user chat template, with no jailbreak suffix; refusals are detected by a fixed list of refusal markers in the first 40 characters, and ASR is the share not refused. Over-refusal uses the first 200 non-contrast XSTest safe prompts with the same rule.

Full details are in our responses to Reviewer Q9v7 (Q2) and Reviewer YQeL (Q1, Q2).

Revisions: Appendix A.3–A.6 (L823–913) now specify the gate-training data, the cross-vocabulary mapping, the safety-evaluation protocol and the training and baseline settings, and Section 3.2 (L213–215) corrects the mapping description. We will release anonymized code for gate training, fusion decoding and evaluation.

W2: MBPP's share of the headline gain and statistical validation.

Thank you; we agree with this analysis, and the estimate is correct (68.0 / 9 = 7.56 of 14.27 points). We investigated it in four parts: (1) the cause of the 0.5% baseline, (2) whether MBPP is a reliable measure here, (3) the gain without MBPP and (4) statistical validation.

(1) Cause of the 0.5% baseline. Our MBPP protocol is zero-shot: the prompt asks for a single Python code block and shows the first test case; we extract the first fenced code block (or the whole completion if there is none), run all tests with an 8-second timeout, and allow 1,024 new tokens. In 198 of the 200 Qwen2.5-Math-7B outputs no Python function can be extracted. The cause is the zero-shot prompt rather than the extraction rule: with a three-shot prompt, the receiver scores 66.5% and MoO 65.5%. The zero-shot MBPP gain thus reflects instruction following transferred from the donor, not improved coding.

(2) MBPP is not a reliable measure for this receiver. Re-training the gate with two more seeds gives MBPP 0.010 and 0.165, versus 0.685 for the reported seed. We therefore no longer rely on MBPP and no longer report the 14.27-point figure.

(3) The gain without MBPP is robust across seeds. Excluding MBPP, MoO improves Qwen-Math from 0.4836 to 0.5682 ± 0.0090 over three seeds (+8.46 points). The gain holds for every seed (+7.55 to +9.36) and comes from the math benchmarks (GSM8K 0.575 → 0.860 ± 0.053; MATH500 0.530 → 0.710 ± 0.048).

(4) Statistical validation. Per-seed results are in our response to Reviewer YQeL (Q4). We are re-running evaluations with per-example logging to compute paired bootstrap intervals.

Revisions: The abstract (L24–26) and introduction (L103–104) report the seed-averaged eight-task gain. Appendix A.5 adds the MBPP protocol, the analysis of the receiver's outputs and the three-shot results (L871–891), and Section 4 adds the per-seed results (Table 4, L416–431). Paired bootstrap intervals will be added once the re-evaluation is complete.

Thank you for the author's detailed response. My main concern has been resolved. Considering the overall quality of this work, I would like to maintain my rating.

Reviewer_YQeLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Mixture of Outputs (MoO), a decoding-time framework for transferring capabilities from frozen donor language models to a frozen receiver. A lightweight MLP uses the receiver's last hidden state to predict independent, token-dependent donor strengths. Donor logits are aligned to the receiver vocabulary, centered, and calibrated to the receiver's logit scale before gated addition and protected rescaling. Only the gate is optimized during fusion training; heterogeneous model pairs additionally require a separately learned vocabulary projection. The evaluation covers nine reasoning benchmarks, alignment transfer across three receiver families, and two-donor reasoning integration. Reported results include a 14.27-percentage-point average improvement for single-donor transfer to Qwen Math and additional gains of approximately two points from joint donors over the best single-donor configuration. Gate diagnostics and fixed-strength comparisons investigate the proposed routing mechanism.

Critical strengths
  • A concrete and flexible mechanism for composing frozen models.

Sections 3.2–3.3 specify a receiver-conditioned gate with independently bounded donor strengths, allowing multiple donors to contribute simultaneously. The formulation accommodates vocabulary alignment and logit-scale calibration while preserving the receiver's generation interface. This is a useful design for reusing specialized models without modifying their checkpoints.

  • Meaningful transfer gains in several evaluated settings.

With AceMath, Qwen Math improves from 0.4304 to 0.5731 on the nine-benchmark average; joint donors further improve this to 0.5923 (Table 8). In alignment transfer, Llama's ASR decreases from 1.0000 to 0.1700 while its capability average increases from 0.3299 to 0.4743 (Table 2). These results support the feasibility of beneficial output-space transfer. Reporting the full task-level results also makes the accompanying regressions and tradeoffs visible.

Critical weaknesses
  • Essential training and evaluation details are missing, limiting reproducibility and interpretation.

Appendix A reports optimization settings and evaluation sample counts, but does not identify the actual gate-training datasets, their sizes and mixture proportions, or their separation from evaluation data. The learned vocabulary projection is central to the heterogeneous experiments, yet its parameterization, training objective, training data, and handling of different token boundaries are unspecified. The alignment evaluation also lacks a sufficiently explicit attack protocol and response-scoring procedure. These omissions affect central results: without these details, it is difficult to reproduce the reported transfer or assess the fairness of the comparisons.

  • The evidence does not sufficiently isolate the benefit of token-dependent routing from alternative fusion choices.

The fixed-strength sweep in Table 10 evaluates only three nonzero coefficients, and the reasoning results in Tables 7–9 do not include corresponding fixed-fusion controls. A learned constant gate trained on the same data would be a particularly informative control for whether context dependence itself matters. Table 3 measures differences in gate strengths rather than downstream performance, while Figure 5 uses a separate configuration and only two illustrative prompts. These diagnostics establish variation in routing but do not establish that this variation causes the reported gains. This distinction matters because the results show substantial tradeoffs: for example, AceMath fusion improves Qwen Base's mathematics scores but reduces MMLU from 0.7260 to 0.6280 and ARC-C from 0.8960 to 0.7940 (Table 7). These regressions do not invalidate the demonstrated transfer, but they make evidence for selective incorporation particularly important.

Questions for authors
  • Training and evaluation clarification: What datasets, sample counts, and mixture proportions were used to train each gate, and how were training examples separated from the evaluation benchmarks? For the alignment results, what exact attack procedure and criteria were used to determine attack success and benign-prompt refusal?

  • Vocabulary-alignment clarification: How is the learned projection parameterized and trained, and how are donor and receiver predictions synchronized when their tokenizers segment the shared text differently? Please specify the projection's training data and objective.

  • Targeted validation of routing: On an existing representative reasoning configuration, how does MoO compare with a learned constant donor strength optimized on the same training data, using identical alignment, calibration, and rescaling? For the two-donor setting, a learned constant strength for each donor would help distinguish the benefit of adding donors from the benefit of token-dependent routing.

  • Reliability of the multi-donor gains: Can the approximately two-point improvements over the best single-donor configuration be accompanied by uncertainty estimates, such as paired bootstrap intervals on the existing evaluation examples? Table 4 reports one random seed, and Table 5 uses relatively small subsets for several benchmarks. Such intervals would help assess evaluation-sample uncertainty, although they would not measure variability across independently trained gates.

6 comments
AuthorsOpenReview ↗

Response to Reviewer YQeL (1/4)

We thank the reviewer for describing MoO as a concrete and flexible mechanism for composing frozen models, for recognizing the transfer gains in several settings, and for noting that the full task-level results make the trade-offs visible. The reviewer's suggested constant-gate control was especially useful: it identifies where token-dependent routing matters. We address each question below.

Q1: Training and evaluation details and the alignment protocol.

Thank you. We answer in four parts: (1) gate-training data and mixture proportions, (2) separation of training and evaluation data, (3) evaluation subsets and (4) the alignment evaluation protocol.

(1) Gate-training data. Reasoning, single AceMath donor: 1,736 AceMath-generated examples (1,000 math, 57.6%; 236 code, 13.6%, kept only if they pass the unit tests; 500 general, 28.8%). Two-donor setting: 3,472 examples (50% the AceMath examples above; 50% R1-Distill-Qwen-1.5B responses to 987 GSM8K and 749 MATH training questions, kept only when the final answer is correct). Alignment: 860 examples (220 code, 220 general, 150 AdvBench harmful, 50 XSTest safe, 220 multilingual).

(2) Train/evaluation separation. Reasoning questions come from the GSM8K and MATH training splits, while evaluation uses test sets. For alignment, AdvBench and XSTest items 1–200 are reserved for evaluation; training draws only from item 201 onward. No validation set is used; we evaluate the checkpoint at the final training step.

(3) Evaluation subsets. Each reasoning benchmark uses the first N items of its test split (N as in Appendix Table 5). In the alignment capability suite, the 200 MMLU items are sampled with seed 0.

(4) Alignment evaluation protocol. All methods use identical prompts and greedy decoding.

  • Attack procedure. The first 200 AdvBench harmful behaviors, each placed in the model's user chat template, with no jailbreak suffix or adversarial optimization.
  • Attack success. A string-matching judge: a response counts as a refusal if it starts with, or contains within its first 40 characters, one of a fixed list of refusal markers. ASR is the share of harmful prompts that are not refused.
  • Benign-prompt refusal. The first 200 XSTest safe prompts after excluding the contrast types, judged by the same marker rule; over-refusal is the share refused.
  • IFEval-I. Instruction-level strict accuracy from the official strict checker.

Revisions: Appendix A.3 (L823–847, Table 9) gives the training data, mixture proportions, train/evaluation separation and evaluation subsets for every configuration. Appendix A.5 (L871–884) gives the attack procedure, the refusal-marker list and the judging rules.

AuthorsOpenReview ↗

Response to Reviewer YQeL (2/4)

Q2: Vocabulary alignment and tokenizer synchronization.

We answer in three parts: (1) a correction regarding the “learned projection”, (2) how the mapping is applied and (3) how donor and receiver are synchronized when their tokenizers segment text differently.

(1) The mapping is not learned. We correct the submission: the Gemma/Qwen cross-vocabulary map is a deterministic sparse mapping built from the two tokenizers' vocabularies, so it has no parameters, training data or objective.

(2) How the mapping is applied. Each donor token is mapped to a receiver token index. At each step the donor distribution is computed with a softmax, its probability mass is summed into the receiver vocabulary through this index, and the log of the result serves as the donor's logits before centering and calibration. Donor tokens whose string also exists in the receiver vocabulary map to that token; the remaining donor tokens fall back to a receiver token derived from their subword decomposition. Receiver tokens that receive no donor mass have their probability clamped at a small ε before the log, so they receive a strongly negative donor logit, which centering, calibration and the final clipping of the fused logits to [−50, 50] keep bounded.

(3) Synchronization. Receiver and donor share text, not tokens. At decoding step t the receiver selects yty_t; its string is appended to the shared text, and the donor re-tokenizes the full text. The donor's KV cache is reused for the longest common prefix of its previous and new token sequences. When a BPE boundary changes, the cache is cropped to that prefix and the suffix is recomputed; if the cache cannot be cropped, the whole sequence is recomputed. During teacher-forced gate training, each example is re-tokenized with the donor tokenizer, and the donor's next-token logits are gathered back to the corresponding receiver positions.

Cross-tokenizer mapping is an inherent source of noise, which is why we treat Gemma/Qwen as a stress test. With the same mapping, fixed fusion (γ = 0.3–0.5), DExperts and a learned constant gate all collapse (Capability Avg. at most 0.21 / 0.11 / 0.14), whereas MoO keeps the receiver's level (0.71).

Revisions: Section 3.2 (L213–215) replaces the “learned projection” description with the deterministic mapping. Appendix A.4 (L848–869) describes the mapping construction, the synchronization procedure and its limitations.

AuthorsOpenReview ↗

Response to Reviewer YQeL (3/4)

Q3: Learned constant donor strength versus token-wise routing.

We thank the reviewer for this precise suggestion and ran exactly this control: one learnable scalar per donor, γk=γmax⁡ σ(βk)\gamma_k = \gamma_{\max}\,\sigma(\beta_k), trained with the same data, objective, alignment, calibration, rescaling and number of steps. The results (Tables R3 and R4) show three things, and they also change how we present the routing diagnostics (point 4).

Table R3. Reasoning suite, nine-task average.

Setting (nine-task avg.)Learned constant gateMoOΔ MoO − constant (pts)
Qwen-Math + two donors0.58600.5923+0.63
Qwen-Base + two donors0.62650.6491+2.26
Qwen-Math + AceMath0.60760.5731−3.45
Qwen-Base + AceMath0.64860.6290−1.96

Table R4. Alignment.

Alignment settingCap. Avg. ↑ASR ↓Over-ref. ↓IFEval-I ↑
Gemma receiver0.69960.01500.12000.7959
Gemma/Qwen, learned constant gate0.14040.76500.06500.4850
Gemma/Qwen, MoO0.71360.01500.12500.7707
Llama receiver0.32991.00000.00500.3445
Llama, learned constant gate0.43120.01000.13500.3950
Llama, MoO0.47430.17000.05500.4190

(1) Two donors (the decomposition the reviewer proposed). With constant strengths, adding the second donor lowers the average (Qwen-Math 0.6076 → 0.5860; Qwen-Base 0.6486 → 0.6265). With token-dependent routing over the same two donors, MoO scores 0.63 and 2.26 points above the two-donor constant gate. A fixed strength selected on the test benchmarks still exceeds MoO by 0.9–1.7 points (see our response to Reviewer Q9v7, Q1); the advantage of routing is that it needs no such selection.

(2) Heterogeneous pair. On Gemma/Qwen the constant gate collapses (Capability Avg. 0.14, ASR 0.77), as do fixed fusion (γ = 0.3–0.5) and DExperts, while MoO stays at the receiver's level. On Llama, the constant gate reaches lower ASR (0.010 vs 0.170) at the cost of higher over-refusal (0.135 vs 0.055), lower capability (0.431 vs 0.474) and lower IFEval-I (0.395 vs 0.419).

(3) Single compatible donor. A learned constant strength is already effective here (3.45 and 1.96 points above MoO): the useful strength does not need to vary across tokens, and the constant gate is a special case within MoO's gate family. Fixed strengths and Proxy-Tuning are likewise strong in this setting (see our response to Reviewer Q9v7, Q1).

(4) Routing diagnostics. We agree that Table 3 and Figure 5 show variation in routing rather than causation. The constant-gate control above now serves as a controlled ablation: it removes token-wise variation while keeping everything else fixed.

Together, these results identify where token-dependent routing matters: when several donors must be combined, and when donor signals are unreliable, as with heterogeneous tokenizers. This is the setting MoO targets. Additional seeds for these controls are in progress.

Revisions: The reasoning constant-gate control is added to Section 4 as part of Table 3 (L399–414); the alignment control is summarized in Section 4 (L462–470) and listed in Appendix C (Table 14). The gate-input diagnostic (Table 3 in the submission, Table 5 in the revision) and Figure 5 are presented as descriptive diagnostics of routing (L490–492).

Reviewer_Q9v7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

MoO combines next-token outputs of a frozen receiver model and 1+ frozen donor models. A small gate uses the receiver's hidden state to predict how strongly each donor should contribute at each decoding step. Donor logits are aligned to the receiver vocabulary and calibrated before fusion. The paper evaluates reasoning and alignment transfer, including combinations of two donors and models with different tokenizers, and then, analyzes the learned gate's behavior

Critical strengths
  1. Reusing capabilities from specialized donors without updating the receiver is a neat and useful goal. Conditioning donor contributions on the current receiver state makes sense, particularly when a donor helps on some tasks but interferes on others. Independent gates also allow several donors to contribute at the same decoding step, which is a good motivation

  2. The task selection is broad and includes math-style reasoning, knowledge, coding, and alignment. Some improvements are very big: Qwen Math with AceMath improves by 14.27% on the 9-task average; alignment experiments show large receiver-relative reductions in attack success for Qwen and Llama. The multi-donor experiments also evidence that combining donors can improve on the best single-donor fusion

Critical weaknesses
  1. Comparisons do not establish the main advantage used to motivate MoO: that learned token-wise routing integrates donor capabilities more effectively than existing fusion methods. Table 1 and the complete reasoning tables compare only receiver-only, donor-only, and MoO outputs, and it is not presented in a clear way. The competing fusion and adaptation baselines are reported for alignment, leaving the reasoning gains without the relevant method comparisons. For alignment, the results show tradeoffs rather than a consistent advantage: on Llama, proxy-tuning at alpha=1 has higher capability and instruction-following scores and lower attack success, although it refuses more benign prompts. The fixed-strength sweep tests only 0, 0.3, 0.4, and 0.5. This is particularly limiting for the heterogeneous setting, where the nonzero fixed coefficients severely damage performance. A better-selected fixed strength or a constant gate trained with the same supervision would help isolate the benefit of token-wise routing. I felt like the others are trying to hide something, or present the results in a much more favorable way to the method and its results than their own results show it. Moreover, the distinction from the adaptive fusion method On Giant's Shoulders, cited in Section 2.2, also needs to be clearer.

  2. Essential experimental details are missing, and the presentation makes the results harder to assess than necessary. I could not find the gate-training datasets, their sizes and split construction, or the parameterization and training protocol for the learned vocabulary projection. The latter is central to the heterogeneous-model results. The SFT-LoRA setup and exact donor-budget regularizer also need specification. Meanwhile, Table 1 shows three mathematical benchmarks but leaves substantial knowledge and coding losses to the appendix. I find this practice highly problematic. Bolding compares MoO with the donor, although improvement over the receiver motivates the method. For Qwen Base with AceMath, the average gain is only 0.15 percentage points despite large math gains, because performance declines elsewhere. Those losses need analysis given the claim that the gate suppresses unhelpful donor signals. Only one training seed is listed, and the smallest gains have no uncertainty estimates. The long paragraphs, dense layout, and 3D and radial plots further impede comparison.

Questions for authors
  1. How does MoO compare with fixed fusion, proxy-tuning, and the closest adaptive fusion method on the reasoning suite? Can you compare against a constant gate trained with the same data and objective to isolate the contribution of token-wise adaptation?

  2. What datasets and splits train the gate and vocabulary projection? Please specify their sizes, the projection's parameterization and objective, the donor-budget regularizer, and the SFT-LoRA baseline configuration, in detail.

  3. Why do some reasoning gains accompany substantial knowledge and coding losses? In particular, why is Qwen Math's receiver-only MBPP score 0.5%, and how much does output formatting contribute to its improvement? How do the Qwen Base losses inform the claim that the gate suppresses unhelpful donor signals?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Q9v7 (1/4)

We thank the reviewer for finding the goal of reusing specialized donors without updating the receiver useful, and for recognizing the motivation for state-conditioned, independent donor gates and the breadth of the evaluation. We take the concern about presentation seriously: the revision now reports every comparison the reviewer asked for, including the settings where simpler baselines are stronger, and states the scope of our claims accordingly. We address each question below.

Q1: Comparison with fixed fusion, proxy-tuning and adaptive fusion on the reasoning suite.

We answer this question in four parts: (1) the requested baselines on the reasoning suite, (2) what these comparisons show about where token-wise routing is needed, (3) the comparison with On Giant's Shoulders, and (4) the alignment trade-offs.

(1) Reasoning-suite baselines. The submission compared fixed fusion, DExperts, Proxy-Tuning and SFT–LoRA on alignment transfer (Figure 4, Table 10). We now add fixed fusion, Proxy-Tuning and a learned constant gate on the reasoning suite (Table R1; nine-task average, same donors, vocabulary alignment and evaluation protocol):

Table R1. Nine-task average on the reasoning suite.

Receiver (donor)ReceiverFixed γ (best of six)†Fixed γ (mean of six)Proxy-Tuning (α = 0.5 / 1.0)‡Learned constant gateMoO
Qwen-Math (AceMath)0.4300.605 (γ = 0.2)0.5810.625 / 0.6020.6080.573
Qwen-Base (AceMath)0.6280.673 (γ = 0.2)0.6470.663 / 0.6520.6490.629
Qwen-Math (two donors)0.4300.601 (γ = 0.3)0.571n/a0.5860.592
Qwen-Base (two donors)0.6280.666 (γ = 0.1)0.636n/a0.6270.649

† Selected among γ ∈ {0.1, 0.2, 0.3, 0.5, 0.7, 1.0} on the test benchmarks, i.e. an oracle choice. ‡ Uses the donor's untuned base model as the proxy base. “Mean of six” averages the six strengths. Fixed-strength and Proxy-Tuning runs were executed on a separate cluster, where receiver scores differ by up to about one point. MoO values use the submitted gate (seed 42); the baselines are single runs. For Qwen-Math + AceMath, the three-seed mean of MoO is 0.537 (Table R5), lower mainly because MBPP varies strongly across seeds; excluding MBPP, the seed mean (0.568) is higher than seed 42 (0.559).

(2) Where token-wise routing is needed. The comparisons separate three settings.

  • Heterogeneous donors. When donor signals pass through a cross-vocabulary mapping, fixed fusion at every strength we tested (γ = 0.3, 0.4, 0.5), DExperts and the learned constant gate collapse, Proxy-Tuning is not applicable, and MoO stays at the receiver's level:

Table R2. Heterogeneous Gemma/Qwen pair (alignment).

Gemma/QwenCapability Avg. ↑ASR ↓
Receiver0.69960.0150
Fixed fusion (γ = 0.3)0.21110.3450
Fixed fusion (γ = 0.4)0.15740.7150
Fixed fusion (γ = 0.5)0.13440.7850
DExperts0.10500.5950
Learned constant gate0.14040.7650
MoO0.71360.0150
  • Multiple donors. The best fixed strength selected on the test benchmarks exceeds MoO by 0.9–1.7 points (Qwen-Math 0.601 vs 0.592; Qwen-Base 0.666 vs 0.649). Without such selection, MoO scores 0.63 and 2.26 points above the learned constant gate and 1.3–2.1 points above the average fixed strength.
  • A single same-vocabulary donor with its base model available. Here simple fixed combinations are strong: Proxy-Tuning (α = 0.5), the best fixed strength and the learned constant gate exceed MoO by 2.0–5.2 points. These baselines rely on information MoO does not use: Proxy-Tuning needs the donor's untuned base model, and the fixed strength and α were chosen on the test benchmarks. MoO reaches its scores without either.

MoO therefore provides one fusion rule that needs neither the donor's base model nor tuning on evaluation data, stays safe when donor signals are unreliable, and composes multiple donors. When the strength can be selected on the evaluation benchmarks, tuned fixed combinations are competitive or stronger with same-vocabulary donors, including two donors, and we report this. The best strength also varies across settings (0.1–0.3 for reasoning, 0.7–1.0 for Qwen alignment) and collapses on the heterogeneous pair, whereas MoO needs no such selection.

AuthorsOpenReview ↗

Response to Reviewer Q9v7 (2/4)

Q1 (continued).

(3) Comparison with On Giant's Shoulders (Fan et al., 2024). The authors have not released implementation code or implementation details, so we clarify the methodological differences. That method sets per-step fusion weights by solving a KL-constrained optimization at inference time, around tuned/untuned small-model pairs whose weights compete. MoO (i) predicts the strength from the receiver's hidden state with a learned gate, with no per-step optimization; (ii) uses independent, bounded donor strengths, so several donors can be active at the same step; and (iii) needs no untuned counterpart and supports heterogeneous tokenizers.

(4) Alignment trade-offs. We agree the alignment results show trade-offs rather than uniform dominance. On Llama, Proxy-Tuning (α = 1) is stronger in capability and IFEval-I at the cost of higher over-refusal. A finer fixed-strength sweep on the Qwen pair traces a similar safety–refusal curve: γ = 0.7 reaches ASR 0.030 with over-refusal 0.060, and γ = 1.0 matches MoO (0.020, 0.080) with similar capability and IFEval-I. MoO's distinct advantage is the heterogeneous pair above, where Proxy-Tuning is not applicable and fixed strengths collapse. The finer sweep for the Llama and Gemma pairs is in progress.

Revisions: Table R1 is added as Table 3 with its discussion (Section 4, L399–414). The constant-gate results on the Gemma/Qwen and Llama pairs are added to Section 4 (L462–470) and Appendix C (Table 14). Section 2.2 is expanded with the comparison to On Giant's Shoulders (L149–151). The fixed-versus-dynamic discussion is revised to describe trade-offs, including the finer fixed-strength sweep (L499–512).

AuthorsOpenReview ↗

Response to Reviewer Q9v7 (3/4)

Q2: Training data, vocabulary projection, budget regularizer and SFT–LoRA configuration.

Thank you for pointing this out; these details were missing. We answer in five parts: (1) gate-training data, (2) train/test separation, (3) the cross-vocabulary mapping, (4) the donor-budget regularizer and (5) the SFT–LoRA baseline.

(1) Gate-training data. All targets are generated by the donors themselves and filtered for correctness. For reasoning, taking the single AceMath donor as an example: 1,736 AceMath-generated examples (1,000 math; 236 code, kept only if they pass the unit tests; 500 general). Two-donor setting: 3,472 examples, i.e. these 1,736 plus 1,736 R1-Distill-Qwen-1.5B responses to GSM8K (987) and MATH (749) training questions, kept only when the final answer is correct. Alignment: 860 examples (220 code, 220 general, 150 AdvBench harmful prompts and 50 XSTest safe prompts taken from item 201 onward, 220 multilingual). Examples longer than 2,048 tokens are dropped, not truncated.

(2) Train/test separation. Reasoning questions come from the GSM8K and MATH training splits; evaluation uses the test sets. AdvBench and XSTest items 1–200 are reserved for evaluation. No validation set is used; we evaluate the checkpoint at the final training step.

(3) Cross-vocabulary mapping. We correct one description in the submission: the Gemma/Qwen map is not learned. It is a deterministic sparse mapping built from the two vocabularies; donor probabilities are summed into the receiver vocabulary through a token index, and their log serves as the donor's logits. It has no parameters, training data or objective.

(4) Donor-budget regularizer. Lbudget=λ(1N∑t∑kγt,k−ρ)2\mathcal{L}_{\text{budget}} = \lambda\big(\frac{1}{N}\sum_t\sum_k \gamma_{t,k} - \rho\big)^2, where N is the number of supervised assistant tokens, ρ=0.3\rho = 0.3 and λ=0.05\lambda = 0.05; total loss LCE+Lbudget\mathcal{L}_{\text{CE}} + \mathcal{L}_{\text{budget}} for reasoning, LCE\mathcal{L}_{\text{CE}} alone for alignment.

(5) SFT–LoRA baseline. LoRA on Qwen2.5-7B-Base (all linear modules, rank 16, α = 32), trained on the same 860 alignment examples with learning rate 1e-4, effective batch 8, 10 epochs (≈1,080 steps, vs 1,200 gate steps), cosine schedule with 5% warmup and cutoff length 2,048.

Revisions: Appendix A.3 (L823–847, Table 9) lists the gate-training data and splits for every configuration; Section 3.2 (L213–215) and Appendix A.4 (L848–869) correct the description of the cross-vocabulary mapping; Section 3.3 (L309–311) gives the donor-budget regularizer, and Appendix A.6 (L893–913) gives its exact form together with the control and baseline settings, including SFT–LoRA. We will release anonymized code for gate training, fusion decoding and evaluation.

AuthorsOpenReview ↗

Response to Reviewer Q9v7 (4/4)

Q3: Knowledge and coding losses, and the 0.5% MBPP baseline.

We answer in three parts: (1) why the receiver scores 0.5% on MBPP, (2) how robust the MBPP gain is, and (3) the knowledge and coding losses and what they imply for the gate.

(1) MBPP baseline. We inspected all 200 receiver outputs of Qwen2.5-Math-7B. In 198 of them no Python function can be extracted (16 reach the 1,024-token limit); one runs but fails the tests, and one is correct. The extraction rule is therefore not the cause; the zero-shot prompt is. With a three-shot prompt, the receiver scores 66.5% and MoO 65.5%, so the zero-shot MBPP gain reflects instruction following transferred from the donor rather than improved coding.

(2) MBPP gain. Re-training the gate with two more seeds shows that MoO's MBPP gain is not stable (0.685 for the reported seed, 0.165 and 0.010 for the others). We therefore no longer rely on MBPP and report the eight-task average without it. Across three seeds, MoO improves Qwen-Math from 0.4836 to 0.5682 ± 0.0090 (+8.46 points), and the gain holds for every seed. It comes from the math benchmarks (GSM8K 0.575 → 0.860 ± 0.053, MATH500 0.530 → 0.710 ± 0.048, GSM-Hard 0.480 → 0.728 ± 0.013).

(3) Knowledge and coding losses. On Qwen-Math these losses are largely seed-specific: for the reported seed, MMLU and ARC-C drop by 8.8 and 13.2 points, while for the two new seeds they drop by at most 1.8 points (MMLU 0.576 / 0.570, ARC-C 0.750 / 0.736, vs. receiver 0.588 / 0.754). Additional seeds for Qwen-Base are still running. With two complementary donors, the Qwen-Base losses are small (MMLU −1.0, ARC-C −2.4, MMLU-Pro −2.8). We agree that a single gate does not always suppress unhelpful donor signals and have softened this claim.

Revisions: The abstract (L24–26) and introduction (L103–104) report the seed-averaged eight-task gain instead of the single-seed 14.27-point figure. Section 4 adds the per-seed results (Table 4, L416–431), with all nine benchmarks per seed in Appendix B.1 (Table 13), and softens the claim that the gate suppresses unhelpful donor signals (L421–422). Appendix A.5 adds the MBPP protocol, the analysis of the receiver's outputs and the three-shot results (L871–891).

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for the constructive feedback and for recognizing the value of composing frozen models through a receiver-conditioned gate with independent donor strengths, as well as the transfer gains across reasoning and alignment. Your questions led to the new baseline comparisons, the learned-constant-gate control, the multi-seed re-training and the reproducibility details reported in our individual responses. The table below summarizes the shared concerns and the corresponding revisions.

ReviewersConcernResponse and manuscript revision
Q9v7, YQeLWhether token-dependent routing helps beyond simpler fusionWe added fusion baselines on the reasoning suite and a learned constant gate trained with identical data and calibration. MoO is one fusion rule trained without evaluation data. With heterogeneous donors, fixed fusion (γ = 0.3–0.5), DExperts and the constant gate collapse while MoO keeps the receiver's level; with two donors, MoO outperforms the learned constant gate (+0.63 / +2.26 points) and the average fixed strength. MoO needs no strength tuning on evaluation data; the full fixed-strength sweep is reported in our response to Reviewer Q9v7 (Q1).
Q9v7, YQeL, BvgYMissing training, mapping and evaluation detailsWe added the full gate-training data, train/evaluation separation, cross-vocabulary mapping and safety-evaluation protocol in Appendix A.3–A.6. We also correct one description: the Gemma/Qwen vocabulary map is a deterministic sparse token mapping, not a learned projection. Anonymized code will be released.
Q9v7, YQeL, BvgYRobustness of the headline gain (MBPP, seeds, statistics)We re-trained the gate with two more seeds and no longer rely on MBPP. Excluding MBPP, MoO improves Qwen-Math from 0.4836 to 0.5682 ± 0.0090 (+8.46 points), with a gain for every seed. The abstract and introduction now report this seed-averaged result; paired bootstrap intervals are being added.
Q9v7, YQeLClaims stronger than the evidence (alignment, routing diagnostics)We revised the claims to match the evidence. Alignment results are presented as trade-offs, and Table 3 and Figure 5 are presented as descriptive diagnostics, with the constant-gate control as a controlled ablation.

We hope these responses address your concerns, and we are happy to clarify further during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.