- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PairQ, a method for adapting frozen mixture-of-experts language models by learning token-conditioned pairwise costs over expert-selection variables. These costs supplement the frozen router’s unary scores, producing a quadratic objective over subsets containing exactly K experts. Selection uses simulated annealing, while an energy-weighted candidate surrogate supplies gradients through hard routing. Only the added routing parameters at the final MoE layer are trained, while the pretrained model parameters and the expert-mixing rule remain fixed. Experiments evaluate two backbones on five mathematical reasoning benchmarks, with additional unary-control, surrogate, and solver analyses. A physical coherent Ising machine also supplies routing decisions throughout one autoregressive generation.
Critical strengths
- A coherent formulation that isolates routing adaptation. PairQ directly scores expert subsets while preserving the pretrained experts, activation cardinality, and native mixing rule. This is a useful experimental setting for studying whether richer selection objectives improve frozen MoEs. The paper also includes evaluations on both 32-expert and 128-expert backbones.
- Useful controlled evidence for the training surrogate. The aligned-versus-cyclic experiment matches initialization, data order, and training settings across three paired seeds. Correctly matching candidate energies with their outputs consistently improves held-out hard-routing NLL over the cyclic control. This supports the presence of meaningful training signal in the surrogate. The paper also clearly acknowledges estimator bias, limited candidate coverage, and the distinction between gradient agreement and successful discrete optimization.
Critical weaknesses
- The default decoder uses 64 annealing reads and 1000 sweeps per routing decision, yet no end-to-end latency, throughput, or search-budget comparison is reported. With relatively small accuracy gains, these measurements will affect the method’s practical significance.
- The training forward pass uses SA routes, so the learned parameters are fitted to SA’s behavior rather than to the energy’s minimizer. Without downstream results under exact decoding (feasible for GPT-OSS), it is hard to tell whether any benefit comes from the energy or from SA’s particular approximation. Fidelity on Qwen3 is not measured.
- The frozen-router AIME scores are below the developers’ figures even at their lowest reasoning settings. The gpt-oss model card cited in the paper lists gpt-oss-20b at 42.1 / 37.1 (AIME24 / AIME25, no tools) at low effort and 80.0 / 72.1 at medium, versus 36.67 / 36.67 here. Qwen3-30B-A3B’s AIME25 is reported at about 71 with thinking and about 22 without, versus 16.70 here. In that regime, small accuracy differences can come from changes in output length or termination rather than reasoning quality.
- Native-router CPT falls below the frozen router on three of five benchmarks for each backbone. This suggests that MathPile CLM training is itself poorly matched to chat-format evaluation. Beating CPT is therefore weak evidence, and an anchored, bounded router may mostly be doing less harm.
Questions for authors
- What fraction of generations hit the 4096-token cap for each method and benchmark, and how do mean output lengths compare between methods? Do the Table 1 conclusions hold under the developers’ recommended generation budgets?
- On GPT-OSS, how do accuracy and NLL change when the trained router is decoded by exact enumeration instead of SA? If accuracy drops, that would suggest the gains come from SA’s behavior rather than from minimizing the learned energy. GPT-OSS has only C(32,4) = 35960 feasible subsets. Yet SA spends 64 reads × 1000 sweeps × 32 single-spin proposals ≈ 2.05M proposals per token, about 57 times the number of feasible subsets. Exhaustive enumeration would be exact and plausibly cheaper.
- How often does PairQ select exactly the native route? What are the distributions of per-token overlap with it, and of the native-logit ranks of swapped-in experts? Mixing still uses the restricted native softmax, so the lower entropy and higher top-1 weight in Table 2 could simply reflect low-logit experts entering the set with negligible weight. What fraction of selected experts receive less than 1% of the mixing weight?
- What is the frozen-native NLL on the GPT-OSS held-out set of Table 3?
- What are inference latency and throughput relative to native Top-K, for SA and for the CIM (including communication)?
- For the 214 CIM decisions, how often does the CIM route match the exact minimizer and the SA route, and does the frozen router also answer that question correctly?
- Is there any plan to include results for an intermediate layer, or for several layers, which would show whether the effect is specific to the final MoE layer?
o1ZB-W1: Search cost
At matched output lengths, simulated annealing (SA) took 1.2419–1.3011 times native generation time on GPT and 1.685–1.720 times on Qwen. Separately, increasing SA reads from 64 to 512 at a fixed Qwen checkpoint lowered NLL from 1.606364 to 1.602179. Each read is an annealing run; all use 1,000 sweeps in this comparison. See XUmm-6 for timing and XUmm-3 for search budgets.
o1ZB-W2: Decoder dependence
Exact GPT decoding also improves over frozen native on AIME24 and held-out NLL. Qwen benefits from more SA reads at a fixed checkpoint (XUmm-3).
o1ZB-W3: Reasoning settings
Methods use matched prompts, greedy decoding and output budgets. Qwen uses
enable_thinking=False, closing an empty thinking block before answer generation; the GPT length control changes only the cap, retaining its prompt and decoding configuration (XUmm-2, XUmm-6).o1ZB-A: Native-router training
Frozen native is the pretrained baseline. Native-gate continued pretraining (CPT) is a separate adaptation control with frozen backbone and experts. PairQ's gains over frozen native are reported independently of CPT (XUmm-B, XUmm-3).
o1ZB-1: Output budget
Changing only the GPT AIME24 cap from 4,096 to 8,192 raises SA correct answers from 11/30 to 18/30 and reduces cap hits from 16/30 to 8/30. At 8,192, SA/exact mean outputs are 4,620.43/5,003.10 new tokens (XUmm-6).
o1ZB-2: Exact decoding
With matched Harmony prompts, greedy decoding and an 8,192-token cap, frozen native scores 16/30 on AIME24; the same causal-language-modeling (CLM) GPT router scores 18/30 under SA and 17/30 under exact decoding. Their respective NLLs on 747,305 held-out targets are 1.874181/1.864217/1.861681. Exact gives lower NLL, while SA answers one more question correctly (XUmm-3). GPT selects four of 32 experts: exact checks all 35,960 legal subsets, while SA uses 64 reads and 1,000 sweeps.
o1ZB-3: Swapped-in experts
Three Qwen checkpoints (seeds 123, 456, 789) each use 256 documents: 65,280 token positions and 522,240 selected-expert instances (eight per position). PairQ matches the native set at 74.380%–76.898% of positions, with mean overlap 6.9113–7.0062 experts. Swapped-in experts (outside native Top-8) have native-rank medians 25–29 and 90th percentiles 62–77. Their bfloat16 (BF16) mixing weights are below 1% in 4.592%–6.937% of swapped-in instances; the fraction among all selected instances is 0.571%–0.901%. With the same logits and BF16 conversion, only 1 out of 522,240 native-reference weights per checkpoint falls below 1%.
o1ZB-4: Frozen-native validation loss
The frozen-native NLL on the original Table 3 evaluation set is 1.613915. This result was computed using the unmodified pretrained base on the same cached text: the first 256 tokens of each of 128 documents, giving 32,640 next-token targets. The corresponding NLLs for PairQ-SA and PairQ-exact are 1.621868 and 1.615968, respectively.
These results belong to the original Table 3 checkpoint and text cache and must be kept separate from the subsequent CLM experiment. The latter uses 128 complete documents and 747,305 targets, with native/SA/exact NLLs of 1.874181/1.864217/1.861681.
o1ZB-5: Throughput
Original/reversed method order, in tokens/s:
Each model uses eight fixed prompts, batch size 1, greedy decoding and 128 new tokens; methods share a GPU within each comparison. After warm-up, each prompt is timed twice and method order is reversed. Timing includes prompt processing, routing, experts and decoding, excluding model loading and tokenization (XUmm-6).
We do not yet have a communication-inclusive CIM latency measurement.
o1ZB-6: Physical-solver comparison
The reported GPT-OSS-20B GSM8K CIM run completes 214 feasible four-expert selections and answers “3 bolts” correctly (Vcjk-5). Matched CIM/SA/exact agreement and the frozen-native answer remain unverified. That comparison requires each submitted QUBO with its returned mask, paired SA/exact solves, and native generation under the same prompt and decoding settings.
o1ZB-7: Adaptation depth
Controlled runs adapt GPT layer 23 or Qwen layer 47 (zero-based), freezing all pretrained parameters. This permits direct output-replacement loss measurement without later attention layers. The construction applies to earlier or multiple MoE layers; current controlled evidence concerns the final layer (XUmm-5).