acceptodds
Under review as a conference paper at ICLR 2027

CH-MoE: Decoupling Learning Credit from Hard Execution Budgets

16 views

Abstract

Sparse mixture-of-experts (MoE) routing often struggles to balance dynamic expert selection for model quality with strict hardware execution budgets for system efficiency. We introduce CH-MoE, a unified routing framework that decouples hard execution budgets from gradient-based learning credit. At the Llama 260M/5B scale, under a strict budget of 80B expert calls, token-local CH-Global improves validation perplexity and macro accuracy over ReMoE and Default MoE while driving substantial load variance reductions (dropping Load CV by over 85%). Rigorous ablation controls demonstrate a structural proof-of-concept: our mask-closed architecture maintains robust language generalization without relying on conventional confidence penalties or proxy regularizations. At the 4.38B scale, our full-sequence allocator (CH-Primary) translates algorithmic frugality into measured hardware speedups, increasing single-A100 throughput by up to 26.03%. Crucially, CH-Primary outperforms a structurally matched same-work scalar baseline by 11.22%, indicating that its distinct allocation distribution provides improved execution efficiency even under identical load-balancing constraints. Furthermore, physical two-node (8A100) execution delivers up to 49.95% higher throughput, driven by a 63.41%–68.32% collapse in receive-route variance. Together, these results establish CH-MoE as a rigorously controlled framework for explicitly uncoupling execution limits from sparse learning signals.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Mixture-of-Experts, Sparse Routing, Budgeted Execution, Load Balancing, Hardware Efficiency

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_D9HfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a CH-MoE routing framework to separate hard execution budgets from the gradient-based learning credit. It has two key components:

  1. CH-Global executes the four highest-ranked experts for each token. The fifth-ranked expert’s routing score participates in a backward-only normalization attachment, but that expert is not executed. The attachment uses an analytic cardinality-event probability to supply an additional training signal.They report CH-Global improves perplexity, downstream accuracy, and load balance compared to that of ReMoE and Default MoE. They then use a four-arm ablation to separate the attachment and the penalty gradient.
  2. CH-Primary allocates three/four experts across a batch to enforce ~ 90% of the top-4's expert call budget. CH-Primary reports throughput improvements over Top-4 on both a single A100 and a physical two-node setup. On a single A100, it also outperforms Scalar-90, a structurally matched baseline with the same expert-call budget.
Critical strengths
  1. The paper provides clear execution semantics for experts along with informative components controls. The formulation needs some presentation work, as it requires tracking multiple policies and models across the methods section, but overall it clearly distinguishes executed work from how backward credit is applied. And the paper is very clear how this work fits as a contribution through the combination of established parts, which is still very useful.
  2. The paper has strong results throughout its well-designed experiments. Controlling the structure of the CH-Primary vs Scalar 90 provides a fair and strong baseline for evaluating the two. The results themselves are good. The physical node measurements are also very convincing for just the CH-Primary.
Critical weaknesses
  1. The learning mechanism provides a very small gain over the two other methods and it is quite unclear where the larger quality gains come from. PPL improvements are modest in table 2 over the same router w/o their intervention. Table S5 shows that the gains are similar when both of the gradients are disabled. Section 5.2 makes larger claims therefore than what their results show, specifically around the hard execution budget which Default MoE already has. The claims need to either be weakened or new results must be produced.

  2. There are no quality results to accompany the speed results. Without this, one cannot assess the practical importance of CH-Primary. A comparison of the proposed method and Scalar-90/Top-4, measuring both quality and speed under the same conditions, would materially strengthen the paper.

Questions for authors
  1. What is the leading explanation for the quality improvements? What other differences separate All-Q-Off from Default MoE?
  2. Is there a quality degradation with the speed improvements?
  3. How much do the quality results change with changes in batching/batch ordering?
7 comments
AuthorsOpenReview ↗

Response to Reviewer D9Hf (Part 1/3: Weakness 1 & Question 1 — Primary Drivers of Quality Gains, Table S5, & Default MoE Differences)

We sincerely thank Reviewer D9Hf for the positive assessment ("Weak Accept") and for recognizing our execution semantics, the Scalar-90 control, and our physical multi-node measurements. We organize our response into three parts matching your Weaknesses and Questions.


Weakness 1 & Question 1: Primary Drivers of Quality Gains, Claim Moderation, & Differences Between All-Q-Off and Default MoE

Weakness 1 & Question 1 Summary

The incremental gain of QQ over the same router without QQ is small, making the main source of quality gains unclear. Claims in Section 5.2 overreach—especially regarding hard execution budgets, since Default MoE already enforces a hard budget. What is the most likely explanation for the quality improvements, and what other differences exist between All-Q-Off and Default MoE?

We appreciate your careful reading and fully agree: because Default MoE already enforces a hard expert-call budget, "enforcing a hard budget" alone is not a unique source of quality improvement. We will moderate our claims in Section 5.2 accordingly.

1. Primary Source of Quality Gains vs. Incremental Role of the Analytic Attachment: We agree that CH-MoE's primary quality advantage over Default MoE stems from the holistic architectural co-design of our shared routing infrastructure, while the analytic attachment provides a modest, penalty-free additional learning gain on top of that foundation:

  • Table S5 (Single-Seed 19701 External Comparison, 260M/5B): Under identical frozen scorers (L=512,B=16L=512, B=16 for PPL; L=256,B=8L=256, B=8 for Macro Acc), moving from Default MoE (27.453 Test PPL, 45.258% Macro Acc) to All-Q-Off (00) (26.894 PPL, 45.793% Acc) already captures the majority of the gain (-0.559 PPL, +0.535 pp Acc), outperforming both Default MoE and ReMoE (27.084 PPL, 45.320% Acc) without QQ enabled. Enabling the analytic attachment (AttachOnly (10)) adds a clean incremental refinement (26.885 PPL, 45.937% Acc).
  • 3-Seed Ablations (Seeds 19701, 28463, 39173, 260M/5B): Relative to the shared All-Q-Off baseline (26.933 PPL, 45.257% Acc, 0.0352 Load CV), enabling the analytic attachment in main-paper Table 2 (Attach Only: 26.905 PPL, 45.317% Acc, 0.0326 Load CV) and in our controlled surrogate ablation (Q-Only: 26.922 PPL, 45.350% Acc, 0.0332 Load CV) yields small, consistent gains across all three metrics without auxiliary penalties—whereas replacing it with the tested comparable-strength Simple Surrogate (sample, averaging the 12 unexecuted experts' routing scores) drops Macro Acc to 44.497% (26.931 PPL, 0.0336 Load CV). The two runs were executed on different A100 server node groups and yielded slightly different results (0.017 PPL and 0.033 pp Macro Accuracy differences); we report both separately for transparency.
  • Scale-Up Verification (4.38B / 120M Tokens): In a matched CH-Primary ablation, disabling the analytic attachment (OFF) degrades validation PPL from 132.54 ± 1.68 to 138.69 ± 1.97 and lowers 7-task macro accuracy from 29.28% to 29.12%.
  • Revised Scope: Rather than attributing overall quality gains to QQ or hard budgets alone, we will frame our shared routing architecture as the primary driver over Default MoE, and the analytic attachment as a safe, penalty-free learning refinement.

2. Key Architectural Differences Between All-Q-Off and Default MoE: As reflected in Table S5 (All-Q-Off (00) vs. Default MoE), although both execute 4 experts per token, they differ fundamentally in three components (not yet isolated one-by-one, and explicitly scoped as a combined architectural difference in revised Section 5.2 and Table S5):

  1. Output Merge Semantics (Renormalization vs. EMA Default Injection): All-Q-Off renormalizes merge weights strictly over actually executed experts, whereas Default MoE applies global softmax weights and injects EMA default output estimates of unexecuted experts into the forward pass.
  2. Learned Token-Level Gain: All-Q-Off applies a learned, token-adaptive output gain gt=2σ(wg⊤ht+bg)g_t = 2\sigma(w_g^\top h_t + b_g), which Default MoE lacks.
  3. Realized-Load Feedback (Selection Bias vs. Feature Estimation): Default MoE uses EMA to estimate unexecuted expert outputs, whereas All-Q-Off uses dual-timescale (fast/slow) EMA to track realized loads and update proportional selection biases bjb_j, shifting routing selection probabilities rather than perturbing forward features.

Thank you again for encouraging a clearer, well-scoped attribution of our architectural gains.

AuthorsOpenReview ↗

Response to Reviewer D9Hf (Part 2/3: Weakness 2 & Question 2 — Joint Quality-Speed Evaluation Under Identical Conditions)

Thank you again, Reviewer D9Hf, for highlighting the need to pair our latency measurements with quality metrics under identical deployment conditions.


Weakness 2 & Question 2: Joint Quality–Speed Evaluation Under Identical Deployment Conditions

Weakness 2 & Question 2 Summary

Speed results lacked paired quality evaluation under identical conditions. Does the speedup come with any quality degradation?

To provide a direct, apples-to-apples answer, we evaluated CH-Primary and Top-4 side-by-side within a unified evaluation suite where all evaluations share the exact same causal attention, expert execution backend, and precision stack:

  • Matched Models & Training: Both share the identical 36-layer architecture (hidden = 3072, intermediate = 12288, 32 heads, 16 experts at MoE layers {8, 17, 26, 35}, n_required = 4) trained in paired runs across 3 seeds (42, 123, 777) on the same 240M-token corpus (14,649 steps, lr = 1.5e-4, weight_decay = 0.01, bf16).
  • Shared Evaluation Stack: Both arms run at batch_size = 8 with left-padding (1 for valid tokens, 0 for padding) and intra-sequence causal masking, sharing the exact same precision and backend (BF16 dense, FP32 MoE, TF32 off, float32_precision = highest, grouped_bmm, no CUDA Graph), identical benchmark_sha256 data across all 7 tasks (7/7), and identical continuation log-likelihood scoring (accuracy_normalized headline):

Table R1: Unified Deployment Evaluation (36-Layer Paired Models, 3 Seeds, batch_size = 8)

CategoryMetricTop-4CH-PrimaryRelative Change
QualityCorpus PPL ↓60.4959.31-1.94% (3-seed CIs < 0)
7-Task NLL ↓BaselineLower on 6/7 tasks-0.5% to -1.6%
7-Task Macro Acc (%) ↑36.13%36.01%-0.12 pp
InferenceTTFT (Prefill) ↓356 ms224 ms-37%
TBT (Decode) ↓29.4 ms27.1 ms-8%
End-to-End Generation Latency ↓4116 ms3697 ms-10%
Generation Throughput ↑502 tok/s555 tok/s+11%
PPL Scoring Throughput ↑5,743 tok/s8,797 tok/s+53%
Peak Memory ↓BaselineReduced-5%
Routing LoadActive Slot Rate1.0000.900-10% active slots
Load CV ↓1.130.55-52%
Effective Experts ↑8.7 / 1613.8 / 16+59%
Dead Expert Fraction ↓2–5%0%Eliminated

Within this unified causal-attention and execution stack (batch_size = 8, grouped_bmm), executing 10% fewer active expert slots (active_slot_rate 0.900 vs. 1.000) delivers -37% TTFT, +11% generation throughput, -52% Load CV, and -1.94% Corpus PPL (60.49 → 59.31, 3-seed CIs < 0), alongside a minor -0.12 pp change in discrete 7-task Macro Accuracy (36.13% → 36.01%).

In addition, under our aligned 4.38B/120M full-sequence protocol comparing Top-4, Scalar-90, and CH-Primary on the identical Triton backend (B=8, L=512), CH-Primary similarly achieves the highest throughput (15,627 ± 256 tok/s vs. 14,051 ± 215 for Scalar-90 and 13,339 ± 269 for Top-4), the lowest Validation PPL (132.54 vs. 134.55 and 137.00), and matched 7-task accuracy (29.28%).

We appreciate this suggestion, as this unified evaluation makes the practical deployment trade-off of CH-Primary clear.

AuthorsOpenReview ↗

Response to Reviewer D9Hf (Part 3/3: Question 3 — Sensitivity to Batching Method and Batch Order)

Thank you again, Reviewer D9Hf, for raising the important practical question of how batch composition and ordering affect CH-Primary.


Question 3: Sensitivity of Quality Results to Batching Method and Batch Order

Question 3 Summary

How much do the quality results change when the batching method or the order of batches is altered?

Because CH-Primary enforces micro-batch quotas, altering batch composition changes token-to-expert assignments. To quantify this sensitivity, we evaluated CH-Primary across 3 seeds (42, 123, 777) under 8 different batching conditions on our 7-task reasoning suite, holding model weights, prompts, scorers, and backend strictly constant:

Table R2: Impact of Batching Perturbations on 7-Task Macro Accuracy (CH-Primary, 3-Seed Mean)

Batching ConditionMacro-7 Acc (%)Change vs. Original B16
Original B16 (Baseline)29.1183%—
Reverse Batch Execution Order29.1183%0.0000 pp
Reverse Row Order within Batches29.2300%+0.1117 pp
Re-batch, Fixed Permutation 10129.1344%+0.0162 pp
Re-batch, Fixed Permutation 20229.1657%+0.0474 pp
Re-batch, Fixed Permutation 30329.1838%+0.0656 pp
Change to B829.1428%+0.0245 pp
Change to B3229.1262%+0.0079 pp

Key Findings:

  1. Batch Execution Order Invariance: Reversing the execution order of identical batches produces exact determinism (0.0000 pp change across all 9 checkpoints when evaluation feedback buffers are frozen).
  2. Bounded Mean Shifts: Across all 8 perturbations (reversing intra-batch rows, reshuffling batches, or changing batch sizes across 8, 16, and 32), the 3-seed mean Macro-7 accuracy stayed tightly bounded between 29.1183% and 29.2300% (maximum shift +0.1117 pp).
  3. Bounded Single-Seed Shifts: Individual seed variations exhibited minor bidirectional shifts strictly within -0.1322 pp to +0.1680 pp (all single-seed absolute changes < 0.17 pp).

These results indicate small changes under the tested conditions rather than universal invariance across arbitrary deployment distributions. We will include Table R2 in the revised Appendix. We sincerely thank you for your thoughtful review, helpful feedback, and constructive assessment of our work.

Reviewer_a1EDOpenReview ↗
Rating
2
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CH-MoE, a routing framework that separates hard expert-execution budgets from differentiable learning credit. A shared binary mask determines expert execution, merge support, and work accounting, while an auxiliary cardinality-event probability provides additional router gradients through stop-gradient attachments. The paper evaluates two policies: CH-Global, which executes four experts per token with a backward-only normalization attachment, and CH-Primary, which allocates three or four experts across a microbatch under explicit branch quotas.

Validation includes a 260M-parameter, 5B-token quality study against ReMoE and Default MoE, component ablations, and execution measurements on a larger GPT-NeoX-based model. The paper reports modest language-modeling improvements, substantial load-CV reductions, and throughput gains on single-A100 and two-node configurations. Additional autoregressive profiling shows smaller generation-throughput gains, although generation quality is not evaluated under that protocol.

Critical strengths
  1. Explicit separation of execution constraints and learning signals. The mask-closed formulation provides a clear contract connecting executed expert calls, merge support, and load accounting. The backward-only attachment offers a concrete way to modify router learning without evaluating inactive experts. This is a useful experimental framework for studying sparse routing under explicit budgets.

  2. Measured system benefits with a meaningful same-work control. The paper reports hardware measurements rather than relying solely on expert-call reductions. Scalar-90 retains the same budget and balanced-weakest allocation structure, making it a useful control: CH-Primary’s improvement over this baseline supports the observation that identical logical call counts can still produce different execution costs. Physical two-node measurements further strengthen the systems evidence within the tested configurations.

Critical weaknesses
  1. The practical relevance of the targeted imbalance is insufficiently established. The paper demonstrates substantial imbalance under its selected baselines, but does not establish whether comparable imbalance persists with the load-balancing mechanisms used in relevant modern MoE deployments, such as those in the Qwen and DeepSeek families. The authors need not demonstrate that every existing strategy fails. However, they should justify why ReMoE, Default MoE, and the evaluated Top-4 baseline adequately represent the problem motivating the work. Without this evidence, it remains unclear whether the substantial improvements address a limitation of current deployments or primarily a weakness of the selected baselines. Scalar-90 is informative within the proposed allocation framework, but does not resolve this broader baseline concern.

  2. The evidence does not sufficiently establish the incremental value of the analytic credit mechanism or a deployment-level quality–efficiency advantage. In Table 2, All-Q-Off already performs close to Attach Only: Test PPL changes from 26.933 to 26.905, and macro accuracy from 45.257% to 45.317%. This suggests that much of the observed benefit may come from the shared routing structure and realized-load feedback. The larger-model comparison against Scalar-90 evaluates the complete learned routing recipe and does not isolate the analytic event’s contribution. Moreover, CH-Primary’s quality measurements use batch-coupled full-sequence scoring, while its autoregressive deployment measurements report speed without corresponding generation-quality evaluation. Consequently, the paper establishes useful results in separate settings, but does not yet demonstrate a convincing quality–latency advantage under one representative deployment protocol.

Questions for authors
  1. Baseline relevance: Which practically relevant inference workloads remain substantially imbalanced after applying modern MoE load-balancing strategies? Can the authors provide a representative comparison, or existing diagnostics, showing that the imbalance targeted here persists under such a baseline? This would materially affect my assessment of the work’s significance.

  2. Incremental contribution: Given the small gap between All-Q-Off and Attach Only, what evidence establishes that the analytic cardinality event is necessary or preferable to simpler alternatives? In particular, are matched CH-Primary results available with its analytic attachment disabled while retaining the budget, allocation structure, and feedback?

  3. Deployment trade-off: Is there a configuration in which CH-Primary’s quality and speed are evaluated under the same causal routing protocol, batch conditions, and execution backend? The reported autoregressive speedups are encouraging, but corresponding quality evidence would be needed to assess whether they represent a favorable deployment trade-off.

8 comments
AuthorsOpenReview ↗

Response to Reviewer a1ED (Part 1/3: Weakness 1 & Question 1 — Baseline Relevance vs. Qwen2.5 & DeepSeek-V3)

We sincerely thank Reviewer a1ED for the constructive and detailed review, and for highlighting the clarity of our mask-closed formulation and the importance of our same-work (Scalar-90) and physical two-node hardware evaluations. We organize our response into three parts matching your Weaknesses and Questions.


Weakness 1 & Question 1: Practical Relevance Against Modern MoE Load-Balancing Strategies (Qwen2.5-Style & DeepSeek-V3-Style)

Weakness 1 & Question 1 Summary

Whether the targeted load imbalance persists under modern MoE load-balancing mechanisms (such as Qwen and DeepSeek families), and which practically relevant inference workloads remain substantially imbalanced after applying such strategies.

To directly address your concern, we evaluated CH-MoE against Qwen2.5-style (standard auxiliary load-balancing loss) and DeepSeek-V3-style (auxiliary-loss-free dynamic bias updates) load-balancing mechanisms under an identical unified training budget and identical inference-style validation micro-batches.

Evaluation Protocol & Measurement Conditions: All three models were trained under identical data, step budgets (grad_accum = 16, seq_len = 2048), and hyperparameters, and evaluated on the exact same fixed held-out C4-en validation windows (40,960 tokens evaluated on Rank 0 across 10 deterministic micro-batches of 2 sequences × 2048 tokens, seed = 0) using next-token prediction (PPL=exp⁡(NLL/Ntoken)\text{PPL} = \exp(\text{NLL}/N_{\text{token}})). Crucially, Load CV is measured directly during validation across these identical inference-style micro-batches (2 × 2048 = 4,096 tokens per micro-batch):

Table R1: Controlled Comparison Against Modern MoE Load-Balancing Mechanisms (C4-en Validation)

Routing / Balancing MechanismFinal Training Loss ↓Val Micro-Batch Load CV ↓Validation PPL ↓
Qwen2.5-Style (Aux Load-Balancing Loss)3.51250.197632.1134
DeepSeek-V3-Style (Aux-Loss-Free Dynamic Bias)3.47860.149331.4228
CH-MoE (Ours)3.33720.011232.1670

(Note: Both Validation PPL and Val Micro-Batch Load CV are computed over the exact same fixed C4-en validation windows totaling 40,960 tokens on Rank 0—10 micro-batches of 2 sequences × 2048 tokens—across all three runs.)

  1. Persistent Micro-Batch Imbalance in Modern Inference Workloads: Although the DeepSeek-V3-style dynamic bias update improves upon the Qwen2.5-style auxiliary loss (0.1976 → 0.1493), under our evaluated validation micro-batch configuration, DeepSeek-V3-style bias balancing retains a Load CV of 0.1493, indicating that substantial token-local imbalance persists despite the use of a modern load-balancing mechanism. Dynamic bias updates regulate batch-level expert loads during training but do not explicitly enforce balanced execution within each inference micro-batch (B = 2, L = 2048). This aligns with explicit observations in both the DeepSeek-V3 Technical Report (Section 4.5.3)—which identifies "load imbalance within certain sequences or small batches" and "domain-shift-induced load imbalance during inference" as persistent efficiency challenges requiring redundant expert deployment (Section 3.4)—and the Qwen load-balancing study (Demons in the Detail, Section 5 & Limitations), which notes that relaxing micro-batch constraints toward global-batch balance induces "local expert load imbalance under the dropless strategy" and extra latency. By enforcing hard quotas directly inside each micro-batch, CH-MoE reduces validation micro-batch Load CV to 0.0112 (a 92.5% reduction vs. the DeepSeek-V3-style baseline).
  2. Hardware Packing Cost vs. Unconstrained PPL: CH-MoE achieves the lowest final training loss (3.3372) and matches the Qwen2.5-style baseline's validation PPL (32.1670 vs. 32.1134). The DeepSeek-V3-style baseline's slightly lower unclipped validation PPL (31.4228) comes alongside a 13× higher validation micro-batch Load CV (0.1493 vs. 0.0112). Such imbalance can increase tile-padding overhead in grouped GEMM kernels and straggler exposure in multi-node All-to-All communication, providing a practical motivation for CH-MoE's strict micro-batch execution balancing.

We will incorporate this comparison and the exact C4-en validation micro-batch protocol into the revised manuscript. Thank you again for raising this important question on modern baseline relevance.

AuthorsOpenReview ↗

Response to Reviewer a1ED (Part 2/3: Weakness 2A & Question 2 — Incremental Contribution of the Analytic Attachment)

Thank you again, Reviewer a1ED, for pushing us to assess the incremental contribution of the proposed analytic attachment across model scales.


Weakness 2 (Part A) & Question 2: Incremental Contribution of the Analytic Attachment (Surrogate Ablation & CH-Primary ON/OFF Comparison)

Weakness 2 (Part A) & Question 2 Summary

Given the small gap between All-Q-Off and Attach Only, what evidence establishes that the analytic cardinality event is necessary or preferable to simpler alternatives, and are matched CH-Primary results available with its analytic attachment disabled while retaining the budget, allocation structure, and feedback?

The small gap between All-Q-Off and Attach Only at the 260M scale demonstrates that our mask-closed execution contract and realized-load feedback already form a sound routing foundation without heavy auxiliary penalties. To assess the incremental contribution of the proposed analytic attachment (which jointly pairs the cardinality event QQ with the 5th-candidate normalization structure), we provide both a surrogate ablation and the exact matched CH-Primary ON/OFF comparison requested:

1. Why the Analytic Attachment Outperforms Simpler Surrogates (260M/5B, 3-Seed Mean): A simple backward surrogate—taking the average routing score of the 12 unexecuted experts directly on the MoE output while removing both QQ and the 5th-candidate normalization attachment (computing no probabilities and giving no special weight to the 5th candidate)—treats all unexecuted experts uniformly, agnostic to the discrete capacity threshold (N=4N=4). In contrast, pairing the 5th-candidate normalization attachment with Q=Pr⁡(∑j=116Bj≥4)Q = \Pr(\sum_{j=1}^{16} B_j \ge 4) weights gradients via cardinality-specific pivotal probabilities ρj=(1−pj)Pr⁡(K−j=N−1)\rho_j = (1 - p_j)\Pr(K_{-j} = N - 1) (Eq. 4), focusing backward credit on threshold-pivotal experts. Holding forward calls (4), budget (80B), and feedback rules and hyperparameters identical across 3 seeds (19701, 28463, 39173):

Table R2: Ablation of Credit Assignment Mechanisms (260M/5B, 3-Seed Mean)

Credit MechanismTest PPL ↓6-Task Macro Acc (%) ↑Test Load CV ↓
All-Q-Off (Disabled Attachment)26.93345.257%0.0352
Simple Surrogate (sample)26.93144.497%0.0336
Q-Only (Full Analytic Attachment)26.92245.350%0.0332

Although the Simple Surrogate slightly lowers Test Load CV (0.0352 → 0.0336), it degrades 6-Task Macro Accuracy by -0.853 pp relative to Q-Only (45.350% → 44.497%), whereas the full analytic attachment (Q-Only) improves all three metrics simultaneously.

2. Matched CH-Primary Results with Analytic Attachment Disabled (4.38B Scale, 120M Tokens): Holding the execution budget (3n+k3n+k calls), balanced-weakest branch allocation structure, and dual-timescale realized-load feedback strictly identical, we compare CH-Primary with the analytic attachment enabled (ON) versus disabled (OFF):

Table R3: CH-Primary Matched ON/OFF Ablation (4.38B Scale, 120M Tokens)

CH-Primary ConfigurationThroughput (tok/s) ↑P50 Latency (ms) ↓Val PPL ↓7-Task Macro Acc (%) ↑
Attachment OFF15,083 ± 234258.99138.69 ± 1.9729.12%
Attachment ON (Paper)15,627 ± 256262.02132.54 ± 1.6829.28%

Disabling the analytic attachment (OFF) substantially degrades language-modeling quality—increasing Validation PPL from 132.54 ± 1.68 to 138.69 ± 1.97 and lowering 7-Task Macro Accuracy from 29.28% to 29.12% (-0.16 pp)—while operating in the same throughput/latency regime (15,627 ± 256 vs. 15,083 ± 234 tok/s; P50 262.02 vs. 258.99 ms).

We appreciate your rigorous feedback, which prompted these controlled ablations to clarify the mechanism's contribution.

AuthorsOpenReview ↗

Response to Reviewer a1ED (Part 3/3: Weakness 2B & Question 3 — Unified Quality-Latency Deployment Trade-off)

Thank you again, Reviewer a1ED, for emphasizing the importance of evaluating quality and execution speed under a single, unified deployment framework.


Weakness 2 (Part B) & Question 3: Deployment Trade-off under a Unified Evaluation Framework

Weakness 2 (Part B) & Question 3 Summary

CH-Primary's quality was previously measured with batch-coupled full-sequence scoring while autoregressive deployment reported only speed. Is there a configuration evaluating CH-Primary's quality and speed under the same causal routing protocol, batch conditions, and execution backend?

To directly answer your question, we evaluated CH-Primary and Top-4 side-by-side within a unified evaluation suite where all evaluations share the exact same causal attention, expert execution backend, and precision stack:

  • Matched Models & Training: Both share the identical 36-layer architecture (hidden = 3072, intermediate = 12288, 32 heads, 16 experts at MoE layers {8, 17, 26, 35}, n_required = 4) trained in paired runs across 3 seeds (42, 123, 777) on the same 240M-token corpus (14,649 steps, lr = 1.5e-4, weight_decay = 0.01, bf16).
  • Shared Evaluation Stack: Both arms run at batch_size = 8 with left-padding (1 for valid tokens, 0 for padding) and intra-sequence causal masking, sharing the exact same precision and backend (BF16 dense, FP32 MoE, TF32 off, float32_precision = highest, grouped_bmm, no CUDA Graph), identical benchmark_sha256 data across all 7 tasks (7/7), and identical continuation log-likelihood scoring (accuracy_normalized headline):

Table R4: Unified Deployment Evaluation (36-Layer Paired Models, 3 Seeds, batch_size = 8)

CategoryMetricTop-4CH-PrimaryRelative Change
QualityCorpus PPL ↓60.4959.31-1.94% (3-seed CIs < 0)
7-Task NLL ↓BaselineLower on 6/7 tasks-0.5% to -1.6%
7-Task Macro Acc (%) ↑36.13%36.01%-0.12 pp
InferenceTTFT (Prefill) ↓356 ms224 ms-37%
TBT (Decode) ↓29.4 ms27.1 ms-8%
End-to-End Generation Latency ↓4116 ms3697 ms-10%
Generation Throughput ↑502 tok/s555 tok/s+11%
PPL Scoring Throughput ↑5,743 tok/s8,797 tok/s+53%
Peak Memory ↓BaselineReduced-5%
Routing LoadActive Slot Rate1.0000.900-10% active slots
Load CV ↓1.130.55-52%
Effective Experts ↑8.7 / 1613.8 / 16+59%
Dead Expert Fraction ↓2–5%0%Eliminated

Key Takeaways:

  1. Autoregressive Acceleration Driven by Load Balance: Under this shared execution backend and precision stack (batch_size = 8, grouped_bmm), reducing active expert slots by 10% (active_slot_rate 0.900 vs. 1.000) cuts load_cv by 52% (1.13 → 0.55) and eliminates dead experts (2–5% → 0%), yielding -37% TTFT, -8% TBT, -10% end-to-end generation latency, +11% generation throughput, +53% PPL scoring throughput, and -5% peak memory.
  2. Quality–Latency Trade-off: Within this unified causal-attention and execution stack, CH-Primary reduces Corpus PPL by -1.94% (60.49 → 59.31, with 3-seed confidence intervals below zero) and lowers NLL on 6/7 tasks, accompanied by a minor -0.12 pp change in discrete 7-task Macro Accuracy (36.13% → 36.01%).

We will replace our earlier cross-setting juxtaposition with Table R4 and this analysis in the revised manuscript. We are grateful for your constructive suggestions throughout the review, which have materially strengthened the empirical rigor of our paper.

Reviewer_X4pLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CH-MoE, a framework that separates hard expert-execution budgets from the gradients used to train routing. CH-Global executes four experts per token and introduces a backward-only normalization attachment based on an analytic cardinality event QQ, without evaluating the fifth candidate expert. CH-Primary selects candidates from four fixed expert groups and allocates three or four expert calls per token under batch-level quotas. Both policies incorporate realized-load feedback.

Critical strengths
  1. Explicit execution accounting. The framework uses the same hard mask to govern expert evaluation, output support, and work accounting. This makes the computational contract clear and allows changes to the backward signal without introducing unaccounted expert evaluations. This is a useful foundation for studying routing under strict budgets.

  2. Measured systems benefits with a meaningful control. The paper reports hardware measurements on both a single A100 and a physical two-node setup. The Scalar-90 comparator retains the same call budget and structural allocation mechanism, making it a useful control beyond simply comparing against a method that performs more expert calls.

Critical weaknesses
  1. The distinct value of the proposed analytic credit signal is not sufficiently established. The backward-only construction is well defined, but the motivation for this particular cardinality event and normalization attachment remains insufficiently connected to task-relevant routing decisions. The absence of an unbiased-gradient guarantee is not itself my objection; a surrogate gradient can be useful. My concern is the evidence supporting this specific choice.

  2. CH-Primary’s fixed grouping introduces a restrictive selection assumption whose benefits appear dependent on initialization. Selecting at most one expert from each group can exclude useful combinations containing multiple experts from the same group. Dense-slice initialization provides a reasonable motivation for this constraint at initialization, but it does not establish that within-group redundancy and cross-group complementarity persist after training.

Questions for authors
  1. Evidence concerning the credit mechanism: What motivates the specific cardinality event QQ and fifth-candidate normalization attachment over a simpler backward-only signal? Beyond the All-Q-Off comparison, is there a controlled comparison that keeps the forward computation, hard budget, and load feedback fixed while replacing the proposed signal with a simpler surrogate of comparable strength? Such evidence would materially affect my assessment of the learning contribution.

  2. Clarification and evidence concerning grouping: Does it compare grouped selection against unrestricted global selection while holding the call budget, deletion policy, and feedback mechanism fixed? In particular, what do the results establish under independent initialization about the cost or benefit of allowing multiple selected experts from the same group?

4 comments
AuthorsOpenReview ↗

Response to Reviewer X4pL (Part 1/2: Weakness 1 & Question 1 — Analytic Event Q vs. Simple Surrogates)

We thank Reviewer X4pL for the insightful comments on our mask-closed execution contract, hardware measurements, and the two core issues regarding credit assignment and grouped candidate selection. We organize our response into two parts matching your Weaknesses and Questions.


Weakness 1 & Question 1: Unique Value and Necessity of the Analytic Cardinality Event (QQ) vs. Simpler Surrogates

Weakness 1 & Question 1 Summary

Insufficient evidence for the unique value of the specific cardinality event QQ and 5th-candidate normalization attachment over a simpler backward-only signal. Is there an experiment fixing forward computation, hard budget, and load feedback while replacing QQ with a simple surrogate of comparable strength?

We greatly appreciate this rigorous question. Testing a simpler backward-only surrogate—one that removes both QQ and the 5th-candidate normalization attachment—directly highlights the mathematical motivation behind our design and allows us to empirically verify its necessity.

1. Mathematical Motivation (Threshold-Pivotal Credit vs. Uniform Unexecuted-Expert Averaging): A simple backward surrogate—such as taking the average routing score of the 12 unexecuted experts as a direct backward-only signal on the MoE output (computing neither cardinality probabilities nor QQ, and giving no special weight to the 5th candidate)—treats all 12 unexecuted experts uniformly, agnostic to how close individual candidates are to the discrete execution threshold (N=4N=4). In contrast, our attachment pairs the 5th-candidate normalization structure with the analytic truncated Poisson-binomial event Q=Pr⁡(∑j=116Bj≥4)Q = \Pr(\sum_{j=1}^{16} B_j \ge 4). Its derivative (Eq. 4) is:

∂Q∂z~j=λsj(ρj−∑i=116siρi)\frac{\partial Q}{\partial \tilde z_j} = \lambda s_j (\rho_j - \sum_{i=1}^{16} s_i \rho_i)

This weights each expert by its cardinality-specific pivotal probability ρj=(1−pj)Pr⁡(K−j=N−1)\rho_j = (1 - p_j)\Pr(K_{-j} = N - 1)—the exact probability that expert jj is decisive in satisfying the active-slot threshold. Rather than applying an undifferentiated average across all 12 unexecuted experts, this conditions backward credit specifically on threshold-pivotal routing decisions.

2. Empirical Evidence (Controlled Surrogate Ablation at 260M/5B): To provide the exact evidence requested, we conducted a controlled ablation at the 260M/5B scale across 3 distinct seeds (19701, 28463, 39173). We held the forward computation (exactly 4 executed calls), hard budget (80B calls), and realized-load feedback strictly identical, while comparing our QQ-based attachment (Q-Only) against a comparable-strength Simple Surrogate (sample, which removes both QQ and the 5th-candidate normalization attachment and instead applies the mean routing score of the 12 unexecuted experts directly as a backward-only credit signal on the MoE output, without computing probabilities or giving special weight to the 5th candidate):

Table R1: Ablation of Credit Assignment Mechanisms (260M/5B, 3-Seed Mean)

Credit MechanismTest PPL ↓6-Task Macro Acc (%) ↑Test Load CV ↓
All-Q-Off (Disabled Attachment)26.93345.257%0.0352
Simple Surrogate (sample)26.93144.497%0.0336
Q-Only (Analytic QQ)26.92245.350%0.0332
  • Key Takeaway: Although averaging the 12 unexecuted experts' routing scores slightly reduces Test Load CV (0.0352 → 0.0336), it severely degrades downstream reasoning, dropping 6-Task Macro Accuracy by -0.85 pp relative to Q-Only (45.350% → 44.497%) and falling well below All-Q-Off (45.257%). Without the 5th-candidate normalization structure and cardinality-pivotal weighting, undifferentiated backward credit across all 12 unexecuted experts perturbs learned task specialization ("destructive balancing"). By contrast, Q-Only simultaneously achieves the best Test PPL (26.922), highest Macro Accuracy (45.350%), and lowest Load CV (0.0332) ("lossless balancing"). We will integrate this surrogate baseline into revised Section 5.2.
AuthorsOpenReview ↗

Response to Reviewer X4pL (Part 2/2: Weakness 2 & Question 2 — Grouped Selection vs. Unrestricted Global Selection)

Thank you again, Reviewer X4pL, for highlighting the structural and initialization aspects of our grouped candidate selection.


Weakness 2 & Question 2: Grouped Selection, Initialization Dependency, and Unrestricted Global Comparison

Weakness 2 & Question 2 Summary

Fixed grouping in CH-Primary restricts candidate selection (at most one expert per group), its gains appear dependent on initialization, and further evidence is needed regarding within-group redundancy vs. cross-group complementarity. Did the paper compare grouped selection with unrestricted global selection under identical budget, deletion policy, and feedback, particularly under independent initialization?

Yes, we conducted this exact controlled comparison. Because Section 5.4 in the main text was condensed, we present the full 2-seed factorial experiment here. We evaluated Grouped Selection versus Unrestricted Global Selection while holding the expert-call budget and deletion policy (Weakest vs. Fixed-tail) fixed, and using the same realized-load feedback rule and hyperparameters (with analytic event credit disabled to isolate structural routing effects):

Table R2: Factorial Comparison of Grouped vs. Global Selection (Relative PPL Reduction %, 2-Seed Diagnostic)

Initialization StrategyDeletion PolicySeed 19931Seed 19932Mean PPL Improvement (Grouped vs. Global)
Independent Init.Weakest+0.08%-0.17%-0.045%
Independent Init.Fixed-tail-0.11%+0.19%+0.040%
Dense-SliceWeakest+2.80%+2.33%+2.565%
Dense-SliceFixed-tail+2.41%+2.31%+2.360%

(Positive values indicate Grouped outperforms Global; negative values indicate Global outperforms Grouped.)

What the Existing Results Show: These results confirm that the benefit of grouped selection is initialization-dependent: its advantage is consistent under Dense-Slice initialization but not under independent initialization. Specifically:

  1. Costs/Benefits Under Independent Initialization (Mixed Directions Near Zero): As shown in the top two rows, when experts are initialized independently at random, the differences between Grouped and Global selection fluctuate closely around zero with mixed directions across seeds (averaging -0.045% under Weakest deletion and +0.040% under Fixed-tail deletion). Thus, under independent initialization, grouped selection offers no consistent quality advantage over unrestricted global selection, though it also does not introduce a severe quality penalty under the tested conditions.
  2. Co-Design Synergy Under Dense-Slice Initialization: When paired with Dense-slice initialization and Weakest deletion, Grouped selection yields a consistent PPL improvement of +2.565% (+2.80% on Seed 19931 and +2.33% on Seed 19932) over Unrestricted Global selection.
  3. Mechanism (Pre-Deletion Slice Coverage): Each of the 4 branches Eb={b,b+4,b+8,b+12}\mathcal{E}_b = \{b, b+4, b+8, b+12\} corresponds to one distinct slice (j mod 4j \bmod 4) of the original dense FFN. Grouped selection guarantees that the four pre-deletion candidates cover all four initialization slice groups at the start of training (before budget pruning drops one candidate for 3-slot tokens). In contrast, unrestricted global selection can nominate multiple redundant copies from the same initialization slice even prior to deletion, creating uneven initial feature coverage. We also acknowledge that directly measuring post-training within-group vs. cross-group functional specialization is a valuable diagnostic direction and will explicitly scope our claims to initialization-coupled structural balance.

We will include Table R2, explicitly note this initialization dependency, and clarify the pre-deletion slice-coverage mechanism in the revised manuscript. We sincerely appreciate your insightful feedback, which has helped us sharpen the scope and empirical grounding of Section 5.4.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer X4pL,

We highly appreciate the constructive comments and insightful suggestions you have offered for our work. As the deadline for our discussion period is approaching, in order for us to have sufficient time to address any additional questions you may have, we kindly encourage you to engage in the ongoing discussion and share any further insights or clarifications you may have.

If you find that our response addresses some of your concerns, would you kindly consider raising your rating score for our paper?

Thank you very much for your time and consideration. We look forward to hearing from you soon.

Best Regards,

Authors of CH-MoE

AuthorsOpenReview ↗

Final Summary & Sincere Thanks to Reviewer X4pL

Dear Reviewer X4pL,

As the discussion period draws to a close, we would like to express our sincere gratitude for your thoughtful, rigorous review and leave a concise summary of how we have addressed each of your weaknesses and questions in our responses and revised manuscript:

  1. Unique Value of the Analytic Attachment vs. Simpler Surrogates (Weakness 1 & Question 1):
    • Mathematical & Empirical Isolation: Unlike a simple backward surrogate that averages unexecuted experts uniformly, our analytic attachment weights gradients by threshold-pivotal probabilities (ρj=(1−pj)Pr⁡(K−j=N−1)\rho_j = (1-p_j)\Pr(K_{-j}=N-1)). In our new 3-seed controlled ablation at 260M/5B holding forward calls (4), budget (80B), and realized-load feedback identical, replacing our full analytic attachment (Q-Only: 45.350% 6-task macro accuracy, 26.922 Test PPL, 0.0332 Load CV) with a comparable-strength Simple Surrogate (sample, applying the mean routing score of the 12 unexecuted experts) degrades 6-task macro accuracy by -0.853 pp (44.497%), confirming that threshold-pivotal credit achieves lossless load balancing.
  2. Grouped vs. Unrestricted Global Selection & Initialization Dependency (Weakness 2 & Question 2):
    • Controlled Factorial Evidence & Scope Refinement: We added the full 2-seed factorial comparison isolating grouped vs. unrestricted global selection under identical budgets, deletion policies, and feedback. We explicitly scoped our claims in revised Section 5.4 to acknowledge this initialization dependency: under independent initialization, differences fluctuate closely around zero (-0.045% under Weakest and +0.040% under Fixed-tail—offering no consistent advantage while avoiding severe penalties), whereas under Dense-Slice initialization, grouped selection yields a consistent +2.565% mean PPL improvement by guaranteeing pre-deletion coverage across all four FFN slices.

Your insightful questions directly helped us tighten both our empirical grounding and the scope of our claims. We will ensure all new results and clarifications are thoroughly incorporated into the revised manuscript. Thank you again for your valuable time and careful evaluation—we earnestly hope these new controlled results resolve your concerns and encourage you to reconsider your rating for our paper.

Best regards,

The authors of CH-MoE

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response: Summary of New Controlled Experiments and Manuscript Revisions

We sincerely thank the Area Chairs and Reviewers X4pL, a1ED, and D9Hf for their thorough, constructive reviews and for recognizing the clarity of our mask-closed execution accounting as well as the value of our physical hardware and same-work (Scalar-90) evaluations.

To directly address every weakness and question raised across the reviews, we have conducted new controlled experiments and refined our claims in the revised manuscript (with detailed point-by-point responses posted under each review):

  1. Practical Relevance Against Modern MoE Load-Balancing Baselines (Reviewer a1ED W1 & Q1): We added a strictly controlled comparison against Qwen2.5-style and DeepSeek-V3-style load-balancing mechanisms evaluated on the exact same fixed held-out C4-en validation windows (40,960 tokens on Rank 0 across 10 micro-batches of 2 × 2048 tokens). While token-local routing retains substantial micro-batch load imbalance (Load CV = 0.1976 and 0.1493, respectively), CH-MoE reduces validation micro-batch Load CV to 0.0112 (a 92.5% reduction vs. DeepSeek-V3-style balancing).
  2. Incremental Contribution of the Analytic Attachment & Scope Moderation (Reviewers X4pL W1 & Q1, a1ED W2 & Q2, D9Hf W1 & Q1):
    • At the 260M/5B scale (3-seed mean), replacing the full analytic attachment (Q-Only: 45.350% 6-task macro accuracy, 26.922 Test PPL, 0.0332 Load CV) with a comparable-strength Simple Surrogate (sample, averaging the 12 unexecuted experts' routing scores) drops 6-task macro accuracy by -0.853 pp (45.350% → 44.497%), supporting the incremental contribution of threshold-pivotal cardinality credit.
    • At the 4.38B scale (120M tokens), a matched CH-Primary ON/OFF ablation shows that disabling the analytic attachment degrades validation PPL from 132.54 ± 1.68 to 138.69 ± 1.97 and lowers 7-task macro accuracy from 29.28% to 29.12%.
    • We moderated our claims in Section 5.2 (and aligned Table S5) to attribute CH-MoE's primary quality gains over Default MoE to the holistic architectural co-design (renormalized execution merge, learned token gain, and dual-timescale realized-load feedback), while framing the analytic attachment as a modest, penalty-free additional learning refinement rather than hard execution budgets or QQ alone.
  3. Unified Quality–Latency Deployment Evaluation (Reviewers a1ED W2 & Q3, D9Hf W2 & Q2): Evaluating paired 36-layer CH-Primary and Top-4 models (3 seeds 42, 123, 777, 240M tokens) within a unified evaluation suite sharing the exact same causal attention, execution backend (grouped_bmm), precision stack (BF16 dense / FP32 MoE, TF32 off), and batch conditions (batch_size = 8), CH-Primary achieves -37% TTFT, +11% generation throughput, -52% Load CV, and -1.94% Corpus PPL (60.49 → 59.31, 3-seed CIs < 0), alongside a minor -0.12 pp change in discrete 7-task macro accuracy (36.13% → 36.01%) under a 90% active-slot budget.
  4. Grouped vs. Global Selection & Batching Perturbation Bounds (Reviewers X4pL W2 & Q2, D9Hf Q3): We added the full 2-seed factorial table isolating grouped vs. unrestricted global selection (explicitly acknowledging its initialization dependency—+2.565% mean PPL gain under Dense-Slice initialization vs. near-zero -0.045% / +0.040% under independent initialization—and clarifying the pre-deletion slice-coverage mechanism) and a 3-seed batching perturbation study across 8 batching conditions (0.0000 pp shift under batch execution reversal; mean shifts bounded within +0.1117 pp).

Once again, we thank the Area Chairs and all reviewers for their time, thoughtful feedback, and constructive engagement.

AuthorsOpenReview ↗

Summary of Our Rebuttal

Dear Area Chair, Senior Area Chair, and Program Chairs,

We would like to express our sincere gratitude for the time and effort you have devoted to our submission. As the discussion period draws to a close, to help minimize your workload, we respectfully provide a concise summary of our rebuttal:

Our Strengths Summarized by Reviewers

We appreciate the reviewers' recognition of the following core aspects of our work:

  • Explicit execution accounting & separation of budget from credit. Reviewer X4pL noted that "the framework uses the same hard mask to govern expert evaluation, output support, and work accounting. This makes the computational contract clear... This is a useful foundation for studying routing under strict budgets." Reviewer a1ED praised the "explicit separation of execution constraints and learning signals," and Reviewer D9Hf highlighted our "clear execution semantics for experts along with informative components controls."
  • Meaningful same-work controls & measured physical hardware speedups. Reviewers X4pL and a1ED both emphasized that Scalar-90 "retains the same call budget and structural allocation mechanism, making it a useful control," while Reviewer D9Hf noted "strong results throughout its well-designed experiments" and "very convincing" physical two-node measurements.

Our Main Revisions and Supplementary Experiments

Our individual responses and planned manuscript revisions incorporate the following controlled experiments and clarifications:

  • Modern MoE load-balancing baselines (Sec. 5.1 / Appendix): Compared CH-MoE against Qwen2.5-style auxiliary loss and DeepSeek-V3-style dynamic bias updates on identical C4-en validation micro-batches (40,960 tokens, 2 × 2048). CH-MoE reduces validation Load CV to 0.0112 (-92.5% vs. DeepSeek-V3-style 0.1493 and -94.3% vs. Qwen2.5-style 0.1976) with the lowest training loss (3.3372).
  • Threshold-pivotal credit vs. simple surrogates & claim moderation (Sec. 5.2 & Table S5): At 260M/5B (3 seeds), replacing our analytic attachment (Q-Only: 45.350% 6-task macro acc) with a comparable-strength Simple Surrogate (sample, averaging the 12 unexecuted experts) drops macro accuracy by -0.853 pp (44.497%). We also moderated our claims to attribute primary quality gains over Default MoE to our shared architectural co-design, framing the analytic attachment as a penalty-free refinement.
  • Matched CH-Primary ON/OFF scale-up ablation (Sec. 5.3): At 4.38B (120M tokens), disabling the analytic attachment degrades validation PPL from 132.54 ± 1.68 to 138.69 ± 1.97 and lowers 7-task macro accuracy from 29.28% to 29.12%.
  • Unified deployment evaluation & quantitative causal audit (Sec. 5.3): Evaluating paired 36-layer CH-Primary and Top-4 models (3 seeds, 240M tokens) under a shared causal-attention and grouped_bmm stack (batch_size = 8) yields -37% TTFT, +11% generation throughput, -52% Load CV, and -1.94% Corpus PPL (60.49 → 59.31, 3-seed CIs < 0) with a -0.12 pp shift in 7-task macro accuracy (36.13% → 36.01%). Furthermore, our controlled DynamicCache causal audit across 2,048 windows (Tables R5–R6) confirms that 79.1% of CH-Primary's mean-NLL advantage over Top-4 survives under strictly zero future-token access (99.2% at pool=64), while token-local CH-Global is exact to 0.0 (prefix/batch) and 2.98e-8 (full-sequence vs. incremental).
  • Grouped vs. global factorial & batching sensitivity (Sec. 5.4 / Appendix): Added the 2-seed factorial table explicitly scoping grouped selection to initialization-coupled balance (+2.565% PPL gain under Dense-Slice vs. -0.045% / +0.040% under independent initialization) and a 3-seed batching study across 8 conditions (0.0000 pp order-reversal shift; ≤ +0.1117 pp mean shift).

Progress During the Discussion

  • Reviewer D9Hf: rating 3 (Weak acceptance), confidence 2. Confirmed on Nov 19 that all concerns were addressed and their review stays weakly positive.
  • Reviewer a1ED: rating 2 (Weak rejection), confidence 3 (lowered from 4). Provided Tables R1–R4; after a1ED's Nov 19 follow-up lowering confidence to 3 over causal attention vs. quota routing, we supplied the controlled DynamicCache causal audit (Tables R5–R6) and committed to explicit step-level vs. full-sequence qualifications in Sec. 5.3.
  • Reviewer X4pL: rating 2 (Weak rejection), confidence 3. Provided both requested controlled experiments (Table R1 simple surrogate; Table R2 independent-init factorial) and scoped Sec. 5.4 accordingly.

We respectfully invite you to examine our full rebuttal and planned revisions. Given the comprehensive controlled evidence addressing every concern raised, we sincerely hope you will find our paper worthy of acceptance. Thank you very much again for your invaluable time and consideration!

Best Regards,

Authors of CH-MoE

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.