- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a three step post training binarization for mixture of experts LLMs. First, the expert weights of each projection are jointly factorized via SVD into a shared 8 bit factor and expert specific binary factors, refined against the routed aggregate output error. Second, binary residual blocks are added greedily by their conditional reconstruction gain per stored bit under a fixed storage budget. Last, the router is binarized and calibrated to preserve the margins between selected and unselected experts. Each step is ablated, and comparisons to binary and low bit PTQ baselines on multiple MoE models are shown.
Critical strengths
- Principled design: three MoE specific binarization challenges are identified and each is addressed by a dedicated objective. The new elements are the routed aggregate loss with cross expert error terms (Eq. 5), residual selection by conditional gain per stored bit (Eq. 9), and margin preserving router calibration (Eq. 12).
- Large and consistent gains over the binary baselines at similar resident storage, also on instruction tuned models, with ablations showing that each component contributes.
Critical weaknesses
Major:
- Budget mismatch on the motivating metric. Tables 2 and 3 match the baselines at 1.11 BPW, but this amortizes a per layer 8 bit shared factor over all experts. On the per token active path, which motivates the paper (Section 1) and the speedup (Section 4.4), BitMoE sits at 1.72 BPW while BiLLM and ARB-LLM remain at 1.11. The headline gains are reported at unequal cost per token.
- Baseline configuration is unreported and the GPTQ rows are hard to reconcile with the literature. No group size or quantized scope is given. MxMoE (ICML 2025, cited by the authors) reports GPTQ on DeepSeek-V2-Lite with the same calibration set at 6.18 perplexity for 3.25 bits and 8.49 for 2.25 bits, against 16.26 and 5e7 at 3 and 2 bits here. AlphaQ (arXiv 2606.04980) shows a similar gap on Qwen3-30B-A3B.
Minor:
- MOBR is not isolated. The router is binarized although its storage is negligible, no full precision router control is given, expert shift in Table 1 is neither defined nor explained, and the cosine similarity metric of Figure 5 is not what MOBR optimizes.
- Storage versus decode performance. The paper should either confine its claims to resident storage, where the reduction is real and large, or show how it translates into decode speed: a description of the kernel for the INT8 shared factor times binary expert product, bytes read per token, and a comparison to FP8 or NVFP4 on the same GPU. A single speedup against a BF16 reference at 42 tok/s does neither.
- Presentation. Notation and terminology are not consistent: “shared experts” versus “shared factors”, the star denotes both teacher quantities and optima, and s^r and s^c, the “(m,1)” superscript, “planes” and the router’s “code and scale representation” are used before being defined. Figure 4 is too dense to serve as an overview. The statement “reconstruction errors are further combined through sparse routing” (Section 3.1) needs explanation. It should be stated that c_t is the always active branch on quantized inputs, what “deployed” routing weights p̄ means, and that all losses are evaluated on the requantized weights. The relation between q^(0) and q̂ should be clarified, and norm and vector notation unified throughout.
Questions for authors
Major:
- How much of the gain over BiLLM and ARB-LLM remains at a matched active path budget?
- Which group size and quantized scope were used for GPTQ and AWQ, and how do you reconcile the 3 bit rows with MxMoE’s numbers on the same model and calibration set?
- Which data types do the row and column scales in Eq. 3 and the residual planes use, how are attention, always active experts, embeddings and norms treated, and are they counted in the 1.11 BPW?
Minor:
- What does BitMoE achieve with a full precision router, and why is cosine similarity of routing probabilities the criterion for MOBR?
- How does BitMoE compare to a ternary PTQ baseline at matched active bits, and to the cited but uncompared STBLLM, DBF, KBVQ-MoE and MxMoE?
3 comments
Thank you for the detailed response and the additional experiments. The manuscript has improved during the rebuttal, and some of my minor points have been clarified. The response is very extensive, with eight parts for my review alone. Since my time for this review is limited, I may have missed some details. Still, the need for such a long response suggests to me that the manuscript is not yet mature enough for acceptance, and the main concerns behind my initial assessment remain. The points below summarize these concerns, which in my view reflect conceptual shortcomings of the study.
- Active path budget (major 1). I agree that the bitmaps and scales of the baselines should be counted. With this accounting BitMoE is more accurate at a lower active budget (about 1.7 versus 2.5 to 2.9 BPW), which strengthens the paper. However, the main result tables do not reflect this. Tables 2 and 3 report only the static BPW and still list BiLLM and ARB-LLM at 1.06 to 1.11 BPW, although the accounting of Table 7 would give them 2.5 to 2.9 BPW there as well. In addition, the storage format of the attention and dense projections of BitMoE is not stated. The reported budgets are therefore neither consistent across tables nor fully reproducible.
- Comparison with baselines (major 2). Thank you for fixing the calibration issue and for revising the claim on 3 bit GPTQ. BitMoE improves on the binary baselines (see the previous point). However, only AWQ and GPTQ were rerun after the fix, so it remains unclear whether BiLLM, ARB-LLM, MoEQuant and NoWag are affected as well. Compared with 3 bit GPTQ, BitMoE now reaches similar accuracy at a lower budget. Compared with 4 bit methods, it needs less memory and bandwidth, but it is also clearly less accurate (59.85 versus 68.07 on Qwen3-30B-A3B). It is therefore unclear whether BitMoE improves the Pareto curve of accuracy versus memory and bandwidth. At the same budget, a smaller model at 4 bit may be the better choice.
- Reliability of the efficiency numbers. Throughput is crucial for judging the method. In your response to Reviewer 7f1H you write: "The illustrative figures above require verification before they are presented as measured results." The same figures appear in Table 11, where the throughput is described as measured. It is therefore unclear which throughput numbers are measured. The model sizes in Table 11 are also about twice as large as the whole model BPW in Appendix A.4.2 implies. For Qwen3-30B-A3B, Table 11 reports 10.94 GB, while 1.41 BPW for about 30.5B parameters gives about 5.4 GB. The FP8 result in your response to me is not part of the revised PDF.
Thank you for the follow-up clarifications and the further corrections. My main concerns are now largely addressed. Tables 2 and 3 report the active path budget for all methods, the baseline configuration is documented, and the comparison with 3 bit GPTQ has been revised. In my view the paper now barely crosses the acceptance threshold, and I will raise my rating from 2 to 3. Two concerns remain.
- Many numbers changed during the rebuttal. This includes the AWQ and GPTQ baselines after the calibration fix, the model sizes after the saving format fix, and the throughput note in the response to Reviewer 7f1H. This limits my confidence in the reported numbers. Please check all tables once more for the final version.
- Changes to the manuscript are still pending. Table 10 still lists 26.96 GB for the 80B model. This is about 2.7 bits per parameter and does not match the corrected sizes in Table 11 (about 1.4 to 1.6 bits per parameter). The storage format of the attention and dense projections (Q1_0, 1.125 BPW) is stated only in the response, and the manuscript does not describe how these weights are binarized. The final version should contain all details needed to reproduce the reported budgets and results.
Follow-up on Our Rebuttal and Revised Manuscript
Dear Reviewer PRjS,
Please forgive this brief follow-up. With fewer than 36 hours remaining in the author–reviewer discussion period, we wanted to kindly ask whether you have had an opportunity to review our rebuttal and the revised manuscript.
In response to your concerns, we clarified the static and active-path budget accounting and added an estimated active-budget comparison. We also corrected an issue in the AWQ/GPTQ calibration procedure and updated the corresponding results, added a BF16-router control to better assess MOBR, and provided further details on decoding performance and comparisons with related methods.
We hope these additions address your principal concerns. If you find that the new evidence resolves them, we would be sincerely grateful if you could reconsider your overall assessment. If any concern remains, even a brief indication would help us provide further clarification promptly.
We understand the demands on your time and sincerely appreciate your careful consideration.
Best regards,
The Authors