- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes TAP-MoE, a task-aware framework for MoE compression and inference under memory constraints. Its core idea is to analyse how different token categories relate to expert-level statistics, then use the token-category distribution of a target workload to estimate expert sensitivities and formulate a precision-allocation optimisation problem. The paper provides theoretical bounds for the statistical transfer and an optimisation analysis of the resulting allocation. It further introduces expert access and prefetching strategies, together with output compensation for quantisation errors (Section C).
Experiments on three MoE models and three workloads (general text, code, and mathematics) show improved perplexity over the evaluated quantisation baselines, alongside reported decoding speedups of 1.25–1.75× on an NVIDIA RTX A6000.
Critical strengths
-
Well-motivated insight and coherent analysis. The paper formulates task-aware expert compression through category-conditioned statistics and workload-specific sensitivity estimation. The resulting optimisation objective has a clear mathematical formulation, and the statistical transfer bounds and optimisation analysis provide a coherent theoretical foundation.
-
Practical deployment extensions. Beyond precision allocation, the paper introduces expert access and prefetching strategies and output compensation for quantisation errors (Section C). These components address practical deployment requirements and are evaluated through experiments and ablations.
Critical weaknesses
none
Questions for authors
Estimate in different context:The method estimates target-task expert sensitivities by reweighting category-conditioned gradient statistics collected from shared calibration data. However, tokens from the same category may exhibit different gradients and routing behaviour in different contexts. Have the authors evaluated whether the estimated expert sensitivities remain stable when the token-category proportions are similar but the within-category context distribution changes? Such an evaluation would help clarify the reliability and scope of the proposed statistical transfer.