IncMoE: Active-Path Precision Increment for Calibration-Free Mixture-of-Expert Quantization

Mixture-of-Experts Models (MoE) achieve superior scaling efficiency, yet their massive parameter volume incurs prohibitive memory footprints and IO transfer overheads during serving. Mixed-precision quantization mitigates these bottlenecks by allocating non-uniform bit-widths across experts. However, existing paradigms suffer from two fundamental limitations: (i) static mixed-precision schemes enforce rigid allocations that fail to adapt to token-dependent routing dynamics, while multi-copy dynamic-precision approaches incur prohibitive memory duplication by storing multiple discrete weight copies; and (ii) conventional allocation heuristics rely on calibration activations that are vulnerable to runtime distribution shifts. To overcome these constraints, we propose IncMoE, a calibration-free, single-copy framework enabling runtime router-conditioned precision refinement. IncMoE reformulates precision from a static expert property into a dynamic trajectory of progressive refinements executed along routed paths. Specifically, each weight tensor is physically encoded once into nested, ordered bit slices comprising a compact base layer and progressively loadable increments, supporting variable precisions without parameter duplication. To identify critical increments without calibration data, we introduce BitInfo, which provides a data-free ranking score for individual bit slices directly from weight distributions to establish priority hierarchies. At runtime, a lightweight scheduler combines instantaneous gating confidences with offline priority lists, dynamically assigning prefix-consistent bit increments to active experts under a target bit budget. Experiments across three representative MoE architectures show that IncMoE approaches full-precision perplexity at an average 6-bit budget and preserves competitive downstream quality at 4 bits while strictly maintaining single-copy storage, establishing a superior accuracy-memory Pareto frontier for MoE serving.