Context-Robust: Context-Informed Activation Calibration for Mixture-of-Experts Language Models

Sparse routing makes activation calibration in Mixture-of-Experts (MoE) models inherently router-conditioned: each expert observes only the calibration tokens routed to it, so a small activation range observed in one calibration realization does not necessarily indicate a stable small range. Consequently, conventional Absmax calibration can select overly fine quantization steps that are sensitive to the routed calibration population. We introduce **Context-Robust (CR)**, a training-free activation calibration method that uses contextualized natural-routing executions to probe routed range instability. CR measures channelwise routed-range discrepancies between original and contextualized calibration collections, aggregates them into a site-level routed range-instability signal, and uses this signal as a one-sided floor on channelwise Absmax quantization steps. The resulting steps remain fixed during inference. Across Qwen3.5-35B-A3B and Gemma-4-26B-A4B, CR consistently improves average downstream performance over Absmax across FP16, AWQ, GPTQ, and SmoothQuant weight configurations, while exhibiting a repeatable task-selective preservation pattern. Controlled W16A8 diagnostics further show substantially reduced sample-level degradation dispersion and upper-tail failures while keeping mean degradation close to Absmax. These results show that router-conditioned range instability provides useful calibration information beyond activation magnitude for MoE activation quantization.