NormSub: Normalization-Affine Subspace Adaptation for Quantization Recovery

Low-bit quantization can substantially degrade model accuracy. Beyond improving quantizers and weight reconstruction, recovery requires understanding which parameters are useful to adapt under a limited budget. We investigate normalization-affine parameters as a structured, low-dimensional update subspace. In a fixed-graph study with matched parameter counts, data, and update budgets, affine updates achieve lower perplexity and higher mean accuracy than the tested weight-coordinate controls, with a small advantage over gradient-selected coordinates. Functional diagnostics further show that task-level recovery need not reduce every local reconstruction error. These findings motivate NormSub, a two-stage affine adaptation framework around a target post-training quantization (PTQ) backend. The pre-stage adapts only normalization-affine parameters on a disposable backend-aligned quantized proxy, then discards the proxy, restores the original floating-point weights, and transfers only the backend-compatible affine state for fresh PTQ reconstruction. The post-stage refines the affine coordinates exposed by the reconstructed graph while freezing its non-affine persistent state. Across the evaluated dense language models and PTQ backends, NormSub improves perplexity and mean zero-shot accuracy. On Qwen3-8B W2A4KV4 with QuaRot+ResComp, the unweighted six-task mean accuracy increases from 49.6% to 58.1%.