acceptodds
Published as a conference paper at NeurIPS 2026

A Method-Class Divide in Sub-4-Bit Quantization: Iterative vs. Single-Pass Sensitivity to Data Composition

Yehong Jiang, Yen-Kuang Chen, Xinmin Tian

Abstract

Block-wise post-training quantization (PTQ) compresses LLMs by minimizing per-block reconstruction error on a small calibration set. We show that the composition of this set (200 samples) is a targeted lever at extreme bit widths, acting only on iterative methods. In a controlled study spanning six data-dependent PTQ methods plus a data-free baseline, six models (1.7B–8B) across three families plus 70B verification, three bit widths, and a 10-task evaluation with four-seed replication, we find a sharp near-cliff method-class divide. In each method's pre-cliff regime (where the model retains predictive function), iterative methods (SignRound V2, OmniQuant, AQLM) gain 0.74–1.93pp on the 10-task average from switching Pile to a fragility-guided dataset (FragCal-DC), while single-pass analytical methods (GPTQ, AWQ) show near-zero response (pp). Both classes are evaluated on functional Pile baselines (0.42–0.61, well above chance); above the cliff, every method becomes data-composition-insensitive. A reconstruction-loss paradox (the method that most reduces its proxy loss gains nothing downstream, while the method that barely moves the proxy gains the most) sharpens the mechanism; fixed-reference held-out evaluation confirms the proxy–downstream disconnect. A wrong-answer contamination ablation retains 87–91% of the improvement and a data-dependent single-pass control (RTN-opt) shows zero response, isolating the iterative optimization loop. A weight-space probe shows SignRound-V2 preserves higher per-group scale heterogeneity than GPTQ on 9 of 9 (model, width) cells; a scale-flatten knockout shows this heterogeneity is necessary for iterative fidelity. We release FragCal-DC, the discriminative-task instance used here, as a drop-in JSON for the evaluated setting.

Related papers

See more related submissions when the market opens.