- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors observe two limitations of existing backpropagation-free PTQ methods: group decisions ignore the error that the remaining unquantized weights can absorb, and discrete refinements keep the quantization grid fixed. To address this, they propose SchurOpt, which uses the Schur complement to obtain a groupwise objective that accounts for compensation by the remaining weights, and alternately optimizes the scale/zero-point and the integer codes. Building on this, the authors augment the optimization objective with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and token weighting to propose SchurQuant. Across eight Llama and Qwen models, SchurQuant achieves the highest mean zero-shot accuracy among the baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.
Critical strengths
- The paper provides a theoretical derivation, giving the method a clear theoretical foundation.
- Experiments show that the method brings clear improvements at low bit widths.
- The ablation studies are thorough, separately validating the contributions of the optimizer and the optimization objective.
Critical weaknesses
- Limited model coverage. The experiments involve only two model families, Llama and Qwen, at 0.6B–13B scale, lacking validation on larger models and other model families, as well as on architectures such as Mixture-of-Experts (MoE).
- Limited baselines. The paper discusses several related methods, but the experiments compare against only a few of them.
- Insufficient efficiency evaluation. As a compression method, the paper only reports the offline quantization time in the appendix, without reporting memory usage or inference metrics such as TTFT and TPOT.
- High quantization cost. The appendix shows that SchurQuant's quantization time is roughly 30–50× that of GPTQ and about 5× that of QEP.
Questions for authors
- How does SchurQuant perform on larger models, other model families, and architectures such as MoE?
- Could the authors report memory usage and inference metrics such as TTFT and TPOT?
- How does SchurQuant compare with the other related methods discussed in the paper?
We appreciate the reviewer's detailed and constructive assessment of our work. We address each point below.
W1/Q1. Larger models, other families, MoE. Added Llama-3.3-70B, Qwen3-30B-A3B (mixture-of-experts, 128 experts per layer, 8 active), and Qwen3.5-9B (a vision-language model with gated linear-attention blocks). Expert tensors are quantized expert by expert with routing-weighted statistics (App. I). On Qwen3.5-9B at 2 bits, the mean over six multimodal benchmarks is 48.17 for SchurQuant against 4.0-7.6 for GPTQ, FOEM, QEP, and CDQuant (full precision 80.81; App. G, Table 13). As a language model, 2-bit SchurQuant reaches 63.61% on Qwen3.5-9B against 61.34% for CDQuant (full precision 75.04%). Results (App. G, Table 12; PPL / mean accuracy in %):
At 2 bits, QEP stays below 47% on Llama-3.3-70B and Qwen3-30B-A3B, and SchurQuant exceeds the strongest baseline, CDQuant, by 1.1 pp on Llama-3.3-70B and by 4.7 pp on Qwen3-30B-A3B. QEP and CDQuant do not handle expert tensors, so on Qwen3-30B-A3B we apply their solvers to each expert on the tokens routed to it.
W2/Q3. More baselines. CDQuant, QuantEase, and LoaQ are now in Tables 1-2 on all eight models and bit widths, and all rows of these tables are backpropagation-free. Mean accuracy (%) over the eight models:
At 2 bits, SchurQuant outperforms the baselines on seven of the eight models. CDQuant is the closest competitor, achieving better results on Llama3-8B but lower accuracy on seven of the eight models. At 4 bits, the differences between methods are small. After applying the same scale tuning to all methods (Sec. 4.4, Table 3), SchurQuant+T achieves 58.91% mean accuracy at 2 bits, compared to 58.60% for CDQuant.
To isolate the effect of the solver, Table 8 (App. D) compares SchurOpt with the CDQuant solver using the same GuidedQuant statistics at 2 bits, before scale tuning. SchurOpt also reduces solver time by a factor of 2.4-8.0:
W3/Q2. Memory and inference metrics. Peak GPU and host memory during quantization are now reported (Table 7; e.g. Qwen3-4B: 47.6 GB GPU, 11.5 GB host). We measured TTFT/TPOT with vLLM on one H100 SXM 80GB GPU (App. H, Table 14). SchurQuant keeps GPTQ's storage format, and latency matches GPTQ within noise at every bit width. With 512-token prompts (ms):
TPOT falls by 53% and 63% relative to BF16. TTFT is higher for all quantized formats.
W4. Quantization cost. The cost in the submission came from recalibrating the full model before every layer, not from SchurOpt. Neither revised configuration needs this. Quantization time for 2-bit Qwen3-4B on one H100 SXM 80GB GPU (App. C, Table 6; SchurQuant took 253.7 min in the submission):
SchurQuant+T takes 17.6 min for Qwen3-4B, 24.5 min for Llama2-7B, and 27.5 min for Qwen3-8B on one H100 SXM 80GB GPU, including the gradient pass and scale tuning. Llama-3.3-70B is quantized by SchurQuant on a single H100 SXM 80GB GPU, keeping one block on the GPU at a time (table under W1/Q1 above). The Llama-3.3-70B quantization process takes approximately 3 hours.