- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents PACE, a mixed-precision post-training quantization method that allocates precision according to the predicted output damage of individual quantization candidates. PACE combines shared output-sensitivity estimation, independently optimized bit-conditioned candidates, and global allocation under a weight-code budget. Experiments across three 7–8B model families show that PACE achieves competitive quality under average 6-bit and 3-bit budgets, ranking first or second across all nine reported W6 comparisons. On Qwen3-8B, PACE is approximately 490× faster to quantize than SFMP while achieving comparable quality.
Critical strengths
1 PACE has a clear motivation and formulation. It estimates the output damage of each quantization candidate using both output sensitivity and quantization error. Precision is then allocated according to the predicted reduction in damage from using more bits.
2 The framework is well designed. It separates shared sensitivity estimation, local candidate optimization, and global precision allocation. Sensitivity estimation requires one backward pass per calibration sequence, and the estimates are reused across groups and bitwidths. The paper also reports runtime and memory usage for each stage and includes metadata in its packed-storage measurements.
3 The experiments provide useful support for the method. Predicted and measured quantization damage show strong agreement on the sampled groups. Under the average 6-bit W4/W8 setting, PACE ranks first or second across all nine comparisons in Table 1.
Critical weaknesses
1 The novelty over previous sensitivity-based quantization methods needs to be clearer. The paper should explain how its damage estimate for each quantization candidate differs from existing methods that use sensitivity and quantization error.
2 The reported storage reductions of about 2.6× and over 4.6× are relative to BF16. The paper should also compare actual storage sizes with other quantization methods.
3 PACE reduces quantization time and storage, but the paper does not show whether it makes LLM inference faster. The paper should also compare inference latency and throughput with other quantization methods.
Questions for authors
No further questions for the authors.
2 comments
Thank you for the clarifications and additional results. These address some of my concerns, but I still feel the novelty is limited. I will keep my original score.
Response to Reviewer n1CN
Thank you for recognizing PACE's formulation, shared estimation framework and competitive quality. We have clarified the method's distinctions and completed actual-storage comparisons.
W1. Distinction from sensitivity-based quantization
Related Work now explains how PACE connects candidate construction and allocation through a stationary reference-output objective. At the unquantized reference, the output residual and gradient are zero, giving H = E[JᵀJ/K]. Combined random output projections estimate weight-level sensitivity with one backward pass per calibration sequence, reused across groups and bitwidths.
HAWQ-V2 weights quantization error by average task-loss Hessian trace. LLM-MQ uses absolute first-order candidate costs. FGMP combines gradient-square sensitivity, weighted clipping and global selection, with a precision-pair score based on the weighted distance between low/high candidates. PACE instead independently optimizes each candidate and allocates by D(low) − D(high), their predicted costs against the same reference. Candidate optimization and allocation therefore follow a common output-fidelity criterion.
Main Table 1 compares damage prediction on shared neutral W4 stripe candidates, using 32 calibration and 64 held-out sequences. Ranges span three seeds.
Supplement B.5 fixes PACE-optimized W3 candidates and budget to test full-model allocation:
W2. Actual storage compared with quantization baselines
The Qwen3-8B storage–quality results are below. At the same W6 code budget, PACE and the compared baselines have similar storage footprints of approximately 5.1–5.2 GiB. Bold indicates the two lowest perplexity increases for each benchmark.
W3. Inference latency and throughput
PACE with a W6 allocation achieves approximately 1.27× faster decoding and 26.5% higher throughput in the tested Qwen3-8B-Base deployment. We use A100-SXM4-80GB. Both PACE and BF16 use decode CUDA Graphs, identical prompts and KV-cache settings, with 3 warmups and 30 timed requests per arm.
Batch 1, 128 input/512 output tokens. Ranges span two sessions with reversed execution order:
E2E latency falls by about 21%. Gains are in decoding and overall generation, with TTFT increasing by 8–10 ms.
We hope our responses address your concerns and strengthen your assessment of PACE.