acceptodds
Under review as a conference paper at ICLR 2027

PACE: Precision Allocation with Cost Estimation for Mixed-Precision Quantization

Abstract

Mixed-precision quantization reduces model storage by assigning different bitwidths under a fixed budget. Its effectiveness depends on accurately estimating which precision decisions best preserve model quality. Existing methods often rely on indirect sensitivity measures that do not reflect the actual output damage caused by different quantization candidates. We introduce PACE (Precision Allocation with Cost Estimation), a second-order mixed-precision quantization method grounded in a stationary output-fidelity objective. PACE efficiently predicts the output damage of concrete precision candidates and globally allocates the available bit budget according to their estimated benefits. Experiments validate the consistency between predicted and measured quantization damage and compare PACE across three 7–8B model families, two perplexity benchmarks, six zero-shot tasks, and average 6-bit and 3-bit settings. Under the average 6-bit setting, PACE ranks first or second across all nine evaluations, with WikiText2 perplexity increases of at most 0.095, while remaining robust under the more challenging average 3-bit setting. On Qwen3-8B, PACE is approximately faster than SFMP, the strongest comparable-quality baseline. Its packed representations provide approximately compression under W6 and more than compression under W3.

Primary area:
foundation or frontier models, including LLMs
Keywords:
quantization, mixed precision, post-training quantization, large language models, memory-efficient ML

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n1CNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PACE, a mixed-precision post-training quantization method that allocates precision according to the predicted output damage of individual quantization candidates. PACE combines shared output-sensitivity estimation, independently optimized bit-conditioned candidates, and global allocation under a weight-code budget. Experiments across three 7–8B model families show that PACE achieves competitive quality under average 6-bit and 3-bit budgets, ranking first or second across all nine reported W6 comparisons. On Qwen3-8B, PACE is approximately 490× faster to quantize than SFMP while achieving comparable quality.

Critical strengths

1 PACE has a clear motivation and formulation. It estimates the output damage of each quantization candidate using both output sensitivity and quantization error. Precision is then allocated according to the predicted reduction in damage from using more bits.

2 The framework is well designed. It separates shared sensitivity estimation, local candidate optimization, and global precision allocation. Sensitivity estimation requires one backward pass per calibration sequence, and the estimates are reused across groups and bitwidths. The paper also reports runtime and memory usage for each stage and includes metadata in its packed-storage measurements.

3 The experiments provide useful support for the method. Predicted and measured quantization damage show strong agreement on the sampled groups. Under the average 6-bit W4/W8 setting, PACE ranks first or second across all nine comparisons in Table 1.

Critical weaknesses

1 The novelty over previous sensitivity-based quantization methods needs to be clearer. The paper should explain how its damage estimate for each quantization candidate differs from existing methods that use sensitivity and quantization error.

2 The reported storage reductions of about 2.6× and over 4.6× are relative to BF16. The paper should also compare actual storage sizes with other quantization methods.

3 PACE reduces quantization time and storage, but the paper does not show whether it makes LLM inference faster. The paper should also compare inference latency and throughput with other quantization methods.

Questions for authors

No further questions for the authors.

2 comments
AuthorsOpenReview ↗

Response to Reviewer n1CN

Thank you for recognizing PACE's formulation, shared estimation framework and competitive quality. We have clarified the method's distinctions and completed actual-storage comparisons.

W1. Distinction from sensitivity-based quantization

Related Work now explains how PACE connects candidate construction and allocation through a stationary reference-output objective. At the unquantized reference, the output residual and gradient are zero, giving H = E[JᵀJ/K]. Combined random output projections estimate weight-level sensitivity with one backward pass per calibration sequence, reused across groups and bitwidths.

HAWQ-V2 weights quantization error by average task-loss Hessian trace. LLM-MQ uses absolute first-order candidate costs. FGMP combines gradient-square sensitivity, weighted clipping and global selection, with a precision-pair score based on the weighted distance between low/high candidates. PACE instead independently optimizes each candidate and allocates by D(low) − D(high), their predicted costs against the same reference. Candidate optimization and allocation therefore follow a common output-fidelity criterion.

Main Table 1 compares damage prediction on shared neutral W4 stripe candidates, using 32 calibration and 64 held-out sequences. Ranges span three seeds.

ScoreHeld-out Spearman
PACE.917–.963
First-order.433
Group trace.738–.785
Fisher.881

Supplement B.5 fixes PACE-optimized W3 candidates and budget to test full-model allocation:

ScoreC4 PPLWiki PPL
PACE13.51598.0657
First-order14.20618.6553
FGMP pair13.51818.1123

W2. Actual storage compared with quantization baselines

The Qwen3-8B storage–quality results are below. At the same W6 code budget, PACE and the compared baselines have similar storage footprints of approximately 5.1–5.2 GiB. Bold indicates the two lowest perplexity increases for each benchmark.

MethodWeight + metadata payload (GiB)delta C4 PPLdelta Wiki PPL
AMQ5.104250.2450.185
MixQuant5.104250.2450.185
MixLLM5.105550.4370.263
OWQ5.104260.0950.077
SliM-LLM5.104260.1390.106
SFMP5.114510.0500.045
KronQ5.129520.0690.052
PACE5.104250.0670.041

W3. Inference latency and throughput

PACE with a W6 allocation achieves approximately 1.27× faster decoding and 26.5% higher throughput in the tested Qwen3-8B-Base deployment. We use A100-SXM4-80GB. Both PACE and BF16 use decode CUDA Graphs, identical prompts and KV-cache settings, with 3 warmups and 30 timed requests per arm.

Batch 1, 128 input/512 output tokens. Ranges span two sessions with reversed execution order:

MetricBF16PACE
TPOT (ms/token)17.152–17.15713.525
Output throughput (tok/s)58.17–58.1973.62–73.64
E2E latency (s)8.796–8.7996.951–6.952
TTFT (ms)31.46–31.8540.22–41.10

E2E latency falls by about 21%. Gains are in decoding and overall generation, with TTFT increasing by 8–10 ms.

We hope our responses address your concerns and strengthen your assessment of PACE.

Thank you for the clarifications and additional results. These address some of my concerns, but I still feel the novelty is limited. I will keep my original score.

Reviewer_Hw7COpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PACE, a post-training mixed-precision quantization method for large language models. It aims to reasonably allocate weight bitwidth under a fixed storage budget to minimize model performance degradation. To address the issue that existing methods rely on indirect sensitivity metrics and struggle to accurately evaluate the practical impact of specific quantization schemes, the authors formulate an optimization objective based on the fidelity of model outputs, and adopt second-order approximation to estimate the output loss induced by quantization perturbation. Experimental results demonstrate that this method can predict quantization loss with high accuracy. It effectively preserves model performance across different models and quantization precision configurations while substantially reducing the computational overhead of quantization.

Critical strengths

This paper includes reasonably extensive experiments, while its theoretical explanations are concise.

Critical weaknesses
  1. The core idea of the paper is to combine second-order sensitivity and actual quantization error to estimate the model loss incurred by different precision candidates, and perform global bitwidth allocation accordingly. However, similar ideas have been explored to varying extents in prior studies such as HAWQ-V2, LLM-MQ, and FGMP. These works also leverage curvature or gradient information to evaluate quantization perturbations and guide mixed-precision allocation via loss prediction. Therefore, the main contributions of PACE appear to lie primarily in the output-fidelity objective, the efficiency of sensitivity estimation, and the combination of existing techniques, rather than proposing a fundamentally new quantization allocation paradigm. The discussion of these highly related works is insufficient, and targeted comparative experiments are lacking, rendering its unique contributions and innovation boundary relative to existing methods unclear.
  2. PACE adopts a diagonal Hessian approximation, which decomposes the output loss caused by quantization into independent losses of each weight group to guide global precision allocation. Nevertheless, this treatment ignores interactions between weights and across different quantization groups, and thus may fail to accurately capture the overall loss when multiple weight groups are quantized simultaneously. Although the paper experimentally verifies the consistency between predicted loss and real loss, it does not thoroughly analyze the applicability of this approximation under complex perturbations and aggressive low-bit quantization. Moreover, while the paper claims to achieve accurate global bitwidth allocation, the optimality based on gain sorting holds directly only under specific grouping conditions. Sufficient solving details and optimality justifications are missing for allocation groups of varying sizes. Hence, the paper needs to further clarify the theoretical foundation of the loss approximation, as well as the effectiveness and optimality guarantees of the global optimization method under general settings.
Questions for authors

see weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Hw7C

Thank you for your assessment. We added targeted scoring, joint-perturbation and diagonal/full experiments. Main-paper and supplement revisions are marked in blue.

W1. What distinguishes PACE from HAWQ-V2, LLM-MQ and FGMP?

PACE connects candidate construction and allocation through a fixed-reference output-fidelity objective. At the reference, output residual and fidelity-objective gradient vanish. Its Hessian has Gauss–Newton form H = E[JᵀJ/K], without stationary task cross-entropy. Random output projections estimate shared weight-level sensitivity with one backward pass per calibration sequence.

Related Work now distinguishes:

  • HAWQ-V2 uses average task-loss Hessian trace, versus PACE's weight-level output sensitivity.
  • LLM-MQ uses absolute first-order costs, versus PACE's stationary output-fidelity quadratic.
  • FGMP uses gradient-square sensitivity, weighted clipping and global selection. Its upgrade score is the weighted squared distance between low/high candidates. PACE independently optimizes each candidate and uses D(low) − D(high), the difference of their costs against a common reference. These criteria generally differ.

Targeted comparisons. W4 damage prediction on shared neutral stripe candidates (main Table 1):

ScoreHeld-out Spearman
PACE.917–.963
First-order.433
Group trace.738–.785
Fisher.881

Scores use 32 calibration sequences and measurements use 64 held-out sequences. Ranges span three seeds.

Across 252 Qwen3-8B projections, all scores share frozen PACE-optimized candidates, grouping, W3 budget and solver. Only scoring changes:

ScoreC4 PPLWiki PPL
PACE13.51598.0657
HAWQ module trace15.21509.2611
LLM-MQ first-order14.20618.6553
Fisher cost13.52398.1200
FGMP pair13.51818.1123

Full results are in Supplement B.5.

W2. Interactions and aggressive low-bit quantization

Full-model W3 results support diagonal scoring at the default estimation budget. Supplement B.5 compares all 10,368 stripes with fixed neutral candidates, calibration inputs, projection draws and exact W3 budget, changing diagonal versus within-stripe full scores and the resulting allocation. Three-seed mean C4/Wiki PPL:

Repeats/inputDiagonalWithin-stripe full
115.4530 / 9.313715.4601 / 9.3932
815.3570 / 9.258715.3480 / 9.2787
3215.3597 / 9.266515.3218 / 9.2355

Diagonal achieves lower mean C4 and Wiki PPL at this budget, with lower Wiki PPL for all three seeds. At 32 repeats, within-stripe full improves C4/Wiki by 0.0379/0.0310 over diagonal at the same repeat budget. Thus diagonal scoring enables effective low-bit allocation at low estimation cost, while richer within-group curvature offers additional gains with more estimation.

Joint interventions test ranking under simultaneous quantization (B.3). We evaluate 48 groups across eight layers in 768 scenarios: W2/W4, two candidate banks, physical groups/stripes, and same-/cross-layer combinations. A.2 distinguishes diagonal, within-group full and joint-full quadratics, the latter retaining all interactions among the selected groups. Eight same-layer two-stripe sets with neutral W2 candidates at full quantization amplitude give held-out results:

Repeats/inputDiagonal SpearmanJoint-full SpearmanDiagonal errorJoint-full error
1.762–.929.786–.81025.6–43.8%37.4–44.8%
32.857.857–.97622.3–25.4%4.1–5.9%

Ranges span three seeds. Error is summed absolute prediction error divided by summed measured damage. Diagonal retains useful joint-damage ordering, while sufficiently estimated joint-full improves magnitude accuracy. Both candidate banks and same-/cross-layer results are included in the supplement.

W2. Unequal group sizes and optimality

Unequal groups are handled by integer programming, not direct gain sorting. Supplement A.5 specifies a parameter-weighted budget, selecting one bitwidth per group to minimize total predicted damage. Allocation averages just 0.88 s across the six main experiments.

We tested 324 cases covering different group sizes, two or three candidate bitwidths, and unattainable budgets. With numerical rescaling, all 211 feasible cases match independently computed optima, and all 113 infeasible budgets are correctly identified. Among these, 54 real-candidate cases also pass exact rational dynamic-programming checks.

Rescaling multiplies all damage coefficients by the same positive constant, preserving the mathematical optimum. It resolves numerical deviations in 37 test cases with tiny original coefficients.

We hope our responses address your concerns and strengthen your assessment of PACE.

Reviewer_hyy3OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to address the issues of mixed-precision quantization (MPQ). The authors argue that previous methods often rely on indirect sensitivity measures that do not reflect the actual output damage caused by different quantization candidates; however, this claim is not validated. Although the proposed method is validated on LLMs, the performance of PACE is not SOTA.

Critical strengths
  1. This paper is easy to read.
  2. This paper provides basic validation of its method.
Critical weaknesses
  1. The abstract of this paper is poorly written. It does not clearly state which task the MPQ is designed for; the later sections of the paper present LLMs, yet there is no analysis of what problems actually exist in LLM quantization.
  2. The claim that "Existing methods often rely on indirect sensitivity measures that do not reflect the actual output damage caused by different quantization candidates" is unreliable because there is no evidence to support it. Yet this is the most important motivation of the paper.
  3. The writing shows a lack of seriousness. Where is the related work section?
  4. The proposed method is second-order, which implies higher cost. Estimation-based methods are not necessarily suboptimal; what matters is their overall performance.
  5. The method's results are mediocre, and it even performs worse than previous methods on multiple results.
Questions for authors

What is meant by "do not reflect the actual output damage caused by different quantization candidates"? Please provide experimental data to rigorously prove this.

1 comment
AuthorsOpenReview ↗

Response to Reviewer hyy3

Thank you for your comments. We have revised the main paper and supplement, with changes marked in blue, and added direct comparisons addressing your central question.

W1. Task and the LLM-specific problem

We have revised the Abstract to explicitly define the task as post-training mixed-precision weight quantization of LLMs under a fixed weight-code budget. We further clarify the LLM-specific allocation challenge: the benefit of a precision upgrade depends not only on group sensitivity but also on the reconstruction error of the corresponding quantization candidate.

W2/Q1. Direct evidence for candidate-specific output damage

Original Figure 2 already shows accurate damage prediction by PACE, with Spearman correlations of 0.918 for W4 and 0.914 for W2. We now add held-out, shared-candidate comparisons to directly evaluate other scores against the same prediction target.

We now compare scores on identical W2/W4/W8 candidates, using 32 calibration sequences for scoring and 64 held-out sequences for measurement, each with 128 tokens. The study covers 256 sampled groups across 16 Qwen3-8B layers, including 32 column stripes. Main-paper Table 1 reports the neutral MSE-optimized stripe candidates, so candidate construction is shared across scores:

TargetPACEWeight errorFirst-orderGroup traceFisher
W2 damage.883–.934.393.433.666–.712.865
W4 damage.917–.963.404.433.738–.785.881
W8 damage.917–.967.482.198.717–.772.895
W2→W4 benefit.873–.919.400.299.654–.699.855
W4→W8 benefit.917–.963.404.425.738–.785.881

Values are held-out Spearman correlations, with ranges across three estimation seeds. PACE improves both damage prediction and the upgrade ranking used for allocation. The supplement's “Additional held-out fixed-candidate comparison” includes both candidate banks, both granularities, Kendall correlation, top-20% overlap and paired intervals.

W3. Related work

We agree that the original manuscript did not sufficiently distinguish PACE from closely related mixed-precision quantization methods. We have added a dedicated Related Work section, organized around candidate construction and precision allocation, and expanded the discussion of HAWQ-V2, LLM-MQ, and FGMP. We further clarify how PACE differs in its reference-output fidelity objective, weight-level sensitivity estimation, and precision-upgrade criterion.

W4. Measured cost of the second-order formulation

Low quantization cost is a central theme throughout the paper. A second-order formulation need not incur high computational cost. PACE avoids constructing a Hessian, using one backward pass per calibration sequence to estimate sensitivity shared across groups and bitwidths.

PACE combines leading quality with low quantization cost. In main-paper Table 2, it ranks first or second in all nine W6 model–metric combinations across three models. Table 3 gives the stage costs, with an average algorithm core time of about 54 s across six settings. On Qwen3-8B, calibration, candidate search/scoring and allocation total 53.43 s. Figure 3 reports quantization times under each method's native calibration protocol: comparable-quality methods require approximately 63 min for KronQ and 7.3 h for SFMP. This delivers a competitive quality–cost trade-off at LLM scale.

W5. Overall rankings and time cost

PACE's competitiveness is consistent across models and precision budgets. We evaluate Qwen3-8B, Mistral-7B and Llama-3.1-8B on C4, WikiText2 and the average accuracy of six zero-shot tasks. The main-paper W6 results and supplement B.4 W3 results give:

BudgetFirst, including tiesSecond, including tiesTop-two coverage
W6549/9
W3549/9
Total10818/18

These are three models × two budgets × three reported metrics. PACE ranks first in 10 and second in eight comparisons. At W6, its WikiText2 PPL increases are just 0.041, 0.018 and 0.095 across the three models.

Even where PACE ranks second, the gap can be small: on Qwen3-8B W6, its C4 PPL increase is 0.067 versus SFMP's 0.050, a difference of just 0.017. Under their respective reported calibration and selection protocols, PACE completes calibration, scoring and allocation in 53.43 s, compared with approximately 7.3 h for SFMP's selection/materialization process. This illustrates the strong quality–cost trade-off.

Thank you for your comments. We hope our responses address your constructive questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.