acceptodds
Under review as a conference paper at ICLR 2027

PPCC: Price-Preserving Calibration Coresets for Post-Training Quantization

10 views

Abstract

Calibration coreset selection is essential for efficient post-training quantization (PTQ) of large language models, as limited calibration data must support reliable layer-wise choices of bitwidths, quantization granularity, and pre-quantization transformations. However, existing criteria based on data diversity or activation coverage do not directly characterize how calibration errors affect these decisions. To address this, we propose Price-Preserving Calibration Coresets (), a framework that reinterprets calibration selection through the preservation of pairwise quantization price gaps. This perspective decomposes price-gap distortion into distribution shift, finite-pool estimation, and coreset compression, where the compression term, which coreset selection controls, motivates a price-weighted moment-matching objective. Building on this foundation, inspired by traditional herding algorithms, we introduce an efficient greedy algorithm with a correction term derived by minimizing the expected terminal discrepancy. Theoretically, we establish finite-pool concentration bounds and prove that the resulting price-weighted squared moment discrepancy is no larger than its expectation under uniform random selection at the same budget. Empirical results show that recovers the full-pool price minimizer at all three tested budgets on Llama-3.2-1B and improves average downstream task scores in most configuration selection settings, with perplexity reductions of up to over the best baseline on Llama-3.1-8B.

Primary area:
optimization
Keywords:
post-training quantization, calibration selection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cFX9OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a theoretically grounded approach for calibration data selection for post-training quantization. The central assumption of the work is that there is no need to know the exact performance of every single hyperparameter combination for quantization but to preserve the relative ranking (i.e., price gap in terms of KL divergence between the full and quantized models) between options. The paper provides a proved theoretical error bound that breaks total price-gap distortion into three components consisting of the distribution shift error (difference between calibration and deployment data), the sampling error from using a finite pool of data, and the calibration data compression error (calibration data representativeness when subsampling). The later is the only term that can directly be controlled by ensuring that the average activations of the selected calibration set match those of the full data (e.g., the whole of C4).

The proposed PPCC algorithm selects an optimal calibration set by greedily sampling one data point at a time by adapting the Herding algorithm. Empirical results show that it offers better downstream performance for Llama 3.1 8B while results on the Qwen3 8B family are somewhat mixed.

Critical strengths
  1. Theoretically grounded and principled method for choosing calibration data for post-training quantization.

  2. The proof of the theoretical error bound that breaks total price-gap distortion is the strongest contribution.

Critical weaknesses
  1. The empirical evidence is somewhat limited (and rather short in the paper). The method is only applied on smallish models (1-8B). Results on Qwen are not as strong as for the Llama model. It would make the work much more complete if the evaluation include larger models (~30B or higher).

  2. In general, the PPCC runtime is higher and more compute demanding compared to models that do not require the model for data selection.

Questions for authors
  1. How applicable is the proposed approach to architectures that are not dense, e.g. MoE?
3 comments

We thank Reviewer cFX9 for the positive assessment of the error decomposition and for the suggestions on model scale, runtime, and MoE.


W1. limited empirical evidence and evaluation on larger models

On Qwen3-8B, PPCC still achieves the highest average score in all four settings for the Base model, including a gain of 1.98 points and the lowest PPL in bitwidth selection, and in all three settings for the Instruct model in Appendix D. We believe that the margins in the other three Base settings are smaller than on Llama because Qwen3-8B is less sensitive to quantization in our setting. For example, in bitwidth selection at 3.25 bits, the WikiText-2 PPL of Qwen3-8B Base stays at or below 9.83 (7.00 in BF16) under every selector, while that of Llama-3.1-8B Base reaches 32.26 (6.24 in BF16), so on Qwen the calibration data has little room to change the result. This agrees with prior findings that Llama-3 models degrade notably under low-bit quantization [1] and that the effect of calibration data differs across model families [2].

  • [1] W. Huang et al. An Empirical Study of LLaMA3 Quantization: From LLMs to MLLMs. arXiv:2404.14047, 2024.
  • [2] M. Williams and N. Aletras. On the Impact of Calibration Data in Post-training Quantization and Pruning. ACL 2024.

To evaluate a larger model, we ran bitwidth selection on Qwen3-32B at 2.75 and 3.00 bits, following the Qwen3-8B Instruct setting in Appendix D (non-thinking mode, AWQ INT2/3/4 with group size 128, N=64N=64). Below we report the scores on GSM8K, AIME24, AIME25, and GPQA and their average.

BitsMethodGSM8KAIME24AIME25GPQAAvg.
2.75Random86.5016.4610.2141.7438.73
ZipCal85.9714.7915.8342.4139.75
CoverCal86.9617.0815.0041.5240.14
PPCC89.4618.7516.0443.3041.89
3.00Random91.2821.0417.5046.6544.12
ZipCal89.5420.4219.3846.6543.99
CoverCal90.4522.2915.2147.3243.82
PPCC91.5822.5020.4249.3345.96

At 2.75 and 3.00 bits, where quantization costs more, PPCC achieves the best score on all four tasks and the highest average, 1.75 and 1.84 points above the strongest baseline, respectively.

We will include these results and discussion in the revision.

W2. runtime and computational cost of calibration data selection

We agree that PPCC takes longer than ZipCal, because ZipCal selects sequences by vocabulary coverage without any forward pass. However, we believe this cost is acceptable, since PPCC takes 12–14 minutes on one H100 GPU for an 8B model, comparable to the other model-aware selector CoverCal, and this cost is paid only once before deployment.

In return, PPCC leads to better configurations in Sec. 5.2. Averaged over all selection settings, the average downstream score of PPCC exceeds that of the strongest baseline by 1.11, 1.53, and 1.37 points on Llama-3.1-8B Base, Qwen3-8B Base, and Qwen3-8B Instruct, where the strongest baseline is ZipCal, Random, and CoverCal, respectively. PPCC is also more reliable, with its WikiText-2 PPL on Llama-3.1-8B staying between 8.47 and 8.71 across the four settings of Table 1, while each baseline exceeds 13 in at least one setting. Since the selected configuration determines the quality of the quantized model throughout its deployment, we consider a one-time cost of about 13 minutes a worthwhile trade-off.

Q1. applicability to non-dense architectures such as MoE

Yes, PPCC can be extended to MoE models. The price is defined for each linear layer and depends on the data only through the input moments of that layer. In an MoE model, the linear layers of each expert can be treated as separate layers, whose input moments are computed over the tokens routed to that expert, and the price-weighted moment matching and Alg. 1 can then be applied to these layers. Matching these moments also takes into account how often each expert is used in the pool, which is relevant to MoE calibration, where some experts receive few calibration tokens.

However, a complete extension is more involved, because our price and the analysis in Secs. 2 and 3 assume that each token passes through the same layers before and after quantization, whereas in an MoE model, quantization can change which experts a token is routed to. Extending the price and the analysis to account for these routing changes requires substantial further work, which goes beyond the scope of this paper.

Thank you for this suggestion. We leave the extension of PPCC to MoE models as future work and will add this discussion to the revision.


We hope the new results on Qwen3-32B and the cost analysis resolve your concerns, and we are happy to provide any additional details if needed.

Thank you for your clarifications, I'm keeping my score. Best of luck with your submission.

Thank you for your reply and your kind wishes. We hope our responses have fully addressed your concerns. Please feel free to let us know if you have any further questions.

Reviewer_vftkOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Price-Preserving Calibration Coreset (PPCC), a calibration coreset selection method designed for quantization configuration selection. Through a series of approximations, the method represents each calibration sample using layer-wise moments and then applies a herding algorithm to select the coreset. Experimental results show that PPCC outperforms Random, CoverCal, and ZipCal in preserving configuration selection quality.

Critical strengths

None

Critical weaknesses

The paper is hard to follow. Sections 3.2 and 3.3 consist almost entirely of propositions and theorems that are unnecessary for understanding the PPCC algorithm. The extensive mathematical treatment of aspects that are not central to the proposed method adds unnecessary complexity. These sections also interrupt the flow of the presentation and make it difficult to follow the notation used in PPCC. Moreover, several important implementation details are missing, including how Hz,lH_{z,l} is computed, how the selected coreset is used, and what candidate configuration pool is used for configuration selection in Table 1.

The paper does not explain how Hz,lH_{z,l} is computed, even though it is essential for determining γ\gamma in PPCC and plays a critical role in the algorithm. This omission hides how much information is used to guide coreset selection for configuration selection.

The paper lacks comparisons with mixed-precision PTQ methods, such as HIGGS [1] and AMQ [2], which can naturally be extended to the configuration selection setting. Evaluating each candidate configuration on a selected calibration coreset is not the only approach to configuration selection. Moreover, the quantities used in PPCC are also relevant to mixed-precision PTQ methods, making these methods particularly important baselines for comparison.

[1] Malinovskii, Vladimir, et al. "HIGGS: Pushing the limits of large language model quantization via the linearity theorem." Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

[2] Lee, Sangjun, et al. "Amq: Enabling automl for mixed-precision weight-only quantization of large language models." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Questions for authors

See Weaknesses

5 comments

We thank Reviewer vftk for the detailed comments.


W1. presentation of Sections 3.2–3.3 and missing implementation details

We understand this concern, but we believe Secs. 3.2 and 3.3 are important for understanding PPCC, since they explain why PPCC only needs to match the calibration pool. Specifically, Prop. 3.3 in Sec. 3.2 shows that the shift term EshiftE_{\mathrm{shift}} is set by the choice of the calibration source rather than by the coreset, which is why we fix C4 as the pool instead of asking the selector to correct the shift. In addition, Thm. 3.4 in Sec. 3.3 shows that the pool term EpoolE_{\mathrm{pool}} is small once the pool is large enough, and the required size grows only with log⁡J\log J, so a moderate pool suffices even for the many configurations in Table 1. Without these results, matching the pool would say nothing about the deployment decision.

For the implementation details, (1) Hz,l\mathbf H_{z,l} is the empirical Fisher of the full-precision model with respect to the output of layer ll, and we describe its computation in our response to W2. (2) In Sec. 5.2, every calibration selector feeds its coreset into the same pipeline, in which PricePTQ estimates its price table on the coreset and selects a configuration under the bit budget, and GPTQ or AWQ then quantizes this configuration with the same coreset. (3) In Table 1, each linear layer of Llama-3.1-8B chooses one candidate from the bitwidths, granularities, and transformations of the corresponding setting. The bitwidth setting fixes GPTQ with group size 128, the transformation setting fixes INT3 with group size 128, the granularity setting fixes GPTQ with INT3, and the mixed setting varies all three. The bit cost includes the storage of quantization scales, and the target is 3.25 bits for the bitwidth, transformation, and mixed settings and 3.5 bits for the granularity setting.

In the revision, we will make the link between Secs. 3.2–3.3 and the PPCC algorithm more explicit and describe the above implementation details more clearly.

W3. comparisons with mixed-precision PTQ methods such as HIGGS and AMQ

First of all, we would like to clarify that HIGGS and AMQ address a different step of the pipeline from PPCC: they decide which configuration each layer receives, while PPCC decides which calibration data a configuration selector should use. PPCC is therefore orthogonal to configuration selectors such as HIGGS, AMQ, and PricePTQ rather than an alternative to them, and in this paper it is designed to preserve the prices that PricePTQ compares. Accordingly, we compare complete pipelines: HIGGS, AMQ, and PricePTQ with a random coreset and with the PPCC coreset.

MethodLlama-3.1-8B (PPL ↓\downarrow)Llama-3.1-8B (Avg. ↑\uparrow)Qwen3-8B (PPL ↓\downarrow)Qwen3-8B (Avg. ↑\uparrow)
HIGGS allocation18.9459.838.8663.21
AMQ12.2061.228.7563.79
PricePTQ + Random14.3460.478.7562.73
PricePTQ + PPCC8.7161.888.6264.71

PricePTQ with the PPCC coreset achieves the lowest PPL and the highest average score on both models. With a random coreset, however, PricePTQ falls behind AMQ, which is expected because AMQ searches the configuration space for more than 17 hours, while PricePTQ selects a configuration in less than one minute. The PPCC coreset closes this gap at an extra cost of about 12 minutes, and in doing so it lowers the PPL of PricePTQ from 14.34 to 8.71 on Llama-3.1-8B and from 8.75 to 8.62 on Qwen3-8B, and raises the average score by 1.41 and 1.98 points, respectively. Overall, PricePTQ with PPCC outperforms both HIGGS and AMQ, and on Llama-3.1-8B it is about 10 times faster than HIGGS and 80 times faster than AMQ.

We will add this comparison and the discussion of HIGGS and AMQ to the revision.

W2. computation of Hz,lH_{z,l} and its role in PPCC

PPCC uses only the full-precision model and the calibration pool: no evaluation data, downstream tasks, or quantized-model evaluations are involved. Hz,l\mathbf H_{z,l} is the Hessian of the KL objective in Eq. (1) with respect to the output zl\mathbf z_l of layer ll (Sec. 2.2), and we estimate it with the empirical Fisher of the full-precision model on a reference set T\mathcal T of 64 sequences taken from the C4 pool,

H^z,l=1∣T∣∑t∈Tgl,tgl,t⊤\widehat{\mathbf{H}}_{z,l}=\frac{1}{|\mathcal{T}|}\sum _ {t \in \mathcal{T}}\mathbf{g _ {l,t}}\mathbf{g _ {l,t}}^\top

where gl,t\mathbf g_{l,t} is the gradient of the next-token cross-entropy with respect to zl\mathbf z_l at token tt. PPCC uses H^z,l\widehat{\mathbf H} _ {z,l} only through the sensitivity γl,j\gamma_{l,j}, which is half the range of the diagonal response entries over the candidates of layer ll (identity below Thm. 3.5):

[M^l(αl)]jj=1∣T∣∑t∈T([ΔWl(αl)Tl(αl)]⊤gl,t)j2,γl,j=12(max⁡α[M^l(αl)]jj−min⁡α[M^l(αl)]jj),[\widehat{\mathbf M} _ l(\alpha _ l)] _ {jj}=\frac1{|\mathcal T|}\sum _ {t\in\mathcal T}\Big(\big[\Delta\mathbf W _ l(\alpha _ l)\mathbf T _ l(\alpha _ l)\big]^\top\mathbf g _ {l,t}\Big) _ j^2 ,\qquad \gamma _ {l,j}=\frac12\Big(\max _ {\alpha}[\widehat{\mathbf M} _ l(\alpha _ l)] _ {jj}-\min _ {\alpha}[\widehat{\mathbf M} _ l(\alpha _ l)] _ {jj}\Big) ,

where ΔWl(αl)\Delta\mathbf W_l(\alpha_l) is computed once from the full-precision weights. Coreset selection is then guided only by these sensitivities and by the per-sample input moments cl,j(x)c_{l,j}(\mathbf x) from forward passes over the pool.

We use the empirical Fisher to estimate the Hessian for two reasons. (1) The empirical Fisher is an efficient and widely used approximation of the Hessian, and it is also used by PricePTQ, so PPCC and the configuration selector rely on the same type of curvature estimate. (2) Since the KL objective is taken with respect to the full-precision model π0\pi_0 itself, its Hessian at the full-precision model is exactly equal to the Fisher information of π0\pi_0, Hz,l=ExEy∼π0(⋅∣x)[gl(x,y)gl(x,y)⊤]\mathbf H_{z,l}=\mathbb E_{\mathbf x}\mathbb E_{y\sim\pi_0(\cdot\mid\mathbf x)}[\mathbf g_l(\mathbf x,y)\mathbf g_l(\mathbf x,y)^\top], where gl(x,y)\mathbf g_l(\mathbf x,y) is the gradient of −log⁡π0(y∣x)-\log\pi_0(y\mid\mathbf x) with respect to zl\mathbf z_l. This follows from the score identity used in the proof of Prop. 2.2. The empirical Fisher differs from it only in that gl,t\mathbf g_{l,t} evaluates gl(x,y)\mathbf g_l(\mathbf x,y) at the next token in the data instead of at a sample from π0\pi_0, so the two coincide when the predictive distribution of the full-precision model matches the next-token distribution of the calibration data.

To check whether this choice matters for PPCC, we computed the Fisher of π0\pi_0 directly in the setting of Sec. 5.1 by sampling four labels per token from the full-precision model on the same 64 sequences, keeping everything else fixed. The resulting γ\gamma has a Pearson correlation of 0.999 and a Spearman correlation of 0.993 with the original γ\gamma. We then reran PPCC with this γ\gamma in the configuration ranking and decision preservation experiment of Sec. 5.1, following the protocol of Fig. 2.

γ\gamma fromN=8N=8 (Top-1 %)N=16N=16 (Top-1 %)N=32N=32 (Top-1 %)N=8N=8 (Kendall τb\tau_b)N=16N=16 (Kendall τb\tau_b)N=32N=32 (Kendall τb\tau_b)
Empirical Fisher961001000.9550.9580.962
Fisher of π0\pi_095991000.9550.9590.962

The two versions of γ\gamma give nearly the same Top-1 recovery and Kendall τb\tau_b, so the selection is insensitive to the source of the labels in the Fisher. We will add these details to the revision.


We hope these details and the new baselines address your concerns, and we would be happy to provide further clarifications if needed.

We thank the authors for their rebuttal, which addresses some of our concerns. We have raised our score accordingly. However, the following concerns remain:

  1. The definition of EshiftE_{\text{shift}} already states that it is determined by the choice of calibration source. Further clarification on this point seems unnecessary.

  2. An essential component of the experimental setup is entirely omitted. The Experiments section never mentions PricePTQ, even though the actual method evaluated is PricePTQ + PPCC.

  3. How does PPCC perform when combined with other configuration selection methods, such as AMQ and HIGGS? Is PPCC specifically designed for PricePTQ, potentially limiting its generalizability to other configuration selection methods?

Reviewer_mQVJOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PPCC (Price-Preserving Calibration Coresets), a calibration data selection framework for post-training quantization (PTQ) of large language models. PPCC aims to preserve the pairwise quantization price gaps among candidate PTQ configurations, thereby supporting consistent configuration selection under limited calibration budgets. The authors formulate the problem through pairwise price-gap preservation and decompose the associated distortion into distribution shift, finite-pool estimation, and coreset compression errors. Based on this analysis, they introduce a price-weighted moment-matching objective and develop a herding-inspired greedy selection algorithm with a finite-population correction. Experiments across multiple LLMs and PTQ configuration settings demonstrate improvements in configuration ranking preservation, optimal configuration recovery, and downstream task performance compared with existing calibration selection methods.

Critical strengths
  1. Well-motivated and decision-oriented problem formulation. The paper addresses a meaningful problem in PTQ calibration data selection. Rather than focusing only on data diversity or activation coverage, the proposed framework directly connects calibration sample selection to the preservation of quantization configuration preferences. This provides a principled and practically relevant perspective on calibration coreset construction.
  2. Solid theoretical motivation and encouraging empirical results. The paper establishes a clear connection between price-gap distortion and price-weighted activation moment matching, providing theoretical motivation for the proposed selection algorithm. The experimental evaluation includes multiple LLMs, PTQ configuration settings, and relevant calibration selection baselines. The improvements in configuration recovery and downstream performance provide encouraging evidence of the effectiveness of PPCC.
Critical weaknesses
  1. Lack of component-level ablation studies. My main concern is the lack of ablation experiments that isolate the contributions of the two key components of PPCC: price-weighted moment matching and the finite-population correction in the herding-based selection algorithm. Although the paper demonstrates improvements over Random, ZipCal, and CoverCal, it remains unclear how much of the performance gain comes from the proposed price-aware weighting and correction mechanism, rather than from standard herding itself. In particular, a comparison with standard herding, price-weighted herding without the correction term, and corrected herding without price weighting would help clarify the individual contributions of these components. Such experiments are important for evaluating the effectiveness and novelty of the proposed algorithm.
  2. Applicability of the fixed candidate response assumption. The theoretical analysis relies on Assumption 3.2, which requires the candidate response matrices to remain fixed during coreset selection. However, practical PTQ methods such as GPTQ and AWQ may depend on calibration data when fitting quantization parameters. It would be helpful to clarify how this assumption is implemented in the experimental pipeline and to what extent the theoretical analysis applies when quantization parameters are refitted using the selected coreset.
Questions for authors

Q1. Component-level ablation studies. Could the authors provide an ablation study comparing (1) standard herding, (2) price-weighted herding without the finite-population correction, (3) corrected herding without price weighting, and (4) the full PPCC method? It would be particularly helpful to compare these variants using the existing evaluation metrics, including Kendall's tau, Top-1 configuration recovery, and downstream performance. This would clarify the individual contributions of price-aware weighting and the correction term.

Q2. Clarification of the fixed candidate response assumption. Could the authors clarify how Assumption 3.2 is satisfied in the actual PTQ experiments? Specifically, are the candidate response matrices fixed before coreset selection, and are the quantization parameters subsequently refitted using the selected calibration samples? Clarifying this procedure would help establish the connection between the theoretical analysis and the practical quantization pipeline.

3 comments

We thank Reviewer mQVJ for the concrete suggestions on the ablation and on Assumption 3.2.


W1 / Q1. component-level ablations of price weighting and finite-population correction

We ran the requested ablation with four variants: (1) standard herding, (2) price-weighted herding without the correction, (3) corrected herding without price weighting, and (4) the full PPCC method. All variants share the same moment features, candidate responses, budget, and initialization, and differ only along two axes:

  • Weighting: the price-aware γl,j\gamma_{l,j} from Thm. 3.5 vs. γl,j≡1\gamma_{l,j}\equiv1.
  • Correction: the standard herding rule arg⁡max⁡x⟨rt,y(x)⟩\arg\max_{\mathbf x}\langle\mathbf r_t,\mathbf y(\mathbf x)\rangle vs. the corrected rule in Eq. (22), which adds the term −λt2t∥y(x)∥22-\frac{\lambda_t}{2t}\|\mathbf y(\mathbf x)\|_2^2.

We first evaluate configuration ranking and decision preservation in the setting of Sec. 5.1 on Llama-3.2-1B. We report Kendall τb\tau_b and the Top-1 recovery rate, and bold marks the best result among the four variants.

Variantγ\gammaCorr.N=8N=8 (Kendall τb\tau_b)N=16N=16 (Kendall τb\tau_b)N=32N=32 (Kendall τb\tau_b)N=8N=8 (Top-1 %)N=16N=16 (Top-1 %)N=32N=32 (Top-1 %)
(1) Standard herding✗✗0.8930.8490.825759198
(2) Price-weighted herding✓✗0.8870.8430.816779398
(3) Corrected herding✗✓0.9450.9560.9599398100
(4) PPCC✓✓0.9550.9580.96296100100

We then evaluate downstream performance in bitwidth selection at 3.25 bits. We report WikiText-2 PPL and the average accuracy over PIQA, WiC, ARC-C, and HellaSwag.

VariantLlama-3.1-8B Base (PPL ↓\downarrow)Llama-3.1-8B Base (Avg. ↑\uparrow)Qwen3-8B Base (PPL ↓\downarrow)Qwen3-8B Base (Avg. ↑\uparrow)
(1) Standard herding13.0861.249.0362.97
(2) Price-weighted herding29.2761.928.9163.71
(3) Corrected herding10.5061.438.7962.36
(4) PPCC8.7161.888.6264.71

PPCC achieves the best Kendall τb\tau_b and Top-1 recovery at every budget, the lowest PPL on both models, and the highest average on Qwen3-8B, while standard herding falls behind on every metric, so the gain does not come from herding itself. The correction is the prerequisite for preserving the ranking.

Without it, the selector is drawn to samples with large feature norms, because the standard rule scores each sample linearly in y(x)\mathbf{y} ( \mathbf{x} ) and ignores the term ∥y(x)∥22\|\mathbf y(\mathbf x)\|_2^2, i.e., how much the sample itself adds to the discrepancy, so both uncorrected variants leave the price-weighted discrepancy J2J_2 two to three orders of magnitude above the corrected ones. Once this bias is removed, price weighting brings further gains, raising Top-1 recovery from 93% to 96% at N=8N=8 and from 98% to 100% at N=16N=16, and lowering the Llama-3.1-8B PPL from 10.50 to 8.71. The two components are therefore complementary, and both follow from the same objective J2J_2 (Lemma 4.1 in the main text).

We will add the ablation study to the appendix of the revision.

W2 / Q2. Is Assumption 3.2 satisfied in the actual PTQ experiments?

Yes. Assumption 3.2 holds exactly wherever our analysis is applied: the candidate responses are frozen before selection, PPCC selects the coreset with them, and the configuration ranking experiment (Sec. 5.1) computes both the pool and the coreset prices with them. The only departure is in the configuration selection experiment (Sec. 5.2), where GPTQ and AWQ are refitted on the selected coreset as in standard PTQ practice. However, the diagnostic below shows that this refitting changes the configuration ranking only slightly and leaves the top-ranked configuration unchanged. We detail both parts of the question below.

Are the candidate responses fixed before coreset selection? Yes. Before selection, we compute the GPTQ weight perturbations ΔWl(αl)\Delta\mathbf W_l(\alpha_l), the AWQ scales, and the empirical Fisher Hz,l\mathbf H_{z,l} once from the full-precision model on a fixed set of reference sequences, and freeze the resulting responses Ml(αl)\mathbf M_l(\alpha_l). PPCC uses these responses only to compute the sensitivities γl,j\gamma_{l,j}, so no quantizer is refitted while the coreset is built.

Are the quantization parameters refitted on the selected coreset? PPCC always selects the coreset with the frozen responses, and our two experiments differ only in how the coreset is used afterwards. In Sec. 5.1, the prices on the pool and on the coreset are both computed with the frozen responses, so Assumption 3.2 holds exactly and the ranking metrics measure the compression error that our theory controls. In Sec. 5.2, the coreset serves as standard calibration data, so PricePTQ estimates its price table on it and GPTQ or AWQ quantizes the selected configuration with it, as in common PTQ practice and for every selector we compare. Sec. 5.1 thus isolates the quantity our theory controls, while Sec. 5.2 tests whether the benefit carries over to the standard pipeline.

To see how far the analysis extends under refitting, write the refitted responses as MlS(αl)=Ml(αl)+ElS(αl)\mathbf M_l^{\mathcal S}(\alpha_l)=\mathbf M_l(\alpha_l)+\mathbf E_l^{\mathcal S}(\alpha_l), where ElS\mathbf E_l^{\mathcal S} is the drift, and let ΔUS\Delta^{\mathcal{S}}_U and ΔSS\Delta^{\mathcal{S}}_S be the price gaps between two configurations computed with these responses on the pool and on the coreset. By bilinearity of the price,

ΔUS(α,β)−ΔSS(α,β)=12∑l⟨Dlα,β,ΔCl(S)⟩+12∑l⟨ElS(αl)−ElS(βl),ΔCl(S)⟩\Delta^{\mathcal{S}}_{U}(\alpha,\beta)-\Delta^{\mathcal{S}}_S(\alpha,\beta)=\frac12\sum_l \left\langle\mathbf D_l^{\alpha,\beta},\Delta\mathbf C_l(\mathcal S)\right\rangle+\frac12\sum_l\left\langle\mathbf E_l^{\mathcal S}(\alpha_l)-\mathbf E_l^{\mathcal S}(\beta_l),\Delta\mathbf C_l(\mathcal S)\right\rangle

The first term is the frozen-response error that Thm. 3.5 bounds and PPCC minimizes. The second term depends only on the difference of the drifts between configurations, so it vanishes when refitting shifts all configurations in the same way.

To check this in practice, we take all 32 candidate configurations of Sec. 5.1 on Llama-3.2-1B, refit GPTQ and AWQ on the PPCC coreset and on three random coresets at N∈{8,16,32}N\in\{8,16,32\}, and evaluate every configuration under the frozen and the refitted responses on a fixed evaluation set of 32 sequences from the WikiText-2 test split, which is used neither for refitting nor for coreset selection. Below we report Kendall τb\tau_b between the rankings on this evaluation set under the frozen and the refitted responses, and how often the best configuration stays the same, averaged over the four coresets.

QuantizerN=8N=8 (Kendall τb\tau_b)N=16N=16 (Kendall τb\tau_b)N=32N=32 (Kendall τb\tau_b)N=8N=8 (same best %)N=16N=16 (same best %)N=32N=32 (same best %)
GPTQ0.9750.9830.985100100100
AWQ0.9830.9840.985100100100

The best configuration on this evaluation set never changes, and every pair whose order changes has frozen prices within 1.7% of each other. Moreover, the configuration selected on the PPCC coreset is identical under the frozen and the refitted responses at every budget and for both quantizers. Refitting therefore causes only a negligible deviation from Assumption 3.2 for configuration selection, which is consistent with the gains of PPCC in Table 1 being obtained with refitted quantizers.

We will state this protocol below Assumption 3.2 in the revision and add the drift diagnostic to the appendix.


We hope the ablation and the refitting analysis address your concerns, and we are happy to provide any additional details if needed.

Dear Reviewer mQVJ,

Thank you again for your constructive feedback. As the discussion period is drawing to a close and we have not yet heard back from you, we would like to kindly follow up with a brief summary of our response.

  • Component-level ablation (W1/Q1). Across standard herding, price-weighted herding, corrected herding, and full PPCC, PPCC performs best on ranking and Top-1 recovery at every budget, so the two components are complementary and the gain does not come from herding itself.
  • Fixed candidate responses (W2/Q2). Assumption 3.2 holds exactly in Sec. 5.1. When GPTQ and AWQ are refitted on the coreset, the ranking barely changes (Kendall τb≥0.975\tau_b \ge 0.975) and the best configuration is always the same.

We hope these updates address your concerns. If you still have any questions, we would be very happy to discuss them further.

Sincerely, Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Summary

Dear Area Chair and Reviewers,

We hope this message finds you well. We sincerely thank the Area Chair and all reviewers for their time and insightful feedback, which has significantly contributed to improving the quality of our paper. We are glad that our rebuttal led Reviewer vftk to raise the rating, and we have addressed the follow-up comments in a further reply with a revised PDF. Reviewer cFX9 has also acknowledged our clarifications and maintained a positive rating. For Reviewer mQVJ, we believe that the requested component ablation and refitting analysis in our rebuttal have well addressed both concerns.

In this letter, we summarize our main contributions and some strengths of our work as highlighted by the reviewers, as well as the main concerns raised by the reviewers and how we have addressed them in our responses.


We would like to first summarize our main contributions and some key strengths of our work as highlighted by the reviewers:

Contributions

  • Formalize calibration coreset selection for PTQ as configuration decision preservation, which requires the coreset to preserve the price gaps that determine the selected configuration.
  • Provide an analytical error bound that decomposes price-gap distortion into distribution shift, finite-pool estimation, and coreset compression, which respectively guide where to draw the calibration data from, how large the calibration pool must be, and how to select the coreset.
  • Propose PPCC, an efficient herding-style algorithm derived from the coreset compression term, whose price-weighted moment discrepancy is provably no larger than that of random selection in expectation, and which empirically outperforms existing calibration selection methods across multiple settings.

Strengths highlighted by the reviewers

  • Theoretically grounded and principled method (Reviewer mQVJ, cFX9).
  • The error bound is the strongest contribution (Reviewer cFX9).
  • Well-motivated and decision-oriented problem formulation (Reviewer mQVJ).
  • Encouraging improvements over existing calibration selection methods (Reviewer mQVJ, vftk).
  • Clear and easy to follow (Reviewer mQVJ, cFX9).

In addition, we summarize below the main concerns raised by the reviewers and the responses we have provided:

  • Theoretical Part:
    • Fixed candidate responses in Assumption 3.2 (Reviewer mQVJ): We have clarified that Assumption 3.2 holds exactly in Sec. 5.1, and shown that refitting GPTQ and AWQ on the coreset barely changes the configuration ranking (Kendall τb≥0.975\tau_b\ge0.975) and never changes the best configuration.
    • Computation of Hz,l\mathbf H_{z,l} (Reviewer vftk): We have clarified that it is the empirical Fisher of the full-precision model on 64 sequences from the C4 pool, so PPCC uses no evaluation data, and shown that the exact Fisher gives nearly the same selection results.
  • Empirical Part:
    • Component ablation (Reviewer mQVJ): We have compared PPCC with standard, price-weighted, and corrected herding, and shown that PPCC achieves the best Kendall τb\tau_b, Top-1 recovery, and PPL, so both components contribute to the gain.
    • Larger models and results on Qwen (Reviewer cFX9): We have added experiments on Qwen3-32B, where PPCC achieves the best score on all four reasoning tasks at both 2.75 and 3.00 bits.
    • Comparison with HIGGS and AMQ (Reviewer vftk): We have shown that PricePTQ with PPCC outperforms both methods on Llama-3.1-8B and Qwen3-8B, and that PPCC also lowers the PPL of AMQ from 12.20 to 11.26.
    • Runtime (Reviewer cFX9): We have clarified that PPCC takes 12–14 minutes for an 8B model and runs only once before deployment, and that PricePTQ with PPCC is still about 10 times faster than HIGGS and 80 times faster than AMQ on Llama-3.1-8B.
  • Possible extensions (Reviewer cFX9): We have discussed how PPCC extends to MoE models by treating the linear layers of each expert as separate layers, and left routing changes caused by quantization to future work.
  • Writing clarity (Reviewer vftk): We have clarified why Secs. 3.2–3.3 are needed and will make Sec. 3 more concise, and the revised PDF now describes how PricePTQ uses the coreset in Sec. 5.2 and Appx. E.2.

We would like to express our gratitude once again to the reviewers for their constructive feedback. We hope that our responses and clarifications have further improved your opinion of our work.

Sincerely, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.