acceptodds
Withdrawn from ICLR 2027

ReCaC: Residual-Guided Calibration via Activation Coverage for LLM Quantization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As a crucial technique for efficient deployment of large language models (LLMs), post-training quantization (PTQ) performs low-bit quantization for LLM relying on a pre-defined calibration dataset. Logically, the calibration dataset determines quantization parameters and so clearly influences PTQ performance. However, effective calibration construction is still underexplored. Particularly, existing methods typically focus on selecting representative and task-related samples while overlooking the complementarity among selected samples, which hardly guarantees coverage of the calibration dataset and limits the generalization ability of quantized models. Therefore, this paper proposes a Residual-guided Calibration via activation Coverage (ReCaC) method, aiming to construct an effective calibration dataset by maximizing the sample coverage in the activation space under a small fixed budget. Specifically, given a pre-defined benchmark for constructing the calibration dataset, our ReCaC uses residual-guided calibration construction (RGC) to progressively seek and pick the samples with the smallest residual score after being added to the currently selected calibration set, which effectively prioritizes complementary activation information and maximizes calibration coverage in activation space. Furthermore, ReCaC can be flexibly extended to multi-domain scenarios by joint global and domain coverage (JGDC), which selects the samples that provide complementary contributions to the overall activation space while maintaining sufficient coverage for each domain. As such, ReCaC can capture complementary activation patterns among heterogeneous domains and so enlarge the coverage of calibration dataset. Experiments are conducted by using different language models across various post-training quantization methods and settings, and results on a dozen downstream tasks demonstrate ReCaC achieves highly competitive or superior performance compared with existing state-of-the-art calibration methods in both iintra-domain, inter-domain and out-of-domain downstream tasks, particularly under the setting of lower-bit quantization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model, model quantization, calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_cRVDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReCaC, a method for constructing a small calibration set (K=128) for post-training quantization of LLMs. Each candidate sample is represented by hidden states from a few layers, concatenated and randomly projected to 384 dimensions. Selection starts from the highest-local-density sample and then greedily adds the candidate that most reduces the residual energy of the whole candidate pool after projection onto the span of the selected set (a greedy column-subset / pivoted-QR style criterion). A multi-domain extension (JGDC) adds a lambda-weighted term that prioritizes the currently least-covered domain. The method is evaluated on Llama-3.1-8B-Instruct, Gemma-2-9B-it and Qwen3.5-9B with GPTQ and AWQ (W3/W4A16) and SpinQuant and FlatQuant (W3/W4A4), against COLA and ZipCal, on 11 intra-domain and 5 held-out tasks, plus OOD tasks (Alpaca, HumanEval, MBPP) and ablations on initialization and lambda. The main claim is that maximizing activation-space coverage gives competitive or better quantized models, especially at 3-bit.

Critical strengths
  1. The selection criterion is simple, deterministic and model-agnostic, operates directly on the activations that PTQ actually calibrates on, and needs no prior about the target task distribution (unlike ACDM). This makes it easy to adopt in existing PTQ pipelines.

  2. The evaluation matrix is broad: three models, four PTQ methods, four bit settings, 16 downstream tasks including 5 held-out tasks and OOD tasks, with full per-task tables in the appendix. Gains are small but consistent in the activation-quantized settings (SpinQuant/FlatQuant W4A4) and large at GPTQ W3A16.

Critical weaknesses
  1. The headline gains rest on a single setting where a baseline collapses, and the rest of the evidence is within noise. All large numbers quoted in the text (17.75 / 20.45 points, 25.83-point drop) come from GPTQ W3A16 on Llama, where COLA reaches mean 36.23 (single-domain) / 30.53 (multi-domain), GSM8k 4.33 / 0.23 and HellaSwag 26.68, i.e. at or below chance. Everywhere else (W4A16, AWQ, all Gemma settings) differences are mostly 0.1 to 1 point from a single run with no variance, even though the random projection is itself a seeded component; the initialization ablation (Table 5) separates density-max from random by 0.19 points. There is no random-selection baseline and no standard 128-sample C4/WikiText default, which is what practitioners actually use, so it is not possible to tell whether any of the three compared methods beats the default, or whether COLA's collapse is a GPTQ configuration issue (group size, act-order and damping are not stated).

  2. Evaluation anomalies undermine the reported numbers. Quantized models exceed the dense model on MMLU-K by large margins: Gemma ReCaC GPTQ-W4A16 46.84 vs dense 33.83 (the dense value is itself implausibly low for Gemma-2-9B-it, and Gemma MMLU-K ranges from 24.87 to 46.84 across settings); Qwen3.5 GPTQ-W4A16 73.71 to 75.12 vs dense 66.36. On Llama, MMLU-M at lambda=1.5 is 32.59 while every other lambda gives 21 to 22 and dense is 24.44. Dense Gemma MMLU-K differs between Table 1 (33.83) and Table 2 (33.93). Since MMLU-K is what separates the methods on Gemma, the Gemma conclusions are not trustworthy as reported. In addition, lambda is selected on the same evaluation tasks that are reported (Table 9 covers the tasks of Table 2), and the lambda=1.5 MMLU-M outlier feeds the reported means.

Secondary: the selection rule is greedy residual-energy subset selection (pivoted QR / column subset selection / OMP-style landmark selection) on projected activations, and none of that literature (CSSP, Nystrom landmarks, kernel herding, k-center greedy coresets) is cited or compared. Fig. 1b measures coverage in exactly the space ReCaC optimizes, so ReCaC leading on coverage holds by construction, and three points do not establish that coverage causes performance. Several details needed for reproduction are missing (see questions).

Questions for authors
  1. Which splits (train/test) and how many candidates per domain form the calibration pool? Is there any overlap with the evaluation data for the intra-domain tasks?
  2. How is a 2048-token sequence reduced to a single activation vector (mean pooling, last token?), and from which hidden state?
  3. What are the GPTQ settings (group size, act-order, damping)? Does COLA's W3A16 collapse on Llama persist under a different damping value or seed, and why does it happen?
  4. Why do quantized models exceed the dense model by 7 to 13 points on MMLU-K for Gemma and Qwen? What is the dense Gemma MMLU-K evaluation setup, and why do Tables 1 and 2 report different dense values?
  5. Can you report random selection and the default C4/WikiText 128-sample set, over at least three seeds including the random projection seed?
  6. Was lambda tuned on held-out data? What are the means without the MMLU-M outlier at lambda=1.5?
  7. What is the selection cost (wall-clock and complexity in the pool size N) relative to the quantization time?
  8. Coverage is capped by r=384 with K=128. How do results change with r=128 or r=1024?
Reviewer_dbmAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

ReCaC addresses calibration set selection for post-training quantization with a fixed budget. It projects activations from selected layers, starts with a high-density candidate, and greedily adds samples that minimize residual activation energy over the candidate pool. The multi-domain extension, JGDC, combines global residual with weighted domain residuals to give less covered domains more attention. The experiments use 128 calibration sequences and evaluate GPTQ, AWQ, SpinQuant, and FlatQuant on two model families across single-domain, multi-domain, and held-out tasks. The gains are strongest in demanding 3-bit settings, while the 4-bit and AWQ results are more mixed.

Critical strengths
  1. The most convincing evidence for ReCaC comes from settings where quantization error is severe and calibration selection can make a substantial difference. With Llama and GPTQ W3A16, Table 2 reports a Mean16 score of 55.00, compared with 50.77 for ZipCal and 31.58 for COLA. The SpinQuant W3A4 result in Table 3 is also encouraging, with 53.61 against 52.25 for COLA and 47.86 for ZipCal. These are meaningful gains that support the practical value of calibration selection at low precision.

  2. I find the selection objective coherent because it accounts for what a candidate adds to the samples already selected, rather than assessing each sample in isolation. Equations 6 through 9 measure the activation energy explained after augmenting the span of the selected set. Equations 10 through 13 carry this idea into the multi-domain setting by balancing global coverage with domain-weighted residuals. This is a label-free way to encourage complementarity, without requiring deployment task labels or changes to the quantizer.

  3. The evaluation gives a useful view of where the method transfers and where it does not uniformly lead. Tables 1 through 3 cover four PTQ methods, including both weight-only and weight-activation quantization, rather than relying on a single aggregate benchmark. Table 4 separately evaluates instruction following and code generation, with ReCaC leading all four columns under SpinQuant W3A4, and Table 5 checks alternative initialization choices. Together, these experiments support competitive performance across several quantizers and task types, even though they do not show uniform superiority.

Critical weaknesses
  1. Reproducing the selector and judging its practical cost are difficult from the description in Sections 3 and 4. Section 3.2 explains that activations from selected layers are concatenated and randomly projected, but leaves unclear how token-level activations become one representation per sequence. Equation 3 describes cosine similarity yet displays an unnormalized dot product. The candidate pool size and sampling procedure, projection distribution, random seeds, and exact scoring implementation are also not specified in these sections. Since Equation 8 appears to require repeated scoring over the pool at every greedy step, wall-clock time and peak memory, with comparisons against COLA and ZipCal, would be important for assessing both reproducibility and practicality.

  2. It is hard to judge the smaller gains without knowing how much they vary across runs. ReCaC uses a random projection and a sampled candidate pool, but Tables 1 through 5 give no repeated runs, uncertainty intervals, or variation across projection and sampling seeds. The initialization comparison in Table 5 is useful, but does not answer this question about the random components. This is particularly relevant to the small Gemma aggregate differences in Tables 1 and 2 and several of the AWQ comparisons.

  3. COLA and ZipCal are relevant competitors, but the comparison is narrower than the discussion of calibration strategies would lead me to expect. Section 2 covers ACDM and generation-based calibration approaches, whereas Sections 4.1 through 4.3 compare only COLA and ZipCal. Results against these two methods do not establish ReCaC's position among the broader set of calibration construction strategies discussed in the paper. Including more of those approaches, or explaining more clearly why these two are the appropriate comparison set, would make the empirical case more persuasive.

Questions for authors
  1. How are token-level activations combined into a single representation for each sequence, and are the vectors normalized before applying Equation 3? Could you also specify the candidate pool sizes and sampling procedure, projection distribution, random seeds, and exact implementation of Equation 8?

  2. How much do the selected calibration sets and downstream scores change across projection and candidate sampling seeds? Repeated-run means and dispersion would be especially helpful for settings where the differences are below one point.

Reviewer_oQVzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReCaC, a calibration data selection method for post-training quantization (PTQ) of large language models. The authors argue that existing calibration methods mainly focus on sample representativeness or task relevance, without sufficiently considering complementarity among selected samples. To address this issue, Residual-Guided Calibration Construction (RGC) builds activation representations through random projection, uses QR decomposition to characterize the subspace spanned by the selected samples, and greedily selects new samples that minimize the overall activation reconstruction residual. Furthermore, Joint Global and Domain Coverage (JGDC) extends the method to multi-domain settings by jointly optimizing global coverage and weighted domain-level coverage to reduce representation imbalance across domains. Experiments are conducted on Llama-3.1-8B, Gemma-2-9B, and Qwen3.5-9B using GPTQ, AWQ, SpinQuant, and FlatQuant across multiple low-bit configurations and downstream tasks.

Critical strengths

Strength 1: The paper addresses complementarity among calibration samples in PTQ and introduces a clear activation-space coverage objective. RGC uses subspace reconstruction residuals to guide sample selection, while JGDC further considers coverage across different domains. The approach is independent of specific quantization algorithms, making it potentially applicable to various PTQ methods.

Strength 2: The paper presents extensive experiments across three LLMs and four PTQ methods, covering single-domain, multi-domain, and out-of-domain evaluations. ReCaC achieves substantial improvements in certain low-bit settings. For example, under single-domain GPTQ W3A16 on Llama-3.1-8B, the Mean11 accuracy increases from 36.23% with COLA to 53.98% with ReCaC, highlighting the importance of calibration data selection under aggressive quantization.

Critical weaknesses

Weakness 1: The proposed method is closely related to existing subspace selection algorithms, raising concerns about its independent novelty. Greedy Column Subset Selection [1, 2] has already explored projection reconstruction residuals and greedy selection for constructing representative data subsets. Similarly, RGC uses QR decomposition and Frobenius reconstruction residuals to progressively expand the selected subspace. Moreover, COVERCAL [3] has introduced coverage maximization into LLM PTQ calibration. The main distinction of ReCaC appears to be its focus on overall activation subspace coverage, but the paper lacks sufficient discussion and direct comparisons with these closely related methods.

Weakness 2: The connection between activation coverage and actual quantization performance is not sufficiently established. Eq. (2) reformulates calibration selection as minimizing activation reconstruction residual, but this objective does not explicitly account for quantization sensitivity, activation outliers, or the losses of different PTQ algorithms. Higher subspace coverage does not necessarily imply lower quantization error. Figure 1(b) only demonstrates the relationship between coverage and accuracy under a limited number of settings, which is insufficient to establish the general effectiveness of this surrogate objective.

Weakness 3: The experiments lack several important and straightforward sample selection baselines. The paper mainly compares ReCaC with COLA [4] and ZipCal [5], without systematically evaluating random sampling, farthest-point sampling, pivoted QR, greedy CSS, or leverage-score selection. In particular, the Random baseline in Appendix A.4 is used only to compare initialization strategies rather than as an independent random calibration method.

Weakness 4: The independent contribution of the multi-domain JGDC component is not sufficiently validated. JGDC combines global residual with weighted domain residuals, which resembles a conventional weighted multi-objective optimization strategy, without providing theoretical guarantees for domain coverage. The experiments mainly report results for the complete ReCaC framework, without adequately isolating the contributions of RGC and JGDC or systematically evaluating different domain weights, domain proportions, and calibration budgets. Consequently, it remains unclear how much JGDC improves over directly applying RGC to the combined candidate pool.

Weakness 5: The computational cost of calibration selection and sensitivity to representation construction are insufficiently analyzed. ReCaC requires extracting multi-layer activations from candidate samples and evaluating the subspace residual after adding each candidate during greedy selection. However, the paper does not report the complete selection wall-clock time, peak GPU memory, or computational scaling with candidate pool size. Furthermore, different models use different activation layers, while all representations are projected to 384 dimensions, without systematic analysis of layer selection, projection dimension, or random seeds. These omissions limit the assessment of scalability and reproducibility.

Questions for authors

Q1: Greedy CSS [1, 2] has already used projection residuals for subspace-based sample selection, while COVERCAL [3] has applied coverage maximization to PTQ calibration. Could the authors clarify the independent methodological novelty of RGC compared with these approaches and provide direct comparisons under the same calibration budget?

Q2: Eq. (2) uses activation reconstruction residual as a surrogate for quantization error. Could the authors further explain the theoretical relationship between these objectives and evaluate the correlation between activation coverage and actual quantization reconstruction error, PPL, or downstream accuracy? In particular, does this relationship remain consistent across different PTQ backends?

Q3: The main comparisons are limited to COLA [4] and ZipCal [5]. Could the authors include simpler baselines such as random sampling, farthest-point sampling, pivoted QR, greedy CSS, and leverage-score sampling to demonstrate the independent benefits of RGC over existing subspace coverage and diversity-based selection methods?

Q4: JGDC combines global residual and weighted domain residuals for multi-domain calibration selection. Could the authors directly compare RGC-only, JGDC, uniform domain sampling, and fixed domain allocation, reporting both per-domain coverage and downstream accuracy? Additionally, could they analyze the effects of different λ\lambda values, domain proportions, and calibration budgets?

Q5: Could the authors report the wall-clock time, GPU-hours, peak memory, and computational scaling of the complete calibration construction process as the candidate pool grows? Could they also analyze sensitivity to activation layer selection, the 384-dimensional random projection, and candidate pool size to assess scalability to larger models and datasets?

[1] Farahat, A. K., Elgohary, A., Ghodsi, A., and Kamel, M. S. Greedy Column Subset Selection for Large-scale Data Sets. arXiv preprint arXiv:1312.6838, 2013.

[2] Altschuler, J., Bhaskara, A., Fu, G., Mirrokni, V., Rostamizadeh, A., and Zadimoghaddam, M. Greedy Column Subset Selection: New Bounds and Distributed Algorithms. In Proceedings of the 33rd International Conference on Machine Learning (ICML), pp. 2539–2548, 2016.

[3] Shihab, I. F., Akter, S., and Sharma, A. Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channels. arXiv preprint arXiv:2604.24008, 2026.

[4] He, B., Yin, L., Zhen, H.-L., Liu, S., Wu, H., Zhang, X., Yuan, M., and Ma, C. Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2025.

[5] Monaco, F. P., Cunegatti, E., Vella, F., and Iacca, G. Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization. arXiv preprint arXiv:2603.16105, 2026.

Reviewer_D7FMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReCaC, a residual-guided calibration selection method that improves activation coverage across domains. Tests on three LLMs and four quantization methods show the largest gains with 3-bit GPTQ.

Critical strengths
  1. Selecting complementary activations makes good use of a small calibration budget, while the multi-domain extension balances domain coverage.
  2. Broad experiments show strong gains under 3-bit GPTQ and improvements on some held-out tasks, supporting better quantization robustness.
Critical weaknesses
  1. Missing random-selection baselines and repeated runs make the selection criterion’s benefit unclear, especially under AWQ.
  2. Key implementation details are missing, and inconsistent averages weaken confidence in the results. For example, Table 8’s GPTQ-W4 density scores average to 53.66, not 45.94.
Questions for authors
  1. Can you compare ReCaC with random selection over multiple seeds, reporting performance and activation coverage?
  2. Please clarify candidate-pool construction, token aggregation, data splits, and λ tuning, and correct the inconsistent averages in Tables 2, 8, and 9.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.