- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper revisits GPTQ-style compensation-based PTQ. GPTQ, GPTAQ and ResComp choose each column's code by RTN and only then push the error onto later columns, treating "error formation" and "error compensation" as separate stages. The authors instead write a per-step objective that jointly optimizes the current code and the ideal continuous compensation of future columns. Eliminating the compensation yields a shifted nearest-code rule with RTN as the zero-shift special case. Since the ideal shift is costly, they split the residual into a weight-state and an input-path part, precompute a per-layer matrix from the latter, and split it into diagonal and off-diagonal pieces. That gives CARE-Q's two parts: DiCR, a rounding shift with a tuned scalar coefficient, and ODPP, an extra lower-triangular propagation term added to the GPTAQ-style update. Evaluation is W2A16 weight-only on QuaRot and FlatQuant-processed models plus W2A4KV4 with FlatQuant, on six Llama-3/Qwen3 models (3B-14B), with Qwen3-32B, Llama-3-70B and raw W3 G128 in the appendix. Metrics are WikiText/C4 perplexity and 8-task zero-shot average. CARE-Q tops nearly all of Table 1 and all of Table 2, at offline cost comparable to GPTAQ/ResComp and with no change to the deployed format.
Critical strengths
- Broad, controlled gains where they matter. All methods share calibration data, seed, damping, act-order and block size, and ResComp's coefficient was further re-tuned. CARE-Q leads in nearly all of Table 1 and across Table 2. Some perplexity gaps are large: FlatQuant-PC Llama-3-8B WikiText 18.3 to 14.7 (Table 1), and W2A4KV4-PC Llama-3-8B C4 38.2 to 29.3 (Table 2). The lead persists at 70B (Table 10), across calibration budgets and calibration sets.
- Cheap, drop-in, and fairly open. CARE-Q is a small change to the GPTQ/GPTAQ loop with per-layer precomputation, run time in line with ResComp (Table 4), and no inference-time change per App. C.15. The full coefficient sensitivities are reported and the chosen pair is not the best test-perplexity cell in Table 13, consistent with selection on a calibration pool. A code repo with scripts is provided. Regardless of the theory, a rounding shift plus propagation term of this cost is something practitioners could try.
Critical weaknesses
- The theory does not support the method as written. The paper frames existing methods as rounding "independently" of compensation. In fact GPTQ rounds each column after prior errors have been compensated, which is the known-correct greedy choice. Under exact compensation the paper's own ideal shift is zero for every column after the first, so plain RTN is already optimal there. A nonzero shift only reflects that GPTAQ's compensation is approximate. The issue is imperfect compensation, not a missing "coupling". DiCR does not follow the derivation. The appendix isolates a self term of -g_t * w_t (Eq. 72) and then states DiCR "does not directly approximate" it; DiCR uses a positive tuned coefficient (e.g. +0.3 on Llama-3-8B) with only positive values gridded. As it stands DiCR reads as a tuned heuristic, and Prop. B.1 does not close the gap since its bound holds for any shift including RTN.
- The margin over GPTAQ is thin, and the nearest prior work is absent. At (rho, eps) = (0,0) CARE-Q is GPTAQ (Table 17: 18.33/22.84/51.4 vs Table 1: 18.3/22.8/51.4), so the paper is GPTAQ plus two tuned terms. Qronos (https://arxiv.org/pdf/2505.11695) tackles the same mismatched-input rounding problem and already compares against GPTAQ on Llama-3/Qwen3, but is neither cited nor compared. Table 3 ablates against GPTQ rather than GPTAQ, which flatters ODPP. Against GPTAQ, ODPP alone moves WikiText 18.33 to 17.26 (Table 17). Accuracy deltas are mostly under a point and come from a single calibration seed with no variance.
Questions for authors
- Which (rho, eps) pair is used for each model and preprocessing setting, and are all eps positive? Please also report eps < 0, in particular eps = -1 on the working column, which is the self term of Eq. 72. If the sign that works is opposite to the derived one, how should Props. 3.1-3.2 be read as the basis for DiCR?
- Under exact least-squares compensation of the Eq. 7 objective, s*_t = 0 for all t >= 2 (Qronos Theorem 3.1). If so, is DiCR really correcting GPTAQ's deferred handling of column t's own input-path mismatch? Please report the ideal shift norm and its correlation with the DiCR surrogate on a few real layers.
- Please re-run the Table 3 ablation against GPTAQ, and report mean and std over calibration seeds for Tables 1-2. Several accuracy gaps are below a point.
- Please add Qronos under your settings (W2A16 with QuaRot/FlatQuant, PC and G128; W2A4KV4), plus the baseline: GPTQ runs on the least-squares target. If CARE-Q still wins, the case is much stronger.
- The (rho, eps) pool uses 128 WikiText + 128 C4 samples. Are these drawn from training splits and disjoint from eval texts? For W2A4KV4, calibration uses WikiText, the same corpus as evaluation. Please clarify the overlap.
6 comments
R3: Theoretical Basis and Coefficient Sign of DiCR (W1 & Q1)
We thank the reviewer for carefully examining the sign in Eq. (72) and its connection to DiCR. As established in Appendix A.3, is the current-column self term of the input-path contribution to the ideal shift, not the complete compensation-induced shift in Proposition 3.1. The negative sign of this isolated self term does not determine the direction of the complete ideal shift, nor does it prescribe the sign of DiCR's surrogate coefficient.
Proposition 3.1 establishes the compensation-conditioned code-selection criterion (Eq. 9). Section 3.3 derives the candidate-independent representation from the input-path discrepancy. Proposition 3.2 identifies its diagonal signal , while complementary off-diagonal information supports ODPP's subsequent compensation. Appendix A.3 further establishes the structural connection between and the ideal shift. DiCR exploits this analytically derived signal through the efficient surrogate , where the signed coefficient controls its direction and magnitude. Thus, DiCR and ODPP draw on complementary components of the same compensation representation to jointly address error formation and subsequent compensation. The full-shift diagnostics in R2 provide further empirical evidence for this connection.
We directly tested the reviewer's requested with the current working column, using the isolated self-term form . We also compared with the same fixed full-precision reference column, isolating the effect of the coefficient sign. All four settings use Llama3-8B under FlatQuant W2A4KV4-PC, with fixed and otherwise identical calibration, quantization, and evaluation protocols:
| DiCR setting | Wiki PPL ↓ | C4 PPL ↓ |
|---|---|---|
| , working column | 21.53 | 84.34 |
| , reference column | 14.54 | 45.86 |
| (ODPP only) | 13.38 | 33.37 |
| , reference column | 12.52 | 29.30 |
Using this isolated self-term form markedly worsens perplexity in this setting. Under the matched reference and coefficient magnitude, yields lower PPL than on both datasets and further improves upon the ODPP-only baseline. These controlled results directly address the sign concern and demonstrate the benefit of the structurally informed DiCR surrogate relative to the isolated self-term form in the tested configuration.
Appendix C.1 specifies one pair per model and preprocessing setting, consistently reused across the corresponding bit widths and PC/G128 quantization granularities without configuration-specific retuning. The selected values include both positive and negative coefficients; model-wise assignments are provided in the released experimental scripts (https://anonymous.4open.science/r/CARE-Q/). Appendix C.8 (Tables 13–16) further shows a favorable region of moderate coefficient choices, rather than gains confined to a single narrowly tuned pair.
Proposition B.1 characterizes the margin-based stability of surrogate-guided candidate decisions, including RTN when the surrogate shift is set to zero. Beyond this general result, CARE-Q derives a shared compensation representation and uses its complementary diagonal and off-diagonal information to guide DiCR and ODPP. The controlled sign tests above and the component ablations in R5 substantiate the effectiveness of this unified design. Together, the theoretical analysis and empirical evidence reinforce CARE-Q's central contribution: coupling error formation and subsequent compensation within a unified quantization error lifecycle.
R4: Comparison with Qronos and LS-GPTQ (W2, Q4)
We thank the reviewer for highlighting Qronos (https://arxiv.org/pdf/2505.11695) and suggesting LS-GPTQ (GPTQ on the least-squares target). Our submission already compares recent compensation-based methods, including GPTAQ (ICML 2025) and ResComp (ICLR 2026); we now add both requested baselines. For each setting, all three quantization methods use the same QuaRot/FlatQuant-preprocessed weights (or original weights for raw W3), calibration samples/seed, quantization grids/clipping, act-order, GPTQ MSE, damping (percdamp=0.01), block/group sizes, and evaluation protocol. Appendix C.1 details the shared setup. Each entry: Wiki/C4 PPL (↓)/eight-task Avg (↑).
W2A16KV16:
| Config | Method | Llama3-8B | Llama3.1-8B | Llama3.2-3B | Qwen3-4B | Qwen3-8B | Qwen3-14B |
|---|---|---|---|---|---|---|---|
| QuaRot-PC | LS-GPTQ | 22.4/26.6/49.1 | 24.9/32.7/50.4 | 42.5/45.5/43.5 | 43.4/46.4/42.0 | 20.4/27.3/51.7 | 15.2/20.8/58.6 |
| Qronos | 22.5/26.8/48.8 | 24.7/32.4/51.1 | 39.6/45.5/44.6 | 42.6/46.3/42.1 | 20.6/27.2/52.0 | 15.6/20.9/59.0 | |
| CARE-Q | 21.3/25.2/49.7 | 18.5/24.7/51.8 | 39.0/42.1/44.5 | 40.3/47.8/42.7 | 19.9/26.6/52.8 | 14.2/20.3/60.8 | |
| QuaRot-G128 | LS-GPTQ | 17.4/22.6/51.0 | 21.1/28.7/51.5 | 35.6/39.5/43.6 | 34.2/43.1/43.8 | 16.9/23.1/54.1 | 13.7/19.2/60.7 |
| Qronos | 18.2/22.7/52.7 | 21.2/28.6/52.0 | 36.0/39.3/43.4 | 32.4/42.2/44.0 | 16.3/23.2/54.3 | 13.6/19.3/60.8 | |
| CARE-Q | 17.0/21.5/51.9 | 17.3/23.8/52.6 | 33.0/36.6/44.9 | 30.5/39.0/43.8 | 17.4/24.1/54.9 | 13.1/19.0/62.3 | |
| FlatQuant-PC | LS-GPTQ | 20.9/23.8/51.6 | 24.5/30.7/53.9 | 40.7/41.4/48.6 | 30.6/32.6/50.5 | 20.7/24.7/55.7 | 16.1/20.2/61.3 |
| Qronos | 21.3/23.9/51.9 | 25.2/30.7/53.6 | 38.2/41.4/48.7 | 31.3/32.4/51.1 | 20.4/24.9/55.1 | 16.2/20.3/61.1 | |
| CARE-Q | 14.7/19.9/52.8 | 16.4/23.3/54.8 | 26.6/32.2/47.6 | 26.6/31.4/52.2 | 16.7/23.4/57.2 | 14.1/19.2/62.7 | |
| FlatQuant-G128 | LS-GPTQ | 17.8/22.0/53.0 | 20.6/27.6/53.3 | 33.3/36.6/47.7 | 24.5/29.2/52.2 | 17.0/22.4/56.1 | 13.6/18.8/62.2 |
| Qronos | 18.1/21.8/52.9 | 21.0/27.7/53.1 | 33.3/36.9/47.6 | 24.8/29.1/51.5 | 17.0/22.3/56.2 | 13.6/18.8/61.3 | |
| CARE-Q | 13.8/19.5/54.0 | 15.4/22.3/54.9 | 25.8/30.0/48.3 | 23.9/30.0/52.5 | 15.3/22.7/57.5 | 12.7/18.4/63.5 |
W2A4KV4:
| Config | Method | Llama3-8B | Llama3.1-8B | Llama3.2-3B | Qwen3-4B | Qwen3-8B | Qwen3-14B |
|---|---|---|---|---|---|---|---|
| FlatQuant-PC | LS-GPTQ | 14.1/55.4/47.4 | 17.2/71.6/47.8 | 31.7/112.2/42.5 | 21.7/64.1/46.4 | 15.3/47.0/50.7 | 13.0/38.8/55.4 |
| Qronos | 14.2/58.3/47.2 | 17.4/70.9/48.3 | 31.1/112.0/43.0 | 21.7/65.3/46.6 | 15.4/49.5/49.4 | 13.0/39.3/56.2 | |
| CARE-Q | 12.5/29.3/49.4 | 14.1/31.9/51.1 | 22.8/43.6/44.8 | 21.8/41.2/48.8 | 14.7/27.3/54.9 | 12.3/23.1/59.8 | |
| FlatQuant-G128 | LS-GPTQ | 13.4/53.2/47.7 | 16.2/64.4/48.9 | 28.3/94.1/43.8 | 20.0/54.9/47.4 | 14.4/40.6/51.7 | 12.3/32.7/56.9 |
| Qronos | 13.5/51.5/47.7 | 16.1/62.3/49.8 | 28.0/91.8/44.1 | 20.0/55.3/47.7 | 14.8/40.9/50.8 | 12.4/32.8/56.4 | |
| CARE-Q | 12.1/26.5/49.6 | 13.5/28.5/51.8 | 22.5/41.0/45.0 | 20.2/37.1/49.6 | 14.2/26.2/54.7 | 11.9/21.6/61.2 |
Raw W3A16KV16-G128:
| Method | Llama3-8B | Llama3.1-8B | Llama3.2-3B | Qwen3-4B | Qwen3-8B | Qwen3-14B |
|---|---|---|---|---|---|---|
| FP16 | 6.1/9.4/67.1 | 7.2/11.4/68.2 | 11.0/16.5/60.9 | 13.7/19.9/64.6 | 9.7/15.4/67.2 | 8.6/13.8/70.3 |
| LS-GPTQ | 8.1/12.3/62.6 | 9.1/13.9/64.7 | 21.7/20.3/56.7 | 15.6/21.7/59.8 | 10.9/16.7/64.2 | 9.7/14.9/68.3 |
| Qronos | 8.1/12.8/62.0 | 9.2/14.3/64.8 | 14.3/20.8/58.3 | 15.5/21.7/61.5 | 10.9/16.7/64.8 | 9.6/14.9/68.1 |
| CARE-Q | 7.9/12.6/63.9 | 8.8/13.8/65.4 | 13.5/19.7/58.8 | 14.3/21.6/60.7 | 11.1/17.0/65.4 | 9.4/14.7/69.0 |
Under W2A16KV16, CARE-Q achieves strong overall results against both added baselines, with particularly pronounced gains in several FlatQuant configurations. Under W2A4KV4, CARE-Q achieves lower C4 PPL and higher eight-task Avg than both baselines in all 12 model/granularity configurations, reducing C4 PPL by 32.4%–61.1% and improving Avg by 0.9–4.3 percentage points against the stronger baseline for each metric. Under raw W3A16KV16-G128, all three quantization methods operate on identical untransformed weights, and CARE-Q also leads on most reported metrics.
Together, these matched comparisons provide strong empirical evidence for CARE-Q's central insight: quantization error formation and subsequent compensation should be addressed as coupled stages of a unified error lifecycle. CARE-Q realizes this principle through a shared second-order input-path compensation representation, whose complementary diagonal and off-diagonal information guide DiCR and ODPP, respectively. We will cite Qronos and incorporate these comparisons in the revision.
We sincerely thank the reviewer for the thoughtful and insightful feedback. We appreciate the recognition of CARE-Q's consistent accuracy improvements, practical efficiency, and careful experimental comparisons. The reviewer's valuable insights have further strengthened our work by deepening our theoretical understanding, clarifying the rationale behind our method, and making our empirical evaluation more comprehensive and convincing. We address these concerns below.
R1: Clarification of GPTQ's Compensation Mechanism (W1)
We appreciate the reviewer's observation and would like to clarify this distinction. As described in Section 3.1 (Eqs. 3–5), GPTQ performs sequential second-order error compensation, and the current working column already incorporates updates from previously quantized columns. Our point concerns current code selection rather than prior compensation. GPTQ still selects the current code using RTN (Eq. 10), which corresponds to the zero-shift case of the ideal selection criterion. However, our additional diagnostics across 32 linear modules of Llama3-8B during CARE-Q quantization reveal nonzero ideal shifts at sampled steps beyond the first processed column, with detailed results for six representative modules (see R2). This supports the practical relevance of considering subsequent compensation when selecting the current code.
R2: Exact LS Compensation and Practical Ideal Shifts (W1 & Q2)
We appreciate the reviewer's reference to Qronos Theorem 3.1. We agree that, under exact least-squares (LS) reoptimization of the remaining weights, the ideal shift vanishes beyond the first processed column. However, CARE-Q's practical sequential updates do not enforce the same LS optimality condition. Consequently, the ideal shift can remain nonzero during practical CARE-Q quantization.
To examine this distinction, we conducted additional diagnostics on Llama3-8B under FlatQuant W2A4KV4-PC. Nonzero sampled ideal shifts were observed in all 32 examined linear modules (O and Down projections from blocks 0, 2, ..., 30). For six representative modules spanning shallow, middle, and deep layers (blocks 0, 16, and 30), we examined 390 sampled column states beyond the first processed column. We report grid-normalized ideal-shift magnitude (RMS/grid) and Pearson correlation with the DiCR surrogate:
Across the six representative modules, the ideal-shift RMS/grid ranges from 0.3915 to 0.7722, with consistently positive Pearson correlations (+0.210 to +0.504) with the DiCR surrogate. These results reveal nonzero compensation-conditioned shifts during practical CARE-Q quantization and their empirical relationship with the DiCR signal. Appendix A.3 establishes the structural origin of this signal in the input-path contribution (see R3). Our matched Qronos and LS-GPTQ comparisons in R4 provide complementary final-model evidence, particularly the substantial C4 PPL improvements under W2A4KV4.
Qronos Theorem 3.1 establishes the zero-shift result under exact LS reoptimization. Our findings show that this result does not directly extend to CARE-Q's practical working states. These findings also reinforce CARE-Q's central perspective: error formation and subsequent compensation are coupled stages of a unified quantization error lifecycle.