- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
G² PTQ extends GPTQ-style weight compensation by incorporating both first- and second-order information under block-wise optimization objectives, using MSE for intermediate blocks and KL divergence for the final block. It refreshes gradient and Hessian estimates before quantizing each block, introduces trust-region scaling to stabilize gradient compensation, and derives efficient implementations. Experiments demonstrate improved perplexity and downstream accuracy over existing PTQ baselines.
Critical strengths
- The paper integrates first-order gradient compensation with second-order information under block-wise supervision, incorporating refreshed gradient and Hessian estimates and trust-region scaling. The efficient implementation makes this more comprehensive optimization framework practical for large-scale PTQ.
- The paper evaluates diverse LLM architectures, including large-scale MoE models, demonstrating improvements in downstream accuracy and distribution alignment. Detailed ablation studies validate the contributions of individual components, while runtime and memory measurements provide useful evidence of practical feasibility.
Critical weaknesses
- The key components of G² PTQ largely build on existing techniques. First-order gradient compensation was introduced by FOEM (AAAI 2026), and block-wise optimization objectives have been extensively studied in prior work such as BRECQ (ICLR 2021) and aespa (Towards next-level post-training quantization of hyper-scale transformers, NeurIPS 2024). The ablation study shows that most of the improvement over GPTQ comes from block-wise alignment, while the additional gains from other components are relatively modest. Moreover, the lack of a direct comparison with FOEM makes it difficult to assess the incremental contribution of the proposed method.
- G$^2$PTQ requires substantially more calibration memory and computation due to additional backward passes and Hessian estimation. For an 8B model, calibration memory increases from 4.62 GB to 14.57 GB, while runtime increases from 0.43 to 1.49 hours. These overheads are likely to become more pronounced for larger models. Without more convincing evidence of improvements under matched computational budgets, it remains unclear whether the additional complexity is justified by the performance gains.
Questions for authors
- Can the authors rewrite the key contribution of G$^2$PTQ over prior work? Which components represent genuinely new contributions rather than combinations or extensions of existing techniques?
- Can the authors provide a direct comparison with FOEM under matched experimental settings? In particular, how much improvement comes from the proposed components beyond the already established block-wise optimization objective?
- Theorem 3 derives a per-coordinate scaling factor, whereas Equation (56) averages these factors within each row. Does the row-wise averaged factor still satisfy the loss-change bound in Equation (50)?
9 comments
Rebuttal by Authors (Part 2)
Q2. (Reviewer's W2 and Q1) The additional calibration cost lacks sufficient generalization evidence... Generative evaluations cover only one 125B MoE model at 4 bits, leaving capability preservation under the main low-bit dense-model settings unclear.
A2 (continued from Part 1).
Comparison with FOEM.
-
Ablation of exact gradient compensation (Appendix E.1, lines 1324–1335). As shown in Table 8, replacing G²PTQ's exact gradient compensation with FOEM's approximate gradient compensation noticeably degrades both distribution alignment and downstream task performance, resulting in a 2.09% decrease in QA accuracy. These results underscore the importance of G²PTQ's exact gradient compensation.
-
Direct comparison with FOEM. We previously reproduced FOEM following the official code, but found that it failed to outperform GPTQ. This finding has also been reported in recent work [1] reproducing FOEM. We therefore did not include it as a baseline and instead carefully selected other strong baselines, such as GPTAQ and GuidedQuant. As shown below, G²PTQ significantly outperforms FOEM across different models and bit widths.
| Model | Method | W2A16 KL ↓ | W2A16 PPL ↓ | W2A16 QA ↑ | W3A16 KL ↓ | W3A16 PPL ↓ | W3A16 QA ↑ | W4A16 KL ↓ | W4A16 PPL ↓ | W4A16 QA ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3 0.6B | GPTQ | 4.73e+00 | 1.58e+03 | 29.92 | 1.50e+00 | 55.60 | 33.09 | 4.58e-01 | 18.33 | 41.57 |
| FOEM | 9.52e+00 | 1.55e+05 | 29.11 | 1.89e+00 | 128.75 | 33.27 | 5.05e-01 | 32.89 | 44.19 | |
| G²PTQ | 1.55e+00 | 64.99 | 31.72 | 3.87e-01 | 16.92 | 39.22 | 1.17e-01 | 12.63 | 45.85 | |
| G²PTQ* | 1.26e+00 | 53.25 | 32.93 | 3.33e-01 | 16.12 | 39.83 | 1.05e-01 | 12.52 | 45.80 | |
| Qwen3.5 2B | GPTQ | 1.42e+00 | 39.95 | 32.18 | 2.28e-01 | 8.70 | 51.29 | 6.36e-02 | 7.00 | 56.57 |
| FOEM | 2.99e+00 | 248.77 | 31.07 | 3.87e-01 | 18.61 | 51.06 | 9.53e-02 | 13.43 | 57.65 | |
| G²PTQ | 1.05e+00 | 25.06 | 36.09 | 1.99e-01 | 8.42 | 51.87 | 5.83e-02 | 6.98 | 57.22 | |
| G²PTQ* | 8.42e-01 | 19.99 | 37.75 | 1.77e-01 | 8.20 | 53.53 | 5.37e-02 | 6.98 | 57.97 | |
| LLaMA3 8B | GPTQ | 1.15e+00 | 18.11 | 38.61 | 1.69e-01 | 4.99 | 64.63 | 4.99e-02 | 4.33 | 68.44 |
| FOEM | 3.62e+00 | 235.18 | 32.22 | 3.32e-01 | 8.61 | 62.38 | 9.30e-02 | 6.73 | 68.03 | |
| G²PTQ | 8.65e-01 | 14.76 | 47.47 | 1.29e-01 | 4.78 | 66.07 | 3.94e-02 | 4.28 | 69.06 | |
| G²PTQ* | 6.12e-01 | 9.18 | 51.16 | 1.22e-01 | 4.75 | 66.56 | 3.75e-02 | 4.27 | 69.41 |
[1] Lee, Gunjun, et al. "SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization." arXiv preprint arXiv:2608.15567 (2026).
Rebuttal by Authors (Part 3)
Q3. (Reviewer's W2) GPTQ requires substantially more calibration memory and computation due to additional backward passes and Hessian estimation... These overheads are likely to become more pronounced for larger models.
A3. The quantization overhead is an acceptable, one-time cost. Once the model is quantized, the higher accuracy benefits all downstream applications, making the upfront cost well worth it.
Although G²PTQ incurs additional calibration costs compared with GPTQ, a highly efficient algorithm for LLM quantization, the overhead remains within a reasonable budget. As shown in Table 5, G²PTQ can quantize an 8B model in 1.49 hours on a single Hopper GPU. The G²PTQ* variant takes an additional 0.75 hours.
Moreover, G²PTQ can scale to 100B+ models. During the rebuttal period, we substantially improved the efficiency of G²PTQ by optimizing multi-GPU communication and computation workflows, building on the efficiency optimizations described in Appendix C.3. The table below reports the quantization overhead of the optimized G²PTQ* implementation, supplementing Appendix E.2, lines 1338–1346.
- Memory cost. The maximum GPU memory required by G²PTQ is bounded by backpropagating a single Transformer block, not the entire model. As shown below, even a massive 753B model can be quantized on 140GB GPUs.
- Time cost. The time required remains well within practical limits for an offline procedure. Specifically, a 753B model can be quantized in 6 hours using only 32 Hopper GPUs, or in less than a day using only 8 Hopper GPUs.
- Calibration dataset size. As discussed in Appendix E.1, lines 1290–1294, G²PTQ is highly sample-efficient and requires as few as 128 calibration samples to outperform GPTQ.
(GLM-5.2, 753B)
| #GPUs | Memory (GPTQ / G²PTQ*) | Time (GPTQ / G²PTQ*) |
|---|---|---|
| 8 | 42.03 / 97.38 GiB | 9.69 / 17.51 hours |
| 32 | 42.03 / 69.46 GiB | 4.70 / 6.00 hours |
(As noted in Appendix E.2, lines 1338–1346, we quantize MoE models using 8K calibration samples with 2K length. The table reports per-GPU memory usage and time. For the 753B model, the overhead is estimated from quantizing the first 10 of its 78 Transformer blocks.)
Rebuttal by Authors (Part 1)
Q1. (Reviewer's W1 and Q1) Can the authors rewrite the key contribution of GPTQ over prior work? Which components represent genuinely new contributions rather than combinations or extensions of existing techniques?
A1. G²PTQ is not merely a combination of existing techniques. To the best of our knowledge, it is the first unified PTQ framework that integrates refreshed exact first- and second-order compensation under a global block-wise objective.
Specifically, our novel technical contributions include:
Section 2 discusses GPTQ variants in detail. Below, we further summarize the key differences between G²PTQ and the existing work mentioned by the reviewer:
Q2. (Reviewer's W1 and Q2) Can the authors provide a direct comparison with FOEM under matched experimental settings? In particular, how much improvement comes from the proposed components beyond the already established block-wise optimization objective?
A2. Block-wise optimization is one of the core contributions of G²PTQ. As detailed in A1, G²PTQ centers on block-wise optimization within the GPTQ framework, using refreshed, exact first- and second-order compensation. To the best of our knowledge, block-wise optimization within the GPTQ framework has not previously been explored.
The other components also matter. We provide additional ablations to complement Table 4. Although block-wise second-order compensation alone does not necessarily improve upon the layer-wise GPTQ baseline because of stale information, incorporating refreshed information consistently improves quantization accuracy. Moreover, as noted in Section 5.3, lines 455–457, “this refreshing mechanism serves as a strict prerequisite for gradient compensation.” Exact gradient compensation, together with KL alignment in the last block, yields further significant improvements—for example, a 6.42% increase in QA accuracy for 2-bit quantization of Qwen3-8B.
(Qwen3-0.6B, W3A16)
(Qwen3-8B, W2A16)