- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper identifies a mismatch in compensation-based PTQ, where codebooks are fitted to initial weights rather than the compensated weights actually being quantized. It proposes 2-Pass Trajectory Calibration (2PTC), which refits codebooks using the first-pass quantization trajectory, supported by a layer-wise reconstruction error analysis. Experiments show some improvements, although the benefits are sensitive to the initial codebook design.
Critical strengths
- The paper identifies a meaningful mismatch between codebook fitting and the actual rounding trajectory in compensation-based PTQ. The proposed two-pass approach is easy to integrate into existing quantization pipelines.
- The paper provides an extensive empirical evaluation on six LLMs, two quantization engines, and three bit-widths across three random seeds. The fitting-target ablation convincingly isolates the benefits of trajectory-based codebook fitting from those of QEP correction. The authors also transparently report negative results with group-wise codebooks, uniform grids, and LeanQuant-style configurations, strengthening the credibility of the experimental findings.
Critical weaknesses
- I believe there is a critical contradictory here. The second-pass codebook is fitted to a trajectory generated using the first-pass codebook, but changing the codebook also changes the rounding decisions and compensation trajectory. Consequently, improving the frozen-trajectory objective does not guarantee improving the actual reconstruction loss. This mismatch weakens the theoretical motivation, and I believe there will be a better solution than the proposed 2PTC.
- The improvement largely comes from imperfections in the original codebook. If the original codebook is more advanced, 2PTC can even have a negative effect (as shown by the comparison with LeanQuant), which raises concerns about the method's robustness and general applicability. The proposed 2PTC provides a simple and practical alternative.
Questions for authors
- Can the authors reconsider this inconsistency and try to develop a more optimal solution using techniques such as a lookahead optimization objective or compensation-aware codebook generation?
- Can the authors provide a theoretical analysis of what benefits or drawbacks 2PTC could bring if the original codebook is theoretically optimal?
- Can the authors add experiments comparing 2PTC with more advanced codebook-based PTQ methods, including QTIP, GPTVQ, and AQLM?
5 comments
Continuation of the reponse
Q1 – Lookahead or compensation-aware codebook. Our per-layer acceptance test already functions as a one-step lookahead on the true post-compensation loss. Designing a fully compensation-aware codebook generator is an exciting direction. We will explicitly discuss both approaches as promising directions for future work in the revision.
Q2 – Theoretically optimal initial codebook. Here it is helpful to distinguish between two notions of optimality:
- Open-loop optimal: Optimal for the unperturbed full-precision weights .
- Closed-loop optimal: Optimal for the actual compensated trajectory that the quantizer rounds.
Any codebook fitted prior to quantization is at best open-loop optimal, as it cannot anticipate the perturbations introduced by error feedback. 2PTC bridges this gap by moving the codebook toward the closed-loop optimum on the observed trajectory (whose distribution shifts very little between passes, as confirmed by our frozen-trajectory accuracy above). Conversely, if an initial codebook were already closed-loop optimal, the refit would leave it essentially unchanged to first order, and the acceptance test guarantees that no layer can degrade.
Q3 – Comparison with other methods. QTIP and AQLM belong to distinct quantization paradigms. Neither uses the GPTQ-style sequential error feedback that 2PTC targets. We will clearly position both works in the revised Related Work section.
GPTVQ is the closest baseline to our approach among the three methods mentioned. We integrated its official implementation into our pipeline. Both methods therefore share the same calibration data, models, evaluation protocol and metrics, and we compare them in terms of effective bits per weight.
On five models, 1D GPTVQ uses 0.08–0.17 more bits per weight than our per-channel codebooks (e.g., 2.19 vs. 2.01 at 2 bits) and requires roughly more quantization time ( vs. on 7B-scale models). In terms of perplexity, 2PTC with QEP outperforms 1D GPTVQ at 2 bits (12/15 model–seed pairs) and is comparable at 3–4 bits. The same holds for its 2D variant. We emphasize that GPTVQ's performance comes at a higher memory and compute cost.
| Bits | Method | Bits/weight | PPL WikiText-2 (mean) | GPTVQ better than 2PTC + QEP | GPTVQ better than 2PTC w/o QEP | Quant. time (7B-scale) |
|---|---|---|---|---|---|---|
| 2 | 1D GPTVQ | 2.19 | 20.57 | 3/15 | 8/15 | |
| 2 | 2PTC + QEP | 2.01 | 16.90 | -- | -- | |
| 3 | 1D GPTVQ | 3.16 | 6.95 | 6/14 | 12/14 | |
| 3 | 2PTC + QEP | 3.03 | 6.73 | -- | -- | |
| 4 | 1D GPTVQ | 4.14 | 6.21 | 6/11 | 9/11 | |
| 4 | 2PTC + QEP | 4.06 | 5.94 | -- | -- |
Importantly, the two approaches are orthogonal. GPTVQ initializes each codebook when its column group is reached, but within each group, the codebook is still fitted before compensating for the remaining columns. Applying 2PTC on top of GPTVQ's codebooks is therefore a natural extension.
Dear Reviewer Srxe,
Thank you again for your efforts and interest in our paper. As the deadline for the author-reviewer discussion phase is approaching, we would like to ask if our responses have effectively addressed the concerns you raised. Should any remain unresolved, please let us know, and we will promptly follow up.
Response to Reviewer Srxe
W1 – Frozen-trajectory objective vs. the actual loss. We agree that updating the codebook also alters the rounding decisions and, consequently, the quantization trajectory. While improving the frozen-trajectory objective does not guarantee a reduction in the actual loss a priori, Eq. 7 bounds this discrepancy in terms of trajectory displacement, and we now directly quantify its empirical behavior through a new set of experiments.
For every layer, we computed the exact objective minimized by GPTQ/LDLQ:
L(G)=∥(W∗−W(G))X∥F2=tr((W∗−W(G))H(W∗−W(G))⊤).
We evaluated it for both the first-pass codebook (G1) and the codebook refitted by 2PTC on the first-pass trajectory (G2). The experiment covers 108 full quantization runs: 6 models × 3 seeds × 3 bit-widths × 2 engines (GPTQ and QuIP), for a total of 31,248 layers. G2 achieves strictly lower reconstruction error than G1 in 99.4% of these layers. The proportion is consistently high in every setting, including 4 bits (99.2% with GPTQ and 99.98% with QuIP). In the few layers where G1 is better, the margin is small. Although the trajectory does change between passes, its effect on the loss is minor, and refitting the codebook on the first-pass trajectory almost always brings the quantized matrix closer to the full-precision one.
Here we attach a table with the differences between how much G2 reduces loss with respect to G1:
To remove any remaining risk of degradation, we now add a per-layer acceptance test to 2PTC that keeps the first-pass codebook whenever L(G1)<L(G2). Both losses are by-products of the two passes, so the test has negligible cost. It acts as a one-step lookahead on the true objective and guarantees that no layer ends up worse than with a single pass.
Finally, we also studied whether further iterations improve the performance of the proposed approach. In the three-pass experiment of Appendix D.3, the codebook is refitted again on the second-pass trajectory and a third quantization pass is run, using the same protocol and configurations as 2PTC. The third pass improves over 2PTC in only 4 of 18 cases at 2 bits and is 1.8% worse on average. This indicates that the main correction happens with the first refit, and that two passes are sufficient.
W2 – Dependence on the initial codebook. We agree that the benefit of 2PTC depends on the initial codebook. With the LeanQuant codebook as published, the second pass does not help: it degrades results at 2 bits and is neutral at 3–4 bits. Specifically, this occurs because LeanQuant's codebook is optimized for a different criterion: its k-means step weights columns by (1/Ukk)4=dk2, whereas the compensation objective weights columns by dk. Refitting with dk2 on the trajectory therefore optimizes a mismatched objective.
This is visible per layer, and the acceptance test handles it. With the LeanQuant codebook, the second pass increases the layer loss. In exactly these cases, our acceptance test (explained above) retains the first-pass codebook.
Nevertheless, our weighting is the stronger baseline. Our Hessian-diagonal weighting already outperforms LeanQuant with a single pass at 3–4 bits (17/17 model–seed pairs). Also, at 2 bits, 2PTC on our simple initialization outperforms one-pass LeanQuant in 18/18 pairs (median −14.5% perplexity). We thus argue that the gain is not only a correction of a weak initial codebook. It reflects the open-loop vs. closed-loop mismatch, which is largest at low bit-widths.