- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The manuscript under review considers scale optimization in PTQ (Post-Training Quantization). The authors note that while there is a surge of articles on how weights are rounded to quantization grids, the precursor question as to how the scale factors defining the quantization grids are mostly treated using simple and ad-hoc approaches. Having observed that under RTN (Round-To-Nearest) quantization the grid assignment leads to a piece-wise quadratic error function on finitely many intervals, they suggest minimizing the error function and find a closed form solution for each interval. To this end,
- Channel-wise: The authors propose their algorithm PiSO (Piece-wise Scale Optimization) that uses calibration data to find the scale factor that minimizes the error function. Considering channel-wise scale under RTN leads to a univariate optimization problem with a closed-form solution for each interval.
- Group-wise: The authors propose two methods for group-wise quantization, independent and sequential groups, where the sequential group method accounts for cross-group interactions. It essentially accounts for quantization error introduced in the previous groups.
- The authors suggest three different methods for integrating PiSo with GPTQ and Qronos. The methods are (a) Decoupling, (b) Layer-wise Interleaving, and (c) Group-wise Interleaving that introduces a tighter decoupling and results to better results.
The manuscript includes theoretical contributions, formalizing the methods and optimization aspects of the algorithm. The results are documented in Appendix C. The manuscript also includes experiments on Llama-3 and Qwen-2.5 with different sizes and different bit sizes, essentially 2, 3 and 4, where perplexity and zero-shot accuracy evaluated using Wikitext2 dataset. Aside from Appendix C where proofs are documents, the manuscripts includes 4 more appendices where details of different aspect of the work are reported.
Critical strengths
- The theoretical treatment is solid.
- The proposed method provides an optimal solution for selecting scale factor in channel-wise RTN.
- The proposed method integrates naturally with GPTQ and Qronos.
- Overall, I consider this work a practically significant advancement in scale optimization for PTQ.
Critical weaknesses
- Exact optimality only holds for channel-wise RTN.
- Group-wise extensions are heuristic.
- The manuscript does not provide any theory explaining as to why PiSO helps GPTQ/Qronos.
- The sensitivity of the method with respect to calibration data requires more discussion.
- The method is mainly a weight quantization, but it may give the impression that it is an activation quantization.
Questions for authors
-
I know that the optimization problems becomes much more difficult if I go to group-wise quantization, but any formality you can add that points at optimality of your suggested methods for group-wise quantization can be helpful.
-
Again any formal result as to why PiSO helps GPTQ/Qronos can be helpful.
-
Your proposed method depends on the calibration data. It is like a parametric optimization where we have some input parameters. In your case, it is the calibration data. This observation then gives rise to the following question: what is the optimal input for your proposed method, i.e. what is the optimal calibration data? There are already some research on optimal calibration data, e.g. ACDM (Activation Centroid Distance Metric), SelectQ, COLA etc. The following link provides the reference for SelectQ: https://www.techrxiv.org/doi/full/10.36227/techrxiv.21456291.v2
-
There are also methods which are 0-shot and calibration free that show good promise. Your method for selecting the optimal scale cannot be applied in such cases. What would be the advantage of using your proposed method over such calibration free methods? Here is the link to one of such methods: https://www.techrxiv.org/doi/full/10.36227/techrxiv.21456291.v2
-
In connection with the last item in Critical Weaknesses section, please comment on challenges in extending your method for activation quantization.
4 comments
Response 2/3
Weaknesses 3 / Question 2: Why PiSO helps GPTQ and Qronos
As the reviewer notes, the composition with error correction arises naturally, and our experiments show consistent gains with the interleaved strategies. Since jointly optimizing scales and assignments is NP-hard, PiSO's optimality does not carry over to these integrations, as stated in Section 3.3. We have, however, analyzed how well PiSO's scales suit GPTQ in a new Appendix B.1 (proofs in Appendix C.1), which is summarized in a new paragraph of Section 3.3. It is built on:
- Gap decomposition (Eq. 17). With the scale selected by PiSO, the best scale for GPTQ (for which no efficient method is known) and the joint optimum, The assignment gap is GPTQ's own suboptimality, which no scale selection can reduce, so we analyze the scale gap, and argue that PiSO's scale is a good proxy for the best GPTQ scale.
- How the scale enters GPTQ's error (Prop. B.1). GPTQ's error is a weighted sum of its squared residuals (Zhang et al., 2025, Prop. 3.2), and each residual differs from the RTN rounding error assumed by PiSO by at most the drift, the error diffused into that weight after its scale was chosen. Intuitively, weights processed early receive little drift, so they are likely to be quantized to the value that PiSO assumed, and with the usual order of decreasing these are the most important weights.
- Scale gap in expectation (Prop. B.2). In expectation, the scale gap splits into a weighting term, bounded by a factor that compares how PiSO and GPTQ weight the rounding errors, and a drift term. Under a standard high-resolution model of quantization noise, the drift only changes GPTQ's expected error through the clipped weights, typically a small fraction of them, and we bound the drift term by the drift that these weights receive. Without clipping, the drift term vanishes, and GPTQ's expected error at the scale minimizing PiSO's expected objective is within a factor of its minimum. Existing error bounds for GPTQ are restricted to this last case, as they assume an unbounded grid or no clipping (Zhang et al., 2025; Chen et al., 2026). The factor is a worst case: for instance, the weighting term does not contribute at all when the weights that share a scale are identically distributed.
- Effect of the integration strategy (Cor. B.3). Moving from to removes the mismatch between the Hessians used by PiSO and GPTQ, and moving to removes from the drift the error diffused from earlier groups.
Since the analysis relies on modeling assumptions, we also measure the scale gap directly, in a controlled experiment on a subset of the weights of three layers of Qwen2.5-1.5B (1,536 channels with 32 inputs each, in groups of 16; Appendix B.2). The reference is the best pair of scales found by an exhaustive grid search that runs GPTQ for all scale pairs of each channel. We report GPTQ's error at each strategy's scales divided by its error at the scales found by the search (median / 90th percentile over channels):
| Scales | 3 bits | 2 bits |
|---|---|---|
| absmax | 1.307 / 1.911 | 1.225 / 1.771 |
| PiSO, decoupled () | 1.041 / 1.179 | 1.038 / 1.160 |
| PiSO, layer-wise () | 1.015 / 1.097 | 1.017 / 1.090 |
| PiSO, group-wise () | 1.008 / 1.077 | 1.010 / 1.076 |
Here no preceding layers are quantized, so the Hessian mismatch of the decoupled strategy is simulated by adding a random perturbation of about 20% to the inputs used by PiSO.
The gap shrinks with each tightening of the interleaving, and with , GPTQ's error at PiSO's scales is within about 1% of its error at the best scales found by the search for the median channel. The exhaustive search limits this experiment to a small setting, but the ordering of the strategies matches our end-to-end results on full models.
This analysis covers GPTQ only, and its extension to Qronos is future work. For Qronos our evidence is empirical: with the interleaved strategies of our default configuration, PiSO improves over Qronos + absmax in 35 of 36 settings (Table 6). Interleaving matters most at 2 bits, where the RTN-quantized model on which the decoupled strategy selects its scales collapses: there, Qronos PiSO improves over Qronos + absmax by 3–7× on Llama, whereas the decoupled strategy does not (Table 1). This is consistent with the mismatch that Cor. B.3 attributes to the decoupled strategy.
Response 3/3
Weakness 4 / Question 3: Calibration data
We thank the reviewer for the reference, and acknowledge that the effect of calibration data is often overlooked in the PTQ literature (e.g., both GPTQ and Qronos calibrate on 128 sequences from a single dataset and do not study the effect of this choice). Regarding the reviewer's question:
-
While our main results use a fixed calibration set (as is common in PTQ), our original submission already included a study of calibration size and domain shift, calibrating on WikiText2, C4, or FineWeb and evaluating on WikiText2 (Figure 2 and Appendix E). RTN + PiSO reaches near-asymptotic perplexity with four sequences on all three datasets, and GPTQ PiSO calibrated on 16 sequences outperforms GPTQ + absmax calibrated on 512. The gains of PiSO over absmax hold whether or not the calibration and evaluation datasets match, and the gap between the two settings grows when PiSO is not used (Appendix E).
-
Selecting the calibration data is a complementary problem, the calibration data is an input to PiSO, which minimizes its layer-wise objective exactly for whatever data it is given. The piecewise structure and the cost of its sweep do not depend on the data, and PiSO does not require to be invertible. Thus, PiSO can directly use data selected by methods such as SelectQ. Formally, the quality of the calibration data enters our layer-wise error decomposition (Remark C.4) through the covariance-deviation term, which such selection methods aim to reduce.
We added this discussion to the revised manuscript in the calibration paragraph of Section 4, and list the study of calibration data selection as future work in Section 5.
Question 4: Zero-shot and calibration-free settings
Our experiments include two calibration-free scale rules: absmax, and OptQuant, the exact minimizer of the data-free weight reconstruction objective, which is PiSO with . For 3-bit channel-wise RTN on Llama-3 (1B/3B/8B, Table 2), they reach perplexities of 2e3/436/1e3 and 9e3/6e4/5e4, respectively, compared with 19.5/10.6/8.99 for RTN + PiSO, which reaches near-asymptotic performance with four calibration sequences (Figure 2). In this regime, neither of these calibration-free rules is competitive with a data-aware scale. This comparison covers scale selection on the same grid, and if the reviewer had a specific calibration-free method in mind, we would be happy to discuss it. Finally, when no real data is available, PiSO can also be calibrated on synthetic data, since it works with any calibration data, however it is obtained/generated.
Weakness 5 / Question 5: Activation quantization
We thank the reviewer for pointing out this possible confusion. Section 2 now states explicitly that only weights are quantized: activations are kept in high precision, and differs from only because the preceding layers have quantized weights. Following the reviewer's question, Section 5 now discusses the extension to activation quantization:
- PiSO applies when activations are quantized too: computing with quantized activations lets the cross-activation objective account for the error that they introduce.
- Static activation scales can be optimized with PiSO as well, since the activation grid assignments are also piecewise constant in each scale. This holds for any grid, so weights and activations may use different ones (e.g., W4A8).
- We leave this extension to future work, together with the joint optimization of weight and activation scales, for instance by alternating PiSO between the two, as EasyQuant does with grid search, but solving each of them exactly.
- Dynamic activation scales are computed at inference time and cannot be optimized in advance, but PiSO can compute optimal per-token scales on calibration data, which could serve as a gold standard to design cheap dynamic rules (e.g., a fixed fraction of the per-token absmax).
We thank the reviewer again for the constructive feedback, which helped us strengthen the theoretical side of the paper. We hope these additions address the reviewer's concerns and, if so, we would be grateful if the reviewer would consider raising their score.
Thank you for your responses to my comments. I will consider your responses when discussing with other reviewers and the AC.
Response 1/3
We thank the reviewer for the careful and constructive review, especially for recognizing our theoretical treatment as solid and describing our work as "a practically significant advancement in scale optimization for PTQ". The reviewer's questions on formal results for the group-wise objectives and for the integration with error correction motivated us to formalize analyses that we had only sketched so far, enriching our theoretical framework without changing our method, experimental results or conclusions. In short, the revised manuscript (changes highlighted in blue) adds:
Most of these additions are in the appendix and strengthen the theoretical analysis of the extensions of our method. We address in more detail each of the points raised by the reviewer below.
Weaknesses 1,2 / Question 1: Optimality beyond channel-wise RTN and formal results for the group-wise objectives
We agree that our exactness guarantee holds for channel-wise RTN, as stated in the abstract ("the globally optimal channel-wise scales under RTN"), and that the group-wise objectives and the integrations with error correction approximate the full problem, as discussed in Sections 3.2 and 3.3. As discussed in Section 3 (Remark C.3), optimizing the grid assignments freely, without RTN, makes the problem an NP-hard integer least-squares problem, even for a fixed scale (we expand on what can still be said for the integration with error correction in our answer to weakness 3 and question 2). For group-wise scales, there is another difficulty: exact minimization would require solving O((G∣G∣)K) closed-form subproblems instead of O(D∣G∣) (Section 3.2). This curse of dimensionality is not specific to PiSO: a joint grid search with R candidates per scale also needs RK evaluations. Tractable objectives are a contribution that can be leveraged regardless of the optimizer (grid-search, PiSO...), and PiSO minimizes them exactly. Following the reviewer's suggestion, the revision provides formal results for both group-wise objectives:
Beyond two groups, the comparison is empirical: under RTN, the sequential objective gives lower perplexity than the independent one in all 24 group-wise configurations of Table 17, by more than an order of magnitude at 2 bits. The new result is referenced from Section 3.2.