acceptodds
Under review as a conference paper at ICLR 2027

PiSO: Scale Optimization in Post-Training Quantization

Abstract

Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations. While much work has focused on how weights are rounded onto the quantization grid, the scaling factor that defines the grid is typically chosen with simple, data-free heuristics. We observe that, under round-to-nearest (RTN) quantization, the grid assignments are piecewise constant in the scale, making the data-aware layer-output error piecewise quadratic over finitely many intervals, each with a closed-form minimizer. Building on this insight, we present PiSO (Piecewise Scale Optimization), an algorithm that leverages calibration data to efficiently compute the globally optimal channel-wise scales under RTN. We extend PiSO to group-wise quantization via tractable objectives. We also propose strategies for integrating it with error-correction methods such as GPTQ and Qronos, which greedily adjust weight assignments to compensate for the rounding errors of previously quantized weights. Experiments on Llama and Qwen models across multiple sizes and bit-widths show consistent improvements in perplexity and zero-shot accuracy, both standalone and combined with error correction, with gains increasing as the bit-width narrows. For instance, on Llama-3, PiSO reduces the 2-bit perplexity of Qronos by up to 7x, and at 3 bits, RTN with PiSO scales alone outperforms GPTQ with absmax scales.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Post Training Quantization, Large Language Models, Foundation Models, Efficient Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JVLEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript under review considers scale optimization in PTQ (Post-Training Quantization). The authors note that while there is a surge of articles on how weights are rounded to quantization grids, the precursor question as to how the scale factors defining the quantization grids are mostly treated using simple and ad-hoc approaches. Having observed that under RTN (Round-To-Nearest) quantization the grid assignment leads to a piece-wise quadratic error function on finitely many intervals, they suggest minimizing the error function and find a closed form solution for each interval. To this end,

  1. Channel-wise: The authors propose their algorithm PiSO (Piece-wise Scale Optimization) that uses calibration data to find the scale factor that minimizes the error function. Considering channel-wise scale under RTN leads to a univariate optimization problem with a closed-form solution for each interval.
  2. Group-wise: The authors propose two methods for group-wise quantization, independent and sequential groups, where the sequential group method accounts for cross-group interactions. It essentially accounts for quantization error introduced in the previous groups.
  3. The authors suggest three different methods for integrating PiSo with GPTQ and Qronos. The methods are (a) Decoupling, (b) Layer-wise Interleaving, and (c) Group-wise Interleaving that introduces a tighter decoupling and results to better results.

The manuscript includes theoretical contributions, formalizing the methods and optimization aspects of the algorithm. The results are documented in Appendix C. The manuscript also includes experiments on Llama-3 and Qwen-2.5 with different sizes and different bit sizes, essentially 2, 3 and 4, where perplexity and zero-shot accuracy evaluated using Wikitext2 dataset. Aside from Appendix C where proofs are documents, the manuscripts includes 4 more appendices where details of different aspect of the work are reported.

Critical strengths
  1. The theoretical treatment is solid.
  2. The proposed method provides an optimal solution for selecting scale factor in channel-wise RTN.
  3. The proposed method integrates naturally with GPTQ and Qronos.
  4. Overall, I consider this work a practically significant advancement in scale optimization for PTQ.
Critical weaknesses
  1. Exact optimality only holds for channel-wise RTN.
  2. Group-wise extensions are heuristic.
  3. The manuscript does not provide any theory explaining as to why PiSO helps GPTQ/Qronos.
  4. The sensitivity of the method with respect to calibration data requires more discussion.
  5. The method is mainly a weight quantization, but it may give the impression that it is an activation quantization.
Questions for authors
  1. I know that the optimization problems becomes much more difficult if I go to group-wise quantization, but any formality you can add that points at optimality of your suggested methods for group-wise quantization can be helpful.

  2. Again any formal result as to why PiSO helps GPTQ/Qronos can be helpful.

  3. Your proposed method depends on the calibration data. It is like a parametric optimization where we have some input parameters. In your case, it is the calibration data. This observation then gives rise to the following question: what is the optimal input for your proposed method, i.e. what is the optimal calibration data? There are already some research on optimal calibration data, e.g. ACDM (Activation Centroid Distance Metric), SelectQ, COLA etc. The following link provides the reference for SelectQ: https://www.techrxiv.org/doi/full/10.36227/techrxiv.21456291.v2

  4. There are also methods which are 0-shot and calibration free that show good promise. Your method for selecting the optimal scale cannot be applied in such cases. What would be the advantage of using your proposed method over such calibration free methods? Here is the link to one of such methods: https://www.techrxiv.org/doi/full/10.36227/techrxiv.21456291.v2

  5. In connection with the last item in Critical Weaknesses section, please comment on challenges in extending your method for activation quantization.

4 comments
AuthorsOpenReview ↗

Response 1/3

We thank the reviewer for the careful and constructive review, especially for recognizing our theoretical treatment as solid and describing our work as "a practically significant advancement in scale optimization for PTQ". The reviewer's questions on formal results for the group-wise objectives and for the integration with error correction motivated us to formalize analyses that we had only sketched so far, enriching our theoretical framework without changing our method, experimental results or conclusions. In short, the revised manuscript (changes highlighted in blue) adds:

  • formal results for both group-wise objectives (new Prop. C.15, alongside the existing Prop. C.13),
  • an analysis of how well PiSO's scales suit GPTQ, with a small controlled experiment that measures it for each integration strategy (new Appendices B.1, B.2 and C.1, summarized in Section 3.3),
  • discussions of calibration data selection (Section 4) and activation quantization (Sections 2 and 5),
  • following another reviewer's request, a default configuration of PiSO and a summary of our existing results under it (end of Section 4 and Table 6), with no new experiments.

Most of these additions are in the appendix and strengthen the theoretical analysis of the extensions of our method. We address in more detail each of the points raised by the reviewer below.

Weaknesses 1,2 / Question 1: Optimality beyond channel-wise RTN and formal results for the group-wise objectives

We agree that our exactness guarantee holds for channel-wise RTN, as stated in the abstract ("the globally optimal channel-wise scales under RTN"), and that the group-wise objectives and the integrations with error correction approximate the full problem, as discussed in Sections 3.2 and 3.3. As discussed in Section 3 (Remark C.3), optimizing the grid assignments freely, without RTN, makes the problem an NP-hard integer least-squares problem, even for a fixed scale (we expand on what can still be said for the integration with error correction in our answer to weakness 3 and question 2). For group-wise scales, there is another difficulty: exact minimization would require solving O((G∣G∣)K)\mathcal O((G|\mathbb G|)^K) closed-form subproblems instead of O(D∣G∣)\mathcal O(D|\mathbb G|) (Section 3.2). This curse of dimensionality is not specific to PiSO: a joint grid search with RR candidates per scale also needs RKR^K evaluations. Tractable objectives are a contribution that can be leveraged regardless of the optimizer (grid-search, PiSO...), and PiSO minimizes them exactly. Following the reviewer's suggestion, the revision provides formal results for both group-wise objectives:

  • Independent objective (already in the original submission, Prop. C.13 in Appendix C): its gap to the full objective is bounded by the spectral norms of the disregarded cross-group blocks of H\mathbf H and G\mathbf G.
  • Sequential objective (new, Prop. C.15 in Appendix C): (i) the gap between the sequential objective and the joint optimum is bounded by a term that only involves the coupling of each group with the groups processed after it, whereas the independent objective discards all cross-group blocks. (ii) For two groups, the sequential objective is never worse than the independent one.

Beyond two groups, the comparison is empirical: under RTN, the sequential objective gives lower perplexity than the independent one in all 24 group-wise configurations of Table 17, by more than an order of magnitude at 2 bits. The new result is referenced from Section 3.2.

AuthorsOpenReview ↗

Response 2/3

Weaknesses 3 / Question 2: Why PiSO helps GPTQ and Qronos

As the reviewer notes, the composition with error correction arises naturally, and our experiments show consistent gains with the interleaved strategies. Since jointly optimizing scales and assignments is NP-hard, PiSO's optimality does not carry over to these integrations, as stated in Section 3.3. We have, however, analyzed how well PiSO's scales suit GPTQ in a new Appendix B.1 (proofs in Appendix C.1), which is summarized in a new paragraph of Section 3.3. It is built on:

  1. Gap decomposition (Eq. 17). With s^\hat s the scale selected by PiSO, sECs^{\mathrm{EC}} the best scale for GPTQ (for which no efficient method is known) and E⋆E^\star the joint optimum, EEC(s^)−E⋆=EEC(sEC)−E⋆⏟assignment gap+EEC(s^)−EEC(sEC)⏟scale gap.E^{\mathrm{EC}}(\hat s)-E^\star=\underset{\text{assignment gap}}{\underbrace{E^{\mathrm{EC}}(s^{\mathrm{EC}})-E^\star}}+\underset{\text{scale gap}}{\underbrace{E^{\mathrm{EC}}(\hat s)-E^{\mathrm{EC}}(s^{\mathrm{EC}})}}. The assignment gap is GPTQ's own suboptimality, which no scale selection can reduce, so we analyze the scale gap, and argue that PiSO's scale is a good proxy for the best GPTQ scale.
  2. How the scale enters GPTQ's error (Prop. B.1). GPTQ's error is a weighted sum of its squared residuals (Zhang et al., 2025, Prop. 3.2), and each residual differs from the RTN rounding error assumed by PiSO by at most the drift, the error diffused into that weight after its scale was chosen. Intuitively, weights processed early receive little drift, so they are likely to be quantized to the value that PiSO assumed, and with the usual order of decreasing HiiH_{ii} these are the most important weights.
  3. Scale gap in expectation (Prop. B.2). In expectation, the scale gap splits into a weighting term, bounded by a factor θ\theta that compares how PiSO and GPTQ weight the rounding errors, and a drift term. Under a standard high-resolution model of quantization noise, the drift only changes GPTQ's expected error through the clipped weights, typically a small fraction of them, and we bound the drift term by the drift that these weights receive. Without clipping, the drift term vanishes, and GPTQ's expected error at the scale minimizing PiSO's expected objective is within a factor θ\theta of its minimum. Existing error bounds for GPTQ are restricted to this last case, as they assume an unbounded grid or no clipping (Zhang et al., 2025; Chen et al., 2026). The factor θ\theta is a worst case: for instance, the weighting term does not contribute at all when the weights that share a scale are identically distributed.
  4. Effect of the integration strategy (Cor. B.3). Moving from ++ to ⊙\odot removes the mismatch between the Hessians used by PiSO and GPTQ, and moving to ⊡\boxdot removes from the drift the error diffused from earlier groups.

Since the analysis relies on modeling assumptions, we also measure the scale gap directly, in a controlled experiment on a subset of the weights of three layers of Qwen2.5-1.5B (1,536 channels with 32 inputs each, in groups of 16; Appendix B.2). The reference is the best pair of scales found by an exhaustive grid search that runs GPTQ for all 1202120^2 scale pairs of each channel. We report GPTQ's error at each strategy's scales divided by its error at the scales found by the search (median / 90th percentile over channels):

Scales3 bits2 bits
absmax1.307 / 1.9111.225 / 1.771
PiSO, decoupled (++)1.041 / 1.1791.038 / 1.160
PiSO, layer-wise (⊙\odot)1.015 / 1.0971.017 / 1.090
PiSO, group-wise (⊡\boxdot)1.008 / 1.0771.010 / 1.076

Here no preceding layers are quantized, so the Hessian mismatch of the decoupled strategy is simulated by adding a random perturbation of about 20% to the inputs used by PiSO.

The gap shrinks with each tightening of the interleaving, and with ⊡\boxdot, GPTQ's error at PiSO's scales is within about 1% of its error at the best scales found by the search for the median channel. The exhaustive search limits this experiment to a small setting, but the ordering of the strategies matches our end-to-end results on full models.

This analysis covers GPTQ only, and its extension to Qronos is future work. For Qronos our evidence is empirical: with the interleaved strategies of our default configuration, PiSO improves over Qronos + absmax in 35 of 36 settings (Table 6). Interleaving matters most at 2 bits, where the RTN-quantized model on which the decoupled strategy selects its scales collapses: there, Qronos ⊙\odot PiSO improves over Qronos + absmax by 3–7× on Llama, whereas the decoupled strategy does not (Table 1). This is consistent with the mismatch that Cor. B.3 attributes to the decoupled strategy.

AuthorsOpenReview ↗

Response 3/3

Weakness 4 / Question 3: Calibration data

We thank the reviewer for the reference, and acknowledge that the effect of calibration data is often overlooked in the PTQ literature (e.g., both GPTQ and Qronos calibrate on 128 sequences from a single dataset and do not study the effect of this choice). Regarding the reviewer's question:

  • While our main results use a fixed calibration set (as is common in PTQ), our original submission already included a study of calibration size and domain shift, calibrating on WikiText2, C4, or FineWeb and evaluating on WikiText2 (Figure 2 and Appendix E). RTN + PiSO reaches near-asymptotic perplexity with four sequences on all three datasets, and GPTQ ⊙\odot PiSO calibrated on 16 sequences outperforms GPTQ + absmax calibrated on 512. The gains of PiSO over absmax hold whether or not the calibration and evaluation datasets match, and the gap between the two settings grows when PiSO is not used (Appendix E).

  • Selecting the calibration data is a complementary problem, the calibration data is an input to PiSO, which minimizes its layer-wise objective exactly for whatever data it is given. The piecewise structure and the cost of its sweep do not depend on the data, and PiSO does not require H\mathbf H to be invertible. Thus, PiSO can directly use data selected by methods such as SelectQ. Formally, the quality of the calibration data enters our layer-wise error decomposition (Remark C.4) through the covariance-deviation term, which such selection methods aim to reduce.

We added this discussion to the revised manuscript in the calibration paragraph of Section 4, and list the study of calibration data selection as future work in Section 5.

Question 4: Zero-shot and calibration-free settings

Our experiments include two calibration-free scale rules: absmax, and OptQuant, the exact minimizer of the data-free weight reconstruction objective, which is PiSO with X=X~=I\mathbf X=\tilde{\mathbf X}=\mathbf I. For 3-bit channel-wise RTN on Llama-3 (1B/3B/8B, Table 2), they reach perplexities of 2e3/436/1e3 and 9e3/6e4/5e4, respectively, compared with 19.5/10.6/8.99 for RTN + PiSO, which reaches near-asymptotic performance with four calibration sequences (Figure 2). In this regime, neither of these calibration-free rules is competitive with a data-aware scale. This comparison covers scale selection on the same grid, and if the reviewer had a specific calibration-free method in mind, we would be happy to discuss it. Finally, when no real data is available, PiSO can also be calibrated on synthetic data, since it works with any calibration data, however it is obtained/generated.

Weakness 5 / Question 5: Activation quantization

We thank the reviewer for pointing out this possible confusion. Section 2 now states explicitly that only weights are quantized: activations are kept in high precision, and X~\tilde{\mathbf X} differs from X\mathbf X only because the preceding layers have quantized weights. Following the reviewer's question, Section 5 now discusses the extension to activation quantization:

  • PiSO applies when activations are quantized too: computing X~\tilde{\mathbf X} with quantized activations lets the cross-activation objective account for the error that they introduce.
  • Static activation scales can be optimized with PiSO as well, since the activation grid assignments are also piecewise constant in each scale. This holds for any grid, so weights and activations may use different ones (e.g., W4A8).
  • We leave this extension to future work, together with the joint optimization of weight and activation scales, for instance by alternating PiSO between the two, as EasyQuant does with grid search, but solving each of them exactly.
  • Dynamic activation scales are computed at inference time and cannot be optimized in advance, but PiSO can compute optimal per-token scales on calibration data, which could serve as a gold standard to design cheap dynamic rules (e.g., a fixed fraction of the per-token absmax).

We thank the reviewer again for the constructive feedback, which helped us strengthen the theoretical side of the paper. We hope these additions address the reviewer's concerns and, if so, we would be grateful if the reviewer would consider raising their score.

Thank you for your responses to my comments. I will consider your responses when discussing with other reviewers and the AC.

Reviewer_ATW9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper studies the problem of scale selection during post-training quantization. They recognize that the layerwise quantization loss under RTN is a piecewise quadratic function of the quantization scale. Based on this observation, they propose an algorithm for optimizing the scales, which can be applied to RTN and other quantization methods such as GPTQ. The authors present numerical experiments, including perplexity and zero-shot accuracy of several benchmarks.

Critical strengths
  • The authors identify the scale optimization problem in quantization as a legit under-developed area of research, and I agree with them.

  • The optimization based on the piecewise quadratic nature of the loss is clever.

Critical weaknesses

The numerical experiments, while expansive, would benefit from a more careful evaluation protocol and discussion. First, the statistical significance and robustness of the reported improvements are unclear. In several cases, the best and second-best methods differ by only 0.01 in perplexity (e.g., 7.98 vs. 7.99), yet no variability across calibration samples or repeated runs is reported. It is therefore difficult to determine whether such differences represent meaningful improvements or experimental noise. Second, several PiSO variants and integration strategies are evaluated, with no single variant consistently dominating. This raises a model-selection concern: it is unclear whether the choice of PiSO variant can be made a priori, or whether the reported gains partly rely on selecting the best-performing variant for each setting. A fixed selection rule, or a comparison using a consistently chosen variant, would make the empirical conclusions more convincing. Third, the zero-shot evaluation is limited to ARC-Easy, ARC-Challenge, and HellaSwag; broader and more contemporary downstream benchmarks would strengthen the evaluation. Finally, the primary experiments focus on relatively small Llama-3 and Qwen-2.5 models, and it would be useful to establish that the conclusions continue to hold for larger and more recent models.

Questions for authors

Can the authors please address any my concerns with the experiments?

3 comments
AuthorsOpenReview ↗

Response 1/2

We thank the reviewer for the careful and honest review, and especially for recognizing scale optimization as an under-developed area of research and the use of the piecewise quadratic structure of the loss as a clever way to approach it. The reviewer's concerns are about the evaluation protocol, and we agree that addressing them makes our empirical conclusions more convincing. In short:

  • we measured the variability of our results across calibration samples with a seed study, and tested the significance of the gains with a paired test on the evaluation set (point 1),
  • the revised manuscript (changes highlighted in blue) adds a default configuration of PiSO, which is the fixed selection rule suggested by the reviewer, and a summary of all our existing results under it (point 2),
  • for broader benchmarks and larger models, we point to results that were already in the appendix of the original submission (point 3).

1. Statistical significance and robustness

We reported a single run per setting, as is common in the PTQ literature (e.g., GPTQ, AWQ and Qronos also report single values, without standard deviations), because we prioritized breadth: nine models from three families, three bit-widths, channel-wise and group-wise scales, and three rounding methods. We believe that this breadth already makes it unlikely that the gains are due to noise: PiSO improves over absmax with the same rounding method in 111 of 114 comparisons (Table 6, see point 2). Moreover, the calibration study of the original submission (Figure 2 and Appendix E) shows that these gains hold across calibration set sizes and sources. Still, we agree with the reviewer that a difference of 0.01 in perplexity is not meaningful, and that variability results would strengthen our claims. Following the reviewer's comment, we measured it in two ways on the 3- and 4-bit channel-wise experiments on Llama-3 (Table 2).

Variability across calibration samples. We repeated the experiments with three seeds, each one sampling different calibration sequences (mean ± standard deviation of the WikiText2 perplexity):

LlamaBitsGPTQ + absmaxGPTQ ⊙\odot PiSOQronos + absmaxQronos ⊙\odot PiSO
3.2-1B319.612.77 ± 0.0815.011.57 ± <0.01
3.2-1B410.49.92 ± 0.0110.09.60 ± 0.02
3.2-3B311.39.15 ± 0.049.838.41 ± 0.01
3.2-3B47.797.64 ± 0.017.647.43 ± 0.01
3.1-8B39.23–8.277.00 ± 0.01
3.1-8B46.49–6.366.18 ± 0.01

The standard deviation is below 0.1 at 3 bits and below 0.02 at 4 bits, while the gains over absmax range from 0.15 to 6.8, so in every setting the gain is more than ten times the standard deviation. The absmax results are from the single run in Table 2 of the paper. We will include the complete table, with the absmax baselines under the same seeds, in the camera-ready version. We will update the response if we have them before the rebuttal closes.

Significance on the evaluation set. We compared each ⊙\odot PiSO model with its absmax baseline on the 123 evaluation sequences of WikiText2 (on a rerun of Table 2), with a paired t-test on the per-sequence negative log-likelihood (Holm-corrected). The table reports the reduction in perplexity and the number of sequences on which PiSO is better, for the 1B / 3B / 8B models:

Perplexity reductionSequences improved (of 123)
GPTQ, 3 bits33.6% / 20.3% / 19.8%123 / 123 / 123
GPTQ, 4 bits4.2% / 2.0% / 2.8%116 / 103 / 114
Qronos, 3 bits22.8% / 15.0% / 15.4%123 / 123 / 123
Qronos, 4 bits4.3% / 2.4% / 3.0%120 / 118 / 122

The improvement over absmax is significant in all 12 settings (t≥10.9t\ge10.9, p<10−19p<10^{-19}), also at 4 bits, and the conclusion is the same with autocorrelation-robust standard errors and with a Wilcoxon signed-rank test. We will include the presented analysis in the camera-ready version.

Regarding the example of 7.98 vs. 7.99, we agree that it is a tie. It comes from group-wise quantization at 4 bits (Table 3, Qwen2.5-3B with groups of 16), where the unquantized model has 7.88 and all PiSO variants with error correction are between 7.98 and 8.01. The paired test agrees: at 4 bits, the decoupled and interleaved strategies do not differ significantly in any setting after correction. Such diminishing returns are expected: each additional bit doubles the number of grid points, so the quantization error, and with it the room for improvement, shrinks exponentially with the bit-width. PiSO is most helpful in the harder settings (2 and 3 bits). At 4 bits the problem is much easier, and we include these results to show that PiSO still helps, or at least does not hurt. For instance, with RTN at 4 bits, PiSO lowers the perplexity from 22.3 to 11.0 on Llama-3.2-1B (Table 2).

AuthorsOpenReview ↗

Response 2/2

2. Choosing the PiSO variant a priori

We thank the reviewer for this suggestion. The revision provides both a fixed selection rule and a comparison that only uses the variant that it selects. Our tables report several variants because one of the goals of the experiments was to study their effect. However, the choice can be made a priori: the design of the method suggests one configuration for each setting, which is the same for every model and bit-width. We now state it as a default configuration at the end of Section 4:

  • With RTN: the cross-activation objective, which accounts for the quantization error of the preceding layers, together with the sequential objective for group-wise scales.
  • With GPTQ or Qronos: the same objective as the error correction algorithm, with the tightest interleaving available (⊙\odot for channel-wise and ⊡\boxdot for group-wise scales).

We then summarized all our existing results using only the runs that follow this configuration, so no selection among variants is involved (new Table 6 in Appendix E, no new experiments):

Number of settings where…Channel-wiseGroup-wiseTotal
PiSO improves over RTN + absmax21/2118/1839/39
PiSO improves over GPTQ + absmax21/2116/1837/39
PiSO improves over Qronos + absmax17/1818/1835/36
the best result overall is obtained with PiSO21/2116/1837/39

The conclusions of the paper hold with this single, consistently chosen configuration, so they do not rely on selecting the best variant for each setting. Of the three comparisons where PiSO does not improve over absmax, two are within 0.01 perplexity (ties, as discussed in point 1), and the third is Qronos on Qwen2.5-1.5B at 2 bits.

3. Broader benchmarks and larger models

Appendix E covers broader benchmarks and other models for a subset of the settings, as mentioned in Section 4. These are not new results: they were already in the original submission. With the number of settings that we evaluate, we could not run a large benchmark suite or larger models on all of them. The main tables therefore report perplexity and zero-shot accuracy on models of up to 8B parameters, so that all the variants could be evaluated in every setting.

Table 16 reports MMLU (general knowledge), IFEval (instruction following) and LongBench (long-context question answering) for the 3-bit Qwen2.5 models (1.5B / 3B):

MMLUIFEvalLongBench
BF1660.1 / 65.527.0 / 47.728.5 / 28.4
RTN + absmax24.6 / 25.59.4 / 11.32.6 / 1.2
RTN + PiSO42.6 / 55.218.1 / 29.215.0 / 17.0
GPTQ + absmax25.7 / 29.910.9 / 14.613.3 / 12.8
GPTQ ⊙\odot PiSO40.2 / 55.616.6 / 27.719.1 / 18.7

Tables 14 and 15 extend the 3-bit channel-wise evaluation to a larger model, Qwen2.5-14B, and to a third model family, Gemma-2:

WikiText2 (↓)0-shot (↑)
Qwen2.5-14B, BF165.7361.6
GPTQ + absmax7.3751.5
GPTQ ⊙\odot PiSO6.4761.7
Gemma-2-9B, BF167.5965.8
GPTQ + absmax8.5261.8
GPTQ ⊙\odot PiSO8.0665.6

The trend is the same as in the main tables. On the additional benchmarks, PiSO improves over absmax on every task and model, with and without error correction (e.g., GPTQ + absmax stays close to chance level on MMLU, while GPTQ ⊙\odot PiSO reaches 40.2 and 55.6). On the additional models, GPTQ ⊙\odot PiSO stays within 0.2 points of the zero-shot accuracy of the unquantized model, whereas GPTQ + absmax loses 10 and 4 points. PiSO optimizes the scales of each linear layer and does not depend on the architecture, so we expect the same behavior on more recent models. If we are able to complete runs on a more recent model before the discussion period closes, we will report them here.


We thank the reviewer again for the constructive feedback, which helped us make the evaluation more rigorous. The reviewer's concerns were about the experiments, and we hope that the seed study, the significance test, the default configuration and the results above address them. If so, we would be grateful if the reviewer would consider raising their score: as the reviewer notes, scale optimization is an under-developed area of research, and we believe that an exact method with consistent empirical gains is a useful contribution to it.

Thank you for the response, I have revisited my score.

Reviewer_QU1hOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces PiSO, a piecewise scale optimization algorithm for post-training quantization of large language models. By leveraging the piecewise-constant nature of round-to-nearest grid mappings as a function of the scaling factor, the method derives globally optimal channel-wise scaling factors for a data-aware layer-output reconstruction objective with polynomial-time complexity. The authors further extend the framework to group-wise quantization through two computationally tractable approximations, and propose three interleaving strategies to combine PiSO with established greedy error correction methods including GPTQ and Qronos. Empirical results across Llama, Qwen, and Gemma model families show consistent improvements in perplexity and zero-shot accuracy over standard scale selection heuristics, particularly at lower bit widths.

Critical strengths
  1. The paper establishes a solid theoretical framework for scale optimization under RTN quantization, with complete formal proofs of the piecewise error structure, closed-form interval minimizers, and asymptotic complexity bounds. The incremental sweep technique reduces the per-channel computational cost dramatically compared to exhaustive grid search, resulting in near-negligible runtime overhead relative to standard PTQ pipelines, which is highly favorable for practical adoption.

  2. The experimental evaluation is thorough and well-controlled, spanning multiple model sizes, bit widths, and quantization granularities, with both perplexity and downstream task metrics. The ablation studies on calibration data efficiency and integration granularity provide useful empirical insights that go beyond simply reporting state-of-the-art numbers.

Critical weaknesses
  1. The stated “global optimality” guarantee is strictly conditional on the RTN quantization assumption, which is not satisfied by modern error-correction PTQ methods that intentionally shift weights away from their nearest grid points to compensate for accumulated rounding errors. The optimization guarantee therefore does not apply to the full joint problem of scale and grid assignment optimization, which is known to be NP-hard.

  2. Some baseline comparisons are not fully equitable. The 2-bit RTN+absmax baseline reflects a minimally optimized configuration rather than a competitive practical baseline, inflating the relative gains of PiSO. Additionally, the Beacon baseline is re-implemented by the authors rather than using an official release, introducing potential implementation bias.

Questions for authors
  1. Given that the optimality guarantee of PiSO is bounded by the RTN assumption, which is violated during greedy error correction, how does the suboptimality gap change as it tighten the interleaving of scale optimization and error correction (from decoupled to layer-wise to group-wise)? Is there a formal way to bound the gap between the PiSO solution and the true joint optimum of scales and grid assignments?

  2. The current work only considers weight quantization. How would the piecewise scale optimization framework extend to settings with joint weight and activation quantization? Would the piecewise-constant structure of the objective still hold when both weight and activation scaling factors are optimized together, particularly for asymmetric quantization schemes?

4 comments
AuthorsOpenReview ↗

Response 1/2

We thank the reviewer for the detailed review, and for recognizing our theoretical framework as solid and the thoroughness of our experimental evaluation. In short:

  • the scope of the optimality guarantee (channel-wise RTN) is stated throughout the original submission, and the revised manuscript (changes highlighted in blue) adds the analysis requested in Question 1 in Appendix B,
  • our 2-bit claims are measured against competitive baselines (such as Qronos + absmax) and not just against RTN + absmax (included for completeness), and our conclusions rest on the established methods AWQ, GPTQ and Qronos, not on Beacon,
  • Section 2 now explicitly states our focus on weight quantization, and Section 5 discusses activation quantization in more detail, as also requested by Reviewer JVLE (Question 2).

These additions, most of them in the appendix, complement the original submission without changing our method, experimental results or conclusions.

Weakness 1 / Question 1: Scope of the optimality guarantee and gap to the joint optimum

We agree with the reviewer that our optimality guarantee holds under RTN and does not extend to the joint optimization of scales and grid assignments, which is NP-hard. This is how the original submission presents it, for example in the abstract, the contributions and Section 3.

Respectfully, we consider that not solving an NP-hard problem is a limitation, but not a weakness of the paper when this limitation has been clearly stated in our (original) manuscript. We provide a guarantee that existing data-aware scale methods do not offer: we show that, under RTN, the optimal data-aware scales can be computed exactly in polynomial time, and we provide the algorithm that computes them. This is one of our key contributions and it is useful on its own, as at 3 bits RTN with PiSO scales already outperforms GPTQ with absmax scales on all Llama models (Table 2). The integration with error correction is a methodological contribution, for which we do not claim optimality and which our experiments support: PiSO improves over GPTQ + absmax in 37 of 39 settings and over Qronos + absmax in 35 of 36 (Table 6). This integration is also principled (Section 3.3): the weights processed first receive little diffused error, so they are likely to be rounded to the grid point that PiSO assumed, and with the usual order of decreasing HiiH_{ii} these are the most important weights.

Question 1 motivated us to formalize this intuition (new Appendices B.1 and B.2, proofs in Appendix C.1, summarized in Section 3.3). The analysis addresses both parts of the question for GPTQ.

Is there a formal way to bound the gap? With s^\hat s the scale selected by PiSO, sECs^{\mathrm{EC}} the best scale for GPTQ and E⋆E^\star the joint optimum (Eq. 17),

EEC(s^)−E⋆=EEC(sEC)−E⋆⏟assignment gap+EEC(s^)−EEC(sEC)⏟scale gap.E^{\mathrm{EC}}(\hat s)-E^\star=\underset{\text{assignment gap}}{\underbrace{E^{\mathrm{EC}}(s^{\mathrm{EC}})-E^\star}}+\underset{\text{scale gap}}{\underbrace{E^{\mathrm{EC}}(\hat s)-E^{\mathrm{EC}}(s^{\mathrm{EC}})}}.

The assignment gap is GPTQ's own suboptimality, which no scale selection can reduce. We bound the scale gap in expectation, under standard assumptions on the rounding errors (Prop. B.2): it splits into a weighting term, bounded by a factor θ\theta that compares how PiSO and GPTQ weight the rounding errors, and a drift term, bounded by the error diffused into the clipped weights. Without clipping, the drift term vanishes, and GPTQ's expected error at the scale minimizing PiSO's expected objective is within a factor θ\theta of its minimum.

How does the gap change as the interleaving tightens? Formally (Cor. B.3), moving from ++ to ⊙\odot removes the mismatch between the Hessians used by PiSO and GPTQ, and moving to ⊡\boxdot removes from the drift the error diffused from earlier groups. We also measured the scale gap on real weights (sub-matrices of three weight matrices of Qwen2.5-1.5B). The table reports GPTQ's error at each strategy's scales divided by its error at the best scales found by an exhaustive search over 1202120^2 scale pairs per channel (median / 90th percentile over channels):

Scales3 bits2 bits
absmax1.307 / 1.9111.225 / 1.771
PiSO, decoupled (++)1.041 / 1.1791.038 / 1.160
PiSO, layer-wise (⊙\odot)1.015 / 1.0971.017 / 1.090
PiSO, group-wise (⊡\boxdot)1.008 / 1.0771.010 / 1.076

The gap shrinks with each tightening of the interleaving. With ⊡\boxdot, GPTQ's error at PiSO's scales is within about 1% of this reference for the median channel, whereas with absmax scales it is more than 20% above. (The decoupled strategy is simulated by perturbing the inputs used by PiSO by about 20%.) The same ordering appears on full models, where group-wise interleaving is the best integration strategy in every configuration of Table 3. The analysis covers GPTQ, and its extension to Qronos is future work (Section 5).

AuthorsOpenReview ↗

Response 2/2

Weakness 2: Baselines

RTN + absmax at 2 bits. We agree that RTN + absmax is not a competitive baseline at 2 bits, and our 2-bit claims do not rest on it. In channel-wise quantization (Table 1), RTN collapses at 2 bits regardless of the scale, also with PiSO, and these rows are reported for completeness. Our claims compare the same error correction method with and without PiSO, together with the standalone methods AWQ and Beacon (WikiText2 perplexity on Llama 1B / 3B / 8B):

2-bit perplexity
AWQ2e6 / 8e5 / 1e6
Beacon52.8 / 72.4 / 25.5
GPTQ + absmax3e3 / 415 / 331
GPTQ ⊙\odot PiSO54.7 / 29.5 / 22.3
Qronos + absmax197 / 68.9 / 49.5
Qronos ⊙\odot PiSO27.9 / 18.9 / 16.6

PiSO reduces the perplexity of Qronos + absmax, the strongest absmax baseline, by 3–7×. Note also that absmax is not our only RTN baseline: the main tables include OptQuant, the exact minimizer of the data-free objective, which is a stronger version of the common grid-search heuristics. To avoid any misreading, the caption of Table 1 and the results paragraph of Section 4 have been updated and now state explicitly that RTN collapses at this bit-width and is reported for completeness. As for Beacon, it has no official code release, so using our own implementation was the only way to include it in the comparison. More importantly, AWQ, GPTQ and Qronos are much more established methods, they were run with publicly available implementations, and PiSO consistently improves over them or in combination with them.

Question 2: Joint weight and activation quantization

Section 2 is now explicit about our focus on weight quantization, and Section 5 discusses activation quantization in more detail, as also requested by Reviewer JVLE. Regarding the reviewer's questions:

  • Piecewise structure. It still holds. The grid assignments of the weights depend only on the weight scale, and those of the activations only on the activation scale, so each of them is piecewise constant in its own scale. For a fixed weight scale, the layer output error is piecewise quadratic in a static activation scale, and vice versa, so PiSO can solve each of the two problems exactly.
  • Joint optimization. When both scales vary, the assignments are constant on products of intervals. Their number is the product of the two, so an exact joint sweep is expensive. A natural approach is to alternate PiSO between the two scales, as EasyQuant does with grid search, but solving each step exactly. We leave this for future work.
  • Asymmetric schemes. The piecewise structure does not depend on the grid, so weights and activations can use different grids and bit-widths (e.g., W4A8), and each scale is still optimized exactly on its own grid.
  • Activations quantized by another method. PiSO can already be used in this case: computing X~\tilde{\mathbf X} with the quantized activations lets the cross-activation objective account for the error that they introduce.

We thank the reviewer again for the feedback. We hope to have addressed all the reviewer's concerns and questions: the scope of our guarantee is stated in the paper and is now complemented with the analysis requested in Question 1, and our conclusions do not depend on the two baselines discussed in Weakness 2. We would therefore appreciate it if the reviewer could raise their score.

I thank the authors for this thorough response and the substantial additional theoretical analysis and empirical gap evaluation added in the appendix. My main concerns on the scope of the optimality guarantee, the interpretation of 2-bit baselines, and activation quantization have been satisfactorily addressed. The expected error bound for GPTQ combined with PiSO and the exhaustive-scale sweep table are valuable additions that strengthen the paper’s argument.

That said, several limitations remain: the optimality guarantee is still restricted to RTN rounding, the theoretical bound only applies under expectation assumptions and for GPTQ (extension to Qronos is left as future work), and hardware/latency evaluation is still absent. These points limit the strength of the contribution.

I am raising my score, though with reservations.

We thank the reviewer for the constructive discussion and for reconsidering their evaluation, and we are glad that our response and the additional analysis addressed their main concerns. As an additional clarification:

  • Scope of the guarantee. We believe that our guarantee has value on its own: under RTN, PiSO computes the optimal data-aware scales exactly, which existing data-aware scale methods do not guarantee. To our knowledge, it had not been shown before that data-aware scale selection can be solved exactly and efficiently, even when restricted to RTN. Its significance also goes beyond this setting, as PiSO empirically improves over established baselines when combined with error correction (e.g., Tables 2 & 3). Furthermore, thanks to the rebuttal discussion, we now have stronger a theoretical analysis to explain this improvement. Regarding the scope itself, we explicitly state it in several places of our original manuscript. If the reviewer would like the scope to be stated in some other place or in a different way, we would be happy to include it in the final version.

  • Analysis in expectation. The bound in Proposition B.1 is deterministic and holds for every weight vector. The comparison between PiSO scale and the best possible scale for GPTQ (Prop. B.2) is stated in expectation because this gives a more informative result. Note that, in the worst case, the diffused error can be as large as the rounding errors themselves. We also measured the error gap introduced by the scale (the scale gap in Eq. 17) directly on a subset of real weights (Appendix B.2). As the reviewer points out, we leave the extension of this analysis to Qronos as future work (Limitations and future work, Section 5).

  • Hardware and latency. PiSO adds an overhead at quantization time only. Once the scales are selected, the inference time is the same as that of a model quantized to the same bit-width with scales selected by any other method. Moreover, the overhead at quantization time is minimal (Table 5, Appendix E, already in the original manuscript), as the reviewer noted in their initial review: "near-negligible runtime overhead relative to standard PTQ pipelines".

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the rebuttal

We thank the reviewers and the Area Chair for their time and for a constructive discussion. We are glad that our responses addressed the reviewers’ main concerns: Reviewers QU1h and ATW9 raised their scores, and the discussion has resulted in a consensus among all three reviewers in favor of acceptance. Many of the points raised were clarified with content already present in the original manuscript. Moreover, while our method, main contributions, and conclusions remain unchanged, we believe that the reviewers' comments and the work done during this rebuttal have strengthened our original submission. We summarize the discussion below:

  • Scope of the guarantee (QU1h, JVLE). The exact guarantee of PiSO holds for channel-wise RTN, as we clearly state in several places of the manuscript (including the abstract and the contributions). The group-wise objectives and the integration with error correction are practical extensions of our method, validated empirically with consistent gains over established methods (AWQ, GPTQ and Qronos).
  • Evaluation and calibration (ATW9, JVLE). The original appendix already contained more benchmarks, larger models and an expanded version of the study on calibration data size.
  • Theoretical analysis (QU1h, JVLE). We extended our analysis of the group-wise objectives to the sequential one (Prop. C.15), with a bound similar to the one we already had for the independent one (Prop. C.13). We also complemented our study of the integration with error correction with a more formal analysis for GPTQ, supported by a small controlled experiment (Appendix B).
  • Evaluation (ATW9). We now state explicitly the default configuration that the design of the method suggests (end of Section 4). When restricting our existing results to this fixed recipe, PiSO with GPTQ improves over GPTQ with absmax in 37 of 39 settings and PiSO with Qronos improves over Qronos with absmax in 35 of 36 (summary presented in Table 6, appendix E). We also reran part of Table 2 with different seeds and ran a significance test (both reported in our response), which confirm that the gains are far larger than the variability.
  • Discussion on activation quantization (JVLE, QU1h). Section 5 now discusses the use of PiSO together with activation quantization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.