acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Numerical Stability in Hessian-Based PTQ: Accumulation-Order Sensitivity

Abstract

Post-Training Quantization (PTQ) is widely used to reduce the computational and memory costs of large language models. Hessian-based PTQ methods such as GPTQ estimate empirical second-order statistics from calibration samples to guide weight quantization. Although the sum of these statistics is invariant to sample order in exact arithmetic, floating-point non-associativity can yield different empirical Hessians depending solely on the order in which the same samples are accumulated. While calibration sample selection is commonly treated as an important experimental condition, accumulation order is typically left uncontrolled. To our knowledge, we provide the first systematic evidence that reordering fixed calibration samples alone affects quantization and downstream performance in Hessian-based PTQ, with variability comparable to calibration resampling. We isolate this effect by varying only accumulation order across Qwen and Llama models, multiple Hessian-based PTQ methods, and weight bit-widths. In some settings, order-induced performance variation exceeds inter-method performance gaps and even reverses method rankings. Further analysis shows that order-dependent perturbations in the empirical Hessian are amplified through subsequent Hessian-dependent operations, ultimately altering quantization decisions. As a practical control, numerical stabilization incorporating compensated summation substantially suppresses order-induced variation. These findings highlight the need to account for accumulation order when evaluating and comparing Hessian-based PTQ methods. Our code is available at https://anonymous.4open.science/r/ReSQ-B3EF.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post-Training Quantization, Hessian-based Quantization, Numerical Stability, Reproducibility

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ykNVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates an overlooked source of variability in Hessian-based post-training quantization: the order in which fixed calibration samples contribute to accumulated statistics. Across six Qwen and Llama models, four quantization methods, and 3-/4-bit weights, changing accumulation order produces downstream variation sometimes comparable to calibration resampling and can reverse method rankings. Fixed-activation diagnostics show that small Hessian differences are amplified during damped inversion. The authors introduce stabilization incorporating Neumaier compensated summation, substantially reducing order sensitivity with modest measured overhead.

Critical strengths
  1. Controlled experiments convincingly isolate accumulation-order sensitivity.
  2. Broad evaluation covers six models, four methods, and two bit widths, supported by numerical diagnostics.
  3. Practical mitigation achieves identical weights across orders in many settings, with approximately 5% measured quantization-time overhead.
Critical weaknesses
  1. Ordering variability is measured on one fixed calibration set per condition, limiting statistical generalization.
  2. Some stabilized implementations also modify normalization and reductions; missing ablations weaken attribution to compensated summation.
Questions for authors
  1. Do variability ratios and ranking reversals persist across multiple fixed calibration sets?
  2. What does each stabilization component contribute, and why does FP64 accumulation retain substantial variability?
4 comments
AuthorsOpenReview ↗

Official Response to Reviewer ykNV

[W1 & Q1] Ordering variability is measured on one fixed calibration set per condition. Do the variability ratios and ranking reversals persist across multiple fixed sets?

Following the reviewer's suggestion, we repeated the ordering experiment on three more calibration sets. Ordering variability appears on all four sets, and a ranking reversal between GPTQ and GPTAQ also occurs on one of the additional sets.

Setup. Besides the calibration set of the paper (Base set), we used three other fixed sets of random WikiText-2 samples (Sets 1–3). On each set, we quantized the model 10 times, accumulating the same samples in a different order each time, for GPTQ and GPTAQ at INT3 on Qwen3-0.6B and Llama-3.2-1B.

Variability. σord\sigma_{\text{ord}} is nonzero on all four sets in every case, between 0.32 and 1.00 (Table R13). Pooled over the four sets, it is 0.66–1.17 times σsamp\sigma_{\text{samp}}, the standard deviation across independently sampled sets. The paper also reports six different fixed sets on Llama-3.1-8B (three corpora, two sizes), all with σord\sigma_{\text{ord}} between 0.33 and 0.61 (Table 1).

Ranking reversals. Pairing the two methods by order seed as in §3.2, GPTQ scores higher than GPTAQ in 2 of the 10 pairs on Set 3 of Llama-3.2-1B, although GPTAQ is ahead on average. A reversal is therefore not limited to the calibration set of the paper. Reversals are less frequent here than in §3.2, because the two methods are further apart on these two models: 3–5 pp on the Base set (Table 3 of the revised manuscript), against 0.3 pp on Qwen3-8B in §3.2.

We have added Table R13 to App. B.2 in the revised manuscript.

Table R13. Ordering variability on four fixed calibration sets (INT3). σord\sigma_{\text{ord}} (in percentage points) over 10 accumulation orders on each set.

ModelMethodBase set (paper)Set 1Set 2Set 3Pooled over setsσsamp\sigma_{\text{samp}}
Qwen3-0.6BGPTQ0.580.700.320.790.620.95
GPTAQ0.340.720.821.000.760.65
Llama-3.2-1BGPTQ0.770.740.870.830.801.02
GPTAQ0.710.740.720.950.790.76

Pooled over sets: the square root of the mean of the four variances.

[W2 & Q2-1] The stabilized implementations also change normalization and reductions. What does each stabilization component contribute?

We thank the reviewer for asking for this breakdown. Besides Neumaier compensation, our stabilized implementations change the normalization, the scaling, and the reduction order, and all of these changes are needed. Leaving out any one of them leaves Ordering variability in at least one of our settings.

The three changes are as follows (App. D.3).

  • Normalization (GPTAQ, BoA, TurboBoA): add up all samples and divide once, instead of rescaling a running mean after every batch. The GPTQ implementation we build on, from the open-source llm-compressor library (v0.10.0.1), already does this. For the ablation, we also ran GPTQ with a running mean.
  • Scaling (GPTQ, GPTAQ): compute 2XX⊤2XX^\top instead of (2X)(2X)⊤(\sqrt{2}X)(\sqrt{2}X)^\top.
  • Reduction (BoA, TurboBoA): use a fixed summation tree when undoing the rotation.

We turned each change on and off, with and without compensation (Table R5-a; Qwen3-0.6B and Llama-3.2-1B, INT3, 10 orders).

  • Compensation alone is not enough. Without compensation, no combination of the changes removes the variability (σord\sigma_{\text{ord}} 0.23–1.42). With compensation but with the running mean kept, it also remains (σord\sigma_{\text{ord}} 0.35–1.17).
  • Scaling is needed as well. For GPTQ on Llama-3.2-1B, compensation with Normalization but without Scaling still gives 2 different models from the 10 orders (σord\sigma_{\text{ord}} 0.12). For GPTAQ on Llama-3.1-8B, the same configuration leaves σord\sigma_{\text{ord}} 0.44 (Table R5-b). With Scaling added, all 10 orders give the same model in both cases.
  • Reduction is needed for BoA on the 8B models. On the two small models, BoA and TurboBoA already give identical models with compensation and Normalization. For BoA on the 8B models, this configuration leaves σord\sigma_{\text{ord}} 0.20 / 0.26, and adding Reduction lowers it to 0.02 / 0.13 (Table R5-b).

Why compensation needs Normalization. We attribute the variability that remains under a running mean to its rescaling step. Compensated summation corrects the rounding error of additions. A running mean also multiplies the partial sum by a ratio such as n/(n+m)n/(n+m) after every batch, and the rounding of that multiplication depends on which samples have been added so far. The correction term records only addition errors, so it does not capture this rounding. Normalizing once at the end turns the accumulation into a plain sum, to which compensation applies.

GPTQ. Our standard FP32 GPTQ already uses Normalization and Scaling, so its standard FP32 and stabilized versions differ only in compensation (App. D.3). There, compensation takes σord\sigma_{\text{ord}} from 0.39–0.77 to 0.00 on all four models in Table 3 of the revised manuscript.

In the revised manuscript, we refer to the full configuration as "Stabilized" instead of "Neumaier" in Table 3 and §5.2, state in §5.1 that all of the changes are required, and add Table R5-a to App. D.4. The 8B-model ablation is Table 2 of the revised manuscript.

Table R5-a. Effect of each change with and without Neumaier compensation (σord\sigma_{\text{ord}}, INT3, 10 orders). Bold: the full stabilized configuration.

MethodChanges appliedQwen3-0.6B, no compensationQwen3-0.6B, with compensationLlama-3.2-1B, no compensationLlama-3.2-1B, with compensation
GPTQNone0.610.770.881.10
Scaling0.801.021.201.15
Normalization0.880.001.110.12
Scaling + Normalization0.580.000.770.00
GPTAQNone0.340.740.710.56
Scaling0.500.550.800.71
Normalization0.350.000.860.00
Scaling + Normalization0.460.000.720.00
BoANone0.270.830.290.47
Normalization0.640.000.300.00
Reduction0.300.490.230.54
Normalization + Reduction0.420.000.390.00
TurboBoANone1.420.590.260.50
Normalization0.940.000.380.00
Reduction0.821.170.750.35
Normalization + Reduction0.950.000.350.00

Table R5-b. Effect of each change with compensation on the 8B models (σord\sigma_{\text{ord}}, INT3, 10 orders). Bold: the full stabilized configuration.

MethodChanges appliedQwen3-8B, with compensationLlama-3.1-8B, with compensation
GPTAQNone0.590.43
Scaling0.730.70
Normalization0.000.44
Scaling + Normalization0.000.00
BoANone0.630.34
Normalization0.200.26
Reduction0.570.36
Normalization + Reduction0.020.13

[Q2-2] Why does FP64 accumulation retain substantial variability?

We thank the reviewer for this question. FP64 accumulation is more precise, but its result is still cast to FP32, and tiny order-dependent differences in the FP64 Hessian can survive that cast. This is why FP64 retains variability. For BoA and TurboBoA, the FP64 control in the paper already matched the stabilized runs, so those results stand as reported. For GPTQ and GPTAQ it did not, and we have now matched it. The GPTQ results are essentially unchanged, and the GPTAQ variability disappears on the two models (Qwen3-0.6B and Llama-3.2-1B).

Precision of each operation. The FP64 control uses the following precision. Standard FP32 and the stabilized runs use FP32 in every step (App. A.2).

  1. Activation conversion: BF16 → FP64
  2. Matrix multiplication: FP64
  3. Scaling: FP64
  4. Accumulation: FP64
  5. Hessian cast: FP64 → FP32
  6. Normalization: FP32
  7. Solving: FP32

Implementation changes. Besides compensation, our stabilized implementations make method-specific changes that reduce rounding outside the summation itself (App. D.3). Two of them concern GPTQ and GPTAQ.

  • Scaling: compute 2XX⊤2XX^\top instead of (2X)(2X)⊤(\sqrt{2}X)(\sqrt{2}X)^\top, which avoids the rounding caused by the inexact value of 2\sqrt{2}.
  • Normalization: add up all samples and divide once, instead of rescaling a running mean after every batch.

BoA and TurboBoA additionally use a fixed reduction order when undoing the rotation (Reduction).

What the FP64 control in the paper used. For BoA and TurboBoA, the FP64 control already used the same settings as the stabilized implementations, so their results in Table 3 are unchanged. For GPTQ, it lacked Scaling. For GPTAQ, it lacked both Scaling and Normalization.

Matching the control (Table R3). For GPTQ, adding Scaling gives σord\sigma_{\text{ord}} of 0.00 / 0.29, against 0.01 / 0.33 in the paper (Qwen3-0.6B / Llama-3.2-1B). For GPTAQ, adding both changes gives 0.00 / 0.00, against 0.48 / 0.80.

FP64 with matched settings. There are 16 cases in which FP64 and the stabilized runs use the same settings: four GPTQ and GPTAQ cases on two models in Table R3, and 12 cases in Table 3 of the revised manuscript (GPTQ and GPTAQ on the other two models, and BoA and TurboBoA on all four models). FP64 leaves nonzero σord\sigma_{\text{ord}} in 10 of them (three for GPTQ, two for GPTAQ, three for BoA, and two for TurboBoA), and the stabilized runs in 2 (none for TurboBoA). FP64 is the more precise accumulation, yet the variability remained, so the benefit of compensation is not explained by higher precision alone.

Where the FP64 variability comes from. We examined the remaining GPTQ case on Llama-3.2-1B, with both changes applied, for two orders of the same calibration set (seeds 777 and 5000). We compared the Hessians of the two orders just before and just after the cast to FP32. Layer 8 is the first layer in which the FP32 Hessians differ (Table R4). Before the cast, the largest difference in its down_proj Hessian is only 2.44×10−42.44 \times 10^{-4} ULP, where ULP is the gap between two neighboring FP32 numbers. After the cast, 2 of its 819228192^2 entries differ by 1 ULP. This happens when a value lies right at the boundary between two FP32 numbers, so the two orders round it in different directions.

This is a concrete path by which order dependence reaches the FP32 computations that follow, and a change in one layer spreads to the later layers. In an additional experiment, an order difference in a single layer changes 14.1% / 13.2% of the integer weights over the whole model (Qwen3-0.6B / Llama-3.2-1B, new App. C.6). With the stabilized accumulation in the same setting, the quantized weights are bit-identical across all ten orders (§5.2).

Table R3. Ordering variability (σord\sigma_{\text{ord}}) of the FP64 control with and without the two changes (INT3, 10 orders).

MethodChanges appliedQwen3-0.6BLlama-3.2-1B
GPTQNeither0.590.75
Scaling only0.310.88
Normalization only (paper)0.010.33
Both (same as stabilized)0.000.29
GPTAQNeither (paper)0.480.80
Scaling only0.700.74
Normalization only0.210.89
Both (same as stabilized)0.000.00

Table R4. Hessian differences between order seeds 777 and 5000 in layer 8 (Llama-3.2-1B, GPTQ INT3, FP64 with both changes).

ModuleBefore cast: largest difference (ULP)After cast: entries that differ
q/k/v (shared)1.12×10−81.12 \times 10^{-8}0 of 204822048^2
o_proj1.53×10−51.53 \times 10^{-5}0 of 204822048^2
gate/up (shared)2.98×10−82.98 \times 10^{-8}0 of 204822048^2
down_proj2.44×10−42.44 \times 10^{-4}2 of 819228192^2 (both by 1 ULP)
AuthorsOpenReview ↗

Gentle Reminder: Follow-up on our response

Dear Reviewer,

Thank you again for your time and thoughtful feedback on our submission.

As the discussion period is approaching its end on November 18 (AoE), we wanted to kindly follow up to see if our previous responses and the revised manuscript have adequately addressed your concerns.

If any points remain unclear or if you have any further questions, please let us know. We would be more than happy to clarify them promptly before the deadline.

Best regards,
Authors

Reviewer_Qn3nOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper shows that, in GPTQ-style PTQ, reordering a fixed calibration set results in variations in FP32 Hessian comparable in magnitude as resampling a different calibration set. Neumaier compensated summation removes the order dependence at about 5% overhead.

Critical strengths
  • A clean, well-controlled demonstration of a reproducibility hazard the PTQ literature ignores, traced from rounding error to changed integer weights.
  • The fix is simple, method-agnostic and cheap, with code released.
Critical weaknesses
  • Practical significance is modest and somewhat overstated: First, limitations beyond reproducibility are not convincingly established; in reality, variations due to calibration data sampling are already tolerated--why can't one tolerate variations due to ordering as well? After all, resulting model quality is the ultimate objective to optimize--do these two sources of variation contribute differently to the resulting model quality? Second, ordering variance is below sampling variance in most configurations, ten runs cannot separate the two, and perplexity barely moves, so the accuracy variation may be small-task evaluation noise on near-equivalent models.
  • Neumaier makes results deterministic, not better, and does nothing about the larger sampling variance; the real recommendation is to report variance, which the paper already concedes. The technique itself is textbook.
Questions for authors
  1. Whether do ordering and sampling variances differ significantly by statistical tests?
  2. How do variations due to data sampling vs. to data ordering contribute to final model quality separately?
  3. Is the accuracy variation mostly evaluation noise? Compare large vs small tasks.
  4. Does a single-GEMM Hessian or tree summation already give order independence?
  5. Which widely used implementations accumulate in FP32 vs BF16?
6 comments
AuthorsOpenReview ↗

Official Response to Reviewer Qn3n

[W1-1a] Variation from calibration sampling is already tolerated. Why not tolerate variation from ordering as well?

We thank the reviewer for this question. Sampling variability is a recognized experimental condition: papers fix and report the calibration set, and a line of work studies how to choose it [1–4]. Ordering variability is not controlled in this way. It remains even when the calibration set is fixed, and it is rarely reported. We see two reasons why it should be controlled rather than tolerated.

  • Mathematically, it should not exist. Reordering the same samples leaves the data and the objective unchanged, and the Hessian is identical in exact arithmetic. The variation therefore carries no information about the data or the method. It is an artifact of rounding, and unlike Sampling variability, it can be controlled at low cost (§5).
  • It makes it hard to tell which method is better. With the calibration set fixed, GPTQ scores higher than GPTAQ under 7 of 10 accumulation orders and lower under the other 3 (Qwen3-8B, INT3, §3.2). So which of the two methods scores higher depends only on the accumulation order.

Model quality is the final objective, and we do not claim that Ordering lowers it on average. Our claim is that accumulation order is an overlooked experimental variable. Unless it is controlled, a difference between two methods or two calibration sets cannot be reliably attributed to them. We show how the two sources differ, with additional experiments, in our response to [W1-1b & Q2].

[1] B. He, L. Yin, H. Zhen, S. Liu, H. Wu, X. Zhang, M. Yuan, and C. Ma. Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization. Advances in Neural Information Processing Systems 38 (NeurIPS 2025), pp. 58531–58572, 2026 (arXiv:2510.10618).
[2] M. Williams, G. Chrysostomou, and N. Aletras. Self-calibration for Language Model Quantization and Pruning. NAACL 2025 (arXiv:2410.17170).
[3] Z. Yang, S. Hou, K. Zheng, Y. Liu, L. He, Y. Chen, and K. Cui. Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models. Findings of EMNLP 2026, pp. 14073–14093.
[4] M. Williams and N. Aletras. On the Impact of Calibration Data in Post-training Quantization and Pruning. ACL 2024 (arXiv:2311.09755).

[W1-1b & Q2] Do variation from data sampling and variation from data ordering contribute differently to final model quality?

We thank the reviewer for this question. Yes. Sampling changes the data, so it can move the average quality. Ordering leaves the data and the objective unchanged, so it adds only spread around that average, and this spread is part of what is measured as Sampling variability.

Sampling moves the average. Hessian-based quantization minimizes ∥WX−W^X∥2\|WX-\hat{W}X\|^2, and the calibration data enter through H=2XX⊤H=2XX^\top. Changing the samples changes XX, and therefore the objective. To quantify this, we calibrated with three data-curation methods, COLA [1], Self-calibration [2], and DPQ [3], in addition to random sampling (Table R6). Depending on the calibration set, the mean accuracy differs by up to 5.0 / 5.5 pp (Qwen3-0.6B / Llama-3.2-1B).

Ordering adds spread around each average. For each calibration set in Table R6, we quantized the model 10 times, accumulating the same samples in a different order each time. The standard deviation of the 8-task mean accuracy across these orders, σord\sigma_{\text{ord}}, is between 0.44 and 0.77 for every set, and the stabilized accumulation reduces it to zero.

Separating the two. A Sampling run draws one calibration set and uses one accumulation order of it. Its variance is therefore the sum of two components: the variance of the order-averaged accuracy across sets, and the average Ordering variance within a set (law of total variance). Writing accuracy as Q(S,π)Q(S,\pi), where SS is the calibration set and π\pi its accumulation order,

Var(Q)=VarS(Eπ[Q∣S])⏟Vset+ES[Varπ(Q∣S)]⏟Vord.\mathrm{Var}(Q)=\underbrace{\mathrm{Var}_S\left(\mathbb{E}_\pi[Q\mid S]\right)}_{V_{\mathrm{set}}}+\underbrace{\mathbb{E}_S\left[\mathrm{Var}_\pi(Q\mid S)\right]}_{V_{\mathrm{ord}}}.

Using four calibration sets with ten accumulation orders each (Table R7), the within-set ordering component VordV_{\mathrm{ord}} accounts for 77.3% / 82.5% of the variance for GPTQ and 94.5% / 78.1% for GPTAQ (Qwen3-0.6B / Llama-3.2-1B). These shares are estimated from four calibration sets.

Implication for comparing calibration sets. On Qwen3-0.6B, random sampling and COLA differ by 0.35 pp in mean accuracy, which is less than σord\sigma_{\text{ord}} (0.58–0.67). A single run of each therefore cannot establish which calibration set is better.

We have added Table R6 and this analysis to the new App. B.3, and Table R7 to App. B.2 of the revised manuscript.

Table R6. Mean accuracy and Ordering variability under different calibration sets (GPTQ INT3). Mean 8-task accuracy and σord\sigma_{\text{ord}} (in percentage points) over 10 accumulation orders.

Calibration setHow the data are chosenQwen3-0.6B: accuracy / σord\sigma_{\text{ord}}Llama-3.2-1B: accuracy / σord\sigma_{\text{ord}}
Random WikiText-2 (paper)random37.78 / 0.5843.99 / 0.77
COLA [1]activation clustering37.43 / 0.6746.08 / 0.67
Self-calibration [2]model-generated text36.56 / 0.4449.51 / 0.72
DPQ [3]task examples the model is uncertain about41.61 / 0.6346.90 / 0.74
Any of the above, with stabilized accumulation–σord\sigma_{\text{ord}} = 0σord\sigma_{\text{ord}} = 0

Each curation method follows its original protocol.

Table R7. Variance components of 8-task mean accuracy (INT3, four sets × ten orders). Standard deviations are in percentage points; the ordering share is Vord/(Vset+Vord)V_{\mathrm{ord}}/(V_{\mathrm{set}}+V_{\mathrm{ord}}).

ModelMethodBetween-set SDWithin-set ordering SDOrdering share
Qwen3-0.6BGPTQ0.3380.62477.3%
Qwen3-0.6BGPTAQ0.1820.75794.5%
Llama-3.2-1BGPTQ0.3700.80382.5%
Llama-3.2-1BGPTAQ0.4150.78578.1%

[1] B. He, L. Yin, H. Zhen, S. Liu, H. Wu, X. Zhang, M. Yuan, and C. Ma. Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization. Advances in Neural Information Processing Systems 38 (NeurIPS 2025), pp. 58531–58572, 2026 (arXiv:2510.10618).
[2] M. Williams, G. Chrysostomou, and N. Aletras. Self-calibration for Language Model Quantization and Pruning. NAACL 2025 (arXiv:2410.17170).
[3] Z. Yang, S. Hou, K. Zheng, Y. Liu, L. He, Y. Chen, and K. Cui. Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models. Findings of EMNLP 2026, pp. 14073–14093.

[W1-2] Ordering variance is below sampling variance, ten runs cannot separate the two, and perplexity barely moves. Is the accuracy variation small-task evaluation noise?

We thank the reviewer for these careful points. The models are not near-equivalent, and the variation is not explained by small-task noise. Two models that differ only in accumulation order give different answers on about 20% of the evaluation questions, as many as two models calibrated on different sets (Table R8). We address the four points in turn.

Magnitude. σord\sigma_{\text{ord}} is below σsamp\sigma_{\text{samp}} in 34 of the 48 configurations in the paper. This is expected, because Ordering variability is one component of Sampling variability (see our response to [W1-1b & Q2]).

Number of runs. We repeated the comparison with 50 Ordering runs and 50 Sampling runs. σord\sigma_{\text{ord}} is 0.60 / 0.75, close to the 0.58 / 0.77 obtained from 10 runs, and it is 75% / 80% of σsamp\sigma_{\text{samp}} (Qwen3-0.6B / Llama-3.2-1B, GPTQ INT3; see our response to [Q1]).

Perplexity. Perplexity barely moves under Sampling as well. In Table 1 of the paper (Llama-3.1-8B, GPTQ INT3), the standard deviation of WikiText-2 perplexity is 0.025 under Ordering and 0.026 under Sampling. Perplexity is insensitive to both sources, and a small change in perplexity does not show that two models behave the same [5].

Evaluation noise. The models differ substantially. Between two orders, 14.9% of integer weights differ (Llama-3.2-1B, §4.3). At the level of answers, we compared every pair of models within the Ordering runs and within the Sampling runs over 25,618 questions from the eight tasks, following [6]. On 19.4% / 20.9% of the questions, exactly one of two Ordering models is correct, against 20.1% / 21.9% for two Sampling models (Table R8). As for task size, σord\sigma_{\text{ord}} does not decrease as the number of questions increases (see our response to [Q3]).

Table R8. Answer flips between quantized models (GPTQ INT3). A flip is a question on which exactly one of the two models is correct. Mean (range) over the 45 pairs among the 10 Ordering models or the 10 Sampling models.

Models comparedQwen3-0.6B flips (%)Llama-3.2-1B flips (%)
Same set, different order (45 pairs)19.39 (17.76–21.33)20.90 (17.89–23.52)
Different sets (45 pairs)20.05 (16.48–23.25)21.91 (17.43–26.52)

[5] A. Jaiswal, Z. Gan, X. Du, B. Zhang, Z. Wang, and Y. Yang. Compressing LLMs: The Truth is Rarely Pure and Never Simple. ICLR 2024.
[6] A. Dutta, S. Krishnan, N. Kwatra, and R. Ramjee. Accuracy is Not All You Need. NeurIPS 2024.


[Q1] Do ordering and sampling variances differ significantly under statistical tests?

We thank the reviewer for suggesting a statistical test. Ten runs have little power to compare two variances, so we ran 50 Ordering runs (one fixed calibration set, a different order each time) and 50 Sampling runs (a different set each time) for GPTQ at INT3 (Table R9).

σord\sigma_{\text{ord}} is smaller than σsamp\sigma_{\text{samp}} on both models, at 75% / 80% of it (Qwen3-0.6B / Llama-3.2-1B). By a two-sided F-test for equal variances, the difference is significant at the 5% level on Qwen3-0.6B (p=0.046p = 0.046) and not on Llama-3.2-1B (p=0.126p = 0.126).

We do not claim that the two variances are equal. Ordering variability is one component of Sampling variability (see our response to [W1-1b & Q2]), so it is expected to be smaller, as observed on both models. Our claim is that it is large enough to affect comparisons between methods ([W1-1a]).

Table R9. Ordering and Sampling variability over 50 runs each (GPTQ INT3). Standard deviations of the 8-task mean accuracy, in percentage points.

Modelσord\sigma_{\text{ord}}σsamp\sigma_{\text{samp}}Ratiopp (F-test)
Qwen3-0.6B0.600.810.750.046
Llama-3.2-1B0.750.940.800.126
Reviewer_QQpiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the impact of floating-point accumulation order on Hessian-based post-training quantization (PTQ). The authors show that changing only the accumulation order of a fixed calibration set can lead to different Hessian estimates, quantized weights, and downstream performance. They further analyze how small numerical perturbations are amplified during Hessian inversion and affect quantization decisions. Experiments across Qwen and Llama models, four PTQ methods, and 3/4-bit quantization demonstrate that accumulation-order variability can be comparable to calibration sampling variability. The authors also apply Neumaier compensated summation to reduce this instability with relatively low computational overhead.

Critical strengths
  1. An important and overlooked problem. The paper shows that accumulation order alone can affect quantization results, even with a fixed calibration set. This is relevant to the reproducibility and fair comparison of Hessian-based PTQ methods.

  2. Solid experimental validation. The authors evaluate multiple models, quantization methods, and bit widths. The numerical analysis also provides useful evidence connecting Hessian perturbations to changes in quantized weights, while Neumaier summation offers a practical way to reduce the observed variability.

Critical weaknesses
  1. Limited methodological novelty. The proposed stabilization relies on Neumaier compensated summation, a well-established numerical technique. The main contribution is therefore the identification and analysis of the problem rather than a new algorithm.

  2. Incomplete analysis and baselines. While the paper demonstrates that small Hessian perturbations can lead to substantial changes in quantized weights, the amplification mechanism is not fully characterized. Comparisons with alternatives such as Kahan or pairwise summation are also missing, making it difficult to assess whether Neumaier is the most suitable solution.

Questions for authors
  1. Can the authors further analyze how small Hessian perturbations lead to changes in a substantial fraction of quantized weights? In particular, how much of this effect comes from local GPTQ error compensation versus error propagation across layers?

  2. How does Neumaier compare with simpler alternatives such as Kahan or pairwise summation in terms of stability, runtime, and memory overhead? Would these alternatives achieve similar results?

  3. Why does compensation outperform the FP64 control? Please list the precision used for matrix multiplication, accumulation, normalization, and solving, and check differences before and after conversion to FP32. Please also explain the effects of the other numerical changes in GPTAQ and BoA/TurboBoA. This would show whether the benefit goes beyond higher-precision accumulation.

7 comments
AuthorsOpenReview ↗

Official Response to Reviewer QQpi

[W1] Limited methodological novelty: Neumaier summation is a well-established technique, so the contribution is the analysis rather than a new algorithm.

We thank the reviewer for this comment. Our work does not aim to propose a new method. As the reviewer notes, its main contribution is the identification and analysis of an important problem that has been overlooked: changing only the accumulation order of a fixed calibration set changes the quantized model and its downstream performance in Hessian-based PTQ.

Neumaier compensated summation is, as the reviewer points out, a well-established technique (§2.3). What we show is that this existing technique, combined with a few implementation changes (a single final normalization, scaling after the matrix product, and a fixed reduction order; see [Q3-3]), is a simple way to reduce the order-induced variability.

We also compared this stabilized accumulation with Kahan summation, pairwise summation, and FP64 accumulation. None of the four removes the variability in every case, but the stabilized accumulation leaves variability in the fewest cases ([W2 & Q2], [Q3-1, 2]).

[W2 & Q1] The amplification mechanism is not fully characterized. How much of the change in quantized weights comes from local GPTQ error compensation versus error propagation across layers?

We thank the reviewer for this question. The change comes almost entirely from propagation across layers. Within a single layer, the effect is very small.

The 14.9% in §4.3 comes from complete quantization runs, so it mixes the two sources. We separated them with two additional experiments (GPTQ INT3, the calibration set of the paper, the five pairs of orders in App. C.4; Table R1).

  • Local effect. We quantize every layer (transformer block) from the same input activations of one reference run (order seed 777) under both orders, so only the accumulation order differs and nothing is inherited from earlier layers. This changes only 0.02% / 0.04% of integer weights (Qwen3-0.6B / Llama-3.2-1B). This number already includes GPTQ's error compensation within the layer. This error compensation passes each rounding error only to later columns of the same row (Eq. 6), so a change stays inside its row: fewer than 1% of rows contain any changed weight.
  • Propagation. We use standard FP32 accumulation, which depends on the order, in a single layer, and the stabilized accumulation, which gave identical weights across orders in our runs, in all other layers. Every difference therefore starts from that one layer. We placed this layer early, in the middle, or late in the model (Table R1). Among the weights in that layer and all later layers, 15.2% / 15.1% differ when the layer is early, 14.7% / 13.4% when it is in the middle, and 11.1% / 9.9% when it is late (Qwen3-0.6B / Llama-3.2-1B). The early and middle values are close to the 15.5% / 14.9% of complete runs. The late value is lower, as few layers follow it.

With the stabilized accumulation in every layer, no integer weight differs. We have added these results as the new App. C.6 in the revised manuscript.

Table R1. Percentage of integer weights that differ between two runs that use different accumulation orders (GPTQ INT3). "From that layer on" counts only the weights in the layer that uses standard FP32 and in all later layers.

Accumulation usedQwen3-0.6B: whole modelQwen3-0.6B: from that layer onLlama-3.2-1B: whole modelLlama-3.2-1B: from that layer on
Local: standard FP32 in each layer0.022–0.039–
Propagation: standard FP32 only at an early layer (layer 2 / 2)14.0815.1613.1715.05
Propagation: standard FP32 only at a middle layer (layer 14 / 8)7.3514.706.7213.43
Propagation: standard FP32 only at a late layer (layer 21 / 12)2.7811.112.489.92
Complete run: standard FP32 in every layer15.46–14.94–
Stabilized accumulation in every layer0–0–

Each value is δint\delta_{\text{int}} (Eq. 7), averaged over the five pairs of orders in App. C.4, except the complete-run row, which uses the five pairs of §4.3. Layers are zero-indexed, given as Qwen3-0.6B / Llama-3.2-1B (28 and 16 layers). The layers before the one that uses standard FP32 are identical in the two runs, so the whole-model value is smaller when that layer is placed later.

[W2 & Q2] How does Neumaier compare with Kahan or pairwise summation in stability, runtime, and memory?

We thank the reviewer for this suggestion. In our comparison, only Neumaier made all 10 orders produce the same quantized model. Kahan costs about the same as Neumaier but did not remove the variability. Pairwise summation did not remove it either, and needs 2.4× the peak memory.

We compared the four variants for GPTQ on Llama-3.1-8B at INT3 (Table R2). All variants use the same FP32 Hessian contributions, the same scaling, and a single normalization after accumulation, so they differ only in how the contributions are added. Kahan is fused into a single kernel like Neumaier, and time and memory are measured as in Table 15.

  • Stability. With Neumaier, the 10 orders give identical quantized weights (σord=0\sigma_{\text{ord}} = 0). With Kahan and pairwise summation, the 10 orders give 10 different models, as with standard FP32, and σord\sigma_{\text{ord}} is of the same size (0.39 and 0.75, against 0.58). As shown in our response to [W2 & Q1], a difference in a few weights spreads through the later layers, so the models differ as long as any difference in the Hessian remains.
  • Why the alternatives fall short. A known limitation of Kahan summation is that its correction is not exact when an incoming term is larger than the running sum, which Neumaier's variant handles (Neumaier, 1974). Pairwise summation slows the growth of rounding error but does not correct it, and although the shape of its tree is fixed, which samples are paired still follows the input order.
  • Cost. Kahan and Neumaier each keep one extra buffer. Both add about 7% to the accumulation time and use the same peak memory (5.43 GiB, against 4.79 GiB for standard FP32). Pairwise summation must keep up to 8 partial sums for 128 samples, so its peak memory is 2.4× that of standard FP32.

We have added this comparison as the new App. D.2 in the revised manuscript and revised the corresponding limitation in App. E.2.

Table R2. Summation alternatives for GPTQ on Llama-3.1-8B at INT3: Ordering variability and cost.

Summationσord\sigma_{\text{ord}}Distinct models (of 10 orders)Accumulation timePeak memory (GiB)
Standard FP320.5810233.1 ± 0.4 s (1.00×)4.79
Kahan0.3910250.1 ± 2.8 s (1.07×)5.43
Pairwise (tree)0.7510233.6 ± 0.6 s (1.00×)11.49
Neumaier0.001249.0 ± 2.5 s (1.07×)5.43

σord\sigma_{\text{ord}} is the standard deviation of the 8-task average accuracy over the 10 orders. Time is mean ± standard deviation over 3 runs.

[Q3-1, 2] Why does compensation outperform the FP64 control? List the precision of each operation and check differences before and after the cast to FP32.

We thank the reviewer for this question. FP64 accumulation is more precise, but its result is still cast to FP32, and tiny order-dependent differences in the FP64 Hessian can survive that cast. This is why FP64 retains variability. For BoA and TurboBoA, the FP64 control in the paper already matched the stabilized runs, so those results stand as reported. For GPTQ and GPTAQ it did not, and we have now matched it. The GPTQ results are essentially unchanged, and the GPTAQ variability disappears on the two models (Qwen3-0.6B and Llama-3.2-1B).

Precision of each operation. The FP64 control uses the following precision. Standard FP32 and the stabilized runs use FP32 in every step (App. A.2).

  1. Activation conversion: BF16 → FP64
  2. Matrix multiplication: FP64
  3. Scaling: FP64
  4. Accumulation: FP64
  5. Hessian cast: FP64 → FP32
  6. Normalization: FP32
  7. Solving: FP32

Implementation changes. Besides compensation, our stabilized implementations make method-specific changes that reduce rounding outside the summation itself (App. D.3). Two of them concern GPTQ and GPTAQ.

  • Scaling: compute 2XX⊤2XX^\top instead of (2X)(2X)⊤(\sqrt{2}X)(\sqrt{2}X)^\top, which avoids the rounding caused by the inexact value of 2\sqrt{2}.
  • Normalization: add up all samples and divide once, instead of rescaling a running mean after every batch.

BoA and TurboBoA additionally use a fixed reduction order when undoing the rotation (Reduction).

What the FP64 control in the paper used. For BoA and TurboBoA, the FP64 control already used the same settings as the stabilized implementations, so their results in Table 3 of the revised manuscript are unchanged. For GPTQ, it lacked Scaling. For GPTAQ, it lacked both Scaling and Normalization.

Matching the control (Table R3). For GPTQ, adding Scaling gives σord\sigma_{\text{ord}} of 0.00 / 0.29, against 0.01 / 0.33 in the paper (Qwen3-0.6B / Llama-3.2-1B). For GPTAQ, adding both changes gives 0.00 / 0.00, against 0.48 / 0.80.

FP64 with matched settings. There are 16 cases in which FP64 and the stabilized runs use the same settings: four GPTQ and GPTAQ cases on two models in Table R3, and 12 cases in Table 3 of the revised manuscript (GPTQ and GPTAQ on the other two models, and BoA and TurboBoA on all four models). FP64 leaves nonzero σord\sigma_{\text{ord}} in 10 of them (three for GPTQ, two for GPTAQ, three for BoA, and two for TurboBoA), and the stabilized runs in 2 (none for TurboBoA). FP64 is the more precise accumulation, yet the variability remained, so the benefit of compensation is not explained by higher precision alone.

Where the FP64 variability comes from. We examined the remaining GPTQ case on Llama-3.2-1B, with both changes applied, for two orders of the same calibration set (seeds 777 and 5000). We compared the Hessians of the two orders just before and just after the cast to FP32. Layer 8 is the first layer in which the FP32 Hessians differ (Table R4). Before the cast, the largest difference in its down_proj Hessian is only 2.44×10−42.44 \times 10^{-4} ULP, where ULP is the gap between two neighboring FP32 numbers. After the cast, 2 of its 819228192^2 entries differ by 1 ULP. This happens when a value lies right at the boundary between two FP32 numbers, so the two orders round it in different directions.

This is a concrete path by which order dependence reaches the FP32 computations that follow, and our response to [W2 & Q1] shows that a change in one layer spreads to the later layers. With the stabilized accumulation in the same setting, the quantized weights are bit-identical across all ten orders (§5.2).

Table R3. Ordering variability (σord\sigma_{\text{ord}}) of the FP64 control with and without the two changes (INT3, 10 orders).

MethodChanges appliedQwen3-0.6BLlama-3.2-1B
GPTQNeither0.590.75
Scaling only0.310.88
Normalization only (paper)0.010.33
Both (same as stabilized)0.000.29
GPTAQNeither (paper)0.480.80
Scaling only0.700.74
Normalization only0.210.89
Both (same as stabilized)0.000.00

Table R4. Hessian differences between order seeds 777 and 5000 in layer 8 (Llama-3.2-1B, GPTQ INT3, FP64 with both changes).

ModuleBefore cast: largest difference (ULP)After cast: entries that differ
q/k/v (shared)1.12×10−81.12 \times 10^{-8}0 of 204822048^2
o_proj1.53×10−51.53 \times 10^{-5}0 of 204822048^2
gate/up (shared)2.98×10−82.98 \times 10^{-8}0 of 204822048^2
down_proj2.44×10−42.44 \times 10^{-4}2 of 819228192^2 (both by 1 ULP)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.