acceptodds
Under review as a conference paper at ICLR 2027

Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape

Abstract

Post-training quantization requires choosing a grid scale, and max-based choices can fail sharply at low bit-widths. For i.i.d. Gaussian weights and activations with sufficiently large effective rank, we prove uniform convergence of the normalized round-to-nearest loss to a Gaussian quantization objective. Its unique optimal scale decreases with the number of levels, while its relative-scale curvature decays approximately exponentially with bit-width. We verify the rank condition for wide random MLPs. Across five LLMs, GPTQ scale sensitivity likewise falls with bit-width: scale choice matters greatly at 2–3 bits but little from 6 bits onward. The Gaussian-optimal scale performs poorly on raw weights; after Hadamard incoherence processing, it matches the best searched rule from 3 bits onward but remains worse at 2 bits.

Primary area:
learning theory
Keywords:
post-training quantization, theory of quantization, quantization scale, low-bit quantization, reconstruction error landscape

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9gorOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies the effect of quantization scale on layer-wise reconstruction loss. The authors show that the normalized reconstruction loss can be approximated by a scalar Gaussian quantization objective under the assumptions that the weights follow a Gaussian distribution and Gram matrices of activations have sufficiently high effective ranks. They further analyze the loss landscape and show that scale sensitivity is significantly higher at low bit-widths. Experimental results on pretrained LLMs are consistent with the theoretical analysis.

Critical strengths

(+) The authors provide rigorous theoretical support for approximating the layer-wise normalized reconstruction loss with a Gaussian quantization objective under their assumptions.

(+) The curvature analysis of the Gaussian objective provides a theoretical explanation for why quantization performance is more sensitive to scale selection at lower bit-widths.

Critical weaknesses

[Contribution]
(-) While the paper provides a rigorous analysis of scale sensitivity under Gaussian weights and a high effective-rank condition, the main conclusion—that scale selection is more critical at very low bit-widths—appears largely intuitive and empirically expected. It is unclear whether the analysis provides a sufficiently strong new understanding of practical PTQ beyond confirming an already familiar trend.

[Validity]
(-) The authors have demonstrated that E(s)\mathcal{E}(s) converges to E∞(s)\mathcal{E}^{\infty}(s) for infinite values of dd. Nevertheless, it remains unclear how accurate this approximation is at practical model dimensions. Therefore, I think the authors need to measure the empirical differences between E(s)\mathcal{E}(s) and E∞(s)\mathcal{E}^{\infty}(s) on real model, e.g., llama, qwen, gemma.

(-) There is a substantial gap between the setting covered by the theoretical analysis and the pretrained LLM experiments. In particular, the effective-rank condition (H2) is verified only for randomly initialized MLPs under isotropic Gaussian inputs, whereas the experiments are conducted on pretrained transformers. As also acknowledged by the authors, it is not established whether the Gaussian-weight and effective-rank assumptions continue to hold after training. An empirical verification of (H2) on the evaluated pretrained models would strengthen the connection between the theory and experiments.

(-) As acknowledged by the authors, the theoretical results are derived only for RTN, whereas the main practical experiments use GPTQ, which performs sequential error compensation and is not covered by the analysis.

(-) The theoretical analysis is restricted to weight-only quantization. It remains unclear whether the derived scale-sensitivity behavior extends to weight-activation quantization settings, where activation quantization errors may substantially alter the reconstruction landscape.

[Presentation]
(-) While the main intuition of the paper is relatively simple, the presentation relies on a large number of propositions and theorems, making the paper unnecessarily difficult to follow. For example, the continuous extension from the discrete quantization level MBM_B to a real-valued mm appears primarily technical and could potentially be moved to the appendix, while the main text could focus directly on the bit-dependent Gaussian objective and its curvature.

Questions for authors

Please refer to the weaknesses. My primary concern is whether the paper offers sufficient novelty beyond formalizing and confirming the familiar trend that quantization scale selection is more critical at lower bit-widths. I would consider raising my rating if the authors can address my concerns.

3 comments

We thank the reviewer for the careful reading and constructive feedback, and for recognizing the rigor of our theoretical analysis and its explanation of scale sensitivity. Below, we clarify the reviewer's concerns.

  1. Contribution. We agree that the greater difficulty of low-bit quantization, and the importance of carefully selecting quantization parameters in this regime, are familiar observations. However, greater quantization error at low precision does not by itself determine how strongly the reconstruction loss responds to a relative perturbation of the scale. Our contribution is to provide a mathematically precise description of this dependence and identify conditions under which it is justified. The central result rigorously establishes that a high-dimensional, random, Hessian-weighted reconstruction landscape converges uniformly over scales to a deterministic scalar Gaussian objective under the stated assumptions. Identifying Gaussian scalar quantization as the uniform limit of this reconstruction landscape requires justification. This derivation provides an explicit object on which to study scale sensitivity. We characterize its optimal scale and derive its curvature with respect to relative scale perturbations, whose numerical evaluation reveals an approximately exponential decrease with bit-width. The analysis complements the qualitative intuition with concrete, quantitative predictions that go beyond the intuition alone and can be tested at finite widths. In particular, the Gaussian limiting description motivated us to investigate HIP preprocessing as a way to improve agreement with the theoretical prediction, which our experiments support. In the updated version, we further develop heuristics to examine this connection beyond initialization (Appendix A.7) and add quantitative experiments assessing when empirical GPTQ landscapes agree with the Gaussian prediction. See also our responses below on applicability after training and finite-dimensional accuracy.

  2. Finite-dimensional accuracy on trained models. We have added direct measurements of the discrepancy between empirical reconstruction landscapes and the Gaussian prediction in Appendix B.5.2. The experiments use Llama-3.2-1B and Llama-3.1-8B at W2 and W3, and they cover different depths and projection types, with input dimensions ranging from 20482048 to 1433614336. We compare the normalized GPTQ landscapes with the fixed Gaussian objective and variance-adjusted references motivated by the HIP heuristics in Appendix A.7. Figure 13 shows that HIP substantially reduces the measured landscape errors. After additionally accounting for GPTQ compensation-induced variance changes, the remaining discrepancies can be comparable to finite-Gaussian sampling errors (Table 9). These measurements provide empirical support for the approximation at practical dimensions beyond what the theoretical bound certifies. See our answer to Question 5 of Reviewer dPUS for the detailed setup and further results.

  3. Effective rank and applicability after training. We agree that verifying the assumptions for randomly initialized MLPs does not establish their validity for pretrained transformers. We have therefore added measurements of effective rank and its relationship with Gaussian approximation accuracy in Appendix B.5.3. However, the absolute effective-rank values should not be interpreted as a sharp finite-width criterion for Gaussian approximation. Condition (H2) is a sufficient asymptotic growth condition, and our estimates were designed to identify the limiting landscape rather than optimize finite-width constants. The reported mean effective ranks are 3.44/5.343.44/5.34 for Llama-1B and 4.07/8.624.07/8.62 for Llama-8B at W2/W3, respectively. HIP preserves these ranks for fixed incoming activations. Higher effective rank is associated with smaller RTN Gaussian approximation errors (Table 10). Diagonal rescaling and whitening also reduce the median landscape errors (Table 12), although these transformations change both the Gram spectrum and the weight coordinates, so they do not isolate the effect of effective rank. Together with the correlations in Table 10, these observations support effective rank as a relevant feature of the Gram matrix for understanding Gaussian approximation accuracy, even though its numerical value alone does not provide a sharp finite-width accuracy criterion. For GPTQ, we additionally measure the effective number of pivot weights,

=\frac{(\sum_i p_i)^2}{\sum_i p_i^2},$$ where $p_i$ are the coefficients weighting the squared compensated residuals in the GPTQ loss. Larger effective counts and smaller maximum normalized pivot weights are associated with smaller variance-adjusted landscape errors (Table 11). The effective count measures how evenly rounding steps are weighted, and these observations are consistent with the self-averaging motivation in Appendix A.7.
  1. Extension to GPTQ. Our rigorous convergence theorem concerns RTN, we provide now an heuristic argument justifying why the Gaussian landscape transfers to GPTQ after HIP. To address sequential compensation, Section 4.2 now develops a heuristic extension to GPTQ after HIP. Under structural assumptions on the Gram matrix, together with block self-averaging, the predicted limiting landscape is
\xrightarrow{\mathbb P} \mathcal L_s := \int_0^1 v_s(\tau)\ \mathcal E^\infty\left( M_B \frac{s}{\sqrt{v_s(\tau)}}\right)\ d\tau,$$ where $v_s(\tau)$ is the variance of the compensated inputs after processing a fraction $\tau$ of the coordinates. For weak weight–Gram alignment $\rho$ we get an expansion $$\begin{aligned} \mathcal L_s &=\sigma^2\mathcal E^\infty(M_B,s/\sigma) +O(\\|\rho\\|_2^2) =\bar v_s\\,\mathcal E^\infty\\!\left( M_B,\frac{s}{\sqrt{\bar v_s}}\right) +O(\\|\rho\\|_2^4), \qquad \bar v_s:=\int_0^1v_s(\tau)\\,d\tau. \end{aligned}$$ with the initial variance of the weights $\sigma^2$, and $\rho$ measuring the asymptotic dependence between the transformed coordinates and the corresponding update direction in the eigenspace of the Gram matrix. Thus, the Gaussian objective is recovered at zero alignment, while adjusting for the mean-compensated variance captures the leading correction at weak alignment. We also measured normalized alignment $\\|\widehat{\rho}_r\\|_2=\\|V_r^\top\theta\\|_2/\\|\theta\\|_2$ with the leading $r\in\\{16,32,64\\}$ Gram eigenvectors in Llama-1B at W3 with HIP (see Table 14). Across ranks and depths, layerwise medians range from $0.033$–$0.079$ for MLP down projections, $0.047$–$0.161$ for attention output projections, and $0.070$–$0.273$ for query projections. These measurements provide qualitative motivation for the weak-alignment expansion, particularly for down and output projections. This provides an intuition for why the Gaussian approximation can hold under GPTQ. The variance-adjusted landscape experiments provide empirical support (Appendix B.5.2, Figure 14 and Tables 8–9). We refer the reviewer to our answer to question 1 of Reviewer dPUS, and question 1 of Reviewer U87M for further details. 5. **Weight–activation quantization.** We agree that understanding how activation quantization influences scale sensitivity is an important extension. We focused first on weight-only quantization to isolate the effect of weight-scale selection, consistent with the original GPTQ formulation. This setting is also practically relevant: it reduces model memory requirements while retaining higher-precision activations, enabling deployment of larger LLMs on limited hardware, as demonstrated by GPTQ and AWQ. 6. **Presentation** Following the reviewer suggestion, we removed the continuous extension from Section 4.3. We are happy to implement any other suggestions the reviewer may have to improve the presentation.
AuthorsOpenReview ↗

Request to engage.

We kindly ask the reviewer to let us know if their concern has been resolved. If not, we are happy to discuss any remaining concerns. If there are no remaining doubts, we would appreciate the reviewer to consider updating their assessment.

Reviewer_U87MOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper studies the scale sensitivity of post-training quantization (PTQ) for large language models. The authors prove that, under a Gaussian weight assumption and the condition that calibration activations satisfy an effective-rank condition, the normalized RTN reconstruction loss converges uniformly with respect to the scale to an asymptotic Gaussian objective;

  • The authors validate this prediction on GPTQ. The paper introduces HIP pre-processing to make post-training weights approximate a Gaussian distribution; after HIP, the analytically derived optimal scaling factor from the theory can achieve performance comparable to that of heuristic scaling rules.

  • Overall, the paper builds a concise theoretical framework and rigorously connects it to real large-model quantization experiments. Scale sensitivity decaying exponentially with bit width is very appealing. However, there are several conceptual gaps between the theory and GPTQ that need to be clarified.

Critical strengths
  • The motivation for the proposition of scale-invariant curvature is clear.
  • This paper establishes a rigorous uniform convergence result between Gaussian scalar quantization and layer-wise PTQ, using curvature as the core quantity to explain scaling sensitivity;
  • Rather than relying only on an indirect metric such as perplexity, it directly probes the local loss landscape, which strengthens the paper’s persuasiveness.
Critical weaknesses
  • The authors state that the theory does not formally cover GPTQ. However, could some clearer discussion be added:
  • 1)Why can the curvature trend transfer?
  • 2)Is there a qualitative argument explaining that even with compensation, GPTQ’s local loss surface still inherits the same bit-dependent behavior?
  • At present, the main paper mainly relies on numerical evidence and lacks a deeper conceptual explanation.
  • Table 1 shows that the raw analytic scale completely fails for Llama-3.1-8B W3 GPTQ. While HIP fixes this, I want more discussion about the conditions under which HIP successfully aligns weights with assumption (H1). Is HIP universally applicable across different model families, architectures, or model sizes? Are there cases where HIP cannot bring weights close enough to Gaussian for the analytic scale to work?
Questions for authors

to see above weaknesses。

4 comments

We thank the reviewer for the careful assessment and constructive suggestions, and we are glad that the reviewer found the paper interesting. We address both concerns below.

  1. Why the Gaussian trend can persist under GPTQ compensation. GPTQ updates change the remaining weights and introduce dependencies, even at Gaussian initialization. Understanding the dependencies introduced by the training process is an additional unresolved problem. This motivates our rigorous focus on RTN at initialization.

    To develop intuition beyond this setting, we fix the trained weights and introduce randomness through HIP. Under the conditions in Section 4.2, the transformed coordinates have Gaussian limits but remain dependent. Inspired by statistical physics, we assume self-averaging: averages over many dependent coordinates concentrate around their expectations. This turns the discrete compensation updates into a discretization of an ODE whose continuum limit describes their accumulated effect (Appendix A.7.2). The resulting analysis suggests that the limiting behaviour is governed by an effective variance trajectory:

    \mathcal E^\infty\left(M_B,\frac{s}{\sqrt{v_s(\tau)}}\right)d\tau.$$ Thus compensation changes the effective variance, while a Taylor expansion around zero alignment gives $$\sigma^2\mathcal E^\infty(M_B,s/\sigma) +O(\\|\rho\\|^2).$$ Here, $\rho$ measures the asymptotic covariance between transformed weights and their corresponding Gram eigenspace coordinates. The correction vanishes when $\rho=0$, recovering the scalar Gaussian objective, and is of quadratic order for small alignment. We also measured normalized alignment $\\|\widehat{\rho}_r\\|_2=\\|V_r^\top\theta\\|_2/\\|\theta\\|_2$ with the leading $r\in\\{16,32,64\\}$ Gram eigenvectors in Llama-1B at W3 with HIP (see Table 14). Across ranks and depths, layerwise medians range from $0.033$–$0.079$ for MLP down projections, $0.047$–$0.161$ for attention output projections, and $0.070$–$0.273$ for query projections. These measurements provide qualitative motivation for the weak-alignment expansion, particularly for down and output projections. These conclusions rely on self-averaging for adaptive GPTQ updates, which remains unproved. The heuristic analysis is developed in Section 4.2 and Appendix A.7, particularly A.7.2. Supporting landscape experiments are provided in Appendix B.5.2, especially Figure 14 and Tables 8–9. Regarding the convergence of the curvature, we refer the reviewer to Answer 3 of Reviewer dPUS.
  2. Applicability of HIP. The principle behind HIP applies to any matrix–vector multiplication, since an orthogonal change of coordinates preserves the output: (WRd⊤)(Rdx)=Wx.(WR_d^\top)(R_dx)=Wx. It is therefore not restricted to a particular model family or architecture. Under the delocalization and moment conditions in Appendix A.7, each transformed coordinate converges to a Gaussian as the width grows. However, this marginal convergence does not establish independence between coordinates or between weights and activations; see our response to Reviewer dPUS, point 1. Thus HIP does not automatically establish (H1). Concentrated weight rows can violate the Gaussian approximation conditions, and Gaussian marginals alone do not guarantee the analytic scale’s accuracy, as the remaining W2 gap illustrates.

Thank you for your response. My concerns have already been addressed. (It is better to add these discussions in the next version. )

Therefore, I keep my assessment unchanged.

We thank the reviewer for the response, and we are happy to hear that all the reviewer's concerns have been properly addressed. We have already incorporated the rebuttal discussion into the main paper, marked in blue. We noticed now that the reviewer's official comment changed from "Thank you for your response. It is better to add these discussions in the next version. I think this article contains a novel idea, strict theoretical proof, and rich experimental results, which can inspire other researchers. Therefore, I keep my assessment unchanged." to the current version. We would like to ask for an explanation of what changed the reviewer's previous opinion, since no further discussion or concerns were raised.

Sorry!

I have kept my score and overall recommendation unchanged. I only removed one descriptive positive sentence from my review for conciseness. I still think that "the paper contains a novel idea, strict theoretical proof, and rich experimental results, which can inspire other researchers. "

Reviewer_dPUSOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper answers the question of how sensitive the layerwise reconstruction loss is to the choice of scale. The paper proves that the reconstruction loss converges to a Gaussian objective given some assumptions. Furthermore, it shows that the optimal scale is strictly decreasing as the resolution increases. The paper ends with an empirical section showing that the choice of scale matters a lot at 2-3 bits and barely matters at 6 and above, and that the Gaussian-optimal scale choice performs poorly on the raw weights but well after HIP is applied.

Critical strengths
  • The paper explores an interesting and relevant question around scale selection for post-training quantization with a theoretical grounding. The theoretical development appears involved and carefully thought out.
  • The empirical work that is provided is extensive, covering several LLMs, multiple scale-selection rules, and a range of bit-widths and granularities, with lots of experiments in the appendix.
Critical weaknesses
  1. The paper's theory assumes i.i.d. Gaussian weights, while PTQ is applied to trained models later. It seems that it only works well after Hadamard incoherence processing, as without it, the results degrade severely (Tables 6,7). Even with HIP, it is unclear to me that the trained weights satisfy the assumptions, since HIP applies the same rotation to both weights and activations. Therefore, the practicality of this is measured purely from the experiments without a bridge between the theory and reality.
  2. The paper's theory relies on an effective-rank condition, but I could not find measurements of effective rank for the models studied. As stated, the condition seems hard to satisfy at realistic layer widths. HIP does not change effective rank, so it is difficult to judge whether this assumption holds, even approximately, in the experimental setting.
  3. Theorem 1 establishes uniform convergence of the RTN loss, but the paper’s main explanation relies on the curvature of the limiting objective. Convergence of loss values does not imply convergence of curvature. Moreover, the empirical GPTQ sensitivity varies substantially with the perturbation window (Figure 3). It is unclear whether the agreement at the 5% window validates the theoretical curvature, beyond a qualitative trend with bit-width.
  4. Even when the assumptions hold, the finite-dimensional bound appears too loose to explain the experiments quantitatively. For example, at (d=4096) and W3, the stated error bound is approximately (4.57), while the minimum of the Gaussian objective is approximately (0.0469). The bound also grows with bit-width, whereas the sensitivity being studied decreases. I do not see how this guarantee establishes useful approximation accuracy at the dimensions used in the experiments.
Questions for authors
  1. Why is initialization the right model for a method applied after training? Is the theory best understood as describing HIP-processed weights specifically?
  2. Since HIP rotates both weights and activations the same way, does it make the weights approximately independent of the activations as it appears Theorem 1 requires? If not, which parts of the theory still apply after HIP?
  3. Could the authors report measured effective ranks of the calibration matrices for representative layers, before and after preprocessing, and compare them to log^8(d)?
  4. What connects the uniform convergence in Theorem 1 to the curvature comparisons in Section 5.3? Could the authors compare empirical and Gaussian losses using exactly the same perturbation windows, and explain how the agreement depends on the window size and layer width?
  5. Can the finite-dimensional bound be tightened enough to become informative at the experimental widths? Alternatively, could the authors report actual uniform approximation errors in controlled settings satisfying the assumptions at those widths, to assess how accurately the limiting objective describes finite-dimensional losses?
6 comments

We thank the reviewer for the valuable feedback and for recognizing the theoretical development and extensive experiments. We address each question individually.

Our Answers.

  1. Initialization, HIP, and applicability after training. The goal of the paper is to study the landscape of post-training quantization reconstruction losses as a function of the quantization scale. To make the problem tractable, we study the scale dependence of PTQ reconstruction losses at i.i.d. Gaussian initialization. This permits a rigorous asymptotic analysis under (H2), which controls ∣H∣op\\|H\\|_{\mathrm{op}} relative to tr⁡(H)\operatorname{tr}(H). Extending this description to trained weights requires a precise knowledge of how the training procedure would change the initialization distribution and the correlation between the components of the parameter vector θ\theta. This knowledge, as far as we know, represents a fundamental problem of the machine learning community beyond the scope of this paper. HIP nevertheless provides a motivation for using the Gaussian objective beyond initialization. Consider deterministic trained weights θd∈Rd\theta_d\in\mathbb{R}^d and Rd=FdSdR_d=F_dS_d, where Sd=diag⁡(ξ1,…,ξd)S_d=\operatorname{diag}(\xi_1,\ldots,\xi_d) contains independent Rademacher signs and FdF_d is an orthogonal Hadamard matrix with entries hik/dh_{ik}/\sqrt d. If ∣θd∣∞/d→0\\|\theta_d\\|_\infty/\sqrt d\to0 and ∣θd∣22/d→σ2>0\\|\theta_d\\|_2^2/d\to\sigma^2>0, then, for each fixed coordinate,
=\frac1{\sqrt d}\sum_{k=1}^d h_{ik}\xi_k\theta_{d,k} \xrightarrow{\mathrm d}Y\sim\mathcal N(0,\sigma^2).$$ However, different coordinates share the same random signs, so they are not independent. Therefore, this marginal Gaussian limit alone does not justify replacing empirical losses with Gaussian expectations. Inspired by self-averaging in statistical physics, we consider additional concentration assumptions for nonlinear averages such as $$\frac1d\sum_{i=1}^d\ell_s(Y_{d,i}) \xrightarrow{\mathbb P}\mathbb E[\ell_s(Y)], \qquad \ell_s(y):=\bigl(y-\Pi_{s,M_B}(y)\bigr)^2.$$ The Hessian-weighted loss requires further control of averages involving both the transformed weights and the Gram directions. This relates to the reviewer’s second question, **HIP does not generally make weights independent of activations**. Indeed, for a deterministic calibration column $x_d$, we get $$\mathbb E_{R_d}\\!\left[(R_d\theta_d)_i(R_dx_d)_i\right] =\frac{\theta_d^\top x_d}{d},$$ showing that a nonvanishing normalized overlap survives the transformation. Consequently, Gaussian-looking weight marginals after HIP do not suffice to apply Theorem 1. Motivated by this concern, Section 4.2 now develops a separate heuristic connection for trained weights after HIP. Under a fixed-rank perturbation model for the Gram matrix and additional self-averaging assumptions, the RTN limit contains an explicit weight–Gram alignment correction. For GPTQ, similar self-averaging assumptions lead to an average of Gaussian quantization losses with a variance that evolves during compensation. This is intuitive, as GPTQ compensation changes the weight distribution during its quantization pass. At weak weight–Gram alignment, this average can be approximated by a Gaussian objective adjusted to the mean compensated variance: $$\bar v_s\\,\mathcal E^\infty\\!\left( M_B,\frac{s}{\sqrt{\bar v_s}}\right), \qquad \bar v_s:=\int_0^1v_s(\tau)\\,d\tau.$$ The experiments in Appendix B.5.2 provide empirical support for these predictions; see in particular Figure 14 and Tables 8 and 9. We additionally measured the normalized alignment $\\|\widehat{\rho}_r\\|_2=\\|V_r^\top\theta\\|_2/\\|\theta\\|_2$ with the leading $r\in\\{16,32,64\\}$ Gram eigenvectors in Llama-1B at W3 with HIP (see Table 14). Across ranks and depths, layerwise medians range from $0.033$–$0.079$ for MLP down projections, $0.047$–$0.161$ for attention output projections, and $0.070$–$0.273$ for query projections. These measurements provide qualitative motivation for the weak-alignment expansion, particularly for down and output projections. Establishing the additional assumptions rigorously and testing their validity more broadly in trained networks remain open.
  1. Measured effective ranks. We have added an empirical analysis of effective rank in Appendix B.5.3. The experiments do not establish (H2) for trained models, but they confirm that higher effective rank implies better Gaussian approximation (Table 10). This is further supported by Table 12, where it is shown how diagonal rescaling and whitening further improve agreement with the Gaussian objective E∞\mathcal{E}^\infty, since they increase the effective rank. These experiments support the theory, identifying the effective rank as a relevant condition for understanding the Gaussian approximation. We agree that the stated bound does not quantitatively explain the observed accuracy at practical widths. Our analysis aims to identify the limiting landscape and establish uniform convergence for fixed bit-width; the estimates were not optimized for sharp finite-width accuracy. Computing the remaining constants alone would not necessarily resolve this limitation. Nevertheless, without such an analysis, it is unclear a priori whether the high-dimensional random reconstruction landscapes should approach a deterministic continuous objective, or what that objective should be. Identifying the limit provides a theoretically justified reference whose accuracy at finite widths can then be investigated experimentally. We have therefore added experiments to assess this accuracy beyond what the numerical bound currently certifies. See also our answer to question 5.

  2. Uniform convergence, curvature, and perturbation windows. We agree that uniform convergence does not imply pointwise curvature convergence. Even more, classical second derivatives are not defined at quantizer switching points. Nevertheless, uniform convergence does imply distributional derivative convergence and fixed-window sensitivity convergence, which is the notion of sensitivity analyse in the experiments. Appendix B.6.1 establishes these results and recovers Gaussian curvature by taking the width limit before shrinking the window. As requested by the reviewer, we added comparisons at matched perturbation windows (Appendix B.6.2, Figure 16). The population Gaussian landscape shows comparatively weak window dependence. To account for finite sampling, we generate an independent row with i.i.d. standard-Gaussian entries and the same input dimension dd as each corresponding LLM row, setting H=IH=I. We select the best searched scale for each Gaussian row and compare the resulting finite-window sensitivities with those of real GPTQ landscapes. These controls reproduce part of the window dependence, but not the strong dependence observed at low bit-widths. The experiments therefore support the common exponential decrease in sensitivity with bit-width, without establishing quantitative curvature agreement. Regarding layer width, we generally observe smaller variance-adjusted landscape errors in larger layers (Appendix B.5.2). However, width varies together with layer type and other structural properties, so these comparisons do not isolate its effect. Varying the dimension of a trained layer without introducing additional factors that could influence the agreement is not straightforward.

4. Finite-dimensional accuracy. We agree that the stated bound is too conservative to explain the experimental accuracy quantitatively. As remarked in our answer to question 2, our analysis aims to identify the limiting landscape. We therefore address the reviewer’s alternative by reporting approximation errors in controlled Gaussian settings and additional measurements on trained models. In Appendix B.5.1, Figure 11 reports maximum absolute landscape errors on the evaluated scale grid for i.i.d. Gaussian weights and independent identity or projector Gram matrices. We compare effective ranks dd, ⌈d⌉\lceil\sqrt d\rceil, ⌈10−5(log⁡d)8⌉\lceil10^{-5}(\log d)^8\rceil, ⌈log⁡d⌉\lceil\log d\rceil, and 88, over widths d∈4096,16384,65536d\in\\{4096,16384,65536\\}. The first three constructions show an overall decrease in error with width, while the logarithmic and fixed-rank settings retain substantially larger errors over the tested range. In Appendix B.5.2, we measure the Wasserstein distance between the weighted distribution of GPTQ-compensated weights and a Gaussian reference. This distance bounds the discrepancy between the corresponding normalized quantization losses. HIP substantially reduces the Wasserstein distance (Figure 12). If we furthermore adjust for the GPTQ-compensated variance, the reported distances are close to those of finite Gaussian samples of matching dimension. For these controls, we draw dd i.i.d. Gaussian values, where dd is the layer’s input dimension, and measure the Wasserstein distance between their empirical distribution and the generating Gaussian distribution. We average this control distance over 100 independent repetitions. The ratios to the Gaussian sampling controls range from 0.950.95 to 1.171.17 across the configurations in Table 8. We also directly measure maximum landscape errors on finite grids covering local and broad scale intervals. Figure 13 shows that HIP substantially improves agreement. As for the Wasserstein distance, accounting for the compensated variance separately at each scale further reduces the discrepancy: at W2, the broad-grid error relative to the Gaussian sampling control decreases from 40.7140.71 to 1.271.27 for Llama-1B and from 45.8145.81 to 1.481.48 for Llama-8B (Table 9). Figure 14 illustrates this improvement for a predetermined row at W2 and W3.

These comparisons show that, after accounting for compensation-induced variance changes, the remaining discrepancies can be comparable to finite-Gaussian sampling errors. They provide empirical evidence of finite-width accuracy beyond what the theoretical bound currently certifies.

Comments on the paper

2 comments
AuthorsOpenReview ↗

A kind request to engage

We thank the reviewers for their feedback. We responded to all questions raised by the reviewers and updated the paper following the rebuttal as follows:

  1. Added Section 4.2 and Appendix A.7 to discuss the applicability of our theory beyond initialization, using HIP preprocessing. We further provide a heuristic extension to the GPTQ loss;
  2. Added experiments and discussion in Section 5 and Appendices B.5–B.6 supporting the proposed theory, as suggested by the reviewers.
  3. Improved the presentation by moving the continuous extension to Appendix A.5.

All changed content is marked in blue. We would be happy to know whether the reviewers' concerns are properly addressed and if there are any remaining doubts. If our answers are satisfactory, we would appreciate it if the reviewers could consider updating their scores.

AuthorsOpenReview ↗

Summary of the rebuttal

We provide a summary of the rebuttal here. We believe we have addressed all the questions and concerns of the reviewers. In particular,

  1. We developed a heuristic explanation of why the Gaussian description extends beyond initialization and to GPTQ after HIP, supported by extensive new experiments. (Reviewers dPUS, U87M, 9gor.)

  2. We added experiments showing improved agreement with Gaussian predictions after HIP and variance adjustment, alongside measurements of effective rank and weight–Gram alignment. (Reviewers dPUS, 9gor.)

  3. We clarified the connection between uniform convergence and finite-window sensitivity, added matched-window comparisons, and improved the discussion of novelty and presentation. (Reviewers dPUS, 9gor.)

We are disappointed by the limited engagement with these revisions. Reviewer 9gor has not responded despite our follow-up. Reviewer dPUS stated that no new effective-rank measurements were provided, although these were explicitly reported, as were other questions that we explicitly addressed but were not recognised. Moreover, he states, for the first time on the last day of the rebuttal phase, that numerical and sign errors were present without identifying the affected equations. Reviewer U87M confirmed that their concerns were addressed.

Given the considerable effort we have put into the rebuttal, we are disappointed by how the process unfolded, regardless of the final decision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.