acceptodds
Under review as a conference paper at ICLR 2027

Theoretically Scaled Quantization: A Data-Free Analytical Framework for LLMs

Abstract

Large Language Models (LLMs) require efficient post-training quantization (PTQ) to reduce memory and computational costs while preserving model quality. Existing PTQ methods commonly rely on calibration data, activation statistics, Hessian information, or iterative optimization, introducing additional preprocessing overhead and scale storage. We propose Theoretically Scaled Quantization (TSQ), a fully data-free analytical PTQ framework that derives quantization parameters directly from pretrained weight statistics. We formulate the TSQ Distortion Objective, , an exact distribution-independent geometric distortion objective for finite multi-region quantization, and define the optimal geometric scaling factor through one-dimensional minimization over . We further derive exact region-wise absolute-error bounds that decrease geometrically with region index and a region-independent relative-error bound of for all non-innermost regions. Under the empirically observed near-Gaussian structure of pretrained LLM weights, we derive the TSQ Governing Optimality Equation, , which analytically characterizes the optimal scaling factor using the normalized dynamic range , eliminating calibration data, activation statistics, Hessian computation, and iterative optimization. Building on the regional distortion analysis, we derive a generalized TSQ distortion law that decomposes normalized distortion into the weight dynamic range, optimized geometric distortion, and quantization resolution. TSQ also employs a scale-efficient representation requiring only one per-row maximum and a single global scaling parameter, from which all thresholds and quantization step sizes are analytically derived, reducing scale storage from to . Experiments across OPT, Llama, and Qwen model families show that TSQ achieves competitive or lower perplexity than AWQ across the evaluated settings while requiring no calibration data, substantially less scale storage, and lower full-model quantization time. These results demonstrate that analytical geometric scaling can provide a data-free and scalable approach to LLM post-training quantization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post-Training Quantization, Large Language Models, Data-Free Methods, Analytical Methods, Geometric Scaling

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kNzzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes TSQ, a data-free multi-region quantization method. Based on regional distortion analysis, it derives a generalized distortion law that decomposes normalized distortion into weight dynamic range, optimized geometric distortion, and quantization resolution.

Critical strengths

None

Critical weaknesses

For W4 (g=32), the BPW is 4.625 for AWQ and would be even lower after rANS coding. This contradicts the paper's claim that "RTN, AWQ and TSQ are entropy-coded with rANS." Only TSQ appears to use rANS, making the comparison clearly unfair.

TSQ's claimed contribution of a "scale-efficient representation" comes at the cost of increasing the weight code bit width, which conflicts with the overall goal of reducing BPW.

Questions for authors

None

Reviewer_swpBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Theoretically Scaled Quantization (TSQ), a fully data-free analytical PTQ framework that answers this question by partitioning weight magnitudes into a sequence of geometrically spaced regions and deriving all quantization thresholds and step sizes directly from weight statistics. Grounded in classical high-resolution quantization theory, TSQ defines a distribution-independent normalized distortion objective and reduces the otherwise intractable multi-region partition problem to a single-variable minimization over a geometric threshold factor k. The authors formally prove that TSQ strictly dominates uniform quantization regardless of the underlying weight distribution, derive tight region-wise absolute and relative error bounds, and establish a generalized distortion law that decomposes quantization error into interpretable contributions from weight dynamic range, geometric partition structure, and bit resolution. Under a Gaussian approximation of LLM weight distributions—validated empirically across multiple architectures—they further derive a closed-form optimality equation that computes the optimal scaling factor from only the normalized weight dynamic range, replacing expensive per-row numerical search with a single scalar root-find and delivering speedups of multiple orders of magnitude. Beyond its theoretical foundations, TSQ improves storage efficiency by reducing scale storage from O(mn/g) to O(m), as all regional parameters can be reconstructed from per-row weight maxima and one global scaling factor. Experiments across Llama, OPT, and Qwen model families show that TSQ matches or outperforms AWQ in perplexity and zero-shot accuracy at matched effective bit-widths, with far faster full-model quantization and no requirement for calibration data, particularly at lower bit precision. Though the current evaluation is limited to small-to-medium decoder-only models and does not include end-to-end inference latency benchmarks with a fused kernel, the work establishes a rigorous, scalable foundation for data-free LLM quantization and demonstrates that analytical geometric scaling can serve as a viable alternative to calibration-based PTQ methods.

Critical strengths
  1. Unlike most existing post-training quantization methods that rely on heuristic search or data-driven calibration, TSQ formalizes the geometric partition problem as a single-variable optimization over a normalized distortion objective, accompanied by complete proofs of its dominance over uniform quantization, asymptotic convergence properties, and tight region-wise error bounds. The derived generalized distortion law further decomposes quantization error into three interpretable components, providing a principled, analytical foundation rather than an empirical basis for low-bit LLM quantization.

  2. By reconstructing all regional quantization parameters from only per-row weight maxima and one global scaling factor, TSQ reduces scale storage complexity from �0�to�1��0� to �1� relative to conventional group-wise quantization methods. Moreover, the closed-form Gaussian optimality equation eliminates the need for iterative parameter search and calibration data passes, cutting full-model quantization time by up to 62% compared to AWQ while matching or improving perplexity and zero-shot accuracy at equivalent effective bit widths. This combination makes it particularly well-suited for offline, privacy-sensitive, or resource-constrained deployment scenarios where calibration data is unavailable or prohibitively costly to obtain.

Critical weaknesses
  1. The work does not prove that geometric spacing constitutes the globally optimal partition structure for quantization error minimization; it only identifies the optimal threshold factor within this fixed functional family. Additionally, the Bennett high-resolution quantization assumption that underpins the distortion objective loses fidelity at extremely low bit widths, where quantization steps become large relative to local variations in weight density, weakening the analytical model’s predictive accuracy in ultra-low-bit regimes.

  2. The empirical validation is confined to small-to-medium LLMs and lacks generalization across broader architectures and model scales. All evaluated models are below 10 billion parameters, so the behavior of the optimal geometric factor and the resulting quantization quality remain untested for large-scale models (30B and above), where weight distribution statistics and normalized dynamic ranges often differ materially. Critically, the method is not validated on linear attention, state-space, or MoE architectures, such as RWKV, Mamba, or mixed-expert models, whose weight profiles deviate substantially from standard dense transformers, leaving the transferability of TSQ’s statistics-driven scaling strategy an open question.

  3. The Gaussian weight assumption, which underpins the closed-form optimality equation, is only cursorily validated via near-zero mean and negligible skewness. The paper provides no quantitative analysis of how the pronounced heavy-tailed kurtosis of real LLM weights biases the analytically derived k away from the exact optimum. Likewise, the per-row parameterization is presented as a natural design choice without ablation or justification against finer-grained group-wise partitioning, and the validity bounds of the Bennett assumption are never explicitly delineated.

Questions for authors
  1. How much systematic bias does the heavy-tailed kurtosis of real LLM weights introduce to the analytically derived optimal k from the Gaussian assumption?

  2. Under what combinations of bit width and region count does the Bennett high-resolution quantization assumption break down and lose predictive accuracy?

  3. How do you expect the optimal geometric factor k and TSQ performance to scale to models above 30B parameters?

  4. Would TSQ’s geometric scaling strategy remain effective on architectures with very different weight statistics, such as linear attention or MoE models?

  5. How does the accuracy-storage tradeoff change if TSQ switches from row-wise to finer group-wise geometric partitioning?

Reviewer_AwBROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Theoretically Scaled Quantization (TSQ), a data-free post-training quantization method that partitions LLM weights into geometric regions and analytically determines region-specific quantization scales using weight statistics. It further employs rANS entropy coding to reduce storage overhead and compares with AWQ

Critical strengths

The calibration-free quantization formulation is potentially useful, as it reduces preprocessing overhead by deriving quantization parameters directly from weight statistics.

Critical weaknesses

The proposed 4-bit quantization requires an additional 2-bit region index for each weight, resulting in a 6-bit fixed-length representation. The authors introduce rANS entropy coding to reduce this overhead, but rANS is an orthogonal compression technique rather than an inherent advantage of the proposed quantization method. Conventional 4-bit PTQ methods do not require such an additional compression stage to achieve 4-bit storage.

Given the abundance of competitive PTQ methods, the experimental evaluation is insufficient. The paper primarily compares against AWQ and RTN, without considering several other relevant methods.

Although the authors acknowledge that rANS decoding costs are not evaluated, this is a fundamental limitation of the proposed approach. Supporting rANS decoding and region-dependent dequantization introduces additional implementation complexity compared with conventional low-bit quantization. Without an actual inference implementation or latency and throughput measurements, it is unclear whether the proposed method offers any practical advantage over existing PTQ methods.

Given these concerns, I am not convinced that the proposed approach establishes a competitive or practical alternative to existing PTQ methods.

Questions for authors

How does the proposed method compare with existing 6-bit weight-only PTQ methods, given that its 4-bit quantization requires an additional 2-bit region index per weight before entropy coding?

How does the proposed method compare with GPTQ, GPTAQ, and HQQ (calibration-free) at comparable effective bitwidths?

Many existing frameworks already support substantial GPU inference acceleration for conventional 4-bit weight-only quantized models. What would the proposed method's end-to-end or at least kernel-level inference performance look like, considering the additional rANS decoding and region-dependent dequantization overhead?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.