acceptodds
Under review as a conference paper at ICLR 2027

Byte-Constrained Layer-Wise Quantization with Cost-Normalized Sensitivity

Abstract

Mixed-precision post-training quantization allocates storage across tensors of different sizes and sensitivities, but nominal bit width omits format metadata. We instantiate a byte-constrained multi-choice knapsack formulation with measured sensitivity and runtime-supported formats, then allocate precision using quality gain per byte. Across five models from three families, the resulting GGUF artifacts lower perplexity relative to six production llama.cpp presets at 25 of 30 matchedsize points. Paired block-bootstrap intervals resolve 23 improvements and one regression. Controlled ablations separate storage accounting from ranking quality. Uncorrected nominal budgets overshoot file-size targets by 10.6% to 17.9%, but nominal and byte pricing produce similar perplexity after size matching. In contrast, raw-sensitivity ranking is worse at 29 of 30 matched-size points, by up to 6.36 perplexity. Across six downstream tasks, Benjamini–Hochberg correction at 5% identifies 21 gains and three regressions among 180 comparisons, with 19 gains on models of at most 3B parameters. A same-model untying control attributes most shared-embedding sensitivity to the language-model head. These results distinguish accurate artifact sizing from effective precision allocation, while exposing the limits of a separable sensitivity objective.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, post-training quantization, mixed-precision quantization, model compression, sensitivity-aware bit allocation

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_8ZcsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a mixed-precision quantization method for running large language models on device. It estimates each tensor’s sensitivity by quantizing it individually and measuring the resulting KL divergence in the model’s output. It then selects precisions to minimize the total estimated sensitivity within a storage budget, using a Lagrangian relaxation followed by greedy precision upgrades. The method uses formats supported by llama.cpp and iteratively builds and measures the GGUF file, adjusting the allocation to meet the target size. Its clearest contribution is better model quality than llama.cpp’s production presets, especially for small models under tight storage budgets. However, the experiments provide less convincing evidence that the allocation strategy improves on existing mixed-precision methods.

Critical strengths
  1. Practical relevance and executable implementation. The paper addresses storage-constrained on-device LLM inference and produces executable GGUF artifacts compatible with llama.cpp.

  2. Improvements over production presets. The approach improves perplexity at 25 of 30 closely matched storage points, with particularly substantial gains for smaller models at tight budgets. Evaluation on additional corpora supports transfer beyond the calibration domain.

  3. Careful statistical evaluation. The paper reports paired confidence intervals and multiple-comparison corrections, helping distinguish meaningful improvements from small fluctuations.

Critical weaknesses
  1. Limited methodological novelty. The core optimization principles are established in prior literature. Reference [1] formulates mixed-precision quantization as a multiple-choice knapsack problem, while HAWQ-V3 [2] optimizes additive per-layer sensitivities under model-size, latency, and computation constraints, with executable hardware deployment. Although the llama.cpp implementation and GGUF-based empirical analysis are useful, the paper does not sufficiently establish what methodological advance or generalizable insight it adds beyond these existing approaches.

  2. Missing comparisons with established mixed-precision allocators. The main evaluation compares against llama.cpp presets. Although the appendix includes GPTQ and AWQ comparisons, these do not directly evaluate the proposed allocation strategy against existing mixed-precision allocators. The paper cites HAWQ-V2, APTQ, LLM-MQ, SliM-LLM, and AMQ, but does not evaluate representative methods under controlled settings. Comparisons using the same quantization formats, calibration data, evaluation harness, and measured storage budgets would help establish the proposed allocator’s contribution.

  3. Limited demonstrated benefit from byte-based pricing. The NOMINAL-MATCHED ablation produces nearly identical perplexity on four of the five models and outperforms the proposed allocation at all six Qwen2.5-7B budgets. These results support byte accounting as a useful sizing mechanism, but provide little evidence that it improves allocation quality once final storage is matched.

[1] https://arxiv.org/pdf/2110.06554 [2] https://arxiv.org/abs/2011.10680

Questions for authors

See weaknesses.

Reviewer_oCfVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper allocates existing llama.cpp quantization formats to tensors under a measured GGUF file-size budget. It profiles isolated tensor perturbations using output KL, optimizes an additive quality-cost surrogate, then builds and retargets the artifact to match the requested size within 0.35%. Across 5 models and 6 budgets each, it improves perplexity over production presets at 25 of 30 matched-size points, with 23 improvements resolved by paired intervals. The largest gains occur at tight budgets on small models. Downstream tests produce 21 gains and 3 losses after correction. Controls examine nominal versus measured storage, sensitivity normalization, profiling data, and the different roles of a tied embedding/head tensor. The contribution is an executable allocation workflow and deployment study, with limited new optimization machinery.

Critical strengths
  1. The comparison measures complete executable artifacts at matched file size and shares importance-matrix calibration between methods. This addresses a practical constraint that nominal bit widths can misrepresent. The nominal-matched control also separates size accuracy from quality improvements (Fig. 1, Tbls. 2 and 4).
  2. Tight-budget improvements are substantial in some settings. For Llama-3.2-1B at the Q2_K_S target, perplexity falls from ~18.8 to ~11.3 at ~529 MiB. The paper reports weaker results and losses as well. Independent-span checks, other corpora, paired intervals, and corrected task tests make the empirical evidence more useful than a single aggregate win count.
  3. The same-model weight-untying control starts with identical logits and attributes 96.1% of the decomposed sensitivity to the output head in the tested setting (Sec. 8, Fig. 5c). This supports a more precise interpretation than calling a large shared embedding intrinsically fragile.
  4. The supplement contains substantive code for reproducing the workflow. Offline checks and recalculation from saved outcomes succeeded, although fresh model inference was not repeated.
Critical weaknesses
  1. The methodological advance over existing mixed-precision allocation is limited. The paper acknowledges that constrained allocation and additive sensitivity objectives are established, but does not compare against a published joint allocator within the same runtime, format palette, and file-size budget. RAW is an internal heuristic. The GPTQ/AWQ experiments use different harnesses and match effective width over selected tensors, so they do not resolve this allocation comparison (Sec. 9, Tbls. 17-18). A focused comparison or a sharper account of which deployment finding changes prior understanding would strengthen the contribution.
  2. The main RAW comparison changes both ranking and search. RAW greedily demotes tensors, whereas the proposed method uses Lagrangian initialization and filling. The fixed-search exponent sweep partially addresses this, but covers 1 model at 3 budgets and normalizes by parameter count rather than the actual bytes saved by each format transition (Sec. 7, Appendix P, Fig. 5). The remaining uncertainty is how much of the reported advantage comes from byte-normalized ranking itself. Holding the search fixed while comparing raw sensitivity, parameter normalization, and transition-specific byte normalization would isolate that claim.
  3. The broader quality benefit is less established than the perplexity results suggest. Of the 21 corrected task gains, 19 occur on models with at most 3B parameters, and improved perplexity can accompany a task regression. Calibration also uses disjoint character spans from the test split, rather than independently sampled training documents. The independent-span and other-corpus controls are useful, but do not test a fully refreshed calibration pipeline. The conclusions should retain this small-model, tight-budget scope and distinguish surrogate ranking from reliable task-quality prediction.
Questions for authors
  1. For the existing exponent-sweep model and budgets, how do raw sensitivity, parameter-normalized sensitivity, and sensitivity per actual byte saved compare when they use the same search procedure? Can this separate the normalization effect from the main RAW comparison's search change?
  2. Which parts of a cited published allocator such as AMQ or SliM could be evaluated with the existing sensitivities, format palette, and measured-size targets? A focused comparison on representative budgets, with any adaptation stated explicitly, would clarify the incremental allocation benefit.
  3. How sensitive are the tight-budget gains to using a training-split calibration sample and recomputing both the importance matrix and sensitivity map? The current independent-span experiment keeps the importance matrix fixed, so it does not answer this narrower pipeline question.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.