- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes AMPCQ, which assigns matrix-specific additive-codebook tiers under a model-level bit budget. It uses activation-normalized profiling, Pareto filtering, greedy return-per-bit allocation, and block-wise codebook recovery for quantization. AMPCQ improves quality over uniform AQLM allocation, but the formulation contains internal inconsistencies and the practical advantage over existing mixed-precision methods is not sufficiently established.
Critical strengths
- A practical allocation framework with strong gains over uniform codebook allocation and useful ablations. At similar reported bit widths, AMPCQ substantially improves perplexity and zero-shot accuracy over uniform AQLM on Llama3-8B and Qwen3-14B. The uniform-versus-mixed ablations also provide clear evidence that matrix-wise tier assignment is useful. Also, the combination of activation-normalized profiling, storage-aware allocation, and heterogeneous single- and dual-codebook tiers is practically motivated. The paper examines distortion metrics, allocation strategies, profiling-set size, and ROI sensitivity as well.
Critical weaknesses
-
The mathematical formulation is internally inconsistent. The main budget constraint treats as if it were a per-weight rate, whereas Appendix A.3 defines it as the total storage of matrix ; multiplying it again by matrix size therefore double-counts that size. The assumptions of Theorem A.2 are not enforced by Pareto filtering, and its discussion of multi-chain greedy allocation under uniform costs is incorrect. The Stage 3 adapter equations are also dimensionally invalid. These issues make it unclear whether the implemented allocator matches the stated optimization problem.
-
Novelty and practical efficiency are insufficiently established. The main contribution largely applies established mixed-precision allocation techniques to codebook tiers. Comparisons with related allocators are limited to adapted versions at one Llama3-8B operating point with Stage 2 disabled. Profiling and quantization costs are not reported, and the throughput advantage is inconsistent: AMPCQ improves batch-1 throughput but is substantially slower than AQLM at the maximum A100 batch setting. The main evaluation is also limited to two dense language models without repeated seeds.
Questions for authors
-
Budget implementation. Does the implementation treat as total storage or as bits per weight? After correcting the budget equation, do the tier assignments, reported average bit widths, and main results remain unchanged?
-
Full-pipeline comparison. Does AMPCQ retain its advantage over Q-Palette-, SliM-LLM-, and other mixed-precision allocation methods when all methods use the same codebook tiers, bit budget, and complete Stage 1–2 pipeline? Please also provide the exact allocation and optimization procedure.
-
Practical cost. What are the wall-clock time and peak memory required for profiling all tiers and quantizing the full model (regarding all resources required for the quantization phase itself)? Under matched batch size and memory constraints, does AMPCQ maintain a latency or throughput advantage over AQLM (regarding the inference phase of the quantized model)?
2 comments
Thanks for the rebuttal. However, I cannot easily follow the revised text. Could you highlight all the changed parts in the revised manuscript for ease of reference? Also, please reflect the additional results formally within the manuscript and refer them here. Hope that the results within the comment above is the essential part, which the full result would be contained within the paper.
We thank the reviewer for the detailed and rigorous feedback. We have addressed the concerns point-by-point with corrected formulations, full-model experiments, and honest cost reporting.
W1 & Q1: Mathematical Formulation and Implementation
W2 & Q2: Novelty and Full-Pipeline Comparison
We agree and have added full-pipeline (Stage 1 + Stage 2) comparisons on all 224 linear matrices of Llama3-8B under identical codebook tiers and budgets.
Q3: Practical Cost and Throughput