- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes a joint low-rank + sparsity + quantization weight compression strategy, where low-rank and sparsity factors are optimized via alternating optimization (Joint-SLR). Then one quantizes the sparse factor via EBQ (non-uniform quantization) and refits the final low-rank correction residual. The approach is validated on the Llama-2/Llama-3 model family on standard LM evals.
Critical strengths
- The proposed method achieves a quite strong quality/compression trade-off, consistently outperforming baselines from the literature. The closest competitor method, ApiQ [12], requires training, whereas Quench is training-free.
- The design choices of Quench—such as the choice of quantizer, order of the quantization operation, and number of iterations—are carefully ablated.
Critical weaknesses
- The idea lacks conceptual novelty. The ideas of alternating sparse + low-rank [1, 2, 3, 4], quantized + low-rank [5, 6, 7], and sparse + quantized [8, 9] appeared in a line of work. The main contribution appears to be the combination of existing techniques and the right choice of order.
- Evaluation involves only outdated models; Llama-2 and Llama-3 were released 3 and 2 years ago, respectively. Evaluation on more recent models (Qwen-3.x, Gemma-4) and reasoning tasks (GPQA-Diamond, AIME, MATH500) or agentic tasks would significantly strengthen the validation.
- There is a related work, ProjQ [10], claiming to achieve strong compression results (low-bit quantization + low-rank) with a moderate accuracy drop. I think it is worth being added to the comparison as a baseline. OBR [11] is also a natural baseline for the sparse + quantized setup.
[1] Stephen Zhang and Vardan Papyan. 2024. OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition. arXiv:2409.13652. Google Scholar.
[2] Mehdi Makni, Kayhan Behdin, Zheng Xu, Natalia Ponomareva, and Rahul Mazumder. 2025. HASSLE-free: A unified Framework for Sparse plus Low-Rank Matrix Decomposition for LLMs. arXiv:2502.00899. Google Scholar.
[3] Mehdi Makni, Xiang Meng, and Rahul Mazumder. 2025. 3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs. Advances in Neural Information Processing Systems, 38. Google Scholar.
[4] Zeliang Zong, Kai Zhang, Zheyang Li, Wenming Tan, Ye Ren, Yiyan Zhai, and Jilin Hu. 2025. 1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models. Findings of EMNLP 2025, pp. 14206–14220. Google Scholar.
[5] Rajarshi Saha, Naomi Sagan, Varun Srivastava, Andrea J. Goldsmith, and Mert Pilanci. 2024. Compressing Large Language Models using Low Rank and Low Precision Decomposition. NeurIPS 2024. Google Scholar.
[6] Yoonjun Cho, Soeun Kim, Dongjae Jeon, Kyelim Lee, Beomsoo Lee, and Albert No. 2025. Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition. Findings of ACL 2025, pp. 14453–14470. Google Scholar.
[7] Hyochan Chong, Dongkyu Kim, Changdong Kim, and Minseop Choi. 2026. NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models. arXiv:2602.06694. Google Scholar.
[8] Jing Liu, Toshiaki Koike-Akino, Ye Wang, Hassan Mansour, and Matthew Brand. 2025. AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descent. arXiv:2506.10205. Google Scholar.
[9] Xianglong Yan, Tianao Zhang, Zhiteng Li, Haotong Qin, and Yulun Zhang. 2025. Progressive Binarization with Semi-Structured Pruning for LLMs. arXiv:2502.01705. Google Scholar.
[10] Wenya Yu, Chao Zhang, Li Wang, Samson Lasaulce, and Merouane Debbah. 2026. ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression. arXiv:2606.00494. Google Scholar.
[11] Hang Guo, Yawei Li, and Luca Benini. 2025. Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs. arXiv:2509.11177. Google Scholar.
Questions for authors
- How well does the method perform for lower bit widths (1–2-bit quantization)?
- How significant is the decoding cost for the sparse-quantized + low-rank representation? Could it significantly slow down the inference speed?
- Does the choice of SparseGPT [13] as a pruning algorithm significantly affect the result? What if one uses magnitude pruning / Wanda [14] instead?
- How long does it take to compress the Llama-3-8B model via QUENCH, for instance?
[12] Baohao Liao, Christian Herold, Shahram Khadivi, and Christof Monz. 2024. ApiQ: Finetuning of 2-Bit Quantized Large Language Model. EMNLP 2024, pp. 20996–21020. Google Scholar.
[13] Elias Frantar and Dan Alistarh. 2023. SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot. arXiv:2301.00774. Google Scholar.
[14] Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. 2023. A Simple and Effective Pruning Approach for Large Language Models. arXiv:2306.11695. Google Scholar.
3 comments
Thank you for your response. The revised framing better reflects the scope of the paper, and the additional experiments with Qwen3 models and low-bit compression provide further evidence of the method’s effectiveness.
After considering the rebuttal, including your responses to my comments and those of the other reviewers, I have decided to raise my score.
For the final revision, I suggest stating the paper’s contributions more explicitly in the main text and giving greater prominence to its key findings.
Thank you for raising your score and for the suggestion. We have updated the revision accordingly. Each results subsection of Section 5 now opens with the finding it supports, and the contributions in the introduction each point to their evidence.
Thank you for the review. Tables G1 and G2 are in the general response.
Novelty: combination and order. We agree that the components exist. The revised paper states the contribution as a rule for combining them, which prior work does not establish: quantise once after an iterative sparse + low-rank stage, then refit the correction.
Existing methods handle quantisation differently:
Our results show that the order matters:
Recent models and reasoning tasks. We added Qwen3-8B-Base; the full table is in our reply to ivHt. With 3BASiL as stage 1, QUENCH beats EoRA 2:4 + 4-bit on all four benchmarks at lower storage. This includes the reasoning benchmarks GSM8K (25.02 vs 20.92) and MathQA (36.95 vs 35.04). Long-generation benchmarks (GPQA-Diamond, AIME, MATH500) did not fit in the compute available during the discussion period.
ProjQ and OBR.
Each covers two of the three components. The question we study, when to quantise relative to an iterative sparse + low-rank stage, arises only when all three are present. We have added both to the related work.
Q1. Lower bit widths. QUENCH with 3BASiL as stage 1, PPL / ARC-C:
3-bit is the lowest width at which the training-free recipe preserves accuracy. At 2 bits, methods with training, such as ApiQ, remain necessary.
Q2. Decoding cost. Inference weight memory falls from 15.1 to 10.0 GiB. With public kernels, prefill throughput is 95% of dense, and batch-1 decoding takes 36.8 ms per token against 24.7 ms. The overhead comes from the 2:4 kernel at small batch sizes and from the separate low-rank product, and a fused kernel would remove it. Full numbers are in our reply to ivHt.
Q3. Choice of pruner. QUENCH perplexity with each pruner inside the loop:
Iterating helps every pruner. SparseGPT stays about 4 points ahead because it also updates the kept weights, while Wanda and magnitude only select the mask.
Q4. Compression time for LLaMA-3-8B. 3.9 h for five rounds on one GPU (1.2 h for one round), with peak GPU memory of 16.3 GiB.