acceptodds
Under review as a conference paper at ICLR 2027

ASQ: An Improved Approach to Unifying Sparsification and Quantization of LLMs

Abstract

Sparsification and quantization are widely adopted for the efficient deployment of large language models (LLMs). However, they are often optimized independently, overlooking their intricate interactions and yielding suboptimal compression trade-offs. To address this, we propose ASQ, an ADMM-based framework that jointly optimizes sparsification and quantization under a unified formulation, supported by theoretical convergence guarantees. At its core, ASQ admits an exact, closed-form joint projection that concurrently resolves sparse support and quantized values by accounting for both their importance to the objective criterion and their representability on a quantization grid. Across multiple model architectures and scales, ASQ substantially outperforms existing joint compression methods under aggressive compression, with the performance gap widening significantly under extreme sparsity and low-bit regimes. Moreover, under hardware-compatible 2:4 sparsity with INT4 quantization, ASQ achieves a 4.78 memory reduction and up to 3.20 end-to-end decode speedup on Qwen3-8B-Base relative to dense BF16 inference.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
sparsity, quantization, model compression, LLMs

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Q2tcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ASQ, an ADMM-based method for jointly sparsifying and quantizing LLM weights. The dense weights are trained with Adam on the language modeling loss, while an auxiliary variable is projected onto a joint sparse and quantized set. For a fixed quantization grid, this projection has a simple closed-form solution, so the sparse support and the quantized values are decided together. The authors further add quantization scale search and a Fisher-based weighting, and they provide a convergence analysis. Experiments on LLaMA and Qwen models show clear gains over existing joint compression methods at high compression rates; the gap is especially large at 90% sparsity with 4-bit weights. The authors also report up to a 3.2x decode speedup over BF16 for a 2:4 sparse INT4 model.

Critical strengths
  1. The formulation is clean, and handling all the discrete decisions in the projection step is a neat idea. The paper is also clear and easy to follow.

  2. The results at high compression are strong and consistent across models. The ablation in Table 1 is convincing as well. Replacing the joint projection with a sequential prune-then-quantize step almost doubles WikiText-2 perplexity at 40x, which shows that the joint projection itself makes a real difference.

Critical weaknesses
  1. My main concern is the training budget. ASQ fine-tunes the model on C4 data, while the baselines only use 128 calibration sequences. The ablation shows that the joint projection helps within ASQ, but Table 1 also suggests that much of the gain over the baselines comes from the training itself. Even with all three components removed, ASQ still reaches a perplexity of 83.6 on Qwen3-8B at 40x, versus 388 for the best baseline. A comparison with one-shot methods followed by fine-tuning on the same data, or with ADMM-based sparsification followed by GPTQ, would make the claim much more convincing. It would also help to report the training cost.

  2. Some claims are stronger than what the experiments support. The abstract says the gap widens in low-bit regimes, but all experiments use 4-bit quantization. The conclusion claims superior trade-offs in hardware-oriented settings. Under 2:4 + INT4, however, ASQ is only marginally better than the best baselines (Table 10), and at 8x it is even behind the best baseline on all four models. The compression rate also ignores the overhead of masks and scales. For instance, the 2:4 + INT4 model is 8x by this formula but only 4.78x in measured memory. I would suggest reporting bits per weight and focusing the claims on the higher compression rates, where the gains are clear. Even there, the models remain far from dense quality, so saying that they retain high fidelity seems too strong.

  3. The theoretical part is weaker than presented. The convergence theorem requires lambda > 5(beta + eps), while the experiments use lambda between 1e-4 and 1e-2. It also does not cover scale search or Fisher weighting, so it is better seen as an analysis of an idealized algorithm. In addition, ADMM-based joint pruning and quantization has been explored before for DNNs, and recent ADMM-based LLM sparsification work is only briefly cited. The paper should discuss how ASQ differs from these.

Minor: the choice of hyperparameters is not explained, and the largest model is 14B. ASQ is also trained on C4 and evaluated on C4 perplexity, which may favor it. The reported speedups come from the 2:4 + INT4 format rather than from ASQ itself.

Questions for authors
  1. What is the training cost of ASQ in terms of tokens, GPU hours and memory?
  2. How does ASQ compare with one-shot methods followed by fine-tuning on the same data?
  3. Does the advantage still hold at lower bit-widths?
  4. How were the hyperparameters tuned?
4 comments

We sincerely thank the reviewer for their careful assessment and constructive suggestions. We particularly appreciate their recognition of the joint projection, informative ablations, and strong results under aggressive compression. Below, we address the reviewer's comments and suggestions with additional experiments and clarifications.

[W1 & Q1 & Q2] Training Budget Fairness and Computational Cost

First, we would like to address the reviewer's main concern regarding the training budget. The ablation study that the reviewer positively evaluated demonstrates the independent contribution of joint projection within the same optimization framework. We would also like to emphasize that ASQ is not a method that applies joint projection in isolation, but is designed to integrate task-aware iterative optimization and joint sparse–quantized projection into a unified ADMM framework.

However, we agree that the comparison with existing baselines did not fully isolate the effect of additional optimization on performance. To investigate this concern, we conducted additional experiments with the reviewer-suggested sequential baseline, ADMM-based LLM sparsification (ELSA [1]) followed by GPTQ [2], using a training budget matched to ASQ.

The following table reports WikiText-2 perplexity (↓) and average zero-shot accuracy (↑) on Qwen3-8B-Base. Values are presented as PPL (Zero-shot Avg.).

Method50% + INT470% + INT490% + INT4
ELSA→GPTQ10.23 (60.01)19.39 (51.72)45.13 (35.10)
ASQ9.91 (67.97)15.10 (59.55)36.18 (45.08)

Even under the same training budget, ASQ achieves lower PPL and higher zero-shot accuracy across all settings, with the PPL gap becoming larger at higher compression ratios. These results indicate that ASQ's advantages cannot be explained solely by the additional training budget. We will incorporate these results into the Experiments section.

We also report ASQ's training configuration and computational cost on Qwen3-8B-Base below.

TokensGPUWall-clock TimePeak Memory / GPU
134M2 × H2003.7–4.0 h111 GB

One point we would like to emphasize is that compression incurs a cost before deployment. In other words, when a compressed model is used repeatedly, this cost can be amortized over an extended period. ASQ achieves better performance than existing methods at high compression ratios and also outperforms the budget-matched baseline. Therefore, we believe that its efficiency should be evaluated in conjunction with the performance achieved.

[W2 & Q3] Clarification of Claims

Next, we acknowledge that some of our claims were somewhat stronger than what the presented experimental results support. We appreciate the reviewer's careful examination of our results and their suggestions for clarifying the scope of our claims.

First, we conducted additional experiments at 3-bit precision, which is lower than the bitwidth considered in our original experiments.

The following table presents WikiText-2 perplexity (↓) on Qwen3-8B-Base under 3-bit quantization.

Method70% + INT380% + INT390% + INT3
SparseGPTQ24.3174.95407.77
SLiM82.05993.81∞
ASQ17.9925.2646.07

As the results show, ASQ also achieves strong performance at a lower bitwidth. We present the main results here and will include the results for all baselines in the paper.

Additionally, we agree that the performance difference in the hardware-compatible INT4+2:4 setting is relatively small, and that the theoretical compression ratio does not fully reflect actual storage costs. Accordingly, we will define effective bits/weight accounting for metadata overhead and incorporate the corresponding storage costs of ASQ. We will also refine our claims that go beyond the experimental evidence.

[W3] Convergence Analysis and Distinction from Prior ADMM Methods

Regarding the concerns about the theoretical analysis, we would first like to clarify the role of our theoretical convergence guarantee. In general, convergence analyses of complex nonconvex optimization algorithms do not aim to provide unconditional guarantees covering every component of the practical implementation. Rather, their significance lies in rigorously characterizing the properties of the core optimization mechanism under appropriate assumptions.

From this perspective, ASQ's theoretical result goes beyond merely demonstrating a convergence trend. Based on exact joint projection onto a nonconvex and discrete sparse–quantized constraint set, we prove that the residual between the continuous and compressed variables converges to zero. Furthermore, we establish finite-time identification, ensuring that the sparse support and quantized values stabilize after a finite number of changes. Finally, we establish convergence to a λ\lambda-stationary point and derive an explicit upper bound on the number of changes in the discrete representation.

Meanwhile, the theoretical condition λ>5(β+ε)\lambda > 5(\beta+\varepsilon) cannot be assessed solely from the absolute value of λ\lambda, as it depends on the smoothness of the objective function and the approximation accuracy of the x-update. Therefore, it is difficult to determine whether this condition is satisfied based only on the values of λ\lambda used in the experiments. However, since we have not verified this sufficient condition under our current experimental settings, we clearly distinguish the scope of the theoretical guarantee as applying to the core formulation.

We also explicitly discuss the scope of the theoretical guarantee in the Discussion section, where we clarify that the convergence guarantee applies to the core exact-projection formulation and provide separate empirical evidence for the optimization dynamics of practical ASQ (Figure 3). Therefore, the theoretical guarantee and empirical validation complement each other in supporting the optimization properties of ASQ.

Additionally, we thank the reviewer for suggesting that we discuss the closely related work on DNNs. Although [3] similarly uses ADMM as a tool for joint compression, it differs from our work in two important aspects: (i) its primary objective is to automatically determine how sparsity and bitwidth should be allocated across layers, and (ii) it handles sparsity and quantization constraints through separate auxiliary variables and projection steps. In contrast, our work is motivated by the need to account for the interaction between sparsification and quantization constraints, and accordingly formulates the problem and proposes a joint projection. We acknowledge that this work should have been discussed in our paper and will clarify these differences in the paper.

[Q4 & Minor] Hyperparameter Selection, Scalability, and Evaluation Protocol

Finally, we would like to address several minor issues raised by the reviewer.

  • Choice of hyperparameters: We selected the learning rate (η\eta) and ADMM penalty parameter (λ\lambda) based on validation. We will clarify the search space and selection procedure in Appendix C.
  • Scalability: We conducted additional experiments on Qwen2.5-32B, and present the main results below. The following table reports WikiText-2 perplexity (↓) on Qwen2.5-32B under INT4 quantization.
Method70% + INT480% + INT490% + INT4
SparseGPTQ12.4130.30654.69
SLiM12.0626.10831.33
ASQ10.4914.4332.97

These results show that our method also achieves strong performance on a larger model. We will include the full results in the paper.

  • C4: Using C4 for optimization is a common practice in LLM compression. Importantly, C4 perplexity is evaluated on a held-out set, separate from the optimization data. We also evaluate ASQ on WikiText-2 and various zero-shot benchmarks, where the performance improvements remain consistent.

Finally, we sincerely thank the reviewer again for their careful assessment of our work and constructive suggestions. We hope that our responses and additional results help address the concerns raised.

References

[1] Lee et al., "The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM," ICLR, 2026.

[2] Frantar et al., "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers," ICLR, 2023.

[3] Yang et al., "Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-Based Approach," CVPR, 2020.

Thank you for the additional experiments. The budget-matched comparison with ELSA followed by GPTQ addresses my main concern, especially at 90% sparsity. The reported training cost and the new INT3 and 32B results also strengthen the evaluation. I am raising my rating from 2 to 3.

We sincerely thank the reviewer for carefully considering our responses and additional experiments, and for raising their rating. We are pleased that the budget-matched comparison, training cost analysis, and additional INT3 and 32B results have helped address your concerns. Your thoughtful feedback and constructive suggestions have been invaluable in strengthening our experimental evaluation and improving the overall quality of our work. We will incorporate these additional results into the revised manuscript. Thank you again for your time and constructive review.

Reviewer_F4fQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ASQ, an ADMM-based method that jointly sparsifies and quantizes LLM weights. The core idea is to decide which weights to keep while accounting for their quantization error, with convergence guarantees under certain assumptions. Experiments on LLaMA and Qwen models show better performance at high compression ratios and practical inference speedups.

Critical strengths
  1. The joint projection makes sense, it accounts for quantization error when selecting weights to retain and has an exact solution.
  2. ASQ shows clear gains at high compression ratios across several models, along with practical inference speedups.
Critical weaknesses
  1. The comparison may involve unequal budgets: ASQ uses iterative training, while the baselines use 128 calibration sequences. Data usage and computational costs are not clearly compared.
  2. Gains are concentrated at high compression ratios. At lower ratios, results are mixed; under 2:4 sparsity, average accuracy improves over the strongest baseline by only 0.11 points.
Questions for authors
  1. The xx -update in Eq. (5) uses a Euclidean penalty, while the zz -update in Eq. (16) uses a weighted distance. As written, these updates no longer directly correspond to the same augmented Lagrangian. Does the implementation also adjust the xx -update and the corresponding dual update, or is the weighted projection used as a heuristic replacement?
  2. I think the most important is how much training data, GPU time, and memory does ASQ require? Do its gains hold under matched data and compute budgets?
  3. Could you compare ASQ with quantization-only methods under the same actual storage budget? For example, 4-bit quantization with 50% sparsity has the same theoretical weight storage cost as dense 2-bit quantization, but sparse representations require additional metadata, such as indices. After accounting for this overhead, does ASQ still offer a better accuracy–storage trade-off, demonstrating an advantage over simply reducing the bit-width?
4 comments

We sincerely thank the reviewer for their careful assessment and constructive suggestions. We particularly appreciate their recognition of the exact joint projection, the strong performance under high compression ratios, and the practical inference speedups. Below, we address the main concerns in detail.

[W1 & Q2] Unequal Training Budgets and Computational Cost

First, we would like to emphasize that ASQ's iterative optimization is not a separately added step, but a core component of the framework, designed to repeatedly combine the x-update, which optimizes the task objective, with the z-update, which performs exact joint sparse-quantized projection (Section 3.2).

Indeed, the ablation study in Table 1 demonstrates the independent contribution of joint projection within the same optimization framework. To further address this concern, we conducted an additional experiment with a training-budget-matched baseline. Specifically, we optimized ELSA [1] using the same training data and gradient-based optimization budget as ASQ, and then applied GPTQ [2] using 128 calibration sequences, forming a sequential baseline.

The following table reports WikiText-2 perplexity (↓) and average zero-shot accuracy (↑) on Qwen3-8B-Base. Values are presented as PPL (Zero-shot Avg.).

Method50% + INT470% + INT490% + INT4
ELSA→GPTQ10.23 (60.01)19.39 (51.72)45.13 (35.10)
ASQ9.91 (67.97)15.10 (59.55)36.18 (45.08)

As the results show, ASQ achieves better performance across all settings even under the same gradient-based optimization budget, with a larger PPL improvement at higher compression ratios. This supports the conclusion that ASQ's performance advantage cannot be explained solely by the additional training budget.

We also report ASQ's training configuration and computational cost on Qwen3-8B-Base below.

TokensGPUsWall-clock TimePeak Memory / GPU
134M2 × H2003.7-4.0 h111 GB

One point worth noting is that compression incurs a cost before deployment. In other words, when a compressed model is used repeatedly, this cost can be amortized over an extended period. ASQ achieves better performance than existing methods at high compression ratios and also outperforms a budget-matched baseline. Therefore, we believe its efficiency should be evaluated in conjunction with the performance achieved.

[W2] Performance Gains Concentrated at High Compression Ratios

ASQ's stronger gains at high compression ratios are consistent with the central motivation behind our approach. ASQ explicitly accounts for the interaction between sparsification and quantization. As compression becomes more aggressive, fewer weights can be retained, and those retained weights must be represented at low precision. Consequently, it becomes increasingly important to consider sparse support selection and quantization compatibility together. Indeed, the ablation in Table 1 shows that removing joint projection leads to a substantially larger performance degradation at higher compression ratios. Thus, ASQ's growing relative advantage in this regime is consistent with its design objective.

Moreover, preserving model performance at high compression ratios is not merely about evaluating a more challenging setting. It is an important research goal because it expands the achievable limits of model compression under constrained memory budgets.

At the same time, we acknowledge that the accuracy improvement over the strongest baseline is small in the INT4+2:4 setting. The significance of this experiment lies in demonstrating competitive model performance and practical inference acceleration under a hardware-supported compression format. We will clarify this distinction in the paper.

[Q1] Euclidean vs. Fisher-Weighted Projection

First, we would like to clarify the consistency of the ADMM updates. In the actual implementation, we retain the Euclidean x-update in Eq. (5) and the dual update in Eq. (7), while applying a Fisher-weighted projection to the z-update. Therefore, as the reviewer correctly points out, Eq. (16) is a sensitivity-aware extension rather than an exact block update directly derived from the original Euclidean augmented Lagrangian.

However, this extension was not introduced arbitrarily; it was designed to better reflect task-loss degradation caused by compression. While Euclidean projection treats the compression errors of all weights equally, individual weights may have different sensitivities with respect to the task loss. To account for these differences, we use the diagonal empirical Fisher as the projection metric, motivated by a second-order approximation.

Importantly, the exactness of the joint projection is preserved under this weighting. For a fixed quantization grid and a diagonal Fisher metric, we can derive a weighted retention gain and obtain an exact solution to the projection subproblem, jointly determining the sparse support and quantized values. The ablation in Table 1 also shows that sensitivity weighting improves compression performance in practice.

[Q3] Comparison with Quantization-Only Methods Under Storage Constraints

We thank the reviewer for the valuable suggestion to compare ASQ with quantization-only methods under a metadata-aware storage budget. Accordingly, we conducted additional comparisons with GPTQ [2] and the rotation-based quantization method SpinQuant [3], using a common storage metric that accounts for the compressed weights, sparsity masks, and quantization scales.

We define the comparison metric as the average number of stored bits per weight for the linear weights in transformer blocks:

  • Quantization-only: Bquant=b+0.125B_{\mathrm{quant}} = b + 0.125.
  • ASQ: BASQ=(1−s)b+1+0.125B_{\mathrm{ASQ}} = (1-s)b + 1 + 0.125.

Here, b denotes the quantization bitwidth and s denotes the sparsity ratio. For all methods, we account for the cost of one FP16 scale per group of 128 weights, corresponding to 0.125 bits/weight. For ASQ, we additionally include a 1-bit bitmap to store the sparse support. This is an explicit bitmap-based storage model also employed in SparseGPT [4].

The following table compares ASQ with quantization-only methods on Qwen3-8B-Base, reporting WikiText-2 perplexity (↓) against effective bits/weight.

BPWGPTQSpinQuantASQ
4.1257.35 (INT4)7.26 (INT4)-
3.1258.98 (INT3)8.34 (INT3)9.91 (INT4, 50%)
2.625--12.18 (INT3, 50%)
2.325--15.10 (INT4, 70%)
2.125375.3 (INT2)108.7 (INT2)-
2.025--17.99 (INT3, 70%)
1.925--22.57 (INT4, 80%)
1.425--46.07 (INT3, 90%)

As shown in the results, quantization-only methods perform better at around 3 bits/weight. However, a different trend emerges under tighter storage budgets. In particular, SpinQuant INT2 achieves a PPL of 108.7 at 2.125 bits/weight, whereas ASQ achieves a PPL of 17.99 with a lower storage cost of 2.025 bits/weight. Furthermore, ASQ maintains a PPL of 46.07 even when the storage budget is reduced to 1.425 bits/weight. Compared with the quantization-only baselines, these results show that ASQ can retain relatively low perplexity under more restrictive storage budgets.

These findings indicate that jointly employing sparsification and quantization can provide a more favorable accuracy-storage trade-off than simply reducing the quantization bitwidth. We will add these results and their interpretation to the Experiments section.

Finally, we sincerely thank the reviewer again for their careful assessment and suggestions that offered a new perspective on our work. We will incorporate their suggestions into the paper. We hope that our responses and additional results help address the reviewer's concerns.

References

[1] Lee et al., "The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM," ICLR, 2026.

[2] Frantar et al., "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers," ICLR, 2023.

[3] Liu et al., "SpinQuant: LLM Quantization with Learned Rotations," ICLR, 2025.

[4] Frantar and Alistarh, "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot," ICML, 2023.

Thank you for the clarifications and additional experiments. The budget-matched and storage-aware comparisons help address my experimental concerns, and the distinction between the base ADMM formulation and the Fisher-weighted extension is now clearer. I have no further questions.

We are pleased that our responses and additional experiments have adequately addressed the reviewer's earlier concerns. We sincerely appreciate the reviewer's careful reconsideration and thoughtful feedback, which have helped us strengthen our experimental evaluation and clarify the distinction between the base ADMM formulation and the Fisher-weighted extension. We will incorporate these additional results and clarifications into the final version of the paper. Thank you again for your constructive engagement and for updating your assessment.

Reviewer_zeenOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes ASQ, a method for joint sparsification and quantization instead of a naive sequential approach. The proposed formulation, under some assumptions and approximations, admits an elegant ADMM-based alternating minimization scheme. The reported results are strong: at 40x theoretical compression ratio, ASQ improves average accuracy by around 4 to 6 percentage points compared to the strongest baselines.

Critical strengths
  1. The paper is well written and easy to follow. The authors do a good job formulating the problem and motivating their proposed approach.
  2. The provided ablations are informative and show the strength of individual components as well as their synergy.
  3. The amount of experimental results is satisfactory and the hardware-compatibility setting is useful.
Critical weaknesses
  1. There are novelty concerns of this work. In particular, prior work ELSA (The Unseen Frontier, ICLR 2026) already uses ADMM and Fisher-based sensitivity for LLM sparsification. Moreover, Automatic Neural Network Compression by Sparsity-Quantization Joint Learning (CVPR 2020) already considered joint sparsification and quantization using ADMM, but this work is not discussed by the authors. The exact joint projection proposed in ASQ is nice, but to me it seems like a relatively straightforward extension of these ideas. I would invite the authors to explain their novelty more clearly.

  2. I am not sure how fair the baseline comparison is. According to Appendix C.1, the baselines use 128 calibration sequences, while ASQ performs an epoch of gradient-based optimization. Thus, it is difficult to understand how much of the improvement comes from the proposed joint optimization and how much from the additional training. A comparison with a sequential approach with a similar optimization budget is necessary. Moreover, the sequential pipeline is not fully covered, since GPTQ could be applied before sparsification or vice versa, but only the latter is discussed.

  3. Rotation-based quantization methods can ouperform vanilla GPTQ under challenging quantization regimes, which makes the baseline selection somewhat questionable. I understand that rotation can also affect sparsity, but this interaction is worth studying. In addition, I invite the authors to discuss the implementation complexity of rotation preprocessing for ASQ, or even including rotations in the alternating minimization process.

  4. While the proposed method jointly optimizes sparsification and quantization, a mechanism for adjusting the trade-off between sparsity and bitwidth, at least globally, is missing. Perhaps using lower bitwidth while keeping more weights would be a better sweet spot at the same memory budget.

  5. The theoretical compression ratio does not account for the metadata overhead of sparse storage formats or quantization parameters. Thus, the claimed 40x compression does not directly translate into an actual 40x reduction in memory. Moreover, under the more practical INT4+2:4 setting, the gain in average accuracy over the strongest baseline is rather small (61.90% versus 61.79%). I think this difference between the theoretical and practical regimes deserves more discussion.

  6. Table 3, raises some questions about hyperparameter tuning, since learning rates and penalty parameters are selected separately for different models and sparsity levels. More details on how they were selected and how sensitive the results are to these choices, especially given the different optimization budgets will provide more clarity. Although the paper specifies symmetric quantization and group size 128, the exact GPTQ baseline settings should also be clarified.

  7. I am not fully convinced that the convergence analysis applies to the complete method. The proof assumes a fixed quantization grid and an exact Euclidean projection, while the practical algorithm additionally uses adaptive scale optimization and Fisher-based weighting. It would be useful to clarify which parts of the theoretical guarantee still hold for the actual implementation.

Questions for authors

Please refer to the weaknesses section.

8 comments

We sincerely thank the reviewer for their thoughtful assessment and constructive suggestions. We particularly appreciate their recognition of our clear formulation, informative ablations, and strong performance under aggressive compression. We are also grateful for the detailed suggestions that encourage us to examine our work from additional perspectives. Below, we address each concern in detail.

[W1] Novelty and Distinction from Prior Work

We agree that ADMM and Fisher-based sensitivity are established techniques. However, ADMM is a general framework for constrained optimization, and our novelty lies not in the use of ADMM itself, but in the problem we formulate and the solution we develop. Specifically, the three works differ in their problem settings and primary contributions:

  • ELSA [1] addresses limitations of surrogate-based objectives in existing LLM pruning methods. It proposes ADMM-based sparsification that directly optimizes the original language modeling objective, but does not address the interaction with quantization constraints.
  • Yang et al. [2] focus on automatically allocating sparsity and bitwidth across layers under a global storage budget. Their ADMM-based formulation alternates between a sparsity projection with bitwidth fixed and a bitwidth-allocation projection with sparsity fixed, using the original weights and an auxiliary copy. Thus, sparsity and quantization are handled through separate conditional projection steps.
  • ASQ (ours) addresses a different problem: jointly determining which weights to retain and which quantized values should represent them. We formulate a single projection onto the intersection of sparsity and quantization constraints, derive a retention gain that directly incorporates quantization distortion into support selection, and provide an exact closed-form solution to this joint projection.

Thus, ASQ's core contribution lies in directly addressing the interaction between sparsification and quantization through an exact joint projection, offering a distinct approach to the compression problem. We further develop an integrated framework incorporating sensitivity weighting and quantization scale optimization. We will clarify these distinctions in the paper.

[W2] Fairness of Baseline Comparisons and Sequential Compression Order

First, we would like to clarify that iterative optimization is not an additional stage appended to ASQ, but an integral part of the framework (§3.2). ASQ is designed to repeatedly couple the x-update, which optimizes the task objective, with the z-update, which performs exact joint sparse–quantized projection. Its performance should therefore be considered in terms of an optimization structure that directly integrates task-aware optimization with joint compression constraints, rather than additional training alone.

Nevertheless, we agree that the contribution of the training budget should be distinguished from that of the proposed optimization framework. To address this concern directly, we added a sequential baseline that applies ELSA [1] using the same training data and gradient-based optimization budget as ASQ, followed by GPTQ [3] with 128 calibration sequences.

The following table reports WikiText-2 perplexity (↓) and average zero-shot accuracy (↑) on Qwen3-8B-Base. Values are presented as PPL (Zero-shot Avg.).

Method50% + INT470% + INT490% + INT4
ELSA→GPTQ10.23 (60.01)19.39 (51.72)45.13 (35.10)
ASQ9.91 (67.97)15.10 (59.55)36.18 (45.08)

Even under the same gradient-based optimization budget, ASQ achieves lower PPL and higher zero-shot accuracy across all settings, with larger improvements in PPL at higher compression ratios. These findings support the conclusion that ASQ's advantage cannot be explained solely by additional training.

We would also like to clarify that SparseGPTQ [4], which was included in our original experiments, is not simply a prune-then-quantize pipeline. It is the one-shot joint sparsification–quantization procedure proposed in §3.5 of the original SparseGPT paper. Nevertheless, to examine the effect of compression order in genuinely sequential pipelines, we additionally evaluated both SparseGPT→GPTQ and GPTQ→SparseGPT. The following table reports WikiText-2 perplexity (↓) on Qwen3-8B-Base.

Method70% + INT480% + INT490% + INT4
SparseGPTQ (joint)20.1374.52388
SparseGPT→GPTQ21.4595.17445
GPTQ→SparseGPT29.24256341115
ASQ15.1022.5736.18

We will include the full results in the paper.

[W3] Rotation-Based Quantization

We thank the reviewer for this interesting perspective. ASQ focuses on jointly optimizing the discrete decisions of sparsification and quantization within a given weight representation. In contrast, rotation-based methods also consider the representation space in which those decisions are made. We view this as a meaningful direction for extending the central motivation of ASQ.

First, we would like to clarify that OBR [5], a joint compression baseline operating on rotated model weights, is already included in our experiments.

Moreover, combining rotation with ASQ admits a natural mathematical extension. Assuming a fixed, function-preserving rotation is applied through appropriate preprocessing, ASQ's iterative optimization and exact joint projection can be performed in the rotated weight space. Let T be a fixed orthogonal transformation and C\mathcal{C} the sparse–quantized feasible set defined in the rotated coordinates. The Euclidean projection then satisfies

= T^{-1}\Pi_{\mathcal{C}}(Tv), \qquad v=x^{t+1}+u^t.$$ Thus, with a fixed rotation and a Euclidean ADMM formulation, the basic structure of the x- and u-updates can be retained under a consistent coordinate representation, while ASQ's existing exact joint projection can be applied as the z-update onto the rotated constraint set. This indicates that rotation and ASQ's joint projection are mathematically compatible and potentially complementary. Nevertheless, applying a fixed rotation as preprocessing and optimizing the rotation itself within alternating minimization are distinct problems. The latter requires maintaining orthogonality constraints while optimizing the rotation, as well as accounting for sensitivity estimation in a changing representation space and inference-compatible reparameterization. These additional challenges arise from optimizing the representation space alongside compression, rather than from changing the underlying joint projection itself. Furthermore, rotation may flatten the weight distribution and thereby facilitate quantization, but it may also weaken the distinctions in weight magnitudes that are useful for sparsification. Since ASQ is motivated by the interaction between these two compression mechanisms, studying how rotations alter that interaction and identifying rotations suitable for joint compression would be a meaningful extension. **[W4] Trade-off Between Sparsity and Bitwidth** We appreciate this valuable suggestion. The primary objective of our work is to jointly optimize sparse support and quantized values under a given sparsity level and bitwidth. Consequently, automatically searching for the optimal sparsity–bitwidth configuration is not a direct objective of the present work. Nevertheless, because ASQ concerns joint compression, we agree that examining this trade-off is an interesting and potentially valuable direction for further analysis. Extending ASQ to automatically allocate compression constraints across sparsity and bitwidth is promising future work.

[W5] Theoretical Compression Ratio and Practical Storage Costs

We thank the reviewer for this valid observation. We recognize that our theoretical compression ratio does not include metadata overhead. In §4, we defined it as a theoretical metric for comparing compression configurations, while the Discussion section addresses the remaining gap between algorithmic performance at aggressive compression ratios and practical hardware execution.

We agree, however, that the distinction between theoretical compression ratios and actual memory savings should be made more explicit. Accordingly, we additionally considered metadata-aware storage costs. Specifically, for the linear weights in transformer blocks, we estimate ASQ's storage cost in bits per original weight as

BASQ=(1−s)b+1+0.125.B_{\mathrm{ASQ}}=(1-s)b+1+0.125.

Here, b is the quantization bitwidth and s is the sparsity ratio. The 1-bit term accounts for a bitmap storing sparse support, and the 0.125 bits/weight term corresponds to one FP16 quantization scale per group of 128 weights. Bitmap-based sparse storage is also considered in SparseGPT [4]. This estimate explicitly accounts for metadata omitted from the original theoretical compression ratio. We will add this storage-aware metric to the Experiments section.

We also acknowledge that the accuracy improvement over the strongest baseline is relatively small in the practical INT4+2:4 setting. The main significance of this experiment is to demonstrate competitive model performance, actual memory savings, and inference acceleration under a sparse INT4 format supported by existing hardware.

At the same time, we believe it is important to distinguish the compression levels efficiently supported by current hardware from those that can be achieved algorithmically. Although extreme compression does not yet translate into equivalent practical acceleration on existing hardware, ASQ demonstrates relatively robust model performance at high compression ratios where prior methods degrade sharply. This is meaningful because it extends the range of performance and compression ratios attainable through joint compression.

Therefore, we view ASQ as demonstrating both practical utility under currently supported hardware settings and algorithmic potential that could be better realized as hardware and execution techniques advance to support higher compression ratios. We will revise the manuscript to distinguish these two aspects more clearly.

[W6] Hyperparameter Selection and GPTQ Baseline Settings

First, the use of different learning rates and penalty parameters across models and sparsity levels in Table 3 is related to the nature of ADMM-based optimization. The penalty parameter controls the relative emphasis on objective minimization and constraint satisfaction, while the learning rate governs the behavior of the inexact optimization performed at each iteration. Appropriate choices can therefore vary with the model's loss landscape and the imposed sparsity constraint. This distinction becomes particularly important for ASQ, which handles strong discrete constraints under aggressive compression.

Our hyperparameter search also revealed a consistent trend: higher sparsity levels tended to favor relatively larger learning rates and smaller penalty parameters. This is consistent with the need to adjust the continuous optimization so that it can better compensate for the degradation introduced by stronger discrete constraints. For example, for Qwen3-8B-Base, as sparsity increases from 50% to 90%, the selected learning rate changes from 2e-5 to 1e-4, while the penalty parameter changes from 1e-2 to 2e-4.

To improve clarity and reproducibility, we will specify the actual search ranges, validation criterion, and configuration-wise selection procedure in the Appendix. For the GPTQ baseline, we use symmetric quantization and a group size of 128, consistent with ASQ. We will also provide the implementation and detailed GPTQ settings in the Appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.