acceptodds
Under review as a conference paper at ICLR 2027

SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

Abstract

Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2–3 bits. Existing backpropagation-free PTQ optimizers have two limitations: group decisions ignore the correction that the remaining continuous suffix can absorb, and discrete refinements typically keep the affine quantization grid fixed. We introduce SchurOpt, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature. It then alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes. With the GPTQ objective fixed, SchurOpt improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points (pp). At higher precision, however, tighter reconstruction does not consistently improve end-model metrics. SchurQuant therefore combines SchurOpt with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and teacher-decision token weighting. Across eight Llama and Qwen models, SchurQuant achieves the highest mean zero-shot accuracy among the evaluated backpropagation-free PTQ baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.

Primary area:
generative models
Keywords:
large language model, quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5HKdOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors observe two limitations of existing backpropagation-free PTQ methods: group decisions ignore the error that the remaining unquantized weights can absorb, and discrete refinements keep the quantization grid fixed. To address this, they propose SchurOpt, which uses the Schur complement to obtain a groupwise objective that accounts for compensation by the remaining weights, and alternately optimizes the scale/zero-point and the integer codes. Building on this, the authors augment the optimization objective with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and token weighting to propose SchurQuant. Across eight Llama and Qwen models, SchurQuant achieves the highest mean zero-shot accuracy among the baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.

Critical strengths
  1. The paper provides a theoretical derivation, giving the method a clear theoretical foundation.
  2. Experiments show that the method brings clear improvements at low bit widths.
  3. The ablation studies are thorough, separately validating the contributions of the optimizer and the optimization objective.
Critical weaknesses
  1. Limited model coverage. The experiments involve only two model families, Llama and Qwen, at 0.6B–13B scale, lacking validation on larger models and other model families, as well as on architectures such as Mixture-of-Experts (MoE).
  2. Limited baselines. The paper discusses several related methods, but the experiments compare against only a few of them.
  3. Insufficient efficiency evaluation. As a compression method, the paper only reports the offline quantization time in the appendix, without reporting memory usage or inference metrics such as TTFT and TPOT.
  4. High quantization cost. The appendix shows that SchurQuant's quantization time is roughly 30–50× that of GPTQ and about 5× that of QEP.
Questions for authors
  1. How does SchurQuant perform on larger models, other model families, and architectures such as MoE?
  2. Could the authors report memory usage and inference metrics such as TTFT and TPOT?
  3. How does SchurQuant compare with the other related methods discussed in the paper?
1 comment

We appreciate the reviewer's detailed and constructive assessment of our work. We address each point below.

W1/Q1. Larger models, other families, MoE. Added Llama-3.3-70B, Qwen3-30B-A3B (mixture-of-experts, 128 experts per layer, 8 active), and Qwen3.5-9B (a vision-language model with gated linear-attention blocks). Expert tensors are quantized expert by expert with routing-weighted statistics (App. I). On Qwen3.5-9B at 2 bits, the mean over six multimodal benchmarks is 48.17 for SchurQuant against 4.0-7.6 for GPTQ, FOEM, QEP, and CDQuant (full precision 80.81; App. G, Table 13). As a language model, 2-bit SchurQuant reaches 63.61% on Qwen3.5-9B against 61.34% for CDQuant (full precision 75.04%). Results (App. G, Table 12; PPL / mean accuracy in %):

ModelMethod2-bit3-bit4-bit
Llama-3.3-70BFP163.86 / 81.22
QEP66.15 / 37.526.04 / 78.604.41 / 80.86
CDQuant12.90 / 72.055.73 / 80.694.45 / 80.84
SchurQuant14.15 / 73.165.78 / 79.874.35 / 80.66
Qwen3-30B-A3BFP168.71 / 77.49
QEP32.31 / 46.779.66 / 72.258.93 / 74.06
CDQuant24.06 / 60.599.88 / 71.709.05 / 74.92
SchurQuant16.88 / 65.269.50 / 74.178.91 / 75.13
Qwen3.5-9BFP168.65 / 75.04
GPTQ41.23 / 42.709.71 / 71.298.93 / 75.38
FOEM95.47 / 40.579.24 / 68.818.84 / 74.56
QEP45.38 / 44.769.33 / 71.488.46 / 75.19
CDQuant26.16 / 61.348.64 / 74.668.19 / 74.51
SchurQuant22.24 / 63.618.72 / 75.048.37 / 75.84
SchurQuant+T11.19 / 68.698.48 / 74.848.20 / 75.16

At 2 bits, QEP stays below 47% on Llama-3.3-70B and Qwen3-30B-A3B, and SchurQuant exceeds the strongest baseline, CDQuant, by 1.1 pp on Llama-3.3-70B and by 4.7 pp on Qwen3-30B-A3B. QEP and CDQuant do not handle expert tensors, so on Qwen3-30B-A3B we apply their solvers to each expert on the tokens routed to it.

W2/Q3. More baselines. CDQuant, QuantEase, and LoaQ are now in Tables 1-2 on all eight models and bit widths, and all rows of these tables are backpropagation-free. Mean accuracy (%) over the eight models:

Method2-bit3-bit4-bit
QuantEase38.9962.4667.30
LoaQ42.3163.7067.59
CDQuant51.6264.4267.41
SchurQuant53.4164.9267.28

At 2 bits, SchurQuant outperforms the baselines on seven of the eight models. CDQuant is the closest competitor, achieving better results on Llama3-8B but lower accuracy on seven of the eight models. At 4 bits, the differences between methods are small. After applying the same scale tuning to all methods (Sec. 4.4, Table 3), SchurQuant+T achieves 58.91% mean accuracy at 2 bits, compared to 58.60% for CDQuant.

To isolate the effect of the solver, Table 8 (App. D) compares SchurOpt with the CDQuant solver using the same GuidedQuant statistics at 2 bits, before scale tuning. SchurOpt also reduces solver time by a factor of 2.4-8.0:

SolverQwen3-8B PPLQwen3-8B Acc.Llama3-8B PPLLlama3-8B Acc.
CD (CDQuant)33.260.134.856.1
SchurOpt28.262.230.856.4

W3/Q2. Memory and inference metrics. Peak GPU and host memory during quantization are now reported (Table 7; e.g. Qwen3-4B: 47.6 GB GPU, 11.5 GB host). We measured TTFT/TPOT with vLLM on one H100 SXM 80GB GPU (App. H, Table 14). SchurQuant keeps GPTQ's storage format, and latency matches GPTQ within noise at every bit width. With 512-token prompts (ms):

ModelFormatTTFTTPOT
Qwen3-8BBF1621.226.49
GPTQ 2-bit31.312.92
SchurQuant 2-bit30.813.05
Llama2-13BBF1629.3610.07
GPTQ 2-bit44.713.58
SchurQuant 2-bit44.153.68

TPOT falls by 53% and 63% relative to BF16. TTFT is higher for all quantized formats.

W4. Quantization cost. The cost in the submission came from recalibrating the full model before every layer, not from SchurOpt. Neither revised configuration needs this. Quantization time for 2-bit Qwen3-4B on one H100 SXM 80GB GPU (App. C, Table 6; SchurQuant took 253.7 min in the submission):

GPTQFOEMQEPLoaQQuantEaseCDQuantSchurQuant
Time (s)297.67345.00339.102682.642971.922329.51458.53

SchurQuant+T takes 17.6 min for Qwen3-4B, 24.5 min for Llama2-7B, and 27.5 min for Qwen3-8B on one H100 SXM 80GB GPU, including the gradient pass and scale tuning. Llama-3.3-70B is quantized by SchurQuant on a single H100 SXM 80GB GPU, keeping one block on the GPU at a time (table under W1/Q1 above). The Llama-3.3-70B quantization process takes approximately 3 hours.

Reviewer_RBZVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SchurQuant, a weight-only PTQ method. Its main idea is to design an offline quantization solver that better accounts for how weights compensate for one another during offline optimization, and pair it with a more suitable calibration objective to improve low-bit quantization accuracy. For the offline optimization problem of minimizing ||W-W_ref||, the paper uses many mathematical formulas to prove that accounting for weights that have not yet been quantized through the Schur complement can improve the accuracy of the optimal solution. The experimental results are very strong for 2-bit PTQ, greatly reducing the original model’s perplexity and clearly outperforming the other baseline methods compared in the paper.

Critical strengths
  1. This paper treats weight-only quantization of large language models as a weight optimization problem and improves the accuracy of the overall offline optimization approach in two ways. First, it uses the Schur complement to adjust later weights while quantizing the current weight. Second, it jointly improves which 2-bit grid point each weight is assigned to and the actual values of the 2-bit grid points, combining this with the Schur complement to form SchurOpt. Although I cannot fully understand the mathematical steps in this part, I think the overall logic of the math and the ideas behind these improvements are very valuable.

  2. The paper shows very strong results for 2-bit quantization compared with GPTQ, FOEM, and QEP, greatly reducing perplexity and increasing zero-shot scores by many points across a range of Llama models.

Critical weaknesses
  1. The contribution is not significant. My biggest concern about this paper is that its contribution to 2-bit PTQ is not significant.
  • First, although the improvements over the other PTQ baselines in the paper are clear, there is a large amount of work on PTQ, and each work has a different focus. The authors should not make their contribution seem larger through their choice of papers for comparison. Please see the Questions section for details.
  • Second, compared with the BF16 baseline, their results show a very clear drop in performance in both perplexity and zero-shot accuracy. This is unacceptable in actual production deployment. Saving some GPU memory with 2-bit PTQ cannot justify such a large drop in performance. This may also be why mainstream large-model serving today rarely uses PTQ, with QAT being more common, or at least does not use PTQ as aggressive as 2-bit. For 4-bit PTQ, however, the gains of SchurQuant over other baselines are very small. Therefore, this paper’s contribution is largely limited to research, and I believe it somewhat falls below the bar for ICLR.
  1. The practical value is limited. The authors themselves acknowledge that the complex SchurOpt algorithm adds substantial overhead during offline quantization. For example, for 2-bit quantization of Llama2-7B, GPTQ takes 7.1 minutes, QEP takes 38.6 minutes, and SchurQuant takes 224.2 minutes. More importantly, for today’s very large models, such as DeepSeek, Kimi, and latest Qwen models with hundreds of billions or even trillions of parameters, would SchurQuant take so long that the time might be better spent on additional QAT experiments? These concerns about infrastructure and deployment are real. Although I understand that it is difficult for the authors to carry out such complete experiments, I believe this weakness should be treated as an issue that affects the paper’s main claims, rather than being mentioned only briefly.
Questions for authors
  1. Regarding the choice of other PTQ methods for comparison in this paper: why did the authors choose these works, and why did they leave out others, such as QuantEase, CDQuant, and LoaQ? Among the included baselines, GPTQ is an early representative work in this field, so its inclusion is understandable. The original FOEM paper seems to focus mainly on 4-bit optimization. Could its poor performance at 2 bits be due to settings that do not match the original paper’s focus? Also, its 4-bit performance as reproduced in this paper seems worse than the 4-bit results reported in the original paper.

  2. The models selected for evaluation in this paper all seem relatively old. Why did the authors choose these models? Have they considered using more advanced and larger models to explore how well SchurQuant works in these inference-only settings with greater practical value?

3 comments

We thank the reviewer for thoughtful and constructive feedback. Below, we provide detailed responses to each point.

W1a. Choice of baselines. All three are now in Tables 1-2 on all eight models, and all rows of these tables are backpropagation-free. Mean accuracy (%) over the eight models:

Method2-bit3-bit4-bit
QuantEase38.9962.4667.30
LoaQ42.3163.7067.59
CDQuant51.6264.4267.41
SchurQuant53.4164.9267.28

At 2 bits, SchurQuant achieves the highest accuracy on seven of the eight. When the same scale tuning is applied to all methods (Sec. 4.4, Table 3), CDQuant achieves 58.60% accuracy at 2 bits, compared with 58.91% for SchurQuant+T.

To separate solver from objective, Table 8 (App. D) runs SchurOpt and the CDQuant solver on identical GuidedQuant statistics (2 bits, before scale tuning):

SolverQwen3-8B PPLQwen3-8B Acc.Llama3-8B PPLLlama3-8B Acc.
CD (CDQuant)33.260.134.856.1
SchurOpt28.262.230.856.4

SchurOpt achieves lower PPL and higher accuracy while using a solver that is 2.4-8.0x faster.

On the plain reconstruction objective with identical calibration statistics (Qwen3-0.6B, first block, 2 bits), the relative output error (%) is 18-23% lower for SchurOpt:

SolverQKV
CDQuant reproduction (OWC →\rightarrow OWC-CD →\rightarrow CD)1.911.448.13
SchurOpt (integer zero-point)1.481.186.58

Q1b. FOEM at 2 bits and its 4-bit numbers. We used the official FOEM implementation in GPTQModel without modifications (α=0\alpha=0, β=0.2\beta=0.2, within the recommended range), on a single H100 SXM 80GB GPU. All methods share the original paper's calibration setup (128 C4 samples, 2,048 tokens) and group size (128). While the original paper evaluates only 3-4 bits, FOEM retains GPTQ's nearest-level rounding with a first-order correction, leading to similar failures at 2 bits.

W1b. Large drop from FP16 at 2 bits. The 2-bit mean over the eight models is now 53.41% without gradients and 58.91% for SchurQuant+T (FP16 68.40%):

2-bit SchurQuant+TAcc. (%)FP16 Acc. (%)PPLFP16 PPL
Qwen3-8B67.5474.0913.659.72
Llama2-13B67.0573.296.874.88

At 4 bits, Qwen3-0.6B and Qwen3-4B are within 0.5 pp of FP16.

W2. Quantization cost. To reduce overhead, we further optimized the implementation of SchurQuant during the discussion period. Quantization time for 2-bit Qwen3-4B on one H100 SXM 80GB GPU (App. C, Table 6):

GPTQFOEMQEPLoaQQuantEaseCDQuantSchurQuant
Time (s)297.67345.00339.102682.642971.922329.51458.53

SchurQuant+T takes 17.6 min for Qwen3-4B, 24.5 min for Llama2-7B, and 27.5 min for Qwen3-8B on one H100 SXM 80GB GPU, including the gradient pass and scale tuning. We quantize Llama-3.3-70B with SchurQuant on a single H100 SXM 80GB GPU, keeping only one block on the GPU at a time. The entire quantization process takes approximately 3 hours. At 2 bits, SchurQuant achieves 73.16% accuracy, compared to 72.05% for CDQuant, 37.52% for QEP, and 81.22% in full precision. Results on the larger models (App. G, Table 12; PPL / mean accuracy in %):

ModelMethod2-bit3-bit4-bit
Llama-3.3-70BFP163.86 / 81.22
QEP66.15 / 37.526.04 / 78.604.41 / 80.86
CDQuant12.90 / 72.055.73 / 80.694.45 / 80.84
SchurQuant14.15 / 73.165.78 / 79.874.35 / 80.66
Qwen3-30B-A3BFP168.71 / 77.49
QEP32.31 / 46.779.66 / 72.258.93 / 74.06
CDQuant24.06 / 60.599.88 / 71.709.05 / 74.92
SchurQuant16.88 / 65.269.50 / 74.178.91 / 75.13

At 2 bits, SchurQuant exceeds the strongest baseline, CDQuant, by 1.1 pp on Llama-3.3-70B and by 4.7 pp on Qwen3-30B-A3B.

Q2. Old and small models. We added Llama-3.3-70B, Qwen3-30B-A3B, and Qwen3.5-9B (App. G, Tables 12-13). Qwen3.5-9B is the most recent model in our evaluation and uses a different block design (gated linear-attention blocks interleaved with full attention). As a language model (PPL / mean zero-shot accuracy in %; FP16: 8.65 / 75.04):

Method2-bit3-bit4-bit
GPTQ41.23 / 42.709.71 / 71.298.93 / 75.38
FOEM95.47 / 40.579.24 / 68.818.84 / 74.56
QEP45.38 / 44.769.33 / 71.488.46 / 75.19
CDQuant26.16 / 61.348.64 / 74.668.19 / 74.51
SchurQuant22.24 / 63.618.72 / 75.048.37 / 75.84
SchurQuant+T11.19 / 68.698.48 / 74.848.20 / 75.16

At 2 bits SchurQuant exceeds CDQuant by 2.3 pp and SchurQuant+T reaches 68.69%; at 3 bits SchurQuant matches full precision (75.04%); at 4 bits all methods are within 1.3 pp of each other.

Thank the authors for the detailed responce with many new results. I'm glad that the authors complemented the accuracy with more PTQ baselines, which is definitely helpful to clarify this work's contribution with other similar works. The authors further make supplementary clarifications about a key difference on backpropagation during offline quantization, which is very important.

Another point is that the authors has optimized the implementation of SchurQuant, largely reducing the offline quantization time, which is very impressive and important, because as I stated in my initial rebuttal comment, these time cost is actually worth deep discussion because the concerns about infrastructure and deployment are real. But I didn't see any optimization details in the rebuttal comments or the appendix. Could the authors briefly discuss about it in the final pdf version?

Overall, the hard effort authors made during the rebuttal phase definitely enhance the main claim and the significance of this work. I have raised my rating from 2 to 3.

We sincerely thank the reviewer for the positive feedback and for acknowledging the improvements we have made during the rebuttal period. We are glad that these improvements and additional experiments helped clarify the contributions and practicality of our research. We also appreciate the suggestion regarding optimizations of the quantization process. We agree that providing details on these improvements is essential for the reproducibility of SchurQuant. In the final version of the paper, we will include further details on these optimizations. Thank you again for your constructive comments and for reconsidering your assessment of our work.

Reviewer_Tti5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Motivated by the concept of Schur Complement, this paper introduces an evaluation mechanism designed to anticipate and leverage future compensation for group-wise quantization errors. The authors validate the performance of their approach under extremely low-bit quantization regimes across a diverse set of LLMs.

Critical strengths
  • By formulating the quantization strategy around the anticipated future compensation of quantization errors, this work effectively addresses a critical blind spot in prior group-wise quantization schemes.

  • The proposed method demonstrates practical value for real-world deployment scenarios characterized by stringent memory and compute constraints, such as edge computing and mobile devices—where extremely low-bit quantization is strictly required.

Critical weaknesses
  • While the empirical results demonstrate clear superiority at the extreme 2-bit regime, the proposed method fails to maintain a consistent advantage over existing baselines in the 3-bit and 4-bit quantization settings.
Questions for authors
  • Broader Architectural and Task Validation: The current empirical evaluation is predominantly restricted to standard decoder-only Transformer-based LLMs and purely language-centric tasks. To fully establish the robustness, versatility, and general applicability of the proposed quantization strategy, further exploration is highly encouraged across Multimodal Large Language Models (MLLMs)—including both unified single-Transformer and Mixture-of-Transformers (MoT) architectures, as well as on complex multimodal downstream tasks.
3 comments

We sincerely appreciate the reviewer's careful evaluation and insightful comments. We believe that addressing these points will help clarify the key contributions and strengthen the presentation of our work. We respond to each concern in detail below.

W1. No consistent advantage at 3 and 4 bits. We agree that the differences are small at higher bit widths, as noted in Sec. 4.3. With finer quantization grids, nearest rounding is already close to the optimal assignment, and at 4 bits, the four strongest backpropagation-free methods differ by at most 0.3 pp in mean accuracy. When the same scale tuning is applied to all methods (Sec. 4.4), the differences among the seven methods are within 0.9 pp at 3 bits and 0.8 pp at 4 bits. SchurQuant+T achieves the highest 4-bit mean accuracy (68.01%, compared to 68.40% in full precision), but only by a small margin. Our main advantage is at 2 bits, which we now make clear in the revised paper.

Q1. Multimodal LLMs and multimodal tasks. We added a vision-language model, Qwen3.5-9B (App. G, Table 13). Its language model, which also uses a different architecture (gated linear-attention blocks interleaved with full attention), is quantized with text-only calibration and the vision encoder kept in full precision. All methods are evaluated with lmms-eval under identical settings (greedy decoding, no thinking, float16) on MMMU, MMBench-EN, AI2D, ChartQA, TextVQA, and POPE:

BitsMethodPPLMMMUMMBench-ENAI2DChartQATextVQAPOPEVLM avg.
-FP168.6556.3383.9384.7886.4883.7889.5880.81
2SchurQuant22.2435.3360.1455.4731.1262.6144.3748.17
2SchurQuant+T11.1940.0060.7442.9453.7269.5685.5358.75
2GPTQ41.2323.440.260.320.040.050.004.02
2FOEM95.4725.220.170.870.000.020.004.38
2QEP45.3822.890.694.310.000.020.004.65
2CDQuant26.1626.7817.100.000.001.950.037.64
3SchurQuant8.7253.5681.1980.1581.9680.1887.2777.39
3SchurQuant+T8.4853.1181.6281.8771.4879.9688.5176.09
3GPTQ9.7145.6770.8862.8672.9675.0988.2969.29
3FOEM9.2448.5668.0465.6178.3678.4988.9471.33
3QEP9.3347.0056.960.522.6041.2046.2632.42
3CDQuant8.6455.2284.1183.5579.7681.0987.5378.54
4SchurQuant8.3753.6782.7384.8485.9282.7288.1079.66
4SchurQuant+T8.2055.3383.9385.9185.6882.9887.5780.23
4GPTQ8.9353.2273.2873.5185.1282.4888.0475.94
4FOEM8.8456.3381.1084.6884.0482.0788.7079.49
4QEP8.4654.8982.0479.6383.7282.7786.7678.30
4CDQuant8.1955.2284.8881.7485.8083.3188.4979.91

At 2 bits no baseline yields a usable vision-language model: the mean over the six benchmarks is 4.0-7.6 for GPTQ, FOEM, QEP, and CDQuant (full precision 80.81). SchurQuant retains 48.17, and SchurQuant+T raises MMMU / ChartQA / TextVQA from 35.33 / 31.12 / 62.61 to 40.00 / 53.72 / 69.56. The gap to full precision remains large at 2 bits, which we report as a limitation of text-only calibration. We also added a mixture-of-experts model, Qwen3-30B-A3B (App. G, Table 12): at 2 bits SchurQuant reaches 65.26% against 60.59% for CDQuant and 46.77% for QEP (full precision 77.49%). Mixture-of-Transformers architectures are left for future work.

I acknowledge the authors' response to the reviews. After thoroughly evaluating the rebuttal and the perspectives shared by other reviewers, I will maintain my original recommendation.

We sincerely appreciate your positive evaluation and constructive feedback. We particularly appreciate your suggestion to explore multimodal models, which led us to discover the substantial performance advantage of our method over existing approaches in the challenging 2-bit quantization setting (48.17% vs. 7.64% for the strongest baseline). We are grateful for your valuable comments that helped us improve our work.

Comments on the paper

1 comment

We thank the reviewers. The main changes are:

  1. Faster, backpropagation-free SchurQuant. We keep the SchurOpt solver unchanged but replace the original teacher-decision token weighting, which required full-model recalibration before each layer, with a forward-only estimate of end-loss sensitivity based on the Fisher information of the output distribution (Sec. 3.5). Together with implementation optimizations, SchurQuant is now 5-6x faster than CDQuant, LoaQ, and QuantEase. Quantizing Qwen3-4B at 2 bits takes 459 s, down from 253.7 min (GPTQ: 298 s).

  2. Accuracy without gradients (Tables 1--2). SchurQuant achieves the highest mean accuracy among seven backpropagation-free methods at 2 bits (53.41% vs. 51.62% for CDQuant), leading on seven of eight models. It also has the highest mean at 3 bits (64.92% vs. 64.42%). At 4 bits, the results are similar across methods (67.28% vs. 67.41%).

  3. Results with gradients (new Sec. 4.4). We additionally evaluate SchurQuant+T, which combines GuidedQuant [1] token weights with end-to-end scale tuning, both adopted from prior work. It achieves 58.91% mean accuracy at 2 bits (Qwen3-8B: 67.54% accuracy and 13.65 PPL, compared to 74.09% and 9.72 in full precision). We apply the same scale tuning to all six baselines. SchurQuant+T retains the highest 2-bit mean (58.60% for tuned CDQuant) and the lowest PPL on seven models. At 3-4 bits, the tuned methods differ by at most 0.9 percentage points. Most of the accuracy improvement comes from scale tuning rather than the solver itself.

  4. Additional baselines and solver comparison. We add CDQuant, QuantEase, and LoaQ across all eight models at 2-4 bits (Tables 1-2). We also compare SchurOpt directly against the CDQuant solver using identical statistics (App. D, Table 8).

  5. Larger, MoE, and multimodal models (App. G). We extend the evaluation to Llama-3.3-70B and the MoE model Qwen3-30B-A3B. At 2 bits, SchurQuant achieves 73.16% and 65.26% accuracy, compared to 72.05% and 60.59% for CDQuant. We also evaluate Qwen3.5-9B on six vision-language benchmarks. At 2 bits, SchurQuant averages 48.17, while the four baselines score between 4.0 and 7.6.

  6. Inference latency (App. H). We add TTFT and TPOT measurements using vLLM.

Mean zero-shot accuracy (%) over the eight models of the main tables (FP16: 68.40):

MethodBackprop.2-bit3-bit4-bit
GPTQno39.2562.6966.86
FOEMno37.5061.9066.46
QEPno41.1862.9566.86
QuantEaseno38.9962.4667.30
LoaQno42.3163.7067.59
CDQuantno51.6264.4267.41
SchurQuantno53.4164.9267.28
CDQuant+Tyes58.6065.5967.75
SchurQuant+Tyes58.9165.5868.01

[1] J. Kim et al., "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance," ICML, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.