acceptodds
Under review as a conference paper at ICLR 2027

QALF: Deployment Error as a Regularizer for Quantization-Aware Low-Rank Factorization

Abstract

Low-rank factorization reduces a network's parameter count, while quantization reduces the bits used to store each parameter. Combining them usually means fitting the factors in floating point and rounding them afterwards. Factors that reconstruct a layer accurately can therefore perform poorly at the precision used for storage. We propose QALF, which accounts for quantization during fitting. Under an independent-noise model, we derive each factor's expected contribution to layer-output error in closed form. Adding these contributions to the input-weighted reconstruction loss gives a first-order approximation to the error after quantization. The quantization model determines the regularizer's form, and the target bit-width sets its weight, with no separately tuned trade-off coefficient. On ResNet-18/ImageNet, QALF outperforms dense GPTQ at comparable stored sizes up to the dense 3-bit budget. At 4-bit weights and 8-bit activations, it improves accuracy over the input-weighted reconstruction baseline by 1.8–2.8 percentage points with GPTQ and 4.1–4.7 with round-to-nearest, the rounding rule the objective models. It also exceeds CP-ADMM on its native grid, the closest quantization-aware factorization method, by 0.95–4.43 points at 4 bits. QALF remains more accurate than a tuned component-norm penalty combined with an optimized gauge, despite having higher kernel reconstruction error. The same derivation extends to matrix factorizations: on MLP down-projections in six language models from 410M to 8B parameters, QALF lowers 3-bit perplexity by 1.0–5.4 against SVD-LLM; its factors lose 71–87% less to rounding. For normalized CP factors, we establish conditions under which the regularizer prevents diverging components and the objective attains a minimum.

Primary area:
optimization
Keywords:
low-rank factorization, CP decomposition, quantization-aware optimization, post-training quantization, neural network compression, low-precision inference, tensor decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jqpNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Considers quantization error when finding the factors of a low-rank approximation, via a closed-form formulation of how quantization of each factor affects the layer output, used as a regularizer whose weight is set by the weight's bit-width.

Critical strengths
  • Novel in deriving both the form and the weight of the quantization regularizer from the deployed quantizer.
Critical weaknesses
  • The paper is hard to read: it is overly dense and the motivation of formulas and numbers is often not spelled out. This applies to all paragraphs, not only to isolated passages. A method figure, a worked example and a clearer presentation of results and ablations would help.
  • The results center on a single model and dataset (ResNet-18 on ImageNet); the other CNNs and the language models are covered only by a single configuration or a single projection per block, so it remains unclear how far the results generalize to other models or modalities.
  • Define terms and variables before their first use, in particular terms that may be familiar to you but not to the broader network compression community, such as CP, orbit and gauge.
  • Clearly define variables such as the compression factor and the stored memory size.
  • Baselines are rerun rather than taken from the original papers and are not validated against published numbers where these exist; for CP-ADMM at 2×/W4A8 the published 67.75 is 0.43 points above the reproduction, which halves QALF’s margin there.
  • The closest prior idea, sensitivity-regularized CP (Phan et al., 2020), is mentioned but not compared against.
Questions for authors

None.

4 comments

(1/2) Thank you for your detailed review and concrete suggestions. In short:

  • Readability: a method figure now opens the paper, a worked example follows one layer through rounding, every term is defined at first use, and Section 6 answers one question per paragraph.
  • CP-ADMM: your check showed that our script rounded CP-ADMM's factors a second time. Corrected, our reproduction matches the authors' tabulated numbers within 0.1; QALF's margin at 2x goes from +0.95 to +0.62, and is +1.86 and +3.63 at 3x and 4x.
  • Phan et al. (2020): now compared; it trails QALF by 2.0 to 3.2 points under every rounder, each used as in its paper.
  • Scope: ResNet-50 and DenseNet-121 join Table 1, and the language-model results now cover six models, two tasks, held-out text and every linear layer.

Readability. We agree the submission was too dense, and we revised it for readers outside tensor decompositions, following each of your suggestions.

Figure 1 now gives the motivation in the introduction. You note that the motivation of formulas and numbers is often not spelled out. The new Figure 1 is drawn for a reader who has never met CP, the tensor factorization we use (defined below). Panels (a) and (b) show the method: one convolution is replaced by three small ones, the factors; QALF adds one term to their fit, which accounts for the rounding to come, with the bit-width as its input; and the rounding itself is left to any existing post-training method. Panel (c) shows the problem QALF solves on one real layer (ResNet-18's layer4.0.conv1, compressed 3x, factors stored in 4 bits). The standard approach, which we call CP-Σ, fits the factors to reproduce the layer's output on calibration inputs as closely as possible in floating point. QALF accepts a slightly worse fit in exchange for factors that survive rounding.

Relative error of the layer's output (%)CP-ΣQALF
In floating point0.480.59
After rounding to 4 bits (RTN)4.301.06
Predicted before rounding: floating-point error + price3.661.07

QALF starts slightly behind and ends with a quarter of CP-Σ's error. The term QALF adds to the fit, which the paper calls the price, is a closed-form estimate of the error that rounding each factor will add. It is computed from the factors before anything is rounded, and added to the floating-point error it gives the last row: within 15% of the measured error after rounding, and it ranks the two methods correctly. A new appendix works through this layer step by step. It is one of the two median layers of the 16 we factorize; across them, QALF's error after rounding is 0.05 to 0.38 times CP-Σ's.

Terms defined before use. Each is now defined where it first appears in the revision:

  • CP (canonical polyadic) decomposition (§1, Figure 1) replaces a k×k convolution by three small ones, a 1×1, a depthwise k×k and a 1×1 convolution, by writing its weights as a sum of R rank-one terms. It is the tensor analogue of a truncated SVD.
  • The gauge (§1, §3.3) is the freedom to rescale the factors without changing the layer: multiplying a column of one factor by 2 and the matching column of another by 1/2 gives the same convolution. The orbit (§3.3) is the set of factorizations these rescalings connect. They all give the same layer but not the same rounding error, which is why QALF chooses one.
  • A rounder (Figure 1) maps the floating-point factors to b-bit values after the fit. Round-to-nearest (RTN) and GPTQ, which corrects each rounding error with the weights not yet rounded, need no training; learned rounders (AdaRound, FlexRound, BRECQ) optimize the rounding on calibration data.
  • The grid (§3.2) is the set of b-bit values a channel can take: symmetric around zero, or affine, shifted by a zero-point.
  • The compression factor (§6.1) is the parameter count of the compressed layers before factorization divided by the count after. At 3x, the factors hold a third of the original parameters.
  • The stored size (the caption of the storage plot) is the memory the deployed network takes: the packed low-bit weights, their quantization scales and every layer left uncompressed.

Results organized by question. Section 6 answers one question per paragraph (for example, "Does the gain survive learned rounders?"), with the setting stated once per table.

(2/2)

Baselines. Thank you for checking CP-ADMM against its published number. It showed that our script rounded CP-ADMM's factors a second time, although they leave its fit already on their 4-bit grid; that second rounding explained most of the 0.43-point gap. With it removed, our reproduction matches the two numbers the authors tabulate within 0.1, and agrees with the two they only plot.

CP-ADMM, ResNet-18PublishedOurs (3 seeds)
2x, W4A867.7567.65 ± 0.08
2x, W8A868.5668.50 ± 0.02
3x, W4A8below 65 (their figure)64.84 ± 0.18
3x, W8A866.5–67 (their figure)66.58 ± 0.12

With the corrected baseline, QALF's margin at 2x and 4 bits is +0.62 (it was +0.95; against the published number it is +0.52). At 3x and 4x it is +1.86 and +3.63. Table 1 and the abstract carry the corrected numbers. CP-ADMM keeps its own rounding, since its quantizer is part of its fit.

The other CP baselines have no published number to match. CP-Σ is QALF without the price, and CP-EPC is CP-ADMM's starting point, rounded like the other methods. Both run on the same checkpoint, calibration images and rounder as QALF, so that only the factors differ. Both are fitted without rounding and rounded once afterwards, like QALF.

The revision also rounds the factors with the three learned rounders. BRECQ runs from its official code and AdaRound from the official QDrop code. For FlexRound we found no released CNN code and implemented it in the QDrop codebase. At 4 bits, on the ResNet-18 checkpoint both papers report on (about 71.0 in floating point), BRECQ reproduces its published result (70.68 against 70.70) and FlexRound comes within 0.32 (69.96 against 70.28).

Sensitivity-regularized CP (Phan et al., 2020). The revision now describes both of Phan et al.'s criteria and compares QALF with both. EPC (Phan et al., 2019) minimizes the intensity, the norms of the rank-one terms, at a bounded reconstruction error; it is Table 1's CP-EPC. Phan et al. (2020) minimize instead the sensitivity, how much the layer changes when every factor entry receives the same small random noise; we call CP with this criterion CP-sens. Like Phan et al., CP-sens keeps the reconstruction error of its starting CP, so it has no coefficient to tune.

QALF's price models the quantizer that will actually be used. Each channel of each factor receives the rounding noise of its own range at the target bit-width, the resulting error is weighted by the layer's real inputs, and its coefficient follows from the bit-width. It predicts the error after rounding, as in the example above. To first order, sensitivity is the special case with the same noise on every entry and no input data.

Deployed exactly as QALF, CP-sens is 2.0 to 3.2 points behind under every rounder:

FactorsRTNGPTQAdaRoundFlexRoundBRECQ
CP-sens62.7864.7565.3865.27 ± 0.0965.27
QALF65.9866.93 ± 0.1267.37 ± 0.0267.27 ± 0.1067.41 ± 0.04

Setting: ResNet-18 at 3x, 4-bit factors, per-channel affine grid, weights only, each rounder used as in its paper. Cells with ± are means over 3 seeds, the others seed 0. The gap narrows to 0.42 only when BRECQ reconstructs against the uncompressed network, which also repairs factorization error; QALF stays ahead on every seed.

One model and dataset. You are right that the evidence sat on ResNet-18. ResNet-50, which the submission covered in one weight-only configuration, now joins Table 1 with DenseNet-121 (3x, W4A8, 3 seeds), and QALF is the most accurate after rounding on both, +0.76 and +0.81 over the best baseline under GPTQ. The language-model table (Table 4) now covers six models from three families, Qwen3.5 included, three seeds each, on WikiText-2; the appendix adds MMLU and GSM8K on four of them and C4 as held-out text. Beyond one projection per block, with every linear layer factorized and 80% of each layer's parameters kept, the rounding damage (the perplexity that rounding adds) is 76–87% smaller for QALF's factors than for SVD-LLM's at 3 bits, and 61–73% smaller at 90% and 4 bits.

If a passage of the revised PDF is still hard to follow, please tell us which one and we will rewrite it this week. If the revision addresses your concerns, we would be grateful if you would reconsider your score.

I thank the authors for the extensive revision. The changes go in the right direction: the method is described more clearly, the comparison to existing work is fairer, and the conclusions are better supported. However, the revision does not only add experiments. It also corrects baseline numbers, changes the language-model evaluation protocol and models, and changes which coupling is used in the experiments. The headline gains are smaller as a result. Given the extent of these corrections, I am not confident that the experimental section has reached a stable state, and I believe it would benefit from a full round of review.

Thank you for reading the revision, and for noting that the method is now clearer and the comparison fairer.

Two of the corrections fixed errors in our evaluation. The first came from your comparison with the published results, which helped us discover that our script rounded CP-ADMM’s factors a second time; with that step removed, our runs match the original authors’ tabulated numbers within 0.1. The second concerned the Llama models, whose test windows were formatted differently from the calibration windows. This mismatch caused the seed instability reported in the submission. Once the two were formatted alike, the baseline became much more consistent across seeds (Llama-3.1-8B: 13.90 ± 3.89 becomes 9.33 ± 0.03, against 8.76 ± 0.02 for QALF). Both corrections improved the baselines and narrowed QALF’s margins, but QALF stays ahead: at 4 bits it beats CP-ADMM on its native grid by 0.62 to 3.63 points, and the gap grows with compression; on the Llama models it removes 68–69% of the rounding damage, close to the 71–76% of the other four models.

The coupling correction concerns our description of which variant each experiment used; the experiments themselves did not change. The new models and tasks, including Qwen3.5 in place of Qwen2.5, were added in response to the reviews.

The derivation, the objective and its coefficient, and the controls reported in the original Table 2 are unchanged, and in every comparison of the submission, QALF keeps its position relative to each baseline.

Reviewer_zWK6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

A - Paper Summary

The paper addresses a fundamental mismatch in model compression: low-rank factorization and quantization are usually applied in sequence. Factors are fitted in floating point and then rounded to low precision. This means a factorization that reconstructs a layer accurately in floating point can perform poorly when quantized. The authors propose QALF (Quantization-Aware Low-Rank Factorization), which fits CP (canonical polyadic) factors for the precision at which they will be stored.

The paper:

  • Derives a closed-form, first-order approximation of the expected output error after all factors are quantized under an independent channelwise noise model. This decomposes into the FP reconstruction loss plus three per-factor "deployment prices" (ΦA,ΦB,ΦC)(\Phi_A, \Phi_B, \Phi_C), weighted by a bit-width-dependent coefficient μ(b)\mu(b).

  • Shows that this regularizer selects a representative along the CP scaling orbit (fixing the gauge freedom) and prevents CP degeneracy (component norm blow-up), establishing well-posedness and coercivity on a canonical slice.

  • Demonstrates empirically that QALF outperforms dense GPTQ at comparable storage, CP-EPC, CP-DMM, and data-aware CP-Σ\Sigma baselines on ResNet-18/ImageNet, and extends the two-factor version to MLP down-projections in six LLMs.

B - Paper Contribution

(1) A closed-form single-factor deployment price under an independent channelwise noise model (Theorem 1);

(2) a coefficient-free objective Jb=L+μ(b)(ΦA+ΦB+ΦC)J_b = L + \mu(b)(\Phi_A + \Phi_B + \Phi_C), with the reconstruction–quantization trade-off fixed by the target bit-width and quantizer model;

(3) well-posedness/coercivity on the canonical slice (Theorem 2), with the orbit-restricted problem reducing to a convex SOCP for the absmax scale;

(4) an extension applied to LLM MLP down-projections.

C - Paper Validation

Models: ResNet-18/ImageNet (primary), ResNet-50, GoogLeNet; six LLMs across three families (Pythia, Qwen, Llama) for the two-factor case.

Baselines: dense GPTQ, CP-EPC, CP-DMM, data-aware CP-Σ\Sigma, plus gauge-only and oracle-tuned-penalty controls.

Bit widths 2–8; 4-bit factors with 8-bit activations the main setting.

Three seeds with shared calibration subsets.

Critical strengths
  1. The objective is coefficient-free and beats an oracle-tuned penalty. The trade-off between reconstruction and quantization is fixed by the target bit-width and quantizer model with no externally tuned trade-off coefficient. Table 2 shows QALF beats a penalty control whose λ\lambda was swept over [10−8,10−3][10^{-8}, 10^{-3}] and selected on the evaluation set in the control's favor. This is important because it removes the single most fragile hyperparameter in penalty-based compression and demonstrates that the gain is not a tuning artifact.

  2. The mechanism is isolated by a clean matched-initialization ablation. QALF and CP-Σ\Sigma share initialization, input weighting, and deployment, differing only by μ(b)(ΦA+ΦB+ΦC)\mu(b)(\Phi_A + \Phi_B + \Phi_C). The resulting +4.1/+4.5/+4.7 point gains under RTN (the rounding rule the price models) are thus attributable to the price rather than to solver or initialization differences. This is consequential because it converts a "our method is better" claim into a mechanistic one.

Critical weaknesses
  1. The positive (CP) results are essentially single-architecture. Only ResNet-18 is evaluated in full depth for the CP case; ResNet-50 and GoogLeNet appear briefly. The LLM evaluation covers only one projection per block. The paper's limitations section concedes that this tests "transfer of the derivation rather than end-to-end compression." This matters because the headline claim is a general design rule, but the evidence is a well-characterized special case.
Questions for authors
  • Can you report QALF's margin over the tuned-penalty+gauge control on the affine grid as the primary deployment result, rather than the symmetric-grid number?

  • How much of the gain survives if the GPTQ-aware coupling (Appendix B.2) is used throughout, given that GPTQ correlates perturbations and the price models independent RTN noise?

2 comments

Thank you for your careful review and for two precise questions.

Q1. The affine grid as the primary result. We agree, and the affine grid is now the primary result in Table 2. On it, QALF stays ahead of the tuned norm penalty plus optimized gauge at both compression ratios and on every seed.

CompressionQALFTuned norm penalty + gaugeMargin
3x66.93 ± 0.1266.43 ± 0.14+0.51
4x64.94 ± 0.1364.37 ± 0.17+0.57

Setting: ResNet-18, 4-bit factors, per-channel affine grid, GPTQ, weights only, 3 seeds; margins from the unrounded means. Against CP-Σ, the same fit with no penalty at all, the gain is +2.67 at 3x and +3.50 at 4x.

The penalty's coefficient λ\lambda is tuned on the evaluation set, which favours the control. Its best value still changes with the grid and the bit-width. At 4x it is 10−510^{-5} on the affine grid and 10−410^{-4} on the symmetric one; at 3x on the symmetric grid it is 10−510^{-5} for 4-bit factors and 10−710^{-7} for 8-bit ones. QALF has nothing to tune, since its coefficient μ(b)\mu(b) is computed from the bit-width.

The paper derives the price for the symmetric grid, so in the table above QALF is fitted with that price and deployed on the affine grid. With the price re-derived for the affine grid, the 3x margin widens to +0.67 (QALF 67.09 ± 0.09, ahead on all 3 seeds).

Q2. The GPTQ-aware coupling throughout. All of the gain survives. App. B.2 gives two GPTQ-aware couplings, a static one and the exact sequential one. We reran the paper's ablation setting with only the coupling changed. Setting: ResNet-18, symmetric grid, 4-bit factors, GPTQ, BN recalibration, seed 0. Accuracy moves by at most 0.07 points, about the spread between seeds (0.06 at 3x), and the gain over CP-Σ is unchanged: +2.0 to +2.1 at 2x, +4.0 at 3x and +6.0 at 4x. These gains are larger than in Q1 because the symmetric grid rounds CP-Σ's factors worse. The coupling does matter before BN recalibration. At 4x, accuracy at that point is 56.44 with the RTN coupling and 58.35 with the static GPTQ one, and recalibration then closes the difference.

On your point that GPTQ correlates the perturbations, the exact coupling models this. On factor AA it predicts GPTQ's actual error to within 16% on every one of the 16 layers (measured over predicted 1.00, from 0.84 to 1.09). The RTN price over-predicts it about threefold (0.31).

Part of the paper already used the static coupling. Table 2, the storage plot and the language-model results were run with it (on LL; RR is priced with the trace), and only Table 1 with the RTN one. The text of §5 and App. B.2 said RTN throughout; the revision corrects it.

Scope beyond ResNet-18 and one projection. You note that the headline claim is a general design rule while the evidence is one well-characterized case. We agree this was the main gap, and the revision tests the rule in four more places:

  • ResNet-50 and DenseNet-121 in Table 1 (3x, W4A8, 3 seeds). ResNet-50 is now tested on both torchvision checkpoints: V1 in the ablation setting, as submitted, and V2 (80.3%) in Table 1; DenseNet-121 is new. QALF is the most accurate after rounding on both, under RTN and GPTQ: +0.76 and +0.81 points over the best baseline under GPTQ. On ResNet-50 the trade of Figure 1 shows at the scale of the network. In floating point QALF is 0.43 below CP-EPC (EPC's intensity criterion, Phan et al., 2019), 78.33 against 78.76, and after GPTQ rounding it is 1.54 above, 78.07 against 76.53.
  • Every linear layer of every transformer block, not only the down-projections. Keeping 80% of each layer's parameters with 3-bit factors, the rounding damage (the perplexity that rounding adds) is 76–87% smaller for QALF's factors than for SVD-LLM's, on six models. Keeping 90% with 4-bit factors, it is 61–73% smaller, on four.
  • Tasks and held-out text on the down-projections, for six models from three families (Qwen3.5 included), 3 seeds each. QALF beats SVD-LLM after 3-bit GPTQ on every model and seed. On MMLU (the Qwen3.5 and Llama models) it keeps 2.0 to 4.0 more points, ahead in all 12 model-seed pairs, and on GSM8K 1.1 to 3.5 more (table in our reply to 2inT). On C4, never used for calibration, its rounding damage is 64–74% smaller.
  • Tucker-2, a second factorization format: pricing each stored factor makes the accuracy lost to rounding 76% smaller at 4 bits and 84% smaller at 3 bits (ResNet-18, 3x, one seed; Appendix D).

The limitations paragraph now describes this scope.

We also reworked the presentation. The introduction now opens on a method figure with a worked example on one layer, and each term is defined where it first appears.

I thank the authors for addressing my questions and concerns, and for making the mentioned revisions to the paper. I will maintain my positive score.

Reviewer_2inTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors formulate each factor’s expected contribution to layer-output error in closed form and show that adding those contributions to the input-weighted reconstruction loss gives a first-order approximation to the error after quantization. They experimentally show that the proposed method is better than GPTQ.

Critical strengths
  • The paper provides a theoretical analysis that the summation of three factors' contributions with the reconstruction loss is a first-order approximation to the expected error once all three factors are quantized.
Critical weaknesses
  • In Table 1, the rounding method is too old-fashioned. To quantize ResNet-18, AdaRound and FlexRound techniques are well-known methods, but the authors do not explore them at all. In addition, SmoothQuant, FlexRound, and LRQ papers show W4A8/W8A8 quantization, but they also do not explore them at all.
  • In Table 3, the authors only investigate the perplexity of WikiText-2. To validate the effectiveness of QALF, they should explore more difficult tasks like MMLU and GSM8K.
  • Overall, the models used (ResNet-18, Pythia, Qwen2.5, Llama-3.2) are too out-dated. To justify the efficacy of QALF, the authors should evaluate it on recent models such as Qwen3.5.
Questions for authors

N/A

2 comments

(1/2) Thank you for your review and for pointing us to learned rounders, harder tasks and recent models: all three are now in the revision.

In short: with AdaRound, FlexRound and BRECQ each used as in its paper, QALF's factors stay 0.8 to 1.0 points above the best baseline's at 3x, and QALF with plain GPTQ is above every CP baseline under any of them. On MMLU, QALF keeps 2.0 to 4.0 more points than SVD-LLM on four models, Qwen3.5 included.

Our submitted Figure 1 compared QALF with GPTQ on the uncompressed network at comparable storage, which made it easy to read QALF as an alternative to GPTQ. The two work at different steps. QALF fits the low-rank factors of each compressed layer in floating point, accounting for the error that rounding them will add (the paper calls it the price), but does not round them itself. A rounder, whether RTN, GPTQ, AdaRound, FlexRound or BRECQ, then quantizes these factors. The new Figure 1 shows the two steps. The question the paper asks is which factors to hand to the rounder. Below we run each rounder on QALF's factors and on those of its two closest baselines: CP-Σ, the same fit without the price, and CP-sens, the closest earlier criterion (Phan et al., 2020).

AdaRound, FlexRound and BRECQ. We ran all three on the factors.

FactorsRTNGPTQAdaRoundFlexRoundBRECQBRECQ, target = uncompressed network
CP-Σ (input-weighted, no price)53.1364.26 ± 0.1966.37 ± 0.0666.44 ± 0.0366.56 ± 0.0866.32 ± 0.09
CP-sens (sensitivity, Phan et al. 2020)62.7864.7565.3865.27 ± 0.0965.2767.21 ± 0.14
QALF65.9866.93 ± 0.1267.37 ± 0.0267.27 ± 0.1067.41 ± 0.0467.63 ± 0.09

Setting: ResNet-18 with its 16 3×3 convolutions factorized to a third of their parameters (3x), 4-bit factors on a per-channel affine grid, weights only. Top-1 accuracy (%); the uncompressed network scores 69.76. Cells with ± are means over 3 seeds, the others seed 0. The last column is BRECQ reconstructing each block against the uncompressed network instead of the factorized one.

QALF's factors are the most accurate under every rounder, and QALF with plain GPTQ (66.93) is above every CP baseline under every learned rounder used as in its paper (at most 66.56; the one exception is CP-sens with BRECQ's uncompressed target, 67.21). The learned rounders help CP-Σ most, because its factors round worst: from GPTQ to BRECQ it gains 2.30 points and QALF 0.48. With the uncompressed network as BRECQ's target (last column), every method can also repair its factorization error, and QALF stays ahead on each of 3 seeds. The margin over CP-Σ grows with compression. Under BRECQ it is +0.29, +0.85 and +1.41 at 2x, 3x and 4x, and under AdaRound +0.44, +1.00 and +1.44 (one seed, except at 3x).

We run BRECQ from its official code and AdaRound from the official QDrop code. We found no released CNN code for FlexRound, so we implemented it in the QDrop codebase, following the quantizer and training loop of its released language-model code. At 4 bits, on BRECQ's ResNet-18 checkpoint, which both papers use (about 71.0 in floating point, against 69.76 for the torchvision model above), BRECQ gives 70.68 against the published 70.70 and our FlexRound 69.96 against 70.28. Some difference from the paper is expected without the authors' CNN code; it applies alike to every set of factors, since the same implementation rounds all of them. The 2- and 3-bit checks are in the revision.

W4A8 and W8A8. Table 1 already reports both on ResNet-18 (4- or 8-bit factors, 8-bit activations). The revision adds ResNet-50, which the submission had only with weights quantized, and DenseNet-121, both at W4A8. We have not run SmoothQuant or LRQ during the discussion. LRQ learns the same kind of weight scaling as FlexRound, with low-rank scaling matrices, so it is a rounder in the sense above and would take QALF's factors as input like the five we ran. SmoothQuant moves activation scale into the weights before the factorization, which then prices their rounding. Our language-model results are weights only, the setting of SVD-LLM and of LRQ's weight-only experiments. The revision cites both and says this in the limitations.

(2/2)

MMLU and GSM8K. Added in the appendix for the Qwen3.5 and Llama models, three seeds each. Setting: in every block the MLP down-projection is factorized to 80% of its parameters, and both factors are rounded to 3 bits by GPTQ (group size 128). The baseline is SVD-LLM, the whitened truncated SVD at the same rank, which is also QALF's starting point. Accuracy in %, 5-shot, three-seed means:

ModelMMLU, uncompressedMMLU after rounding, SVD-LLM → QALFGSM8K, uncompressedGSM8K after rounding, SVD-LLM → QALF
Qwen3.5-0.8B48.341.2 → 44.139.56.1 → 7.3
Qwen3.5-2B53.951.6 → 53.759.320.8 → 24.3
Llama-3.2-3B56.639.5 → 43.526.33.4 → 5.0
Llama-3.1-8B65.352.3 → 54.850.610.6 → 13.8

On MMLU, QALF keeps 2.0 to 4.0 more points per model and is ahead in all 12 model-seed pairs. On GSM8K it keeps 1.1 to 3.5 more points and is ahead in 10 of 12; the other two trail by 0.2, inside the seed spread (± 0.5 to 2.3). GSM8K is far more fragile than MMLU to the factorization itself, before any rounding. SVD-LLM's factorization takes Llama-3.1-8B from 50.6 to 20.2 on GSM8K, against 65.3 to 59.0 on MMLU.

Perplexities are in the revised Table 4, and the appendix adds C4 as text never used for calibration. QALF is ahead on all six models and every seed, and its rounding damage (the perplexity that rounding adds) is 64–74% smaller than SVD-LLM's on C4 and 68–76% smaller on WikiText-2.

Recent models. Qwen3.5-0.8B and 2B (2026) join Table 4, as you suggested, in place of the two Qwen2.5 models. Table 4 keeps Llama-3.2-3B and Llama-3.1-8B, and the two Pythia models as its smallest cases. On the CNN side, ResNet-50 moves into Table 1 and DenseNet-121 is new (3x, W4A8, 3 seeds), with QALF the most accurate under RTN and GPTQ on both. We keep ResNet-18 as the main CNN benchmark because it is where AdaRound, BRECQ and FlexRound report their own results, so our runs of them can be checked against the published numbers.

If the revision addresses your concerns, we would be grateful if you would reconsider your score.

Comments on the paper

1 comment

We thank the reviewers for their feedback.

QALF fits low-rank factors for the precision at which they will be stored, with a closed-form rounding-error term whose coefficient follows from the bit-width. zWK6 wrote that the objective "removes the single most fragile hyperparameter in penalty-based compression and demonstrates that the gain is not a tuning artifact", and that the matched-initialization ablation "converts a 'our method is better' claim into a mechanistic one". jqpN found the method "novel in deriving both the form and the weight of the quantization regularizer from the deployed quantizer", and 2inT noted the first-order analysis of the three factors' contributions.

Generalization beyond ResNet-18 (zWK6, jqpN). ResNet-50 and DenseNet-121 are now in Table 1: at 3× compression with W4A8, QALF with GPTQ improves on the best baseline by 0.76 and 0.81 points. Tucker-2 is tested as a second factorization format.

Learned rounders (2inT). We ran AdaRound, FlexRound and BRECQ on ResNet-18; QALF's factors are the most accurate under each of them.

Language models (zWK6, 2inT). We replaced the two Qwen2.5 models with Qwen3.5, the recent family 2inT asked for, and added MMLU, GSM8K, factorization of every linear layer, and C4 as held-out text. With the down-projections factorized, QALF is ahead of SVD-LLM after rounding on every model and seed in perplexity, on WikiText-2 and C4, and on MMLU, and keeps 1.1 to 3.5 more points on GSM8K.

Readability (jqpN). A method figure, a worked example on one layer, and every term defined at first use.

Baselines (jqpN). Our runs of CP-ADMM match its published results within 0.1 points, and our new BRECQ and FlexRound runs match theirs within 0.02 and 0.32 at 4 bits. The revision also fixes two evaluation errors, explained in our reply to jqpN: our script rounded CP-ADMM's factors a second time, and the Llama test windows were formatted unlike the calibration windows (the Llama-3.1-8B baseline now reads 9.33 ± 0.03 across seeds, against 13.90 ± 3.89 submitted). Both fixes strengthened the baselines and narrowed QALF's margins, but QALF stays ahead: its 4-bit margin over CP-ADMM is now 0.62–3.63 points (0.95–4.43 submitted), and on Llama it removes 68–69% of SVD-LLM's rounding damage. A third correction concerns only the text, which now states which coupling each experiment used; no experiment changed. The derivation, the objective and its coefficient, and the gains over CP-Σ, the same fit without the rounding-error term, are unchanged (1.8–2.8 points with GPTQ, 4.1–4.7 with RTN), and QALF is the most accurate method after rounding in every column of Table 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.