- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Considers quantization error when finding the factors of a low-rank approximation, via a closed-form formulation of how quantization of each factor affects the layer output, used as a regularizer whose weight is set by the weight's bit-width.
Critical strengths
- Novel in deriving both the form and the weight of the quantization regularizer from the deployed quantizer.
Critical weaknesses
- The paper is hard to read: it is overly dense and the motivation of formulas and numbers is often not spelled out. This applies to all paragraphs, not only to isolated passages. A method figure, a worked example and a clearer presentation of results and ablations would help.
- The results center on a single model and dataset (ResNet-18 on ImageNet); the other CNNs and the language models are covered only by a single configuration or a single projection per block, so it remains unclear how far the results generalize to other models or modalities.
- Define terms and variables before their first use, in particular terms that may be familiar to you but not to the broader network compression community, such as CP, orbit and gauge.
- Clearly define variables such as the compression factor and the stored memory size.
- Baselines are rerun rather than taken from the original papers and are not validated against published numbers where these exist; for CP-ADMM at 2×/W4A8 the published 67.75 is 0.43 points above the reproduction, which halves QALF’s margin there.
- The closest prior idea, sensitivity-regularized CP (Phan et al., 2020), is mentioned but not compared against.
Questions for authors
None.
4 comments
(2/2)
Baselines. Thank you for checking CP-ADMM against its published number. It showed that our script rounded CP-ADMM's factors a second time, although they leave its fit already on their 4-bit grid; that second rounding explained most of the 0.43-point gap. With it removed, our reproduction matches the two numbers the authors tabulate within 0.1, and agrees with the two they only plot.
| CP-ADMM, ResNet-18 | Published | Ours (3 seeds) |
|---|---|---|
| 2x, W4A8 | 67.75 | 67.65 ± 0.08 |
| 2x, W8A8 | 68.56 | 68.50 ± 0.02 |
| 3x, W4A8 | below 65 (their figure) | 64.84 ± 0.18 |
| 3x, W8A8 | 66.5–67 (their figure) | 66.58 ± 0.12 |
With the corrected baseline, QALF's margin at 2x and 4 bits is +0.62 (it was +0.95; against the published number it is +0.52). At 3x and 4x it is +1.86 and +3.63. Table 1 and the abstract carry the corrected numbers. CP-ADMM keeps its own rounding, since its quantizer is part of its fit.
The other CP baselines have no published number to match. CP-Σ is QALF without the price, and CP-EPC is CP-ADMM's starting point, rounded like the other methods. Both run on the same checkpoint, calibration images and rounder as QALF, so that only the factors differ. Both are fitted without rounding and rounded once afterwards, like QALF.
The revision also rounds the factors with the three learned rounders. BRECQ runs from its official code and AdaRound from the official QDrop code. For FlexRound we found no released CNN code and implemented it in the QDrop codebase. At 4 bits, on the ResNet-18 checkpoint both papers report on (about 71.0 in floating point), BRECQ reproduces its published result (70.68 against 70.70) and FlexRound comes within 0.32 (69.96 against 70.28).
Sensitivity-regularized CP (Phan et al., 2020). The revision now describes both of Phan et al.'s criteria and compares QALF with both. EPC (Phan et al., 2019) minimizes the intensity, the norms of the rank-one terms, at a bounded reconstruction error; it is Table 1's CP-EPC. Phan et al. (2020) minimize instead the sensitivity, how much the layer changes when every factor entry receives the same small random noise; we call CP with this criterion CP-sens. Like Phan et al., CP-sens keeps the reconstruction error of its starting CP, so it has no coefficient to tune.
QALF's price models the quantizer that will actually be used. Each channel of each factor receives the rounding noise of its own range at the target bit-width, the resulting error is weighted by the layer's real inputs, and its coefficient follows from the bit-width. It predicts the error after rounding, as in the example above. To first order, sensitivity is the special case with the same noise on every entry and no input data.
Deployed exactly as QALF, CP-sens is 2.0 to 3.2 points behind under every rounder:
| Factors | RTN | GPTQ | AdaRound | FlexRound | BRECQ |
|---|---|---|---|---|---|
| CP-sens | 62.78 | 64.75 | 65.38 | 65.27 ± 0.09 | 65.27 |
| QALF | 65.98 | 66.93 ± 0.12 | 67.37 ± 0.02 | 67.27 ± 0.10 | 67.41 ± 0.04 |
Setting: ResNet-18 at 3x, 4-bit factors, per-channel affine grid, weights only, each rounder used as in its paper. Cells with ± are means over 3 seeds, the others seed 0. The gap narrows to 0.42 only when BRECQ reconstructs against the uncompressed network, which also repairs factorization error; QALF stays ahead on every seed.
One model and dataset. You are right that the evidence sat on ResNet-18. ResNet-50, which the submission covered in one weight-only configuration, now joins Table 1 with DenseNet-121 (3x, W4A8, 3 seeds), and QALF is the most accurate after rounding on both, +0.76 and +0.81 over the best baseline under GPTQ. The language-model table (Table 4) now covers six models from three families, Qwen3.5 included, three seeds each, on WikiText-2; the appendix adds MMLU and GSM8K on four of them and C4 as held-out text. Beyond one projection per block, with every linear layer factorized and 80% of each layer's parameters kept, the rounding damage (the perplexity that rounding adds) is 76–87% smaller for QALF's factors than for SVD-LLM's at 3 bits, and 61–73% smaller at 90% and 4 bits.
If a passage of the revised PDF is still hard to follow, please tell us which one and we will rewrite it this week. If the revision addresses your concerns, we would be grateful if you would reconsider your score.
I thank the authors for the extensive revision. The changes go in the right direction: the method is described more clearly, the comparison to existing work is fairer, and the conclusions are better supported. However, the revision does not only add experiments. It also corrects baseline numbers, changes the language-model evaluation protocol and models, and changes which coupling is used in the experiments. The headline gains are smaller as a result. Given the extent of these corrections, I am not confident that the experimental section has reached a stable state, and I believe it would benefit from a full round of review.
Thank you for reading the revision, and for noting that the method is now clearer and the comparison fairer.
Two of the corrections fixed errors in our evaluation. The first came from your comparison with the published results, which helped us discover that our script rounded CP-ADMM’s factors a second time; with that step removed, our runs match the original authors’ tabulated numbers within 0.1. The second concerned the Llama models, whose test windows were formatted differently from the calibration windows. This mismatch caused the seed instability reported in the submission. Once the two were formatted alike, the baseline became much more consistent across seeds (Llama-3.1-8B: 13.90 ± 3.89 becomes 9.33 ± 0.03, against 8.76 ± 0.02 for QALF). Both corrections improved the baselines and narrowed QALF’s margins, but QALF stays ahead: at 4 bits it beats CP-ADMM on its native grid by 0.62 to 3.63 points, and the gap grows with compression; on the Llama models it removes 68–69% of the rounding damage, close to the 71–76% of the other four models.
The coupling correction concerns our description of which variant each experiment used; the experiments themselves did not change. The new models and tasks, including Qwen3.5 in place of Qwen2.5, were added in response to the reviews.
The derivation, the objective and its coefficient, and the controls reported in the original Table 2 are unchanged, and in every comparison of the submission, QALF keeps its position relative to each baseline.
(1/2) Thank you for your detailed review and concrete suggestions. In short:
Readability. We agree the submission was too dense, and we revised it for readers outside tensor decompositions, following each of your suggestions.
Figure 1 now gives the motivation in the introduction. You note that the motivation of formulas and numbers is often not spelled out. The new Figure 1 is drawn for a reader who has never met CP, the tensor factorization we use (defined below). Panels (a) and (b) show the method: one convolution is replaced by three small ones, the factors; QALF adds one term to their fit, which accounts for the rounding to come, with the bit-width as its input; and the rounding itself is left to any existing post-training method. Panel (c) shows the problem QALF solves on one real layer (ResNet-18's layer4.0.conv1, compressed 3x, factors stored in 4 bits). The standard approach, which we call CP-Σ, fits the factors to reproduce the layer's output on calibration inputs as closely as possible in floating point. QALF accepts a slightly worse fit in exchange for factors that survive rounding.
QALF starts slightly behind and ends with a quarter of CP-Σ's error. The term QALF adds to the fit, which the paper calls the price, is a closed-form estimate of the error that rounding each factor will add. It is computed from the factors before anything is rounded, and added to the floating-point error it gives the last row: within 15% of the measured error after rounding, and it ranks the two methods correctly. A new appendix works through this layer step by step. It is one of the two median layers of the 16 we factorize; across them, QALF's error after rounding is 0.05 to 0.38 times CP-Σ's.
Terms defined before use. Each is now defined where it first appears in the revision:
Results organized by question. Section 6 answers one question per paragraph (for example, "Does the gain survive learned rounders?"), with the setting stated once per table.