- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Mixed-precision post-training quantization must assign per-layer bit-widths under a global memory budget using a small calibration set. This is a combinatorial problem whose sensitivity estimates can be slow to compute and fragile when calibration data are corrupted. The authors derive Cantelli-based probabilistic global and layerwise bounds on the quantization error that separate the error propagated from earlier layers from the local perturbation introduced at each layer. They turn the local term into a separable mean-plus-dispersion score, computed with one full-precision pass per calibration image and used by a solver-free greedy allocation rule.
Critical strengths
The allocation stays stable when the calibration inputs are corrupted. This is shown through matched backends, a CLADO control with identical candidate bit-widths, rank-correlation diagnostics of the allocations, and a replication on BSDS500.
In matched timings the allocation stage is 28 to 2,570 times faster than the studied baselines, and the scores can be reused when only the budget changes. This low cost makes per-layer allocation feasible, which closes the gap to CLADO at the tightest budget
The experimental protocol is careful and transparent, with shared quantizers and block partitions in each comparison, paired confidence intervals, qualitative examples chosen by fixed random seeds.
Critical weaknesses
The claim that the probabilistic nature of the approach brings robustness to corrupted data is not tested by any ablation of the score, such as a mean-only, max-based, data-free or varying delta score variant. The robustness may come from the dominance of weight-quantization error rather than from the probabilistic analysis, and this would change the main message of the paper.
The algorithm is a heavily simplified version of the theory, and the derived bounds are never computed in experiments. The theory therefore motivates the method but does not support it directly.
The evaluation uses only one denoising network, weight-only quantization and two masking corruptions, and the large gains appear only against CLADO. It is unclear whether the conclusions hold more broadly.
The image pairing with baselines could not be fully verified. The claim of improving the state of the art is too strong.
Questions for authors
How was CLADO’s loss computed under corrupted calibration, and does its collapse also appear with milder corruptions?
2 comments
Thank you for the detailed response and the additional experiment. I checked the image pairing in the diffusion experiment and found that the authors had corrected the related descriptions. My concern about this part has therefore been resolved.
I still have concerns about the theoretical part. The upper bound reported in the response is around to times larger than the actual error. The authors also acknowledged that the proposed score is heuristic. In this regard, the theory seems to serve mainly as motivation for the method, rather than providing direct support for the algorithm or the experimental results.
The other concerns have only been partially addressed. In the DRUNet experiments, the mean, max, and combined scores produced the same assignments across all 10 corruption conditions. The advantage of the combined score was also shown only in a limited set of experiments. Regarding the rank-matching experiment raised by the Bukf reviewer, the evaluation was performed with only one block quantized at a time. It is still unclear whether the same behavior holds when multiple blocks are quantized simultaneously at low bit-widths. The method is also still limited to weight-only quantization. I think these limitations and additional results should be included in the revised paper.
I also have a remaining concern about the CLADO comparison. According to the response, CLADO compares the output from a corrupted input against a clean image, while the proposed score does not require such a target. This makes it difficult to tell whether CLADO's performance drop under masking is due to its actual robustness or simply to the difference in the loss definition. The current results do not seem sufficient to distinguish between these possibilities. It would be helpful to include a CLADO variant that uses the FP32 output from the same corrupted input as its target.
Considering these remaining issues, I will keep my score at 2.
Thank you for your review. The additional experiments are described in our responses to m1Uc and Bukf.
1. Score ablations
Variants perform similarly on DRUNet. On heterogeneous FFDNet calibration, our score gains 4.7 dB over the mean and 3.5 dB over standard deviation alone. See m1Uc, point 1, for all ablations and allocation stability.
The data-free score ∑ℓ∈g∥Wℓ−Qb(Wℓ)∥F reaches 21.3 dB on the CIFAR-10 DDPM (A6), versus 25.1 dB for our score, on 32 images. Thus, removing dependence on calibration data does not necessarily improve quality.
These ablations support the combined score in the tested settings. We will revise the abstract to present robustness as an empirical finding.
2. Role of weight-quantization error
The controlled ablations in m1Uc, point 1, change only aggregation. Their PSNR gains show that the aggregation statistic matters even with the same local perturbations and allocation algorithm.
3. Link between the theory and the algorithm
Recomputing scores from partially quantized activations changes PSNR by at most 0.02 dB in our tests. See m1Uc, point 3, for the protocol and the equivalent exact decomposition supporting FP32 activations.
We have now evaluated recursive bounds on DRUNet. They exceed the actual error by factors of about 1041 with operator norms and With data-estimated factors, the gap drops to about 108 but still too loose to be used as the exact score for the algorithm. That is why we introduce our heuristic.
4. Scope of the evaluation
On FFDNet, we exceed CLADO by 0.9 dB with clean calibration and 2.9 dB with mostly periodic calibration noise. See Bukf, points 2 and 3, for the 10 corruptions, cross-dataset tests, and second diffusion model.
The benefits are not limited to CLADO. The paper already reports a 1.8 dB gain over OMPQ-ORM at 4 bits, with much faster allocation than OMPQ and AIMET. In an FFDNet comparison with calibration intensities multiplied by 4, AIMET loses 3.5 dB, versus 0.13 dB for our method. The final gap is 1.96 dB in our favor, at about 4.5 bits for both methods. In the DRUNet comparison with the AIMET backend under photon noise, AIMET changes 18 of 36 blocks, versus one for our method.
The paper and our experiments use weight-only quantization. Extending the local score to activation quantization is an interesting direction for future work. For the same input z and fixed quantizers, we can separate the local weight and activation errors:
Wz-Q_W(W)Q_A(z)=(W-Q_W(W))z+Q_W(W)\bigl(z-Q_A(z)\bigr).\tag{4}
This identity provides a starting point for the extension.
5. Image pairing in diffusion
We directly checked the initial noise tensors and the first U-Net input for 1,000 images. They are identical across FP32, our method, and the baseline examined.
On this verified set, our output is closer to FP32 in 992 of 1,000 cases. The mean PSNR gain is 10.1 dB, with a 95% confidence interval of [9.85,10.34] dB. This confirms the gain under directly verified image pairing. We will replace "improves the state of the art" with "matches or improves the studied baselines."
6. CLADO loss and milder corruptions
We use CLADO's official code for individual and pairwise sensitivities, followed by integer quadratic optimization. Following its task-loss formulation, we use the denoising loss
L(Q)=\operatorname{mean}\lVert f_Q(\widetilde x)-y\rVert_2^2,\tag{5}
where y is the clean image. Both methods use the same blocks, quantizer, candidates, and budget. Under corruption, only the calibration input changes.
The degradation appears with only 10% of pixels masked: at 4 bits, CLADO drops from 34.41 to 28.15 dB. Our allocation remains unchanged and gives higher PSNR on all 100 test images.