- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper modifies a recent quantization algorithm by Feng et al, which aims at B-bit quantizing and reconstruction of a vector x. The algorithm by Feng et al. relies on multiplying x by a diagonal Radamacher matrix, followed by normalized Hadamard matrix, and flooring a dithered and compressed version of each entry. The novelty of the current paper with respect to Feng et al. is the choice of the dither, where a pairwise independent noises U_i replace a scalar noise. The authors show unbiasedness and smaller variance w.r.t Feng et al.
Critical strengths
A simple and clear improvement of an existing quantization algorithm.
Critical weaknesses
- I’m afraid that the authors did not do a good job in translating and simplifying this AI-generated proof. The paper is extremely jargon heavy, and would probably not mean much to readers that did not read Feng et al. very recently. Some examples:
a. I didn’t see the point of Section 1.3 (“Technical overview”). I did not mean much to me in first pass (e.g., \Phi undefined, Q_U undefined) , and all details are explained later, so second read is not helpful either. I believe it can be removed.
b. The role, definition, and intuition behind G_B in Section 2.2 are missing.
c. The term “q” in Proposition 3.2 apparently refers to something in Algorithm 1, without it being noted anywhere in that proof or in that section.
d. I could not decipher why (3.5) is true. More detailed transitions would have helped.
- There are no experiments to validate the performance in practice, and code is not provided for reproducibility.
Questions for authors
A few suggestions that the authors might consider in order to improve the paper’s score:
-
Implement their algorithm in code and upload to a public Github repository.
-
Provide a better and more self-contained introduction to Feng at al.
-
Demonstrate improved practical performance over state-of-the-art quantizers, in particular that of Feng et al.
-
Improve writing and notations, several suggestions are given above.
Additional space could be saved by removing Section 1.3, and avoiding unnecessary display equation (an annoying tendency of AI generated proofs), e.g., (1.1), (1.2), among others.
3 comments
I thank the authors for considering my comments and suggestion. I ask the authors to make their code publicly available to the research community via an openly available online repository. , I am raising my score to “weak accept.”
Thank you for your kind response. We have uploaded our code as supplementary material on OpenReview, where it is publicly available. Since the supplementary material stays attached to the paper, we think it is a convenient place for the community to access the code.
We thank the reviewer for the detailed comments on presentation, and we have revised the paper accordingly.
Writing. We revised the writing to explain each object in plain terms before it is used. In particular, we rewrote the technical overview in Section 1.3 (see the general response). It now explains, in order, what we use from Feng et al., why a single stage with a shared dither fails (with the exact example x=y=e1), and why pairwise-independent dithers fix it.
Section 2.2 now explains the role of GB: mapping the cell midpoint back through the nonlinear F−1 would be biased, and GB corrects this bias so that averaging over the dither gives exactly t. Appendix A recalls its construction from Feng et al. (their Lemma A.8).
The vector q in Proposition 3.2 is now defined in the proof: q=(q1,…,qd) is the vector of decoded scalars in Algorithm 1, so that qi=QUi(zi) and x^=DH⊤q/d.
We also expanded the derivation of Equation (3.5) in the proof of Proposition 3.3. Since D2=I and H⊤H=I, we have x=DH⊤z/d, so x^−x=DH⊤e/d with e=q−z. Since D is symmetric, ⟨y,DH⊤v⟩=⟨HDy,v⟩, and (HDy)i=∥y∥2wi/d by the definition of wi. Together these give ⟨y,x^−x⟩=d1∑i(HDy)iei=d∥y∥2∑iwiei, which is Equation (3.5).
Experiments. We added experiments in Section 6 and Appendix B, following the setup of the recent arXiv version of Feng et al. (summarized in the general response). For example, at a fixed total number of bits per coordinate, our inner-product error is 24 to 43 times smaller than that of the two-stage quantizer of Feng et al. on random query–data pairs. The experiments also include a synthetic example where a shared dither fails. For x=y=e1, the normalized error of our quantizer with one shared dither grows linearly in d and reaches about 104 at d=4096, while that of our quantizer with pairwise-independent dithers stays between 1 and 5. We will release the code with the final version.