- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper concerns random rotations which are used in quantization of neural networks. The theory of TurboQuant assumes that the random rotation is a dense matrix chosen according to a Haar measure, but practical implementations need to have a fast way to multiply with the random rotation matrix, so they use Hadamard-structured random matrices. Previous works have addressed this theoretical gap by bounding the distance (e.g. Wasserstein) of probability laws arising from the Hadamard-structured versus the dense random matrices. This paper instead directly bounds the resulting quantization distortion, giving an "improved" error bound for the distortion that can not be deduced from the previous results about distances of probability laws.
Critical strengths
New error bound. The paper provides a new error bound on a distortion measure, which can not be deduced from the previous bounds that focused on distance of probability laws (and not on the final distortion).
Critical weaknesses
Readability. The theory is compressed to two pages of dense theorems that are nontrivial to parse. The paper doesn't provide intuition or discussion of the results. The language in the paper also reads "claudish", and indeed the AI use statement suggests that the paper was written mainly by AI. (e.g. "yardstick", "What is not new.", "What matters more than the rotation.", "The harness is sound, and Qwen’s keys are the problem.") Specialized notation/terms are not explained (e.g. vNMSE, SIFT, etc). The formatting also breaks in places, for example many expectations are rendered lower, e.g. at the start of line 197 or in corollary 1.
Especially in times where the mathematical results themselves are discovered/proved by AI, the value comes from clearly communicating and explaining the results and putting them in context, and the paper fails at that.
Questions for authors
- What is the purpose and context of the experiments? You label "two-RHT" as "yours", but as you say in the paper, this has been studied in previous papers, no? I generally don't understand the relation between the theory and the experiments: The theory seems to provide a new error bound for an existing method, while the experiments seem to compare different methods (?). Shouldn't the experiments focus on demonstrating that the error bound holds in practice and how it can potentially inform certain choices (e.g. of hyperparameters) in the method?