- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AngularQ, a framework for low-bit LLM quantization-aware training based on random sign embeddings and a radial-angular decomposition of quantized inner products. The method uses hard low-bit forward operators while supplying geometry-aware surrogate gradients: a realized sampled radial component and a corrected tangent-space angular component. The paper proposes a unified sampled gradient intended to be efficient on GPUs and provides a teacher-student stationarity result.
Critical strengths
Novel perspective on surrogate gradients for extreme low-bit training. The analysis of multiple embedding families and the treatment of their finite-sample concentration properties provide more theoretical grounding than typical low-bit QAT methods.
Critical weaknesses
-
Compared with QuEST, AngularQ changes several components at once: the embedding construction, hard forward representation, quantizer/codebook, normalization and radial parameterization, and backward rule. The authors explicitly frame the comparison as one between complete operators, which is reasonable for end-to-end benchmarking, but this prevents attributing the large gains specifically to the proposed angular surrogate gradient.
-
In practice, the embedding is sampled once and fixed throughout training, while the weights and activations adapt to that same realization. Thus, the expectation-based unbiasedness statements do not directly characterize the gradient trajectory of the trained model. This issue is especially relevant for the discrete Rademacher/Walsh-Hadamard cases, where the guarantees are asymptotic and require delocalization assumptions, and for multibit Haar-frame training, where the paper explicitly uses a Gaussian joint-kernel approximation rather than claiming exact finite-dimensional population unbiasedness.
Questions for authors
-
Can the authors provide a matched-forward ablation that holds the hard forward operator, embedding/frame, normalization, codebook, and training schedule fixed while changing only the backward rule?
-
For the practical Haar-frame multibit implementation, can the authors quantify how closely the Gaussian joint-kernel approximation and the resulting supplied gradient match the corresponding finite-dimensional Haar population quantities during training?
-
Can the authors report measured training cost and inference cost for the practical implementations, including wall-clock throughput, peak memory, and any overhead from materializing or applying the embedding frames?
3 comments
Response to Authors
Thank you for the detailed clarification and for addressing both concerns.
Regarding R4-03, I appreciate the distinction between the population-level unbiasedness identities and conditional unbiasedness along a training trajectory with a fixed realization of the embedding. It is helpful to explicitly acknowledge that, once the frame is sampled and fixed, the learned weights and activations may adapt to that realization; hence the stated identities should not be interpreted as conditional unbiasedness of the actual training gradients.
The proposed uniform stationarity/recovery extension appears to be the right direction for addressing this issue. In the revision, please make the quantifiers and scope of the result especially explicit: namely, whether the high-probability event over the sampled frame holds uniformly over all admissible frame-dependent iterates/weights encountered during training, and which geometric assumptions must remain satisfied along that trajectory. It would also be useful to clearly separate (i) the population surrogate-gradient identity, (ii) the fixed-frame training behavior, and (iii) the uniform recovery guarantee in the main text, so readers do not conflate these claims.
Regarding R4-04, thank you for adding measured training-update time, peak GPU allocation, and packed structured-runtime/storage measurements. Reporting the cost of materializing and applying the frames is important, and the reported storage figures help clarify the practical overhead. Please ensure that the final paper presents these results with sufficient experimental context—for example, hardware, batch size, model/configuration, precision, and the corresponding baseline/QAT implementations—so that the wall-clock and memory comparisons are directly interpretable. If inference costs are not materially changed, it would also help to state this explicitly and report the relevant end-to-end inference measurements where applicable.
These additions substantially improve the paper, although I would like to verify the precise statement and proof of the uniform fixed-frame guarantee in the revision before considering this concern fully resolved.
Response to Reviewer qKuy: follow up
We thank the reviewer for the thoughtful follow-up and constructive suggestions. The manuscript has also been updated, with the changes highlighted in a contrasting color (purple). We have clarified the scope and quantifiers of the uniform fixed-frame recovery guarantee (please see the highlighted text following Theorem 2.2), included its proof in the appendix, and further distinguished population unbiasedness from fixed-frame training behavior (highlighted text following Eq. 7). We have also expanded the discussion of training and inference costs with the requested experimental details and qualifications (please see the updated experimental section and appendix). We appreciate the reviewer's careful consideration of these revisions.
Response to Reviewer qKuy
We thank the reviewer for the comments and suggestions. Please find below our detailed response.
R4-01. Matched forward, frame, normalization, codebook and schedule
Yes. In the added 140M experiment, both arms use identical hard forwards, frame tensors, normalization, binary codebook, FP initialization, training data and schedule. Only the full operand backward changes. Unified AngularQ achieves NLL 3.84313 versus 3.92692 for the adapted trust-masked STE, reducing PPL by 8.04% (Table 1). This answers the attribution question without changing the representation or quantizer.
R4-02. Finite-dimensional Haar multibit approximation
We evaluated the Gaussian joint-kernel approximation against a finite-dimensional Haar reference at the actual model widths, codebooks and trained-state correlations. At the 0.6B multibit endpoint, the maximum sampled relative kernel-derivative error is 0.0463%. Median supplied-gradient cosines against the population reference are 0.977 for inputs and 0.982 for weights. These measurements support the accuracy of the approximation in the examined trained states. The 0.0463% quantity concerns the kernel derivative, rather than the entire network gradient.
R4-03. Fixed frames, Rademacher/Hadamard and delocalization
We appreciate the reviewer’s thoughtful observation. The unbiasedness statements in our work are population-level identities: for fixed operands w,x independent of the random embedding R, the expected sampled gradient agrees with the gradient of the smooth population operator,
=\nabla_w\mathbb{E}_{\mathcal{R}}\left[\mathcal{Q}_i(w,x)\right]$$ with exact or asymptotic interpretations depending on the embedding distribution. These identities characterize the underlying expected angular geometry of the quantized operator and provide a principled target for constructing its backward surrogate. They do not assert conditional unbiasedness along the training trajectory, where the embedding is fixed and the learned weights and intermediate activations may depend on its realization. Importantly, our sampled gradients explicitly retain information from the realized embedding and quantized forward codes, rather than relying solely on the population gradient. We also recognize that this adaptivity issue is a shared concern raised by the reviewer and Reviewer jtZN. To address it theoretically, we have strengthened our stationarity-to-recovery result to a uniform guarantee under additional geometric nondegeneracy conditions (please see response R3-01 to Reviewer jtZN). Specifically, the new result removes the requirement that the candidate $w$ be fixed independently of $\mathcal{G}$ and establishes a high probability recovery, that holds simultaneously for all admissible frame-dependent weights satisfying the stated geometric conditions. The key observation is that the restricted null-space property in the original proof already holds uniformly over all binary-code discrepancies. We will include this extension as the Uniform Recovery corollary, immediately following Theorem 2.2, with its proof in the appendix. This extension strengthens the theoretical foundation of AngularQ by establishing that exact recovery can be certified uniformly, even when the candidate weights adapt to the same fixed random embedding used during training. ### R4-04. Wall-clock and memory overhead > Can the authors report measured training cost and inference cost for the practical implementations, including wall-clock throughput, peak memory, and any overhead from materializing or applying the embedding frames? Table 5 includes measured training-update time and peak GPU allocation, including materializing and applying the dense frames; sharing limits 0.6B frame storage to 56 MiB. Table 6 adds packed structured-runtime measurements with all model buffers counted: **4.28× smaller resident storage**, with only **8.56 KiB** of transform state at 1.7B. At a common 4 GiB allocator cap, packing permits four times as many concurrent sequences and increases aggregate throughput by **2.99×** at a 512-token prompt and **2.83×** at a 2K prompt. Table 4 provides trained structured-frame accuracy evidence at 140M and 0.6B; the 1.7B hardware prototype establishes execution costs.