W1 [Major] The local gradient identity does not support the current strength of the theoretical framing. Locations: pp. 1–2, abstract/contributions; pp. 3–5, Eqs. 2–9; pp. 13–14, Appendix B.2.
The exact expectation in Eq. 2 does upper-bound the discrete minimum, but its first-order truncation J^(1) generally does not retain that upper-bound property, nor does its gradient necessarily provide a descent direction for the original expectation. A directly reproducible example is a grid {0,1}, s=1, z=0, mu=0.4, and L(w)=(w−0.2)^2. The MAP value is 0 and the discrete minimum loss is 0.04. The exact expectation is E[L]=0.28, with derivative 0.6 with respect to mu. The first-order surrogate at the MAP value is 0.04−0.4mu=−0.12, with derivative −0.4. Thus, the surrogate can fall below the discrete minimum and its gradient can point in the opposite direction to the gradient of the exact expectation.
This example does not refute the identity that STE is the gradient of this first-order surrogate. It limits the stronger interpretation that this identity establishes general optimization validity or eliminates the heuristic aspect. I suggest explicitly restricting the claim to a local surrogate interpretation, or supplying error/descent conditions and evidence that they apply at extreme low precision.
Another point requiring clarification is the gradient with respect to quantization parameters. The main text says that gradients through the expected weights are retained, whereas Appendix B.2 recommends STE pass-through based on the MAP term (a) for s, z, and g. MAP weights change with learnable codebook values and scales. Omitting Hessian-related terms therefore requires explicit stop-gradient definitions and conditions. Please provide automatic-differentiation pseudocode matching the actual implementation rather than using distribution peakedness as a blanket justification for all parameters.
W2 [Major] The claim that STE cannot learn non-uniform codebooks is too broad, and the ablations do not isolate the key factors. Locations: p. 5, §2.5; p. 7, §3.5; pp. 8–9, §5.1/Table 1.
With fixed hard assignments and a quantized value s(g_k−z), the selected g_k still has a direct gradient equal to s times the sum of the relevant upstream gradients. Cancellation of a derivative with respect to g under a particular parameterization of the SR expectation does not imply that every training scheme using STE for latent-weight updates is unable to learn non-uniform codebooks. The standard baseline fixes g to integer values and therefore cannot itself test the capability of “STE plus a learnable codebook.” An initialization ablation conducted only at 3-bit precision is insufficient to explain the main gains at 2-bit and ternary precision.
Suggested remedy: In the principal extreme low-bit settings, compare “Laplace plus a fixed non-uniform codebook,” “Laplace plus a learnable codebook,” and “hard nearest-neighbor assignments plus STE latent updates and direct codebook learning.” Control initialization, training budget, and actual storage. This would separate the effects of the assignment distribution, codebook capacity, and initialization.
W3 [Major] The complexity comparison contains a definite error. Location: p. 9, §5.2.
A factorized general softmax has K logits per weight, so its parameter storage/explicit probability tensor is O(NK), not K^N. The latter is the size of an enumeration of all joint assignments. The paper’s first-order expected-weight calculation likewise does not require enumerating joint assignments. Using one latent value in the Laplace construction does reduce the number of learnable parameters, and a fused kernel can avoid writing the probability tensor to global memory. These are different advantages from the claim that a general softmax must enumerate K^N configurations.
For the implementation described, which iterates over K terms, the computation also cannot simply be called linear in bit-width b when K=2^b. Please correct the distinctions among parameter storage, temporary memory, operation count, and the complexity of enumerating the exact expectation, and revise the efficiency comparison accordingly.
W4 [Major] The actual training protocol is not uniquely specified, limiting reproducibility and the connection between theory and experiments. Locations: p. 7, §3.5; pp. 13–14, Appendices B.2/C and Table 3.
Table 3 specifies sigma0=sigmamin=1.0 and a cosine schedule, whereas the immediately following text says that sigma decreases linearly from 2.0 to 0.9. This is not an inconsequential typo: both distribution sharpening and the justification for omitting gradient terms depend on temperature. The description of a “Laplace forward” pass in the main text also needs to be reconciled with the earlier hard-MAP forward description through the actual implementation.
Please provide the temperature configuration actually used, stop-gradient rules, constraints on the grid values, and learning rates, together with the necessary low-cost gradient checks. Where the appendix states that correction-term experiments did not help, the corresponding results and settings should be reported.
W5 [Major] The compression and deployment conclusions lack accounting for the actual representation. Locations: pp. 6–7, §§3.1/3.4; p. 12, related work; pp. 14–15, Appendix D/Table 4.
The labels “1.58/2/3-bit” primarily describe codebook cardinality. The storage precision and sharing scheme of the learnable per-group values are not sufficiently specified. For example, with group size 64 and K=4, storing the codepoints independently for each group in FP16 would add 16×4/64=1 bpw for the codebook alone. This is an illustration of why byte-level accounting is necessary, not an assertion that the implementation necessarily uses that representation. Similarly, K=3 and log2(3) alone do not establish that fixed-width storage achieves 1.58 bpw.
Table 4 explicitly reproduces Llama throughput from another paper. It is background evidence, rather than a measurement of an exported Qwen model using the non-uniform codebooks learned here. It cannot by itself establish that the present format preserves the same throughput in FLUTE.
Please report the actual serialized size, codebook/scale/zero-point overhead, and ternary packing scheme, and demonstrate export correctness and throughput for at least one checkpoint from this paper. This does not require developing an entirely new inference framework.
Positioning relative to prior work (supporting context):
The assertion that STE has lacked a principled explanation needs to be distinguished from existing work. Shekhovtsov and Yanush’s “Reintroducing Straight-Through Estimators as Principled Methods for Stochastic Binary Networks” and Ajanthan et al.’s “Mirror Descent View for Neural Network Quantization” already explain STE from other modeling or optimization perspectives. They do not automatically cover this paper’s multilevel MAP/SR derivation, but the novelty should be framed as this particular interpretation rather than as the first theoretical justification of STE in general.
https://openreview.net/pdf?id=F8lXvXpZdrL
https://proceedings.mlr.press/v130/ajanthan21a.html
This is a suggestion about novelty positioning. The main grounds for rejection are the inconsistencies between the claims and the implementation/experimental evidence discussed above, rather than the absence of a single citation.
Thank you for the positive review! Please find our replies below:
As to the assumptions and scope, the equivalence requires only that the assignment distribution be stochastic rounding and the first-order expansion around the hard assignments. It holds for any differentiable loss and any sorted grid, uniform or otherwise. Indeed, you can generate many different differentiable proxies for quantization just by changing the assignment distribution (Laplace categorical, softmax, or anything else), and STE falls out as a special case for stochastic rounding.
On the matter of \sigma, you are correct, we did not conduct a formal ablation study on its values and annealing schedule. Based on our preliminary experiments, we did find that the final performance is sensitive to the ending value of \sigma (setting it too small tended to hurt performance), but we did not explore this part of the hyperparameter space in depth.