acceptodds
Under review as a conference paper at ICLR 2027

SQuAT: Spherical Lattice Vector Quantization-Aware Training

Abstract

Vector quantization (VQ), in particular lattice-based methods, offer strong rate-distortion trade-offs for low-bit large language models (LLMs), yet their behavior under quantization-aware training (VQ‑QAT) remains poorly understood. We scale lattice VQ‑QAT to billion-parameter LLMs and multi-billion-token training regimes, and introduce a shape–gain VQ‑QAT approach, SQuAT, that decomposes weights into direction and magnitude: normalized directions are quantized using an E8-derived spherical codebook, while gains are quantized independently with low-bit scalar quantizers. We analyze the interaction of shape-gain parameterizations and straight-through estimators (STE) in the context of VQ optimization, and show that factored STEs induce tangent-space updates for directions, yielding an implicit form of spherical (Riemannian) optimization. To enable scalability, we develop a distributed exact nearest-neighbor assignment scheme, in which workers quantize disjoint weight blocks and communicate only discrete indices, reducing VQ-QAT training time by 12.8x and allowing us to scale it to 10B tokens. Across instruction-tuned LLaMA-3.2-1B/3B, Qwen3-4B, and base Qwen3-1.7B, SQuAT consistently outperforms standard VQ‑QAT STE under identical quantizers and initialization, and achieves SOTA VQ-QAT accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Quantization, Quantization-Aware Training, Vector Quantization, Lattice Quantization, LLMs

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_J7ZNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes SQuAT, a quantization-aware training (QAT) method for vector quantization. It decomposes weight vectors into shape and gain components and applies separate straight-through estimators (STEs) to the two quantizers. The method uses E8-derived spherical lattice codebooks, including the proposed E8S codebook. It is evaluated against several post-training quantization (PTQ) and QAT baselines across multiple models and benchmarks.

Critical strengths
  • The factored STEs are well motivated and have a clear geometric interpretation under idealized SGD updates.
  • The evaluation results demonstrate competitive accuracy compared with existing methods across different models and benchmarks.
  • The implementation is efficient, with training time approaching that of BF16 in the reported benchmark.
Critical weaknesses
  • The evaluation is limited to E8-derived codebooks (E8S and E8P12). The generalizability to other vector quantization formats remains unclear. In addition, E8S currently lacks an efficient dequantization kernel for inference, limiting its practical deployment.
  • The method introduces two changes simultaneously: spherical shape-gain parameterization and factored STEs. Table 10 compares three configurations: Euclidean parameterization with joint STE, Euclidean parameterization with factored STE, and spherical parameterization with factored STE (SQuAT). However, the fourth configuration, spherical parameterization with joint STE, is missing. Although it is analyzed theoretically in Appendix B.1, an empirical comparison would help disentangle the contributions of the parameterization and STE design.
  • Some results appear inconsistent across tables. For example, Tables 1 and 10 report identical Spherical results but different Standard VQ-QAT results for Llama-3.2-1B. Similarly, Tables 2 and 5 report different results for 1B-token INT-QAT using WSD and Hadamard rotation for Llama-3.2-1B.
Questions for authors

None. Please reply to the weaknesses.

3 comments
AuthorsOpenReview ↗

General Response + Response to Weaknesses and Questions

We thank all reviewers for their constructive feedback. We are glad that the reviewers found the factored STE and the spherical parameterization novel and well motivated (rztw, J7ZN), the gradient analysis clear (gQL3), and the implementation efficient, with practical training speedups (gQL3, J7ZN). We have revised the paper accordingly, and all changes are highlighted in yellow. Since the revision merges Tabs. 1 and 2 and adds new tables, this rebuttal uses the numbering of the submitted version and marks new tables as “revised Table X”.

Decoding kernels. As requested by gQL3 and J7ZN, we developed fast decoding kernels for E8S and added them to the revised paper (Sec. 5 and App. F). Following standard practice for low-bit weight quantization, we measure at batch size one, the memory-bound regime of autoregressive decoding, on an NVIDIA L40S (the memory bandwidth class of the RTX 4090 used by QuIP#). We report end-to-end decode throughput at context length 512, averaged over three seeds, against the official QuIP# E8P12 kernel. Percentages give the change relative to E8P12 at 2.0 bpw.

CodebookbpwQ4B (tok/s)L3B (tok/s)
FP161685106
E8P12 (QuIP#)2.0273329
E8S2.0271 (-0.8%)334 (+1.4%)
E8S-SG (SQuAT)2.0274 (+0.7%)333 (+1.3%)
E8P12-SG2.25243 (-11.0%)299 (-9.1%)
E8S-SG (SQuAT)2.25254 (-6.8%)314 (-4.5%)

At 2.0 bpw E8S decodes as fast as E8P12, and both run 3.1 to 3.2 times faster than FP16. At 2.25 bpw E8S-SG decodes 4.8% to 5.1% faster than E8P12-SG. App. F gives the kernel design, table sizes, GEMV latencies and comparisons with INT2 kernels.

Seeds on all three models. As requested by rztw, we repeated the comparison of SQuAT and Standard VQ-QAT with three seeds on L3B and Q4B at 1B tokens (App. E.2, revised Tables 9 and 10). SQuAT reaches a perplexity of 10.94 ± 0.03 on L3B and 10.02 ± 0.02 on Q4B, against 11.09 ± 0.04 and 10.19 ± 0.02 for Standard VQ-QAT. Together with L1B (Table 8), the perplexity differences exceed the standard deviation for all models and budgets, and the ranges do not overlap in any 1B-token experiment. We have started 5B-token runs and will post the results here if they finish in time, or include them in the final version.

Response to weaknesses and questions

W1. Only tested E8 codebooks, and E8S has no efficient inference kernel.

We agree that the evaluation covers only E8-derived codebooks, and Sec. 7 of the submitted version stated this limitation (line 467), together with the lack of an efficient E8S dequantizer (line 471). The second point no longer holds, as described in the General Response. We have since built batch-one decoding kernels for E8S that decode as fast as E8P12 at 2.0 bpw (Sec. 5 and App. F). We revised the limitations part and kept the accuracy claims limited to the two codebooks we test.

The method itself does not depend on E8. SQuAT is a training rule for any spherical codebook with exact nearest-neighbor assignment and a scalar gain quantizer. The two codebooks we test are also built differently. E8P12 is a structured subset designed for fast decoding, while E8S consists of whole lattice shells. Shape-gain training with SQuAT improves both (Tab. 3, and Tabs. 12 and 13 without Hadamard rotations).

W2. Missing Spherical parameterization with a joint STE.

We agree that Tab. 10 does not include this configuration, and we analyze it in App. B.1. It is a less natural choice, since with spherical parameters the Euclidean weight exists only as an intermediate quantity formed to place the STE. App. B.1 shows that it yields the same tangent update for the direction and radial update for the gain as SQuAT, with one difference. Its gradients use the unquantized direction and gain, whereas SQuAT uses the quantized values from the forward pass, so SQuAT’s updates are consistent with the model actually seen during training. This consistency is the specific contribution of the factored STE on top of the spherical parameterization. We are running this configuration on L1B at 1B tokens and will post the results before the end of the discussion period if they finish in time, and add them to Tab. 10 of the final version otherwise.

W3. Some results are inconsistent across tables. We checked the experiment logs and confirmed that the main text has the most recent results. Appendix Tabs. 5 and 10 came from earlier reruns and were not updated. Code, hyperparameters, seeds and setup were unchanged, and we attribute the differences to non-deterministic distributed training across A100 and H100 nodes. The conclusion is unchanged, since SQuAT (13.85 / 58.8 / 40.3) outperforms the Standard baseline in both Tab. 1 (14.22 / 58.4 / 39.3) and the submitted Tab. 10 (14.10 / 58.7 / 37.5). We updated Tab. 10 (revised Tab. 12) and are recomputing Tab. 5, which we will post here if it finishes in time and otherwise include in the final version.

Thank you for the reply. I look forward to the pending experimental results. Overall, I maintain my positive assessment of the paper.

Thank you for your positive assessment. As promised, the runs with spherical parameterization and a joint STE have now finished. The table completes all four combinations of parameterization and STE with Wiki2 perplexity after 1B tokens at 2.25 bpw

ParameterizationSTEL1BQ4B
Euclideanjoint (Standard VQ-QAT)14.2210.17
Euclideanfactored (App. B.2)14.0110.15
Sphericaljoint (App. B.1)13.8610.16
Sphericalfactored (SQuAT)13.8510.07

Both the parameterization and the STE choice are needed for the best results. SQuAT gives the lowest perplexity on both models, and on Q4B it is the only configuration that improves over Standard VQ-QAT. SQuAT also improves over the spherical joint STE variant on downstream tasks for Q4B

Q4BCSRMMLUGSM8KIFEval
Spherical, joint STE (App. B.1)70.1567.0776.1981.01
SQuAT71.6967.2977.0382.26

As noted in our rebuttal and in App. B.1, the joint STE variant is somewhat artificial since the Euclidean weight w=γsw = \gamma s is formed only to place the STE. We will add these results to Table 12.

Reviewer_gQL3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SQuAT for low-bit vector QAT. It separates each weight block into direction and magnitude, quantizes both, and applies separate STEs while differentiating through direction normalization. This changes the training gradients compared with applying one STE to the full quantizer. The paper also introduces E8S spherical codebooks and speeds up nearest-neighbor assignment through fused kernels, index caching, and distributed computation. Experiments on 1B–4B models with up to 10B training tokens show improvements over Standard VQ-QAT on several metrics. Comparisons with scalar INT-QAT are more favorable at 2.25 bpw than at 2.0 bpw.

Critical strengths
  1. The paper explains how parameterization and STE placement change the training gradients, even when the forward quantizer is the same. The gradient analysis gives a clear account of how direction normalization affects the updates.
  2. The main comparison with Standard VQ-QAT uses the same quantizer and PTQ initialization. SQuAT improves perplexity and several generative task scores. For example, Llama-3B GSM8K improves from 47.0 to 51.9 after 1B training tokens.
  3. The experiments cover different token budgets, bit allocations, and codebooks. The appendix includes ablations of gain quantization, shape updates, and Hadamard rotations, as well as three-seed results on Llama-1B.
  4. The implementation reduces training time from 62.5 to 4.9 seconds per iteration through fusion, caching, and distributed assignment. The reported breakdown shows how these optimizations make exact-assignment VQ-QAT more practical.
Critical weaknesses
  1. The paper limits the exact tangent-step interpretation to plain SGD. Its connection to the AdamW updates used in the experiments remains unclear, since coordinate-wise preconditioning and momentum can change the update direction. The geometric explanation therefore does not fully characterize the actual training dynamics.
  2. The evaluated models are small, with at most 4B parameters, and come from only the Llama and Qwen families. The results do not establish the method's effectiveness on larger models or MoE architectures, which limits the evidence for generalization.
  3. Shape-gain decoding introduces additional steps, but its impact on inference efficiency is not evaluated. The reported acceleration concerns training rather than inference. This is particularly relevant to E8S, which the paper acknowledges currently lacks an efficient dequantizer.
Questions for authors
  1. Why do the Llama-1B Standard results differ between Tables 1 and 10 while the SQuAT results match? For example, Standard MMLU is 39.3 in Table 1 but 37.5 in Table 10. Please clarify any differences in seeds, configurations, or versions of the results.
  2. Appendix E.4 concludes that neither spherical parameterization nor STE factorization alone is sufficient. How does Table 10 support this conclusion without testing spherical parameterization with a joint STE, the variant discussed in Appendix B.1?
  3. Please specify the GPU model used for Figure 3 and whether the reported training time includes the teacher forward pass. What is the peak training memory usage?
  4. For E8P12-SG and E8S, what are the practical inference costs and main decoding limitations? In particular, what prevents an efficient E8S dequantizer, and how does separate gain decoding affect runtime?
4 comments
AuthorsOpenReview ↗

Response to all reviewers (1/2)

We thank all reviewers for their constructive feedback. We are glad that the reviewers found the factored STE and the spherical parameterization novel and well motivated (rztw, J7ZN), the gradient analysis clear (gQL3), and the implementation efficient, with practical training speedups (gQL3, J7ZN). We have revised the paper accordingly, and all changes are highlighted in yellow. Since the revision merges Tabs. 1 and 2 and adds new tables, this rebuttal uses the numbering of the submitted version and marks new tables as “revised Table X”.

Decoding kernels. As requested by gQL3 and J7ZN, we developed fast decoding kernels for E8S and added them to the revised paper (Sec. 5 and App. F). Following standard practice for low-bit weight quantization, we measure at batch size one, the memory-bound regime of autoregressive decoding, on an NVIDIA L40S (the memory bandwidth class of the RTX 4090 used by QuIP#). We report end-to-end decode throughput at context length 512, averaged over three seeds, against the official QuIP# E8P12 kernel. Percentages give the change relative to E8P12 at 2.0 bpw.

CodebookbpwQ4B (tok/s)L3B (tok/s)
FP161685106
E8P12 (QuIP#)2.0273329
E8S2.0271 (-0.8%)334 (+1.4%)
E8S-SG (SQuAT)2.0274 (+0.7%)333 (+1.3%)
E8P12-SG2.25243 (-11.0%)299 (-9.1%)
E8S-SG (SQuAT)2.25254 (-6.8%)314 (-4.5%)

At 2.0 bpw E8S decodes as fast as E8P12, and both run 3.1 to 3.2 times faster than FP16. At 2.25 bpw E8S-SG decodes 4.8% to 5.1% faster than E8P12-SG. App. F gives the kernel design, table sizes, GEMV latencies and comparisons with INT2 kernels.

Seeds on all three models. As requested by rztw, we repeated the comparison of SQuAT and Standard VQ-QAT with three seeds on L3B and Q4B at 1B tokens (App. E.2, revised Tables 9 and 10). SQuAT reaches a perplexity of 10.94 ± 0.03 on L3B and 10.02 ± 0.02 on Q4B, against 11.09 ± 0.04 and 10.19 ± 0.02 for Standard VQ-QAT. Together with L1B (Table 8), the perplexity differences exceed the standard deviation for all models and budgets, and the ranges do not overlap in any 1B-token experiment. We have started 5B-token runs and will post the results here if they finish in time, or include them in the final version.

AuthorsOpenReview ↗

Response to weaknesses and questions (2/2)

W1. The tangent-step interpretation assumes SGD, but training uses AdamW.

Our analysis concerns the gradient, and that part holds for any optimizer. Because the loss depends on the shape parameter u only through u/‖u‖, the surrogate gradient with respect to u is exactly tangent to the sphere (Eq. 11), and renormalization after every update acts as a retraction whatever step the optimizer takes. What changes under AdamW is the step. Preconditioning and momentum can give the update a small radial component, which only changes ‖u‖, leaves the forward pass unchanged and rescales the effective angular step. SQuAT with AdamW is therefore an adaptive step on tangent gradients followed by the same retraction, closely related to Riemannian adaptive optimization (Bécigneul & Ganea, ICLR 2019). We revised Sec. 3.2 to state that the exact tangent step holds for SGD, while the tangent gradient and the retraction hold for any optimizer.

W2. Only dense models up to 4B parameters from two families.

We agree, and Sec. 7 already lists 7B+ models and broader architectures as future work. The revision adds MoE models to this list and limits the empirical claims to the tested dense models. Our focus was on long training horizons rather than model size. VQ-QAT has mainly been studied with short budgets, for example 1B tokens for PV-Tuning, and we scale it to 10B tokens to compare with INT-QAT under the same budgets. Our longest run, Qwen3-4B on 10B tokens, took four days on eight A100-80GB GPUs. Since SQuAT acts on each linear layer independently, larger dense models and MoE expert weights require no change to the method.

W3 and Q4. Inference cost and decoding limits. At submission we focused on the training method, and we have since built E8S kernels that closely match E8P12 (General Response). The extra cost of E8S is its codebook table, copied into shared memory for each layer. It is 64 KiB for the codebooks we use, and storing sign variants once gives smaller tables, such as a 16 KiB variant that decodes 1.7% to 3.6% faster than the official E8P12 kernel. The gain adds almost no latency, since it is applied once per block of 8 weights on the accumulated output while the kernel waits for weights from memory. NVIDIA GPUs have 100 to 228 KiB of shared memory per SM (L40S to H100), so the table fits easily. On mobile GPUs with about 32 KiB per compute unit, the 64 KiB table does not fit, while E8P12 needs only 1 KiB. We added this limitation to the discussion (Sec. 7).

Q1. Why do the L1B Standard results differ between Tabs. 1 and 10?

Thank you for highlighting the inconsistency. We checked the experiment logs and confirmed that the results in the main text are the most recent and authoritative. Table 10 was created from earlier reruns and not updated later. Code, hyperparameters, seeds and training setup were unchanged, and we attribute the difference to non-deterministic operations in distributed training across A100 and H100 nodes. The main conclusion is unchanged, since SQuAT (13.85 / 58.8 / 40.3) outperforms the Standard baseline in both Table 1 (14.22 / 58.4 / 39.3) and the submitted Table 10 (14.10 / 58.7 / 37.5). We have updated Table 10 (revised Table 12) for consistency.

Q2. Tab 10 does not test spherical parameterization with a joint STE (App. B.1).

We included this variant in App. B.1 to complete the analysis of Tab. 4. It is a less natural choice, because with spherical parameters the Euclidean weight is not an optimization variable and is formed only to place the STE. App. B.1 shows that its update already has the geometry of SQuAT, a tangent step with retraction for the direction and a radial step for the gain, so the spherical parameterization alone provides this structure. The two differ only in where the gradients are evaluated. The joint STE evaluates them at the unquantized shape and gain, while SQuAT uses the quantized values of the forward pass (Eq. 21 compared with Eq. 12). At 2 bits per weight the quantized direction differs noticeably from the unquantized one, so the joint STE updates the gain for a direction the model never uses. Removing this mismatch is the specific contribution of the factored STE. We are running this configuration on L1B at 1B tokens and will post the results here before the end of the discussion period if they finish in time, and add them to Tab. 10 of the final version otherwise.

Q3. Which GPU was used for Fig. 3, did the timing include the teacher forward pass, and what is the peak memory?

The timings in Fig. 3 were measured on an NVIDIA A100-SXM4-80GB GPU and include the teacher forward pass. The peak memory usage for the Llama-3.2-1B model was 64.8 GB per GPU. We used our standard setup (App. C) with gradient checkpointing disabled. The revision adds these details to Sec. 3.4 and App. C.

Thank you for the response. The authors have added an E8S decoding kernel, which achieves a speed close to E8P12 at 2.0 bpw on the L40S with batch size 1. The GPU, teacher forward-pass cost, peak training memory, and the actual optimizer updates and their scope of applicability have also been clarified. The comparison with spherical parameterization using a joint STE and evaluations on other models are still pending.

Thank you for the response. We are glad that our rebuttal resolved your concerns on decoding speed, where E8S matches E8P12, and that it clarified the GPU, teacher cost, peak training memory, and the optimizer updates and their scope.

As promised, the runs with spherical parameterization and a joint STE have now finished. The table completes all four combinations of parameterization and STE with Wiki2 perplexity after 1B tokens at 2.25 bpw

ParameterizationSTEL1BQ4B
Euclideanjoint (Standard VQ-QAT)14.2210.17
Euclideanfactored (App. B.2)14.0110.15
Sphericaljoint (App. B.1)13.8610.16
Sphericalfactored (SQuAT)13.8510.07

Both the parameterization and the STE choice are needed for the best results. SQuAT gives the lowest perplexity on both models, and on Q4B it is the only configuration that improves over Standard VQ-QAT. SQuAT also improves over the spherical joint STE variant on downstream tasks for Q4B

Q4BCSRMMLUGSM8KIFEval
Spherical, joint STE (App. B.1)70.1567.0776.1981.01
SQuAT71.6967.2977.0382.26

As noted in our rebuttal and in App. B.1, the joint STE variant is somewhat artificial since the Euclidean weight w=γsw = \gamma s is formed only to place the STE. We will add these results to Table 12.

We understand that the point on larger models is still open. A 7B run did not fit on our GPUs within the discussion period and we will include it in the final version. SQuAT acts on each linear layer independently and needs no change for larger dense or MoE models.

We hope this resolves your remaining questions and that you will consider raising your score.

Reviewer_rztwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a quantization-aware training (QAT) method for low-bit LLM compression. The core contributions are threefold: (1) applying the factored straight-through estimator (STE) separately to the shape (direction) and gain (magnitude) components of the weight vector; (2) proposing a spherical parameterization of shadow weights; (3) developing a new E8-derived spherical codebook (E8S) supporting fractional bitwidths, and a distributed exact nearest-neighbor assignment scheme achieving a 12.8× speedup for billion-token-scale VQ-QAT. I am particularly satisfied with the unique STE decomposition and spherical parameterization ideas in this work. To the best of my knowledge, such methods are relatively novel and unique in existing VQ-QAT literature. Experiments conducted across LLaMA-3.2-1B/3B, Qwen3-4B, and Qwen3-1.7B models also reveal performance gains across diverse tasks. Honestly, I initially favored accepting this work, but the vague table design and insufficient analysis of the experimental results prevent me from accepting it. I list several major points below and hope the authors will fully consider them before this work is to be accepted.

Critical strengths
  1. Methodology novelty. The authors show that a factored STE over shape-gain variables induces a tangent-step and retraction update on the unit sphere, which is novel and unexplored in existing literature. The comparisons with Standard VQ-QAT and the two alternative parameterizations in Appendix B further highlight the contribution of each design choice.
  2. Practical scalability. The distributed assignment scheme, combining fused Triton kernels, index caching, and sharded workers, is a good engineering contribution. The 12.8× speedup demonstrated in Figure 3 also addresses a real bottleneck to enable the multi-billion-token experiments.
  3. E8S codebook design. The construction of fractional-bitwidth spherical codebooks via shell enumeration and farthest-point sampling (Appendix A) is well-motivated and fills a real gap: unlike E8P12, which is fixed at 2.0 bpw, E8S enables finer-grained Pareto-optimal compression points inaccessible to scalar methods.
Critical weaknesses
  1. (Major) Table 1 requires more comprehensive analysis. The paper claims that Spherical VQ-QAT “clearly improves” over Standard VQ-QAT on WikiText2 perplexity and GSM8K. However, Table 1 shows that the CSR metric for LLaMA-3.2-3B is worse under Spherical VQ-QAT at both 1B and 5B token budgets. The authors only mention this issue without any mechanistic analysis. Moreover, across nearly all metrics and models, the performance advantage of Spherical over Standard VQ-QAT shrinks substantially from 1B to 5B tokens. I worry that the performance gain may be a transient effect of limited training rather than a stable structural advantage.
  2. (Major) Table 2 requires clearer designs and analysis of the effect on spherical training. Table 2 omits Standard VQ-QAT as a baseline, so I have no idea how much of the observed gain over INT-QAT or PTQ baselines comes from the spherical training rule versus the structural advantage of VQ over scalar quantization. Meanwhile, Table 2 simultaneously varies method type, bitwidth, and token budget across rows without explicit controls, introducing multiple confounds. I also don’t know what the bold number in Table 2 means, so it needs more annotations.
  3. (Major) More insights and analysis should be presented in Sec. 5. The analysis of all tables and figures in Sec. 5 is mostly descriptive and restates numbers without mechanistic interpretation. Given the authors’ promising ideas and methods, the insufficient analysis of the results significantly undermines the quality of this work. I strongly recommend including more insight into why the results were obtained, especially their connections to the proposed methods.
  4. (Major) Statistical validation is insufficient beyond L1B. Table 8 provides multi-seed evaluation only for LLaMA-3.2-1B. The other two models (L3B, Q4B) have no seed analysis. This is quite important, as Spherical VQ-QAT underperforms Standard VQ-QAT on CSR with L3B and Q4B models in Table 1.
  5. (Major) Figures 4 & 6 need more comparisons. Figure 4 presents Pareto fronts for only one model (LLaMA-3.2-1B), one metric (WikiText2 PPL), and one training budget (1B tokens), which is a narrow basis for a Pareto optimality claim. Figure 6 in the appendix partially addresses this by extending to four evaluation panels (WikiText2 PPL at two sequence lengths, CSR, and MMLU), but still covers only one model and one training budget. Critically, neither figure includes a Standard VQ-QAT Pareto curve as a reference. Without this comparison, it is impossible to attribute the observed Pareto advantage over INT-QAT to SQUAT’s spherical training mechanism. The authors are encouraged to add Standard VQ-QAT reference points to at least Figure 4, and to extend the Pareto analysis to at least one additional model in the main text.
  6. (Moderate) Table 3 requires bold annotations and more analysis. Table 3 shows that E8S outperforms E8P12 even without SQUAT, but the authors don’t present further analysis. Moreover, Table 3 contains no bold entries, and the “Bits” column format (2.0 + 0.25) is explained only briefly as “codebook + scale/gain bits” in the caption, without clarifying the semantic difference between, e.g., 2.25 + 0.0 and 2.0 + 0.25 at the same total bpw.
Questions for authors

See Weaknesses part above.

3 comments

Response to all reviewers

We thank all reviewers for their constructive feedback. We are glad that the reviewers found the factored STE and the spherical parameterization novel and well motivated (rztw, J7ZN), the gradient analysis clear (gQL3), and the implementation efficient, with practical training speedups (gQL3, J7ZN). We have revised the paper accordingly, and all changes are highlighted in yellow. Since the revision merges Tabs. 1 and 2 and adds new tables, this rebuttal uses the numbering of the submitted version and marks new tables as “revised Table X”.

Decoding kernels. As requested by gQL3 and J7ZN, we developed fast decoding kernels for E8S and added them to the revised paper (Sec. 5 and App. F). Following standard practice for low-bit weight quantization, we measure at batch size one, the memory-bound regime of autoregressive decoding, on an NVIDIA L40S (the memory bandwidth class of the RTX 4090 used by QuIP#). We report end-to-end decode throughput at context length 512, averaged over three seeds, against the official QuIP# E8P12 kernel. Percentages give the change relative to E8P12 at 2.0 bpw.

CodebookbpwQ4B (tok/s)L3B (tok/s)
FP161685106
E8P12 (QuIP#)2.0273329
E8S2.0271 (-0.8%)334 (+1.4%)
E8S-SG (SQuAT)2.0274 (+0.7%)333 (+1.3%)
E8P12-SG2.25243 (-11.0%)299 (-9.1%)
E8S-SG (SQuAT)2.25254 (-6.8%)314 (-4.5%)

At 2.0 bpw E8S decodes as fast as E8P12, and both run 3.1 to 3.2 times faster than FP16. At 2.25 bpw E8S-SG decodes 4.8% to 5.1% faster than E8P12-SG. App. F gives the kernel design, table sizes, GEMV latencies and comparisons with INT2 kernels.

Seeds on all three models. As requested by rztw, we repeated the comparison of SQuAT and Standard VQ-QAT with three seeds on L3B and Q4B at 1B tokens (App. E.2, revised Tables 9 and 10). SQuAT reaches a perplexity of 10.94 ± 0.03 on L3B and 10.02 ± 0.02 on Q4B, against 11.09 ± 0.04 and 10.19 ± 0.02 for Standard VQ-QAT. Together with L1B (Table 8), the perplexity differences exceed the standard deviation for all models and budgets, and the ranges do not overlap in any 1B-token experiment. We have started 5B-token runs and will post the results here if they finish in time, or include them in the final version.

Response to weaknesses and questions (1/2)

W1. The Tab. 1 gains shrink at 5B tokens, and L3B CSR is worse.

We read the advantage as structural rather than transient. SQuAT’s factored STE follows the structure of the shape-gain quantizer, training direction and gain separately for the quantizers they are encoded with, while Standard VQ-QAT applies one STE to the reconstructed block. Standard VQ-QAT does not close the gap with more tokens. On Q4B its perplexity stays at 10.17 from 1B to 5B tokens while SQuAT remains below it. SQuAT is also more token efficient. On L1B it matches after 1B tokens what Standard VQ-QAT reaches after 5B (13.84 against 13.85, Table 8), and on the generation tasks it does so in 7 of 9 pairs of model and task (Tables 1 and 7).

Downstream gains persist at 5B. SQuAT improves GSM8K on all three models at both budgets (Table 1), and its IFEval gains reach 2.8, 4.7 and 3.5 points at 5B (Table 7). L3B CSR is the one metric where Standard VQ-QAT is ahead at both budgets in Table 1. Over three seeds at 1B tokens, however, the means are 65.80 for SQuAT against 65.70 for Standard VQ-QAT (revised Table 9), so this gap is within seed noise. On the same model, SQuAT is better in perplexity, GSM8K, LongBench and IFEval at 5B.

W2. Tab. 2 has no Standard VQ-QAT row, varies several factors at once, and does not define bold.

We agree that Tab. 2 can be clearer. Since Tabs. 1 and 2 report the same runs at 1B and 5B tokens, the revision merges them into one table grouped by bit rate and token budget, with Standard VQ-QAT, SQuAT and INT-QAT side by side. Every comparison then holds bit rate and token budget fixed, and boldface marks the best method within each group. GSM8K moved to the appendix with the other generation tasks. The merged table also shows where the gains come from (WikiText2 perplexity, 2.25 bpw).

ModelTokensINT-QATStandard VQ-QATSQuAT
L1B1B15.0814.2213.85
L3B1B11.4911.0410.96
Q4B1B10.9610.1710.03
L1B5B14.4213.7713.64
L3B5B11.0310.8810.84
Q4B5B10.5710.1710.06

Going from INT-QAT to Standard VQ-QAT improves perplexity by 0.45 to 0.86 points at 1B tokens, but the two methods use different initialization procedures (App. C), so this gap compares the two training pipelines as a whole. Going from Standard VQ-QAT to SQuAT improves perplexity by a further 0.08 to 0.37 points, and since both share the quantizer and the initialization, this gap isolates our training rule.

W3. Sec. 5 is mostly descriptive.

We agree that Sec. 5 should explain the results, and the revised Sec. 5 now does so. First, our ablations in Tabs. 3, 9 and 10 isolate where SQuAT's improvement comes from. The factored STE passes gradients to direction and gain separately, so both are trained for the quantizer they are encoded with. SQuAT improves both E8P12 and E8S at 2.25 bpw (Tab. 3), and together with the spherical parameterization it improves all three models on perplexity and MMLU (Tab. 10). In Tab. 9, deriving the gain from the shadow weights raises L1B perplexity from 13.85 to 14.01, while freezing the shape weights keeps it at 13.85 but lowers CSR and MMLU. The learned gain therefore carries most of the perplexity improvement, and training the shape adds downstream accuracy.

Second, E8S makes the rate adjustable in small steps (Tab. 3, Figs. 4 and 6). Each E8S codebook takes whole lattice shells in order of increasing norm and fills the remaining codewords with a subset of the next shell, so the same rule gives a codebook for any index size. One more index bit per block of eight weights changes the rate by 0.125 bpw, independently of the gain allocation, which lets us sweep 1.5 to 3.0 bpw and choose the best split between shape and gain at each rate. Standard INT-QAT reaches fractional rates only through scale overhead, and E8P12 is designed for a fixed 16 bits per block. The E8S construction is also beneficial on its own, since E8S outperforms E8P12 at the same rate even without shape-gain (Tab. 3).

Third, the accuracy gap between lattice VQ-QAT and INT-QAT is larger at lower training budgets (Tab. 2). At 2.25 bpw on L1B the perplexity gap is 1.23, 0.78 and 0.59 at 1B, 5B and 10B tokens. Part of this gap reflects the different initialization procedures (App. C), and one possible explanation for the narrowing is that the representation matters less once the weights have adapted to the quantizer over a long training run. We find this an interesting direction for follow-up work.

W4. Seeds are only reported for Llama-1B. We agree, and we ran three seeds for L3B and Q4B at 1B tokens, as described in the General Response (revised Tabs. 9 and 10). With L1B at 1B and 5B tokens (Tab. 8), SQuAT has lower perplexity than Standard VQ-QAT in every matched pair of seeds on all three models.

W5. The Pareto plots have no Standard VQ-QAT reference and cover one model. The term "Pareto front" was not the right choice, and the revision replaces it throughout. Figs. 4 and 6 are ablations of how a fixed bit budget should be split between shape and gain bits within SQuAT. They show that the best number of gain bits grows with the total rate. At 1.5 bpw 0.125 gain bits is best, around 2.0 bpw 0.125 to 0.25 bits, and at 3.0 bpw 0.375 bits (Fig. 4). They also show that E8S adjusts the rate in steps of 0.125 bpw, independently of the gain allocation, and compare these rates with INT-QAT at 2.0, 2.25 and 3.0 bpw.

W6. Tab. 3 bit notation, bold entries and analysis. The revision (revised Tab. 2) makes the notation explicit. In "A + B", A is the bits per weight for the codebook index and B the bits per weight for the scale or gain. At 2.25 bpw, 2.0 + 0.25 stores a 16-bit index (65,536 codewords) and a 2-bit gain per block of 8 weights. 2.25 + 0.0 stores an 18-bit index (262,144 codewords) with one scale per output channel, which costs under 0.01 bpw. App. A describes how E8S builds a codebook for any index size, which gives these fractional rates. We regrouped the rows and bolded the best result in each group. The revised text draws three conclusions from the table.

First, a per-block gain pays off only in the shape-gain form. Without it, the extra 0.25 bpw is better spent on a larger codebook, and for E8S 2.25 + 0.0 beats 2.0 + 0.25 (14.24 against 14.53 perplexity). With shape-gain, the same 2.0 + 0.25 budget reaches 14.01 when the gain is derived from the shadow weights (Tab. 9) and 13.85 with SQuAT, which supports the argument of Sec. 2.3.

Second, SQuAT improves both codebooks. At 2.25 bpw it reaches 13.94 with E8P12-SG and 13.85 with E8S-SG, against 14.67 and 14.53 without shape-gain.

Third, E8S is slightly better than E8P12 even without SQuAT (14.78 / 58.1 / 35.8 against 14.85 / 57.6 / 35.7 at 2.0 bpw), because it has slightly lower distortion. On Gaussian data with an optimized scale, its normalized MSE is 0.0906 against 0.0914 for E8P12. E8P12 is built for compact decoding, with 256 table entries plus sign and parity bits, while E8S is built for angular coverage from whole lattice shells.

As the discussion closes today, we hope our response has addressed your concerns. Following your suggestions, we made every table a controlled comparison with Standard VQ-QAT. We also added three-seed runs and E8S decoding kernels that match the speed of E8P12, and expanded the analysis in Sec. 5. Your initial review mentioned that you were considering acceptance, and we hope these changes now meet that bar. We are happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.