- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes BW-QAT, a representation-first framework for 2-bit weight-only quantization-aware training of LLMs. It introduces a finite 16-dimensional Barnes–Wall lattice representation that encodes 16 weights into a fixed 32-bit payload. The framework incorporates Codec-Consistent Gain Calibration (CCGC) to improve initial quantization quality and decision-preserving initialization to retain continuous residual information for effective hard-forward QAT with a straight-through estimator. Experiments on Qwen3 and Llama-3.2 models demonstrate improved perplexity and downstream accuracy over matched lattice, scalar-QAT, and vector-QAT baselines, particularly in low-data settings, with additional comparisons under matched training-time budgets.
Critical strengths
-
Novel Quantization Representation. The paper introduces a finite 16-dimensional Barnes–Wall (BW16) lattice representation with a fixed 2-bit-per-weight payload. Unlike conventional scalar or vector quantization approaches, BW-QAT provides a structured finite representation with exact encoding and decoding, codec-consistent gain calibration, and hard-forward QAT. Importantly, the same discrete representation is maintained throughout training and deployment. This constitutes a meaningful representation-level contribution to extremely low-bit LLM quantization.
-
Strong Empirical Improvements with Detailed Comparisons. The paper demonstrates substantial improvements over existing scalar-QAT and vector-QAT baselines, particularly in low-data settings. For example, on Qwen3-1.7B, BW-QAT improves the six-task average accuracy from 52.17% to 58.30% over LC-QAT and reduces WikiText-2 perplexity from 18.90 to 14.02. The experiments also include matched training-data budgets, matched wall-clock comparisons, multiple random seeds, and ablation studies that isolate the effects of the representation and initialization. These controlled evaluations strengthen the empirical evidence supporting the proposed method, despite its higher computational cost.
Critical weaknesses
-
Limited Methodological Novelty. Although the finite BW16 representation is a potentially useful contribution, the overall framework builds upon established lattice-based vector quantization and STE-based QAT methods, including QuIP#, Lattice-based QAT, and LC-QAT. In particular, the proposed decision-preserving initialization closely resembles conventional QAT approaches that maintain continuous latent weights while performing quantized forward passes. Its novelty beyond initializing the latent weights from compensated targets is not sufficiently established. Furthermore, the representation comparisons in Table 1 involve substantially different numbers of active lattice points, making it difficult to determine whether the improvements arise from the specific Barnes–Wall lattice geometry or simply from a larger effective reconstruction space. More detailed comparisons are needed to evaluate the contributions.
-
Limited Experimental Scope The experimental evaluation is restricted to three relatively small models (Qwen3-0.6B, Llama-3.2-1B, and Qwen3-1.7B), without results on commonly used 7B/8B or larger LLMs. This is particularly important given the substantial computational overhead of BW-QAT. For example, Table 4 shows that BW-QAT requires approximately 14.5 times more QAT time than LC-QAT under the same training-label budget. Although the matched-time experiments demonstrate a remaining performance advantage, the method's scalability and computational practicality are insufficiently validated. Moreover, the downstream evaluation focuses primarily on commonsense reasoning and question-answering benchmarks, while more challenging tasks such as mathematical reasoning, code generation, and instruction following are not evaluated. In addition, the lack of inference latency and throughput measurements further limits the assessment of the proposed representation's practical deployment benefits.
Questions for authors
1. Methodological Novelty. How does the proposed decision-preserving initialization fundamentally differ from conventional STE-based QAT methods that maintain continuous full-precision latent weights during quantized forward passes? Table 2 primarily compares initialization from compensated targets against initialization from quantized codewords. Could the authors also clarify or evaluate the difference between initializing from the original full-precision weights and the proposed compensated targets?
2. Contribution of Barnes–Wall Geometry. In Table 1, BW16 has substantially more active lattice points (237.3M) than Product-E8 (83.2M) and JS-E8 (182.0M), despite the same nominal storage budget. How can the authors establish that the reported improvements arise from the specific Barnes–Wall lattice geometry rather than the larger effective reconstruction space? Could the authors provide comparisons with approximately matched codebook cardinalities or additional analyses isolating the effects of lattice geometry, codebook capacity, and energy shaping?
3. Scalability to Larger LLMs. The experiments only cover models up to 1.7B parameters. Given the considerable computational overhead of the proposed finite BW16 quantizer, how would BW-QAT scale to widely used 7B/8B or larger LLMs?
4. Practical Inference Efficiency. Although the paper reports serialized compression rates and evaluates independently reloaded quantized models, it does not provide inference latency or throughput measurements. How efficiently can the proposed BW16 representation be decoded and executed during inference compared with existing scalar and vector quantization methods? Does the exact lattice decoding introduce substantial runtime overhead, and are specialized inference kernels required?
5. Standard Benchmarks. Is there evidence that the improvements extend to more challenging capabilities, such as mathematical reasoning, code generation, and instruction following?