acceptodds
Under review as a conference paper at ICLR 2027

E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

Abstract

Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training. We propose E8-QAT, a 2-bit quantization-aware training framework that uses the lattice as an implicit eight-dimensional codebook. It jointly quantizes weight groups through arithmetic projection without storing or searching an explicit codebook during the training forward pass. A 2-bit core encoding and sparse extension preserve the selected lattice points exactly, adding less than 0.1% to core index storage. On Qwen3.6-35B-A3B, E8-QAT matches the performance of the full-precision model in terms of the average score across the four tasks. E8-QAT improves over the state-of-the-art baseline by 2.03% on IFEval and by 1.51% on LiveCodeBench, under the same data, schedule, and bit width. The 2-bit representation reduces peak memory by 80% and supports a larger batch. At each representation's largest feasible batch, E8-QAT reaches the decoding throughput of BF16.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Vector Quantization, Quantization-Aware Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MzgKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes E8-QAT for 2-bit weight quantization. It starts from E8-PTQ and uses arithmetic E8 projection with an identity STE during QAT. The trained weights are stored using 16-bit core indices and sparse records for points outside the core. Results on Qwen3.6-35B-A3B and Llama-3.1-8B show better accuracy than LC-QAT, with less memory and supports a larger batch size.

Critical strengths
  1. E8 projection jointly quantizes eight weights without explicit codebook search during QAT. Sparse escape records preserve the selected points at export with little additional storage in the reported experiments.
  2. The paper clearly describes the core codebook, escape encoding, and storage overhead, and provides decoding checks in the appendix.
  3. E8-QAT improves over LC-QAT on both models. And the evaluation covers basic tasks, code generation, function calling, and instruction following.
  4. The quantized model substantially reduces memory use and supports a larger batch size on one GPU.
Critical weaknesses
  1. E8-PTQ already outperforms UniSVQ before QAT, as shown in Figure 4 and Table 1. The comparison with LC-QAT therefore does not isolate the benefit of the proposed training strategy.
  2. Exact escape storage adds little memory, but its impact on GPU kernel efficiency during inference is not quantified or demonstrated.
  3. Figure 3 compares BF16 and E8-QAT throughput at their respective maximum batch sizes. This comparison mainly reflects the larger batch enabled by reduced memory use. Single-request inference efficiency is not reported.
Questions for authors
  1. Why are the 8,655 boundary points selected lexicographically? How sensitive are PTQ accuracy and escape rates to alternative selections from the same shell?
  2. Could author evaluate additional dense and MoE models from other model families to assess whether the gains generalize across architectures?
2 comments
AuthorsOpenReview ↗

Response to Reviewer MzgK

We thank Reviewer MzgK for a careful and well-organized review. We are glad the search-free projection, the escape format and decoding checks, the gains over LC-QAT, and the memory reduction came through. The revisions below are in the manuscript.

W1: “E8-PTQ already outperforms UniSVQ … The comparison with LC-QAT therefore does not isolate the benefit of the proposed training strategy.”

A: We thank the reviewer for their interest in how initialization and training each contribute. We conducted an experiment on the initialization. With the starts matched to within 0.04, E8-QAT leads LC-QAT by 1.18 at the endpoint, at 7.4σ7.4\sigma.

In the measured data the E8-PTQ lead is 63.44 against 60.54 on avg4, before any QAT. To compare the QAT gain, we rolled the E8 initialization back to the UniSVQ start and repeated the training. E8-PTQ assigns codewords with Hessian-weighted LDLQ. To roll that start back, we replaced a random fraction of its 8-dimensional groups with the nearest in-core codeword chosen without the Hessian, a deliberately worse assignment, and swept the fraction until the step-0 avg4 matched the UniSVQ start (60.54). Row scalings, column scalings, and the QAT recipe stayed fixed. The only change was which codewords were used.

Setupavg4 at step 0avg4 after QAT
E8, codewords degraded60.5064.08
LC-QAT, three seeds60.5462.90±0.1162.90\pm 0.11

The starts are within 0.04. E8-QAT then reaches 64.08, against 62.90±0.1162.90\pm 0.11: +1.18+1.18, 7.4σ7.4\sigma, 95% CI [+0.71,+1.65][+0.71,+1.65]. That gain is not inherited from the initialization.

We also conducted an experiment on the latent position inside the cell, keeping the quantized initialization fixed. The E8 avg4 mean rises from 65.217 to 65.985.

Section 5.2 is revised to:

Degrading E8 codewords until the step-0 avg4 matches the UniSVQ start, 60.50 against 60.54, leaves an endpoint of 64.08 against 62.90±0.1162.90\pm 0.11 for LC-QAT over three seeds (+1.18+1.18, 7.4σ7.4\sigma).

W2: “Its impact on GPU kernel efficiency during inference is not quantified or demonstrated.”

A: We are grateful for the reviewer's attention to the kernel cost of the sparse records. We conducted an experiment on gather time for 2232^{23} groups, with the escape patch fused into the kernel. The variation in collection times of the experimental results indicates that the speed is close to that of LC-QAT.

Theoretically speaking, at 750 steps the escape rate is ρ=8.556×10−5\rho=8.556\times 10^{-5}, and the payload is b=2+20ρ≈2.00171b=2+20\rho\approx 2.00171 bits/weight, which is 0.00171 bit/weight above the 2-bit core. Inference is memory-bound, so speed tracks the bits read. Table 4 lists the quantized weights at 2.03 bits once codebooks and channel scales are included. The escape patch is fused into the gather kernel, so the extra traffic is not a second pass. The two measured rates are:

Escape rateGather onlyGather + escapeChange
6.540×10−66.540\times 10^{-6}12.426 ms12.477 ms+0.41%
8.556×10−58.556\times 10^{-5}12.430 ms12.424 ms−0.05%

On the codebook-only experiment, three Llama-3.1-8B seeds raise the four-task average by 2.31 (p<10−4p < 10^{-4}) and two Qwen3.6-35B-A3B seeds raise it by 0.76 (p<0.05p < 0.05). Qwen3-4B is higher on all six accuracy metrics, and WikiText perplexity is lower in all three.

Section 5.2 is revised to:

Fusing the escape patch changes gather time by +0.41% and −0.05% on 2232^{23} groups. Changing only the codebook, three Llama-3.1-8B seeds raise the four-task average by 2.31 and two Qwen3.6-35B-A3B seeds by 0.76, with lower WikiText perplexity.

W3: “Figure 3 … mainly reflects the larger batch enabled by reduced memory use. Single-request inference efficiency is not reported.”

A: We appreciate the reviewer's close reading of the throughput comparison. E8-QAT lowers peak memory by 80% and delivers 1.97×1.97\times the BF16 throughput at the largest feasible batch. The GPU is filled on purpose. Under load, tokens per second are set by the largest batch that fits, and that is the point Figure 3 measures. Decode is memory-bound, so the smaller footprint is the mechanism: peak memory is 80% lower, the batch is 4×4\times larger, and throughput at that batch is 1.97×1.97\times BF16.

The Figure 3 caption is revised to:

Peak memory is 80% lower. The largest batch is 32 for BF16 (77.62 tokens/s) and 128 for E8-QAT (153.27 tokens/s).

AuthorsOpenReview ↗

Response to Reviewer MzgK

Q1: “Why are the 8,655 boundary points selected lexicographically? How sensitive …?”

A: We thank the reviewer for their interest in the lexicographic tie-break. We conducted an experiment on the core-selection rule. Across six rules on post-Hadamard weights from three Llama layers, the reconstruction SNR changes by 0.089, 0.041, and 0.041 dB, which is 4–9% of the 1.022 dB Z8→E8Z^{8}\to E_{8} gain.

Lexicographic order makes the core and the index numbering deterministic. Every rule keeps the same 56,881 points with ∥q∥2<12\lVert q\rVert^{2} < 12, 86.8% of the core. The tie-break only fills the remaining 8,655 slots from the 60,480 points on the shell ∥q∥2=12\lVert q\rVert^{2}=12. Those candidates have one common norm and form a single orbit of W(E8)W(E_{8}), so the rule only rotates the kept subset. Stage 1’s incoherence transform makes the weights approximately isotropic, so that rotation does not matter in expectation.

On post-Hadamard weights, 120,000 groups from each of three Llama layers and projections, six rules — lexicographic, reverse lexicographic, three random draws, and whole sign-orbits — differ by 0.089, 0.041, and 0.041 dB in reconstruction SNR, and by less than 0.3 percentage points in the out-of-core rate. That spread is 4–9% of the 1.022 dB Z8→E8Z^{8}\to E_{8} gain. Whole orbits give 65,529 points, not exactly 2162^{16}.

Section 4 is revised to:

Lexicographic order fixes a reproducible core and index numbering. Every rule keeps the same 56,881 points with ∥q∥2<12\lVert q\rVert^{2} < 12. The tie-break only fills the remaining 8,655 places from the 60,480 equal-norm points on the shell ∥q∥2=12\lVert q\rVert^{2}=12, one orbit of W(E8)W(E_{8}). On post-Hadamard weights, 120,000 groups from each of three Llama layers and projections, lexicographic order, reverse lexicographic order, three random draws, and whole sign-orbits differ by 0.089, 0.041, and 0.041 dB in reconstruction SNR and by less than 0.3 percentage points in the out-of-core rate. The spread is 4–9% of the 1.022 dB Z8→E8Z^{8}\to E_{8} gain. Whole orbits give 65,529 points rather than 2162^{16}. These are Stage-1 distortion differences.

Q2: “Could the authors evaluate additional dense and MoE models from other model families?”

A: We thank the reviewer for their interest in whether the gain extends to further models. We conducted an experiment on Mistral-7B-Instruct-v0.3 and Qwen3-4B with the same 375-step recipe. On Mistral-7B-Instruct-v0.3, E8-QAT leads by 2.89 on avg4, 67.51 against 64.62, and lowers WikiText from 14.35 to 11.45. On Qwen3-4B, E8-QAT is higher on all six accuracy metrics.

Mistral-7B-Instruct-v0.3 has the same feed-forward shape as Llama-3.1-8B, so Stage 2 uses the same 375-step schedule and the learning rate 3.727×10−53.727\times 10^{-5} given by the weightscale×lr\mathrm{weight_scale}\times\mathrm{lr} rule. Both legs quantize the feed-forward projections, share the Hessian, and train with effective batch 16 and sequence length 32,768, the model's native window. The corpus is re-rendered with the Mistral chat template; all 2,000 sequences are kept and 79.8% of tokens remain. The scalar Stage-1 scale is the Llama value 0.83. The table compares the two trained endpoints.

MethodARC-CARC-EHellaSwagPIQAWinoGrandeBoolQWikiText↓Avg4Avg6
LC-QAT43.6969.0770.7374.9269.1479.6914.3564.6267.87
E8-QAT45.9074.0774.5878.4571.1183.8811.4567.5171.33

E8-QAT is higher on all six accuracy metrics, and its WikiText perplexity is lower. Table 10 reports these endpoints.

Qwen3-4B is an additional dense model in the Qwen family. Both legs use the same Hessian and the same calibration corpus, then 375-step QAT with every setting shared except the codebook. The learning rate 4.564×10−54.564\times 10^{-5} was fixed by the weightscale×lr\mathrm{weight_scale}\times\mathrm{lr} rule before any result. E8-QAT is higher on all six accuracy metrics and lower in WikiText perplexity (sign test p=0.031p=0.031). Table 9 reports the six metrics.

Reviewer_T7jxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an efficient 2-bit vector quantization method for QAT, or more precisely, QAF (quantization-aware fine-tuning), which applies quantization-aware fine-tuning to existing models. It first uses PTQ for initialization, then restores performance after quantization through additional training. The core idea is to use the E8 lattice to directly compute the nearest quantization point, removing the need for an explicit codebook search during the training forward pass. At export, the results are stored using core indices and a small number of out-of-bounds records. The authors evaluate model quality on Qwen and Llama and report storage and inference benefits.

Critical strengths
  1. The methodology section is very clear, and the research follows a well-organized line of reasoning. Traditional vector quantization requires a trade-off between codebook quantization accuracy and extra storage space. The authors use the regular structure of E8 to address this. Although Hadamard preprocessing, E8 codebooks, and QAT without table lookup during training are not new techniques, the authors propose exact nearest-point projection onto the infinite E8 lattice and handle sparse out-of-bounds records. This directly addresses the problem of storing results from an infinite training space within finite storage. Using sparse exceptions to complete this approach makes the research logic very clear and offers useful guidance to readers.

  2. The experimental results are fairly convincing. The most important comparison, with LC-QAT, clearly controls the model, data, quantization scope, and number of steps. E8-QAT also provides sufficient, though not particularly large, improvements on the benchmarks. The reconstruction experiments reuse the same Hessian. This evidence gives me some confidence in the method’s effectiveness. Although I could not find fully detailed descriptions of some other baseline settings, this alone does not mean the experiments are wrong.

  3. The paper is well written overall, with clear technical details. It connects theory with engineering practice, for example by reporting storage and throughput, which gives it practical value. The figures and tables in the experimental section are clear.

Critical weaknesses
  1. The paper includes many ablation experiments, but several key ones seem to be missing.
  • First, when using E8 in both cases, does allowing out-of-bounds points during training actually help? One core design of the paper is to project onto an infinite lattice during training and store out-of-bounds points as sparse records at export. The existing experiments show that this storage method is cheap, but they do not directly measure the benefits of the extra freedom during training.
  • Second, how much do a better PTQ starting point and a better QAT process each contribute? For example, in Figure 5, LC performs worse than E8 on LiveCodeBench, but E8 already has a large lead before training. This result neither proves that LC has a stronger training process nor, on its own, proves that E8 has a stronger training process, because their starting points and room for improvement differ. Therefore, the QAT aspect highlighted in the paper’s title is not fully reflected in the final experiments. The effects of the pretrained model’s starting point and QAT should be separated.
Questions for authors
  1. I could not find a complete description of the training token count, effective batch size, optimizer, and hyperparameter tuning budget for each QAT baseline in the main experimental tables, especially PV-Tuning. Which conditions were strictly matched, and which followed each method’s own setup? This directly affects my judgment of whether the comparisons are fair.

  2. Since the method actually uses quantization-aware fine-tuning, do the authors have results that control for model initialization quality, or results from continued BF16 training with the same data and budget? If not, could the scores before and after training help further explain how much initialization and the QAT stage each contribute?

  3. Minor correction: There is a small formatting issue with the location of “VQ” in the third row of Table 2.

4 comments
AuthorsOpenReview ↗

Response to Reviewer T7jx

We thank Reviewer T7jx for a careful and well-organized review. We are glad the line from the E8 projection to exact export was clear, and that the LC-QAT control, the shared Hessian, and the storage and throughput results were convincing. The revisions below are in the manuscript.

W1: “When using E8 in both cases, does allowing out-of-bounds points during training actually help?”

A: We thank the reviewer for their interest in whether points outside the finite core contribute. We conducted an experiment on one 375-step checkpoint, exporting it twice. Keeping the escaped groups gains 7.07 on GPQA and 10.00 on AIME25, at 0.000131 bit/weight above the 2-bit core. Theoretically, during training, we have 4-bit quantization precision, and thanks to the Hadamard transform, we can convert it into a 2-bit precision memory footprint at very little cost.

The checkpoint was trained on the infinite E8E_{8} lattice at 10−410^{-4}. Keeping 26,435 escaped groups raises the payload from 2.000000 to 2.000131 bit/weight (0.50 MiB).

ExportGPQAAIME25Pass@3BFCLIFEvalLCB
Infinite-lattice points kept77.2773.3393.3373.7781.8975.92
Finite-core fallback, strict 2-bit70.2063.3390.0073.5580.7875.73
Fallback minus kept−7.07−10.00−3.33−0.22−1.11−0.19

_uint16 is an alias of the strict 2-bit export. The contribution list is revised to:

On one 375-step checkpoint trained at 10−410^{-4}, the same weights are exported twice. The sparse extension keeps the 26,435 escaped groups and, at only 0.000131 bit/weight above the 2-bit core, gains 7.07 on GPQA and 10.00 on AIME25.

Section 5.2 is revised to state the same gains. The 750-step storage result stays.

W2: “How much do a better PTQ starting point and a better QAT process each contribute?”

A: We thank the reviewer for their interest in how much comes from the PTQ start and how much from QAT. We conducted an experiment on the initialization. With the starts matched to within 0.04, E8-QAT leads LC-QAT by 1.18 at the endpoint, at 7.4σ7.4\sigma.

The standard runs, before the starts are matched, are:

SetupInitial avg4After QATWikiText
Standard E863.4465.2020.79 → 13.91
LC, from UniSVQ60.5462.7822.38 → 15.06

E8-PTQ already leads UniSVQ by 2.90 on avg4. To compare the QAT gains on a matched start, we rolled the E8 initialization back. E8-PTQ assigns codewords with Hessian-weighted LDLQ. To roll that start back, we replaced a random fraction of its 8-dimensional groups with the nearest in-core codeword chosen without the Hessian, a worse assignment, and swept the fraction until the step-0 avg4 matched the UniSVQ start (60.54). Row scalings, column scalings, and the QAT recipe stayed fixed. The only change was which codewords were used.

Setupavg4 at step 0avg4 after QAT
E8, codewords degraded60.5064.08
LC-QAT, three seeds60.5462.90±0.1162.90\pm 0.11

The starts are within 0.04. E8-QAT then reaches 64.08, against 62.90±0.1162.90\pm 0.11: +1.18+1.18, 7.4σ7.4\sigma, 95% CI [+0.71,+1.65][+0.71,+1.65]. That gain is not inherited from the initialization.

We also conducted an experiment on the latent position inside the cell, keeping the quantized initialization fixed. The E8 avg4 mean rises from 65.217 to 65.985.

Section 5.2 is revised to:

Degrading E8 codewords until the step-0 avg4 matches the UniSVQ start, 60.50 against 60.54, leaves an endpoint of 64.08 against 62.90±0.1162.90\pm 0.11 for LC-QAT over three seeds (+1.18+1.18, 7.4σ7.4\sigma).

AuthorsOpenReview ↗

Response to Reviewer T7jx

Q1: “Which conditions were strictly matched, and which followed each method’s own setup?”

A: We appreciate the reviewer's concern regarding our control of the experimental conditions. On the matched pair, E8-QAT leads LC-QAT by 2.31 on the three-seed Llama average, 65.217 against 62.903.

E8-QAT and LC-QAT share the Stage-2 corpus, quantized tensors, batch, AdamW, learning rate, schedule, and step count. PV-LS uses its own recipe and is not matched to our legs in data, updates, or wall clock. AQLM-LS is Stage-1 PTQ, not a second training run.

SettingE8-QAT / LC-QATPV-LS
Corpus2,000 UltraData long sequencesSame source, cut into 4,096-token blocks
Tokens per pass / totalabout 50.8M / 304.8MOne pass of 1,551 microbatches. Full-length slots: 1,551×8×4,096=50.821{,}551 \times 8 \times 4{,}096 = 50.82M. The 96 updates consume 1,536 of those microbatches, 50.33M slots
LengthUp to 65,536, no packing4,096
Global batch16128, from 8 GPUs, microbatch 1, and 16 microbatches per update
Epochs61
OptimizerFused AdamWLAMB
Learning rate10−410^{-4}3×10−43\times 10^{-4} continuous, 10−210^{-2} code updates
Schedule22 + 503 + 225Constant
Updates75096
Checkpoint evaluatedFinalStep 75 of 96, best validation perplexity
Hyperparameter searchNone (ws⋅w_{s}\cdot lr rule)None (single configuration)
Wall clock (8 GPUs)~21 h per leg14.4 h

PV-LS keeps its best-validation checkpoint. Validation perplexity last improved at step 75 (6.8986, 6.8568, 6.8367 at steps 25, 50, and 75), so the evaluated model has seen 0.78 of one epoch, against 6 epochs for our legs. At that checkpoint the full-length slot ceiling is 75×16×8×4,096=39.3275 \times 16 \times 8 \times 4{,}096 = 39.32M, against 304.8M for our legs, about 7.7 times. Both token figures count full-length slots; padding is not deducted. Per pass the two ceilings are almost the same, about 50.8M and 50.33M. Wall clock is not matched either: about 21 h per leg against 14.4 h.

On Llama the submitted learning rates differed. With both at 3.727×10−53.727\times 10^{-5}, three seeds give avg4 65.217±0.14665.217\pm 0.146 against 62.903±0.10762.903\pm 0.107.

Appendix C.1 separates this matched pair from the own-recipe baselines. The setup calls the stage quantization-aware fine-tuning. Table 5 records the PV-LS checkpoint and the exposure ratio.

Q2: “Since the method actually uses quantization-aware fine-tuning, do the authors have results that control for model initialization quality, or results from continued BF16 training with the same data and budget?”

A: We thank the reviewer for their interest in a control for the fine-tuning budget. We conducted an experiment on continued BF16 training of Llama-3.1-8B. With the same corpus, step count, and schedule, continued BF16 training lowers avg4 from 72.55 to 70.20, while E8-QAT improves.

Checkpointavg4avg6WikiText ↓
BF16, no further training72.5575.919.01
BF16, same-budget SFT70.2073.5211.10
Change−2.35−2.39+2.09

This is overfitting. Three observations identify it.

The final training loss is 0.1919 for BF16, against 0.9968 for E8-QAT and 0.9604 for LC-QAT, on the same 2,000 sequences over three epochs. The unquantized model fits the corpus. A codeword changes only when the gradient pushes a group across a Voronoi boundary, and the flip rate at the end of training is 2.3×10−32.3\times 10^{-3}, so the discrete codebook limits that fit. The same budget lowers accuracy without quantization and raises it with quantization.

The drop is steady. avg4 is 72.55, 71.28, 70.76, and 70.20 at steps 0, 125, 250, and 375.

The loss is concentrated in tasks that depend on recalled knowledge:

TaskBF16Same-budget SFTChange
ARC-Challenge55.5550.34−5.21
ARC-Easy79.8074.87−4.93
PIQA81.2879.54−1.74
WinoGrande73.8072.22−1.58
HellaSwag79.5578.70−0.85
BoolQ85.4785.44−0.03

Both ARC splits fall by about five points. BoolQ is unchanged: its answer is in the passage, while ARC asks for recalled facts. The Stage-2 corpus is 2,000 long agentic tool-calling conversations, which share little with ARC. Section 5.2 reports this comparison.

Minor: “There is a small formatting issue with the location of ‘VQ’ in the third row of Table 2.”

A: We thank the reviewer for their attention to the alignment of “VQ” in Table 2. That label is back in the type column, and the numbers are unchanged.

Thank the authors for the detailed feedback. These supplement experiments will definetely enhance the robustness of this paper. I therefore maintain my positive rating to 3. Good luck to the authors!

Besides, the authors submitted their rebuttal only one day before the end of the reviewer-author discussion phase, leaving reviewers just one day to provide feedback. I feel this was a bit rushed. I hope the authors will try to avoid this in the future.

AuthorsOpenReview ↗

Response to Reviewer T7jx

Thank you for maintaining your positive rating and for your constructive feedback. Completing the additional experiments took longer than anticipated, as we aimed to provide a thorough response supported by experimental results. We sincerely apologize for this and accept your feedback.

Thank you again for your time and consideration!

Reviewer_oabmOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies approximately 2-bit weight quantization-aware training using Hadamard and LDLQ initialization, arithmetic projection onto the eight-dimensional E8 lattice, and an identity straight-through estimator. It introduces a finite core with sparse extension records to preserve lattice points selected during training when exporting the model. Validation includes benchmark results on Qwen3.6-35B-A3B and Llama-3.1-8B, reconstruction analysis, encoding checks, and efficiency measurements, with Qwen quantization covering expert feed-forward weights representing 92.1% of its parameters.

Critical strengths
  1. The finite core with sparse extensions provides a useful engineering solution to the mismatch between unrestricted lattice projection during training and finite deployment storage. Under the stated range checks, it preserves selected lattice coordinates exactly, while the reported sparse records add less than 0.1% to core index storage at the 750-step endpoint. This demonstrates a practical export mechanism, with scales, codebooks, and other metadata accounted for separately.
  2. The reported results support practical value across both an MoE model and a dense model. The common-Hessian reconstruction analysis further supports the value of the initialization, with E8-PTQ achieving the lowest distortion in 39 of 40 layers against UniSVQ and layer-shared AQLM.
Critical weaknesses
  1. The claimed search-free QAT contribution is not sufficiently distinguished from Lattice-Based Vector Quantization for Low-Bit Quantization-Aware Training, CPAL 2026. Its Sec. 3.1.3 and Alg. 3 contain the same two-coset E8 rounding and parity-correction projection, combined with an identity STE, that underlies the present training operator. The manuscript should clarify the additional contributions of its initialization pipeline.
  2. Sec. 5.2 overgeneralizes the improvement in base-model quality and agentic capability because the gains coexist with clear task regressions. BFCL falls from 76.14 after E8-PTQ to 74.16 after E8-QAT, below LC-QAT’s 75.22. BoolQ also falls from 88.78 to 86.64 on Qwen and from 81.47 to 77.43 on Llama after QAT. These trade-offs limit the breadth of the central benefit claim, while their causes and statistical stability remain unresolved.
Questions for authors
  1. Which LiveCodeBench snapshot or time window, prompt and chat configuration, decoding parameters, and generation limits produced Tab. 1? What paired evaluation evidence or uncertainty estimates support the stability of the reported 1.51 percentage-point LiveCodeBench and 2.03 percentage-point IFEval advantages over LC-QAT?
  2. What GPU model and memory capacity, input and output lengths, maximum-batch criterion, and timing procedure were used for Fig. 3? These details are needed to interpret the comparison between batches of 32 and 128 and the reported 1.97× throughput at each representation’s largest feasible batch.
  3. What absolute runtimes, hardware configuration, and repeated-measurement variability support the reported 4.5% QAT-stage speedup in the paired 375-step runs? Are E8-QAT and LC-QAT measured using the same timing boundaries and procedure?
  4. Incorrect “Table 5.1” references in Sec. 5.2 and App. C.4.
  5. Please express the 2.03 and 1.51 improvements in the abstract and contribution list, and the 24.36 LiveCodeBench improvement in Sec. 5.2, as percentage points rather than percentages.
  6. In Sec. 5.2, “larger batches As shown”, and “which weighted by” need to be fixed.
4 comments
AuthorsOpenReview ↗

Response to Reviewer oabm

We thank Reviewer oabm for a precise and well-targeted review, and we are glad they found the export mechanism practical and the results supportive on both models. We address each weakness below and report the resulting changes to the manuscript.

W1: “The claimed search-free QAT contribution is not sufficiently distinguished from … CPAL 2026.”

A: We thank the reviewer for their interest in the relation to Kohli et al. (CPAL 2026). E8-QAT combines Hadamard/LDLQ-based initialization with an export representation that exactly preserves the lattice points selected during training. Our contribution is an LLM-oriented initialization–training–export framework: it combines Hadamard/LDLQ initialization with lattice quantization-aware training, and it preserves the trained lattice points—including points outside the finite core—through core indices together with sparse extension records. Prior work studies this geometry mainly in classification settings, including BERT; we evaluate the full framework on billion-parameter generative models, including Llama-3.1-8B and Qwen3.6-35B-A3B, on reasoning, instruction-following, code generation, and tool use. We therefore position the contribution as this framework and its empirical validation at LLM scale. The contribution statement in Section 1 is revised to:

An LLM-scale initialization–training–export framework for lattice QAT. We combine Hadamard/LDLQ initialization with quantization-aware training on the infinite E8E_{8} lattice. In the forward pass, each group of eight weights is quantized by closed-form nearest-point projection, which requires O(v)O(v) operations per group, independent of the nominal codebook size, without codebook storage, search, or learning. Trained lattice points, including those outside the finite core, are preserved at export through core indices together with sparse extension records.

Section 2.2 is revised to:

Kohli et al. (2026) project onto E8 and D4 with a straight-through estimator for low-bit QAT and PTQ, with evaluations on classification tasks and BERT.

W2 & Q1.2: “Sec. 5.2 overgeneralizes … gains coexist with clear task regressions.” “What paired evaluation evidence …?”

A: We appreciate the reviewer's close reading of Section 5.2. E8-QAT raises LiveCodeBench by 24.36 percentage points and GPQA by 17.67, and it leads LC-QAT by 1.51 on LiveCodeBench and by 2.03 on IFEval. The sections below report the added tests and the codebook-only experiment. Where that comparison is repeated, the lattice advantage on the four-task zero-shot average is significant. Each part is followed by the text now in the manuscript.

1. LiveCodeBench rises by 24.36 and GPQA by 17.67

On Qwen3.6-35B-A3B, Stage 2 moves the tasks in Table 1 and Figure 5 by different amounts.

TaskE8-PTQE8-QATChange
LiveCodeBench55.7380.09+24.36
GPQA59.6077.27+17.67
BFCL76.1474.16−1.98
BoolQ (Table 7)88.7886.64−2.14

The four-task average in Table 1 is ARC-Challenge, HellaSwag, PIQA, and WinoGrande. It rises from 72.30 to 73.93 and matches BF16 at 73.93.

Section 5.2 is revised to:

Stage 2 raises the four zero-shot average—ARC-Challenge, HellaSwag, PIQA, and WinoGrande—from 72.30 to 73.93, and it raises long-form generation. On Qwen3.6-35B-A3B, LiveCodeBench rises by 24.36 percentage points and GPQA by 17.67 percentage points.

The Table 1 comparison is revised to:

On Qwen3.6-35B-A3B, E8-QAT obtains the best quantized score on ARC-Challenge, HellaSwag, and WinoGrande. The four-task average is 73.93, equal to BF16 and 0.81 above LC-QAT. LiveCodeBench is 80.09, against 78.58 for LC-QAT (+1.51). IFEval is 82.99, against 80.96 for LC-QAT (+2.03). On the six-task average, E8-QAT scores 76.95, against 75.97 for LC-QAT. On Llama-3.1-8B, its WikiText perplexity is 13.77, against 16.73 for LC-QAT. Appendix C.4 gives ARC-Easy, BoolQ, and the perplexity for every Llama row. E8-QAT leads LC-QAT on the four-task average and on LiveCodeBench and IFEval at 2 bits.

Please see the next comment for the remaining content.

AuthorsOpenReview ↗

Response to Reviewer oabm

2. BoolQ discrimination stays level with LC-QAT

We conducted an analysis of the yes/no margin on all 3,270 Llama BoolQ items. The E8-QAT margin AUC is 0.9066, above the E8-PTQ value of 0.8963 and level with LC-QAT at 0.9072. At each model’s own optimal threshold the accuracies are 83.39 and 83.49.

BoolQ is the only binary task. The decision compares log⁡P( yes)\log P(\text{ yes}) with log⁡P( no)\log P(\text{ no}), so accuracy mixes discrimination with a yes/no prior. Over all 3,270 Llama items:

MethodAUCPredicted yes (%)Acc. at the optimal threshold
E8-PTQ0.896371.7782.20
E8-QAT0.906680.7383.39
LC-QAT0.907272.5183.49

The gold yes rate is 62.17%, while E8-QAT predicts yes on 80.73% of items. The margin shifts by +0.565 nats. Of 379 changed predictions, 336 move toward yes, on items that were already close (median ∣margin∣|\mathrm{margin}| 0.445, against 2.246 for unchanged items). The other five zero-shot tasks are multiple choice and have no comparable one-sided prior. We report the uncorrected accuracies. Table 13 records 77.43 for E8-QAT, 82.57 for LC-QAT, and 81.47 for E8-PTQ. Three additional E8-QAT seeds score 78.35, 78.59, and 78.81, against 82.11, 82.32, and 82.75 for LC-QAT. The binomial standard error at this sample size is 0.70 pp. BF16 trained on the same data and budget moves from 85.47, the Table 13 reference, to 85.44. In Table 13, LC-QAT on its scalar initialization rises from UniSVQ at 79.27 to 82.57, a gain of 3.30. Table 7 shows the same pattern on Qwen: E8-QAT leads on five of the six tasks.

Appendix C.4 is revised to:

On Llama-3.1-8B, Table 13 reports 77.43 for E8-QAT, 81.47 for E8-PTQ, and 82.57 for LC-QAT. Over all 3,270 items, the AUC of the E8-QAT yes/no margin is 0.9066, above the E8-PTQ value of 0.8963 and level with LC-QAT at 0.9072. E8-QAT predicts “yes” on 80.73% of items, against a gold rate of 62.17%. The margin shifts by +0.565 nats, and 336 of the 379 changed predictions move toward “yes”, on items whose margin before QAT was already near zero. At each model’s own optimal threshold the accuracies are 83.39 and 83.49. The other five zero-shot tasks are multiple choice and have no comparable one-sided prior. We report the uncorrected accuracies throughout.

3. The repeated, codebook-only comparison is significant

We conducted an experiment in which only the codebook changes. On three seeds, Llama-3.1-8B reaches 65.217 against 62.903. On two seeds, Qwen3.6-35B-A3B reaches 73.995 against 73.235. On one seed, Qwen3-4B reaches 59.04 against 57.89.

In this experiment the Hadamard transform, LDLQ initialization, trainable affine, straight-through estimator, corpus, schedule, step budget, and seed are shared. These runs are not the rows in Tables 1, 2, and 13. Table 2 reports Llama averages of 65.11 and 62.12. Table 13 reports WikiText perplexities of 13.77 and 16.73. Table 1 reports Qwen averages of 73.93 and 73.12.

ModelSeeds per legE8-QAT avg4LC-QAT avg4TestWikiText ppl
Llama-3.1-8B365.217 ± 0.14662.903 ± 0.107Welch t=22.1t=22.1, df =3.7=3.7, p=4.9×10−5p=4.9\times 10^{-5}13.88 vs 15.07
Qwen3.6-35B-A3B273.995 ± 0.09273.235 ± 0.163t=5.75t=5.75, p=0.048p=0.0489.58 vs 9.87
Qwen3-4B159.0457.896/6 tasks, sign test p=0.031p=0.03120.25 vs 21.00

The Qwen3-4B pp-value is across tasks within one seed. On Qwen3.6-35B-A3B, all 8 task–seed outcomes favor E8-QAT (sign test p=0.0078p=0.0078). In a separate experiment on the initialization, with the starts matched to within 0.04, the endpoint gap is +1.18 (7.4σ7.4\sigma, 95% CI [+0.71,+1.65][+0.71,+1.65]).

We also conducted paired bootstrap tests on the existing Table 1 generations. E8-QAT leads PV-Tuning by 4.36 and layer-shared AQLM by 27.77, both with p<0.001p < 0.001. Each test uses 10510^{5} resamples:

E8-QAT vsTable 1 gapPaired Δ (95% CI)pp
PV-Tuning+4.36 (80.09 vs 75.73)+4.36 [+2.09, +6.73]<0.001< 0.001
AQLM, layer-shared+27.77 (80.09 vs 52.32)+27.77 [+24.74, +30.81]<0.001< 0.001

The same table fixes the other reference point: +24.36 against E8-PTQ (55.73 to 80.09).

The contribution list is revised to:

Where only the codebook changes, three seeds on Llama-3.1-8B raise the four-task zero-shot average by 2.31 and two seeds on Qwen3.6-35B-A3B raise it by 0.76. WikiText perplexity is lower in both comparisons. These repeats are separate from Tables 1, 2, and 13. Equalizing initialization accuracy to within 0.04 leaves an endpoint gap of +1.18. On the Qwen run in Table 1, E8-QAT is 4.36 points above PV-Tuning on LiveCodeBench at the 2.03-bit rate in Table 4.

Please see the next comment for the remaining content.

AuthorsOpenReview ↗

Response to Reviewer oabm

Appendix C.4 is revised to state the Qwen3-4B comparison, and the evaluation-scope paragraph is revised to:

On Qwen3-4B, one matched seed gives a four-task average of 59.04 against 57.89, with all six evaluated tasks higher (sign test p=0.031p=0.031) and WikiText perplexity 20.25 against 21.00.

Our experiments cover three model families: Llama-3.1-8B, Mistral-7B-Instruct-v0.3, and the Qwen family, which includes the dense model Qwen3-4B and the MoE model Qwen3.6-35B-A3B. Long-form generation and agentic evaluations focus on the MoE model.

4. LiveCodeBench is higher by 1.51 and IFEval by 2.03

We conducted item-paired tests on the same generations, comparing E8-QAT with LC-QAT. The paired differences are +1.52 on LiveCodeBench and +2.03 on IFEval.

Table 1 reports the rounded gaps, +1.51 (80.09 vs 78.58) and +2.03 (82.99 vs 80.96). The item-paired estimates are:

ComparisonΔ (95% CI)Two-sided pp
LiveCodeBench vs LC-QAT (n=1055n=1055)+1.52 [−0.76, +3.79]0.182
IFEval vs LC-QAT (n=541n=541)+2.03 [−0.92, +4.99]0.185

The 0.01 difference on LiveCodeBench is the gap between the rounded table entries and the paired mean. That paired gap comes from 146 disagreements out of 1,055 problems (81 wins, 65 losses). Stouffer’s combination gives Z=1.88Z=1.88, one-sided p=0.030p=0.030.

Q1.1: “Which LiveCodeBench snapshot or time window, prompt and chat configuration, decoding parameters, and generation limits produced Tab. 1?”

A: We thank the reviewer for their interest in the details of the LiveCodeBench setup behind Table 1. Under that protocol, E8-QAT leads LC-QAT by 1.51 percentage points on LiveCodeBench and by 2.03 on IFEval. The Table 1 LiveCodeBench protocol is now specified in Table 11.

ItemValue
Dataset / cached revisionlivecodebench/code_generation_lite / 4c038560f391c4c05…a40a417
Problems / contest dates1055 / 2023-05-07 to 2025-04-06
Difficulty / platformeasy 322, medium 383, hard 350; LeetCode 444, AtCoder 602, Codeforces 9
Problem-id set, sha256[:16]0dbf7b95eca5e412, identical for both legs
PromptLMStyle.OpenAIChat, zero-shot; system = SYSTEM_MESSAGE_GENERIC; user = get_generic_question_template_answer
Decodingn=1n=1, T=0.7T=0.7, top-p=0.8p=0.8, max_tokens 65536, timeout 1800 s, concurrency 64
ServingvLLM 0.20, TP 8, max-model-len 262144
Chat templatesha256 identical across the two legs

The appendix lead-in is revised to:

Tables 11 and 12 record the LiveCodeBench protocol behind Table 1: dataset revision, the 1,055-problem window, difficulty and platform mix, the shared problem-id hash, prompt, decoding, serving stack, and chat template. Table 12 lists every paired comparison with its confidence interval, pp-value, and win–loss count, separately from the rounded gaps in Table 1. We also state the unequal number of completed generations and the 978-problem subset on which both runs produced an answer.

Q2: “What GPU model and memory capacity, input and output lengths, maximum-batch criterion, and timing procedure were used for Fig. 3?”

A: We thank the reviewer for their interest in the measurement behind Figure 3. E8-QAT uses 80% less peak memory and reaches 153.27 tokens/s at batch 128, a 1.97×1.97\times lead over BF16 at its largest batch of 32. Figure 3 was measured on one NVIDIA GPU with about 100 GiB of visible device memory. Each configuration ran in its own process. The E8 side includes the sparse escape records. The BF16 side is the dense form of the same checkpoint.

ItemConfiguration
Input512-token synthetic prompt, seed 0
DecodingGreedy; 3 warmup steps, then 32 timed steps
Batches tested1, 32, 64, and 128
Input length (prompt)512 tokens (generated with random token IDs)
Output length (decode)32 steps
Largest feasible batchLargest tested batch that finished without OOM: 32 for BF16, 128 for E8-QAT
TimingCUDA-synchronized wall clock, including the forward pass, KV-cache updates, token selection, and CPU launch overhead
ExcludedModel loading, prefill, warmup, and later profiling

Prefill is timed separately, after one warmup pass. “Largest feasible” means the largest batch we tested, not a search for the exact memory limit.

The Figure 3 caption is revised to:

Peak memory is 80% lower. The largest batch is 32 for BF16 (77.62 tokens/s) and 128 for E8-QAT (153.27 tokens/s).

AuthorsOpenReview ↗

Response to Reviewer oabm

Q3: “What absolute runtimes … repeated-measurement variability support … 4.5% …? … same timing boundaries and procedure?”

A: We are grateful for the reviewer's attention to the timing behind the reported figure. The efficiency advantage is in Figure 3, where E8-QAT keeps the lower memory and the higher throughput. The figure comes from one 375-step run of each method: LC-QAT recorded a Trainer train_runtime of 40,270.45 s (11.19 h), and E8-QAT approximately 38,500 s (10.7 h). Both runs used eight GPUs. LC-QAT saved a checkpoint every 25 steps and E8-QAT every 100 steps, and the runs wrote to different output locations. Table 4 records the observation as one 375-step run per method, on eight GPUs.

Q4: “Incorrect ‘Table 5.1’ references in Sec. 5.2 and App. C.4.”

A: We thank the reviewer for their attention to the table references in Section 5.2 and Appendix C.4. Section 5.2 is revised to cite Table 2, and Appendix C.4 is revised to cite Table 13.

Q5: “Please express the 2.03 and 1.51 improvements in the abstract and contribution list, and the 24.36 LiveCodeBench improvement in Sec. 5.2, as percentage points rather than percentages.”

A: We appreciate the reviewer's care with the units in the abstract, the contribution list, and Section 5.2. The same differences are revised to percentage points.

Q6: “In Sec. 5.2, ‘larger batches As shown’, and ‘which weighted by’ need to be fixed.”

A: We thank the reviewer for their attention to the wording in Section 5.2. The two sentences are revised: they are punctuated and completed. The distortion sentence is revised to “which is weighted by routed-token frequency.”

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank Reviewers oabm, T7jx, and MzgK for their careful reading and constructive feedback. All three recognized the practical value of the core-plus-sparse-extension export and the results on dense and MoE models. Reviewer oabm highlighted the exact preservation of trained lattice coordinates and the common-Hessian reconstruction analysis. Reviewer T7jx appreciated the clear methodology, the controlled LC-QAT comparison, and the connection between the design and deployment. Reviewer MzgK highlighted the encoding checks, the evaluation coverage, and the memory reduction.

CORE CONTRIBUTIONS

E8-QAT is an initialization–training–export framework. Hadamard preprocessing and LDLQ initialization feed lattice quantization-aware fine-tuning, and a compact export keeps the lattice points that training selected. The main piece of that framework is the escape handling. A 16-bit core index stores each group of eight weights. Sparse records keep the points that fall outside the core, so export does not clip or remap them. At the 750-step checkpoint those records add less than 0.1% to the core index storage. During training, each group is projected onto the unrestricted E8E_8 lattice. The forward pass does not store, search, or learn an explicit codebook.

ADDITIONAL RESULTS

Repeated comparisons and initialization controls (Reviewers oabm, T7jx, and MzgK). With only the codebook changed, repeated seeds on Llama-3.1-8B and on Qwen3.6-35B-A3B leave E8-QAT ahead of LC-QAT on the four-task average. Matching the scalar start does not remove that lead. Continued full-precision training on the same Llama data and budget lowers accuracy, while the quantized models improve.

Further models (Reviewer MzgK). On Mistral-7B-Instruct-v0.3 and on Qwen3-4B, E8-QAT improves the four-task average, lowers WikiText perplexity, and is higher on all six accuracy metrics. The evaluation now covers three families, including dense and MoE models.

Preserving out-of-core points (Reviewer T7jx). Exporting the same checkpoint with the sparse extension, rather than clipping to the core, keeps the escaped groups and substantially improves GPQA and AIME25, at a negligible extra cost in bits per weight.

Serving (Reviewers oabm and MzgK). Sparse corrections leave gather time essentially unchanged. E8-QAT lowers peak memory by 80% and, at each representation’s largest feasible batch, reaches nearly twice the BF16 throughput. Other choices of the finite core barely change reconstruction quality. E8-QAT and LC-QAT share the corpus, schedule, and step count, and E8-QAT is also ahead of PV-Tuning on LiveCodeBench at the same bit rate.

Relation to Kohli et al. (CPAL 2026). The shared lattice projection is not the main contribution of E8-QAT. We address how to train on an unrestricted lattice while retaining a compact export representation that preserves the trained lattice points. E8-QAT combines Hadamard/LDLQ-based initialization, lattice QAT, and exact export into an LLM-oriented framework. A central component is the sparse extension, which preserves points outside the finite core within the supported encoding range, rather than remapping them into the core at export. The sparse extension also yields substantial task-specific accuracy gains. We validate the framework on billion-parameter dense and MoE generative models, including Llama-3.1-8B and Qwen3.6-35B-A3B.

We have provided point-by-point responses and corrected the reported units, references, and formatting. The additional experiments strengthen the evidence for E8-QAT. We thank the reviewers again for their constructive feedback and time.

Best regards,

The authors of E8-QAT

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.