acceptodds
Under review as a conference paper at ICLR 2027

CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search

Abstract

The best two-bit weight quantizers for large language models, such as QTIP and Proteus, share a recipe. Each weight matrix is first rotated by a random orthogonal transform so that its entries look Gaussian, and this rotation has to be undone at every decoding step. The rotated weights are then encoded with a trellis or lattice code under a Euclidean search, and the layer Hessian is used only in the error-feedback step between coding blocks. We show that this recipe leaves part of the Hessian unused. Error feedback turns the loss into a weighted sum of per-coordinate rounding errors, and the weights, the diagonal of the Hessian's LDL factorization, are computed by existing quantizers but never read. We put these weights into the search's distance, the Viterbi branch metric, so that the search follows the curvature within each coding block. This also explains what the rotation is for. The rotation removes this within-block variation, so after it the weight has little left to exploit. Weighting in the native basis and rotating are therefore substitutes. We confirm this on three models: the weighted native search matches the baselines' rotation, a full-dimension randomized Hadamard, to within about one point of downstream accuracy, and weighting after the rotation gains little. Around this search we build CurveTQ, a trellis codec with no rotation. The rotation also flattened the amplitude and the marginal shape of the weights, which we handle directly with a factored scale field and a closed-form quantile table. In either basis, error feedback passes each block's residual on to the next, so we store a start state per coding block that lets the trellis adapt to it. At two bits CurveTQ is 1-3 points higher in mean downstream accuracy than QTIP and Proteus on three 4-8B Instruct models, even after both are given our start state, which by itself lifts either baseline by 1-3 points. It also leads on a 35B mixture of experts, to our knowledge the first trellis-coded result on such a model. With no rotation to undo, our decoder is the fastest of the three at all tested batch sizes and bit widths.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training quantization, weight-only quantization, trellis-coded quantization, low-bit LLM, Hessian-aware quantization, model compression, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RFpFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks what the random rotation used by state-of-the-art two-bit trellis quantizers actually accomplishes, and argues it does three separable things: it evens out local weight amplitude, it makes the weight marginal Gaussian, and it removes within-block variation in a curvature weight that error feedback already computes but that existing coders never read inside the search. Placing that weight in the Viterbi branch metric is argued to substitute for the third effect, while the first two are handled directly by a factored per-group scale field and a per-layer quantile table, yielding CurveTQ, a trellis codec that applies no transform at decode. The paper also replaces the baselines' tail-biting block termination with a block-local trellis whose start state is stored, and transfers that component to the baselines. Validation covers six checkpoints including a mixture of experts, a rotation-by-metric factorial, component and termination ablations, and a fused CUDA decoder measured against released baseline kernels.

Critical strengths

S1. The paper reframes a component everyone uses rather than adding another codec.

Treating the random rotation as three separable jobs, and then asking which of them could be taken over by a quantity the solver already computes, is more informative than a further comparison between codes. The supporting observation — that existing trellis and lattice quantizers derive this curvature quantity in order to run error feedback and then discard it inside the search — is genuine, and it is actionable independently of whether the substitution claim holds in full.

S2. The rotation-free claim is carried through to a measured decode result, with the gain correctly attributed.

The authors implement a fused dequantize-and-multiply kernel and report it as the fastest of the three codecs at every batch size and bit width tested. They then add the baselines' transform back into their own kernel, so the throughput gain is attributed to the transform a rotation-free decoder does not pay for rather than to implementation effort the baselines did not receive. This is where the paper's claim and its evidence line up most cleanly. The reported rate is likewise charged for every stored quantity, including the side information that carries most of the margin.

Critical weaknesses

W1. The experiment that tests the paper's central claim is the thinnest part of the evaluation, and one contrast within it runs the other way.

The rotation-by-metric factorial is the only direct test of the substitution claim on which the title and abstract rest. It is run on three checkpoints with a small number of calibration slices per cell, and on one of them the unrotated Euclidean cell is reported as collapsing across the task suite, so every contrast involving it is omitted — leaving two checkpoints on which the full design can be read. Within that design, the contrast asking whether rotating still helps once the curvature weight is in place is positive and consistent across slices on two of the three models, with the quantile table and the scale field already present in every cell. A claim of this prominence would be better supported on more checkpoints, and on an account of what the rotation still contributes once all three of the paper's substitutes are in place.

W2. The comparison frame does not support the strength of the claimed margin.

The proposed build is charged a higher rate than every baseline it is compared against, and the strongest baseline — the same codec given the paper's own start-state component, at a lower rate — is better on perplexity on most of the models tested. The stated downstream margin holds on a subset of the models and does not appear at the higher bit widths also reported. Separately, no method is fine-tuned anywhere, while the extended related work converts the value of a post-quantization fine-tuning stage into effective bits and places it above the spread between any two codes compared here; whether the ordering survives that stage is deferred. The margins are therefore established in a regime that the paper itself identifies as not the one the baselines' published numbers come from.

Questions for authors
  1. Could the factorial be run on more checkpoints, including ones where the unrotated Euclidean cell does not collapse? As reported, the full design is readable on two models.

  2. Could any comparison be reported at matched rate, either by raising the baselines to your rate or reducing yours to theirs? The strongest baseline currently sits below your rate and above your build on perplexity on most models.

4 comments
AuthorsOpenReview ↗

Response to Reviewer RFpF (1/2): the factorial on six checkpoints, and rate

We thank the reviewer for the careful reading and for stating both concerns precisely. We agree with both. We have run the factorial on three more checkpoints (W1/Q1) and answer the comparison frame with the rate accounting, a three-slice replication, the higher bit widths and a uniform fine-tuning stage (W2/Q2). Table, figure, section and equation numbers refer to the submitted paper; the tables in this reply are unnumbered.

W1 / Q1 — the factorial on more checkpoints. We ran the 2×2 design of Table 3 (full-dimension RHT × {Euclidean, 1/di21/d_i^2}, same format and rate, four disjoint calibration slices, paired contrasts) on three more checkpoints: Llama-3.1-8B (14-task suite), Qwen3-8B (34-task suite) and Mistral-7B (the four zero-shot tasks of Table 1's Llama-2-7B column); 48 builds. Contrasts as in Table 3, mean accuracy points: P1/S1 = weighted − Euclidean, native/rotated; R-e/R-w = rotated − native, Euclidean/weighted; SUB = rotated Euclidean − native weighted. Mean ± s.d. over the four slices; (k): slices of four on which the contrast is positive.

CheckpointP1S1R-eR-wSUB
Table 3 of the paper
Qwen3-4B+2.44 ± 0.57+0.36 ± 0.51+3.54 ± 0.71+1.46 ± 0.42+1.10 ± 0.63
Llama-2-7B+4.29 ± 1.04+0.08 ± 0.53+4.68 ± 1.40+0.48 ± 0.14+0.39 ± 0.66
Llama-3-8B—−0.13 ± 0.19—−0.06 ± 0.67+0.07 ± 0.84
New
Llama-3.1-8B+17.26 ± 3.52 (4)−0.28 ± 0.39 (1)+18.29 ± 3.49 (4)+0.76 ± 1.00 (3)+1.03 ± 0.84 (3)
Qwen3-8B+2.80 ± 0.59 (4)−0.33 ± 0.30 (1)+3.49 ± 0.19 (4)+0.36 ± 0.80 (2)+0.68 ± 0.64 (4)
Mistral-7B+3.50 ± 0.86 (4)−0.63 ± 0.89 (1)+4.42 ± 0.52 (4)+0.28 ± 0.91 (2)+0.91 ± 0.98 (3)

The full design is now readable on five checkpoints. The native Euclidean cell does not collapse on Qwen3-8B and Mistral-7B, where it is 2.8 and 3.5 points behind on average, and is 13–22 points per slice behind on Llama-3.1-8B (perplexity 10.4–12.9 against 9.6), the Llama-3 fragility again. The three findings of Section 4.2 hold on each new checkpoint: the weight gains in the native basis (P1 positive on all 12 slice comparisons), adds nothing after the rotation (S1 within ±0.7), and the native weighted search is within about a point of the rotated Euclidean one (SUB +0.7 to +1.0). R-w, the contrast the reviewer asks about, is +0.3 to +0.8 points on the three new checkpoints, positive on two or three slices of four, and moves perplexity by less than 0.2 on all three. With the weight, the scale field and the table in place, what the rotation still adds is small but not zero: the mean R-w contrast ranges from −0.06 to +1.46 points across the six checkpoints and is below one point on five of them. The factorial does not identify what this remainder is; it may be within-block Hessian coupling that a diagonal weight does not capture, or an interaction with the transformed weight distribution. The revision adds the three checkpoints to Table 3 and states the contribution in this form rather than as zero.

W2 / Q2 — the comparison frame.

Rate. Every bits-per-weight figure counts every stored quantity (Appendix J): 2.287 = 2.000 code + 0.069 scale field and table + 0.219 start state, against 2.222 for Proteus+start and 2.190 for QTIP+start. The whole gap to Proteus+start is the per-(row, 64-group) scale field. The baselines raised above our rate are in Appendix D: the +start+g128 rows (2.31 / 2.34 bpw; single calibration slice) give them our start state and a 128-group scale, which moves the +start rows by −0.7 to +0.8 points; against them CurveTQ is +1.26 [+0.29, +2.22] (Qwen3-4B), +0.36 [−0.54, +1.26] (Llama-3-8B) and +1.02 [+0.13, +1.94] (Llama-3.1-8B) over Proteus+start+g128, and +2.6 to +2.7 with intervals excluding zero over QTIP+start+g128 (Table E.1). Even with the baselines raised above our rate, CurveTQ is ahead in all six contrasts, five of them with intervals excluding zero. A rotated code has little use for a group scale because the rotation has already evened out local amplitude; in the native basis the scale is what keeps a two-bit grid from collapsing on outlier channels (Appendix C). The 0.069 bpw is the price of doing that job as side information rather than as a decode-time transform, which is charged in time instead: adding the transform to our decoder costs 4–11% of batch-1 throughput, against 3–4.5% more weight bytes for our rate (Section 4.4).

AuthorsOpenReview ↗

Response to Reviewer RFpF (2/2): accuracy over three calibration sets, and fine-tuning

W2 / Q2 — the comparison frame (continued).

Perplexity and downstream accuracy. On perplexity we agree. We have rebuilt every trellis row of Table 1 on two further calibration sets B and C (128 sequences of 2048 C4 tokens each, disjoint from the paper's set A, same command lines, full suite); against the two start-state baselines over the three slices (paired on the same slice, positive favours CurveTQ; mean ± s.d., with the slices on which CurveTQ is ahead):

2-bit, CurveTQ (2.29 bpw) vsProteus+start (2.22): Δ\Delta perplexityProteus+start: Δ\Delta downstreamQTIP+start (2.19): Δ\Delta perplexityQTIP+start: Δ\Delta downstream
Qwen3-4B+1.48 ± 0.35 (3/3)+1.41 ± 0.50 (3/3)+1.77 ± 0.72 (3/3)+2.48 ± 0.88 (3/3)
Llama-3-8B+0.07 ± 0.17 (2/3)+0.85 ± 0.40 (3/3)+0.29 ± 0.03 (3/3)+1.65 ± 0.80 (3/3)
Llama-3.1-8B+0.12 ± 0.02 (3/3)+0.66 ± 0.67 (3/3)+0.30 ± 0.09 (3/3)+2.08 ± 0.97 (3/3)
Qwen3-32B−0.01 ± 0.01 (0/3)+0.05 ± 0.14 (2/3)+0.04 ± 0.04 (3/3)+0.41 ± 0.19 (3/3)
Llama-2-7B*−0.06 ± 0.06 (0/3)−0.03 ± 0.21 (1/3)+0.07 ± 0.03 (3/3)−0.02 ± 0.55 (1/3)

*Four zero-shot tasks, as in Table 1.

CurveTQ and Proteus+start are at perplexity parity on four of the five models and CurveTQ is 1.5 lower on Qwen3-4B; the margin is downstream: CurveTQ is ahead on every slice of the three 4–8B models, by 0.7–1.4 points on average (1.7–2.5 over QTIP+start), and a tie on Qwen3-32B and Llama-2-7B. Against QTIP+start CurveTQ has the lower perplexity on every slice of all five models. The revision states the two-bit claim in this form in the abstract and Section 4.1. In the revision these two-bit differences form a new bottom panel of Table 1, and the full three-slice results, every method at two and three bits, are in Tables E.3 and E.4.

Higher bit widths. At three bits the +start baselines draw level, as Section 4.1 says: over the three slices every contrast against Proteus+start is within ±0.4 points and 0.07 perplexity (+0.4 to +0.8 over QTIP+start), and at four bits every trellis codec is within a point. The two-bit margin is the only one we claim, and the revision words it so.

Fine-tuning. We have since applied one post-quantization stage to every method: QTIP's own end-to-end stage with its published recipe (per-channel scales and the unquantized parameters trained, trellis indices frozen, 512 sequences of 4096 tokens, 4 epochs), on Qwen3-4B and the three Llama models, three calibration slices each, 60 fine-tuned models. Every method gains and the baselines gain more (Qwen3-4B: +1.9 points for CurveTQ, +2.1 to +2.8 for the baselines; their perplexity falls by 2.2–3.9, ours by 0.9), so the lead narrows but the ordering survives: after fine-tuning CurveTQ has the highest accuracy on all four models and the lowest perplexity on three (a tie on Llama-2-7B). Over Proteus+start, pooled over the three slices:

CurveTQ − Proteus+start, points [95% CI over tasks]before fine-tuningafter fine-tuning
Qwen3-4B+1.41 [+0.95, +1.88]+1.11 [+0.54, +1.72]
Llama-3-8B+0.85 [+0.28, +1.62]+0.65 [+0.15, +1.27]
Llama-3.1-8B+0.66 [+0.02, +1.33]+0.52 [−0.28, +1.34]
Llama-2-7B−0.03 (tie)+0.29 (tie)

The published QTIP and Proteus numbers include their fine-tuning stages; Table 1 compares every method without one, and this experiment with the same one for all. The revision adds it as Appendix K, with all five methods before and after fine-tuning and CurveTQ's lead over each (Tables K.1–K.4), and says in Section 4.1 that the margin narrows under fine-tuning and is no longer distinguishable from zero on Llama-3.1-8B.

Thank you for your response.

After careful consideration, I decided to maintain my score. I remain open to the discussions and decisions of other reviewers.

Thank you for reading our responses and for the careful review. Your questions led directly to the three new checkpoints in Table 3, the replication over three calibration sets and the common fine-tuning stage, which together make the paper's evidence considerably stronger. We appreciate your positive assessment of the paper.

Reviewer_gwNbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies weight-only post-training quantization of LLMs. This paper is motivated by the fact that QTIP and Proteus use a random rotation to make weights easier to build the codebook, while their Euclidean trellis search does not use the within-block curvature information. Therefore, this paper proposes a method named CurveTQ. This method puts an LDL-derived weight in the Viterbi branch metric and removes the inference-time rotation during the dequantization. It also uses a stored start state for each coding block, a factored scale field, and a quantile emission table. A controlled experiment on three models finds that weighted search is within about one percentage point of rotated Euclidean search in downstream accuracy, while weighting after rotation adds little. In calibration-only comparisons, CurveTQ improves two-bit accuracy over QTIP and Proteus on three models of sizes between 4B and 8B. It also achieves performance that is approximately tied with baselines on a 32B model. Its decoding process (dequantization) has the highest measured throughput among the three implementations on an RTX 3090 GPU.

Critical strengths
  • The idea is justified. Table 3 changes only the rotation and the branch metric within one codec. Weighting helps in the native basis and adds little after rotation. Figures 2 and 3 connect this pattern to within-block curvature variation.

  • The evaluation under 2-bit quantization shows competitive performance with stronger baselines. The paper adds its stored start state to QTIP and Proteus. The start state itself yields a transferable 1–3 point gain at two bits. Also, CurveTQ retains a smaller downstream margin on three models (4B - 8B).

Critical weaknesses
  • From my understanding, this paper aims to remove the rotation for trellis or lattice code quantization. Its argument is that by considering the Hessian within blocks, one can achieve the same quality (of the codebook search results), and this removes the need for rotation during dequantization/decoding. I wonder why this can instead achieve even better performance than existing methods. Moreover, it would be better to add more evaluation of inference speed to justify the design.

  • The practical comparison lacks post-quantization tuning. The QTIP and Proteus baselines (the original papers) also include fine-tuning experiments. The paper notes that tuning can change perplexity more than the spread between codecs. It is unclear whether on-par performance still holds for the complete pipelines of other baselines.

  • The writing of this paper is a bit hard to follow. For example, in the Preliminaries, many notations are undefined (including wr,D,Uw_r, D, U), making it hard to understand this paper's idea.

Questions for authors
  • Do you have a matched post-quantization tuning result for CurveTQ and the start-state-equipped QTIP/Proteus baselines on one representative two-bit model?

  • What measurable property/quantity of a model or Hessian-based statistics predicts when native weighting will perform worse/better than rotation?

5 comments
AuthorsOpenReview ↗

Response to Reviewer gwNb (1/3): where the two-bit gains come from, and speed

We thank the reviewer for the careful reading and the accurate summary, and for recognizing that Table 3 isolates the rotation and the branch metric within one codec and that our start state transfers to QTIP and Proteus. The questions are well taken. W1 in particular points to something the submission did not make clear enough, namely where the two-bit gains come from, so we begin with a short summary of the method and its evidence and then answer each point. Table, figure, section and equation numbers refer to the submitted paper; the tables in this reply are unnumbered.

What CurveTQ does, in one paragraph. QTIP and Proteus rotate the weights so that a Euclidean trellis search works, and pay for the rotation at every decode step. (1) We replace the rotation by a curvature weight that the error feedback already computes. The weighted native search matches the rotated Euclidean search to within about a point (Table 3, now on six checkpoints) and does not beat it; what it buys is a decoder with no transform, 1.04–1.10× QTIP and 1.12–1.36× Proteus at batch 1. (2) The accuracy comes from two stored components. The per-block start state is the larger: worth 4–5 points in our codec (Table 4) and 1–3 points when we give it to QTIP and Proteus (the +start rows of Table 1), for 0.22 bpw and 1–3% of decode throughput. The native basis then keeps a per-module quantile emission table and a factored per-group scale field, which a rotated, Gaussian-coded design has little use for; they are the main remaining difference from the +start baselines (on Qwen3-4B, removing the table alone brings CurveTQ back to Proteus+start, Appendix D). (3) At two bits this gives 2.8–4.8 points over QTIP and Proteus at their published rates on the 4–8B models, 0.7–2.5 points over the same codecs given our start state (ties on Qwen3-32B and Llama-2-7B), at the highest decode throughput of the three.

W1 — why better than the rotation rather than equal, and more speed evaluation. We thank the reviewer for this question; it goes to the core of the paper, and the submission did not answer it as directly as it should have. As summarized above, the "better" is not the weight, which only substitutes for the rotation (on the three checkpoints added since submission, Llama-3.1-8B, Qwen3-8B and Mistral-7B, the difference stays within about a point); it is the start state and the native-basis table and scale field. Table 4 isolates the table on Qwen3-4B (removing it costs 2.13 perplexity and 1.2 [0.2, 2.2] points). Rebuilt on three calibration slices, the margin over Proteus+start is 0.7–1.4 points on the three 4–8B models, positive on all nine (model, slice) pairs, a tie on Qwen3-32B and Llama-2-7B, and parity in perplexity except on Qwen3-4B; at three bits the start-state baselines are within 0.8 points of CurveTQ, and at four bits all trellis codecs are within a point.

We also agree that the speed claim deserved a more careful evaluation. Since submission we have re-reported the RTX 3090 benchmark as mean ± s.d. over three independent runs, with every QTIP cell measured on our machine (its released kernels with fusion added): CurveTQ is the fastest of the three in all 18 configurations, 1.04–1.10× QTIP and 1.12–1.36× Proteus at batch 1 and 1–13% ahead at batch 64 and 128. The within-implementation control of Section 4.4 shows where the margin comes from: adding the baselines' Hadamard to our own decoder costs 4–11% of batch-1 throughput. The revision adds the standard deviations to Tables 5 and I.1.

AuthorsOpenReview ↗

Response to Reviewer gwNb (2/3): matched fine-tuning and presentation

W2 / Q1 — matched post-quantization tuning. We thank the reviewer for this suggestion and agree that it matters: the published QTIP and Proteus results include a fine-tuning stage, whereas our comparison did not. Following the reviewer's suggestion, we ran the matched comparison on four two-bit models. We applied one post-quantization stage to every method: QTIP's own end-to-end stage with its published recipe (per-channel scales and the unquantized parameters trained, trellis indices and code tables frozen, 512 sequences of 4096 tokens, 4 epochs), to CurveTQ, QTIP, Proteus and the two +start baselines on Qwen3-4B, Llama-3-8B, Llama-3.1-8B and Llama-2-7B, three calibration slices each (60 fine-tuned models). Two-bit mean accuracy before → after, mean over the three slices, and CurveTQ's lead after fine-tuning (paired bootstrap over tasks pooling the slices):

ModelCurveTQProteus+startQTIP+startCurveTQ − Proteus+startCurveTQ − QTIP+start
Qwen3-4B51.9 → 53.750.5 → 52.649.4 → 51.4+1.11 [+0.54, +1.72]+2.31 [+1.53, +3.09]
Llama-3-8B52.8 → 54.651.9 → 54.051.1 → 53.6+0.65 [+0.15, +1.27]+1.01 [+0.42, +1.73]
Llama-3.1-8B53.7 → 55.853.1 → 55.351.6 → 55.0+0.52 [−0.28, +1.34]+0.88 [−0.09, +1.92]
Llama-2-7B*63.0 → 64.263.0 → 63.963.0 → 64.0+0.29 (tie)+0.23 (tie)

*Four zero-shot tasks, as in Table 1.

Every method gains and the baselines gain more (on Qwen3-4B their perplexity falls by 2.2–3.9, ours by 0.9), so the lead narrows but the ordering survives: CurveTQ remains the most accurate on all four models and has the lowest perplexity on three (Qwen3-4B 10.80 against 11.06 and 11.26; a tie with Proteus+start on Llama-2-7B at 5.98). Its lead over Proteus+start is distinguishable from zero on Qwen3-4B and Llama-3-8B, no longer on Llama-3.1-8B, and Llama-2-7B is a tie. This is the same stage applied to every method, not each baseline's own pipeline. The revision adds it as an appendix and states in Section 4.1 that the two-bit margin narrows under fine-tuning; we are grateful for the suggestion, which makes the comparison considerably more complete.

W3 — presentation. We thank the reviewer for this candid and helpful feedback, and we agree: the submission used several symbols, including wrw_r, DD and Uˉ\bar U, before defining them, and asked too much of the reader; the examples the reviewer gave show exactly where the text falls short. We have rewritten for readability rather than added material. Concretely: a notation paragraph at the start of Section 3 defining wrw_r (row rr of WW, one output channel, with error er=wr−w^re_r = w_r - \hat w_r) and the LDL factorization H=(I+Uˉ)D(I+Uˉ)⊤H = (I + \bar U) D (I + \bar U)^\top of the damped input Hessian in the coding order (Uˉ\bar U strictly upper triangular; D=diag(D1,…,DC)D = \mathrm{diag}(D_1, \dots, D_C) with Di=1/di2D_i = 1/d_i^2 the LDL conditional residual variance of input coordinate ii given the coordinates coded after it, did_i the corresponding diagonal element of the inverse-Hessian Cholesky factor, so that the Viterbi branch weight is ωi=Di\omega_i = D_i); Section 3.1 opening with what the Viterbi search selects, defining the block quantities DbD_b and ηb\eta_b where they are first used, and split into exact identity, implemented surrogate and empirical result; a compact table of the codec's components (weight, start state, scale field, emission table) beside the schematic of Figure 1, with what each stores, what it costs at decode and which table tests it (the full version, with the Table 4 row that ablates each, is in Appendix H); a short paragraph at the start of Section 4 separating the comparison of complete codecs (Section 4.1) from that of the two searches inside one codec (Section 4.2); and the HiiH_{ii} control and the three-slice replication moved into the main text so that the claim can be read from one place.

AuthorsOpenReview ↗

Response to Reviewer gwNb (3/3): what predicts weighting versus rotation

Q2 — what predicts when native weighting beats or trails the rotation. We thank the reviewer for this question. The short answer is that we do not yet have a reliable way to predict whether native weighting or the rotation is better on a given model, and we do not claim one. Γb\Gamma_b predicts, matrix by matrix, how much weighting gains over the unweighted search (Spearman +0.67 to +0.99, Figure 2); for the remaining gap to the rotation we have no predictor, and the revision says so in Section 3.2. On the six checkpoints of Table 3 the rotated Euclidean search is slightly ahead, by 0.07 to 1.10 points, and on the five where it can be measured, weighting alone recovers 69–94% of the rotation's gain over the unweighted native search.

We should also have made clearer that native weighting is one component of CurveTQ, not the whole method: Table 3 compares the two searches inside the same codec, Table 1 the complete codecs. Inside the codec, weighting is slightly weaker than the rotation; what it buys is a decoder with no transform. What puts CurveTQ ahead of QTIP and Proteus at their published rates, at two bits on all five models of Table 1, are the stored components summarized above, which they do not have: the per-block start state and, for the native basis, the per-module quantile emission table and the factored group scale. Their cost is small: the start state, the largest, takes 0.22 bpw and 1–3% of decode throughput, and with all of them CurveTQ is still the fastest decoder. Our claim is therefore not that weighting beats the rotation, but that it enables a rotation-free codec with competitive or better accuracy and higher decoding throughput.

We thank the reviewer again. The questions led directly to the summary above, the notation paragraph and the matched fine-tuning comparison, and we believe they make the revised paper easier to follow and its claims better supported. We would be glad to clarify anything that remains unclear.

Reviewer_NKcVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper incorporates coordinate-wise sensitivity derived from the Hessian/LDL factorization into the Viterbi branch costs of a trellis. Together with a native-basis quantile table, factored scales, and per-block start states, this enables low-bit weight quantization without inference-time rotations. Its contribution is to turn established ideas in weighted coding into a decodable LLM codec with empirical support for its mechanism and a decoding-speed advantage, while demonstrating that the start-state component also benefits QTIP and Proteus.

Critical strengths
  1. The central mechanism is tested through direct controls. Table 3 on pp. 7–8 compares rotation × weighting at the same format and rate, using a full-dimensional randomized Hadamard transform (RHT). Four calibration slices and the Haar-rotation control in Appendix B support the finding that the additional benefit of weighting becomes smaller after rotation. This is more informative than comparing only final perplexity (PPL).
  2. Rate comparisons are handled carefully. Table 1 on p. 6 includes strengthened baselines such as QTIP/Proteus+start, and Appendix D also compares baselines equipped with both start states and group scales. Appendix J on pp. 34–35 explicitly accounts for approximately 0.219 bits per weight (bpw) of start-state overhead and reports a byte-level serialization/decoding audit. The results cannot simply be attributed to hidden metadata costs.
  3. The paper retains valuable qualifications and negative results. Proposition 2 on p. 15 explicitly rules out a universal substitution claim for arbitrary positive-definite Hessians. In the search ablations on pp. 8–9, PPL and downstream performance rank the variants differently. At three and four bits, performance is close to that of strengthened baselines. These disclosures increase the paper’s credibility.
Critical weaknesses

W1 [Moderate] The interpretation as an LDL-specific mechanism should be narrowed. Locations: p. 4, §3.1; p. 15, Remark 2; p. 17, Appendix B; p. 25, Appendix H. The exact column-wise loss identity does not imply that the implemented block search exactly optimizes the Hessian objective. The implementation omits within-block coupling and uses a column factor that also conditions on later columns within the same block. These approximations are disclosed and are not hidden errors. However, Appendix B reports a difference of only approximately 0.18 downstream points between H_ii and 1/d_i² when feedback is enabled. The evidence more directly establishes that channel-sensitivity weighting is effective than that LDL conditional curvature uniquely explains the gains. Suggested remedy: Distinguish the exact identity, the implemented surrogate, and the empirical substitution result in the main text. Bring the H_ii control into the main text and explain the choice of LDL. If its principal advantage is that it requires no additional statistics, state that directly. There is no need to insist that LDL must outperform H_ii.

W2 [Moderate] The stability of small end-to-end advantages remains insufficiently established. Locations: p. 7, Table 3; pp. 21–23, Appendix E. The task-bootstrap intervals measure sensitivity to the task mix; they do not cover variability from rebuilding the quantized model with a newly sampled calibration set. Appendix E explicitly acknowledges a per-build standard deviation of approximately 0.3–0.6 points, whereas several comparisons with strengthened baselines differ by only approximately 0.2–1 point. For example, Qwen3-4B exceeds Proteus+start by +0.86, with a task-bootstrap interval of [-0.21, +1.88]; Llama-3-8B exceeds Proteus+start+g128 by +0.36, with an interval of [-0.54, +1.26]. These results do not establish a stable advantage in every setting. Suggested remedy: Reuse the existing multiple-calibration-slice protocol for the one or two most consequential comparisons with strengthened baselines and report paired differences. Describe results at three/four bits and on larger models as broadly comparable. The rotation-free mechanism and start-state contributions can remain valid independently of these small margins.

W3 [Moderate] Generalization of the “fastest decoder” claim should be constrained by the implementation conditions. Locations: p. 9, §4.4; pp. 32–33, Appendix I. The batch-1 QTIP numbers for Llama-2-7B come from the original paper, whereas CurveTQ is measured in the present study. Reproducing FP16 throughput alone does not ensure complete comparability of the quantized-kernel timings. The Proteus kernel is a reimplementation based on its description, which the appendix correctly distinguishes from reproducing official performance. Some large-batch differences are small and require information about measurement variability. Suggested remedy: Remeasure the released QTIP kernels on the same machine and software stack, reporting independent timing repetitions and their variability. Identify the Proteus results as those of this reimplementation directly in the main text. The existing +rotation control provides stronger evidence for the practical benefit of removing rotations than a ranking across different implementations does.

Questions for authors

Q1. On which matrices do H_ii and LDL weighting select different paths? Do these differences improve end-to-end quality, or are the two weights effectively equivalent in the current experiments? Q2. Does CurveTQ’s advantage over Proteus+start persist across calibration slices? Please distinguish calibration-set variability from task-bootstrap variability. Q3. Does the speed gap persist when QTIP is remeasured on the same machine? Does the smallest throughput difference exceed the variability across repeated timings?

3 comments
AuthorsOpenReview ↗

Response to Reviewer NKcV (1/3): LDL interpretation

We thank the reviewer for a careful and constructive review. We agree with the three weaknesses, have run the suggested remedies, and give the results and the text changes we have made. Table, figure, section and equation numbers refer to the submitted paper; the tables in this reply are unnumbered.

W1 / Q1 — the LDL interpretation. We agree: the evidence shows that channel-sensitivity weighting in the native basis is effective, not that LDL conditional curvature uniquely explains the gain. The revision says so in Section 3.1, distinguishing (a) the exact identity (Lemma 1, block-factor feedback), (b) the implemented surrogate (the column-wise LDL diagonal Di=1/di2D_i = 1/d_i^2 as an additive Viterbi weight, with column-factor feedback and without modelling within-block off-diagonal coupling) and (c) the empirical substitution result, with the HiiH_{ii} control moved into the main text.

Q1. We re-encoded 256 complete rows of every linear layer of four models with both weights (exact encodes, since the feedback runs row by row; one calibration slice) and rebuilt the HiiH_{ii} arm end to end on four calibration slices for two of them. (k/4): slices on which Δ\Delta has the sign of the mean.

Modelcells where the paths differo_proj / down_projq/k/v/gate/upheld-out layer loss, HiiH_{ii} vs 1/di21/d_i^2end to end: Δ\Delta perplexity (4 slices)Δ\Delta downstream (points)
Qwen3-4B50%66% / 60%36%−2.5%+0.29 ± 0.13 (4/4)+0.18 ± 0.12 (4/4)
Llama-3-8B51%59% / 54%45–49%+0.7%−0.09 ± 0.04 (4/4)−0.21 ± 0.43 (3/4)
Qwen3-8B62%64% / 66%60%−0.1%——
Llama-3.1-8B52%59% / 54%46–50%+0.7%——

The two weights choose different paths on half of the cells, most on o_proj and down_proj (Remark 2: where Di/Hii=1−Ri2D_i/H_{ii} = 1-R_i^2 varies within a block), and the difference does not reach the end-to-end result: the differences are small and point in opposite directions: on Qwen3-4B accuracy favours HiiH_{ii} and perplexity 1/di21/d_i^2 on all four slices, and on Llama-3-8B the directions reverse. The two weights are effectively equivalent in the current experiments; the unweighted search on the same slices is 2.4 (Qwen3-4B) and 21.6 (Llama-3-8B) points behind both. The block-exact diagonal [Db]ii[D_b]_{ii} changes the path on 3–5% of the cells and the end-to-end result by −0.06 ± 0.37 and −0.29 ± 0.17 points, so the choice of diagonal matters little; this does not test the within-block coupling that both diagonals omit. We chose 1/di21/d_i^2 because it comes from the same factorization the feedback already uses and because, at a fixed target, its path has the lower block-exact loss (HiiH_{ii}'s is 1.5–6% higher); both weights are free, and we do not claim that 1/di21/d_i^2 outperforms HiiH_{ii}. The revision also clarifies the difference between the exact block-factor identity and the loss induced by the implemented column-factor feedback, without asserting that the resulting correction is necessarily positive semidefinite, and rewords the corresponding sentence of Appendix H.

AuthorsOpenReview ↗

Response to Reviewer NKcV (2/3): calibration-set variability

W2 / Q2 — calibration-set variability. We agree that the task-bootstrap interval does not cover rebuilding on a new calibration set, and we have run the suggested protocol on every trellis row of Table 1: two further calibration sets B and C (128 sequences of 2048 C4 tokens each, disjoint from the paper's set A), the same command lines, the full suite, 120 new builds. Below, Δ\Delta is the paired difference on the same slice (positive favours CurveTQ), mean ± s.d. over the three slices, (k/3) the slices with CurveTQ ahead; the task-bootstrap column is Appendix E (slice A). We report calibration-build variability separately from the paired task-bootstrap intervals; the latter do not by themselves quantify uncertainty across independently rebuilt calibration sets.

Model (2-bit)task-bootstrap Δ\Delta acc. [95% CI]slices A/B/C: Δ\Delta acc. ± s.d. (k/3)slices A/B/C: Δ\Delta ppl ± s.d. (k/3)
CurveTQ (2.29 bpw) − Proteus+start (2.22)
Qwen3-4B+0.86 [−0.21, +1.88]+1.41 ± 0.50 (3/3)+1.48 ± 0.35 (3/3)
Llama-3-8B+1.17 [+0.09, +2.39]+0.85 ± 0.40 (3/3)+0.07 ± 0.17 (2/3)
Llama-3.1-8B+1.44 [+0.30, +2.76]+0.66 ± 0.67 (3/3)+0.12 ± 0.02 (3/3)
Qwen3-32B+0.23 [−0.37, +0.90]+0.05 ± 0.14 (2/3)−0.01 ± 0.01 (0/3)
Llama-2-7B*—−0.03 ± 0.21 (1/3)−0.06 ± 0.06 (0/3)
CurveTQ (2.29 bpw) − QTIP+start (2.19)
Qwen3-4B+2.03 [+1.14, +2.97]+2.48 ± 0.88 (3/3)+1.77 ± 0.72 (3/3)
Llama-3-8B+2.17 [+0.99, +3.68]+1.65 ± 0.80 (3/3)+0.29 ± 0.03 (3/3)
Llama-3.1-8B+3.10 [+1.39, +5.16]+2.08 ± 0.97 (3/3)+0.30 ± 0.09 (3/3)
Qwen3-32B+0.23 [−0.52, +0.96]+0.41 ± 0.19 (3/3)+0.04 ± 0.04 (3/3)
Llama-2-7B*—−0.02 ± 0.55 (1/3)+0.07 ± 0.03 (3/3)

*Four zero-shot tasks, as in Table 1; no task-bootstrap interval is reported for them.

Q2. The margin over Proteus+start persists on the three 4–8B models: positive on all nine (model, slice) pairs, and pooled over the three slices with a task bootstrap +1.41 [+0.95, +1.88] on Qwen3-4B, +0.85 [+0.28, +1.62] on Llama-3-8B and +0.66 [+0.02, +1.33] on Llama-3.1-8B. On Qwen3-32B and Llama-2-7B the methods are tied, and perplexity is a tie everywhere but Qwen3-4B. At three bits every contrast against Proteus+start is within ±0.4 points and 0.07 perplexity, confirming "draw level". The +start+g128 rows were not rebuilt; the Llama-3-8B contrast there (+0.36) is within noise, and the revision describes it so (Appendix D). The revision adds the three-slice table to Appendix E and states the two-bit result as a 0.7–1.4-point downstream margin on the 4–8B models that reproduces across calibration sets, with three and four bits and the 32B model described as comparable.

AuthorsOpenReview ↗

Response to Reviewer NKcV (3/3): decode throughput

W3 / Q3 — decode throughput. We thank the reviewer for this point and agree. The figures in Tables 5 and I.1 were already means over three independent runs (each run's figure the median over decoding steps, as Appendix I says), but the submission reported neither the spread nor, for Llama-2-7B at batch 1, our own measurement of QTIP, whose published numbers we quoted there. We now report the mean ± s.d. over the three runs for every configuration, and have measured QTIP on our machine for those cells (tokens/s; one RTX 3090, QTIP's CUDA-graph decode setting; QTIP runs its released kernels with fusion added, which reproduce its published numbers to within 1.5%; Proteus is our implementation of the published description):

ModelbitsQTIPProteusCurveTQCurveTQ/QTIPCurveTQ/Proteus
batch 1
Llama-2-7B2126.62 ± 0.33105.90 ± 0.19139.03 ± 0.221.098×1.313×
3119.02 ± 0.37103.47 ± 0.11125.67 ± 0.511.056×1.215×
4110.61 ± 0.26102.58 ± 0.21114.91 ± 0.401.039×1.120×
Qwen3-4B2153.61 ± 0.89117.94 ± 0.05160.78 ± 0.721.047×1.363×
3144.20 ± 0.63115.43 ± 0.31151.65 ± 1.921.052×1.314×
4137.21 ± 0.10113.23 ± 0.26146.19 ± 0.251.065×1.291×
batch 64
Llama-2-7B21011.5 ± 1.41038.0 ± 0.61081.6 ± 1.71.069×1.042×
3999.4 ± 1.71024.7 ± 0.21071.3 ± 2.81.072×1.045×
4985.4 ± 3.61011.3 ± 0.21063.1 ± 1.81.079×1.051×
Qwen3-4B21006.9 ± 0.91011.7 ± 3.71020.9 ± 0.81.014×1.009×
31002.3 ± 2.31002.9 ± 0.61015.5 ± 1.41.013×1.013×
4997.3 ± 1.61000.4 ± 1.71012.1 ± 0.31.015×1.012×
batch 128
Llama-2-7B21714.5 ± 2.81830.2 ± 2.81927.2 ± 5.51.124×1.053×
31702.5 ± 2.21805.3 ± 1.01916.5 ± 1.01.126×1.062×
41689.8 ± 2.11781.9 ± 4.31907.1 ± 3.31.129×1.070×
Qwen3-4B22185.8 ± 0.92227.2 ± 6.42267.8 ± 2.61.038×1.018×
32178.1 ± 4.62212.4 ± 2.92246.4 ± 1.71.031×1.015×
42162.8 ± 4.52193.8 ± 5.52236.1 ± 3.21.034×1.019×

Q3. The gap persists: with QTIP measured here, Llama-2-7B at batch 1 is 1.04–1.10× QTIP (1.05–1.09× with the published numbers), and CurveTQ is the fastest in all 18 configurations. The closest comparison relative to its spread, Qwen3-4B at batch 64 and two bits against Proteus (+9.2 tokens/s, 0.9%), is 2.4 times the two standard deviations combined. The revision adds the standard deviations to Tables 5 and I.1, replaces the quoted QTIP cells by our measurements, states the protocol in Appendix I, identifies Proteus as our implementation in Section 4.4, and presents the within-implementation rotation control (adding the Hadamard to our own decoder costs 4–11% at batch 1) as the primary evidence for the cost of the transform.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: our contributions and what the revision adds

General response: our contributions and what the revision adds

We thank the AC and all three reviewers for their time and constructive feedback. For convenience, we summarize our contributions and what the revision adds. All changes are marked in blue in the revised PDF.

1. Contributions. QTIP and Proteus, the strongest prior two-bit trellis quantizers, rotate every weight matrix and must undo the rotation at every decoding step. Replacing this rotation by a curvature weight in the trellis search is one of our four contributions. By itself it does not raise accuracy; it does two things. First, speed: the decoder has no transform to undo. Second, room for accuracy: it raises the accuracy ceiling, which (b) and (c) below then reach. The other three contributions are: (a) a stored start state per coding block, worth 1–3 points of accuracy; (b) a per-layer table that fits each layer's own weights; (c) one scale per 64 weights. (b) and (c) turn the room the weight creates into accuracy. So CurveTQ is faster because of the weight and more accurate because of (a)–(c).

2. What the revision adds: every part of the evidence is now stronger.

  • Three calibration sets (Table 1, Appendix E). Every trellis row of Table 1 was rebuilt on two more disjoint calibration sets, 120 new builds. On the three 4–8B models, CurveTQ leads on every calibration set: by 2.8–4.8 points over QTIP and Proteus, and by 0.7–2.5 points over the same codecs given our start state.
  • Fine-tuning (Section 4.1, Appendix K). We applied one common fine-tuning stage, QTIP's own recipe, to all five trellis methods on four models and three calibration sets, 60 fine-tuned models. CurveTQ remains the most accurate method on all four models. On the three 4–8B models it leads QTIP and Proteus by 2.1–4.0 points, every interval excluding zero. Even when the baselines are given our contribution (a), the start state, CurveTQ still leads them by about 1 point on average.
  • Six models in the rotation-versus-weight test (Table 3). We added three more models to this test, 48 new builds. The conclusion holds on all six: the weight can replace the rotation.
  • Decoding speed (Table 5, Figure 4, Appendix I). Every QTIP number is now our own measurement of its released kernels, over three runs with standard deviations, and a new figure plots accuracy against speed. CurveTQ is the fastest of the three in all 18 settings: 1.04–1.10× QTIP and 1.12–1.36× Proteus at batch 1. On Qwen3-4B, its 3-bit model runs as fast as QTIP's 2-bit model and is 13 points more accurate.

Smaller changes: a notation paragraph and a component table make Section 3 easier to follow; the H_ii control is moved into the main text and extended to a second model; the margins are stated more precisely in the abstract and Section 4.

3. One question we want to make easy to assess. A question raised in the discussion helped us sharpen the paper: since the weighted search only matches the rotation in accuracy, and the decode gain over QTIP is 4–10%, what is the contribution of the search? Our answer is that the weight is the enabler rather than the payoff: it removes the rotation at no cost in accuracy, bits or decode work, and the rotation-free codec it permits is both more accurate and faster than the rotated ones (1.04–1.10× QTIP and 1.12–1.36× Proteus at batch 1). For scale, the two previous codecs in this line reported speed parity with their predecessors (QTIP: 1.01× QuIP#, 188 vs 186 tokens/s; Proteus: "comparable throughput", 1.01× QTIP at batch 1). We are grateful for the question and would welcome any further guidance on what would make this case clearer.

We thank all reviewers again; their questions made the paper considerably stronger.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.