- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks what the random rotation used by state-of-the-art two-bit trellis quantizers actually accomplishes, and argues it does three separable things: it evens out local weight amplitude, it makes the weight marginal Gaussian, and it removes within-block variation in a curvature weight that error feedback already computes but that existing coders never read inside the search. Placing that weight in the Viterbi branch metric is argued to substitute for the third effect, while the first two are handled directly by a factored per-group scale field and a per-layer quantile table, yielding CurveTQ, a trellis codec that applies no transform at decode. The paper also replaces the baselines' tail-biting block termination with a block-local trellis whose start state is stored, and transfers that component to the baselines. Validation covers six checkpoints including a mixture of experts, a rotation-by-metric factorial, component and termination ablations, and a fused CUDA decoder measured against released baseline kernels.
Critical strengths
S1. The paper reframes a component everyone uses rather than adding another codec.
Treating the random rotation as three separable jobs, and then asking which of them could be taken over by a quantity the solver already computes, is more informative than a further comparison between codes. The supporting observation — that existing trellis and lattice quantizers derive this curvature quantity in order to run error feedback and then discard it inside the search — is genuine, and it is actionable independently of whether the substitution claim holds in full.
S2. The rotation-free claim is carried through to a measured decode result, with the gain correctly attributed.
The authors implement a fused dequantize-and-multiply kernel and report it as the fastest of the three codecs at every batch size and bit width tested. They then add the baselines' transform back into their own kernel, so the throughput gain is attributed to the transform a rotation-free decoder does not pay for rather than to implementation effort the baselines did not receive. This is where the paper's claim and its evidence line up most cleanly. The reported rate is likewise charged for every stored quantity, including the side information that carries most of the margin.
Critical weaknesses
W1. The experiment that tests the paper's central claim is the thinnest part of the evaluation, and one contrast within it runs the other way.
The rotation-by-metric factorial is the only direct test of the substitution claim on which the title and abstract rest. It is run on three checkpoints with a small number of calibration slices per cell, and on one of them the unrotated Euclidean cell is reported as collapsing across the task suite, so every contrast involving it is omitted — leaving two checkpoints on which the full design can be read. Within that design, the contrast asking whether rotating still helps once the curvature weight is in place is positive and consistent across slices on two of the three models, with the quantile table and the scale field already present in every cell. A claim of this prominence would be better supported on more checkpoints, and on an account of what the rotation still contributes once all three of the paper's substitutes are in place.
W2. The comparison frame does not support the strength of the claimed margin.
The proposed build is charged a higher rate than every baseline it is compared against, and the strongest baseline — the same codec given the paper's own start-state component, at a lower rate — is better on perplexity on most of the models tested. The stated downstream margin holds on a subset of the models and does not appear at the higher bit widths also reported. Separately, no method is fine-tuned anywhere, while the extended related work converts the value of a post-quantization fine-tuning stage into effective bits and places it above the spread between any two codes compared here; whether the ordering survives that stage is deferred. The margins are therefore established in a regime that the paper itself identifies as not the one the baselines' published numbers come from.
Questions for authors
-
Could the factorial be run on more checkpoints, including ones where the unrotated Euclidean cell does not collapse? As reported, the full design is readable on two models.
-
Could any comparison be reported at matched rate, either by raising the baselines to your rate or reducing yours to theirs? The strongest baseline currently sits below your rate and above your build on perplexity on most models.
4 comments
Response to Reviewer RFpF (2/2): accuracy over three calibration sets, and fine-tuning
W2 / Q2 — the comparison frame (continued).
Perplexity and downstream accuracy. On perplexity we agree. We have rebuilt every trellis row of Table 1 on two further calibration sets B and C (128 sequences of 2048 C4 tokens each, disjoint from the paper's set A, same command lines, full suite); against the two start-state baselines over the three slices (paired on the same slice, positive favours CurveTQ; mean ± s.d., with the slices on which CurveTQ is ahead):
| 2-bit, CurveTQ (2.29 bpw) vs | Proteus+start (2.22): perplexity | Proteus+start: downstream | QTIP+start (2.19): perplexity | QTIP+start: downstream |
|---|---|---|---|---|
| Qwen3-4B | +1.48 ± 0.35 (3/3) | +1.41 ± 0.50 (3/3) | +1.77 ± 0.72 (3/3) | +2.48 ± 0.88 (3/3) |
| Llama-3-8B | +0.07 ± 0.17 (2/3) | +0.85 ± 0.40 (3/3) | +0.29 ± 0.03 (3/3) | +1.65 ± 0.80 (3/3) |
| Llama-3.1-8B | +0.12 ± 0.02 (3/3) | +0.66 ± 0.67 (3/3) | +0.30 ± 0.09 (3/3) | +2.08 ± 0.97 (3/3) |
| Qwen3-32B | −0.01 ± 0.01 (0/3) | +0.05 ± 0.14 (2/3) | +0.04 ± 0.04 (3/3) | +0.41 ± 0.19 (3/3) |
| Llama-2-7B* | −0.06 ± 0.06 (0/3) | −0.03 ± 0.21 (1/3) | +0.07 ± 0.03 (3/3) | −0.02 ± 0.55 (1/3) |
*Four zero-shot tasks, as in Table 1.
CurveTQ and Proteus+start are at perplexity parity on four of the five models and CurveTQ is 1.5 lower on Qwen3-4B; the margin is downstream: CurveTQ is ahead on every slice of the three 4–8B models, by 0.7–1.4 points on average (1.7–2.5 over QTIP+start), and a tie on Qwen3-32B and Llama-2-7B. Against QTIP+start CurveTQ has the lower perplexity on every slice of all five models. The revision states the two-bit claim in this form in the abstract and Section 4.1. In the revision these two-bit differences form a new bottom panel of Table 1, and the full three-slice results, every method at two and three bits, are in Tables E.3 and E.4.
Higher bit widths. At three bits the +start baselines draw level, as Section 4.1 says: over the three slices every contrast against Proteus+start is within ±0.4 points and 0.07 perplexity (+0.4 to +0.8 over QTIP+start), and at four bits every trellis codec is within a point. The two-bit margin is the only one we claim, and the revision words it so.
Fine-tuning. We have since applied one post-quantization stage to every method: QTIP's own end-to-end stage with its published recipe (per-channel scales and the unquantized parameters trained, trellis indices frozen, 512 sequences of 4096 tokens, 4 epochs), on Qwen3-4B and the three Llama models, three calibration slices each, 60 fine-tuned models. Every method gains and the baselines gain more (Qwen3-4B: +1.9 points for CurveTQ, +2.1 to +2.8 for the baselines; their perplexity falls by 2.2–3.9, ours by 0.9), so the lead narrows but the ordering survives: after fine-tuning CurveTQ has the highest accuracy on all four models and the lowest perplexity on three (a tie on Llama-2-7B). Over Proteus+start, pooled over the three slices:
| CurveTQ − Proteus+start, points [95% CI over tasks] | before fine-tuning | after fine-tuning |
|---|---|---|
| Qwen3-4B | +1.41 [+0.95, +1.88] | +1.11 [+0.54, +1.72] |
| Llama-3-8B | +0.85 [+0.28, +1.62] | +0.65 [+0.15, +1.27] |
| Llama-3.1-8B | +0.66 [+0.02, +1.33] | +0.52 [−0.28, +1.34] |
| Llama-2-7B | −0.03 (tie) | +0.29 (tie) |
The published QTIP and Proteus numbers include their fine-tuning stages; Table 1 compares every method without one, and this experiment with the same one for all. The revision adds it as Appendix K, with all five methods before and after fine-tuning and CurveTQ's lead over each (Tables K.1–K.4), and says in Section 4.1 that the margin narrows under fine-tuning and is no longer distinguishable from zero on Llama-3.1-8B.
Thank you for your response.
After careful consideration, I decided to maintain my score. I remain open to the discussions and decisions of other reviewers.
Thank you for reading our responses and for the careful review. Your questions led directly to the three new checkpoints in Table 3, the replication over three calibration sets and the common fine-tuning stage, which together make the paper's evidence considerably stronger. We appreciate your positive assessment of the paper.
Response to Reviewer RFpF (1/2): the factorial on six checkpoints, and rate
We thank the reviewer for the careful reading and for stating both concerns precisely. We agree with both. We have run the factorial on three more checkpoints (W1/Q1) and answer the comparison frame with the rate accounting, a three-slice replication, the higher bit widths and a uniform fine-tuning stage (W2/Q2). Table, figure, section and equation numbers refer to the submitted paper; the tables in this reply are unnumbered.
W1 / Q1 — the factorial on more checkpoints. We ran the 2×2 design of Table 3 (full-dimension RHT × {Euclidean, 1/di2}, same format and rate, four disjoint calibration slices, paired contrasts) on three more checkpoints: Llama-3.1-8B (14-task suite), Qwen3-8B (34-task suite) and Mistral-7B (the four zero-shot tasks of Table 1's Llama-2-7B column); 48 builds. Contrasts as in Table 3, mean accuracy points: P1/S1 = weighted − Euclidean, native/rotated; R-e/R-w = rotated − native, Euclidean/weighted; SUB = rotated Euclidean − native weighted. Mean ± s.d. over the four slices; (k): slices of four on which the contrast is positive.
The full design is now readable on five checkpoints. The native Euclidean cell does not collapse on Qwen3-8B and Mistral-7B, where it is 2.8 and 3.5 points behind on average, and is 13–22 points per slice behind on Llama-3.1-8B (perplexity 10.4–12.9 against 9.6), the Llama-3 fragility again. The three findings of Section 4.2 hold on each new checkpoint: the weight gains in the native basis (P1 positive on all 12 slice comparisons), adds nothing after the rotation (S1 within ±0.7), and the native weighted search is within about a point of the rotated Euclidean one (SUB +0.7 to +1.0). R-w, the contrast the reviewer asks about, is +0.3 to +0.8 points on the three new checkpoints, positive on two or three slices of four, and moves perplexity by less than 0.2 on all three. With the weight, the scale field and the table in place, what the rotation still adds is small but not zero: the mean R-w contrast ranges from −0.06 to +1.46 points across the six checkpoints and is below one point on five of them. The factorial does not identify what this remainder is; it may be within-block Hessian coupling that a diagonal weight does not capture, or an interaction with the transformed weight distribution. The revision adds the three checkpoints to Table 3 and states the contribution in this form rather than as zero.
W2 / Q2 — the comparison frame.
Rate. Every bits-per-weight figure counts every stored quantity (Appendix J): 2.287 = 2.000 code + 0.069 scale field and table + 0.219 start state, against 2.222 for Proteus+start and 2.190 for QTIP+start. The whole gap to Proteus+start is the per-(row, 64-group) scale field. The baselines raised above our rate are in Appendix D: the +start+g128 rows (2.31 / 2.34 bpw; single calibration slice) give them our start state and a 128-group scale, which moves the +start rows by −0.7 to +0.8 points; against them CurveTQ is +1.26 [+0.29, +2.22] (Qwen3-4B), +0.36 [−0.54, +1.26] (Llama-3-8B) and +1.02 [+0.13, +1.94] (Llama-3.1-8B) over Proteus+start+g128, and +2.6 to +2.7 with intervals excluding zero over QTIP+start+g128 (Table E.1). Even with the baselines raised above our rate, CurveTQ is ahead in all six contrasts, five of them with intervals excluding zero. A rotated code has little use for a group scale because the rotation has already evened out local amplitude; in the native basis the scale is what keeps a two-bit grid from collapsing on outlier channels (Appendix C). The 0.069 bpw is the price of doing that job as side information rather than as a decode-time transform, which is charged in time instead: adding the transform to our decoder costs 4–11% of batch-1 throughput, against 3–4.5% more weight bytes for our rate (Section 4.4).