acceptodds
Under review as a conference paper at ICLR 2027

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

Abstract

Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy. Ternary weights have mostly been obtained by training from scratch or long fine-tuning. We introduce RoleQ, a quantization framework that places scales by the functional roles of Transformer units rather than by the layout of the weight matrix, as conventional formats do. Vectors that read from the residual stream take one scale per output unit; those that write to it, one per input unit. The rule is structural, needs no search, calibration or stored state, and holds on integer and ternary grids. In a rotated basis it matches a per-layer axis search at half the solver work, and at ternary it cuts perplexity by up to half against layout-based placement in two families at every size from 0.6B to 14B, where per-row scales leave MMLU at chance. Our lightweight post-training pipeline converts an existing checkpoint to 1.59 bits per weight in a few GPU-hours, a fraction of the cost of learned ternarization, and leads every post-training ternary method at matched or lower memory-resident bits on perplexity, zero-shot accuracy and MMLU, most widely on knowledge. A sparse second ternary plane placed by a per-unit Fisher saliency keeps the matmul multiplication-free and recovers what ternary loses first: at 8B, a fraction of a bit returns about half the MMLU that conversion costs, well ahead of random or Hessian-based placement at equal bits.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training quantization, ternary quantization, low-bit large language models, weight-only quantization, model compression, efficient inference, quantization scales, rotation-based quantization, GPTQ, sub-2-bit

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.60 over 5 reviews, above 70% of the others.
Reviewer_B8BYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RoleQ, which places quantization scales according to reader/writer roles rather than using a uniform axis. The conversion pipeline combines this rule with learned rotations, ternary GPTQ, and scale distillation. An optional second ternary plane is assigned to units selected by Fisher-weighted quantization error.

Critical strengths

The scale-placement rule is simple, and the conversion results are promising. In Table 1, on Qwen3-8B-Base before distillation, RoleQ-T with secondary planes achieves 56.04 MMLU at 2.157 bpw, compared with 34.82 at 2.281 bpw for QTEA under the paper's accounting.

Critical weaknesses
  1. The novelty is limited. As acknowledged in Section 2, the rotation setup follows QuaRot and SpinQuant, the l4 objective comes from OptRot, and the Fisher allocator follows FGMP. The main additions are therefore the role-based scale rule and functional-unit allocation. The new rotation objective also shows no clear advantage over the existing alternative: in Table 13(b), s2(opt) and s2(l4) achieve nearly identical perplexity after distillation (20.26 vs. 20.27), while l4 performs better before distillation.

  2. The contribution of role-based placement to the final results remains unclear. Table 2 isolates placement under random rotation without distillation, whereas the main results use learned rotations and distillation. Table 9 lacks a matched per-OC control at approximately 1.59 bpw under the full pipeline. With grouped scales, per-OC and RoleQ are already nearly tied after distillation (51.33 vs. 50.89 MMLU). Moreover, Table 8 only compares RoleQ with AdaDim in local reconstruction error. The claimed halving of solver work is relative to a two-way GPTQ search, not AdaDim's RTN-based selection followed by one GPTQ pass. The practical advantage over AdaDim is therefore not established.

Questions for authors
  1. Could the authors compare RoleQ with per-OC placement on Qwen3-8B-Base at approximately 1.59 bpw, using identical learned rotations and the same distillation budget?

  2. How does RoleQ compare with AdaDim's original axis-selection procedure in final accuracy and measured conversion time?

  3. With the Fisher score and total bit budget fixed, does allocating secondary planes to functional units outperform allocation to ordinary fixed blocks?

  4. Why choose s2(opt) when s2(l4) matches it after distillation and performs better before distillation?

2 comments

W1 (novelty; s2(opt) vs s2(l4)). Our contributions are the role-based placement rule in a rotated basis, an exact per-vector ternary step within GPTQ, and secondary-plane allocation using recall-conditioned Fisher saliency. The reader refit is adopted from Z-FOLD. The first two are linked: because the rule puts a writer's scale on the axis GPTQ sweeps, each unit gets its own grid, fitted jointly with its codes on the compensated vector, which a per-OC scale does not allow. In Table 13's solver control that fit is worth 1.25× on top of the axis's own 1.23× (1.7B, PTQ). Section 3 identifies the rotation slots and objectives as existing components. Opt directly minimises ternary reconstruction error; l4 is a kurtosis surrogate. We use l4 in s1 because it gives better model quality. In s2, opt and l4 give nearly identical perplexity after SD (20.26 vs 20.27 at 1.7B), and the PTQ difference (70.86 vs 73.66) is within seed variation. Both are data-free at the same cost, so we retain the direct objective where model quality is comparable.

W2 / Q1 (per-OC at 1.59 bpw under the full pipeline). Common response Section 2. The per-OC control under the full recipe with SD at 8B, two seeds, reads 28.53 / 50.33 / 33.40 → 11.48 / 59.94 / 48.91 against the role axis's 22.42 / 50.39 / 33.32 → 11.33 / 60.33 / 47.59. The g128 pair you cite (51.33 vs 50.89) is consistent with a small post-SD effect at 8B: the task differences are small relative to seed variation and perplexity differs by 1-2%. At PTQ, per-OC perplexity is 27% higher; at 14B the advantage persists after SD on all three measures. We revised Section 4.2 to reflect this distinction. The rule also requires no axis search or stored flags, and we saw no measurable runtime overhead in the 8B comparison. Planes naturally refine each unit's fitted grid. Per-OC writer scales span units and do not support that extension; this control therefore concerns one-plane models.

Q2 (AdaDim end to end, with conversion time). We ran AdaDim as published at 1.7B (two seeds, r1 and recipe bases) and 8B (recipe basis). Rotations reduce the outlier effects that make low-bit quantization difficult. At 8B it puts 89% of the matrices on the role axis and reads 23.16 → 11.32 WikiText2, 61.0 zero-shot and 49.3 MMLU after SD against RoleQ-T's 22.20 → 11.27, 60.1 and 44.9 (seed 45), inside the seed ranges on every instrument. The selection takes 1.6 s against 102-124 s of GPTQ kernels per model at 1.7B (1-2%) and 12 s against 325 s at 8B (3.7%), so the conversion-time claim is replaced by this measurement. In the recipe basis at 1.7B the search picks the role axis for 91-92% of the matrices and reads 72.8 / 86.5 PTQ, 20.3 / 20.9 after SD against RoleQ-T's 73.7 / 72.6 and 20.3 / 20.7 (WikiText2); in r1, 94-95% agreement and a tie. Without a rotation it picks the opposite axis on 91% of the matrices and is 2.3× better at PTQ (554 vs 1,246).

Q3 (units vs fixed blocks). With blocks the size of a unit vector, reader blocks coincide with rows. For writers (W_O, W_out), units are columns, whereas row-wise blocks span units. Unit-aligned planes naturally refine individually fitted vector grids. Two measurements support this choice. First, the one-plane solver control separates the axis effect (1.23×) from the exact per-unit fit (a further 1.25×; Table 13, 1.7B PTQ). These are not measurements of the secondary-plane block variant. Second, Fisher scores are more concentrated and stable across units than across equal-sized blocks at 1.7B: the top 30% of W_O units hold 80% of the score, versus 63% for blocks; within W_out layers the medians are 52% and 44%. Split-half rank correlation is 0.97-0.98 for units and 0.89-0.92 for blocks.

Q4 (why s2(opt)). Answered under W1.

Reviewer_B8BYOpenReview ↗

Response to Authors

Thank you for the new experiments and the revised claims. However, they confirm my main concern. Under the full pipeline at 8B, RoleQ and per-OC tie after SD (per-OC is even ahead on MMLU), and RoleQ ties AdaDim end to end, where the axis search costs only 1–4% of GPTQ time. The gains in Table 1 therefore appear to come mainly from rotation, SD and secondary planes rather than from the placement rule. I could not find the table supporting the 14B post-SD claim. Q3 (units vs. fixed blocks at a fixed budget) remains unanswered. I keep my score.

Reviewer_y68gOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new design principle of scale assignment for quantized linear layers in the attention block. It assigns quantization scale depending on the role of the linear layer, resulting in row-wise assignment for the layers that read from the residual stream, and column-wise for the layers that write to the residual stream. It introduces adaptations of the existing methods such as GPTQ, Scale Distillation, and Fisher-Placed Planes to the new principle. The paper validates the design empirically in multiple configurations, mostly outperforming the baseline algorithms.

Critical strengths
  • The paper presents an interesting principle of scale assignment, which considers the layer’s function to determine which dimension should be scaled.
  • Authors show that existing compression methods can be adapted to this principle, including GPTQ, rotations, scale distillation and fisher-placed planes
  • Authors conduct an extensive empirical analysis proving the benefit of the principle in practice for various configurations. They also provide an extensive ablation study in the appendix.
Critical weaknesses
  • The intuition behind the method is hard to understand. The authors empirically show that this scale assignment strategy works, but haven’t formally shown why it would improve the performance of the model.
  • Authors mention that “RoleQ matches AdaDim’s reconstruction quality at half the solver work of a two-way search” as the first contribution, but don’t provide any AdaDim results in the main text. Scale distillation is presented in the contributions list. However, the topic of scale optimization is not covered in the related work section.
  • The readability of the paper is poor
    • Some paragraphs are hard to read. For example, paragraph 3 starts with the following sentence: “Our central insight is that quantization should allocate bits by function”. The next sentence talks about assigning scales. I don’t see any connection between bit allocation and scale assignment. For me, these are two different problems, bit allocation questions, how many bits to assign, scale assignment questions which scale to choose for the quantized values.
    • It’s not clear what is the focus of the paper. Apart from scale selection, the authors consider GPTQ, rotations, scale distillation and Fisher planes. All of them are presented with some modifications, some of which the authors explain in more detail than the others. Moreover, it’s not clear for the reader that the list of modifications is comprehensive.
    • Some paragraphs can be broken down into multiple. In Section 3.2, this one paragraph communicates 4-5 separate ideas. I can say the same about Section 3.4.
    • Abbreviations and very narrow terms are not defined, yet used. Per-OC, per-IC, QK and OV circuits - should be defined before being used. Providing a reference to related work is not enough.
Questions for authors
  • RoleQ-T combines several methods such as rotations, gptq, scale distillation and Fisher planes. What is the individual impact of each method? How much does RoleQ lose in accuracy if we keep removing the methods from RoleQ in the order of their presentation, assuming they are orthogonal?
  • What is the formal intuition behind the principle? Why does it work in theory?
2 comments

Q1 (the individual impact of each component, removed in order). The components are not orthogonal, so we report the cumulative ladder in the order of presentation, each rung the previous one plus one component, at 8B (seed 45, Table 1 protocol) and at 1.7B (two seeds). 8B-Base, WT2 / zs7 / MMLU:

rungPTQ+ SD
GPTQ ternary, per-OC, no rotation724.1 / 39.64 / 24.5327.27 / 46.34 / 28.73
+ the placement rule, no rotation205.6 / 40.16 / 25.3222.98 / 50.84 / 32.00
+ r1 (Hadamard)31.00 / 49.45 / 32.6012.04 / 59.75 / 45.01
+ the recipe rotation (s1 l4, s2 opt, r4) = RoleQ-T22.20 / 50.30 / 31.8411.27 / 60.06 / 44.87
+ Fisher planes (2.157 bpw)11.32 / 59.51 / 56.049.30 / 65.89 / 62.75

(The r1 row's task columns are Table 9's seed-45 cells; the full-recipe per-OC control is in common response Section 2.) At 1.7B the same ladder on WikiText2 is 1245.9 → 119.8 → 73.7 at PTQ and 67.5 → 23.7 → 20.3 after SD for none → r1 → recipe (Table 13). Rotation and SD account for large gains in this ladder. Placement has a large effect without rotation (3.5× at PTQ and 19% after SD at 8B), but a small effect after SD in the recipe basis at 8B (1.4%); the difference is clearer again at 14B. Adding planes gives the largest MMLU gain among the later rungs.

Q2 / W1 (formal intuition). Aligning writer scales with the GPTQ sweep allows each unit's grid to be fitted jointly and exactly with its codes when the sweep reaches that unit, after compensation for earlier rounding errors. Per-OC grids span columns and must be fixed before the sweep. Table 13's solver control (1.7B, PTQ) separates these effects: with the solver held fixed, changing the axis reduces perplexity by a factor of 1.23 (138.6 → 112.8); the exact fit adds a factor of 1.25 (→ 90.0). The axis effect itself is supported by ablations, not a derivation. Mixing the unit axis with r4 reduces the writer-side advantage from 1.63× to 1.17× (Table 13d). AdaDim's criterion agrees with the rule on most matrices in rotated bases, but favours per-input-channel scales for readers without rotation and achieves 2.3× lower perplexity at 1.7B. We therefore state the rule for rotated bases, where rotations reduce outlier effects in low-bit quantization. Absorption of gain errors by downstream norms does not explain the gap: the gain component accounts for 13-17% of unit error under both placements (Section 3.1).

W2 (AdaDim in the main text; scale optimization in related work). Rotations reduce outlier effects in low-bit quantization, so we evaluate axis selection in these bases. The 96.4% / 14.3% agreement moves into the main text with the end-to-end AdaDim rows: at 1.7B the search agrees with the rule on 91-95% of the matrices in rotated bases and ties after SD, at 1-4% of the GPTQ time; at 8B it puts 89% on the role axis and reads 23.16 → 11.32 against 22.20 → 11.27, task columns inside the seed ranges. "Half the solver work" is replaced by that measurement. Related Work now covers scale optimization (AWQ, SqueezeLLM, EfficientQAT, PV-Tuning and QuIP#-style fine-tuning), and describes SD as restricting updates to scales and norm gains with codes frozen.

W3 (readability). We define per-OC, per-IC, QK and OV at first use. The introduction now explains scale placement directly and separates it from secondary-plane allocation. We split the mechanism discussion into solver behavior and empirical evidence, shorten the contribution list, and leave detailed numerical comparisons to the tables. Section 3 distinguishes our placement rule, exact ternary step and Fisher-guided allocation from the adopted rotation setup, GPTQ, Z-FOLD reader refit, KL distillation and FGMP allocator.

I thank the authors for addressing my concern.

After reading the updated manuscript, I can confirm that readability of the paper has improved. My question regarding the effect of each component and AdaDim have also been successfully addressed.

The intuition of the method itself is still not trivial. I thank the authors for the explanation they gave, and I think I have a better understanding of it now, but I believe there should be a better and more direct way to show why this idea works. However, it feels more like an empirical phenomenon, and I understand the challenge behind formalizing it.

Overall, I am increasing my score for this paper to 3

Reviewer_NBNWOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RoleQ assigns scales by Transformer vector role: per output unit for readers and per input unit for writers. RoleQ-T combines this rule with rotations on ternary GPTQ, optional scale distillation, and Fisher-selected secondary planes.

Critical strengths

The scale-placement rule is simple and interpretable, with useful controlled experiments in a rotated basis.

Critical weaknesses
  1. Section 3.5 uses MMLU-style and ARC items for the knowledge Fisher without specifying enough split/item information to verify separation from final benchmark tests;
  2. Table 3 further lacks a matched-budget text-Fisher control for the post-trained 8B comparison, leaving the source of the knowledge-specific gain insufficiently isolated.
  3. Appendix A counts log2(L) information bits, excludes packing overhead, and reports the quantized body separately from FP16 embeddings/head. Calling this rate “memory-resident bpw” can misrepresent actual storage.
  4. Table 8 explicitly acknowledges that “half the solver work” refers to two-way GPTQ search, whereas AdaDim selects axes using RTN before one GPTQ pass. The local reconstruction comparison is informative, but does not establish half the end-to-end cost of original AdaDim. These disclosed limitations leave the practical storage and conversion-cost advantage less established than the quality results.
  5. More harder evaluation tasks are required (just like previous works: math, code etc). It is already difficult to discern differences prior to the quantization stage (language modeling tasks); the true divergence emerges in how the model is protected and addressed.
  6. Only an end-to-end speedup comparison can truly demonstrate the advantages of ultra-low-bit quantization. Although the authors acknowledge the limitations, the reviewers consider such a comparison to be essential. If the contribution is limited merely to comparisons on language modeling tasks or metrics like bits-per-word (BPW), its value to the research community would be minimal.
  7. The organization of the tables—particularly those in the appendices—is overly chaotic, with notes that lack clarity and precision, making it difficult for human readers to follow the information accurately. While the author may have intended to provide a comprehensive set of relevant data, it is essential to ensure that the presentation remains concise and clear, rather than being cluttered with an excessive number of table headers and annotations.
Questions for authors

All the weakness need feedback.

2 comments
AuthorsOpenReview ↗

Comment 1/2: W1-W4

Thank you for the detailed review. We have run the requested controls and evaluations, and will correct the accounting, AdaDim claim and presentation. We address all seven points below.

W1 (Fisher item splits). The submitted probe pool included MMLU test, TriviaQA validation and ARC-Easy test items: 1.9% of the MMLU and 10.9% of the ARC-Easy evaluation sets contributed gradients to the selector. We should have disclosed this overlap. Although the items select where secondary planes are placed, rather than fitting their values, that selection can still affect evaluation performance. We therefore replaced the pool with MMLU validation + dev from the same recall subjects, TriviaQA train and ARC-Easy train, with no items used in a reported evaluation, and reran Tables 3 and 12 and Figure 3. The rerun preserves similar unit rankings and reproduces all five budget-sweep points within the seed ranges, including the advantage over random allocation. After scale distillation, results remain close at both sizes; at 8B, MMLU is 61.9 with disjoint items versus 61.8 originally, with similar perplexity and zero-shot accuracy (two seeds each). We also evaluated knowledge sets absent from both probe pools. On CommonsenseQA, the disjoint knowledge Fisher scores 71.2%, compared with 68.6% for text Fisher and about 61–62% for random and Hessian×weight selection. SciQ and NQ-open are less informative because performance is near the ceiling and floor, respectively; IFEval shows little difference between the Fisher selectors. These checks preserve the main findings, and the revision uses the evaluation-disjoint pool throughout, including generation results.

W2 (matched-budget text-Fisher control at post-trained 8B). Agreed that Table 3 needs it, and we have run it: +0.40 bits, two seeds, the Table 3 protocol. The text Fisher scores WT2 14.23 / zero-shot 64.05 / MMLU 59.77, against the knowledge Fisher's 14.94 / 64.74 / 61.89, random's 15.74 / 63.07 / 55.58 and Hessian×weight's 17.68 / 62.32 / 52.80. It is the best selector on perplexity and 2.1 MMLU points behind the knowledge Fisher on both seeds, the 1.7B pattern (2.7 points there), while still 4.2 above random and 7.0 above Hessian×weight. The revised table includes this control and reports the trade-off between perplexity and MMLU.

W3 (bit accounting). You are right that the name was misleading. The rate counts log₂ L information bits per code plus every parameter a decoder needs to rebuild the weight (scales, zero points, group and plane membership, norm gains, the r4 sign vector), under one rule for every method in Table 1; it excludes the fp16 embedding and head and the packing overhead of a 2-bit code. We will call it "information bits (decoder-complete)", state the exclusions in one sentence, and give the stored bytes where a packed release exists. Bodies only, Qwen3-8B: RoleQ-T one plane 1.42 GB (1.63 stored bits per weight, five ternary codes per byte, against 1.590 information bits); RoleQ-T + planes 1.94 GB (2.23 vs 2.157); QTEA's released pack 1.98 GB (2.28 vs 2.281: codes 1.44 GB, fp32 scales 0.44 GB, fp8 residuals 0.10 GB). Our files add an int8 embedding (0.62 GB) and an int5 head (0.39 GB), 2.43 / 2.95 GB complete. CAT-Q and Qronos release dense fp16 / fp32 weights (16 / 31 GB) and PT²-LLM only its parameters. Stored bits track information bits within 0.1 bits per weight for these packed formats. The distinction matters for deployment even though it does not change their ordering in this comparison.

W4 (AdaDim). You are right: "half the solver work" compared against a two-way GPTQ search, which is not AdaDim's procedure. We will replace that claim with an end-to-end measurement. We ran AdaDim as published (RTN on both axes, its criterion, one GPTQ pass on the chosen axis) on Qwen3-1.7B-Base and 8B-Base, in the r1 and recipe bases, timing the selection and GPTQ in the same job. Rotations reduce the outlier effects that make low-bit quantization difficult. At 8B in the recipe basis the search puts 89% of the matrices on the role axis and reads 23.16 / 51.2 / 34.1 → 11.32 / 61.0 / 49.3 against RoleQ-T's 22.20 / 50.3 / 31.8 → 11.27 / 60.1 / 44.9 (seed 45), inside the seed ranges on every instrument. At 1.7B it picks the role axis for 91-92% of the matrices in the recipe basis, ties after SD (20.3 / 20.9 vs 20.3 / 20.7 WikiText2) and trails at PTQ (72.8 / 86.5 vs 73.7 / 72.6); in r1 it agrees on 94-95% and ties. The selection costs 1.6 s against 102-124 s of GPTQ kernels (1-2%; 3.7% at 8B). Without a rotation it picks the opposite axis for 91% of the matrices and is 2.3× better at PTQ (554 vs 1,246), so the rule is stated for rotated bases. In a rotated basis AdaDim's search selects the role axis for most matrices, and RoleQ needs neither the search nor stored axis flags. Table 8's reconstruction-level agreement (96.4% rotated vs 14.3% unrotated) moves into the main text beside the end-to-end rows.

AuthorsOpenReview ↗

Comment 2/2: W5-W7 and the contribution

W5 (harder tasks). We agree that perplexity and multiple-choice accuracy do not fully capture model capability. We added GSM8K, MATH-500 and HumanEval, together with IFEval on the post-trained models (common response Section 5). The results show a shared limitation of the low-bit post-training methods tested: before distillation, all perform at or near zero on MATH-500 and HumanEval. RoleQ-T retains more generation accuracy in the reported comparisons, although the distillation stages are not all matched. At 8B, comparing both models after SD, planes raise GSM8K accuracy from 21.7% to 42.6%, with gains on mathematics and code generation as well. Planes also improve instruction following, but substantial losses remain: the 8B planes model retains 82% of full-precision MMLU accuracy and only 20% of MATH-500 accuracy. We made this limitation explicit in the paper and report the full results and stages in the appendix.

W6 (end-to-end speed). We added the requested comparison. The format runs in our own CPU runtime (AVX-512 kernels for ternary weights and 8-bit activations; the r4 slot as a fast Walsh-Hadamard with a stored sign vector; the second plane as a sparse descriptor field; the writer scale on its axis). On the same Sapphire Rapids cores at 8 threads RoleQ-T one plane decodes at 25.1 tok/s (1.81 GB per token) against the ternary formats of llama.cpp, TQ2_0 at 17.5 (2.31 GB) and TQ1_0 at 15.9 (1.98 GB), and of bitnet.cpp, TL2 at 21.0 (Llama-3-8B-1.58, 2.53 GB); llama.cpp's 4-bit integer Q4_0 gives 14.8 and fp16 5.2; the per-OC twin runs at 25.2 and the planes export at 17.9 (2.33 GB). TL2 uses a different model.

W7 (table organization). We will state the shared conventions (basis, stage, seeds, instruments and cell macros) once at the head of the appendix, remove repeated notes and redundant tables, and group the remaining results by question.

On the contribution. Our goal is a practical post-training route to low-bit models. On Qwen3-8B-Base, the planes configuration reaches 62.8 MMLU and 42.6 GSM8K at 2.16 information bpw in under ten GPU-hours. The contributions are role-based scales in a rotated basis, an exact ternary step in GPTQ, and Fisher-guided plane allocation; distillation is adopted. We apply the same information-bit accounting rule across methods and provide a runtime for the format. Your soundness concerns led us to rerun the selector with evaluation-disjoint items and add the text-Fisher control; the results and limitations are reported above. We welcome further questions about these results.

Reviewer_8o82OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RoleQ, a quantization framework that assigns scales according to the functional roles of Transformer weight vectors: per-row scales for readers of the residual stream and per-column scales for writers. Its ternary conversion pipeline, RoleQ-T, combines learned rotations, a role-aware adaptation of GPTQ, and optional scale distillation with frozen ternary codes. It further allocates sparse secondary ternary planes to functional units using a Fisher-weighted sensitivity score. Experiments across Qwen3 and Llama models evaluate scale placement, conversion quality, and secondary-plane allocation, with particular attention to retaining task accuracy and factual knowledge at low bit rates. The evaluation focuses on accuracy and storage accounting rather than measured inference speed.

Critical strengths
  1. Clear and interpretable central contribution. The role-aligned scale-placement rule provides a concrete structural rationale for choosing between per-input- and per-output-channel scales. The paper clearly distinguishes the placement rule from the conversion pipeline and the optional secondary-plane allocation.
  2. Extensive controlled experiments. The paper examines scale placement under shared solver, rotation, and calibration settings, and provides additional comparisons of rotation objectives, solver components, scale distillation, and secondary-plane selectors. These experiments help separate several sources of improvement beyond the performance of the complete pipeline.
Critical weaknesses
  1. The Fisher-weighted score estimates sensitivity to one-plane quantization error, but does not directly estimate the loss reduction achievable by adding a second plane.
  2. The observation that Hessian×weight selection underperforms random placement is interesting, but its cause remains unclear. Please analyze the allocation across layers and matrix types, and test whether the gap persists when these allocation proportions are matched.
  3. Although the ternary representation potentially reduces memory traffic and arithmetic cost, the paper reports neither kernel-level nor end-to-end inference speedups. The practical efficiency benefits therefore remain unverified.
Questions for authors
  1. Does the Fisher score predict the actual benefit of an additional plane? For a representative subset of units, could the authors add a second plane to one unit at a time and measure the resulting change in the target loss? Please report how well the proposed score ranks these marginal benefits.
  2. What explains the gap between Hessian×weight and random selection? Please show how Fisher, Hessian×weight, and random selection distribute the additional bit budget across layers, matrix types, and reader/writer roles.
  3. Can the proposed representation deliver practical inference speedups? Could the authors provide kernel latency for representative linear-layer shapes and end-to-end prefill and decoding latency for at least one model, covering both the single-plane and secondary-plane variants? Measurements should include the necessary unpacking, scale application, runtime rotations, and secondary-plane overheads, with matched hardware and workload settings.
1 comment

W1 / Q1 (does the score predict the marginal benefit of a second plane). You are right that the score does not directly measure the benefit of adding a plane. It is the Fisher-weighted one-plane error, used as a proxy for the sensitivity to that error. We measured the marginal effect itself on the post-trained 1.7B one-plane checkpoint. We selected 420 units stratified by kind, Fisher decile and depth, added a plane to each unit separately, and recorded the change in KL to the fp model and in perplexity. No score reliably ranks the signed effect in this test (Spearman ≤ 0.12 for every selector; the effect of one unit in 420k is at the measurement floor, and 33-55% of units in each selector's top decile hurt when added alone). The Fishers do rank the magnitude of the effect (|ΔKL| Spearman 0.47-0.52, precision at the top 10% 0.33-0.36 against 0.10 by chance), while Hessian×weight, residual magnitude and the PTQ-error attribution do not (−0.06 to −0.04). We also ran the selector you describe, score₂ = Σᵢ Fᵢ (rᵢ² − r₂ᵢ²), the Fisher-weighted error reduction the second plane achieves (one plane fit per unit, minutes on a GPU), as an arm at the same budget. Its results are close to those of the original score (in-house PTQ 9.74 vs 10.22, 5.49 vs 5.52 after SD, MMLU 41.35 vs 41.35), and its allocation differs in allocation by kind by less than one percentage point. We therefore retain the simpler score, while acknowledging that it does not predict whether an individual plane helps.

W2 / Q2 (why Hessian×weight loses to random; allocation). The allocation tables by kind, depth and role for the three selectors at 1.7B (+0.47 bits) and 8B (+0.40) are part of the appendix in our revision, and they suggest that allocation across depth is important. Hessian×weight puts 78% of its planes in the last quarter of the layers and 99% in the last half (random 25% per quarter, the knowledge Fisher 76% in the first half), and gives W_V and W_K half the Fisher's share. It ranks by error magnitude, which correlates +0.96 with the realized PTQ error of a unit, while the knowledge Fisher anti-correlates (−0.47). In the matched-proportion test: Hessian×weight scores under the Fisher's kind × layer quotas are close to the Fisher on every instrument (in-house PTQ 9.89 vs 10.15, after SD 5.574 vs 5.579, MMLU 40.5 vs 40.4), and Fisher scores under Hessian×weight's quotas produce results similar to Hessian×weight (18.96 vs 17.95; MMLU 34.3 vs 31.4). At this budget, the results point to allocation across kinds and layers as the main measurable factor; differences from ranking within quotas remain within seed variation. We will qualify the explanation in Section 3 accordingly.

W3 / Q3 (speed). Common response Section 4. On the same Sapphire Rapids cores at 8 threads, RoleQ-T one plane decodes at 25.1 tok/s reading 1.81 GB per token, against the ternary formats of llama.cpp, TQ2_0 at 17.5 (2.31 GB) and TQ1_0 at 15.9 (1.98 GB), and of bitnet.cpp, TL2 at 21.0 (Llama-3-8B-1.58, 2.53 GB); llama.cpp's 4-bit integer Q4_0 gives 14.8 and fp16 5.2. The planes export reaches 17.9 tok/s at 2.33 GB per token; prefill at 8 threads is 36.7 tok/s for one plane against TQ2_0's 51.1. The single-threaded ternary matvec streams 15-20 GB/s at the 8B shapes across the ISA ladder. The TL2 result uses a different model, so it is not a same-model comparison.

Reviewer_VGx4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a post-training pipeline that converts pretrained LLMs to ternary weights. Its core idea, RoleQ, is a fixed rule for where quantization scales go. Matrices that read from the residual stream (Q, K, V, gate, up) get one scale per row. Matrices that write to it (O, down) get one scale per column. Each read/write vector therefore has its own scale, which is not shared across residual coordinates. The full pipeline, RoleQ-T, has four stages:

  • a weight-only learned rotation (an ℓ4 objective in slot r1, an exact-ternary-error objective in r2, and a Hadamard r4);
  • GPTQ with an exact per-vector ternary fit and a closed-form scale refit for reader rows;
  • optional "scale distillation" (SD): roughly 400 KL steps that train only the scales and reintroduced norm gains, with the codes frozen;
  • optional sparse second ternary planes, assigned to whole units ranked by a Fisher-weighted residual saliency.

Experiments cover Qwen3 0.6B–14B (Base and post-trained) and Llama-3.1-8B, reporting WikiText2 perplexity, a seven-task zero-shot average and 5-shot MMLU. The paper also re-derives every method's "memory-resident" bpw, counting all decoding metadata. On that basis RoleQ-T+SD at 1.59 bpw beats Qronos, PT²-LLM and QTEA, and with planes (2.16 bpw) it keeps most of the full-precision MMLU (62.8 vs 76.8 on Qwen3-8B-Base). The authors are open about the limits: the placement rule only holds in a rotated basis, QTIP/GuidedQuant stay ahead at 2 bpw, and no latency is measured.

Critical strengths
  1. Retained knowledge at post-training cost, under uniform bit accounting. The paper shows that existing ternary PTQ methods score 26–35 MMLU on Qwen3-8B-Base, close to chance, even when their perplexity is moderate. RoleQ-T retains 47.6 at 1.590 bpw and 62.8 at 2.157 bpw (Table 1), at a cost of a few GPU-hours. All rates, including the baselines', are recomputed with one rule ("can a decoder reconstruct the weight from the counted bits alone?", App. A, Table 5). This matters because it changes how sub-2-bit claims in this literature should be compared, and it backs the paper's argument that perplexity alone hides knowledge loss.

  2. A mechanistic hypothesis that makes testable predictions, with negative results reported. The reader/writer rule is not just tuned empirically. It predicts that the rule should match a reconstruction-error axis search only after rotation (96.4% vs 14.3% agreement with AdaDim, Table 8). It also predicts that r4, the one slot that mixes a unit axis, should shrink specifically the writer half of the margin (1.63× → 1.17×, Table 13d). Both predictions hold. The scope limits are stated (rotated basis only; QTIP is better at 2 bpw, Table 10). This makes the placement result credible even where its effect size is small.

Critical weaknesses
  1. The headline claim of leading at matched bits depends on scale distillation, which only one baseline receives. At the PTQ stage, the one-plane RoleQ-T (22.42 / 50.39 / 33.32) is only marginally ahead of Qronos (22.80 / 46.67 / 25.93), and it loses to QTEA on perplexity (15.66) while roughly tying on zero-shot and MMLU (Table 1). The large gaps (11.33 / 60.33 / 47.59) appear only after SD, which is KL distillation on 3,200 × 512 RedPajama tokens. The authors apply SD to only one baseline, PT²-LLM. That baseline's rerun is weak to begin with: 67.95 ppl on the C4 draw versus 25.73 on its own WikiText2 draw (Table 14), so it is not a strong test of whether SD explains the gap. The paper itself says that "the lower-rate advantage therefore depends on distillation" (§4.1). The abstract's claim to "lead every post-training ternary method … on perplexity, zero-shot accuracy and MMLU" should be qualified accordingly. Applying SD to Qronos and QTEA would be the fairer comparison.

  2. The placement rule, presented as the central contribution, has a modest and fragile effect in the final recipe, and the knowledge-Fisher gains may be inflated by the choice of probe items.

    • (a) On integer grids, the placement gain is about 1.01× at Qwen-8B/14B (§4.2). On ternary, the 0.6B and Qwen-8B gaps are within seed noise (gap/range 0.6× and 0.7×, Table 7a). With g128 group scales, role-axis and per-OC placement are on par (App. D, Table 9). In the recipe basis after SD at 1.7B, per-OC scales give 21.34 against 20.26 (Table 13a), about 5%. The large ternary effects (Llama-8B 58 vs 114; 14B MMLU 38.5 vs 25.4) occur at PTQ in regimes 9–18× full-precision perplexity, which are far from usable. An 8B per-OC one-plane run in the recipe basis with SD is missing, so it is unclear how much of Table 1 the placement rule actually accounts for.
    • (b) The knowledge Fisher used for planes on the post-trained model (Table 1, Table 3: +6.2 MMLU over random) is accumulated on "MMLU-style multiple choice, TriviaQA cloze and ARC" items (§3.5). MMLU and ARC-E/C are also evaluation targets. The paper does not state which splits these items come from, or whether the evaluated questions or subjects overlap with them. If they do, the selector is tuned to the test distribution, and the comparison against random or Hessian placement is not clean.
Questions for authors
  1. [Clarification] Which datasets and splits are the 720 knowledge-Fisher items drawn from? Is there any overlap, at the item or subject level, with the MMLU test set and the ARC-E/C test sets used in evaluation? Could you report Table 3 with a Fisher built only on sources disjoint from the evaluation suite (e.g. TriviaQA only), or evaluate on a knowledge benchmark absent from the probe pool? This determines whether the knowledge-Fisher gains on the post-trained model can be taken at face value.
  2. [Clarification] Please apply the same scale distillation to Qronos and QTEA. Please also report an 8B per-OC one-plane run in the recipe basis with SD. Together these would separate the contributions of format, rotation and distillation in Table 1.
  3. [Clarification] Table 12 lists full-precision perplexity as 5.64 for post-trained Qwen3-1.7B, while Table 6 gives 16.67 WikiText2. The after-SD values (e.g. 5.45 for the text Fisher) are also below full precision. What exactly is measured in the after-SD column?
  4. [Clarification] App. D says the 512×512 calibration draw "favours the baselines". Table 14, however, shows it improving the RoleQ body by 2.25–2.36× at Llama-8B and 1.7B, against 1.13–1.24× for PT²-LLM. At 8B the effects are comparable. Please reconcile.
  5. [Suggestion] The motivation is deployment: bandwidth and multiplication-free matmuls. The format adds per-column writer scales, a runtime r4 Hadamard and irregular sparse second planes with unit membership. Even a single CPU LUT-kernel measurement (bitnet.cpp / T-MAC style) comparing tokens/s against a uniform ternary baseline would substantiate that these do not erode the practical benefit.
  6. [Suggestion] For the post-trained (chat) models, add at least one generative evaluation (e.g. GSM8K or IFEval). Log-likelihood multiple-choice tasks can miss the failure modes that matter in deployment, as the authors acknowledge in §5.
1 comment

Thank you for the careful review and for spelling out the two testable predictions. We respond to each point below.

W1 / Q2 (SD on the baselines). Common response Section 1; two details for this thread. PT²-LLM's own-draw codes keep their WikiText2 advantage through SD (11.90 against 15.51 for the C4-draw codes) but have nearly identical off-corpus results (C4 21.24 / 21.23, MiniPile 14.44), against RoleQ-T + SD's 11.33 / 18.50 / 12.56 at 0.245 bpw less. Qronos' codes were recovered exactly; its +SD row distils those codes. The abstract now clarifies that the lead is the result after a brief distillation of the scales alone. We also made the PTQ comparison clearer. In Section 4.1, the one-plane row is close to Qronos on perplexity and leads on both task measures, while QTEA at its higher rate has lower perplexity than our one-plane row before SD.

W2a / Q2 (per-OC in the full recipe at 8B). Common response Section 2. You characterise it correctly: at ternary precision and 8B, the placement effect after SD is small, as Section 4.2 now states. The clearer gains are at the PTQ stage (27% higher perplexity for per-OC at 8B) and, after SD, at 14B (Table 7). The rule also requires no axis search or stored flags and has no measurable runtime overhead in our 8B comparison. Planes naturally refine each unit's fitted grid; per-OC writer scales span units and do not support that extension.

W2b / Q1 (Fisher items). As detailed in common response Section 3, the submitted probe pool included MMLU and ARC-Easy test items (1.9% and 10.9% of their evaluation sets), alongside TriviaQA validation items. We replaced these with evaluation-disjoint splits, retaining the same recall subjects. The rerun preserves similar unit rankings and reproduces the full Figure 3 sweep within the seed ranges. After scale distillation, results remain close at both model sizes; at 8B, knowledge-Fisher selection retains MMLU leads of 6.3 points over random and 9.1 over Hessian×weight. On CommonsenseQA, which appears in neither probe pool, it also outperforms text Fisher and both controls. SciQ and NQ-open are less informative because performance is near the ceiling and floor, respectively. The revision uses the disjoint pool throughout.

Q3 (Table 12's 5.64). The after-SD column of Table 12, and Figure 3, report our in-house chat-corpus perplexity (64 held-out UltraChat-style conversations; fp 5.64), not WikiText2. SD trains on a disjoint draw of the same corpus, so after-SD values just under fp are possible on that instrument (5.45 for the text Fisher). The caption mislabelled the instrument. We will correct it and give the Fisher row's WikiText2 value beside it (24.2 after SD, fp 16.67).

Q4 (Table 14's caption). The caption overstates the result; the new measurements contradict it. We measured PT²-LLM at 1.7B on both draw shapes and both corpora: the 512 × 512 shape moves PT²-LLM 1.05× on its WikiText2 draw (66.4 → 63.5) and 0.99× on the C4 draw (211.6 → 213.3), against 2.36× for RoleQ-T; at 8B it moves PT²-LLM 1.13-1.24× and us 1.12×. The shape favours us at 1.7B and has a similar effect on the two methods at 8B. We will correct the caption and add the two rows to Table 14. The corpus change, rather than the shape change, accounts for most of PT²-LLM's shift from 66 to 213 at 1.7B: under its own protocol (WikiText2, 128 × 2048) its code yields 66.4 with MMLU at chance, versus 20.5 after SD for RoleQ-T; the stages differ.

Q5 (kernel measurement). Low-bit weights make CPU decoding practical by reducing memory traffic. Our quality improvements preserve this benefit: on Qwen3-8B with eight Sapphire Rapids threads, RoleQ-T decodes at 25.1 tok/s versus 5.2 for llama.cpp fp16. Role-aligned scales add no measurable overhead relative to our per-OC control (25.2 tok/s). Secondary planes trade some speed for accuracy, reaching 17.9 tok/s, comparable to llama.cpp's ternary TQ2_0 at 17.5. Common response Section 4 gives the full comparison, including the limits on thread scaling and prefill speed.

Q6 (generative evaluation). Common response Section 5 reports reasoning and code generation. We also test instruction following with IFEval on post-trained Qwen3-8B. Adding Fisher planes raises prompt-level strict accuracy from 12.8% to 15.4%, versus 25.3% at full precision. The improvement extends to individual instructions, but substantial losses remain. Knowledge and text Fisher perform similarly here: the knowledge-specific advantage on MMLU does not extend to instruction following.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common response 1/2: distillation, placement, the Fisher items

We thank the reviewers for their thoughtful feedback. Several questions came up across reviews, so we address them here and provide further detail in the individual responses.

1. Scale distillation on the baselines' codes (VGx4 W1/Q2, B8BY W2). The submission distilled only PT²-LLM's codes, and only on our C4 draw. We have now applied the same scale distillation (SD) to the codes of all three baselines under the Table 1 protocol: the same teacher-cache draw, 400 steps, codes frozen, each method's own continuous parameters and the reintroduced norm gains trainable. WikiText2 / zero-shot / MMLU, PTQ → +SD:

  • PT²-LLM, its own WikiText2 draw (1.835 bpw): 25.73 / 43.23 / 26.96 → 11.90 / 57.65 / 38.83
  • Qronos + HIP + MagR (1.591 bpw, the one baseline at our rate; its codes recovered exactly from the released pipeline, the per-matrix Hadamard kept as a runtime op): 22.80 / 46.67 / 25.93 → 14.12 / 54.67 / 33.16
  • QTEA (2.281 bpw): 15.66 / 50.60 / 34.82 → 10.15 / 60.28 / 50.19
  • RoleQ-T (1.590): 22.42 / 50.39 / 33.32 → 11.33 / 60.33 / 47.59; with planes (2.157): 11.32 / 59.51 / 56.04 → 9.30 / 65.89 / 62.75

After distillation, each baseline trails a RoleQ-T row at equal or fewer bits on all three measures. At the matched rate, Qronos has 25% higher WikiText2 perplexity and scores 5.7 points lower on zero-shot and 14.4 points lower on MMLU. SD helps the baselines substantially, but does not close the gap in these comparisons. The revised Table 1 includes these +SD rows and distinguishes the PTQ comparison: before SD, the one-plane row is close to Qronos on perplexity (22.42 vs 22.80) and leads on both task measures; the planes row leads QTEA on all three.

2. Per-OC placement inside the full recipe with SD at 8B (VGx4 W2a, B8BY W2/Q1). We add the per-OC control under the full recipe with SD at 8B, two seeds (Qwen3-8B-Base, recipe basis, 1.589 vs 1.590 bpw; WT2 / zs7 / MMLU):

placementPTQ+SD
RoleQ-T (role axis)22.42 / 50.39 / 33.3211.33 / 60.33 / 47.59
per-OC28.53 / 50.33 / 33.4011.48 / 59.94 / 48.91

At PTQ, per-OC perplexity is 27% higher. After SD, the perplexity gap is 1.4%, within the seed range; the task differences are also small relative to seed variation (the role-axis seeds span 44.9-50.3 on MMLU). Section 4.2 now states this explicitly: at 8B, SD largely closes the placement gap for a one-plane model. Across tested sizes and families, PTQ perplexity improves by up to 2×. At 14B the advantage remains after SD on all three measures (Table 7). The rule also requires no axis search or stored axis flags and uses the same kernels. Rotations reduce outlier effects in low-bit quantization; in these bases, AdaDim's search selects the role axis for most matrices (Section 4.1).

Secondary planes naturally extend the unit-based format: each vector has its own fitted grid, refined by another plane. Writer vectors are columns; per-OC scales span units and do not support the same unit-wise extension. This one-plane control does not isolate placement with planes.

3. The knowledge-Fisher items (VGx4 W2b/Q1, NBNW W1/W2). In the submitted version, the 720 probe items were drawn with a fixed seed from MMLU test (4-5 per recall subject), TriviaQA validation and ARC-Easy test: 1.9% of the MMLU test set and 10.9% of ARC-Easy's contributed gradients to the selector. The submission should have disclosed this overlap. Although the items choose where a second plane goes rather than its values (which are fitted on calibration text), they can still affect evaluation performance. We therefore recomputed the Fisher on an evaluation-disjoint pool (MMLU validation + dev of the same subjects, TriviaQA train, ARC-Easy train; no item appears in a reported evaluation) and reran Table 3, Table 12 and Figure 3. The results are similar: unit rankings have Spearman correlations of 0.9-0.98 at 8B in the same basis, and the post-SD results are close at both sizes (8B: 14.94 / 64.74 / 61.89 vs 15.30 / 64.57 / 61.82, two seeds each; the disjoint selector is 6.3 MMLU points above random). No conclusion of the paper changes: every selector ordering and the Figure 3 trend are reproduced. The revision uses the disjoint pool throughout. We also tested a matched-budget text-Fisher control at 8B (14.23 / 64.05 / 59.77: best on perplexity, 2.1 MMLU points behind the knowledge Fisher) and evaluated on knowledge sets absent from both pools. On CommonsenseQA after SD, knowledge Fisher scores 71.2, text Fisher 68.6, random 62.1 and Hessian×weight 61.2 (fp 78.7). SciQ is near ceiling and NQ-open near the floor for all arms, so they are less informative here.

AuthorsOpenReview ↗

Common response 2/2: speed, generation, text changes

4. Speed (VGx4 Q5, 8o82 W3/Q3, NBNW W6). The format runs in our own CPU runtime: AVX-512 kernels for ternary weights and 8-bit activations, the r4 slot as a fast Walsh-Hadamard with a sign vector, the second plane as a sparse field, the writer scale on its axis. We measured decode on the same Sapphire Rapids cores against llama.cpp on the same model (Qwen3-8B) and bitnet.cpp on its Llama-3-8B-1.58, a different model.

engineweight formatGB read / tokentok/s, 8 threads16 threads
oursternary, RoleQ-T one plane1.8125.122.6
oursternary, per-OC twin1.8125.222.3
oursternary + planes2.3317.915.3
llama.cppternary (TQ2_0)2.3117.529.1
llama.cppternary (TQ1_0)1.9815.927.4
llama.cpp2-bit integer (Q2_K)3.0814.022.8
llama.cpp4-bit integer (Q4_0)4.4214.824.0
llama.cppfp16 (F16)15.15.28.4
bitnet.cppternary (TL2)2.5321.033.3

At 8 threads, the one-plane model matches the per-OC twin and outpaces the external runtimes tested, with the fewest bytes read. Secondary planes increase traffic and reduce speed. There are limitations: our current runtime does not scale beyond 8 threads in this test, where llama.cpp and bitnet.cpp do, and at 8 threads our one-plane prefill is slower than llama.cpp's ternary TQ2_0 (36.7 vs 51.1 tok/s). The table is now part of the appendix.

5. Generation (VGx4 Q6, NBNW W5). We add GSM8K (5-shot), MATH-500 (Minerva 4-shot) and HumanEval (0-shot) for the Table 1 methods at the stages below, and IFEval for the post-trained block (prompt-level strict accuracy: fp 25.3, TWLA 18.7 at 4.15 bpw, Fisher planes 15.4, g128 13.7, CAT-Q 12.9, one plane 12.8; the ternary rows keep 50-60% of fp). Accuracy in %, MMLU in parentheses:

rowGSM8KMATH-500HumanEval
Qwen3-8B-Base fp (76.8)81.456.864.0
Qronos + HIP + MagR, PTQ (25.9)0.00.00.0
QTEA, PTQ (34.8)5.21.20.0
PT²-LLM codes + our SD (36.7)4.61.22.4
RoleQ-T + SD, 1.590 bpw (47.6)21.75.95.5
RoleQ-T + planes + SD, 2.157 bpw (62.8)42.611.118.0
Qwen3-14B-Base fp (80.7)79.256.257.9
QTEA, released (47.8)13.21.80.6
RoleQ-T + planes + SD (68.3)61.023.422.9

The generation results show a shared limitation of the low-bit post-training methods tested: all PTQ rows are at or near zero on MATH-500 and HumanEval. RoleQ-T retains more generation accuracy in these comparisons, with no baseline at or below 2.3 bpw exceeding its corresponding row, although the SD stages are not all matched. Substantial losses remain: the 8B planes row retains 82% of fp MMLU accuracy but only 20% of fp MATH-500 accuracy. We discuss this limitation in Section 5 and give the full results in the appendix.

6. Text changes. We will state the PTQ-stage comparison beside the abstract's lead; rename "memory-resident bpw" to "information bits (decoder-complete)" and give the stored bytes where a packed release exists (NBNW W3); replace "half the solver work" with the end-to-end AdaDim measurement (Section 4.1); define per-OC / per-IC / QK / OV at first use, split the long paragraphs, add scale optimization to Related Work and state the appendix conventions once; and correct the Table 12 and Table 14 captions (VGx4 thread).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.