acceptodds
Under review as a conference paper at ICLR 2027

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

Abstract

Smaller activation outliers do not necessarily imply better low-bit quantization: their alignment with the quantizer matters. We introduce PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4. The affine offsets represent the energy in this subspace without widening the range within the group. We formulate rotation design as a Ky Fan trace maximization and derive a closed-form solution that is provably optimal for this alignment objective. Compact Householder transformations and their compact-WY representation enable gradient-free construction and efficient application at both foldable and online sites. A predictive range law further connects unaligned activation energy and group size to quantization-relevant variation. Experiments on Llama, Qwen, and Mistral span dense models up to 70B parameters and a 30B mixture-of-experts model. Under W4A4KV4, PrismQuant sets the state of the art on Llama-3.2-3B among the compared methods in both perplexity and accuracy. On Llama-3.1-70B, it attains 3.85 perplexity and 72.46% average zero-shot accuracy, only 0.22 percentage points below the full precision. In the deployment study on Llama-3.1-8B, our optimized implementation achieves 1.51× prefill and 1.22× CUDA Graph decode speedups over matched FP16 baselines, with 56.34% lower decode peak memory and only 2.35% additional Graph decode latency over Hadamard.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Model Quantization, Low-Bit Inference, Activation Quantization, W4A4KV4, Rotation-Based Quantization, Quantizer-Aware Optimization

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_NeUzOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PrismQuant, an orthogonal rotation method for group quantizers, aimed at reducing activation errors in W4A4KV4 quantization of large language models. The authors exploit the property that the constant direction within a group does not increase the quantization range, align the dominant activation energy to these directions via spectral decomposition, and prove the optimality of the energy-capture objective based on the Ky Fan principle. In implementation, Householder reflections, the Compact-WY representation, and grouped Hadamard transforms are employed, combined with weight folding and low-rank online correction to reduce computational overhead. Experiments cover Llama, Qwen, Mistral, and MoE models, and the results show that the method effectively reduces quantization error and improves perplexity and task performance on most tested models.

Critical strengths
  1. This paper designs an orthogonal rotation based on the geometric structure of the quantizer itself, rather than focusing solely on the distribution of activation outliers, and provides a clear optimization objective and optimal solution through theoretical analysis, which offers a certain degree of innovation and theoretical value.
  2. The experiments cover Dense and MoE models of different scales, and the effectiveness of the method is verified through multiple ablation studies. In addition, the authors consider the computational overhead of structured rotation, demonstrating good potential for engineering applications.
Critical weaknesses
  1. The paper emphasizes using the affine offset of asymmetric quantizers to absorb activation energy, but experiments show that the method still achieves similar performance advantages under symmetric quantization. This suggests that the main benefits may stem more from activation flattening and energy localization, and the contribution of the core mechanism still needs to be further clarified. Moreover, the theoretical optimality of energy alignment does not directly guarantee optimal practical quantization performance.
  2. The deployment performance experiments and the main accuracy experiments adopt different quantization configurations, and the performance tests use random weights. Therefore, although the existing results verify the execution efficiency of structured rotation, they cannot yet fully demonstrate that the complete method can simultaneously achieve the reported accuracy and speed benefits in real models.
Questions for authors
  1. Could you further explain the respective contributions of spectral alignment, activation flattening, and affine offset absorption to the performance improvement, as well as the relationship between theoretical optimality and practical quantization performance?
  2. Could you clarify what additional deployment overhead the complete method may introduce under the asymmetric group quantization configuration consistent with the main accuracy experiments, and whether such overhead would affect the existing performance conclusions?
2 comments
AuthorsOpenReview ↗

Response to Reviewer NeUz

Thank you for the careful and insightful review. We were glad that you describe the rotation as designed "based on the geometric structure of the quantizer itself". Your two questions probe exactly what that claim must support, and answering them has made the paper more precise. New experiments and analyses are in Appendix F (pp. 39–44, in blue).

W1 / Q1. "... the respective contributions of spectral alignment, activation flattening, and affine offset absorption ... as well as the relationship between theoretical optimality and practical quantization performance?"

1. Contributions of the three mechanisms (Appendix F.2, Table 17, p. 41). You are right: at g=128g=128, flattening and localization carry the larger part of the gain, and offset absorption accounts for 16–40% (now stated in Sec. 1 and App. A.4). Our original framing leaned too much on the offset, and your observation corrected it. The three act in sequence, and ablations on Llama-3.1-8B (g=128g=128, k=max⁡k=\max, 3 seeds) isolate each:

  • Spectral alignment (Table 7, p. 19): selects which energy is moved. With the construction fixed, replacing our all-token spectral subspace by an outlier-token subspace drops the captured energy from f=0.324f=0.324 to 0.1520.152 and loses 26% of the PPL gain (+0.016 of 0.061).
  • Flattening and localization vs. offset absorption (Table 17, p. 41): with the Householder factor and permutation fixed, we move the aligned anchors from the constant Walsh column to a nonconstant one of the same group. This keeps the captured energy, the single-group localization, and the flat magnitude, and exposes the level to the range instead of the offset.
    • Asymmetric INT4: Hadamard 6.402, constant column 6.333 (−0.069), nonconstant column 6.361 (−0.041). Removing offset absorption forfeits 0.028 PPL (3.8 SE): offset absorption contributes ~40% of the gain, and the remaining ~60% is obtained without it.
    • Symmetric INT4 (no offset): the constant (6.399) and nonconstant (6.402) columns match within noise, yet PrismQuant still beats Hadamard (6.489) by −0.090 PPL. With one group per token (g=dg=d; Table 9a, p. 20), the offset's share rises to 37–69%.

2. Theoretical optimality vs. practical performance (Sec. 4.3, p. 9; App. F.2, p. 41). We agree: Proposition 1 guarantees optimality for the alignment objective (Eq. 6), not for perplexity, and your question prompted us to state this explicitly. Maximizing that objective minimizes residual energy, which bounds the sum of squared group ranges (Eq. 26). The mean quantization step also depends on the crest-factor ratio of the residual (Eq. 30, p. 26; 0.87–0.92 at the down-projection input, Table 12), which a second-moment objective does not control. This is why PPL is not strictly monotone across ranks k (Table 3). With the construction fixed, PPL tracks the captured energy f monotonically, across subspace estimators (Table 7) and solver passes (Table 8).

W2 / Q2. "... what additional deployment overhead the complete method may introduce under the asymmetric group quantization configuration ... and whether such overhead would affect the existing performance conclusions?"

It does not affect the conclusions. Your concern was a fair description of the submission, and we have now deployed the group-128 asymmetric Llama-3.1-8B checkpoint of Table 1 with real weights and integer kernels (Appendix F.1, pp. 39–40).

  • Joint accuracy and speed on a real model (Table 20, p. 43): loaded on an A40 with real GPTQ weights and group-128 asymmetric INT4 activations, the checkpoint reaches 6.99 PPL (identical to Table 1), 317 ms prefill, and 16.75 tokens/s decoding, 1.72× faster than FlatQuant's official kernels (9.74 tokens/s, 7.14 PPL).
  • Real-weight verification (Table 16, p. 40): re-running the Appendix D timings with real weights reproduces all 12 entries of Table 13 to two significant figures (1.5× prefill, 1.2× Graph decode vs. FP16, −57% memory; per-token symmetric backend of Appendix D).
  • Where the overhead comes from (Table 15, Eq. 41, pp. 39–40): (1) Online rotation: +2.35% Graph decode latency and +44 MB over Hadamard, independent of the quantizer format since the rotation outputs FP16 before quantization. (2) Integer GEMM (Eq. 41, operation count): rescaling every 128 columns and adding the offset product costs 2d/g = 1.6% extra MACs per layer and 16/(4g) = 3.1% weight storage for the precomputed integer group sums.

We hope this answers both of your questions. If it does, we would be grateful if you would consider raising your score, and we are glad to clarify anything further.

AuthorsOpenReview ↗

A brief follow-up before the discussion period closes

Thank you again for your careful review. As today is the last day of the discussion period, we would like to check whether our response has answered your two questions, or whether anything remains unclear.

In brief:

  • Mechanism (W1 / Q1). Your observation was right. The new attribution on Llama-3.1-8B (Table 17, p. 41) shows that flattening and localization carry the larger part of the gain and that offset absorption accounts for about 40%. The paper is reframed accordingly, and Section 4.3 now states that optimality holds for the alignment objective.
  • Deployment (W2 / Q2). The checkpoint of Table 1 is now deployed with real weights in the group-128 asymmetric configuration, and its perplexity (6.99, identical to Table 1) and latency are read from the same loaded model (Table 20, p. 43).

If any point is still open, we would be glad to address it today. If these two points are now resolved, we would be sincerely grateful if you would consider updating your rating to reflect that. Your questions led to the mechanism analysis in Appendix F.2, which we consider one of the most valuable additions to the paper.

Reviewer_sDQhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes PrismQuant, a PTQ method which introduces a solution to construct rotation matrices that improves upon conventional Hadamard rotations. Through this rotation matrix design, the method aligns dominant activation eigenvectors with the group-constant subspace (components can be represented by the existing offsets). Based on the eigendecomposition and the Ky Fan maximum principle, the paper derives a closed-form optimum for the proposed alignment objective, proving that this alignment maximizes the expected activation energy captured by the target subspace. Experiments on Llama/Qwen/Mistral support the accuracy improvements on quantized models under W4A4KV4 settings. This work also evaluates inference performance on an Nvidia A40 GPU.

Critical strengths
  1. This work delivers a fresh perspective, and also a technically feasible design. In model quantization, methods that improve accuracy often introduce additional overhead. However, this work cleverly uses existing offsets to achieve its optimization objective. It treats the subspace spanned by the group-constant directions as a container, seeking a certain rotation that directs as much energy as possble into it. It is a good design and provides a new direction for research on optimizing rotation matrices.
  2. The mathematical deiviations support the soundness of the method. And of course, the optimality is with respect to the defined optimization objective.
Critical weaknesses
  1. I find the paper provides reasonably thorough theoretical and empirical support for the advantages of PrismQuant over conventional Hadamard rotations under grouped INT4 quantization. However, comparisons with more advanced transformation-based methods remain insufficient, such as FlatQuant. Although FlatQuant is not strictly a rotation method, it is a relevant baseline: the main goal of this work is to improve the accuracy of quantized models, and methods like FlatQuant offer strong accuracy under INT4 quantization and also incorporate runtime inference optimizations. A direct comparison would help demonstrate whether PrismQuant offers a clear advantage over FlatQuant in either accuracy or inference performance.
  2. Some mismatches: in the main accuracy results, the activation quantization uses group-128 asymmetric INT4, whereas the end-to-end performance evaluation uses per-token symmetric INT4 activation quantization. Although this mismatch is acknowledged in Appendix D, it still raises the question of whether the reported accuracy and the performance can be achieved at the same time.
Questions for authors
  1. Could you compare PrismQuant with FlatQuant under INT4 quantization in terms of accuracy and inference performance?
  2. Could you evaluate accuracy and inference performance on the same quantized model using group-128 asymmetric INT4 activations?
1 comment
AuthorsOpenReview ↗

Response to Reviewer sDQh

Thank you for the encouraging review. Your description of the group-constant subspace "as a container" is exactly the picture we hoped to convey. As you note, optimality is "with respect to the defined optimization objective", and Section 4.3 now states this explicitly. Both of your experimental requests are addressed in Appendix F (pp. 39–44, in blue).

W1 / Q1. "Could you compare PrismQuant with FlatQuant under INT4 quantization in terms of accuracy and inference performance?

Yes (Appendix F.3, Tables 18–20, pp. 42–43). Without gradient training, PrismQuant reaches a little bit lower perplexity than FlatQuant at the same zero-shot accuracy, and decodes 1.72× faster on the same GPU, with a 5% longer prefill.

You are right that FlatQuant is the baseline to compare with: it is one of the strongest W4A4 methods, and its released kernels and checkpoints are what made a deployed, same-GPU comparison possible. We ran it from its official repository in its default configuration on Llama-3.2-3B and Llama-3.1-8B, with our perplexity protocol and evaluation harness. Our run reproduces its reported 8B PPL (7.02 vs. 7.01).

MethodTransform obtained by3B PPL ↓3B Avg. (%) ↑8B PPL ↓8B Avg. (%) ↑
FlatQuantGradient descent (15 epochs) + learned clipping8.7260.197.0265.34
HadamardFixed, no calibration9.0459.297.2165.37
PrismQuant (k=8)Closed-form spectral alignment8.6060.806.9965.36
PrismQuant (k=max)Closed-form spectral alignment8.5861.236.9265.43
  • Accuracy (Tables 18–19, p. 42): PrismQuant has lower PPL on both models, and the eight-task means are level: all differences are within two standard errors (0.63 points for a difference of two means), and each method leads on some tasks.
  • Protocol, stated plainly (App. F.3): each method keeps its own quantizer. We use GPTQ weights and group-128 asymmetric activations (0.25 more metadata bits); FlatQuant uses round-to-nearest weights, per-token symmetric activations, and trained transforms and clipping. We therefore do not present this as a matched-protocol win. What it shows is that a training-free closed-form rotation reaches the accuracy of a gradient-trained affine transform.
  • Deployed inference on the same GPU (Table 20, p. 43): each method's real quantized Llama-3.1-8B checkpoint is loaded on one NVIDIA A40 (2048-token prompt, batch 1, eager mode, median of 20 runs). FlatQuant's official packed-INT4 kernels: 7.14 PPL, 302 ms prefill, 9.74 tokens/s. PrismQuant (k=8): 6.99 PPL, 317 ms prefill, 16.75 tokens/s (1.72× faster decoding).

W2 / Q2. "Could you evaluate accuracy and inference performance on the same quantized model using group-128 asymmetric INT4 activations?

Yes. You identified the right gap, and the PrismQuant row of Table 20 (p. 43) is exactly this joint evaluation on a single model (Appendix F.1, pp. 39–40).

  • Same model, same configuration (Table 20, p. 43): the Llama-3.1-8B checkpoint of Table 1 (real GPTQ weights, group-128 asymmetric INT4 activations) runs with integer kernels on an A40. Read from that one loaded model: 6.99 PPL (identical to Table 1), 317 ms prefill, and 16.75 tokens/s decoding.
  • Real-weight verification of the FP16 speedups (Table 16, p. 40): re-running the Appendix D timings with real weights reproduces all 12 entries of Table 13 to two significant figures: 1.5× prefill and 1.2× Graph decode speedup over FP16, with −57% peak memory.
  • Cost of the rotation (Table 15, p. 39): against Hadamard in the same backend, +2.35% Graph decode latency and +44 MB. The rotation outputs FP16 before the quantizer is invoked, so this cost does not depend on the activation format.
  • Scope: the text and tables now distinguish the per-token symmetric backend of Appendix D (Tables 13 and 16, CUDA Graph decoding) from the eager-mode group-128 asymmetric integer deployment (Table 20).

Thank you again for your support of this work. Your two constructive requests helped us build a more complete framework and a clearer paper, and we would be glad to answer any further questions.

Reviewer_Hdp7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper focuses on the subspace spanned by the per-group constant directions as the target. It formalizes a spectral problem with a closed-form optimum over the full second moment, aiming to find a rotation that carries the most activation energy into it. The solution is implemented by a rank-k compact-WY factor that folds into weights or runs online at the same cost. Further, the proposed range law captures the unaligned energy, which is formalized as a function of group size and metadata budget. Experiments over eight tasks compared to five existing methods show that the proposed method achieves noticeably lower perplexity and higher average accuracy.

Critical strengths

The motivation of the paper is well explained, as is the methodology. Experimental results support the proposed methodology.

Critical weaknesses

It would be great to have insights explained along with observed improvements in perplexity and accuracy, e.g., why Off-Q and BASE-Q perform better on some tasks.

Questions for authors

Q1: Why are these five existing methods being selected for comparison? Is there any specific reason for excluding DuQuart in Table 1?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Hdp7

Thank you for the positive review, and for the kind remark that "the motivation of the paper is well explained, as is the methodology". You asked for what the submission lacked: insight alongside the numbers, and a clear account of the baselines. Both are added in Appendix F (pp. 39–44, in blue).

W1. "It would be great to have insights explained along with observed improvements in perplexity and accuracy, e.g., why Off-Q and BASE-Q perform better on some tasks."

We agree, and have added a mechanism attribution (App. F.2, p. 41) and a task-by-task analysis (App. F.4, Table 21, p. 44) to answer both points.

  1. Where the improvements come from (Fig. 2, Tables 7 and 17): A dense Hadamard spreads each dominant direction across all groups. PrismQuant instead places each leading eigendirection in a single group as a constant vector and leaves it zero elsewhere. Across all 28 down-projection inputs of Llama-3.2-3B (Fig. 2, p. 3), this shrinks the within-group range by 25% and the INT4 error by 40%. With the construction fixed, PPL improves monotonically with the captured energy f (Tables 7–8, p. 19). At g=128, offset absorption accounts for 16–40% of the gain across three models, and the larger part is obtained without it, through flattening and localization (Table 9; Table 17, p. 41).
  2. Why OffQ, BASE-Q, or other baselines score higher on some tasks (App. F.4, Table 21, p. 44): We compared all 24 model–task pairs of Table 1 against one fixed PrismQuant row per model, the one with the best average (k=max on 3B and 8B, k=8 on 70B). A published baseline is higher on 7 of 24 pairs:
    • Within noise: 6 of the 7 gaps, including every gap to OffQ and BASE-Q, are smaller than one binomial standard error on that task (0.76 to 2.21 points). The published rows also come from other pipelines and harness versions, so gaps of this size cannot be attributed to a method.
    • The single exception depends on the rank: on 8B SIQA, k=max trails OSTQuant by 1.63 points (1.4 SE), but at k=8 PrismQuant scores 47.24, above OSTQuant (47.13) and every other published row.

Q1. "Why are these five existing methods being selected for comparison? Is there any specific reason for excluding DuQuant in Table 1?

The five are the baselines reported in the two source papers of Table 1 (OffQ for 3B; BASE-Q for 8B and 70B). There was no reason to exclude DuQuant, and thanks to your question we have now run it, together with FlatQuant, from their official repositories (Appendix F.3, Tables 18–19, p. 42):

Method3B PPL ↓3B Avg. (%) ↑8B PPL ↓8B Avg. (%) ↑
DuQuant (official repo)10.3556.728.1962.59
FlatQuant (official repo)8.7260.197.0265.34
Hadamard9.0459.297.2165.37
PrismQuant (k=8)8.6060.806.9965.36
PrismQuant (k=max)8.5861.236.9265.43
  • DuQuant (App. F.3, p. 43): PrismQuant has 1.2–1.8 lower PPL and is higher on all 8 tasks of both models (+4.1–4.5 points in the mean on 3B, +2.8 on 8B; more than 4 SE). Figure 2 (p. 3) shows why: outside the first and last layers, DuQuant's block rotations leave little energy in the group-constant subspace, so its group range and quantization error stay close to Hadamard's.
  • FlatQuant: PrismQuant has lower PPL on both models at the same accuracy, with a closed-form rotation in place of gradient training.

Presentation. We also took your rating as a prompt to tighten the writing: R2 is now described identically in Sec. 3.4 and Alg. 1, App. A.1 states which baseline rows are quoted and which are run by us, and each timing names its backend.

Thank you again for your support. Asking for the insight behind the numbers and for broader baseline coverage is the kind of reviewing that moves the efficient-AI community forward, and it has made this paper more useful to that community. We would be glad to answer any further questions.

Reviewer_gL7sOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes PrismQuant, a quantization method that aligns dominant activation directions with group-constant directions, allowing asymmetric quantizers to represent their shared levels through offsets. It derives an optimal solution for the alignment objective and implements the rotations using compact Householder transformations.

Critical strengths
  1. The paper provides a principled formulation of rotation design and a compact implementation that avoids expensive dense online rotations. The rotations are constructed from calibration statistics without gradient-based optimization.

  2. The method improves perplexity over the matched Hadamard baseline and achieves average zero-shot accuracy within 0.22 percentage points of the full-precision Llama-3.1-70B reference.

Critical weaknesses
  1. The accuracy and efficiency results are not demonstrated jointly in the same deployed model. Appendix D reports timing with random weights and a per-token symmetric quantization backend, whereas the accuracy experiments use grouped asymmetric quantization. This leaves it unclear whether the reported accuracy improvements and speedups can be achieved simultaneously.

  2. The novelty relative to OffQ requires sharper justification. As described in the paper, OffQ already rotates activation directions into group-wise components absorbed by asymmetric offsets. PrismQuant contributes a spectral formulation, different calibration statistics, and compact execution, but the conceptual distinction is less substantial than the framing suggests.

Questions for authors

See Weakness

1 comment
AuthorsOpenReview ↗

Response to Reviewer gL7s

We are grateful for your thoughtful review. You recognize our "principled formulation of rotation design" and "compact implementation that avoids expensive dense online rotations," and both weaknesses identified real gaps in the submission. We address each with new evidence in Appendix F (pp. 39–44, in blue).

W1. "... leaves it unclear whether the reported accuracy improvements and speedups can be achieved simultaneously."

They can, and you were right that the submission did not show it. Appendix D timed random weights in a per-token symmetric backend, while Table 1 used group-128 asymmetric activations. Both are now measured on one deployed checkpoint.

  • One deployed model (Table 20, p. 43). The Llama-3.1-8B checkpoint of Table 1 (real GPTQ weights, group-128 asymmetric INT4 activations) runs with integer kernels on an A40. Read from that single loaded model: 6.99 PPL (identical to Table 1), 317 ms prefill, 16.75 tokens/s decoding (2048-token prompt, batch 1, eager). Under the same protocol, FlatQuant's official packed-INT4 kernels give 7.14 PPL, 302 ms, and 9.74 tokens/s: PrismQuant decodes 1.72× faster at lower PPL.
  • Real vs. random weights (Table 16, p. 40). Re-timing FP16, Hadamard, and PrismQuant with real weights reproduces all twelve entries of Table 13 to two significant figures: 1.5× prefill and 1.2× Graph decoding over FP16, with 57% lower peak memory.
  • Cost of the rotation (Table 15, p. 39). Against Hadamard in the same backend: +2.35% Graph decode latency and +44 MB, with prefill at parity. The rotation writes FP16 output before the quantizer is invoked, so this cost is independent of the activation format.

Scope. The FP16-relative speedups come from the per-token symmetric backend of Appendix D (CUDA Graph); Table 20 is an eager-mode comparison. The Abstract, Introduction, and Appendix D now name the backend behind each number.

W2. "The novelty relative to OffQ requires sharper justification ... the conceptual distinction is less substantial than the framing suggests.

Thank you for pressing on this. The spectral formulation, full-moment statistics, and compact execution are not add-ons to OffQ. They follow from inverting the problem, from an outlier-driven heuristic to a quantizer-driven optimization, and this changes when and why the rotation works.

OffQ (outlier-driven)PrismQuant (quantizer-driven)
PremiseFind extreme tokens; route their principal direction into one channel per group for the offset to absorbThe range-null subspace S\mathcal{S} is fixed by the group partition; find the orthogonal map that carries the most energy into S\mathcal{S}
StatusA feasible point, set by token selectionClosed-form global optimum (Proposition 1)
Without an offsetNothing absorbs the routed channelFull gain retained (−0.090 PPL, Llama-3.1-8B)

Three controlled measurements:

  1. Optimality matters (Table 7, p. 19). OffQ's top-1 token selection, placed in our pipeline with all else fixed, captures less than half the energy at the down-projection input (f=0.152f=0.152 vs. 0.324), reduces group range less (0.84 vs. 0.75 of Hadamard's), and costs +0.016 PPL on Llama-3.1-8B. It also relies on the BOS token: with BOS excluded, it falls to Hadamard's perplexity.
  2. The mechanism is broader than offsetting (Tables 9, 17). Each aligned direction becomes uniform within one group and zero elsewhere. Under symmetric per-group INT4, with no offset, PrismQuant therefore still gains −0.090 PPL over Hadamard (−0.069 with an offset), and the offset accounts for only 16–40% of the gain at g=128g=128.
  3. End to end (Table 1). On Llama-3.2-3B, where direct comparison is possible, PrismQuant leads in perplexity and accuracy (8.58 / 61.23% vs. 8.78 / 60.80%). It also adds a range law (Sec. 3.5), 70B and MoE results, and a deployed integer checkpoint (W1).

OffQ is an arXiv preprint without public code, so the ICLR 2027 Reviewer Guide does not require a comparison; we agree it is important and have sharpened Section 2 accordingly.

Presentation (score 1). We take this rating to heart and apologize for the effort the submission required. We revised (i) the placement of R2R_2, now identical across Sec. 3.4, Alg. 1, and App. A.1; (ii) optimality, scoped to the alignment objective (Sec. 3.2, 4.3); (iii) the backend, named with every timing; (iv) the ablation protocols (App. A.1). We would be grateful for pointers to any passage that remains hard to follow. We hope the deployed measurement and the sharpened comparison with OffQ resolve your two concerns. If they do, we would be grateful if you would consider raising your score, and we are glad to clarify anything further.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Comment to All Reviewers

Summary of the Revision (to All Reviewers)

We thank all four reviewers for their careful and constructive feedback. The reviews recognize our quantizer-geometry formulation and its closed-form optimum (gL7s, NeUz, sDQh) as well as the clarity of our motivation (Hdp7). The primary concerns raised are: (1) whether the reported accuracy gains and inference speedups hold simultaneously on the same deployed model (gL7s, sDQh, NeUz), (2) the conceptual and empirical distinction from OffQ and the specific contribution of each mechanism (gL7s, NeUz), and (3) direct comparisons with FlatQuant and DuQuant alongside insights into per-task results (sDQh, Hdp7).

We address all three points with new experimental evidence in Appendix F (pp. 39–44, highlighted in blue). Revisions throughout the main text and earlier appendices are also marked in blue.

LocationNew Evidence & AnalysesAddressed To
App. F.1, Tables 15–16 (pp. 39–40), Eq. 41The Llama-3.1-8B checkpoint from Table 1 (real GPTQ weights, group-128 asymmetric activations) deployed with integer kernels: 6.99 PPL (identical to Table 1), 317 ms prefill, and 16.75 tokens/s decode. Latency benchmarks vs. FP16 repeated with real weights.gL7s, sDQh, NeUz
App. F.2, Table 17 (p. 41)Mechanism attribution on Llama-3.1-8B isolating affine offset absorption vs. intra-group flattening and energy localization.NeUz, gL7s
App. F.3, Tables 18–20 (pp. 42–43)FlatQuant and DuQuant evaluated from their official repositories: WikiText-2 PPL, 8 zero-shot tasks, and deployed inference on the same GPU.sDQh, Hdp7
App. F.4, Table 21 (p. 44)Task-by-task statistical comparison against published baselines.Hdp7

Summary of Text Revisions:

  • Abstract, Sec. 1, Sec. 5, and App. D explicitly state the backend behind each timing measurement and reference the deployed group-128 asymmetric checkpoint.
  • Sec. 2 sharpens the distinction from outlier-driven rotations (OffQ) with supporting empirical measurements.
  • Sec. 3.4, Sec. 4.3, Alg. 1, and App. A.1 unify the description of R2R_2, clarify that theoretical optimality holds with respect to the alignment objective, and detail the ablation protocols.
AuthorsOpenReview ↗

A brief reminder before the discussion period closes

Dear Reviewers,

Thank you again for your careful and constructive reviews. As the discussion period closes on November 18 AOE, we are writing with a brief reminder that our responses and the revised PDF are available.

We would be grateful if you could let us know whether the revision resolves the concerns you raised, or whether anything remains that we could clarify in the remaining time.

Thank you for your time and effort.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.