- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that residual-stream rotation R1 requires fully fusing the RMSNorm scale γ into the following weights, which can enlarge within-block magnitude disparities and increase MXFP4 quantization error. The proposed GaBR permutes channels by their cross-layer mean |γ| so that similarly scaled channels fall into the same 32-channel block, factors γ into a block-constant scale G retained in RMSNorm and a residual D fused into the weights, which stays exactly equivalent under 32×32 block-diagonal rotations, and alternately optimizes G and the block rotation. Applied to OSTQuant and PeRQ, GaBR is evaluated on seven Llama and Qwen3 models under MXFP4 W4A4 and reports improvements in WikiText-2 perplexity and seven-task average accuracy over its parent methods on most models.
Critical strengths
The paper identifies a concrete, format-specific issue: fusing the RMSNorm scale γ into downstream weights can increase MXFP4 quantization error. This observation is relevant to any offline rotation method targeting MXFP4.
The proposed factorization is exactly function-preserving and adds no inference-time operators, since a block-constant scale commutes with block-diagonal rotations. This makes the method easy to plug into existing pipelines such as OSTQuant and PeRQ.
Critical weaknesses
Compared with OSTQuant, GaBR-OST simultaneously applies block rotation, channel permutation, and G, but no result isolates the effect of block rotation alone. The ablations are also limited to four and three models, and their results are inconsistent. It is therefore unclear whether the improvements come from GaBR’s components or from block rotation itself, which weakens the paper’s main claim.
Questions for authors
-
Tables 4 and 7 show that the benefit of the retained block scale G is not consistent across models. Since the Γ = GD factorization ablation is conducted on only four models, please extend it to all seven models and explain in which cases retaining G is beneficial.
-
Tables 5 and 8 show that the effect of the gamma-aware permutation is not consistent across models. Since the gamma-permutation comparison is conducted on only three models, please report the results for all seven models.
-
Compared with OSTQuant + Qronos, GaBR-OST simultaneously introduces the 32 × 32 block-diagonal R1, the gamma-aware permutation, and the retained block scale G, so the source of the improvement cannot be identified. Please add results for OSTQuant + Qronos with only the block-diagonal R1 to separate the contribution of block rotation itself from that of the GaBR components.
-
DuQuant++ is excluded from the main comparison (Table 3) because its additional online activation transforms result in a different inference cost, but the paper reports no measurements of latency, FLOPs, or quantization time. Please measure the inference latency and offline quantization time of GaBR and DuQuant++ on the same hardware.
4 comments
Thanks for the additional experiments. The additional results show that G lowers PPL on all seven models. However,m the effect of the permutation is still inconsistent across models, and on some models block rotation alone gives similar or better results, so the contribution of the GaBR components seems limited. Regarding DuQuant++, GaBR avoids online transforms, but its offline preparation takes about five times longer, and its Mean 7 is still lower on most models. The reported inference advantage is also measured in BF16 rather than in W4A4. I will keep my score.
Thank you for your feedback. GaBR fuses its rotation offline, adding no online computation to its parent PeRQ or OSTQuant pipeline. Our BF16 comparison isolates online-transform overhead, while efficient native W4A4 kernels can provide further inference acceleration.
Although offline preparation takes about five times longer in our comparison, this one-time cost is amortized over deployment and does not affect inference latency.
Your subsequent comment discusses RFS, EWGS, and Fourier-derived gradients, which are not part of our submission. Could you please check whether it was intended for another paper?
Sorry for the confusion. My previous comment about RFS, EWGS, and Fourier-derived gradients was meant for another paper and was posted here by mistake. Please disregard it.
Extended Ablations and Efficiency Evaluation
Thank you for your helpful suggestions on the ablations and efficiency evaluation. We provide additional results and clarify the experimental settings below.
Q1. Effect of retaining G across models
We now report the G ablation on all seven models under W4A4 MXFP4 with Qronos.
Retaining G improves performance on most of the tested models.
Q2. Gamma-permutation comparison across models
We expanded Table 5 under W4A4 MXFP4 with Qronos to cover all four orderings on all seven models.
Each entry reports PPL ↓ / Mean7 (%) ↑.
Although performance varies across orderings, Gamma-sort remains competitive across models and achieves the best Mean7 on several models.
Q3. OSTQuant + Qronos with only block-diagonal R1
We added the requested control by replacing only dense R1 with independent 32×32 orthogonal blocks in OSTQuant + Qronos. The control keeps the original 100-step training procedure, channel order, and full γ fusion, without Gamma-sort or retained G.
Each entry reports PPL ↓ / Mean7 (%) ↑.
Full GaBR achieves lower PPL on five of seven models and higher Mean7 on five of seven, indicating benefits beyond block rotation.
Q4. Same-hardware latency and offline quantization time
We evaluated Qwen3-1.7B on the same hardware.
Although GaBR requires longer preparation, this cost is incurred entirely offline. These timings include different reconstruction backends, rather than isolating GaBR’s incremental cost.
GaBR’s residual-stream R1 is fused into the weights offline, adding no independent online operator. We measured additional online-transform overhead using the same BF16 model graph and KV-cache settings, excluding down-projection transforms from both paths.
Online timing uses a 512-token input prefix and 32 cached one-token decoding steps. TPOT excludes prefix prefill and reports the median of five trials. This software comparison measures additional online-transform overhead, not native W4A4 deployment latency.