acceptodds
Under review as a conference paper at ICLR 2027

H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference

Abstract

The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language model (LLM) inference. NVFP4's micro-block design, such as a group size of 16, offers strong representational flexibility for capturing local weight distributions and isolating outliers, but it also introduces a large and highly sensitive space of per-group scaling factors. Existing post-training quantization (PTQ) methods primarily focus on refining quantized weight values, leaving this scale-selection step underexplored. To address this gap, we propose H-Scale, a lightweight post-processing method for NVFP4 per-group scale refinement. Instead of minimizing plain weight reconstruction error, H-Scale selects hardware-valid group scales using a diagonal second-order proxy derived from calibration activations, thereby targeting layer output perturbation more directly. It is designed as a drop-in replacement for RTN-style scale selection in diverse NVFP4 pipelines, requires only modest offline calibration, and introduces strictly zero overhead at inference time. Under a fixed evaluation protocol, experiments on mainstream LLMs show that H-Scale generally improves a broad range of NVFP4 baselines and brings several variants closer to the BF16 reference.

Primary area:
foundation or frontier models, including LLMs
Keywords:
AI Infrastructure, LLM Quantization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zZrgOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes H-Scale, a post-training refinement method for NVFP4 weight quantization. Instead of selecting per-group scales by minimizing plain weight reconstruction error, H-Scale searches a local set of hardware-valid scales using a diagonal-Hessian-weighted reconstruction objective derived from calibration activations. The method is designed as a drop-in replacement for RTN-style scale selection and improves the average accuracy of several NVFP4 PTQ pipelines, including GPTQ, 4over6, ArcQuant, GPTAQ, and MR-GPTQ, without changing inference-time cost.

Critical strengths
  • H-Scale combines Hessian-aware error weighting with hardware-valid NVFP4 scale search and can be applied as a lightweight refinement on top of multiple existing PTQ methods.
  • The method generally improves average downstream accuracy across different PTQ baselines, model sizes, and model families.
Critical weaknesses
  • The Hessian proxy is estimated from calibration activations, so the selected scales may be sensitive to the calibration dataset and distribution. The paper studies calibration-set size, but does not sufficiently evaluate robustness to different calibration datasets or domain mismatch.
  • The actual optimization is still a local discrete scale sweep, but change the target to Hessian-based weighted error.
Questions for authors
  • H-Scale relies on activation-derived Hessian statistics. How sensitive is the method to the choice and distribution of the calibration dataset? In particular, does calibration-domain mismatch lead to overfitting or degraded generalization?
  • In Figure 1(b), “Best Scale (theory)” can have a higher weight MSE than H-Scale or Weight-Guided Scale. Could the authors clarify this behavior?
  • The original MR-GPTQ paper reports that standard GPTQ and MR-GPTQ are generally competitive on NVFP4, with standard GPTQ achieving the strongest average recovery in some real-kernel evaluations. In this paper, MR-GPTQ is consistently below GPTQ across several tables, both with and without H-Scale. Could the authors clarify whether this gap is expected under the evaluation setup here and which implementation or configuration differences relative to the original MR-GPTQ evaluation may explain it?
3 comments

Thank you very much for your constructive feedback. We sincerely hope that our responses address your questions and concerns.

Q1: The actual optimization remains a local discrete scale sweep.

R1:

Please see our response R1 to Reviewer 9XUh. We agree with your assessment that the solver is indeed a local discrete search. Nevertheless, we would like to emphasize that we have made unique contributions to NVFP4 that previous work on discrete scale search could not provide.

Q2: Does calibration-domain mismatch cause overfitting or reduced generalization?

R2:

Please see our response R3 to Reviewer XbHx. Domain mismatch affects the absolute error. However, in our three-domain cross-evaluation, H-Scale consistently retains reconstruction gains over uniform scale search and consistently achieves accuracy improvements.

Q3: “Best Scale (theory)” may have a higher weight MSE than H-Scale or Weight-Guided Scale.

R3:

We have updated Section 3.4 of the paper. Please see L253–256 and L265–294.

Best Scale (theory) and Weight-Guided Scale optimize different objectives. The 'Best Scale' aims to shift error toward directions with weaker activations and reduce output error, and it may not reduce weight rounding error. We have explained this in Section 3.4 and the Figure 1 caption, and changed the legend to “Output-scale ref.” to avoid suggesting that “Best Scale” is globally optimal.

Q4: MR-GPTQ consistently achieves lower accuracy than GPTQ.

R4:

Please see our response R2 to Reviewer 9XUh. All our MR-GPTQ settings are consistent with those of the original paper, and the observed gap is consistent with the findings of prior work.

Thanks for the response. I will keep my score since it is positive

Thank you for your follow-up and for maintaining your positive assessment of our work. If you have any further questions, we would be happy to address them.

Reviewer_9XUhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work proposed a Hessian-diagonal guided search to find optimal quantization scales for NVFP4 quantization. Specifically, the method runs grid search over several values below and above the initialization and picks the one with the smallest output error. The H-Scale approach is validated on several LLM from the Qwen family as a complement to different base quantization methods.

Critical strengths
  • The approach is simple and plug-and-play and can be combined with various base quantization methods.
  • The improvements seem to be pretty robust and consistent across models and base quantization methods.
Critical weaknesses
  • The main weakness is the lack of conceptual novelty. Quantization statistic (scales and zeros) search, specifically Hessian-guided, is an established idea with numerous works applying it to quantization. Specifically, NeUQI [1] adopted Hessian diagonal to find optimal zero point. While the target was low-bit integer quantization and optimal zero-point search the overall approach is quite similar. DASH-Q [2] adopts Hessian diagonal to find optimal scale and zero point and optimizes them via alternating optimization. llama.cpp adopts Hessian diagonal to tune the scale LLM-FP4 searches for exponent bias, which parameterizes scale, jointly with FP format, optimizing layerwise reconstruction error. Therefore, the main contribution appears to be adaptation of Hessian scale weighting to NVFP4.

[1] Lin, Li, Xinyu Hu, and Xiaojun Wan. "NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs." arXiv preprint arXiv:2505.17595 (2025).

[2] Kim, Jaemin, et al. "Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate." Proceedings of the Sixth European Workshop on Machine Learning and Systems. 2026.

[3] https://github.com/ggml-org/llama.cpp/pull/4861

[4] Liu, Shih-yang, et al. "Llm-fp4: 4-bit floating-point quantized transformers." Proceedings of the 2023 conference on empirical methods in natural language processing. 2023.

Questions for authors
  • MR-GPTQ is expected to be better in general than GPTQ. Does the MR-GPTQ integration include all components - small Hadamard rotations, MSE scale search and static activation reordering?
  • How does the method perform when combined with activation quantization? The main benefit of using NVFP4 is the use of efficient FP4 GEMMs on Blackwell architecture requiring both weights and activations to be quantized.
  • What are the standard deviations on the provided benchmarks results? Are the differences statistically significant?
3 comments

Q1: Conceptual novelty and practical contributions.

R1:

We respectfully but firmly disagree with the assessment of a lack of conceptual novelty for the following reasons:

  1. We are the first to demonstrate, in NVFP4, that Hessian-based search can directly replace RTN and integrate seamlessly with existing state-of-the-art methods to further improve accuracy. Please note that LLM evaluations inherently involve considerable uncertainty, while NVFP4 itself incurs a relatively small accuracy drop. It is rare for a state-of-the-art PTQ method to consistently outperform other methods on every dataset. Even RTN can outperform conventional state-of-the-art methods on some datasets (please see Table 2, where RTN outperforms ArcQuant and GPTQ). Previous work has not developed Hessian-based search into a plug-and-play module. Our method consistently improves all of these state-of-the-art methods, which is itself a novel contribution and is important for NVFP4.

  2. Please note that the Hessian-based methods you previously mentioned are very difficult to apply to larger LLMs (100B+). They often introduce large-scale search operations, and the enormous search space makes the search unacceptably time-consuming. Our approach requires only a very lightweight search over 16 scales, introducing almost no additional quantization cost. We provide an example on Qwen3.5-397B-A17B, comparing the quantization performance of RTN + H-Scale and SignRoundV2. SignRoundV2 is a Hessian-based method of the type you referred to.

The original quantization-pipeline times for RTN + H-Scale and full SignRoundV2 are 47.60 and 677.88 minutes, respectively. Our method is not only more lightweight but also achieves higher accuracy.

MethodGPQAAIME24AIME25LCB-v5Average
RTN + H-Scale76.43 ± 2.0476.67 ± 5.7755.56 ± 7.7050.18 ± 1.5664.71 ± 2.49
SignRoundV2 (full)76.94 ± 1.2773.33 ± 3.3353.33 ± 6.6747.43 ± 1.6262.76 ± 1.93

We have included the complete results in Section 4.3.1 and Appendix J.

Q2: Components and performance of MR-GPTQ.

R2:

Our full MR-GPTQ control includes all three components: small Hadamard rotations, scale search, and static activation reordering. We have updated the manuscript to clarify this in lines 368–376.

Please note that the original MR-GPTQ paper already explicitly states in its real-kernel evaluation in Section 5: “For NVFP4, standard GPTQ provides the highest recoveries on average”. Subsequent works citing this paper, including KBBQ (Tables 2 and 6), FOCUS (Table 1), and Schur Replay (Tables 1 and 2), have also reported results in which MR-GPTQ underperforms GPTQ under NVFP4.

Q3: Quantization of both weights and activations.

R3:

All experiments in this paper use the W4A4 setting, meaning that both weights and activations are quantized. Please note that Table 13 in Appendix G already specifies this setting; we have also explicitly clarified it in L367 of the revised main text.

Q4: Standard deviations, confidence intervals, and interpretation of results.

R4:

Please note that Appendix A already provides the standard deviations and confidence intervals corresponding to the main tables. The results support the conclusion that our accuracy exceeds that of prior work. We further report standard deviations and confidence intervals in Appendices J–L.

Thank you for your response and clarification. Your comments and the additional results partially address my concerns. However, I still consider the proposed Hessian-based scale to be a relatively incremental contribution, so I have decided to maintain my original score.

Dear Reviewer 9XUh,

Thank you for reading our response and confirming that our clarifications resolved your technical questions on MR-GPTQ, W4A4 evaluation, and statistical significance. While we respect your perspective, we would like to respectfully clarify why we view H-Scale as more than an incremental port of prior curvature-weighted scale methods ([1–4]):

  1. Uncovering a systematic blind spot across modern NVFP4 methods: Recent NVFP4-specific methods (MR-GPTQ, 4over6, ArcQuant, SignRoundV2) introduce substantial complexity—Hadamard rotations, adaptive range modes, or gradient-based rounding/scale tuning—yet still leave consistent accuracy on the table due to suboptimal E4M3 scale selection. Notably, even calibration-light RTN + H-Scale outperforms gradient-tuned SignRoundV2 final (Appendix J), and H-Scale consistently improves every orthogonal PTQ family tested (RTN, GPTQ, GPTAQ, 4over6, ArcQuant) up to 397B MoE models (~48 min). If this were a trivial carryover, these gains would already have been captured by heavier NVFP4 PTQ pipelines.

  2. Why prior scale-optimization formulations ([1–4]) do not directly transfer to hardware-native NVFP4: NeUQI [1] and DASH-Q [2] rely on continuous scale/zero-point updates (closed-form or alternating optimization) on uniform integer grids, while LLM-FP4 [4] searches real-valued channel/tensor exponent biases that depart from native hardware formats. Hardware-native NVFP4 has no zero-point and restricts both weights (E2M1) and g=16 micro-scales (E4M3) to sparse, non-uniform floating-point grids—where continuous optima break down once snapped back to E4M3. Identifying that direct curvature-weighted search over adjacent hardware-valid E4M3 codewords resolves this discrete bottleneck—and integrates cleanly into sequential Cholesky error-compensation loops unlike standalone heuristics [3]—addresses a challenge specific to sub-byte micro-scaling formats.

  3. Simplicity vs. Incrementality under strict hardware constraints: Because Blackwell Tensor Cores strictly fix the W4A4 layout, any runtime-modifying design breaks zero-overhead deployment. Within this rigid design space, achieving broad, composable gains via a minimal formulation is an intentional strength rather than a sign of incremental scope.

Once again, we would like to thank you for your constructive discussion and the valuable reference materials you provided. We look forward to receiving your further response.

Reviewer_XbHxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

H-Scale refines NVFP4 group scales by searching 16 neighboring E4M3 values and minimizing quantization error weighted by the diagonal of the calibration activation Gram matrix. The weighting approximates linear-layer output reconstruction error. The method retains the NVFP4 representation and can replace scale selection within several PTQ pipelines without adding inference operations.

The main weight-only experiments cover Qwen3-4B, LLaMA-3.1-8B, and Qwen3-30A3 models, using 4,096 calibration sequences of length 8,192, or approximately 33.55 million tokens. Appendix G adds W4A4 evaluation of the language component of Qwen3.8-27B. Average benchmark scores improve across the tested pipelines, although the gains vary by task and baseline.

Critical strengths
  1. The intervention is simple and compatible with the target format. Searching representable local scales avoids introducing extra precision, metadata, or runtime branches. The analysis also tests full layer-output error rather than reporting only improvement in the weighted surrogate. Figure 1 gives a useful example where lower weight MSE does not imply better output reconstruction.

  2. The empirical coverage is meaningful for PTQ. The actual method is evaluated on billion-parameter models, including 30B-class MoE checkpoints and a 27B multimodal checkpoint, rather than only on small proxy models. Improvements recur across several quantization pipelines and model families. The matched-candidate ablations in Table 5 and Appendix G are particularly useful, and Appendix A reports decoding variability rather than relying solely on single-run scores.

Critical weaknesses
  1. The distinction from prior activation-weighted scale search is not established. Section 3.4 characterizes SignRoundV2 as unweighted MSE search, but its original version already weights reconstruction error using calibration activations (Section 3.3, Equation 9). H-Scale uses different statistics and hardware-representable candidates, yet the unweighted control does not isolate their value. A matched NVFP4 comparison with prior activation-based weighting is needed to support the claimed conceptual advance. Reference: SignRoundV2, original version.

  2. The specific weighting benefit is less established than the overall refinement gain. Table 5 improves GPTQ from 63.90 to 64.39 with unweighted search and to 64.50 with H-Scale: only 0.11 points come from weighting in that comparison. Other pipelines show larger gains, but calibration-set sensitivity remains untested. Reconstruction diagnostics use calibration data, and three decoding repeats do not measure that sensitivity. Appendix I also leaves score aggregation from the stated AIME and LiveCodeBench beams undefined, limiting interpretation of small differences.

Questions for authors
  1. Can H-Scale and SignRoundV2’s activation-based weights be compared using identical NVFP4 candidates, calibration data, and compensation? Please correct the characterization of the prior objective and isolate the benefit of the proposed statistic.

  2. Does the weighting advantage persist across calibration samples and disjoint reconstruction data? Please report uncertainty for that comparison and specify how AIME and LiveCodeBench candidates are aggregated into scores.

4 comments

Q1: Comparison with SignRoundV2.

R1:

Thank you for your constructive comments. We have corrected our description of SignRoundV2 (lines 105–107 of the paper) and added the corresponding experimental comparisons in Section 4.3.1 and Appendix J.

Please note that our method substantially outperforms SignRoundV2 on Qwen3.5-397B-A17B:

MethodAverageGPQAAIME24AIME25-15LCB-v5Quantization time (min)
Original BF16 model64.17 ± 1.4778.28 ± 1.0177.78 ± 3.8548.89 ± 3.8551.73 ± 1.97--
Plain RTN63.02 ± 2.8875.42 ± 1.0572.22 ± 5.0955.56 ± 10.1848.86 ± 1.4921.54
RTN + H-Scale64.71 ± 2.4976.43 ± 2.0476.67 ± 5.7755.56 ± 7.7050.18 ± 1.5647.60
Full SignRoundV262.76 ± 1.9376.94 ± 1.2773.33 ± 3.3353.33 ± 6.6747.43 ± 1.62677.88

In the NVFP4 settings we tested, SignRoundV2 incurs greater offline optimization cost without showing a statistically significant accuracy advantage.

  1. The full pipeline is substantially more expensive. SignRoundV2 performs 200 gradient-based optimization steps per block, whereas H-Scale directly searches for scales. The SignRoundV2 pipeline takes approximately 14 times as long.

  2. The NVFP4 initialization grid we examined contains redundant computations. SignRoundV2 enumerates 102 multiplicative factors, but many factors map to the same scale after conversion to E4M3. H-Scale directly checks neighboring scales, making candidate evaluation more direct.

  3. The benefits of additional optimization are inconsistent. Please see Appendix J: on Qwen3.8-27B, SignRoundV2 adds tuning of scales and rounding, but no additional accuracy advantage is evident.

Q2: The benefit of weighting itself is less well supported than the overall benefit of scale optimization.

R2:

We strongly disagree with this assessment for the following reasons:

  1. LLM evaluation scores are inherently uncertain. A nonsignificant score improvement on a single state-of-the-art baseline is insufficient to conclude that our gains are limited. On both 4over6 and ArcQuant in Table 5, we obtain substantially higher scores than weight-guided scale search.

  2. Please see Appendix K. We have added experiments on Qwen3.8-27B, and the results clearly demonstrate substantial benefits from weighting itself.

MethodAverageGPQAAIME24AIME25-15LCB-v5IFBench
RTN59.57 ± 0.2871.26 ± 0.5268.89 ± 1.5955.28 ± 1.5856.58 ± 0.7145.84 ± 0.27
Uniform scale search60.57 ± 0.2672.98 ± 0.4572.29 ± 1.5053.33 ± 0.4258.49 ± 0.3845.75 ± 0.23
H-Scale61.01 ± 0.2072.76 ± 0.2572.08 ± 1.4655.83 ± 0.4258.21 ± 0.7946.16 ± 0.17

Q3: Using different calibration samples and reconstruction data.

R3:

We add Appendix L, which evaluates the effect of calibration datasets from three different domains on reconstruction error. The benefit persists within the tested domains. The following table reports the reduction in held-out output error achieved by H-Scale relative to uniform scale search:

Calibration → evaluationWebMathCode
Web35.19%35.43%35.35%
Math33.63%37.22%36.33%
Code32.98%35.93%37.81%

Starting from RTN, we also evaluate all H-Scale models for three seeds in each of the three calibration domains, with the original BF16 model as a reference. The five-task average increases from 59.44 for RTN to 61.09 (Web), 60.79 (Math), and 60.92 (Code), respectively. The downstream results are shown below:

Method / calibrationAverageGPQAAIME24AIME25-15LCB-v5IFBench
BF1662.9774.3974.3858.3360.8646.91
RTN59.4471.8768.5454.1756.7445.87
H-Scale (Web)61.09 ± 0.2672.93 ± 0.4572.64 ± 0.6754.58 ± 1.1059.02 ± 0.1446.27 ± 0.23
H-Scale (Math)60.79 ± 0.5871.67 ± 0.4171.53 ± 0.3255.83 ± 2.5059.00 ± 0.5145.93 ± 0.39
H-Scale (Code)60.92 ± 0.0873.25 ± 0.7072.29 ± 0.7555.28 ± 0.8758.47 ± 1.0245.33 ± 0.14

Q4: The manuscript does not explain how the multiple candidate outputs (beams) for AIME and LiveCodeBench are aggregated into scores.

R4:

We follow the standard evaluation methods for AIME and LiveCodeBench. Please see the description in Appendix J.1. In all our AIME and LiveCodeBench evaluations, each answer is graded independently. We first compute the fraction of correct answers, c/n, for each question, then average equally across questions and report pass@1, without selecting the best answer or using majority voting. In the 27B comparison, each run generates 16 answers per AIME question and 10 answers per GPQA/LCB question.

Dear Reviewer XbHx,

We wanted to briefly check in and see if our revised SignRoundV2 characterization/matched NVFP4 comparisons (Section 4.3.1, Appendix J) and cross-calibration/disjoint reconstruction analyses addressed your questions. Please let us know if any further clarification or analysis would be helpful—we would be glad to provide it. Thank you again for your time and thoughtful review.

Thank you for the additional experiments and clarifications. The held-out reconstruction and cross-calibration results address several evaluation concerns, but my central concern remains unresolved: comparing RTN + H-Scale with full SignRoundV2 does not isolate the contribution of the proposed weighting statistic. A comparison with prior activation-based weighting under identical scale candidates, calibration data, and compensation remains necessary. The new downstream ablation shows a 0.44-point average gain over uniform search, concentrated mainly in AIME25, rather than a consistent improvement across tasks.

There is also an unresolved setting discrepancy: the original Section 4.1 describes weight-only quantization, whereas the rebuttal states that all experiments use W4A4. Please clarify which setting produced the original tables and whether any results were rerun. Overall, the rebuttal strengthens the evidence for practical utility but does not sufficiently establish the distinct contribution. I therefore maintain my rating of 2 (Weak rejection).

Thank you for your response.

  1. Comparison with SignRoundV2. We futher add experiments on Qwen3.5-397B-A17B comparing H-Scale with weight-only guided scale search and full SignRoundV2. As shown below, H-Scale achieves higher average accuracy than weight-only guided scale search in this setting.
MethodAverageGPQAAIME24AIME25-15LCB-v5Quantization pipeline time (min)Block-solver time (min)
Original BF16 model64.17 ± 1.2478.28 ± 1.0177.78 ± 3.8548.89 ± 3.8551.73 ± 1.97----
Plain RTN63.02 ± 3.8775.42 ± 1.0572.22 ± 5.0955.56 ± 10.1848.86 ± 1.4921.542.46
RTN + weight-only guide search62.93 ± 2.5774.05 ± 1.0771.88 ± 2.0954.56 ± 9.4851.23 ± 2.1925.5414.56
RTN + H-Scale64.71 ± 1.6676.43 ± 2.0476.67 ± 5.7755.56 ± 7.7050.18 ± 1.5647.6015.42
Full SignRoundV262.76 ± 1.8476.94 ± 1.2773.33 ± 3.3353.33 ± 6.6747.43 ± 1.62377.88345.04

We also acknowledge the substantial variability inherent in LLM evaluation. Nevertheless, these results demonstrate H-Scale's practical advantages over full SignRoundV2 in offline quantization cost and observed average accuracy.

  1. Clarification of the quantization setting. In the original manuscript, we intended to emphasize the integration of H-Scale into the weight quantization stage of NVFP4 pipelines, while the evaluation was conducted under W4A4. We acknowledge that our previous wording was imprecise, and have revised the manuscript to state this setting clearly and remove the ambiguity.

We appreciate your constructive feedback and look forward to your response. We would be happy to provide further results to address any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their careful reading and constructive feedback. We have revised the manuscript and expanded the experiments to address the concerns about H-Scale's contribution, comparisons with prior methods, the benefit of weighting, calibration robustness, and evaluation details. Below, we summarize the main clarifications and additions; the individual responses provide the detailed results and references.

  1. Clarifying the contribution of H-Scale. Our contribution is a lightweight, plug-and-play Hessian-guided scale-search module for NVFP4 that can directly replace RTN-style scale selection and integrate with existing PTQ pipelines to further improve accuracy. By evaluating only 16 candidate scales, H-Scale keeps the search lightweight and makes this refinement practical for large models, including Qwen3.5-397B-A17B. We respectfully maintain that this integration and scalability constitute a meaningful NVFP4-specific contribution.

  2. Expanding the comparison with SignRoundV2. We have corrected the description of SignRoundV2 and added comparisons on Qwen3.8-27B and Qwen3.5-397B-A17B in Section 4.3.1 and Appendix J. On the 397B model, RTN + H-Scale achieves a four-task mean of 64.71, compared with 62.76 for full SignRoundV2, while requiring less offline quantization time. The comparison also highlights practical differences: full SignRoundV2 performs 200 gradient-based optimization steps per block, and its initialization enumerates 102 multiplicative factors, many of which map to identical E4M3 scales. H-Scale directly checks neighboring scales. The additional tuning in SignRoundV2 does not consistently produce an accuracy advantage in our comparisons.

  3. Evaluating weighting separately and testing calibration robustness. Appendix K adds a direct comparison of RTN, uniform scale search, and H-Scale on Qwen3.8-27B, complementing the 4over6 and ArcQuant results in Table 5. Appendix L evaluates three calibration domains, with three calibration seeds per domain and reconstruction data excluded from calibration. Across all nine calibration/evaluation-domain combinations, H-Scale reduces held-out output error by 32.98%–37.81% relative to uniform scale search. The corresponding five-task averages improve from 59.44 for RTN to 61.09, 60.79, and 60.92 for Web-, Math-, and Code-calibrated H-Scale, respectively. These experiments address both the contribution of weighting and its behavior under the tested calibration changes.

  4. Clarifying the quantization setting and MR-GPTQ configuration. We explicitly state the W4A4 setting, in which both weights and activations are quantized. We also clarify that our full MR-GPTQ control includes small Hadamard rotations, scale search, and static activation reordering. As detailed in our response to Reviewer 9XUh, the original MR-GPTQ paper and subsequent NVFP4 studies also report settings in which standard GPTQ outperforms MR-GPTQ.

  5. Explaining score aggregation and result interpretation. Appendix J.1 specifies that candidates are graded independently: each question receives its correct fraction, c/n, and questions are averaged equally. The 27B comparison uses 16 candidates per AIME question and 10 per LiveCodeBench question, without best-candidate selection or majority voting. We also clarify the reporting of standard deviations and confidence intervals. Section 3.4 and the Figure 1 caption explain why optimizing output reconstruction can increase ordinary weight MSE, and the legend now uses “Output-scale ref.” to avoid implying globally optimal scales.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.