- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes H-Scale, a post-training refinement method for NVFP4 weight quantization. Instead of selecting per-group scales by minimizing plain weight reconstruction error, H-Scale searches a local set of hardware-valid scales using a diagonal-Hessian-weighted reconstruction objective derived from calibration activations. The method is designed as a drop-in replacement for RTN-style scale selection and improves the average accuracy of several NVFP4 PTQ pipelines, including GPTQ, 4over6, ArcQuant, GPTAQ, and MR-GPTQ, without changing inference-time cost.
Critical strengths
- H-Scale combines Hessian-aware error weighting with hardware-valid NVFP4 scale search and can be applied as a lightweight refinement on top of multiple existing PTQ methods.
- The method generally improves average downstream accuracy across different PTQ baselines, model sizes, and model families.
Critical weaknesses
- The Hessian proxy is estimated from calibration activations, so the selected scales may be sensitive to the calibration dataset and distribution. The paper studies calibration-set size, but does not sufficiently evaluate robustness to different calibration datasets or domain mismatch.
- The actual optimization is still a local discrete scale sweep, but change the target to Hessian-based weighted error.
Questions for authors
- H-Scale relies on activation-derived Hessian statistics. How sensitive is the method to the choice and distribution of the calibration dataset? In particular, does calibration-domain mismatch lead to overfitting or degraded generalization?
- In Figure 1(b), “Best Scale (theory)” can have a higher weight MSE than H-Scale or Weight-Guided Scale. Could the authors clarify this behavior?
- The original MR-GPTQ paper reports that standard GPTQ and MR-GPTQ are generally competitive on NVFP4, with standard GPTQ achieving the strongest average recovery in some real-kernel evaluations. In this paper, MR-GPTQ is consistently below GPTQ across several tables, both with and without H-Scale. Could the authors clarify whether this gap is expected under the evaluation setup here and which implementation or configuration differences relative to the original MR-GPTQ evaluation may explain it?
3 comments
Thanks for the response. I will keep my score since it is positive
Thank you for your follow-up and for maintaining your positive assessment of our work. If you have any further questions, we would be happy to address them.
Thank you very much for your constructive feedback. We sincerely hope that our responses address your questions and concerns.
Q1: The actual optimization remains a local discrete scale sweep.
R1:
Please see our response R1 to Reviewer 9XUh. We agree with your assessment that the solver is indeed a local discrete search. Nevertheless, we would like to emphasize that we have made unique contributions to NVFP4 that previous work on discrete scale search could not provide.
Q2: Does calibration-domain mismatch cause overfitting or reduced generalization?
R2:
Please see our response R3 to Reviewer XbHx. Domain mismatch affects the absolute error. However, in our three-domain cross-evaluation, H-Scale consistently retains reconstruction gains over uniform scale search and consistently achieves accuracy improvements.
Q3: “Best Scale (theory)” may have a higher weight MSE than H-Scale or Weight-Guided Scale.
R3:
We have updated Section 3.4 of the paper. Please see L253–256 and L265–294.
Best Scale (theory) and Weight-Guided Scale optimize different objectives. The 'Best Scale' aims to shift error toward directions with weaker activations and reduce output error, and it may not reduce weight rounding error. We have explained this in Section 3.4 and the Figure 1 caption, and changed the legend to “Output-scale ref.” to avoid suggesting that “Best Scale” is globally optimal.
Q4: MR-GPTQ consistently achieves lower accuracy than GPTQ.
R4:
Please see our response R2 to Reviewer 9XUh. All our MR-GPTQ settings are consistent with those of the original paper, and the observed gap is consistent with the findings of prior work.