- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents NADA, a distortion-aware post-training quantization framework for NVFP4 W4A4 inference. The authors observe that reducing activation outliers or dynamic range does not necessarily reduce quantization error due to the non-uniform E2M1 codebook. To address this issue, NADA introduces a three-stage optimization framework: (1) Distortion-Aware Activation Channel Clustering (DACC), which reorganizes channels to reduce block-level quantization distortion; (2) Bounded Iterative Top-k Rescaling (BITR), which selectively rescales important channels to balance activation and weight quantization errors; and (3) Analytical Block-Scale Optimization (ABSO), which performs a lightweight, closed-form scale refinement for dynamic activation blocks at runtime. The first two stages are calibrated offline, while ABSO is integrated into the online activation quantization kernel. Experiments across eight language models ranging from 135M to 70B parameters demonstrate improved perplexity and downstream-task accuracy compared with several NVFP4 PTQ baselines. Ablation studies evaluate the contributions of individual components, while end-to-end inference experiments demonstrate competitive throughput in vLLM and a 1.22–1.29× speedup over the official Four-Over-Six implementation in the reported Transformers setup.
Critical strengths
-
Well-motivated, distortion-aware optimization for NVFP4 quantization. The paper identifies an important limitation of conventional outlier suppression: reducing activation dynamic range does not necessarily minimize quantization error under the non-uniform E2M1 codebook. The proposed framework addresses this issue through channel clustering, selective rescaling, and block-scale refinement. In particular, the mechanism-level experiments demonstrate that DACC reduces quantization distortion compared with several alternative channel-ordering strategies, providing empirical support for the proposed design.
-
Comprehensive experimental validation with practical deployment results. The paper evaluates eight models ranging from 135M to 70B parameters, including perplexity, downstream-task accuracy, and component-level ablations. Importantly, the authors also provide end-to-end inference measurements that account for the additional online optimization overhead. The reported 1.22–1.29× throughput improvement over the official Four-Over-Six implementation, together with competitive vLLM performance, strengthens the practical significance of the proposed approach.
Critical weaknesses
-
Limited methodological novelty. The proposed framework largely builds upon existing techniques, including channel permutation, activation-weight rescaling, and analytical block-scale optimization. In particular, the closed-form scale update in ABSO is mathematically equivalent to the fixed-code least-squares optimization previously introduced in SOAR. While the integration of these techniques is practically meaningful, the paper does not sufficiently establish the novelty of its individual components or clearly distinguish its contributions from closely related work.
-
Insufficiently controlled comparisons to substantiate the claimed accuracy–efficiency advantages. Although the paper reports promising results, it lacks direct comparisons with recent efficient block-scale optimization methods such as ScaleSearch and ScaleSweep. Furthermore, the reported throughput improvements over Four-Over-Six involve different quantization implementations, making it difficult to isolate the benefits of the proposed scale optimization algorithm. Given that accuracy improvements are relatively small on some larger models, these missing comparisons weaken the evidence for a substantial advantage over existing methods.
Questions for authors
-
Novelty of ABSO. SOAR already introduces a closed-form least-squares block-scale update under fixed code assignments, followed by E4M3 projection and code reassignment. Could the authors clarify the methodological differences between ABSO and SOAR, beyond applying this approach to dynamic activation quantization?
-
Comparison with existing block-scale optimization methods. Recent methods such as ScaleSearch and ScaleSweep also investigate efficient NVFP4 block-scale optimization. Could the authors clarify why these methods were not included in the experimental comparisons and how NADA's accuracy–efficiency trade-off compares with them?
-
Attribution of inference speedup. The paper reports a 1.22–1.29× throughput improvement over Four-Over-Six, while the two approaches use different activation-quantization implementations. Could the authors clarify how much of this improvement comes from the proposed ABSO algorithm rather than differences in kernel implementation and fusion strategies?
3 comments
Q3: Attribution of inference speedup
We clarify that ABSO reduces the arithmetic work of block-scale selection through one-step analytical update. We have added a work-complexity analysis in Appendix O.1. Under this counting model, ABSO requires 33.3% fewer multiplications and 51.6% fewer additions/subtractions than Four-Over-Six.
We have added operator-level measurements in Table 25 in Appendix O, showing that ABSO achieves the lowest CUDA Graph latency for most tested input shapes. At (M,K)=(2048,14336), ABSO reduces latency by 40.0% compared with Four-Over-Six. We have also added an overhead analysis in Tables 27–28 in Appendix P, separating transformation overhead from fused quantization and GEMM costs.
Dear Reviewer wPGY,
Thank you again for your valuable feedback on our paper. With the discussion phase nearing its deadline, we would like to check if our clarifications have effectively resolved the concerns you raised. If there are any remaining issues, please let us know so we can follow up immediately.
We thank the reviewer for recognizing the distortion-aware motivation, the DACC mechanism analysis and the practical deployment evaluation. Based on your feedback, we have (1) clarified NADA’s contributions and differences from prior work; (2) expanded baseline and five-seed comparisons and highlighted NADA’s accuracy gains on compact models in the main results; and (3) added vLLM benchmarks, arithmetic-work analysis, operator measurements, and overhead breakdowns. Revisions are highlighted in blue, with rearranged results and added recovery scores in green. We hope these analyses address your concerns and that you will reconsider your assessment and score. We welcome follow-up questions.
W1 & Q1: NADA’s contributions and the relationship to SOAR
We clarify that NADA targets codebook distortion that range reduction alone cannot resolve. Under NVFP4’s nonuniform E2M1 codebook, the error depends on which channels share a scale, how they are rescaled and how the runtime block scale is chosen. NADA addresses these coupled decisions with different, inexpensive surrogate objectives. Table 29 in Appendix Q provides the detailed comparison; the main distinctions are summarized below.
ABSO's role is to refine dynamic activation scales with a single analytical update after DACC and BITR, whereas SOAR performs iterative weight-scale optimization. The single-step budget and fused implementation account for the latency and memory constraints of runtime activations. Table 10 in Appendix H and Tables 25–26 in Appendix O show the resulting balance between accuracy and efficiency.
W2 & Q2: Comparison coverage and experimental control
Additional baselines. We added comparisons with ScaleSweep and ScaleSearch. These methods had not released code at the time of our original submission. We recently obtained ScaleSweep’s official code and vLLM implementation. Tables 18–22 in Appendix N include the additional accuracy comparisons, while Tables 23–24 in Appendix O show NADA’s efficiency advantage within the same ScaleSweep vLLM framework. ScaleSearch is evaluated using our reproduction and project adaptation.
Controlled efficiency comparison. We clarify that Table 3 compares NADA and Four-Over-Six within the same Hugging Face Transformers framework used by the official Four-Over-Six implementation for native NVFP4 inference. Four-Over-Six’s official vLLM integration uses pseudo-quantization; its optimized real-NVFP4 inference path uses a CUDA quantizer and NVFP4 GEMM under HF Transformers. We use that path for the comparison.
Magnitude of the gains. We clarify that the largest accuracy gains occur on compact models, which are harder to quantize. Larger models retain performance closer to BF16 after quantization, leaving less room for improvement and smaller differences across methods. Tables 18–22 in Appendix N provide five-seed evidence of consistent compact-model improvements, with PPL reductions of up to 6.05%.