acceptodds
Under review as a conference paper at ICLR 2027

NADA: Unlocking Fast and Accurate NVFP4 PTQ through Distortion-Aware Optimization

Abstract

NVFP4 provides a wide effective dynamic range while retaining hardware-native FP4 computation on NVIDIA Blackwell GPUs, enabling high quantization fidelity with efficient low-precision inference. However, accurate NVFP4 post-training quantization (PTQ) remains challenging, particularly for activations. Simply suppressing outliers does not guarantee lower activation quantization error, largely because the nonuniform E2M1 codebook induces uneven distortion. We identify that NVFP4 activation reconstruction is governed by three interdependent and sequentially coupled decisions at the tensor, channel, and block levels. This paper proposes an NVFP4 Activation Distortion-Aware (NADA) optimization framework for fast and accurate NVFP4 PTQ. NADA consists of tensor-level Distortion-Aware Activation Channel Clustering (DACC), channel-level Bounded Iterative Top- Rescaling (BITR), and block-level Analytical Block-Scale Optimization (ABSO). DACC and BITR are calibrated offline to reorganize activation channels and selectively rescale sensitive channels, respectively, while ABSO performs a search-free analytical scale update for dynamic activation blocks at runtime. Across eight models ranging from 135M to 70B, NADA achieves state-of-the-art accuracy while preserving high end-to-end inference efficiency. Code is available at https://anonymous.4open.science/r/NADA-5B92.

Primary area:
optimization
Keywords:
Large Language Models, Post-Training Quantization, NVFP4, Weight-Activation Quantization, Microscaling Data Formats, Distortion-Aware Quantization, Efficient LLM Inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wPGYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents NADA, a distortion-aware post-training quantization framework for NVFP4 W4A4 inference. The authors observe that reducing activation outliers or dynamic range does not necessarily reduce quantization error due to the non-uniform E2M1 codebook. To address this issue, NADA introduces a three-stage optimization framework: (1) Distortion-Aware Activation Channel Clustering (DACC), which reorganizes channels to reduce block-level quantization distortion; (2) Bounded Iterative Top-k Rescaling (BITR), which selectively rescales important channels to balance activation and weight quantization errors; and (3) Analytical Block-Scale Optimization (ABSO), which performs a lightweight, closed-form scale refinement for dynamic activation blocks at runtime. The first two stages are calibrated offline, while ABSO is integrated into the online activation quantization kernel. Experiments across eight language models ranging from 135M to 70B parameters demonstrate improved perplexity and downstream-task accuracy compared with several NVFP4 PTQ baselines. Ablation studies evaluate the contributions of individual components, while end-to-end inference experiments demonstrate competitive throughput in vLLM and a 1.22–1.29× speedup over the official Four-Over-Six implementation in the reported Transformers setup.

Critical strengths
  1. Well-motivated, distortion-aware optimization for NVFP4 quantization. The paper identifies an important limitation of conventional outlier suppression: reducing activation dynamic range does not necessarily minimize quantization error under the non-uniform E2M1 codebook. The proposed framework addresses this issue through channel clustering, selective rescaling, and block-scale refinement. In particular, the mechanism-level experiments demonstrate that DACC reduces quantization distortion compared with several alternative channel-ordering strategies, providing empirical support for the proposed design.

  2. Comprehensive experimental validation with practical deployment results. The paper evaluates eight models ranging from 135M to 70B parameters, including perplexity, downstream-task accuracy, and component-level ablations. Importantly, the authors also provide end-to-end inference measurements that account for the additional online optimization overhead. The reported 1.22–1.29× throughput improvement over the official Four-Over-Six implementation, together with competitive vLLM performance, strengthens the practical significance of the proposed approach.

Critical weaknesses
  1. Limited methodological novelty. The proposed framework largely builds upon existing techniques, including channel permutation, activation-weight rescaling, and analytical block-scale optimization. In particular, the closed-form scale update in ABSO is mathematically equivalent to the fixed-code least-squares optimization previously introduced in SOAR. While the integration of these techniques is practically meaningful, the paper does not sufficiently establish the novelty of its individual components or clearly distinguish its contributions from closely related work.

  2. Insufficiently controlled comparisons to substantiate the claimed accuracy–efficiency advantages. Although the paper reports promising results, it lacks direct comparisons with recent efficient block-scale optimization methods such as ScaleSearch and ScaleSweep. Furthermore, the reported throughput improvements over Four-Over-Six involve different quantization implementations, making it difficult to isolate the benefits of the proposed scale optimization algorithm. Given that accuracy improvements are relatively small on some larger models, these missing comparisons weaken the evidence for a substantial advantage over existing methods.

Questions for authors
  1. Novelty of ABSO. SOAR already introduces a closed-form least-squares block-scale update under fixed code assignments, followed by E4M3 projection and code reassignment. Could the authors clarify the methodological differences between ABSO and SOAR, beyond applying this approach to dynamic activation quantization?

  2. Comparison with existing block-scale optimization methods. Recent methods such as ScaleSearch and ScaleSweep also investigate efficient NVFP4 block-scale optimization. Could the authors clarify why these methods were not included in the experimental comparisons and how NADA's accuracy–efficiency trade-off compares with them?

  3. Attribution of inference speedup. The paper reports a 1.22–1.29× throughput improvement over Four-Over-Six, while the two approaches use different activation-quantization implementations. Could the authors clarify how much of this improvement comes from the proposed ABSO algorithm rather than differences in kernel implementation and fusion strategies?

3 comments

We thank the reviewer for recognizing the distortion-aware motivation, the DACC mechanism analysis and the practical deployment evaluation. Based on your feedback, we have (1) clarified NADA’s contributions and differences from prior work; (2) expanded baseline and five-seed comparisons and highlighted NADA’s accuracy gains on compact models in the main results; and (3) added vLLM benchmarks, arithmetic-work analysis, operator measurements, and overhead breakdowns. Revisions are highlighted in blue, with rearranged results and added recovery scores in green. We hope these analyses address your concerns and that you will reconsider your assessment and score. We welcome follow-up questions.

W1 & Q1: NADA’s contributions and the relationship to SOAR

We clarify that NADA targets codebook distortion that range reduction alone cannot resolve. Under NVFP4’s nonuniform E2M1 codebook, the error depends on which channels share a scale, how they are rescaled and how the runtime block scale is chosen. NADA addresses these coupled decisions with different, inexpensive surrogate objectives. Table 29 in Appendix Q provides the detailed comparison; the main distinctions are summarized below.

ComponentSpecific issueNADA’s contribution
DACCMagnitude-based grouping can place values in high-distortion regions of the shared E2M1 grid.Groups channels using a codebook-distortion proxy and token-wise cluster maxima. Unlike the joint second-moment sorting in PermuQuant, this explicitly accounts for E2M1 codebook alignment and the dynamically changing shared scale during grouping.
BITRFull output reconstruction for channel-scale candidates increases calibration cost.Introduces a bounded, surrogate-based search restricted to influential channels. To our knowledge, it is the first to use a normalized activation–weight reconstruction surrogate, thereby reducing the cost of AWQ-style search.
ABSOSearching or repeatedly refining activation scales adds recurring inference work.To the best of our knowledge, the first to employ a single analytical update after DACC/BITR. It fuses the remaining activation transformations with quantization and packing, avoiding candidate-scale enumeration.

ABSO's role is to refine dynamic activation scales with a single analytical update after DACC and BITR, whereas SOAR performs iterative weight-scale optimization. The single-step budget and fused implementation account for the latency and memory constraints of runtime activations. Table 10 in Appendix H and Tables 25–26 in Appendix O show the resulting balance between accuracy and efficiency.

W2 & Q2: Comparison coverage and experimental control

  1. Additional baselines. We added comparisons with ScaleSweep and ScaleSearch. These methods had not released code at the time of our original submission. We recently obtained ScaleSweep’s official code and vLLM implementation. Tables 18–22 in Appendix N include the additional accuracy comparisons, while Tables 23–24 in Appendix O show NADA’s efficiency advantage within the same ScaleSweep vLLM framework. ScaleSearch is evaluated using our reproduction and project adaptation.

  2. Controlled efficiency comparison. We clarify that Table 3 compares NADA and Four-Over-Six within the same Hugging Face Transformers framework used by the official Four-Over-Six implementation for native NVFP4 inference. Four-Over-Six’s official vLLM integration uses pseudo-quantization; its optimized real-NVFP4 inference path uses a CUDA quantizer and NVFP4 GEMM under HF Transformers. We use that path for the comparison.

  3. Magnitude of the gains. We clarify that the largest accuracy gains occur on compact models, which are harder to quantize. Larger models retain performance closer to BF16 after quantization, leaving less room for improvement and smaller differences across methods. Tables 18–22 in Appendix N provide five-seed evidence of consistent compact-model improvements, with PPL reductions of up to 6.05%.

Q3: Attribution of inference speedup

We clarify that ABSO reduces the arithmetic work of block-scale selection through one-step analytical update. We have added a work-complexity analysis in Appendix O.1. Under this counting model, ABSO requires 33.3% fewer multiplications and 51.6% fewer additions/subtractions than Four-Over-Six.

We have added operator-level measurements in Table 25 in Appendix O, showing that ABSO achieves the lowest CUDA Graph latency for most tested input shapes. At (M,K)=(2048,14336), ABSO reduces latency by 40.0% compared with Four-Over-Six. We have also added an overhead analysis in Tables 27–28 in Appendix P, separating transformation overhead from fused quantization and GEMM costs.

Dear Reviewer wPGY,

Thank you again for your valuable feedback on our paper. With the discussion phase nearing its deadline, we would like to check if our clarifications have effectively resolved the concerns you raised. If there are any remaining issues, please let us know so we can follow up immediately.

Reviewer_KG2iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NADA, a distortion-aware post-training quantization method that enables NVFP4 W4A4 quantization. It combines three techniques: distortion-aware channel clustering (DACC), selective channel rescaling (BITR), and analytical block-scale optimization (ABSO). Experiments across multiple model families demonstrate improved quantization accuracy while maintaining competitive inference efficiency.

Critical strengths
  1. The proposed methods, DACC, BITR, and ABSO, address complementary aspects of W4A4 quantization. Their integration into a single pipeline is practical.
Critical weaknesses
  1. The novelty is limited. Most of the discussed findings and methodologies are relatively similar to what people have already discussed in previous literatures.

  2. Improvements over the baselines are minimal, and statistical significance is unclear, as shown in Table 1, 2, and 13.

Questions for authors
  1. How does the introduced ABSO method differ from SOAR's proposed method? Meanwhile, you also need to justify the novelty of the other sections.
3 comments

We thank the reviewer for recognizing that DACC, BITR and ABSO address complementary W4A4 quantization problems within a practical pipeline. Following your suggestions, we have (1) clarified each component’s contribution and its relationship to SOAR and other prior work; (2) added operator benchmarks to support ABSO’s accuracy–efficiency trade-off; and (3) added five-seed results and emphasized the experimental evidence of NADA’s accuracy improvements on compact models. Changes are highlighted in blue, with rearranged results and added recovery scores in green. We hope these additions address your concerns and would appreciate your reconsideration of the score. Please let us know if further clarification is needed.

W1 & Q1: Contributions and the relationship to SOAR

We clarify that our contribution lies in identifying how nonuniform distortion under the E2M1 codebook contributes to reconstruction error and proposing targeted methods to reduce it. NADA addresses this problem through distortion-aware channel clustering, selective channel rescaling and single-step runtime scale refinement. The specific contributions are summarized below and detailed in Table 29 in Appendix Q.

ComponentProblem addressedDesign and distinction from related work
DACCSimilar channel magnitudes do not guarantee low distortion when channels share a block scale.Groups channels using a codebook-distortion proxy and token-wise cluster maxima. This accounts for E2M1 codebook alignment and the changing shared scale during grouping, which joint second-moment sorting in PermuQuant does not directly optimize.
BITRRepeated output reconstruction makes broad channel-scale search costly for W4A4 calibration.To our knowledge, we are the first to searches selected influential channels with bounded updates and a normalized activation–weight reconstruction surrogate, reducing the cost of AWQ-style search.
ABSOActivation-scale optimization lies on the inference critical path.To the best of our knowledge, our work is the first to uses one analytical update after DACC/BITR and fuses the remaining activation transformations with quantization and packing, avoiding candidate-scale enumeration.

We clarify that SOAR applies iterative refinement to weight quantization, whereas ABSO uses a single analytical update for runtime activations after offline DACC/BITR preprocessing. Because activation optimization contributes directly to inference latency and memory overhead, ABSO limits the update budget and integrates the update into a fused activation kernel. Table 10 in Appendix H and Tables 25–26 in Appendix O support this accuracy–efficiency trade-off.

W2: Magnitude and statistical reliability of the gains

We clarify that NADA’s gains are most pronounced on smaller models, which are more sensitive to NVFP4 quantization. On larger models, quantized performance is already closer to BF16, leaving smaller differences among methods. The five-seed results in Tables 18–22 in Appendix N and Table R2 show consistent improvements on compact models, with PPL reductions of up to 6.05%.

Table R2: NVFP4 W4A4 perplexity on the two smallest models with calibration seed 0. Lower is better, and the best quantized result in each column is boldfaced.

MethodSmolLM2-135MQwen3-0.6B
WKT2 ↓C4 ↓WKT2 ↓C4 ↓
BF1615.44421.35520.95730.329
RTN21.38928.93026.20536.558
GPTQ21.11828.14825.68035.745
MR-GPTQ21.90629.35826.84636.346
4/6 + AWQ21.24128.72626.10137.471
ARCQuant————
ScaleSearch20.64927.86125.97936.484
PermuQuant-LLM25.03433.15325.03235.698
ScaleSweep-MSE20.77628.06925.28236.330
ScaleSweep-WMSE19.40926.18224.47034.705
NADA18.84125.26623.14633.428
NADA-WMSE-W18.33524.70223.43033.023

Thank you for the response and additional experiments. My main concern regarding methodology novelty and the accuracy improvements are still unresolved. In particular, the distinction between ABSO and SOAR appears to be in the application setting and implementation rather than the underlying methodology. Meanwhile, the benefit is still only large on very small models (<1B), where quantization may not matter most. Therefore, I will maintain my original score.

Thank you for taking the time to review the additional experiments and for explaining where you remain unconvinced.

Methodological novelty. We agree that ABSO shares SOAR’s fixed-code least-squares update, and we do not claim that update as a new estimator. NADA’s methodological contribution also concerns the decisions made before the block scale is chosen. DACC uses estimated E2M1 distortion to decide which channels should share a scale. Under that grouping, BITR iteratively rescales selected channels using a normalized measure of weight and activation reconstruction error. Refining the scale of a fixed block cannot change which channels share that scale or how they are rescaled. NADA makes both decisions during offline calibration so that each runtime activation block needs only one analytical scale update in Table 10 in Appendix H. This is the rationale for the offline–online split: channel grouping and scale search are done once offline, while inference applies the stored transformations and a single scale update. Table 25 in Appendix O confirms that one update has lower operator latency than three across all tested shapes.

Magnitude and relevance of the gains. We agree that the largest gains occur on the smallest models and that improvements on the two 8B models are modest. The gains do, however, extend beyond 1B parameters. On Qwen3-1.7B, NADA-WMSE-W achieves lower perplexity than each of the four added baselines on both WikiText-2 and C4 for all five calibration seeds (Tables 18–22 in appendix N). Standard NADA also achieves 94.77% average recovery across ten downstream tasks, compared with 92.86% for MR-GPTQ, the strongest baseline in Table 2. The individual task scores are reported in Table 11 in appendix I.

Compact-model quantization is useful for local inference on memory-constrained devices, including laptops, personal computers, and in-vehicle computing platforms. NVIDIA’s DRIVE AGX Thor documentation (Table 4), for example, lists NVFP4 support for Qwen2.5-0.5B, a model below 1B parameters. For an in-vehicle assistant that processes user inputs locally, reducing the weight footprint leaves more memory for the KV cache even when the original weights already fit. This can support longer conversations or more concurrent requests. Thank you for raising the question of where these gains matter in practice.

Reviewer_vVHtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NADA, a distortion-aware post-training quantization framework for NVFP4 W4A4 inference of LLMs. This work is motivated by the observation that reducing activation range does not guarantee lower quantization error because the non-uniform E2M1 codebook makes error depend on where normalized values fall relative to the code points. They decompose activation quantization into three coupled decisions and address each with a dedicated stage: an offline channel permutation that groups channels into blocks to reduce codebook distortion, an offline selective rescaling of the most sensitive channels within each block, and an online closed-form block-scale update that replaces candidate-scale search. Evaluation on eight models from 135M to 70B parameters shows consistently lower perplexity and higher downstream recovery than existing NVFP4 PTQ baselines, with gains concentrated on smaller models, while deployment on an RTX 5090D remains within 12% of the fastest quantized baseline under vLLM and achieves 1.22–1.29× the throughput of the official Four-Over-Six implementation.

Critical strengths
  1. The paper offers a clear and well-supported diagnosis of why outlier suppression is insufficient for NVFP4: the non-uniform E2M1 codebook creates distortion regions, so a narrower activation range can still increase error if values land near large code-point gaps. The evidence for this in Figure 1 and Appendix B, which separates weight-side from activation-side degradation after rotation, is specific to FP4 formats rather than inherited from integer-quantization intuition, and it motivates the method directly.
Critical weaknesses
  1. The accuracy improvements are small in absolute terms and their significance is not established. On the 8B models the perplexity gaps to the strongest baseline are a few hundredths, on Llama3-70B NADA is behind ARCQuant on WikiText-2, and the seed study in Appendix M reports standard deviations of up to 0.3 perplexity for NADA alone while no variance is reported for any baseline. Without that, it is difficult to conclude that the ranking in Table 2 reflects a real difference rather than calibration noise, which weakens the central accuracy claim.
  2. The efficiency claim is framed against a favorable comparison. The 1.22–1.29× throughput advantage is measured against Four-Over-Six in a Hugging Face generation loop at a few hundred tokens per second, which is not a representative serving setting, whereas under vLLM NADA is 3–12% slower than RTN, GPTQ, and MR-GPTQ because part of its activation-side transformation cannot be folded offline. Four-Over-Six is not evaluated under vLLM, so the reader cannot tell whether “fast” holds in the setting that matters for deployment.
Questions for authors
  1. The seed study in Appendix M covers only models up to 1.7B. On Qwen3-1.7B, the margin between the seed-averaged NADA result and MR-GPTQ is within one standard deviation on both corpora, and the main table uses the single most favorable seed on WikiText-2; on the 8B models the margins are a few hundredths of perplexity with no variance reported. Could the authors report seed variation for the baselines and for the 8B models, or otherwise show that the ranking in Table 2 is robust?
  2. The throughput advantage over Four-Over-Six is measured in a Hugging Face generation loop, while under vLLM NADA is 3–12% slower than RTN, GPTQ, and MR-GPTQ. Could the authors run Four-Over-Six under the vLLM configuration of Table 9, or provide a kernel-level comparison of the activation-quantization cost of ABSO versus candidate-scale search, so that the efficiency claim can be assessed in a representative serving setting?
  3. ScaleSweep is cited alongside Four-Over-Six as an NVFP4 block-scale method but is not compared, and PermuQuant appears only as an ordering criterion in Appendix E rather than as a complete method. Could the authors explain these omissions or include them in the main comparison?
5 comments

We thank the reviewer for recognizing our practical motivation and the evidence on the NVFP4 codebook distortion. In response to your comments, we have (1) added baseline comparisons and five-seed accuracy results; (2) clarified the Transformers comparison and added vLLM and operator benchmarks; and (3) highlighted the results showing NADA’s accuracy gains on compact models. Revisions are highlighted in blue, with rearranged results and added recovery scores in green. We hope this evidence addresses your concerns and merits reconsideration of your score. We welcome further questions.

W1 & Q1: Magnitude and robustness of the accuracy gains

We respectfully clarify that NADA achieves significant and robust accuracy improvements, particularly on smaller models that are more difficult to quantize. For larger models, quantized performance is already close to the BF16 reference, leaving limited room for further improvement and resulting in relatively small differences across quantization methods.

We added five-seed results in Tables 18–22 in Appendix N to demonstrate the magnitude and robustness of these gains on compact models. NADA achieves PPL reductions of up to 6.05% and consistently outperforms the baselines on these models.

W2 & Q2: Efficiency under vLLM and HF Transformers

We clarify that we conduct efficiency experiments using both vLLM and Hugging Face Transformers to ensure fair comparisons. Four-Over-Six uses the Hugging Face Transformers framework for end-to-end inference. Table 3 evaluates NADA and Four-Over-Six within this same framework.

We added Tables 23–24 in Appendix O to further demonstrate NADA’s substantial efficiency advantage through another fair comparison. Tables 25–26 in Appendix O also demonstrate ABSO’s efficiency advantage at the operator level.

Q3: ScaleSweep and PermuQuant

Because ScaleSweep and PermuQuant had not released their code at the time of the original submission, a reliable and fair comparison was not feasible.

ScaleSweep’s official open-source code was recently released, including its vLLM implementation. We added the corresponding comparison in Tables 18–22 in Appendix N and Table R1. PermuQuant is evaluated using our reproductions and project adaptations. NADA’s WMSE-W variant reduces PPL by up to 6.05% against the strongest added baseline on compact models, while NADA delivers 4.95%–13.48% higher throughput than the faster ScaleSweep variant.

Table R1: NVFP4 W4A4 perplexity on four compact models with calibration seed 0. Lower is better, and the best result in each row is boldfaced.

MethodSmolLM2-135MQwen3-0.6BLlama3.2-1BQwen3-1.7B
WKT2 ↓C4 ↓WKT2 ↓C4 ↓WKT2 ↓C4 ↓WKT2 ↓C4 ↓
ScaleSweep-MSE20.77628.06925.28236.33011.20816.49420.53124.640
ScaleSweep-WMSE19.40926.18224.47034.70510.97416.05817.98323.314
PermuQuant-LLM25.03433.15325.03235.69811.63817.14017.98324.345
NADA18.84125.26623.14633.42810.99615.98517.12023.208
NADA-WMSE-W18.33524.70223.43033.02310.90215.85617.23023.102

I appreciate the authors for the detailed response. For W1&Q1, the added results have addressed my concerns on robustness and significance of the accuracy gain. For W2&Q2, the new experiments in Tabel 23-26 basically resolve my question, but I would appreciate a further explaination as detailed below. For Q3, the comparison with ScaleSweep and PermuQuant shows consistent accuracy advantages of proposed method, and I am good with it.

My remaining concern is, while ABSO exhibits substantial operator-level speedups, NADA however is slower than RTN/GPTQ/MR-GPTQ at larger batch sizes under vLLM. Could the authors briefly analysis and discuss why the operator-level improvements don’t translate into a consistent end-to-end throughput advantage?

Thank you for your timely reply and we are glad to see that the added results address your concerns.

We clarify that ABSO exhibits substantial operator-level speedups compared to search-based scale-selection methods, rather than the range-based AbsMax scale for activation quantization used by RTN/GPTQ/MR-GPTQ. NADA's primary advantage over the AbsMax scaling used in RTN/GPTQ/MR-GPTQ lies in improved quantization accuracy rather than reduced latency.

The activation block-scale strategies considered here fall into three categories:

Table R3: Activation block-scale strategies.

StrategyRepresentative MethodsProcedureTrade-off
Range-based scalingAbsMaxDirectly computes the scale from activation statistics, without explicit optimization.Low computational cost, but potentially higher quantization error due to sensitivity to outliers.
Analytical scalingABSODerives the scale analytically through a closed-form solution with fixed codes, without iterative search.Achieves a favorable balance between reconstruction accuracy and computational efficiency.
Search-based scalingFour-Over-Six, ScaleSearch, ScaleSweepSearches by evaluating candidate values against a reconstruction objective (e.g., quantization MSE).Can achieve higher reconstruction accuracy, but can incur high latency cost.

NADA uses analytical refinement to improve accuracy beyond AbsMax while limiting the latency cost of scale optimization. The following tables show its throughput in Table R4 and perplexity in Table R5 relative to the AbsMax-based baselines.

Table R4: End-to-end throughput under vLLM (output tokens/s, higher is better).

Batch sizeRTN+AbsMaxGPTQ+AbsMaxMR-GPTQ+AbsMaxNADA
1126.50126.24125.97127.40
2249.58248.67248.56253.00
4487.34485.89485.81481.75
8902.10899.54898.98863.96
161604.421603.751601.161482.28

Table R5: NVFP4 W4A4 perplexity. Lower is better.

ModelMetricRTN+AbsMaxGPTQ+AbsMaxMR-GPTQ+AbsMaxNADA
SmolLM2-135MWKT221.38921.11821.90618.841
C428.93028.14829.35825.266
Llama3.2-1BWKT211.52311.06811.22310.996
C416.92316.18316.40015.985
Qwen3-1.7BWKT218.54617.96817.65217.120
C424.16623.89823.45923.208
Llama3-8BWKT26.8696.7746.8646.711
C410.68310.52010.62910.441

NADA achieves up to a 10.78% reduction in PPL compared to AbsMax-based RTN/GPTQ/MR-GPTQ, while incurring at most a 7.61% throughput reduction across the evaluated batch sizes. These results characterize the accuracy-efficiency trade-off of the NADA pipeline. Although analytical scaling can be largely fused with existing operations, fusion cannot completely eliminate its additional computational overhead. As batch size increases, more activation elements must be processed, potentially making this residual overhead more pronounced.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Remarks by the Authors

Dear ACs, SACs, and PCs,

Thank you for the time and care you have devoted to the review process. We would like to summarize what the reviewers recognized in NADA, what we added during rebuttal, and where the remaining disagreement lies.

Reviewer vVHt described our analysis of NVFP4 codebook distortion as a “clear and well-supported diagnosis of why outlier suppression is insufficient for NVFP4.” Reviewer wPGY singled out the mechanism experiments showing that DACC reduces distortion relative to alternative channel orderings and recognized the value of end-to-end measurements that include online quantization overhead. Reviewer KG2i noted that the three components “address complementary aspects of W4A4 quantization” and that “their integration into a single pipeline is practical.” The discussion centered on three questions:

  1. How large and consistent are the accuracy gains? We added five-seed comparisons on six models, including both 8B models, against ScaleSearch, PermuQuant-LLM, and two ScaleSweep variants (Tables 18–22). The largest gains occur on compact models, but they extend beyond 1B parameters. On Qwen3-1.7B, NADA-WMSE-W achieves lower perplexity than all four added baselines on both WikiText-2 and C4 for every seed. Standard NADA also reaches 94.77% recovery across ten downstream tasks on this model, compared with 92.86% for the strongest baseline in Table 2. The revised main results make the compact-model gains more visible, while the appendix retains the larger-model results, including settings where other methods perform better.

  2. What does NADA contribute beyond SOAR’s scale update? We agree that ABSO shares SOAR’s fixed-code least-squares update. NADA also addresses two decisions that refining a fixed block scale cannot change: DACC determines which channels share the scale, and BITR selectively rescales them using both weight and activation reconstruction errors. Table 10 illustrates why these offline decisions matter for inference. On SmolLM2-135M, three online updates give slightly lower average perplexity than one without preprocessing. After DACC and BITR, one update gives lower average perplexity than three (22.054 vs. 22.118). This supports a single online update in the tested setting. Table 29 details the distinctions from related methods.

  3. When does faster scale refinement improve inference throughput? In the controlled Transformers comparison, NADA achieves 1.22–1.29× the throughput of the official Four-Over-Six implementation (Table 3). This is a result for the complete deployment, not a speedup attributable to ABSO alone. In the added Qwen3-8B vLLM benchmark, NADA delivers 4.95%–13.48% higher throughput than the faster ScaleSweep variant at each tested batch size. Against the faster, simpler AbsMax-based RTN/GPTQ/MR-GPTQ configurations, NADA trades some throughput for accuracy at larger batches, with a maximum throughput reduction of 7.61%. The operator measurements, arithmetic analysis, and runtime breakdowns explain why these comparisons give different results (Appendices O–P).

Following the additional experiments and discussion, Reviewer vVHt raised their rating to weak acceptance, while retaining a reservation about the magnitude of the gains. Reviewer KG2i maintained their rating, citing the overlap with SOAR and the concentration of gains on smaller models. We appreciate the reviewers’ careful engagement. Their questions helped us state more precisely both the contribution of the method and the settings in which it offers the clearest benefit.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.