acceptodds
Under review as a conference paper at ICLR 2027

MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention

Abstract

The quadratic cost of attention limits diffusion-based video generation. MXFP4 attention offers a promising path toward lower-cost inference, but direct quantization often degrades generation quality. We identify two numerical causes: power-of-two shared scaling creates a clipping–underflow trade-off within MXFP4 blocks, while quantization in the online softmax loop breaks row-wise normalization, so the attention weights no longer sum to one. We propose MXAttention, a data-free post-training quantization framework with two components. First, Universal Optimal Scaling (UOS) exploits the periodic structure of power-of-two microscaling to minimize a global MXFP4 quantization-error objective. It yields the closed-form, distribution-independent boundary Qmax = 7.25, without calibration or per-layer search. Second, Pre-Normalization Quantization (PNQ) uses the same quantized exponential tiles in the normalizer and output updates, preserving row-wise normalization. On Wan2.2 and HunyuanVideo, MXAttention substantially improves frame-level similarity to FP16 outputs. It preserves FP16-level generation quality with less than 0.01 absolute degradation on all reported VBench metrics and achieves video quality competitive with strong NVFP4-based baselines. We further evaluate LLM quantization on Qwen3-14B, observing accuracy gains for attention quantization, weight–activation quantization, and their combination. On one Ascend 950-series NPU, MXAttention delivers these accuracy gains at nearly the same attention latency as the PNQ-enabled MXFP4 baseline (0.13% difference). MXAttention retains a 2.70× attention speedup and a 1.84× total video-generation task speedup over BF16 attention.

Primary area:
generative models
Keywords:
post-training quantization, low-bit inference, attention quantization, MXFP4, microscaling floating-point formats, diffusion transformers, video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3K4TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MXAttention is a data-free, training-free framework for MXFP4 attention. It combines Universal Optimal Scaling (UOS) and Pre-Normalization Quantization (PNQ) to reduce quantization error and preserve attention normalization while retaining low-precision inference efficiency.

Critical strengths
  1. The paper presents training-free, efficient MXFP4 attention, which is useful for practical deployment.
  2. The paper is well written and easy to follow, with clear motivation and an accessible presentation of the proposed method.
Critical weaknesses
  1. Although the paper provides theoretical proofs, the overall novelty of UOS and PNQ appears limited. In my view, these are primarily engineering refinements.
  2. MXAttention does not show a clear overall accuracy advantage over NVFP4 + Sage, particularly on HunyuanVideo in Table 1.
  3. If MXFP4 used one shared scale per 16 elements, as in NVFP4, how would its accuracy change?
  4. How would the method perform on models with fewer denoising steps, such as a four-step Wan2.2 model?
  5. The baseline comparison is not fully fair: MXAttention uses Hadamard rotation, whereas this preprocessing is not reported for the NVFP4 baselines.
  6. Quantitative quality evaluation primarily relies on VBench. Could the authors report additional assessments, such as VisionReward [1] scores or human preference results, to better substantiate the generation quality?

[1] Jiazheng Xu et al. VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(13): 11269–11277, 2026.

Questions for authors

Please refer to Weaknesses.

4 comments

Thank you for recognizing the practical motivation and clarity of our work. We have added matched comparisons and VisionReward evaluations on Wan2.2 and HunyuanVideo. This comment addresses W1–W3; the following comment presents the rotation and quality results.

W1. Contributions of UOS and PNQ

Our contribution starts from the analysis of two sources of MXFP4 attention error: the shared-scale trade-off and normalization mismatch.

UOS provides a theoretical basis for selecting the shared scale. Lemma 1 and Eq. (6) show that the distribution of block maxima enters the derivative of the expected error through a nonnegative factor. The sign of the derivative is determined by the E2M1 quantization grid. Theorem 1 establishes q⋆=29/4=7.25q^\star=29/4=7.25 as a global minimizer of the objective in Eqs. (3)–(4), under the stated assumptions. Corollary 1 extends this result to the specified class of weighted objectives. This gives a common scaling rule without calibration or per-layer search. In the new matched downstream experiments, changing q=7q=7 to q=7.25q=7.25 reduces mean MSE by 3.85% on Wan2.2 and 20.70% on HunyuanVideo, with rotation, PNQ, block size, and precision policy held fixed.

PNQ addresses the normalization error introduced by quantization. Section 3 and Appendix I show how inconsistent numerator and denominator weights produce a row-dependent scale error in tiled attention. Appendix R measures an average row sum of 0.9266 after offline quantization of normalized attention weights across 80 Wan2.2 tensors. This motivates preserving normalization for each row. Equation (10) uses the same quantized exponential weights in both updates; Appendices H–J analyze online rescaling and establish that the resulting weights sum to one in exact arithmetic.

INT-FlashAttention and QFlash also use quantized exponentials in both updates, and we will include them in Related Work. Our contribution is the MXFP4 error analysis, the explicit normalization analysis, and the demonstrated quality recovery.

PNQ also supports computing the row sum on the Cube unit through

P^ij[V^j,1]=[P^ijV^j,P^ij1].\widehat{P}_{ij}[\widehat{V}_j,\mathbf{1}]=[\widehat{P}_{ij}\widehat{V}_j,\widehat{P}_{ij}\mathbf{1}].

Our implementation uses the same quantized weights and exactly represented constant-one entries for both outputs, removing a separate Vector row reduction. Online rescaling and final normalization remain on the Vector unit. The complete UOS + PNQ + Q/K Hadamard configuration achieves 2.67× attention speedup and 1.63× total video-task speedup over BF16 on Ascend 950.

W2. Value of MXFP4 attention and comparison with NVFP4 + Sage

MXAttention recovers 92.9% and 95.8% of the Imaging Quality gap between direct MXFP4 and FP16 on Wan2.2 and HunyuanVideo, respectively. The comparison with NVFP4 + Sage is model-dependent: MXAttention leads on all five reference-based metrics on Wan2.2, while NVFP4 + Sage leads on these metrics on HunyuanVideo.

Recovering MXFP4 quality has practical value on hardware with native MXFP4 support. For example, AMD documents native MXFP4 support on MI350X/MI355X and conversion or BF16 emulation for NVFP4 checkpoints [1]. Our Ascend results provide a separate deployment example.

There is also evidence of efficient MXFP4 computation. MR-GPTQ/QuTLASS reports up to approximately 15% higher MXFP4 throughput than NVFP4 for the evaluated B200 linear-layer configuration [2]. WUSH reports up to 5.8× speedup over BF16 for evaluated RTX 5090 linear layers, including transformation and quantization [3]. These results motivate the format; our attention and video-task speedups are measured directly on Ascend.

W3. A shared scale for 16 elements

This comparison would help isolate the effect of scaling granularity. Our work targets the OCP MXFP4 format, which specifies 32 E2M1 values per E8M0 shared scale [4]. A 16-element group would double the number of scales. We have not evaluated that configuration. The reported quality recovery and acceleration use the standard 32-element format.

References

[1] AMD: NVFP4 to MXFP4.

[2] MR-GPTQ/QuTLASS, Section 5, Figure 5.

[3] WUSH, Section 5, Figure 2.

[4] OCP Microscaling Formats, Table 1.

This comment presents the matched rotation comparison (W5) and additional quality assessment (W6).

W5. Applying the same Q/K Hadamard rotation to NVFP4 + Sage

We evaluated NVFP4 + Sage with the same Q/K Hadamard rotation on both models. Each comparison uses 48 videos with matched prompts and seeds. All methods share the same precision policy within each model: Wan2.2 uses high precision at steps 38 and 39, with no first-layer fallback; HunyuanVideo uses four-bit attention throughout denoising.

In the table, Sage denotes NVFP4 + Sage, and +H adds the same Q/K Hadamard rotation. MX denotes the complete MXAttention configuration.

MetricWan SageWan Sage+HWan MXHunyuan SageHunyuan Sage+HHunyuan MX
Subject ↑0.94590.94700.94910.95750.95660.9563
Imaging ↑0.69000.69340.68470.61190.60330.6050
Aesthetic ↑0.63480.62560.63550.60290.60790.6080
Cosine ↑0.89880.88350.92160.97100.97460.9655
SSIM ↑0.47660.45380.51900.75240.77650.7198
PSNR ↑13.9313.4015.1619.7420.4118.59
FID ↓126.18139.07112.60105.9593.60136.28
LPIPS ↓0.43780.47340.38610.26790.24330.3089
MSE ↓3402.33866.02462.9918.7793.51112.4
VisionReward ↑0.08320.08170.07940.09180.09720.0889

Rotation has model-dependent effects: it improves the five reference-based metrics on HunyuanVideo and worsens them on Wan2.2. MXAttention retains its advantage on all five Wan2.2 reference-based metrics against both NVFP4 + Sage configurations.

W6. Quality evaluation beyond VBench

Thank you for suggesting VisionReward. We have added it on both models using the same 48-video evaluation sets.

The original Table 1 also reports Cosine similarity, SSIM, PSNR, FID, and LPIPS alongside the three VBench metrics. VisionReward adds an assessment based on learned human preferences.

MethodWan2.2HunyuanVideoWan2.2 + first-layer fallback
Direct MXFP40.0573−0.00600.0716
NVFP40.08420.08610.0766
NVFP4 + Sage0.08320.09180.0825
NVFP4 + Sage + Hadamard0.08170.0972—
MXAttention0.07940.08890.0859

The first two columns use the precision policies described above. The last column is a separate Wan2.2 comparison with the first layer kept in high precision for every method, while retaining the same step fallback. A dash indicates an unavailable result.

MXAttention improves VisionReward over direct MXFP4 on both models and achieves scores close to the original NVFP4 + Sage baseline: 0.0794 versus 0.0832 on Wan2.2, and 0.0889 versus 0.0918 on HunyuanVideo. With the same first-layer fallback, MXAttention scores 0.0859 versus 0.0825 for NVFP4 + Sage on Wan2.2.

The HunyuanVideo ablation also supports the contribution of each component. With rotation fixed, adding PNQ increases VisionReward from 0.0558 to 0.0768; adding UOS increases it further to 0.0889. Together with the reference-based metrics and the visual examples in the paper, these results support substantial quality recovery under MXFP4.

Thank you again for your helpful suggestions; we hope the additional comparisons provide a clearer picture of MXAttention's contribution and practical value, and we would be glad to discuss these results further.

We have now completed the two Wan2.2 experiments you suggested: one with 16 elements per shared scale and one with a distilled four-step model.

W3. 16 elements per shared scale. We changed the group size from 32 to 16 for Q, K, V, and P, keeping the E2M1 values and E8M0 scales. We evaluated 48 videos per configuration. All four configurations used the same Q/K Hadamard rotation, with no high-precision fallback for any denoising step or for layer 0. The baseline used the OCP scaling rule without PNQ.

ConfigurationGroup sizeSubject ↑Imaging ↑Aesthetic ↑VisionReward ↑PSNR ↑LPIPS ↓
OCP scaling rule + rotation320.92500.61600.60250.033813.300.6193
OCP scaling rule + rotation160.93090.61860.61590.051212.870.6267
MXAttention320.93570.67300.62160.068914.980.4010
MXAttention160.94570.67880.62480.077315.230.4024

For MXAttention, reducing the group size to 16 improved all three VBench scores reported in Table 1, as well as VisionReward and PSNR. LPIPS changed little. UOS and PNQ continued to help at the smaller group size: with 16 elements per scale, adding them to the rotation baseline raised Imaging from 0.6186 to 0.6788 and PSNR from 12.87 to 15.23 dB.

W4. Four-step Wan2.2. We used Wan2.2-T2V-A14B with the four-step LoRA weights from LightX2V (Seko V2.0, rank 64). We evaluated 48 videos per configuration, using videos generated by the same four-step model with high-precision attention as the reference.

MetricHigh-precision referenceNaive MXFP4MXAttention
Subject ↑0.95150.95840.9506
Imaging ↑0.67430.60450.6588
Aesthetic ↑0.62540.61580.6226
VisionReward ↑0.06910.06270.0685
Cosine ↑—0.90730.9191
SSIM ↑—0.40370.4348
PSNR ↑—13.3814.30
Mean per-video FID ↓—206.8994.20
LPIPS ↓—0.60890.4131
MSE ↓—3331.32959.3

On this four-step model, MXAttention closed 77.9% of the gap in Imaging between naive MXFP4 and the high-precision reference. VisionReward rose from 0.0627 to 0.0685, close to the reference score of 0.0691. The videos also matched the reference more closely: LPIPS fell by 32.2% and PSNR rose by 0.92 dB relative to naive MXFP4.

Thank you for suggesting both experiments. We would be happy to answer any further questions about the settings or results.

Reviewer_hb1ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes MXAttention to recover quality lost when attention is computed using MXFP4, which represents values with 4 bits and uses a scale shared by each group of values. The scale choice is important as it can make small values round to zero and large values get clipped. To address this, this paper proposes UOS (Universal Optimal Scaling), which is a mathematically derived rule to choose the scale without requiring calibration or a separate search for each layer. Another challenge associated with using MXFP4 is that the weighted output uses quantized values while its normalising sum uses the original values. As a result, the sum then does not match the weights used to form the output. The paper proposes PNQ (Pre-Normalization Quantization) to fix this using the same quantized values in both calculations. MXAttention also uses a fixed Hadamard rotation to reduce outliers before quantization.

The authors test whether MXAttention preserves video quality using Wan2.2 and HunyuanVideo. They measure generated video quality and similarity to videos produced with FP16 attention and compare against other low precision methods. They also remove components in stages to check how much rotation, PNQ and UOS contribute individually. On Qwen3-14B, they measure performance on question answering, reasoning and text prediction tasks. These experiments show improved quality and accuracy compared to direct MXFP4 quantization. Separate measurements on an Ascend NPU check how long attention takes to compute and how long the full video-generation task takes.

Critical strengths
  1. The main strength of this paper is that UOS gives a mathematical basis for choosing the quantization scale without calibration needed. In MXFP4, each block of 32 values shares one power of 2 scale. Each value is divided by the shared scale before rounding. If the scale is too large, small values could become so small that they round to zero. If the scale is too small, large values could exceed the range the format can represent and get clipped. The paper derives a cutoff of 7.25 to manage this trade-off. This threshold is optimal for the chosen error objective across different distributions of block maxima. Also, it is simple to apply as just need to change a constant in the scale rule and no separate search for each layer.

  2. The other strength is the good recovery in video quality shown by the ablations. PNQ makes the weighted output and its normalising sum use the same quantized values. With rotation kept unchanged, adding PNQ substantially improves imaging quality on both video models. UOS improves it further. This shows that correcting scaling and normalization can recover much of the quality lost with direct MXFP4 quantization.

Critical weaknesses
  1. The first critical weakness is novelty since INT-FlashAttention and QFlash already use the same quantized exponentials in the numerator and normalising sum in tiled attention. As a result, the authors need to explain what PNQ adds beyond adapting this to MXFP4. For UOS, Appendix G connects it to OAS, a previous relevant method. When both methods use blocks of 32 values, the OAS scaling rule corresponds to a cutoff of 7 while UOS uses 7.25. This cutoff controls the shared scale used to represent the values in each block and affects the balance between preserving small values and clipping large ones. The paper provides a theoretical justification for 7.25, which is a contribution. However, the paper does not show how much this improves actual video or LLM quality compared to using 7 with the same rotation, PNQ and block size. This would isolate the practical benefit of the new scaling rule. The comparison should also be described carefully since the full OAS method uses a smaller block size.

  2. The quality and speed results are also not for exactly the same configuration. The quality experiments apply UOS to the exponential values used in attention, but the timed kernel keeps its existing quantizer for those values. The paper does not show the quality of the timed version or the speed of the full version used for quality evaluation. Since the authors say that the method recovers quality while keeping the speed, the paper needs to show that both are achieved together.

Questions for authors
  1. What exactly is new in PNQ compared with INT-FlashAttention and QFlash? Both already use the same quantized exponentials to calculate the weighted output and the sum used to normalise it. So both parts use the same rounded values. This seems to be the main idea behind PNQ.

  2. Appendix G says the OAS scaling rule is equivalent to using a cutoff of 7 when the block size is 32 while UOS uses 7.25. These cutoffs choose the shared scale for the values in each block. So they affect which small values are preserved and which large values get clipped. How much does using 7.25 actually improve video or LLM quality over 7? Can you show this with the same block size, rotation and PNQ, so we can see the gain from the scaling rule itself?

  3. The quality experiments use UOS on the exponential tiles. However, the timed kernel keeps its existing quantizer for these tiles. So the quality and speed numbers come from different configurations. Can you show the quality of the timed version or the speed of the full version used in the quality tests? We need to know whether the reported quality gains are still there in the implementation that gives the reported speed.

5 comments

Thank you for highlighting the scaling analysis and the quality recovery in the ablations. We have completed the requested comparisons on both video models and measured the full MXAttention implementation. This comment addresses PNQ; Part 2 covers the scaling comparison and deployment results.

Q1. PNQ versus INT-FlashAttention and QFlash

Both INT-FlashAttention and QFlash use quantized exponentials in the numerator and normalizing sum. We acknowledge this shared principle and will cite and discuss both methods in Related Work.

PNQ's contribution is the explicit analysis of normalization mismatch in MXFP4 attention, together with its normalization guarantee and downstream validation. Section 3 and Appendix I show that inconsistent weights in the numerator and denominator introduce a row-dependent scale error that persists through online-softmax updates.

Appendix R measures this effect on 80 Wan2.2 attention tensors: direct quantization of normalized attention weights gives an average row sum of 0.9266, with zeroing and saturation accounting for most of the net deficit. These are offline measurements of quantized normalized weights. An average correction cannot enforce normalization for every row.

Equation (10) uses the same quantized exponential weights in both updates. Appendices H–J account for block scales and subsequent online rescaling, and prove that the resulting effective attention weights sum to one in exact arithmetic. The analysis identifies the error being corrected and establishes the normalization property for tiled MXFP4 attention.

The quality results show why this matters. With Q/K rotation fixed and UOS disabled, adding PNQ on HunyuanVideo improves Imaging Quality from 0.4982 to 0.5989 and VisionReward from 0.0558 to 0.0768.

PNQ also enables a useful implementation on Ascend. We compute the weighted output and row sum together on the Cube unit using

P^ij[V^j,1]=[P^ijV^j,P^ij1].\widehat{P}_{ij}[\widehat{V}_j,\mathbf{1}]=[\widehat{P}_{ij}\widehat{V}_j,\widehat{P}_{ij}\mathbf{1}].

Both results use the same quantized exponential tile and scales, with exactly represented constant-one entries. This removes a separate Vector-unit row reduction over the exponential tile. The Vector unit still performs online rescaling and final normalization.

We will clarify these contributions and their relationship to the shared numerator–denominator principle in prior work.

Q2. Matched q=7q=7 versus q=7.25q=7.25 comparison

We evaluated both thresholds on 48 videos per model with matched prompts and seeds, keeping block size 32, Q/K Hadamard rotation, PNQ, and the precision policy fixed. The threshold for Q, K, V, and the exponential tiles is the only change. Wan2.2 retains high precision at steps 38–39, with no layer-0 fallback; HunyuanVideo uses 4-bit attention throughout denoising.

MetricWan2.2: 7Wan2.2: 7.25HunyuanVideo: 7HunyuanVideo: 7.25
Subject ↑0.94840.94910.95930.9563
Imaging ↑0.68660.68470.59250.6050
Aesthetic ↑0.63420.63550.60580.6080
Cosine ↑0.91850.92160.94720.9655
SSIM ↑0.51950.51900.71260.7198
PSNR ↑15.0415.1618.7918.59
FID ↓109.54112.60142.60136.28
LPIPS ↓0.39050.38610.31020.3089
MSE ↓2561.42462.91402.71112.4
VisionReward ↑0.07810.07940.08140.0889

The q=7.25q=7.25 rule reduces mean MSE by 3.85% on Wan2.2 and 20.70% on HunyuanVideo. HunyuanVideo also gains 0.0125 in Imaging Quality and 0.0075 in VisionReward; Wan2.2 gains 0.12 in PSNR and improves LPIPS by 1.13%. The table includes the trade-offs across all evaluated metrics.

The theoretical contribution is the derivation of a common optimal threshold under the paper's objective. Lemma 1 and Eq. (6) show that the block-maximum distribution contributes a nonnegative factor to the derivative. Theorem 1 establishes q⋆=29/4=7.25q^\star=29/4=7.25 as a global minimizer of Eqs. (3)–(4), under the stated assumptions. Corollary 1 extends this result to nonnegative block weights fixed as qq varies. The rule requires no calibration or per-layer search.

We also agree that the OAS comparison needs a precise scope: q=7q=7 is OAS-equivalent at block size 32, while the full OAS method uses smaller blocks. This experiment isolates the threshold at a fixed block size.

Q3. Quality and speed of the same configuration

We have implemented and timed the complete configuration used in the quality experiments. It achieves 2.67× attention speedup and 1.63× total video-task speedup over BF16.

The original timed implementations in both Table 5(a) and Table 5(b) used the native OCP quantizer for the exponential tiles. The updated implementation uses UOS with q=7.25q=7.25 for Q, K, V, and the exponential tiles, PNQ, and fixed Q/K Hadamard rotation, matching the quality configuration.

MethodAttention latency (ms)Total video-task time (s)
BF16594.0422941.851
Naive MXFP4401.6882285.723
Full MXAttention222.3741801.513

The complete attention implementation is also 1.81× faster than naive MXFP4.

Attention measurements use the same Ascend 950-series NPU and protocol for all methods: B=1B=1, H=40H=40, D=128D=128, and sequence length 75,776. We report the median of six round medians, with five warm-up calls and 48 timed calls per round. Synchronized wall-clock timing includes layout conversion, Q/K rotation where applicable, Q/K/V quantization and scale computation, native attention, and output layout conversion.

The video-task measurements use Wan2.2 at 1280×720 resolution, 81 frames, and 40 denoising steps. We use the same prompt and seed across methods and average two independent complete tasks per method. Quantized methods retain BF16 at steps 38–39. Timing includes startup, model loading, generation, and saving.

This directly supplies the requested speed measurement for the full configuration used in the quality tests.

We appreciate your constructive comments, which helped us clarify the contributions and strengthen the experimental evidence, and would welcome any further questions or suggestions during the discussion.

Reviewer_yFqVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MXAttention, a training- and calibration-free framework for improving MXFP4 attention accuracy. It introduces Universal Optimal Scaling, which derives a fixed scaling threshold under a specified quantization-error objective, and Pre-Normalization Quantization, which preserves normalization by using the same quantized exponential weights in the softmax denominator and output accumulation. These components are combined with a fixed Hadamard rotation. Experiments on Wan2.2, HunyuanVideo, and Qwen3-14B demonstrate improved accuracy over standard MXFP4 and competitive video quality against NVFP4-based baselines. On a single Ascend 950-series NPU, the paper reports 2.70× attention and 1.84× video-generation task speedups over BF16.

Critical strengths
  1. Principled and lightweight numerical design. UOS derives a closed-form scaling threshold under an explicit error objective, while PNQ preserves normalization using consistent quantized weights. Neither requires training or calibration, making the approach practical for deployment under MXFP4’s restrictive scaling format.
  2. Substantial quality recovery with deployment evidence. Experiments recover much of the video quality lost by standard MXFP4 on both Wan2.2 and HunyuanVideo. Complementary LLM evaluations and NPU timing results support the practical relevance of these improvements beyond tensor-level quantization error.
Critical weaknesses
  1. The contribution over closely related methods remains insufficiently established. PNQ shares the numerator–denominator consistency principle already present in QFlash, yet its distinct algorithmic or analytical contribution is not explained. This matters because Table 2 attributes substantial quality recovery to PNQ. Meanwhile, although UOS has theoretical and tensor-level validation, its downstream advantage over the OAS-equivalent q=7 rule is not directly quantified under matched settings.
  2. The connection between the quality and deployment results is incomplete. The quality configuration applies UOS to exponential tiles, whereas the timed attention kernel retains its native exponential quantizer. The paper does not quantify the quality impact of this deployment difference, leaving uncertainty about whether the measured implementation achieves the quality reported in Table 1.

QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

Questions for authors
  1. Clarification. Which aspects of PNQ’s update rule or analysis are new relative to QFlash, whose Algorithm 1 also uses quantized exponentials for both numerator and denominator updates?
  2. Evidence request. Given the existing theoretical and tensor-MSE validation, what downstream gains does q=7.25 provide over the OAS-equivalent q=7 rule when rotation, PNQ, block size, and precision policy are held fixed?
  3. Clarification and evidence request. Does the end-to-end implementation in Table 5(b) use the native exponential quantizer described for Table 5(a)? What quality does the deployed configuration achieve on the Table 1 evaluation set?
3 comments

Thank you for recognizing the numerical design and quality recovery of MXAttention. We have added matched scaling comparisons on both video models and measured the complete configuration used in the quality experiments. We address PNQ here, followed by scaling and deployment in Part 2.

Q1. PNQ and QFlash

PNQ contributes an analysis of normalization mismatch in MXFP4 attention, its effect on the output, and a quantization placement that preserves row-wise normalization. QFlash and INT-FlashAttention already use quantized exponentials in both the numerator and denominator. We acknowledge this shared principle and will include both methods in Related Work.

Section 3 and Appendix I show that using different weights for the output accumulation and the normalizing sum introduces a row-dependent scale error. This error persists through the tiled online-softmax updates. Appendix R further quantifies the issue: across 80 Wan2.2 attention tensors, directly quantizing normalized attention weights produces an average row sum of 0.9266. The decomposition identifies zeroing and saturation as the main sources of the net deficit. These are offline measurements of quantized normalized weights. Correcting the average deficit alone cannot ensure that each row sums to one.

Equation (10) uses the same quantized exponential weights in both updates. Appendices H–J analyze the effective full-row weights, including block scales and subsequent online rescaling, and establish that the resulting attention weights sum to one in exact arithmetic. This addresses the normalization error explicitly under MXFP4 quantization.

The downstream effect is substantial. On HunyuanVideo, with rotation fixed and UOS disabled, adding PNQ raises Imaging Quality from 0.4982 to 0.5989 and VisionReward from 0.0558 to 0.0768.

PNQ also supports an efficient implementation on Ascend. The output accumulation and row sum can be computed together on the Cube unit:

P^ij[V^j,1]=[P^ijV^j,P^ij1].\widehat{P}_{ij}[\widehat{V}_j,\mathbf{1}]=[\widehat{P}_{ij}\widehat{V}_j,\widehat{P}_{ij}\mathbf{1}].

The constant-one entries are represented exactly, and both results use the same quantized exponential tile and scales. This removes a separate row reduction over the exponential tile on the Vector unit; online rescaling and final normalization remain on the Vector unit.

Our contribution therefore includes the MXFP4 normalization-error analysis, the explicit normalization guarantee for tiled attention, and the demonstrated quality and implementation benefits. We will make the relationship to QFlash and INT-FlashAttention explicit.

Q2. Downstream comparison of q=7q=7 and q=7.25q=7.25

With all other settings fixed, q=7.25q=7.25 reduces mean MSE by 3.85% on Wan2.2 and 20.70% on HunyuanVideo.

Each comparison uses 48 videos with matched prompts and seeds. Both configurations use 32-element blocks, the same Q/K Hadamard rotation, PNQ, and precision policy. The only change is the threshold applied to Q, K, V, and the exponential tiles. Wan2.2 retains high precision at steps 38–39, with no layer-0 fallback; HunyuanVideo uses 4-bit attention throughout denoising.

MetricWan2.2: 7Wan2.2: 7.25HunyuanVideo: 7HunyuanVideo: 7.25
Subject ↑0.94840.94910.95930.9563
Imaging ↑0.68660.68470.59250.6050
Aesthetic ↑0.63420.63550.60580.6080
Cosine ↑0.91850.92160.94720.9655
SSIM ↑0.51950.51900.71260.7198
PSNR ↑15.0415.1618.7918.59
FID ↓109.54112.60142.60136.28
LPIPS ↓0.39050.38610.31020.3089
MSE ↓2561.42462.91402.71112.4
VisionReward ↑0.07810.07940.08140.0889

On HunyuanVideo, Imaging Quality increases by 0.0125 and VisionReward by 0.0075. On Wan2.2, LPIPS improves by 1.13% and PSNR by 0.12. The table reports the trade-offs across all evaluated metrics.

UOS also provides a theoretical basis for choosing this threshold. Lemma 1 and Eq. (6) show that the block-maximum distribution enters the derivative of the objective as a nonnegative factor. Theorem 1 establishes q⋆=29/4=7.25q^\star=29/4=7.25 as a global minimizer of the objective in Eqs. (3)–(4), under the stated assumptions. Corollary 1 extends the result to nonnegative block weights fixed as qq varies. This supplies a common scaling rule without calibration or per-layer search.

As noted in Appendix G, q=7q=7 is the OAS-equivalent rule at block size 32. The full OAS method uses smaller blocks, so this experiment isolates the scaling threshold at the same block size.

Q3. Alignment of quality and deployment configurations

Yes. The original implementations in both Table 5(a) and Table 5(b) retained the native OCP exponential quantizer, while the quality experiments used UOS for the exponential tiles.

We have now implemented and timed the full configuration: UOS with q=7.25q=7.25 for Q, K, V, and the exponential tiles, PNQ, and fixed Q/K Hadamard rotation. This matches the algorithmic configuration used in the quality evaluation.

MethodAttention latency (ms)Total video-task time (s)
BF16594.0422941.851
Naive MXFP4401.6882285.723
Full MXAttention222.3741801.513

The full configuration achieves 2.67× attention speedup and 1.63× total video-task speedup over BF16. It is also 1.81× faster than naive MXFP4 in the attention measurement.

For attention, all methods use the same Ascend 950-series NPU and measurement protocol, with B=1B=1, H=40H=40, D=128D=128, and sequence length 75,776. We report the median of six round medians; each round contains five warm-up calls and 48 timed calls. Synchronized wall-clock timing includes layout conversion, Q/K rotation where applicable, Q/K/V quantization and scale computation, native attention, and output layout conversion.

For the video task, we use Wan2.2 at 1280×720 resolution, 81 frames, and 40 denoising steps, with the same prompt and seed across methods. Each result is the mean of two independent complete tasks. Quantized methods retain BF16 at steps 38–39. The reported time covers the complete task, including startup, model loading, generation, and saving.

These measurements establish the performance of the full configuration used for the reported quality results.

Thank you again for your thoughtful feedback; we hope the clarifications and additional results address your concerns, and we would be happy to discuss any remaining questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Comment to All Reviewers

Dear Area Chair and Reviewers,

Thank you for the care you took in reviewing our paper. Two questions in particular helped us improve it: whether the implementation we timed used the same quantization settings as the quality experiments, and how much the choice of 7.25 over 7 matters for the generated videos. We have updated the manuscript with direct comparisons and summarize the results here.

1. Quality and speed with the full MXAttention configuration (Table 5; Appendix S). The kernel timed in the original submission used its native quantizer for the unnormalized exponentials, while the quality experiments in Table 1 used UOS on those tiles. We have now timed both attention and the complete Wan2.2 video generation task with UOS on Q, K, V and the exponential tiles, PNQ, and the fixed Q/K Hadamard rotation. On one Ascend 950-series NPU, attention takes 222.37 ms with MXAttention and 594.04 ms with BF16. The complete video task takes 1,801.51 s and 2,941.85 s, respectively. The abstract now reports 2.67× and 1.63× from these runs. The earlier 2.70× and 1.84× referred to the previous implementation. The quality results in Table 1 are unchanged.

2. What UOS and PNQ contribute (Sections 4.1–4.2; Tables 2, 4(b), and 9). We derive 7.25 as the optimal boundary for the MXFP4 quantization-error objective in the paper. To test its downstream effect, we compared it with 7 on 48 prompts per model, holding group size (32), rotation, PNQ, and precision settings fixed. Using 7.25 reduced the MSE of the generated videos against their FP16 references by 3.85% on Wan2.2 and 20.70% on HunyuanVideo. Cosine similarity and VisionReward also improved on both models.

We also clarify that INT-FlashAttention and QFlash already use quantized exponentials in both updates. Our PNQ analysis identifies the normalization error in direct MXFP4 attention: when the denominator uses the original exponentials but the output uses their quantized values, the effective weights need no longer sum to one. With rotation held fixed, correcting this mismatch improves Imaging Quality by 0.0542 on Wan2.2 and 0.1007 on HunyuanVideo. Appendix S shows how we use the same quantized tile to compute both the row sums and the weighted-value sums on the Cube unit in one multiplication.

3. Additional video evaluations (Tables 11–13). We added VisionReward scores and tested NVFP4 + Sage with the same Q/K rotation as MXAttention on both video models. On Wan2.2, we also compared 16 values per shared scale with the standard 32 and evaluated the four-step LightX2V model. On the four-step model, Imaging Quality rises from 0.6045 with direct MXFP4 to 0.6588 with MXAttention, compared with 0.6743 for the high-precision reference.

We appreciate the time all three reviewers spent on these comparisons. Your suggestions helped us explain where the quality gains come from and measure what they cost in the deployed implementation. We would be happy to clarify any of the new results.

AuthorsOpenReview ↗

A breif follow-up on the revised manuscript

Dear Area Chair and Reviewers,

Thank you again for your thoughtful reviews and the time you have devoted to our paper. Your suggestions have helped us strengthen the experimental evidence and explain our contributions more clearly.

As the discussion period draws to a close, we would like to briefly follow up on our revised manuscript and responses. The additional experiments and main revisions are summarized in our “Comment to All Reviewers.”

We would be grateful for your feedback on whether these updates address your concerns. If any questions remain, we would be happy to clarify them during the remaining discussion time.

Thank you for your time and consideration.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.