- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Prefix-Point (PXP), an adaptive non-uniform numerical format for post-training quantization of diffusion models. The key observation is that activation distributions vary not only in scale but also substantially in shape across transformer layers and denoising timesteps, making a single fixed non-uniform codebook suboptimal. PXP retains an exponent-mantissa representation but encodes exponent levels using variable-length prefix codes, such that exponent regions contributing larger quantization errors receive shorter prefixes and consequently more mantissa bits.
Critical strengths
- The paper identifies a meaningful and underexplored failure mode of existing diffusion quantization methods.
- PXP is conceptually simple but non-trivial.
- Results at very low precision are unusually strong.
Critical weaknesses
- The strongest concern is that the baseline comparison does not fully disentangle format design from calibration flexibility. PXP uses a separate calibrated prefix table for each layer and timestep, while the reported NF/AF/SF/BOF baselines use a single codebook shared across layers and denoising steps. I would like to see the results that use a single fixed PXP format shared across layers and timesteps.
- The evaluation deliberately removes advanced diffusion PTQ techniques, making the practical state-of-the-art comparison incomplete. The paper explicitly evaluates all formats using absolute-max scaling without equivalent transformations, rotations, learned clipping, or advanced calibration, in order to isolate the format effect. However, the Related Work simultaneously argues that PXP is orthogonal to existing timestep-aware calibration and transformation methods and can be combined with them. That important claim is not empirically verified.
- Hardware efficiency claims are compelling but should be interpreted more narrowly than some of the presentation suggests. The reported speedups come from a custom RTL design synthesized with Yosys/ABC using the NanGate 45 nm library, not from wall-clock execution on an existing GPU/NPU/accelerator.
- The Related Work section would be more complete by including additional representative PTQ methods for diffusion models, such as SVDQuant, EDA-DM, TFMQ-DM, and DilateQuant.
Questions for authors
- How much of PXP's gain comes from the prefix-point representation itself versus the fact that PXP is calibrated separately for every layer and timestep? Could the authors provide an equally fine-grained adaptive baseline under a comparable metadata/storage budget?
- The paper argues that PXP is orthogonal to existing diffusion PTQ calibration/transformation methods. Could the authors integrate PXP into one representative strong diffusion PTQ pipeline and report whether the gain remains after such transformations?
8 comments
(Question 1 - part 1/2) Global, layer-only, timestep-only and per layer-and-timestep PXP codebooks.
Thank you for highlighting the ambiguity in attributing quality gains to the format itself vs. layer- and timestep-specific codebooks. The table below compares four codebook granularities: global, per-layer, per-timestep, and joint per-layer-and-timestep (as reported in the paper) for 256×256 ImageNet generation using DiT-XL/2 (CFG = 1.5) over 50 denoising steps at W4A6. For ease of comparison, we also include the formats evaluated in Table 1 of the main paper.
| Method | Bit-width | Granularity | IS ↑ | FID ↓ | sFID ↓ | Precision ↑ | Recall ↑ |
|---|---|---|---|---|---|---|---|
| FP16 (Baseline) | 236.11 | 5.40 | 17.57 | 0.81 | 0.67 | ||
| INT | W4A6 | Global | 2.58 | 220.06 | 422.06 | 0.03 | 0.08 |
| NF | W4A6 | Global | 53.62 | 44.36 | 162.97 | 0.44 | 0.56 |
| AF | W4A6 | Global | 72.87 | 39.56 | 43.64 | 0.45 | 0.70 |
| SF | W4A6 | Global | 34.41 | 77.10 | 62.21 | 0.27 | 0.62 |
| BOF | W4A6 | Global | 7.53 | 160.27 | 74.08 | 0.11 | 0.27 |
| PXP (Ours) | W4A6 | Global | 121.96 | 19.00 | 59.60 | 0.62 | 0.64 |
| PXP (Ours) | W4A6 | Per-layer | 179.90 | 10.05 | 23.82 | 0.72 | 0.67 |
| PXP (Ours) | W4A6 | Per-timestep | 158.91 | 12.62 | 35.52 | 0.69 | 0.65 |
| PXP (Ours) | W4A6 | Per-layer and timestep | 182.50 | 9.70 | 23.50 | 0.73 | 0.67 |
The results show that both per-layer and per-timestep codebooks provide substantial improvements over a global codebook, with per-layer codebooks offering the greatest benefit and achieving performance close to that of per-layer-and-timestep codebooks. Notably, even with a single global codebook, PXP substantially outperforms all existing formats in IS, FID and precision. These results highlight two complementary advantages of PXP: its flexible quantization levels provide a more effective format than existing alternatives, while its adaptivity enables codebooks to be optimized for individual layers and timesteps. Unlike existing formats, which are fixed based on assumed prior distributions, PXP naturally supports such fine-grained adaptation.
(Question 1 - part 2/2) Comparison with equally fine-grained adaptive baseline.
For comparisons against equally fine-grained adaptive baselines, we compare PXP with AFP [1] and FloatMax [2], both of which independently determine the floating-point format for each layer and timestep, offering the same adaptation granularity as PXP.
The table below compares the generation results of AFP, FloatMax, and PXP on DiT-XL/2 (CFG=1.5) over 50 denoising steps at W4A6 quantization precision. All methods use absolute-maximum scaling with a group size of 128 and are calibrated using 64 samples.
| Method | Bit-width | IS ↑ | FID ↓ | sFID ↓ | Prec ↑ | Rec ↑ |
|---|---|---|---|---|---|---|
| FP16 (Baseline) | 236.11 | 5.40 | 17.57 | 0.81 | 0.67 | |
| AFP | W4A6 | 90.10 | 27.81 | 32.75 | 0.52 | 0.70 |
| FloatMax | W4A6 | 32.30 | 62.48 | 206.60 | 0.34 | 0.54 |
| PXP (Ours) | W4A6 | 182.50 | 9.70 | 23.50 | 0.73 | 0.67 |
The results show that AFP and FloatMax still perform significantly worse than PXP. AFP performs better due to its greater flexibility in jointly optimizing the exponent width and exponent bias, whereas FloatMax restricts the exponent width to 2 or 3 for hardware efficiency and uses a fixed exponent bias. Notably, although AFP and FloatMax employ the same granularity as PXP (i.e., per-layer-and-timestep formats), they still fall short of PXP in generation quality. We also refer the reviewer to Appendix I, where we compare PXP with FP-Cal, which uses an identical format to AFP but selects the optimal format based on minimum MSE, whereas AFP uses minimum KL divergence. These results highlight the advantages of PXP's additional flexibility in adapting mantissa widths across exponent levels.
References:
[1] Liu, Fangxin, et al. "Improving neural network efficiency via post-training quantization with adaptive floating-point." 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2021.
[2] Chung, Seoho, et al. "FloatMax: An Efficient Accelerator for Transformer-based Models Exploiting Tensor-wise Adaptive Floating-Point Quantization." 2024 IEEE 42nd International Conference on Computer Design (ICCD). IEEE, 2024.
General Response to Reviewer sDX5
We sincerely thank you for your positive assessment of our work and for recognizing the significance of the identified quantization challenge and PXP's strong low-precision performance. We also appreciate your constructive feedback and insightful questions. We address each comment in detail below.