acceptodds
Under review as a conference paper at ICLR 2027

OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers

13 views

Abstract

Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation balances coordinate variances and maps activations toward a stable, known marginal across inputs, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video without per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image DiTs to W2A4 with usable generation quality.

Primary area:
generative models
Keywords:
Quantization, Diffusion Transformer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_F4X2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OrbitQuant quantizes image and video diffusion transformers without calibration. It normalizes weight rows and activation tokens, applies a shared randomized permuted block-Hadamard rotation, and uses a Lloyd-Max codebook based on TurboQuant's rotated-vector distribution. Folding the rotation into the weights avoids an inverse activation transform. Tests cover three image and four video backbones, with the strongest results at W2A4 on FLUX. The paper also reports speedups and memory savings from optimized kernels.

Critical strengths
  1. The construction is simple and reusable across modalities. Equations 4-8 show how folding the rotation into the weights preserves an unquantized projection with just one forward activation transform. No checkpoint-specific calibration is needed.

  2. The low-bit image gains are substantial. At W2A4, GenEval is 0.604 on FLUX.1-schnell and 0.475 on FLUX.1-dev, while the implemented rotation and smoothing baselines score near zero (Table 1). Table 5 also shows sizeable W3A3 gains over AdaTSQ, supported by rotation and quantizer ablations.

Critical weaknesses
  1. The timed implementation is unclear. Figure 5 reports W4A4 speedups, while Appendix E reports preserved quality with INT8 Lloyd-Max centroids and possible degradation at lower precision. The GEMM operand types, index packing/expansion, centroid approximation/scaling, and buffer costs are unspecified, as is quality from the exact optimized path being timed. Appendix D.5 only measures BF16 fake-quantization overhead. I therefore cannot tell which accuracy and storage benefits accompany the speedups, or how the implementation compares with optimized INT8.

  2. Some superiority claims are too broad. Some W4A4 image gains also look small beside the seed variation: FLUX.1-dev scores 0.640 versus ConvRot's 0.639, and FLUX.1-schnell's three-seed mean is 0.690 +/- 0.012 versus ConvRot's single reported 0.695. Many baselines are imported from other papers; matched settings and uncertainty estimates are needed to support a general W4A4 lead.

Questions for authors
  1. What operand precisions, packed formats, centroid approximation/scaling, and buffer sizes are used in Figure 5? Please report GenEval/VBench quality from that same optimized path and compare it with an optimized INT8 baseline under matched settings.
1 comment

We greatly appreciate your efforts in reviewing our work and providing valuable feedback, which encourages us to improve our work.

W1/Q1. Formats, buffers and quality of the timed path, and comparison with INT8.

Operand precisions and packed format. In OrbitQuant, "W4A4" means a 16-entry codebook, so each quantized weight or activation coordinate takes one of 16 levels (4 bits of information). Weights are stored as packed 4-bit indices, two per byte, and expanded to their INT8 codebook values inside the GEMM. Activations are quantized by a fused hook that emits the INT8 codebook value of each coordinate directly. The GEMM runs INT8 × INT8 → INT32 on INT8 Tensor Cores and writes BF16 in its epilogue. AdaLN layers are quantized to W4 (g64 RTN).

Centroid approximation and scaling. The codebook values (centroids) come from Lloyd–Max and are non-uniform, so they cannot be used directly as INT4 operands. We scale each codebook so that its largest value maps to 127 and round its 16 values to INT8. The scale is folded into the per-row and per-token norms applied in the epilogue.

Buffer sizes. On FLUX.1-dev, transformer weights take 10.2 GiB versus 22.17 GiB in BF16, of which 6.0 GiB is AdaLN stored in BF16. Packing AdaLN would save a further ≈4.4 GiB. Activations are passed to the GEMM as INT8, one byte per element.

Quality from the same path. The quality tables in the paper use fake quantization with floating-point codebook values. Table R1 reports quality through the kernel path that Figure 5 times. GenEval on FLUX.1-schnell matches fake quantization within seed spread, and VBench on Wan2.1-14B stays within 0.6 points on every dimension, so rounding the codebook to INT8 has no measurable effect on quality.

Table R1. W4A4 quality with fake quantization and through the kernel path. GenEval is mean ± sd over 3 seeds. VBench (%) on Wan2.1-14B.

ModelPathGenEval
FLUX.1-schnellFake quant0.690 ± 0.012
Kernel0.686 ± 0.007
ModelPathImagingAestheticMotion Smooth.DynamicBackgroundSubjectSceneOverall Consist.
Wan2.1-14BFake quant64.160.297.562.595.693.632.826.1
Kernel63.8560.0097.5661.9495.6793.5532.5726.19

Comparison with INT8. Tables R2 and R3 compare OrbitQuant with a W8A8 INT8 path (per-channel weights, per-token activations) on the same stack on one H200.

Table R2. Latency and peak memory with SageAttention and torch.compile (FLUX at 1024², Wan2.1-14B at 480×832, 81 frames). GenEval at 512², 3 seeds.

ModelMethodLatencyPeak memory (GiB)GenEval
FLUX.1-schnellW8A8 INT80.61 s/image25.90.692 ± 0.009
OrbitQuant W4A40.62 s/image21.90.686 ± 0.007
FLUX.1-devW8A8 INT85.99 s/image25.90.650 ± 0.004
OrbitQuant W4A46.13 s/image21.90.636 ± 0.001
Wan2.1-14BW8A8 INT8220.4 s/video28.9Table R3
OrbitQuant W4A4236.8 s/video22.4Table R3

Table R3. VBench (%) on Wan2.1-14B with both kernel paths on the same stack.

MethodImagingAestheticMotion Smooth.DynamicBackgroundSubjectSceneOverall Consist.
W8A8 INT865.3461.1097.2972.5096.0393.6832.7026.22
OrbitQuant W4A463.8560.0097.5661.9495.6793.5532.5726.19

Both paths use INT8 Tensor Core MMAs (s8 × s8 → s32). W8A8 is slightly faster, partly because OrbitQuant does extra work around the GEMM, applying the online rotation and codebook lookup to each activation token and expanding 4-bit weight indices to INT8 values inside the GEMM. In exchange, it stores 4-bit weights and uses 15–23% less peak memory (4.0–6.5 GiB), at a small quality cost. Dynamic Degree is 10.6 points lower, a motion reduction that most W4A4 methods share while frame quality and temporal consistency are preserved.

W2. Breadth of claims and W4A4 comparisons.

Your comment is right that our W4A4 claims were too broad. To compare under the same settings, we ran SVDQuant and ConvRot in our pipeline with the same three seeds as OrbitQuant (Table R4). Since ConvRot defines only W4A4, below 4 bits we extend it with QuaRot's quantization path (block-sequential GPTQ with weight clipping and per-token activation RTN) on top of its rotation.

Table R4. GenEval Overall on FLUX.1-schnell, mean ± sd over 3 seeds, all run in our pipeline.

MethodW4A4W3A3W2A4
SVDQuant0.686 ± 0.0050.616 ± 0.0060.000
ConvRot0.690 ± 0.0050.219 ± 0.0040.525 ± 0.009
OrbitQuant0.690 ± 0.0120.674 ± 0.0060.595 ± 0.008

At W4A4, the three methods are within seed spread. On FLUX.1-dev, OrbitQuant's three-seed mean (0.639 ± 0.004) is also level with ConvRot (0.639). We therefore claim parity at W4A4 rather than a lead. The advantage holds at 3 bits and below, where the matched gap exceeds the seed spread. Figures 12–14 of the revised appendix show the same comparison qualitatively on FLUX.1-dev. The revision narrows the claims in the abstract, contributions, Section 5.2 and conclusion accordingly.

Reviewer_2c4tOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OrbitQuant combines normalized distributional codebooks with randomly permuted block-Hadamard rotations for calibration-free quantization of diffusion transformers. The same dimension-specific codebook is used for weights and activations, with paired rotations preserving the unquantized linear map. Experiments span image and video generators and report strong low-bit image results, alongside measured inference speedups over BF16.

Critical strengths
  1. The low-bit image results are the most interesting part of the paper. At three bits, OrbitQuant improves substantially over AdaTSQ on all three backbones. At two-bit weights, it still produces useful FLUX outputs where the tested rotation and smoothing alternatives largely fail.
  2. Using the same calibration-free recipe across three image and four video backbones is appealing. The larger video models in the appendix are particularly useful evidence that the codebook construction transfers beyond the models used to illustrate its marginal distributions.
  3. The permutation ablation gives a clear reason to prefer this construction over plain block-Hadamard rotation at low precision. Its quality is also close to dense rotation, with much lower measured rotation cost.
  4. The method is simple to apply: the weight rotation is absorbed offline, and each dimension uses a fixed codebook. Avoiding checkpoint-specific calibration makes the recipe attractive for quantizing new checkpoints and generation configurations.
Critical weaknesses
  1. Several GenEval totals disagree with their task scores. For Z-Image-Turbo W4A4, the OrbitQuant subscores average to 0.763 rather than 0.767. Since the lead over AdaTSQ is small, I would like to see the tables regenerated from evaluator outputs before relying on the claimed ranking.
  2. The video discussion treats Overall Consistency as an aggregate VBench score, although it is one dimension. Eight dimensions are reported rather than the full benchmark. The results therefore do not establish the claimed overall ranking; on HunyuanVideo, DVD-Quant also performs better on several of the reported dimensions.
  3. Appendix E says that the nonuniform centroids need INT8 representation. It is unclear how this relates to the W4A4 kernels used for the reported speedups, or whether the quality tables use the same representation. This connection is important to understanding the actual quality and efficiency tradeoff.
  4. The small four-bit leads are less convincing than the low-bit results. The reported three-seed variation is comparable to some of these differences, while several baseline scores come from other papers. A matched comparison with the strongest baseline is needed to establish an improvement under the same generation and evaluation settings.
Questions for authors
  1. Several GenEval totals disagree with their task scores. For Z-Image-Turbo W4A4, the OrbitQuant subscores average to 0.763 rather than 0.767. Since the lead over AdaTSQ is small, I would like to see the tables regenerated from evaluator outputs before relying on the claimed ranking.
  2. The video discussion treats Overall Consistency as an aggregate VBench score, although it is one dimension. Eight dimensions are reported rather than the full benchmark. The results therefore do not establish the claimed overall ranking; on HunyuanVideo, DVD-Quant also performs better on several of the reported dimensions.
  3. Appendix E says that the nonuniform centroids need INT8 representation. It is unclear how this relates to the W4A4 kernels used for the reported speedups, or whether the quality tables use the same representation. This connection is important to understanding the actual quality and efficiency tradeoff.
  4. The small four-bit leads are less convincing than the low-bit results. The reported three-seed variation is comparable to some of these differences, while several baseline scores come from other papers. A matched comparison with the strongest baseline is needed to establish an improvement under the same generation and evaluation settings.
1 comment

We greatly appreciate your valuable feedback and comments, which have helped us improve our work. We address your concerns point by point below.

W1/Q1. GenEval totals.

We thank the reviewer for checking this. The printed Overall values were correct, but the Color attribution cells of the OrbitQuant and ConvRot rows were swapped when the ConvRot row was added to Table 1, and we have corrected them. We re-checked every GenEval row against the evaluator outputs, and the remaining mismatches are transcription slips in collapsed rows (Overall ≤ 0.007) that do not affect any comparison. The W4A4 margins over the best baselines remain small (0.001 to 0.008, against a seed spread of up to ±0.012), and AdaTSQ's code and evaluation settings are not released, so its seed variation is unknown. We therefore no longer claim a W4A4 ranking and describe W4A4 as parity with the strongest baselines.

W2/Q2. VBench interpretation.

Your point is correct that Overall Consistency is a single VBench dimension, not an aggregate score. We report eight of the benchmark's dimensions, following the prior works whose baseline numbers we compare against, so these results do not establish an overall ranking on the full benchmark. On HunyuanVideo, DVD-Quant is better on five of the eight dimensions, and OrbitQuant on Imaging Quality, Motion Smoothness and Scene. DVD-Quant's code and evaluation settings are not released, so its quoted rows may differ from our setup. The revision removes the Overall Consistency ranking claim and instead summarizes Table 2 by the average of the eight reported dimensions, on which OrbitQuant scores highest on Wan 2.1-1.3B and CogVideoX-2B.

W3/Q3. INT8 centroids and the W4A4 kernels.

In OrbitQuant, "W4A4" means a 16-entry codebook, so each quantized weight or activation coordinate takes one of 16 levels (4 bits of information). The codebook values (centroids) come from Lloyd–Max and are non-uniform, so they cannot be used directly as INT4 operands. We scale each codebook so that its largest value maps to 127 and store its 16 values as INT8. Weights are stored as packed 4-bit indices and decoded to their INT8 codebook values inside the GEMM. Activations are quantized by a fused hook that emits the INT8 codebook value of each coordinate directly. The GEMM then runs INT8 × INT8 → INT32 on INT8 Tensor Cores, and the codebook scale is folded into the per-row and per-token norms applied in the epilogue. The speedups therefore come from INT8 compute with 4-bit weight storage, not from INT4 Tensor Cores.

The quality tables use fake quantization with floating-point codebook values rather than this INT8 representation. Table R1 compares the two. GenEval on FLUX.1-schnell matches within seed spread, and VBench on Wan2.1-14B stays within 0.6 points on every dimension, so rounding the codebook values to INT8 has no measurable effect on quality. The revision describes this kernel representation in Section 5.5.

Table R1. Quality at W4A4 with fake quantization (floating-point codebook) and through the kernel path (INT8 codebook). GenEval is mean ± sd over 3 seeds, in eager mode. VBench (%) is on Wan2.1-14B at 480×832, 81 frames, 50 steps.

ModelPathGenEval
FLUX.1-schnellFake quant0.690 ± 0.012
Kernel0.686 ± 0.007
ModelPathImagingAestheticMotion Smooth.DynamicBackgroundSubjectSceneOverall Consist.
Wan2.1-14BFake quant64.160.297.562.595.693.632.826.1
Kernel63.8560.0097.5661.9495.6793.5532.5726.19

W4/Q4. Seed variation and quoted baselines.

Your comment is right that the W4A4 leads are within seed variation. To compare under the same conditions, we reran the strongest baselines with public code in our pipeline with the same three seeds as OrbitQuant (Table R2). Since ConvRot defines only W4A4, below 4 bits we extend it with QuaRot's quantization path with GPTQ on top of its rotation. AdaTSQ has not released its code, so its rows remain quoted from its paper.

Table R2. GenEval Overall on FLUX.1-schnell, mean ± sd over 3 seeds.

MethodW4A4W3A3W2A4
SVDQuant0.686 ± 0.0050.616 ± 0.0060.000
ConvRot0.690 ± 0.0050.219 ± 0.0040.525 ± 0.009
OrbitQuant0.690 ± 0.0120.674 ± 0.0060.595 ± 0.008

At W4A4, the three methods are within seed spread, so we claim parity rather than a lead. At W3A3 and W2A4, OrbitQuant exceeds both baselines. Figures 12–14 of the revised appendix show the same comparison qualitatively on FLUX.1-dev, where all three methods produce clean images at W4A4 and the differences appear below 4 bits.

Reviewer_N2YyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Orbit, a calibration-free post training quantization scheme for diffusion Transformers. The method combines randomized permuted block-Hadamard rotation and normalization to fit weights and activations to an approximate fixed distribution, so precomputed non-uniform codebooks can be used. Rotation sharing and fusion reduce computational overhead. The method requires no calibration data and no fine-tuning. It achieves good low-bit generation quality, reduces memory usage and speeds up inference.

Critical strengths
  1. It does not need activation calibration, and there is no need to adjust quantization ranges for different diffusion timesteps. The codebook is only determined by dimension and bit width. Magnitude information is recovered through per-channel or per-token norms.
  2. Weights and activations are processed in the same rotated space. No inverse rotation is needed during inference. RPBH avoids expensive Haar transforms.
  3. The W2A4 experimental results are promising. It can still maintain acceptable generation quality for some tasks at this extremely low bit setting.
Critical weaknesses
  1. Its core components overlap with prior work such as RobuQ and HIGGS. The paper needs to more clearly explain the additional contributions of RPBH and the shared-basis design for diffusion transformers.
  2. The provided theory does not give a bound on the distance between the coordinate distribution after RPBH rotation and normalization, and the spherical marginal distribution used to build the Lloyd–Max codebook. It also does not provide a theoretical bound on how this distribution mismatch affects quantization error.
  3. Several strong diffusion quantization baselines only appear in quantitative tables and are not included in visual comparisons. The evaluation of PTQ preprocessing overhead covers a limited set of methods.
Questions for authors
  1. What are the core advantages of OrbitQuant compared with RobuQ and HIGGS? Can the existing ablation experiments tell apart these differences?
  2. Are the quality metrics and speed metrics tested under the same inference setup? How is the overall execution path and cost calculated?
5 comments

We deeply appreciate your constructive reviews and comments, which have been invaluable in helping us strengthen our work. We address your concerns below and will integrate all feedback into the revised manuscript.

W1 / Q1. Relation to RobuQ and HIGGS.

We thank the reviewer for these pointers. Both share our high-level principle of rotating, normalizing, and quantizing with a Gaussian-matched grid, but differ in what is quantized, how the rotation is built, and which grid is used. We discuss and compare against them under matched settings below.

RobuQ is a QAT framework whose RobustQuantizer can also run as calibration-free W4A4 PTQ. OrbitQuant differs in three ways.

  • Quantizer. RobuQ uses a per-token standard-deviation scale and, in its released code, an integer-aligned uniform grid. OrbitQuant uses the ℓ2 norm and a symmetric Lloyd–Max codebook.
  • Rotation. RobuQ uses a full Hadamard transform without permutation. OrbitQuant applies a uniform random permutation followed by block-diagonal randomized Hadamard transforms on power-of-two blocks (RPBH).
  • Branch. RobuQ adds a rank-16 full-precision low-rank branch. OrbitQuant uses none.

We ran RobuQ's official code in our pipeline with the same layers, AdaLN treatment, evaluation and seeds, without its low-rank branch. Its integer-aligned grid leaves only one positive level at 2 bits and clips large positive weights, which accounts for most of the gap at 2-bit weights.

Table R1. GenEval Overall, mean ± sd over 3 seeds.

ModelMethodW3A3W2A4W2A3
FLUX.1-schnellRobuQ-style0.659 ± 0.0070.466 ± 0.0420.362 ± 0.040
OrbitQuant0.674 ± 0.0060.595 ± 0.0080.552 ± 0.007
FLUX.1-devRobuQ-style0.553 ± 0.0010.284 ± 0.0070.182 ± 0.004
OrbitQuant0.580 ± 0.0020.460 ± 0.0140.389 ± 0.011

HIGGS is a data-free, weight-only LLM quantizer that applies a block Hadamard transform and rounds to a Gaussian MSE-optimal grid, for which it recommends a p = 2 vector grid. OrbitQuant differs in two ways.

  • Activations. OrbitQuant also quantizes activations online with a scalar codebook, which takes a single LUT lookup. A p = 2 grid of HIGGS needs a nearest-neighbor search over up to 256 points per coordinate pair, which is 12.7–14.5× slower in our implementation.
  • Permutation. OrbitQuant adds a random permutation before the block transform, which balances mass across blocks (Proposition 1).

We extended HIGGS to W+A with per-channel weight and per-token activation scales for both methods, the granularity of standard integer kernels, instead of its per-block scales.

Table R2. GenEval Overall, mean ± sd over 3 seeds. Time is s/image at W2A4 in the same fake-quantization framework on one H200 (FLUX.1-schnell at 512², 4 steps, FLUX.1-dev at 1024², 50 steps). For p = 1 we use the scalar Lloyd–Max grid.

ModelMethodW3A3W2A4W2A3Time
FLUX.1-schnellHIGGS (p = 2)0.646 ± 0.0040.608 ± 0.0110.533 ± 0.0119.09
HIGGS (p = 1)0.635 ± 0.0100.556 ± 0.0130.443 ± 0.0150.70
OrbitQuant0.674 ± 0.0060.595 ± 0.0080.552 ± 0.0070.72
FLUX.1-devHIGGS (p = 2)0.476 ± 0.0090.404 ± 0.0080.254 ± 0.016337.4
HIGGS (p = 1)0.441 ± 0.0010.365 ± 0.0150.185 ± 0.01022.7
OrbitQuant0.580 ± 0.0020.460 ± 0.0140.389 ± 0.01123.3

With p = 2, HIGGS scores below OrbitQuant in five of six settings (by 0.02 to 0.14) and is 0.013 higher in the sixth (FLUX.1-schnell at W2A4), while running 12.7–14.5× slower. With a scalar grid (p = 1), it matches our speed but scores lower in all six settings.

These comparisons align with our existing ablations. HIGGS with p = 1 differs from OrbitQuant by the permutation, and its gap matches the effect isolated in Table 4 (W2A4 0.595 vs. 0.558 without the permutation). Figure 7 fixes the rotation and varies the activation quantizer, showing that the Lloyd–Max codebook outperforms uniform INT at similar cost.

We have added Tables R1 and R2 and this discussion to Appendix D.4 of the revised version.

W2. Marginal discrepancy and distortion.

Your comment is correct that Proposition 1 and Appendix B control only the mean, variance and magnitude of the rotated coordinates. They do not bound how far the rotated distribution is from the spherical marginal used to build the codebook, or how much extra distortion this causes. Such a bound cannot hold for every input. If a block receives only one nonzero coordinate, every rotated coordinate in that block takes just two values, which is far from the design marginal. Recent theory on Hadamard-based quantizers reports the same limitation for a single transform and fixes it with a second one [1]. A second transform would double the online activation rotation, so we instead check how much this matters in practice.

Figure 3 measures the marginal discrepancy, where RPBH brings the Kolmogorov–Smirnov distance to N(0, 1/d), the Gaussian approximation of the design marginal, to 0.001–0.003, comparable to a dense Haar rotation. Table R3 measures the resulting distortion as the ratio of realized to design MSE, where 1.0 means the codebook performs exactly as designed. For weights, the ratio is within 0.1% of 1 at both the median and the 95th percentile. For FLUX.1-schnell activations, the median is 1.00 and the 95th percentile is at most 1.09. Z-Image-Turbo activations have larger tails (95th percentile up to 1.77), mostly in its layers with 256-entry Hadamard blocks.

Table R3. Realized MSE / design MSE of the Lloyd–Max codebook (1.0 = as designed). Median / p95 over layers for weights and over (layer, timestep) for activations, on 16 prompts.

ModelTensorb = 2b = 4
FLUX.1-schnellweight1.000 / 1.0000.999 / 1.001
activation0.999 / 1.070.998 / 1.09
Z-Image-Turboweight1.000 / 1.0011.000 / 1.001
activation1.06 / 1.431.09 / 1.77

The revision points to this scope right after Proposition 1 and adds the discussion and Table R3 to Appendix B.1.

[1] Ben-Basat, Ran, et al. "Quantizing with randomized hadamard transforms: Efficient heuristic now proven." arXiv preprint arXiv:2605.06014 (2026).

W3. Baselines in visual and cost comparisons.

We added SVDQuant and ConvRot to the FLUX.1-dev qualitative comparisons at W4A4, W3A3 and W2A4, shown in Figures 12–14 of the revised appendix, and report a matched accuracy comparison (Table R4a) and their preparation cost (Table R4b) below. Since ConvRot defines only W4A4, below 4 bits we extend it with QuaRot's quantization path with GPTQ on top of its rotation.

At W4A4, all three methods produce clean images and reach similar GenEval scores. Below 4 bits, OrbitQuant outperforms both on GenEval. ConvRot scores higher at W2A4 than at W3A3, which indicates that its 3-bit activation quantization, rather than its 2-bit weights, is the main bottleneck.

In Table R4b, SVDQuant's preparation, dominated by calibration, takes about 10.8 hours on one H200, 842× OrbitQuant's. ConvRot is calibration-free at W4A4 and takes 1.14× OrbitQuant's time. Its sub-4-bit GPTQ extension needs calibration data and takes 12.0×. AdaTSQ's code has not been released, so we could not include it.

Table R4a. GenEval Overall on FLUX.1-schnell, mean ± sd over 3 seeds.

MethodW4A4W3A3W2A4
SVDQuant0.686 ± 0.0050.616 ± 0.0060.000
ConvRot0.690 ± 0.0050.219 ± 0.0040.525 ± 0.009
OrbitQuant0.690 ± 0.0120.674 ± 0.0060.595 ± 0.008

Table R4b. One-time preparation cost on FLUX.1-schnell, measured on H200.

MethodCalibration (s)Application (s)Total (s)RelativePeak memory (GiB)
OrbitQuant–46.0046.001.00×32.64
ConvRot (W4A4)–52.3952.391.14×32.53
ConvRot + GPTQ (W3A3)3.55548.56552.1212.0×36.00
SVDQuant38,376348.138,724842×54.1

Q2. Execution path and overhead accounting.

The quality tables in the paper use fake quantization with floating-point codebook values, while Figure 5 times the kernel path. To connect the two, we now evaluate quality through the kernel path itself (Table R5a). On FLUX.1-schnell, GenEval through the kernels matches fake quantization within seed spread. On Wan2.1-14B, VBench through the kernels stays within 0.6 points of fake quantization on every dimension. The fake-quant tables therefore reflect what the timed path produces.

*Table R5a. Quality at W4A4 through fake quantization and through the kernel path. GenEval is mean ± sd over 3 seeds. VBench (%) is on Wan2.1-14B at 480×832, 81 frames, 50 steps.

ModelPathGenEval
FLUX.1-schnellFake quant0.690 ± 0.012
Kernel0.686 ± 0.007
ModelPathImagingAestheticMotion Smooth.DynamicBackgroundSubjectSceneOverall Consist.
Wan2.1-14BFake quant64.160.297.562.595.693.632.826.1
Kernel63.8560.0097.5661.9495.6793.5532.5726.19

Kernel path. Weights are stored as packed 4-bit indices and decoded to INT8 codebook values inside the GEMM. A fused activation hook rotates each token, normalizes it, and maps each coordinate to its INT8 codebook value with a single LUT lookup. The same hook absorbs the preceding LayerNorm, AdaLN modulation, GELU and concatenation. The GEMM runs in INT8 × INT8 → INT32, and the codebook scale and the per-row and per-token norms are folded into the epilogue that writes BF16.

Overhead accounting. Table R5b breaks down GPU time per image. Weight decoding and the INT32-to-BF16 conversion are inside the GEMM time. Rotation, activation codebook lookup and INT8 emission are inside the hook, which accounts for 10.8% of the total. Other dtype conversions are included in "Other". Modules executed in BF16 are timed separately. These include the embedders and the AdaLN layers. AdaLN weights carry the same W4 (g64 RTN) values as in fake quantization but are stored and executed as BF16 linears. On FLUX.1-dev, packing them would reduce memory by a further ≈4.4 GiB.

Table R5b. Sum of GPU kernel time per image, FLUX.1-schnell W4A4, eager mode, one H200, full pipeline including text encoders and VAE.

ComponentmsShare
W4A4 GEMM (incl. weight decode, epilogue)252.438.6%
Activation hook (rotation, LUT, INT8 emission, fused ops)70.810.8%
BF16 linears (AdaLN, embedders, other BF16 GEMMs)17.72.7%
Attention183.228.0%
QK norm + RoPE21.83.3%
VAE decode (conv + GroupNorm)≈70.610.8%
Other (residual adds, dtype copies, misc.)≈36.95.6%
Total653.4100%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.