- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
OrbitQuant quantizes image and video diffusion transformers without calibration. It normalizes weight rows and activation tokens, applies a shared randomized permuted block-Hadamard rotation, and uses a Lloyd-Max codebook based on TurboQuant's rotated-vector distribution. Folding the rotation into the weights avoids an inverse activation transform. Tests cover three image and four video backbones, with the strongest results at W2A4 on FLUX. The paper also reports speedups and memory savings from optimized kernels.
Critical strengths
-
The construction is simple and reusable across modalities. Equations 4-8 show how folding the rotation into the weights preserves an unquantized projection with just one forward activation transform. No checkpoint-specific calibration is needed.
-
The low-bit image gains are substantial. At W2A4, GenEval is 0.604 on FLUX.1-schnell and 0.475 on FLUX.1-dev, while the implemented rotation and smoothing baselines score near zero (Table 1). Table 5 also shows sizeable W3A3 gains over AdaTSQ, supported by rotation and quantizer ablations.
Critical weaknesses
-
The timed implementation is unclear. Figure 5 reports W4A4 speedups, while Appendix E reports preserved quality with INT8 Lloyd-Max centroids and possible degradation at lower precision. The GEMM operand types, index packing/expansion, centroid approximation/scaling, and buffer costs are unspecified, as is quality from the exact optimized path being timed. Appendix D.5 only measures BF16 fake-quantization overhead. I therefore cannot tell which accuracy and storage benefits accompany the speedups, or how the implementation compares with optimized INT8.
-
Some superiority claims are too broad. Some W4A4 image gains also look small beside the seed variation: FLUX.1-dev scores 0.640 versus ConvRot's 0.639, and FLUX.1-schnell's three-seed mean is 0.690 +/- 0.012 versus ConvRot's single reported 0.695. Many baselines are imported from other papers; matched settings and uncertainty estimates are needed to support a general W4A4 lead.
Questions for authors
- What operand precisions, packed formats, centroid approximation/scaling, and buffer sizes are used in Figure 5? Please report GenEval/VBench quality from that same optimized path and compare it with an optimized INT8 baseline under matched settings.
We greatly appreciate your efforts in reviewing our work and providing valuable feedback, which encourages us to improve our work.
W1/Q1. Formats, buffers and quality of the timed path, and comparison with INT8.
Operand precisions and packed format. In OrbitQuant, "W4A4" means a 16-entry codebook, so each quantized weight or activation coordinate takes one of 16 levels (4 bits of information). Weights are stored as packed 4-bit indices, two per byte, and expanded to their INT8 codebook values inside the GEMM. Activations are quantized by a fused hook that emits the INT8 codebook value of each coordinate directly. The GEMM runs INT8 × INT8 → INT32 on INT8 Tensor Cores and writes BF16 in its epilogue. AdaLN layers are quantized to W4 (g64 RTN).
Centroid approximation and scaling. The codebook values (centroids) come from Lloyd–Max and are non-uniform, so they cannot be used directly as INT4 operands. We scale each codebook so that its largest value maps to 127 and round its 16 values to INT8. The scale is folded into the per-row and per-token norms applied in the epilogue.
Buffer sizes. On FLUX.1-dev, transformer weights take 10.2 GiB versus 22.17 GiB in BF16, of which 6.0 GiB is AdaLN stored in BF16. Packing AdaLN would save a further ≈4.4 GiB. Activations are passed to the GEMM as INT8, one byte per element.
Quality from the same path. The quality tables in the paper use fake quantization with floating-point codebook values. Table R1 reports quality through the kernel path that Figure 5 times. GenEval on FLUX.1-schnell matches fake quantization within seed spread, and VBench on Wan2.1-14B stays within 0.6 points on every dimension, so rounding the codebook to INT8 has no measurable effect on quality.
Table R1. W4A4 quality with fake quantization and through the kernel path. GenEval is mean ± sd over 3 seeds. VBench (%) on Wan2.1-14B.
Comparison with INT8. Tables R2 and R3 compare OrbitQuant with a W8A8 INT8 path (per-channel weights, per-token activations) on the same stack on one H200.
Table R2. Latency and peak memory with SageAttention and torch.compile (FLUX at 1024², Wan2.1-14B at 480×832, 81 frames). GenEval at 512², 3 seeds.
Table R3. VBench (%) on Wan2.1-14B with both kernel paths on the same stack.
Both paths use INT8 Tensor Core MMAs (s8 × s8 → s32). W8A8 is slightly faster, partly because OrbitQuant does extra work around the GEMM, applying the online rotation and codebook lookup to each activation token and expanding 4-bit weight indices to INT8 values inside the GEMM. In exchange, it stores 4-bit weights and uses 15–23% less peak memory (4.0–6.5 GiB), at a small quality cost. Dynamic Degree is 10.6 points lower, a motion reduction that most W4A4 methods share while frame quality and temporal consistency are preserved.
W2. Breadth of claims and W4A4 comparisons.
Your comment is right that our W4A4 claims were too broad. To compare under the same settings, we ran SVDQuant and ConvRot in our pipeline with the same three seeds as OrbitQuant (Table R4). Since ConvRot defines only W4A4, below 4 bits we extend it with QuaRot's quantization path (block-sequential GPTQ with weight clipping and per-token activation RTN) on top of its rotation.
Table R4. GenEval Overall on FLUX.1-schnell, mean ± sd over 3 seeds, all run in our pipeline.
At W4A4, the three methods are within seed spread. On FLUX.1-dev, OrbitQuant's three-seed mean (0.639 ± 0.004) is also level with ConvRot (0.639). We therefore claim parity at W4A4 rather than a lead. The advantage holds at 3 bits and below, where the matched gap exceeds the seed spread. Figures 12–14 of the revised appendix show the same comparison qualitatively on FLUX.1-dev. The revision narrows the claims in the abstract, contributions, Section 5.2 and conclusion accordingly.