acceptodds
Under review as a conference paper at ICLR 2027

CentriQ: Calibration-Free Quantization of Diffusion Transformers via Exact Mean Centering

Abstract

Diffusion transformers (DiTs) achieve state-of-the-art image generation, but their sampling cost limits deployment. Quantizing both weights and activations to 4 bits reduces this cost, yet existing methods fall short in one of two ways. Calibration-based methods are tied to a specific checkpoint and prompt distribution, whereas data-free Hadamard rotation, effective for LLMs, loses quality on DiTs. We show that this loss has a structural cause. Adaptive layer-norm conditioning adds a per-token mean to the activations, and at the widths of the evaluated DiTs, the Hadamard rotations used by data-free methods cannot spread this mean uniformly across coordinates. A single dominant direction therefore survives the rotation and sets the quantization range. We introduce CentriQ, a calibration-free quantizer that centers each token before rotation and restores the mean exactly through a rank-1 full-precision branch, so that per-token scales follow in closed form without data. Weights are fitted under a robust objective that tracks the dense mode of each group and discounts heavy tails. Across three DiTs, CentriQ matches the quality of calibrated SVDQuant at 4 bits, whereas calibration-free weight quantizers with plain per-token activation quantization collapse or degrade substantially. CentriQ outperforms the strongest calibration-free method reported to date at 2-bit weights. It is also the first calibration-free method to retain usable image quality at 2-bit activations.

Primary area:
generative models
Keywords:
Diffusion Models, Quantization, Calibration-free, DiTs, Image Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zU9kOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CentriQ, a calibration-free post-training quantization framework for diffusion transformers. The paper observes that adaptive LayerNorm introduces a token-dependent channel mean into DiT activations, and argues that conventional Hadamard rotations may concentrate this DC component into a few coordinates, thereby dominating the activation quantization range. CentriQ subtracts the per-token channel mean before rotation and restores its contribution exactly through a full-precision rank-1 correction. The remaining centered activation is processed by a factorized Hadamard transform and dynamically quantized. The paper further introduces RoDS, an L_p-based grouped weight quantizer designed for heavy-tailed DiT weights.

Critical strengths
  1. The paper identifies an interesting failure mode of Hadamard-based activation quantization in adaLN-based DiTs.
  2. At W4A4, the component ablation does show that centering is useful.
  3. The paper is clear and well presented.
Critical weaknesses
  1. Mean-centered DiT quantization is already closely related to existing work. HadaNorm already studies mean-centered transformations for DiT quantization, while ConvRot explicitly addresses the mean component using regular group-wise Hadamard transforms.
  2. The main calibration-free baseline comparison is not sufficiently strong. Table 1 primarily compares CentriQ against RTN, NF4, and HQQ combined with plain per-token activation quantization. These are not competitive calibration-free DiT activation quantizers. Stronger related methods such as ConvRot and OrbitQuant are discussed by the paper itself but are absent from the main fidelity comparison.
  3. The “calibration-free” claim should be qualified more carefully. The paper states that the rotation layout and clipping percentile were selected using a disjoint 1,000-prompt MJHQ development split.
  4. The practical efficiency benefit is weak. On FLUX.1-schnell, the reported end-to-end latency improves only from 1.77 s in BF16 to 1.69 s with W4A4 CentriQ.
Questions for authors
  1. Can the authors report transformer-only latency, activation preprocessing latency, rotation latency, INT4 GEMM latency, epilogue latency, peak memory, and throughput separately?
8 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Thank you for your detailed and constructive feedback. We greatly appreciate your recognition that our paper identifies an interesting failure mode of Hadamard-based activation quantization in adaLN-based DiTs, that centering is useful at W4A4, and that the paper is clear and well presented. We address your major questions and concerns below.

W1. Overlap with HadaNorm and ConvRot. We agree centering is not new and will say so in Section 2. The difference is which mean is removed. HadaNorm removes a per-channel mean over tokens (with calibrated scales), which leaves the per-token adaLN offset in place. ConvRot mixes only within small regular-Hadamard blocks, so the mean is not spread across the full width. CentriQ removes the per-token mean exactly (Wx = W(x − μₜ1) + μₜ·W1) and then rotates globally, without data.

W2. The calibration-free baselines in Table 1 are weak; ConvRot and OrbitQuant are missing. We agree that RTN, NF4 and HQQ with plain per-token activations are weak baselines. Following the reviewer's suggestion, we added the closest calibration-free activation quantizers to the fidelity comparison. We reimplemented ConvRot (its release uses NVFP4/FP8, not our INT4 grid) and HadaNorm (no public code) under identical weights, INT4 grid and clip, and also reproduced calibrated SVDQuant in our own harness (MJHQ-5k):

MJHQ-5k, W4A4 unless notedFIDIRPSNRLPIPS
PixArt-Σ
FP1616.660.952––
ConvRot-style36.570.73315.010.365
HadaNorm-style42.570.35411.800.513
SVDQuant (calibrated, ours)19.270.87114.770.312
CentriQ (no calibration)19.100.88516.180.278
FLUX.1-schnell
BF1619.450.966––
ConvRot-style18.880.92618.130.181
HadaNorm-style18.900.89612.470.433
SVDQuant (calibrated, ours)18.730.94418.370.171
CentriQ (no calibration)18.800.93918.400.173
FLUX.1-schnell, W4A2
ConvRot-style42.580.26613.020.507
HadaNorm-style43.170.28112.930.512
CentriQ (no calibration)35.590.39413.620.457

On PixArt-Σ, CentriQ is clearly better than both alternatives (FID 19.10 vs 36.57 and 42.57). On FLUX at 2-bit activations, the gap is also large (FID 35.59 vs 42.58 and 43.17). On FLUX at W4A4, all methods are close, and calibrated SVDQuant is slightly ahead on FID, IR and LPIPS. The SVDQuant rows are our reproduction in the same pipeline, not the published numbers. We have added these results to the revised paper (Table 5, in red).

OrbitQuant reports only GenEval and uses a Lloyd–Max codebook rather than an integer grid, so we compare on its own protocol: at W2A4, CentriQ scores 0.647 vs 0.604 for OrbitQuant. In addition, on a 1,000-prompt paired check, an OrbitQuant-style activation path (permuted block-Hadamard on our INT4 grid) reaches PixArt-Σ FID-1k 131.7, vs 47.7 for CentriQ.

W3. Qualifying "calibration-free". CentriQ collects no activation statistics; scales and W·1 are closed-form. The disjoint development split was used once to fix two global settings shared by all models. These settings transfer unchanged to a model we never tuned on (FLUX.1-dev, 500 MJHQ prompts):

FLUX.1-dev, 500 MJHQ prompts, 28 stepsCalibrated on FLUX.1-dev?FID-500 ↓
BF16–77.71
SVDQuant (official FLUX.1-dev checkpoint)yes77.66
CentriQ (settings from the paper, unchanged)no77.59

CentriQ matches the model-specific calibrated baseline without any data from the new model. (FID on 500 images is higher than FID on 5,000 and is not comparable to the MJHQ-5k numbers above.) We have added this definition of calibration-free to Section 1 (in red).

W4. The efficiency benefit is weak. The latency in Section 5.1 came from an unoptimized prototype without the engine-level fusions of SVDQuant's Nunchaku engine. We have now optimized CentriQ the same way and run both 4-bit methods in the same engine on the same layers, with identical settings for all methods (1024², 4 steps, batch 1, mean ± std):

Transformer (s)BF16BF16 + compileSVDQuantCentriQ
A100, INT4 (150 imgs)1.762 ± 0.0081.407 ± 0.0051.245 ± 0.0051.289 ± 0.005
RTX PRO 6000, NVFP4 (50 imgs)1.166 ± 0.0011.006 ± 0.0010.646 ± 0.0000.664 ± 0.000
End to end, A100 / RTX2.000 / 1.3381.644 / 1.1781.480 / 0.8171.522 / 0.834

CentriQ is within 2.8–3.5% of calibrated SVDQuant and 1.37× (A100) / 1.76× (RTX) faster than BF16.

Q1. Per-component latency, memory, throughput.

A100, ms per imageBF16SVDQuantCentriQ
Centering + rotation (one fused kernel)––22.8
Activation quantization–118.7122.8
INT4 GEMM + fused epilogue–428.8441.9
BF16 GEMM956.1 (all layers)262.1263.7
Attention302.6297.6299.0
Norm / RoPE / elementwise503.5132.4132.7

Centering and rotation run as one fused kernel, and the epilogue is fused into the GEMM, so neither can be timed separately. Peak memory: 36.3 GB (BF16), 26.6 GB (SVDQuant), 26.8 GB (CentriQ). Throughput: 0.66 / 1.20 img/s on A100 / RTX (BF16: 0.50 / 0.75). All added to the revised paper.

We hope these results address the reviewer's concerns, and we would be glad to clarify any remaining issue.

AuthorsOpenReview ↗

Kind request to check our responses

Dear Reviewer, thank you for your detailed review. Your feedback have helped us strengthen the paper. As the discussion period is coming to an end, we wanted to kindly check whether you have had a chance to look at our response. Following your suggestions, we have:

  1. Stronger baselines: added ConvRot- and HadaNorm-style activation paths and calibrated SVDQuant to the main comparison. CentriQ is clearly better than both alternatives on PixArt-Σ and at 2-bit activations.
  2. OrbitQuant: compared on its own protocol.
  3. Calibration-free: qualified the claim, and showed the settings transfer unchanged to FLUX.1-dev.
  4. Efficiency: reported per-component latency, memory and throughput.

We hope these additions address your concerns, and we would be grateful if you could take them into account in your final assessment. We would be glad to clarify any remaining points.

Reviewer_Z7KVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CentriQ addresses calibration-free DiT quantization by removing each token's channel mean before rotation and restoring its contribution through a full-precision rank-one branch. It combines this with online activation scaling and robust weight quantization. Experiments cover three DiT families, W4A4 quality and fidelity, simulated sub-4-bit configurations, and A100 latency.

Critical strengths
  1. Clear diagnosis and targeted intervention. The exact decomposition in Eq. (4) directly addresses mean-induced quantization error. In Table 2, adding centering and reinjection reduces FLUX W4A4 FID from 25.65 to 18.46, supporting the proposed mechanism.

  2. Evidence across architectures and precisions. Shared settings preserve W4A4 quality across three DiT families. Table 6 shows benefits with existing weight quantizers; simulated FLUX W2A4 reaches 0.647 GenEval versus OrbitQuant's reported 0.604, and W4A2 retains 0.573.

Critical weaknesses
  1. The comparative activation-path advantage remains incompletely established. Table 2 isolates centering against rotation-only, and Section 2 distinguishes HadaNorm/ConvRot. However, Table 1 uses published SVDQuant results and untransformed activations for RTN/NF4/HQQ, without a matched comparison to those closer alternatives. Table 6 shows small RoDS gains on FLUX/SANA but a larger gain on PixArt. A feasible activation-path comparison would strengthen the central contribution; HadaNorm's unavailable implementation is a recognized constraint.

  2. The deployment advantage is incompletely characterized. Latency improves from 1.77 s to 1.69 s over six images, without a matched low-bit timing baseline. This supports modest acceleration, while sub-4-bit results remain simulated. Conversion time and total storage/peak memory, beyond the disclosed 4.5-bit grouped representation, would help quantify the benefit of avoiding calibration.

This is a borderline weak rejection. I would consider raising the rating to weak acceptance if the rebuttal establishes the activation-path advantage through controlled comparisons and clarifies the deployment trade-off.

Questions for authors
  1. With the weight quantizer and activation grid fixed, how does centering with exact reinjection compare with the closest feasible mean-handling or Hadamard alternative?

  2. Can you report repeated latency measurements with variability against Nunchaku-based SVDQuant under identical execution settings?

  3. What are the one-off conversion time, total checkpoint size, and peak memory, including padding and auxiliary branches, relative to the calibrated baseline?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Thank you for your thoughtful feedback, and for clearly stating what would change your assessment. We greatly appreciate your recognition of the clear diagnosis and targeted intervention, and of the evidence across architectures and precisions. We address both conditions below.

Q1. Activation path vs. the closest alternatives, with weights and grid fixed. Following the reviewer's suggestion, we reimplemented HadaNorm (no public code) and ConvRot (its release uses NVFP4/FP8, not our INT4 grid) with the RoDS weights, INT4 grid and clip fixed, changing only the activation transform. We also reproduced calibrated SVDQuant in our own harness (MJHQ-5k):

MJHQ-5k, W4A4 unless notedFIDIRPSNRLPIPS
PixArt-Σ
FP1616.660.952––
ConvRot-style36.570.73315.010.365
HadaNorm-style42.570.35411.800.513
SVDQuant (calibrated, ours)19.270.87114.770.312
CentriQ (no calibration)19.100.88516.180.278
FLUX.1-schnell
BF1619.450.966––
ConvRot-style18.880.92618.130.181
HadaNorm-style18.900.89612.470.433
SVDQuant (calibrated, ours)18.730.94418.370.171
CentriQ (no calibration)18.800.93918.400.173
FLUX.1-schnell, W4A2
ConvRot-style42.580.26613.020.507
HadaNorm-style43.170.28112.930.512
CentriQ (no calibration)35.590.39413.620.457

Bold marks the best quantized value. As in Section 2, the methods differ in which mean they remove: HadaNorm removes a per-channel mean over tokens and ConvRot mixes only within small blocks, while CentriQ removes the per-token adaLN mean exactly and rotates globally. On PixArt-Σ, CentriQ is clearly better than both alternatives (FID 19.10 vs 36.57 and 42.57). On FLUX at 2-bit activations, the gap is also large (FID 35.59 vs 42.58 and 43.17). On FLUX at W4A4, all methods are close, and calibrated SVDQuant is slightly ahead on FID, IR and LPIPS. The SVDQuant rows are our reproduction in the same pipeline, not the published numbers. We have added these results to the revised paper (Table 5, in red).

Q2. Repeated latency vs. Nunchaku SVDQuant under identical settings. The latency in Section 5.1 came from an unoptimized prototype without the engine-level fusions of SVDQuant's Nunchaku engine. We have now optimized CentriQ the same way and run both 4-bit methods in the same engine on the same layers, with identical settings for all methods (1024², 4 steps, batch 1, mean ± std):

Transformer (s)BF16BF16 + compileSVDQuantCentriQ
A100, INT4 (150 imgs)1.762 ± 0.0081.407 ± 0.0051.245 ± 0.0051.289 ± 0.005
RTX PRO 6000, NVFP4 (50 imgs)1.166 ± 0.0011.006 ± 0.0010.646 ± 0.0000.664 ± 0.000
End to end, A100 / RTX2.000 / 1.3381.644 / 1.1781.480 / 0.8171.522 / 0.834

CentriQ is within 2.8–3.5% of calibrated SVDQuant and 1.37× (A100) / 1.76× (RTX) faster than BF16. The remaining gap is CentriQ's fused centering-and-rotation pass (22.8 ms) and its wider side branch (13.1 ms), the price of needing no calibration.

Q3. Conversion time, checkpoint size, peak memory.

  • Conversion: 3.1 min (PixArt-Σ) and 9.3 min (FLUX) without data, vs 13.0 h for SVDQuant on FLUX.
  • Checkpoint: the FLUX checkpoint is 13.03 GiB with the paper's layer set, which keeps adaLN and the single-block output projections in BF16. With the same quantized layers as SVDQuant (6.28 GiB), CentriQ would be 6.32 GiB, including padding, scales and W·1.
  • Peak memory: 26.8 GB (CentriQ) vs 26.6 GB (SVDQuant) vs 36.3 GB (BF16).

We hope these results establish the activation-path advantage and clarify the deployment trade-off.

AuthorsOpenReview ↗

Adding per component latency

Per-component latency

A100, ms per imageBF16SVDQuantCentriQ
Centering + rotation (one fused kernel)––22.8
Activation quantization–118.7122.8
INT4 GEMM + fused epilogue–428.8441.9
BF16 GEMM956.1 (all layers)262.1263.7
Attention302.6297.6299.0
Norm / RoPE / elementwise503.5132.4132.7

Centering and rotation run as one fused kernel, and the epilogue is fused into the GEMM. All added to the revised paper.

AuthorsOpenReview ↗

Kind request to check our responses

Dear Reviewer, thank you for clearly stating what would change your assessment. We have addressed both conditions:

  1. A controlled activation-path comparison with weights, grid and clip fixed. CentriQ beats HadaNorm- and ConvRot-style transforms on PixArt-Σ (FID 19.10 vs 42.57 / 36.57) and on FLUX W4A2 (35.59 vs 43.17 / 42.58).
  2. Latency against Nunchaku SVDQuant with mean ± std. CentriQ is within 2.8–3.5% of SVDQuant. We also report conversion time, checkpoint size and peak memory.

We would be grateful if you could take a look, and we are happy to address any further concerns.

AuthorsOpenReview ↗

Kind reminder: both stated conditions addressed

Dear Reviewer, as the discussion period is coming to an end, we wanted to kindly follow up. We have addressed both conditions you stated (controlled activation-path comparison and deployment trade-off). If our response fulfills your request, we would be grateful if you could consider raising your rating.

Reviewer_hoG8OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

CentriQ studies calibration-free weight and activation quantization for diffusion transformers. It identifies token-wise means as a source of outliers after Hadamard rotation. The method centers each token, quantizes the rotated residual, and restores the removed component through a full-precision rank-one branch. A robust weight objective complements this activation path. Experiments cover three image-generation models and two quality benchmarks, with additional compositional evaluation on FLUX.1-schnell. W4A4 quality is strong. Lower-bit simulations remain promising, while an INT4 implementation provides a modest end-to-end speedup on an A100 GPU.

Critical strengths
  1. The paper identifies a concrete activation failure mode. Mean-centering removes the offset through an exact linear decomposition. Activation-range diagnostics support the mechanism (Section 4).

  2. W4A4 preserves quality across three DiT families and two benchmarks. The activation ablation shows clear gains beyond rotation alone, including FID improving from 25.65 to 18.46 on FLUX (Tables 1–2).

  3. The design quantizes both operands without fitting activation calibration statistics. Fixed settings, weight-quantizer ablations, and a working INT4 implementation strengthen the practical evidence.

Critical weaknesses
  1. The main fidelity comparison needs a clearer common protocol. Table 1 imports SVDQuant results, while CentriQ uses the current evaluation harness. Matching references and metric implementations are not established. This limits interpretation of the reported LPIPS advantage.

  2. Acceleration is measured only on FLUX.1-schnell and one A100 GPU. Latency improves from 1.77 to 1.69 seconds. The six-image average does not report variability. Lower-bit experiments use simulation, so their hardware benefits remain unverified.

Questions for authors
  1. Table 1 imports SVDQuant results, while CentriQ consistently improves LPIPS. Please clarify whether both results use matching full-precision references, seeds, schedulers, and LPIPS implementations. Were any protocol differences identified when reproducing SVDQuant for GenEval? This would help interpret the perceptual-fidelity advantage over the calibrated baseline.

  2. The method uses fixed rotation and clipping settings without per-checkpoint activation calibration. Were development choices shared across checkpoints, or selected separately for each one? Please identify which settings transfer unchanged to a new fine-tune. This would clarify the practical meaning of calibration-free deployment beyond the evaluated models.

  3. Section 5.1 reports 1.77 s for BF16 and 1.69 s for CentriQ. Does the BF16 transformer also use CUDA graphs? Please summarize any existing timing breakdown for the transformer and centering path, if available. This would clarify whether the modest end-to-end gain reflects kernel savings or pipeline overhead.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We appreciate the reviewer's positive feedback and recognition of the concrete failure mode we identify, the strong W4A4 quality across DiT families, and the practical evidence of our INT4 implementation. We address the reviewer's questions below.

The main fidelity comparison needs a common protocol. Q1. Do SVDQuant and CentriQ use matching references, seeds, schedulers and LPIPS? Any protocol differences for GenEval? Thank you for raising this important point. Table 1 imported the published SVDQuant numbers. Following the reviewer's suggestion, we have regenerated SVDQuant in our pipeline with the same prompts, seeds, scheduler and metric code, swapping only the transformer (public Nunchaku checkpoint for FLUX; our own calibration for PixArt-Σ, which has no public checkpoint):

MJHQ-5k, W4A4 unless notedFIDIRPSNRLPIPS
PixArt-Σ
FP1616.660.952––
ConvRot-style36.570.73315.010.365
HadaNorm-style42.570.35411.800.513
SVDQuant (calibrated, ours)19.270.87114.770.312
CentriQ (no calibration)19.100.88516.180.278
FLUX.1-schnell
BF1619.450.966––
ConvRot-style18.880.92618.130.181
HadaNorm-style18.900.89612.470.433
SVDQuant (calibrated, ours)18.730.94418.370.171
CentriQ (no calibration)18.800.93918.400.173
FLUX.1-schnell, W4A2
ConvRot-style42.580.26613.020.507
HadaNorm-style43.170.28112.930.512
CentriQ (no calibration)35.590.39413.620.457

The ConvRot- and HadaNorm-style rows are our reimplementations under identical weights, INT4 grid and clip.

Without any calibration data, CentriQ matches calibrated SVDQuant on FLUX.1-schnell and is closer to the full-precision images on PixArt-Σ. We have added these matched rows to the revised paper (Tables 2 and 5, in red).

For GenEval, we did identify a protocol difference: our earlier SVDQuant run used Nunchaku's evaluation script, which has a different seed scheme and maximum text length. Rerun in our pipeline under the same protocol as CentriQ, SVDQuant scores 0.664, comparable to CentriQ (0.670) and full precision (0.667).

Q2. Were development choices shared across checkpoints? Which settings transfer to a new fine-tune? All settings are shared across models and none is chosen per checkpoint; scales and W·1 are closed-form, so a new model only needs the 3–9 min conversion. To test transfer to a new fine-tune, we applied CentriQ with all settings frozen to FLUX.1-dev, which the paper does not evaluate, and compared it with SVDQuant calibrated specifically for that model:

FLUX.1-dev, 500 MJHQ prompts, 28 stepsCalibrated on FLUX.1-dev?FID-500 ↓
BF16–77.71
SVDQuant (official FLUX.1-dev checkpoint)yes77.66
CentriQ (settings from the paper, unchanged)no77.59

FID on 500 images is higher than FID on 5,000 and is not comparable to the MJHQ-5k numbers above.

CentriQ matches the model-specific calibrated baseline without any data from the new model.

Acceleration is measured on one GPU, over six images, without variability. Q3. Does BF16 also use CUDA graphs? Is there a timing breakdown? Thank you for this question. The latency in Section 5.1 came from an unoptimized CentriQ prototype, which used Nunchaku’s 4-bit GEMMs but none of the engine-level fusions that SVDQuant’s Nunchaku engine uses. Following the reviewer’s suggestion, we optimized CentriQ in the same way and re-measured all methods under one protocol on two GPUs, with the same execution settings (CUDA graphs off) for every method, including BF16. The earlier 1.77 s BF16 figure matches the new transformer-only time (1.762 s); 2.000 s is end to end.

Transformer (s)BF16BF16 + compileSVDQuantCentriQ
A100 (150 imgs)1.762 ± 0.0081.407 ± 0.0051.245 ± 0.0051.289 ± 0.005
RTX PRO 6000 (50 imgs)1.166 ± 0.0011.006 ± 0.0010.646 ± 0.0000.664 ± 0.000

Kernel breakdown on the A100 (ms per image):

ComponentBF16SVDQuantCentriQ
Centering + rotation (one fused kernel)––22.8
Activation quantization–118.7122.8
INT4 GEMM + fused epilogue–428.8441.9
BF16 GEMM956.1 (all layers)262.1263.7
Attention302.6297.6299.0
Norm / RoPE / elementwise503.5132.4132.7

With the same optimizations, CentriQ is within 2.8–3.5% of calibrated SVDQuant without any calibration, and its quantized layers are 1.18× (A100) and 2.42× (RTX PRO 6000) faster than BF16. We have added these results to Section 5.1 (in red).

AuthorsOpenReview ↗

Kind request to check our responses

Dear Reviewer, thank you again for your positive review. Your questions were very helpful in strengthening the paper. Following your suggestions, we have:

  1. Common protocol: regenerated SVDQuant with the same references, seeds, scheduler and metric code. CentriQ matches calibrated SVDQuant.
  2. Transfer: with all settings frozen, CentriQ matches SVDQuant calibrated for FLUX.1-dev (FID 77.59 vs 77.66).
  3. Latency: repeated measurements with mean ± std and a per-kernel breakdown, in the same engine as SVDQuant.

We hope these results resolve your remaining concerns, and we would be grateful if you could take them into account in your final assessment. We are happy to clarify anything further.

AuthorsOpenReview ↗

Kind reminder: all questions addressed

Dear Reviewer, thank you again for your positive review. As the discussion period is coming to an end, we wanted to kindly check whether you had a chance to look at our responses on the common protocol, transfer and latency. We would be grateful if you could take them into account in your final assessment.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of the Rebuttal

We thank all reviewers for their thoughtful and constructive feedback. We greatly appreciate the recognition that our paper identifies a concrete failure mode of Hadamard-based activation quantization in DiTs (all reviewers). Moreover, the reviewers acknowledged that exact centering with rank-1 re-injection is a targeted fix (FLUX W4A4 FID 25.65 → 18.46, Table 2), and that W4A4 quality is preserved across three DiT families (Z7KV, hoG8). Following the reviewers' suggestions, we have conducted the following additional experiments:

  • Controlled activation-path comparison (zU9k, Z7KV). With the weights, INT4 grid and clip fixed, CentriQ clearly outperforms ConvRot- and HadaNorm-style transforms on PixArt-Σ (FID 19.10 vs 36.57 and 42.57) and at 2-bit activations on FLUX.1-schnell (35.59 vs 42.58); at W4A4 on FLUX all methods are close.
  • SVDQuant under a common protocol (hoG8). Regenerated in our pipeline with identical references, seeds and metric code, CentriQ matches calibrated SVDQuant on FLUX.1-schnell without any calibration data (FID 18.80 vs 18.73), is comparable on GenEval (0.670 vs 0.664), and is closer to the full-precision images on PixArt-Σ.
  • Transfer to a new checkpoint (hoG8, zU9k). With all settings frozen, CentriQ matches an SVDQuant model calibrated specifically for FLUX.1-dev (FID 77.59 vs 77.66).
  • Matched latency (all reviewers). The latency in Section 5.1 came from an unoptimized prototype, without the engine-level optimizations that SVDQuant's Nunchaku engine uses. We have now applied the same optimizations to CentriQ and run both methods in the same fused engine, on the same quantized layers, averaged over many images. CentriQ is within 2.8–3.5% of calibrated SVDQuant on two GPUs, without any calibration, and up to 1.76× faster than BF16.

These additional experiments have strengthened the paper and clarified its contributions, and we have added all of them to the revised version (marked in red).

Area_Chair_VyoBOpenReview ↗

Reminder: Please Respond to Author Rebuttals

Dear Reviewers,

As the discussion period is coming to an end, please review the authors’ responses and provide a brief follow-up on whether your concerns have been addressed.

Thank you.

AuthorsOpenReview ↗

Summary of the discussion

Dear Area Chair(s),

Thank you for reminding the reviewers. Following our earlier note, we would like to respectfully summarize the status of the discussion.

Reviewer hoG8 (weak accept): no response yet.

  • Asked for a common protocol with SVDQuant, transfer to a new checkpoint, and repeated latency measurements.
  • All addressed: CentriQ matches calibrated SVDQuant, including on FLUX.1-dev with all settings frozen.

Reviewer Z7KV (weak reject; would consider weak accept if two points addressed): no response yet.

  • The two points: controlled activation-path comparisons, and a clear deployment trade-off.
  • Both addressed: CentriQ beats ConvRot- and HadaNorm-style transforms under identical settings, and runs within 3.5% of SVDQuant's latency without its 13 h of calibration.

Reviewer zU9k (weak reject):

  • Original review: named ConvRot and OrbitQuant as missing baselines, and HadaNorm as the closest related work. We added all three in the rebuttal.
  • First reply: did not engage with these results, and asked for "more recent and competitive methods" without naming any.
  • Second reply: named SVDQuant, ConvRot, OrbitQuant, ViDiT-Q and SegQuant. All except SegQuant were already compared in the paper (Tables 1, 2, 5 and 8), and SegQuant does not evaluate W4A4. The Reviewer Guide asks reviewers to consult supplementary material "before asking authors."
  • Using reported numbers, CentriQ outperforms ViDiT-Q at W4A4 and W2A4, OrbitQuant at W2A4, and ConvRot on every metric.
  • ConvRot and OrbitQuant are arXiv-only. Per the Reviewer Guide, authors are not required to compare against such papers, and "the lack of such comparisons cannot be a basis for rejection." We included them out of goodwill.

We would be very grateful if you could take this into account in your assessment.

Thank you for your time and support!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.