- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Q-Drift, a training-free fix for sampling with quantized diffusion models. During a calibration phase, it runs the full-precision and quantized models side by side and estimates the variance of the quantization error at each step. At sampling time, this variance is used to rescale the update. Model weights are not touched. Experiments cover six models and three samplers. FID goes down in all seven, and CLIP score stays about the same. Under milder quantization, the results are mixed.
Critical strengths
1.The method is simple and cheap. After calibration it only needs one scalar per step, so it is easy to drop into an existing quantized pipeline. I also liked the SDXL check with only 10 calibration runs: even in the three stress cases the authors picked, the result stays close to the 1,000-run version (Table 3). This suggests calibration cost is not a real concern.
2.The gains are not just one FID number. The method is tested on both U-Net and transformer backbones, e.g. PixArt-Sigma goes from 48.85 to 44.06 FID and Sana from 17.32 to 16.54 (Table 1). Appendix E adds KID and paired bootstrap intervals. Six of the seven intervals support an improvement, and the authors are upfront that the small FLUX.1-dev gain is not significant.
Critical weaknesses
1.My main concern is the gap between the theory and what is actually implemented. In Section 4.2, the correction strength is derived by treating the quantization error as independent, zero-mean Gaussian noise. But the actual update keeps the real error, does not subtract its mean, and scales it together with the prediction. So the error is neither zero-mean nor independent across steps in practice. Matching the variance of an idealized noise model does not show that this update brings the sampling distribution closer to the full-precision one. Appendix D looks at spatial statistics, but not the correlation across steps, which is what the SDE argument depends on. Right now I see this as a reasonable heuristic motivated by the SDE view, not a correction with a marginal-preserving guarantee. The paper should say this more clearly.
2.The experiments don't show that this specific scaling rule is what matters. Table 3 compares different variance estimates, but there is no simple baseline such as a constant multiplier or a tuned per-step schedule. Without that, it is hard to know whether the gain comes from the variance-based rule or just from damping the update a bit. The comparison with earlier correction methods is only on SDXL/Euler, where some parts of those methods can't be used anyway. Also, in Table 5, four of the six SVDQuant W4A4 settings get slightly worse FID after correction. So it is not clear when Q-Drift should be preferred over a simpler adjustment.
Questions for authors
1.Along the corrected trajectories, how large is the mean of the quantization error, and how correlated is it across steps? If the mean is not removed, why should the residual variance alone decide the correction strength?
2.Could you compare against a constant multiplier and a simple step-dependent multiplier, both tuned on a separate validation set and run with the same number of steps? I think this is the key experiment for judging the contribution.
3.Appendix C.1 calls the flow-matching version an empirical extension. Which of the marginal-preservation arguments are meant to hold for that version, and for the multistep solver?
2 comments
Kind request to check our response
Dear Reviewer, we ran the comparison with a tuned constant multiplier, which you called the key experiment. We would be grateful if you could check our response and consider reconsidering your score.
Thank you for the constructive review, and for recognizing the simplicity of Q-Drift, the robustness of calibration with only 10 runs, and the statistical care of our evaluation. We address your questions below.
W1/Q1. Error mean and cross-step correlation. Following the reviewer's suggestion, we measured them along the trajectories (FLUX.1-schnell W3A4, 1,000 paired trajectories):
The mean is small (1–4% of the std), which is why the residual variance sets the correction strength. The error is, however, correlated across steps (0.17–0.51; PixArt-Σ 0.26–0.39). The quantized prediction also carries less variance than FP and is anti-correlated with its error, so each Euler step is too short. We agree that Q-Drift is a heuristic motivated by the SDE view, not a correction with a marginal-preservation guarantee, and the paper will state this clearly.
W2/Q2. Constant and step-dependent multipliers tuned on a validation set. We ran the key experiment as specified, tuning on 1,000 validation prompts and evaluating on held-out test prompts (5,000 for FLUX, 1,000 for SDXL-Turbo) with the same steps and seeds:
A constant tuned by FID reaches a lower FID, but it over-lengthens the step and loses perceptual quality (ImageReward −0.05 on FLUX, −0.07 on SDXL-Turbo). Tuning the constant by ImageReward instead does not help: on SDXL-Turbo validation, every constant (1–16×) scores below the uncorrected model (ImageReward 0.655–0.812 vs 0.823), and the best one (1×: FID 58.69, ImageReward 0.812) is worse than Q-Drift on both metrics (58.38, 0.824). Q-Drift improves FID with ImageReward essentially unchanged and needs no FID search; the tuned constant also changes about 20× across models (0.20 on FLUX, 0.066 on SDXL-Turbo, 0.02 on SDXL, 0.01 on Sana). On FLUX, the step-dependent multiplier matches Q-Drift's FID with lower ImageReward. Note that the gain comes from lengthening the step, the opposite of damping: shrinking it (c* < 0) worsens FID (54.29 vs 52.90). We will add these baselines to the main table and move Table 5 into the main text.
Q3. Scope of the arguments.
Given the measured correlations, none of these is a guarantee, and we will state this explicitly.