acceptodds
Under review as a conference paper at ICLR 2027

FROM PREDICTABLE DISCREPANCY TO RESIDUAL VARIANCE: CORRECTING QUANTIZED DDIM SAMPLING

Abstract

Post-training quantization (PTQ) reduces diffusion-model inference cost, but denoiser-output discrepancies perturb subsequent sampling states. We introduce State-Conditioned Predictive Error Correction (SPEC), a two-stage output corrector for a fixed quantized denoiser under DDIM-family sampling. At sparse refresh steps, its temporal branch learns an output-matching offset for the quantized denoiser's time condition from full-precision supervision on the same latent state; the DDIM grid and transition coefficients remain nominal. A full-resolution branch is then trained on trajectories recollected with this adjustment active to predict the remaining same-state output discrepancy. On held-out trajectories, the Predictable Error Ratio (PER) shows that SPEC reduces the summed squared same-state output discrepancy between the quantized and full-precision denoisers by 82.7%–88.1% across the evaluated bitwidths. For stochastic DDIM, Student-\(t\)-estimated variance reallocation (\(t\)-VR) models the remaining post-SPEC residual variance and reallocates the nominal stochastic variance between its estimated contribution and the explicit Gaussian innovation. With 100-step stochastic DDIM on CIFAR-10 at W4A8, FID falls from 6.84 for Q-Diffusion to 5.83 for SPEC and 5.66 with \(t\)-VR.

Primary area:
generative models
Keywords:
Diffusion Models, Post-Training Quantization, Low-Bit Quantization, Model Compression, Efficient Deep Learning, Quantization Error Correction, Generative Models, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tvKDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets the per-step output discrepancy of a post-training-quantized diffusion denoiser under DDIM-family sampling. With the quantized backbone and DDIM schedule fixed, the authors propose SPEC, a 27.9K-parameter sampling-time corrector that adjusts the quantized denoiser’s time input at sparse steps and then predicts the remaining output error, trained in two stages on recollected trajectories. For stochastic DDIM, t-VR additionally fits a Student-t to the post-correction residual and subtracts its estimated variance from the injected Gaussian noise.

Critical strengths

S1. The decomposition of quantization error into a predictable same-state component and a post-correction residual, measured by PER, gives a clear diagnostic of what the corrector actually removes.

S2. Training the error branch on trajectories recollected after the temporal branch is active is a sound design choice that aligns training states with deployment states.

S3. The corrector is lightweight (27.9K parameters, 0.36 GPU-h offline on CIFAR-10) and leaves the quantized backbone and DDIM schedule untouched, so it is orthogonal to backbone-side PTQ methods.

Critical weaknesses

W1. (Marginal and statistically uncertain gains.) The improvements over Q-Diffusion on the same backbone are small relative to the reported noise: 4.89→4.81 (±0.03) on CIFAR-10 W4A8 at η=0, 3.97→3.90±0.02 and 5.55→5.49±0.05 at η=0.5, and 10.78→10.63 on LSUN-Bedroom W8A8 with only 3k samples, while LSUN-Church W8A8 regresses (7.50→7.56). Moreover, on CIFAR-10 the quantized model already has lower FID than FP32 (3.80 vs 4.22 at η=0; 4.31 vs 5.78 at η=1), so FID cannot tell whether SPEC recovers the full-precision behavior it is trained to match. Since the method’s stated objective is fidelity to FθF_\theta, Table 1 should report distance-to-FP32 metrics (e.g., LPIPS or MSE between SPEC and FP32 samples generated from the same initial noise), along with multi-seed results and larger sample counts on LSUN.

W2. The DPM-Solver++ transfer test is too thin to support any claim about the method’s scope. Appendix C.2 applies the DDIM-trained SPEC to deterministic third-order singlestep DPM-Solver++ without retraining, and FID roughly doubles at both W8A8 and W4A8. The authors hypothesize stage-wise residual coupling and a state-distribution mismatch as the cause, but provide no diagnostic to test it and do not explain how the DDIM-grid refresh set and integer time offsets are mapped to solver stages. Although they state that solver-matched training would be needed, they never run it, even though the staged training recipe is solver-agnostic and costs under 1 GPU-h. It therefore remains unclear whether the failure is a design limitation or a training-distribution mismatch. The few-step multistep regime that matters most for low-bit PTQ is also untested, and since t-VR requires stochastic sampling, the full pipeline is demonstrated only for stochastic DDIM.

Questions for authors

Q1. For readers’ ease of following, please define in equation form (i) the SmoothL1 function in Eq. (8), including β\beta and the element reduction, and (ii) the “Relative Error Energy” on the y-axis of Figure 1, including which states are pooled and how it relates to PER in Eq. (4). On (i), with the default β=1\beta=1 and discrepancies mostly far below 1, SmoothL1 reduces to MSE/2, so please clarify the chosen β\beta and why SmoothL1 was preferred over MSE.

Q2. What is the performance when the state-conditioned temporal branch is replaced by a per-timestep constant offset lookup table? Please also report the distribution of the learned oracle offsets δt∗\delta^*_t across timesteps, including their sign, magnitude, and variance. If the within-timestep variance is small, state-conditioning would appear unnecessary; we would welcome the authors’ view on this.

Q3. In Table C-1, training the error branch with the MSE objective alone reduces IS from 9.36 to 8.54, below the uncorrected Q-Diffusion baseline. What does the corrector sacrifice in this setting, and why does adding the directional and magnitude terms recover IS while leaving FID nearly unchanged?

Reviewer_gNdeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Model quantization enables running deep neural networks with reduced cost. However, the quantization error may hurt the model’s ability. For diffusion models, this issue is more pronounced as the sampling process will run the quantized model iteratively, leading to accumulated error. To address this issue, this paper introduces State-Conditioned Predictive Error Correction (SPEC), a two-stage output corrector.

Critical strengths
  • The paper targets an important issue
Critical weaknesses
  • The proposed method is not well motivated and has limited novelty. The idea of correcting input temporal conditions has been proposed by StepbaQ[b], and the idea of variance relocation has been introduced by PTQD[a]. Unlike StepbaQ or PTQD, which are calibration-based methods that can smoothly work with pretrained quantized models, the proposed SPEC in this work requires an additional network for temporal correction and discrepancy prediction. Such a modification greatly harms the applicability of the method, as it requires an additional network and training cost. Moreover, unlike prior works providing clear theoretical discussion to justify the correction, this work does not provide the motivation to opt for a more complicated solution.
  • The comparison with several important prior works is missing, such as [a, b, c]. Therefore, the advantage over prior calibration-only methods is unclear, providing limited motivation to add an additional network in the correction design.

[a] PTQD: Accurate post-training quantization for diffusion models. NeurIPS 2023

[b] Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps, ICLR 2024.

[c] StepbaQ: Stepping backward as Correction for Quantized Diffusion Models, NeurIPS 2024.

Questions for authors
  • The paper is poorly organized, and the writing quality is low. The notations are not clearly introduced. For example, notation such as \hat{e_t} is used in Eq. 6 but is only introduced in Eq. 10. Additionally, in the introduction section, the authors do not mention that the SPEC method is the proposed method in the third paragraph, which is confusing.
  • The experiment does not cover popular text-to-image generation models such as Stable Diffusion, PixArt-α, and few-step methods like SDXL-Turbo. Therefore, whether this method applies to those advanced models remains unknown.
  • The experimental setups are not consistent. Tables 2 and 3 do not include a comparison with prior correction methods. In Table 1, the number of baselines for different models also varies.
Reviewer_uj91OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a two-stage correction method for sampling with a fixed quantized diffusion model. The first stage adjusts the denoiser’s time input to better match the full-precision prediction without changing the DDIM sampling grid. The second stage learns the remaining output correction using trajectories collected with the first stage enabled. For stochastic sampling, the method additionally fits a Student-t distribution to the remaining residuals and uses the estimated variance to adjust the DDIM update. Experiments evaluate output discrepancies, image quality, and computational overhead across several models and quantization settings.

Critical strengths
  1. Training the correction in two stages addresses a genuine dependency: adjusting the time input changes the states encountered later in sampling. Recollecting trajectories before training the second branch accounts for this change, and the held-out discrepancy measurements support the effectiveness of the correction.

  2. The image-quality improvements are substantial in several settings. At W4A8, FID improves from 14.80 to 11.88 on U-ViT and from 23.69 to 19.67 on ImageNet. The component ablations also distinguish the benefit of output correction from the additional effect of variance reallocation.

Critical weaknesses
  1. The experiments do not establish that the Student-t fit estimates residual variance more accurately. Table 5 shows that the Student-t fit improves FID, but not that it estimates residual variance more accurately.

  2. The covariance-contraction analysis depends on conditions that are only partly checked. In particular, a small average residual mean does not establish conditional isotropy or zero correlation with inherited noise.
    Table 5 shows that the Student-t fit improves FID, but not that it estimates residual variance more accurately.

  3. The residual variances are calibrated before variance reallocation changes the sampling trajectories and are then kept fixed. The paper does not check whether those estimates still describe the residuals encountered by the final sampler.

Questions for authors
  1. How do the Student-t and Empirical-VR variance estimates compare with residual second moments measured on held-out trajectories?

  2. How large are the residual–noise cross-covariances in practice, and do they support the covariance-contraction explanation in Appendix A.3?

  3. How much do the residual variances change after enabling t-VR? Does recalibration on the resulting trajectories materially change the results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.