acceptodds
Withdrawn from ICLR 2027

Limits of Fixed Score Scaling in Diffusion Models under Gaussian Mixtures

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion-based image reconstruction involves a perception–distortion (PD) tradeoff. Fixed score scaling adjusts this tradeoff without retraining by applying the same score multiplier at every noise level. For Gaussian sources, doubling the score in the variance-preserving probability-flow ordinary differential equation recovers the posterior mean and minimizes mean-squared error (MSE). However, this result does not guarantee posterior-mean recovery for non-Gaussian sources. For every nontrivial finite Gaussian mixture model (GMM) with positive weights and positive-definite component covariances, we prove a positive lower bound on excess MSE over all real constant multipliers. The bound extends to deterministic denoising diffusion implicit model (DDIM) sampling on sufficiently fine grids over expanding multiplier intervals. To approximate the posterior mean, we propose Simple Endpoint Correction, which combines two constant-scaling endpoints with coefficients selected on a calibration set. Experiments show lower held-out negative log-likelihood for GMMs than for single Gaussians on the evaluated image latents. Endpoint correction reduces posterior-mean error on synthetic GMMs and image-space MSE for DiffC-SD2.1, ResULIC, and ELIC+DDIM/DSG.

Primary area:
generative models
Keywords:
Score Based Diffusion Model, Gaussian Mixture Model, Probability flow ODE, Posterior Mean Estimation, Diffusion Sampler, Denoising Diffusion Probabilistic Model, Denoising Diffusion Implicit Model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pqxrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proves that for any nontrivial finite Gaussian mixture model with positive component weights and positive-definite covariance matrices, the excess mean-squared error has a strictly positive lower bound over all real constant multipliers. This lower bound also extends to deterministic denoising diffusion implicit models under the stated conditions.

To approximate the posterior mean, the paper proposes Simple Endpoint Correction, which combines two constant-scaling endpoints. Experiments show that GMMs fit image latents better than single Gaussian models. Endpoint correction also reduces posterior-mean error on synthetic GMMs and image-space MSE for DiffC-SD2.1, ResULIC, and ELIC + DDIM/DSG.

Critical strengths

The paper provides substantial theoretical analysis, and its central result is valuable. It proves that, for any nontrivial finite GMM, no fixed score multiplier can exactly recover the posterior mean, and it extends this result to DDIM.

Critical weaknesses
  1. The connection between the theory and the experiments is relatively weak. The paper mainly studies posterior-mean approximation and MSE minimization, but it does not establish that better posterior-mean approximation necessarily improves distortion at a fixed perception level. Section 5.3 uses the Wasserstein distance only for one-dimensional synthetic GMMs, whereas Section 5.4 uses LPIPS and does not compare the methods at matched LPIPS levels.
  2. The experimental results raise some questions. In Table 5, for ResULIC, the MSE and LPIPS values for the Native, Constant, and Effective constant methods are identical across all datasets.
Questions for authors
  1. Section 5.3 shows that fixed score scaling incurs excess distortion in the PD tradeoff at the level of theoretical distributions. Does SEC address this issue?
  2. Why are the values for Native, Constant, and Effective constant exactly identical for the ResULIC model?
  3. The paper claims to compare SEC with a dense grid of constant multipliers. What multiplier values were selected for the Constant method in Table 5?
  4. Why do some images in Figure 3 have borders while others do not?
Reviewer_z7RBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that constant score scaling cannot recover the posterior mean for nontrivial finite Gaussian mixtures, even with exact scores, and extends this result to DDIM under suitable grid conditions. It proposes Simple Endpoint Correction (SEC), which combines two sampling endpoints, and evaluates it on synthetic distributions and real-image compression benchmarks.

Critical strengths
  1. Understanding the inference-time controls for the perception–distortion tradeoff is an important problem.
  2. The experiments include several practical diffusion-based methods and real-image datasets.
Critical weaknesses
  1. My main concern is the motivation, or equivalently, the error metric. The error metric G(λ)G(\lambda) in Eqs. (5)–(6) and the theoretical results in Section 3 take m(y)=E[X0∣Y=y]m(y)=\mathbb E[X_0\mid Y=y] as the target output. However, under the assumed Gaussian observation model, Tweedie’s formula (Eq. 3) already expresses this posterior mean through the score function, up to an affine transformation. Hence, an exact score recovers m(y)m(y) directly, while a learned score immediately provides an estimate. So why is the diffusion sampler used to approximate a quantity that is already given by the score used in the sampling procedure?
  2. The practical significance of the theoretical results remains unclear. The result establishes a strictly positive, distribution-dependent gap without providing a quantitative characterization of its magnitude.
  3. The experimental comparisons do not sufficiently establish SEC’s advantage. SEC requires two sampling trajectories, but the baselines are not compared under equal computational budgets.
Questions for authors

See Weakness.

Reviewer_1MrMOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper analyzes the posterior mean MSE bound of the integrated endpoint based on fixed score scaling for ideal continuous-time and numerical finite-grid integration, and proposes a paradigm to improve the posterior mean recovery called the Simple Endpoint Correction. Experiments demonstrate that real image latent prior can be better described by GMM, verification of the theoretical posterior mean error, and demonstrates superior performance of the new proposal.

Critical strengths

None

Critical weaknesses

The paper needs a holistic rewriting to improve the logic consistency and fill the logical and information gaps. Firstly, because the paper is about handling image reconstruction PD tradeoff, one should clearly define and introduce the problem setup to sufficiently motivate why it is significant to study GMM, such as by bringing the definitions in Section 5.3 upfront. The reader needs to infer the intent and only until the Experiment section 5.1 to see the argument that “GMM better fits the image latent than Gaussian”. It is also unclear why posterior mean recovery, the minimum-distortion endpoint, is the best endpoint to achieve when the PD tradeoff is a multi-objective problem. Furthermore, it is not clear what is the design implication of the lower bound: the new proposal SEC looks like an independent product that is not sufficiently motivated following the theoretical analysis and is not a satisfying response to the presented difficulty of fixed score scaling. Also a minor point is that Related Work does not play the role of “Preliminaries” or “Background” and should be the place to give, for example, sharp contrast of the paper’s contribution against other possible candidate frameworks in literature.

Questions for authors

If your goal is only to recover the posterior mean, and posterior mean can be directly recovered from the Tweedie’s formula (your eq. (3)) so a one-step integration,, why is it necessary to integrate over multiple steps and why is the posterior mean error important at all?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.