acceptodds
Under review as a conference paper at ICLR 2027

Revisiting Deterministic Diffusion through Reverse Transition Kernels

Abstract

Deterministic diffusion samplers replace stochastic reverse dynamics with an iterative reconstruction map, but the conditions under which this iteration is stable remain poorly understood. We study Directly Denoising Diffusion Models (DDDMs), which refine an estimate of the clean sample by a fixed-point iteration at each noise level. Through a reverse transition kernel formulation, each reverse-time step defines a residual energy, and Tweedie's identity yields a certificate of local strong convexity for this energy—a sufficient condition for stable refinement. We audit this certificate at scale on trained models and report three findings. First, the derivative that governs convergence is taken with respect to the sampler's own running estimate, whereas Tweedie's identity concerns derivatives with respect to the noisy observation; on trained networks these are measurably different operators, and only the latter exhibits the symmetry Tweedie predicts. Second, the curvature term in the certificate can be computed exactly rather than bounded, tightening it by a median factor of –. Third, across configurations spanning variance-preserving models under linear and cosine schedules and variance-exploding models, the energy is strongly convex more than twice as often as even the exact certificate detects (% versus %, with the worst-case bound certifying almost none); we trace the remaining conservatism to a single geometric assumption—that the certificate's two terms act along a common worst-case direction, where in measurement they are nearly orthogonal. A one-dimensional system with closed-form posterior statistics reproduces the predicted contraction rate to within % and shows that the certificate, not the sampler, loses resolution at low noise: where certification fails, the iteration itself still converges.

Primary area:
generative models
Keywords:
Deterministic Diffusion, Reverse Transition Kernels, Diffusion Models, Reconstruction Stability, Generative Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dPBNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper develops a theoretical framework for analyzing the convergence and stability of iterative refinement in DDDMs using residual energy, Jacobian analysis, and strong-convexity conditions. Its main contribution is identifying why existing stability certificates are overly conservative and proposing tighter curvature-based bounds. Experiments across image and molecular diffusion models validate the theory, showing rapid refinement convergence and substantial gaps between theoretical certificates and actual Hessian geometry. The results also reveal how training objectives and diffusion settings influence refinement stability.

Critical strengths

The paper provides a principled analysis of DDDM refinement stability through fixed-point contraction and strong-convexity conditions. This strengthens the theoretical understanding of deterministic diffusion sampling and offers useful tools for analyzing convergence behavior.

Critical weaknesses
  1. While the paper provides theoretical convergence guarantees and tighter stability certificates, it does not establish a clear connection between these properties and improvements in generation quality, sampling efficiency, or diversity. This limits the practical significance of the proposed framework.
  2. The proposed certificates remain overly conservative, failing to detect strong convexity in many cases where it is empirically observed. Moreover, the guarantees do not extend to the evaluated molecular diffusion models, raising concerns about the framework's applicability beyond specific image diffusion settings.
Questions for authors
  1. Do the proposed stability certificates correlate with reconstruction quality, sampling efficiency, or generation diversity? Evidence of such a relationship would help establish the practical relevance of the theoretical results.

  2. Why do the proposed certificates fail for molecular diffusion models? Is this primarily due to overly conservative bounds or fundamentally different refinement dynamics? Clarifying this distinction would help assess the applicability of the theoretical framework.

  3. In the experiments, why do you use 3999 sampling steps, which is not common in diffusion sampling?

7 comments

For molecular diffusion on GEOM-LongRange (molecules with more than 100 atoms), the sufficient condition in Proposition 2 is not satisfied across the evaluated diffusion states: the clamped certificate numerator Δ=max⁡(0,1−L)2−RBsq\Delta = \max(0,1-L)^2 - RB_{\mathrm{sq}} is non-positive, resulting in μbound≤0\mu_{\mathrm{bound}} \leq 0. This means that our bound cannot certify local strong convexity for the evaluated molecular models, although it does not imply that their reconstruction energies are necessarily non-convex. To investigate the role of the curvature estimate, we compare the original B_sq-based estimate with the direction-aware quantity R * ||H_u||_op. Relative to MSE, Pseudo-LPIPS reduces the estimate-slot Lipschitz constant by 78% and 39%, and the residual-weighted directional curvature by 87% and 40%, under the VP-cosine and VE schedules, respectively. Notably, under VP-cosine, the B_sq-based certificate remains unchanged despite the substantial reduction in directional curvature, whereas the direction-aware certificate captures this change.

The generic curvature bound may be too conservative to capture changes in the local geometry of molecular reconstruction. However, since the sufficient condition remains unsatisfied, we cannot determine whether its failure is entirely due to bound conservativeness or also reflects the local geometry of the molecular models. Thus, a non-positive certificate indicates that strong convexity cannot be guaranteed, not that the reconstruction energy is necessarily non-convex.

AuthorsOpenReview ↗

Correlation with molecular reconstruction quality

To further investigate whether the stability measures are related to the physical properties of generated molecular structures, we examined the correlations between the stability-related quantities and molecular geometry. We found that the minimum Hessian eigenvalue, λmin⁡(∇2g)\lambda_{\min}(\nabla^2 g), was positively correlated with mean bond length (ρ=1.0\rho=1.0) and mean and median bond angles (ρ=0.9\rho=0.9). The gradient-normalized minimum Hessian eigenvalue showed positive correlations with both bond-length and bond-angle measurements (ρ=0.9\rho=0.9--1.01.0). We also observed some negative relationships. For example, the Lipschitz constant of the reconstruction map, LL, was negatively correlated with maximum bond length (ρ=−0.9\rho=-0.9), while the Hessian symmetry error was negatively correlated with maximum bond angle (ρ=−0.9\rho=-0.9). These observations suggest that some stability-related quantities are associated with variations in molecular reconstruction quality.

Table 1. Strongest positive correlations between stability measures and molecular geometry.

Stability measureGeometry metricSpearman correlation (ρ\rho)
Minimum Hessian eigenvalue (λmin⁡(∇2E)\lambda_{\min}(\nabla^2 E))Mean bond length+1.0
Gradient-normalized minimum Hessian eigenvalueMedian bond length+1.0
Gradient-normalized minimum Hessian eigenvalueMedian bond angle+1.0
Gradient-normalized minimum Hessian eigenvalueMean bond angle+1.0
Minimum Hessian eigenvalue (λmin⁡(∇2E)\lambda_{\min}(\nabla^2 E))Median bond angle+0.9
Minimum Hessian eigenvalue (λmin⁡(∇2E)\lambda_{\min}(\nabla^2 E))Mean bond angle+0.9
Gradient-normalized minimum Hessian eigenvalueMean bond length+0.9
Minimum Hessian eigenvalue (λmin⁡(∇2E)\lambda_{\min}(\nabla^2 E))Median bond length+0.9

Table 2. Additional observed correlations.

Stability measureGeometry metricSpearman correlation (ρ\rho)
Contraction margin (1−L1-L)^2Mean bond angle+0.6
Hessian symmetry errorMean bond length-0.7
Minimum Hessian eigenvalue (λmin⁡(∇2E)\lambda_{\min}(\nabla^2 E))Maximum bond angle+0.6
Curvature-related residualMaximum bond length-0.7
Contraction margin (1−L1-L)^2Median bond length+0.6
Reconstruction residual (RR)Maximum bond length+0.7
Reconstruction residual (RR)Minimum non-bonded distance+0.7
Lipschitz constant of the reconstruction map (LL)Minimum non-bonded distance-0.6
Reconstruction residual (RR)Mean bond length-0.6
Curvature measureMaximum bond length-0.7
Lipschitz constant of the reconstruction map (LL)Median bond length+0.6

Table 3. Molecular geometry measurements for certified molecules (n=3n=3).

Geometry metricCertified (n=3n=3)
Mean bond length0.029734
Median bond length0.022467
Maximum bond length0.110362
Mean bond angle3.543294
Median bond angle2.960807
Maximum bond angle15.940434
Number of clashes0
Fraction of clashes0
Minimum non-bonded distance1.741740

The positive correlations indicate that molecules with higher values of certain stability measures also tend to exhibit larger values of the corresponding bond-length and bond-angle measurements. For the three certified molecules, the mean bond-length deviation was 0.0297 Å, and the mean bond-angle deviation was 3.54°, with no detected non-bonded clashes. These measurements provide a descriptive characterization of the reconstructed geometries.

However, given the small number of certified molecules, we do not interpret the observed correlations between stability measures and molecular geometry as statistically meaningful. In particular, these results are insufficient to establish a general relationship between the proposed stability conditions and physical reconstruction quality. In particular, these observations are insufficient to establish a general relationship between the proposed stability conditions and physical reconstruction quality. A larger evaluation would be needed to investigate such relationships reliably.

AuthorsOpenReview ↗

Molecule Refinement Dynamics

We traced the molecular refinement instability to the way the refinement head is implemented. For each node, the network predicts a single scalar correction, which the sampler broadcasts across all 74 node features. Thus, the refinement map can be written as T(x^)=x−F(x^)174T(\hat{x}) = x - F(\hat{x})\mathbf{1_74}, where FF is the scalar-valued refinement head and xx is fixed during refinement. Differentiating this map gives JT=−174(∇x^F)⊤J_T = -\mathbf{1_74}(\nabla_{\hat{x}}F)^\top. Since this is a rank-one outer product, its spectral norm is exactly ∥JT∥2=74∥∇x^F∥2\lVert J_T\rVert_2 = \sqrt{74}\lVert\nabla_{\hat{x}}F\rVert_2. Consequently, contraction requires ∥∇x^F∥2<1/74≈0.116\lVert\nabla_{\hat{x}}F\rVert_2 < 1/\sqrt{74} \approx 0.116. In our experiments, the measured gradient norms range from 0.145 to 7.10, exceeding this threshold in all 160 evaluated configurations.

We also evaluated a model with a 74-dimensional refinement head to check whether predicting separate corrections for each feature would improve stability. This removes the 74\sqrt{74} amplification introduced by scalar broadcasting: the ratio ∥Jself∥2/∥∂F/∂x^∥2\|J_{\mathrm{self}}\|_2/\|\partial F/\partial\hat{x}\|_2 is 1.0 across all 160 configurations, compared with approximately 8.6 for the scalar head. However, the refinement remains unstable: the measured Jacobian norms range from 1.01 to 10.74, and none of the evaluated states satisfies the contraction condition. Since the two models also differ in their encoders, training objectives, and budgets, this is not a controlled ablation. Nevertheless, it shows that removing the scalar-broadcasting amplification alone is not sufficient to ensure stable refinement.

We identified an additional feedback pathway in the molecular sampler: the running estimate enters the network both as the refinement input and through the node features, which are overwritten with the broadcast estimate after each refinement step. Consequently, the Jacobian governing the refinement map in Proposition 2 captures only the first pathway, while the implemented sampler also propagates the estimate through its conditioning inputs.

We empirically isolated these two pathways across five model checkpoints. At epoch 20, for example, the estimate-input Jacobian norm,∥∂F∂x^∥2\left\|\frac{\partial F}{\partial \hat{x}}\right\|_2, is numerically zero, and refinement converges in two steps when the node features are held fixed. However, the full sampler still diverges. This demonstrates that contraction with respect to the explicit refinement input alone is insufficient to guarantee stability when the estimate also re-enters through the conditioning features.

We therefore clarify this as a scope condition of Proposition 2 rather than a counterexample to the proposition: its contraction guarantee applies to the refinement map under the stated input dependence, but does not account for the additional conditioning feedback introduced by this molecular sampler. Unlike the molecular implementation, the image-model samplers considered in our experiments do not introduce this additional feedback pathway.

Reviewer_48m4OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces an explaination of conditions of stable refinement on top of DDDM's, which are refined using fixed point iteration. The authors define a "proxy energy" which they argue to be strongly convex and a better estimate than the true energy of the score function. They characterize the stability of the proxy energy and demonstrate its behavior on image and molecular generation tasks.

Critical strengths

The authors explore DDDM's and highlight the stability of the fixed point iteration problem using a proxy energy. The convergence using this proxy energy is demonstrated for image and molecular generation tasks.

Critical weaknesses

My main concern with this manuscript is that the proxy representation of the energy is not a strong representation of the fixed point iteration of DDDM. The experimental results reveal mixed signals, which leads me to recommend rejection. Below, I detail the reasons for my specific stance.

  1. Missing Core Insight: The paper is written significantly by AI, which affects its readability throughout. A core insight or takeaway which can benefit the community is completely absent from this work.

  2. Limited Applicability: The paper demonstrates the strong convexity of "g" in Proposition 1. However, for their results to hold broadly, it depends on the gradient of the learnt score which is completely ignored. Additionally, Theorem 2, which is related to the learnt denoiser is shown as not to hold in molecular generation results (Section 6) tasks. The authors provide no explainations as to why this is the case.

  3. No Discussion on Convergence Speed: The relation of strong convexity to the number of refinement steps required for convergence is not discussed. Demonstrating a gurantee and sufficient condition for the number of iteration steps would be crucial for the work to have merit.

Questions for authors
  1. Could the authors provide a gurantee on how Proposition 2 affects the number of steps required for convergence of DDDM? This is specified as Goal 4 but completely missing from the manuscript.

  2. I'm unclear what the experiments are supposed to demonstrate. Table 1 shows the convergence on CIFAR in two iterations. However, the result on GEOM-Long-Range shows the opposite result, i.e the residual increases in the second iteration. Can the authors explain why this is happening?

3 comments
AuthorsOpenReview ↗

score network derivatives

We thank the reviewer for pointing this out. Our curvature analysis account for the derivatives of the learned score network through the reconstruction map Fθ,kηF_{\theta,k\eta}, which is constructed using the learned score. In Proposition 2, we derive the Hessian of the reconstruction energy as

∇2g(z)=1σ2[(I+JF)⊤(I+JF)+∑iriHFi].\nabla^2 g(z)=\frac{1}{\sigma^2}\left[(I+J_F)^\top(I+J_F)+\sum_i r_i H_{F_i}\right].

Here, JFJ_F and HFiH_{F_i} are the Jacobian and output-wise Hessians of the reconstruction map. Hence, the curvature depends on both the first- and second-order derivatives of the learned map.

We also give the following lower bound:

\geq \frac{\max(0,1-L)^2-RB_{\mathrm{sq}}}{\sigma^2},$$ This gives a **sufficient** condition for strong convexity and shows how the network derivatives and reconstruction residual affect it. However, we do not assume that every learned network satisfies this condition.
AuthorsOpenReview ↗

Molecular Refinement Instability and Core Insight

We thank the reviewer for raising this important distinction. Our additional analysis suggests that the failure of the proposed convergence certificates for the molecular diffusion model is not primarily attributable to the conservativeness of the bounds, but rather to fundamentally different refinement dynamics arising from the model architecture. We traced the molecular refinement instability to the way the refinement head is implemented. For each node, the network predicts a single scalar correction, which the sampler broadcasts across all 74 node features. Thus, the refinement map can be written as T(x^)=x−F(x^)174T(\hat{x}) = x - F(\hat{x})\mathbf{1_74}, where FF is the scalar-valued refinement head and xx is fixed during refinement. Differentiating this map gives JT=−174(∇x^F)⊤J_T = -\mathbf{1_74}(\nabla_{\hat{x}}F)^\top. Since this is a rank-one outer product, its spectral norm is exactly ∥JT∥2=74∥∇x^F∥2\lVert J_T\rVert_2 = \sqrt{74}\lVert\nabla_{\hat{x}}F\rVert_2. Consequently, contraction requires ∥∇x^F∥2<1/74≈0.116\lVert\nabla_{\hat{x}}F\rVert_2 < 1/\sqrt{74} \approx 0.116. In our experiments, the measured gradient norms range from 0.145 to 7.10, exceeding this threshold in all 160 evaluated configurations.

We also evaluated a model with a 74-dimensional refinement head to check whether predicting separate corrections for each feature would improve stability. This removes the 74\sqrt{74} amplification introduced by scalar broadcasting: the ratio ∥Jself∥2/∥∂F/∂x^∥2\|J_{\mathrm{self}}\|_2/\|\partial F/\partial\hat{x}\|_2 is 1.0 across all 160 configurations, compared with approximately 8.6 for the scalar head. However, the refinement remains unstable: the measured Jacobian norms range from 1.01 to 10.74, and none of the evaluated states satisfies the contraction condition. Since the two models also differ in their encoders, training objectives, and budgets, this is not a controlled ablation. Nevertheless, it shows that removing the scalar-broadcasting amplification alone is not sufficient to ensure stable refinement.

Core Insight. We thank the reviewer for this comment. We agree that the main insight of the paper needs to be stated more clearly. Our main observation is that the apparent stability of a deterministic diffusion update does not necessarily mean that the underlying reconstruction map is stable. In particular, the sampling-step factor can make the update appear contractive even when the learned reconstruction map itself is expansive. Similarly, good denoising performance does not necessarily imply that the reconstruction energy has positive curvature. We see this in our Stable Diffusion experiments as well. The schedule-scaled Euler map appears contractive, but the direct clean-latent prediction map is expansive and has negative minimum Hessian eigenvalues across all our evaluations. This is precisely the distinction we want our framework to capture: whether stability comes from the learned reconstruction map itself or simply from the sampling discretization. We will make this point clearer in the revised paper.

I thank the authors for their response. I have raised by score by one point.

Reviewer_STWuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper revisits DDDM from RTK perspective, and studies the convergence condition and curvature for the solver of DDDM. The paper then derives a certificate, and performs an audit on the proposed certificate with empirical experiments.

Critical strengths
  1. The paper proposes a certificate that can be used to gauge the stability of DDDM sampling.
  2. A transparent layout of when the certificate is most useful.
  3. Useful takeaways/lessons, like the Tweedie formula not being predictive if used on noisy output of the denoiser. No guarantee from the theorem that is known, but the empirical evidence is still appreciated.
Critical weaknesses
  1. The paper generally has a presentation problem. A lot of framing and theorems that largely don't matter. For example the Tweedie use case and the strong convexity of the energy objective. To be transparent, the paper feels more like a research diary to me than a well-structured research paper.
  2. The usefulness of the certificate is questionable. Convergence of Picca's iteration only depends on L < 1, and other part of the certificate seems to include things that do not matter for DDDM sampling.
  3. The experiments will need further expansion. The CIFAR-10 is far from an empirical scale nowadays, and analysis on latent code of things like Stable Diffusion should be performed additionally. The molecular experiments also lack sufficient details about themselves, and it's hard to gauge how useful any information there actually is.
  4. The contribution is unclear in general. Summing from the previous points.
Questions for authors
  1. Can the certificate be used if the Newton-style method or gradient descent is used on the energy objective, instead of the Picca iteration used by DDDM? Will it be more predictive?
  2. If a certificate is employed on sampling of a real-world model, what should the employer take away from the statistics?
1 comment

I appreciate the author's effort into the response. I'll keep my original assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Model Architecture for Molecule Experiments

We represent each molecule as a 3D graph, with atoms as nodes and bonds as edges. Each atom is described by a 74-dimensional feature vector, while each edge has a 4-dimensional feature vector. The model embeds the node and edge features and incorporates timestep and molecular-context embeddings to condition the denoising process. The backbone consists of stacked equivariant graph convolutional layers (EGNN) [1], which update node representations and 3D coordinates while preserving rotational and translational equivariance. To capture interactions beyond local molecular neighborhoods, we use a Graph GPS-style architecture [2] that combines EGNN-based local message passing with a global sequence-mixing module. Finally, node representations are aggregated using sum pooling and passed through an MLP output head. The resulting graph-level representation has dimension 74. The updated coordinates are returned by the equivariant graph layers. We use EquivStableLapPE [3] to encode graph structure from precomputed Laplacian eigenvectors.

[1] Sartoras et al. , https://arxiv.org/abs/2102.09844

[2] Rampášek et al., https://arxiv.org/abs/2205.12454

[3] Wang et al., https://arxiv.org/abs/2203.00199

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.