acceptodds
Withdrawn from ICLR 2027

SelfVerify: Typicality-Guided Sampling with Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models are trained to predict noise under a prescribed Gaussian corruption process, suggesting that their own noise predictions may provide an internal signal for candidate selection. We introduce SelfVerify, which compares patchwise moments of predicted noise with a Gaussian reference and retains the lowest-scoring trajectory. SelfVerify requires neither a separately trained external reward or preference model nor additional denoiser evaluations, avoiding reliance on external-verifier training and preference data for selection. We interpret the resulting score as a noise-statistical typicality proxy, with preference and alignment gains treated only as downstream empirical outcomes. Controlled noise-process interventions show that predicted-noise statistics respond to selected departures from the training process, including variance changes missed by a patch-mean statistic. On SANA-base, SelfVerify improves several preference metrics under standard classifier-free guidance. Motivated by the possibility that guidance-induced off-manifold behavior may weaken this signal, we additionally evaluate SelfVerify under CFG++, an independently proposed manifold-constrained guidance method, and observe larger relative selection gains. A scored-branch control further shows that the prediction used for ranking materially affects performance. Consistent gains on GenEval2 and AudioLDM2 further suggest that the signal transfers across tasks and modalities. Overall, predicted-noise typicality provides a useful internal sampling signal without establishing formal data typicality, manifold proximity, or a causal link from typicality to preference.

Primary area:
generative models
Keywords:
Diffusion Models, Inference-Time Selection, Statistical Typicality

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HzeAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SelfVerify, an inference-time trajectory selection method for diffusion models that uses the model’s own predicted-noise statistics as an internal selection signal, avoiding the need for an external reward or preference model. The method scores intermediate candidates according to how closely their local predicted-noise statistics match a Gaussian reference and retains the most typical trajectory for continued sampling. The paper evaluates the method across multiple image-generation settings and an audio-generation task. Overall, the results suggest that predicted-noise typicality can provide a useful signal for reward-free inference-time selection.

Critical strengths
  1. Conceptually simple internal selection signal: SelfVerify uses statistics from the diffusion model's own predicted noise to rank intermediate trajectories, without introducing a separately trained reward or preference model. The idea is straightforward and provides an alternative perspective on inference-time selection using signals already available during sampling.

  2. Broad experimental coverage: The paper evaluates the proposed method across multiple diffusion backbones, sampling configurations, generation tasks, and modalities. This gives a relatively broad view of how the proposed selection signal behaves under different settings rather than restricting the evaluation to a single model or benchmark.

Critical weaknesses
  1. Weak theoretical foundation of the proposed typicality criterion: At an intermediate diffusion time, the ideal marginal distribution qtq_t is given by a Gaussian-smoothed version of the data distribution, i.e., a convolution between the scaled data distribution and Gaussian noise. Accordingly, even for an ideal noise predictor, ϵ^(xt,t)=E[ϵ∣xt,t]\hat{\epsilon}(x_t,t)=\mathbb{E}[\epsilon\mid x_t, t] does not generally follow N(0,I)\mathcal{N}(0,I). It is therefore unclear why closeness of local predicted-noise moments to a standard Gaussian should provide a meaningful notion of trajectory typicality or correlate with generation quality. Additional design choices, such as equal weighting and the patch scale, are also largely heuristic.

  2. Limited empirical improvements: The reported quantitative gains are generally small and inconsistent across models and settings, with several main-table configurations showing only marginal improvement over the vanilla baseline. Moreover, the absolute HPSv2 and ImageReward scores are very low in some settings, raising concerns that the underlying candidate pool may already contain severely degraded generations. In such regimes, small numerical increases in these metrics are difficult to interpret as meaningful improvements in generation quality. The paper also provides only a small number of qualitative examples, which are selected from successful cases and therefore provide limited additional evidence.

Questions for authors
  1. Could the authors provide a more principled justification for using N(0,I)\mathcal{N}(0,I) as the reference distribution for ϵ^\hat{\epsilon}? Since even an ideal noise predictor does not generally follow a standard Gaussian, it is unclear why proximity to N(0,I)\mathcal{N}(0,I) should indicate a better trajectory.

  2. How should the improvements in HPSv2 and ImageReward be interpreted in settings where the absolute metric values are already very low and the gains over the vanilla baseline are marginal? Could the authors provide more qualitative generation results to better demonstrate the improvements brought by the proposed method?

  3. How sensitive is the proposed score to the patch scale pp, especially for small values such as p=2p=2? In this case, each patch contains only four values, so the estimated local mean and variance can have substantial finite-sample fluctuations. Could the authors clarify whether this estimation noise affects the reliability of the trajectory ranking and provide further analysis or justification for the choice of pp?

Reviewer_A7BaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes to verify sample quality by matching the noise predicted during the denoising trajectory with a gaussian distribution. As training is done with gaussian noise, anything other than gaussian during inference would mean out of distribution inference. The motivation behind this work is to perform reward-free pruning of model outputs for higher quality without black box metrics and fine-tuning.

Critical strengths

S1. The paper proposes training free improvement in sample quality by drawing random samples and only allowing high quality ones to finish. This saves compute and time during sampling and can improve user experience in generative AI by removing obvious failures during generation.

S2. The central motivation of the work is supported with a toy problem and a sanity test, and with experiments on SANA, SDXL, etc.

Critical weaknesses

W1. The authors talk about CFG++ gains in the introduction and abstract, but even self verify with SANA CFG++ does not perform better than vanilla SANA with CFG. It seems like gains in CFG++ comes from some candidates being completely broken and Self-Verify steering against those, rather than Self-Verify identifying better images. Thus while the paper is written for Self-Verify to improve image quality, it might be better suited to reducing failure rate as opposed to judging better samples. The authors need to analyse this better to identify the proper use of their methodology. In experiments with standard CFG, the gains seem to be modest.

W2. The authors do not cite and compare with other methods with perform training-free and reward-free trajectory analysis like "Not All Noises Are Created Equally" and "ANSE"

W3: Paper argues that the methodology works better from patch variance but p=1 also improves upon the baseline. So a global magnitude test would work as well. The paper discusses this in appendix but the main text never talks about this while this is an important ablation over the methodology.

Questions for authors

See Weaknesses

Reviewer_uZNnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SELFVERIFY, which selects diffusion trajectories using patch-wise statistics of the model’s predicted noise. It combines three Gaussian-reference discrepancies and retains the lowest-scoring candidate at an intermediate sampling step. Experiments show improvements over uniform candidate selection, primarily on SANA-base, with weaker or mixed results on other backbones.

Critical strengths

1.The method is simple and requires neither an external evaluator nor additional denoiser evaluations for scoring. 2.Experiments show that internal noise predictions contain useful selection information, particularly on SANA across multiple evaluation metrics and tasks.

Critical weaknesses

1.The related-work discussion lacks a focused comparison with the closest trajectory-selection approaches. Although avoiding an external evaluator is a clear distinction, the specific contribution relative to existing selection methods remains insufficiently established. 2. The connection between the statistical motivation and the final selection rule remains incomplete. For example, the equal-weight combination and guidance-dependent choice of scoring branch remain partly heuristic. 3. The empirical benefits are most evident on SANA, with weaker or inconsistent gains on other backbones. Comparisons also use different NFE budgets without measured end-to-end runtime. This limits the evidence for general applicability and practical computational advantages.

Questions for authors
  1. Why should selecting the trajectory whose predicted-noise statistics are closest to a standard Gaussian lead to better final images? Could the authors provide some theoretical insights for this selection rule? 2.Could the authors compare SELFVERIFY with more representative selection baselines and additional backbones? This would help clarify its comparative advantages and generality.
  2. How does SELFVERIFY compare with the baselines under the same end-to-end time budget, including decoding and external-evaluator costs? The current comparisons at different NFE budgets do not establish this practical trade-off.
  3. Table S6 shows that p=1 achieves the largest Soft-TIFA gains across all reported selection steps for SANA/CFG++, and Table S5 shows similar results. Given that these controls reduce to noise-energy ranking, could the authors justify the need for patch grouping and demonstrate its added benefit?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.