- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes SelfVerify, an inference-time trajectory selection method for diffusion models that uses the model’s own predicted-noise statistics as an internal selection signal, avoiding the need for an external reward or preference model. The method scores intermediate candidates according to how closely their local predicted-noise statistics match a Gaussian reference and retains the most typical trajectory for continued sampling. The paper evaluates the method across multiple image-generation settings and an audio-generation task. Overall, the results suggest that predicted-noise typicality can provide a useful signal for reward-free inference-time selection.
Critical strengths
-
Conceptually simple internal selection signal: SelfVerify uses statistics from the diffusion model's own predicted noise to rank intermediate trajectories, without introducing a separately trained reward or preference model. The idea is straightforward and provides an alternative perspective on inference-time selection using signals already available during sampling.
-
Broad experimental coverage: The paper evaluates the proposed method across multiple diffusion backbones, sampling configurations, generation tasks, and modalities. This gives a relatively broad view of how the proposed selection signal behaves under different settings rather than restricting the evaluation to a single model or benchmark.
Critical weaknesses
-
Weak theoretical foundation of the proposed typicality criterion: At an intermediate diffusion time, the ideal marginal distribution is given by a Gaussian-smoothed version of the data distribution, i.e., a convolution between the scaled data distribution and Gaussian noise. Accordingly, even for an ideal noise predictor, does not generally follow . It is therefore unclear why closeness of local predicted-noise moments to a standard Gaussian should provide a meaningful notion of trajectory typicality or correlate with generation quality. Additional design choices, such as equal weighting and the patch scale, are also largely heuristic.
-
Limited empirical improvements: The reported quantitative gains are generally small and inconsistent across models and settings, with several main-table configurations showing only marginal improvement over the vanilla baseline. Moreover, the absolute HPSv2 and ImageReward scores are very low in some settings, raising concerns that the underlying candidate pool may already contain severely degraded generations. In such regimes, small numerical increases in these metrics are difficult to interpret as meaningful improvements in generation quality. The paper also provides only a small number of qualitative examples, which are selected from successful cases and therefore provide limited additional evidence.
Questions for authors
-
Could the authors provide a more principled justification for using as the reference distribution for ? Since even an ideal noise predictor does not generally follow a standard Gaussian, it is unclear why proximity to should indicate a better trajectory.
-
How should the improvements in HPSv2 and ImageReward be interpreted in settings where the absolute metric values are already very low and the gains over the vanilla baseline are marginal? Could the authors provide more qualitative generation results to better demonstrate the improvements brought by the proposed method?
-
How sensitive is the proposed score to the patch scale , especially for small values such as ? In this case, each patch contains only four values, so the estimated local mean and variance can have substantial finite-sample fluctuations. Could the authors clarify whether this estimation noise affects the reliability of the trajectory ranking and provide further analysis or justification for the choice of ?