Correcting Diffusion Hallucinations via Uncertainty-guided Trajectory Steering

Diffusion models have emerged as state-of-the-art image generative models, yet they still suffer from *hallucinations*, i.e., unrealistic artifacts or implausible structures absent from the training data distribution. Prior work attributes this issue to *mode interpolation*, where the model generates samples that lie between nearby data modes but fall outside the actual data support. Motivated by this perspective, we propose a lightweight image editing method for correcting hallucinations in diffusion-generated images. The key idea is to steer the sampling trajectory along directions where the diffusion model’s posterior mean estimator exhibits the highest uncertainty, thereby pushing the generation process away from unsupported, uncertain regions and toward nearby realistic modes, while largely preserving the original semantic content. Notably, our method requires neither backpropagation nor external supervision. Experiments across multiple datasets show that it consistently reduces hallucinations while remaining faithful to the original image, achieving a more favorable validity–fidelity trade-off than existing techniques.