Evaluating Generative Models via Diffusion Energy Divergence

Generative models now produce diverse outputs, including depth maps, point clouds, video, whose quality is naturally assessed with task-specific distances between individual samples, e.g. AbsRel for depth, Chamfer distance for point clouds. Turning these distances into divergences between distributions is difficult, though, as the standard tools like energy distance and distance-induced maximum mean discrepancy (MMD) require negative type, a global condition that several commonly used distances violate on real data. We introduce the Diffusion Energy Divergence (DED), which uses a task-specific distance only to connect samples to their nearest neighbors and weight the resulting graph edges. The normalized graph Laplacian is positive semidefinite for any symmetric, nonnegative weights, and its heat semigroup therefore yields a Euclidean diffusion embedding in which energy distance is a valid divergence. This embedding depends on the original distance only through the neighbor lists and edge weights. We fit the graph once on real data and place all evaluation and generated samples on it by a Nyström extension, giving every generator the same diffusion geometry. A permutation test combines the divergence with an off-support check and has exact size under the null for any such distance. On NYU depth maps, energy distance computed directly on AbsRel ranks two depth degradations in reverse order of severity (Spearman $\rho =-0.89$ for both), while DED ranks them correctly ($\rho =+0.96$ and $+1.00$). On VBench, DED detects mode collapse in 16 of 20 generators, although the collapse leaves the expected VBench scores unchanged.