Making Reconstruction FID Predictive of Diffusion Generation FID

It is well known that the reconstruction FID (rFID) of a VAE correlates poorly with the generation FID (gFID) of a latent diffusion model. We propose __interpolated FID (iFID)__, a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each dataset element, we retrieve its nearest neighbor in latent space, interpolate between their latent representations, decode the interpolated latent, and compute the FID between the decoded samples and the original dataset. We provide theoretical intuition and examples explaining why iFID correlates well with gFID by connecting to recent results on diffusion generalization. Empirically, we construct a VAE benchmark comprising 22 pre-trained VAE models and 15 VAE metrics, along with 44 diffusion models trained for class-conditional image generation on ImageNet. Using this benchmark, we show that iFID correlates strongly with gFID, achieving Pearson and Spearman correlation coefficients of approximately $0.85$. The source code is provided in supplementary materials.