QID: Quantile Inception Distance for Generative Model Evaluation

Generative model development requires repeated comparisons with a dataset of real images, often using only a small batch of generated samples at each checkpoint. We introduce Quantile Inception Distance (QID), a fast, sample-efficient metric that represents feature distributions by projected quantile displacements from a common reference. This optimal transport embedding provides coordinates for comparing generated distributions with the real dataset and with one another. We derive a finite-dimensional representation of this embedding, enabling efficient comparison of small generated batches with the full real dataset. We also derive fQID, a Gaussian approximation for faster scoring. We evaluate QID on over one million real and generated images across ImageNet, COCO and CIFAR-10. In small-batch ImageNet comparisons with Inception features, QID reduces pairwise ranking errors nearly fivefold relative to FID and roughly halves the variance of score differences between models relative to MIND. On ImageNet, QID achieves higher pairwise ranking agreement than polynomial and RBF kernel estimators with 2,000–10,000 generated images in both Inception and DINOv2 space. Speedups reach $89\times$ over full-reference kernel estimation and $234\times$ over FID.