A Domain-Aligned Encoder Evaluation Framework for FID-like Quality Metrics on Synthetic 3D Medical Imaging

Fréchet Inception Distance (FID) is the de facto standard for evaluating synthetic image data. For 2D natural images, FID's validity has been shown to depend strongly on the features deemed salient by the used image encoder. For 3D medical imaging, FID is either omitted due to 2D-3D incompatibility or is implemented with the Med3D image encoder as feature extractor, although this substitution has not been scrutinized. We present a framework that evaluates Med3D and other encoders along multiple criteria with controlled signals, including domain-relevant distortions and synthesis-derived fidelity and empirical-coverage signals, which ranks encoders without requiring expert annotation. Additionally, we conduct an exploratory analysis of alignment with expert visual perception, covering overall, anatomical and vessel quality as well as anatomical and pathological variation. In a use-case on Time-of-Flight Magnetic Resonance Angiography with six candidate encoders, Med3D showed no detectable response to vessel masking or to controlled changes in empirical coverage. Self-supervised ResEncL encoders pre-trained on OpenMind responded to most controlled signals, with ResEncL-OM-S3D stronger than Med3D in five of six and ranking first overall. Expert-rating associations were weak for all encoders and did not separate them, reinforcing the need for systematic encoder evaluation. The framework facilitates reliable evaluation of synthetic medical imaging and is designed to transfer to other medical imaging modalities.