- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a training-free IQA metric based on diffusion inversion. The central idea is that high-quality images invert to latents that more closely resemble Gaussian noise. The proposed DIGS demonstrates a meaningful correlation with human perceptual judgments, although CLIP-IQA consistently outperforms the two proposed variants.
Critical strengths
-
New empirical insight: The Gaussianity of inverted diffusion latents correlates with image quality and potentially reflects compatibility between observations and learned generative priors.
-
The authors measure deviations in the skewness and kurtosis of standardized inversion latents from their Gaussian reference values, and consider other statistical aspects to formulate a training-free IQA metric.
Critical weaknesses
I appreciate the authors’ investigation and insight of the statistical properties of diffusion inversion and associate it to the image quality. However, as an IQA metric, the proposed DIGS (and DIGS*) has several limitations:
-
Inferior performance. Other methods, such as CLIP-IQA, consistently outperform both proposed variants on SROCC and PLCC across all three datasets. CLIP-IQA also uses a pretrained prior and does not require image-quality labels or additional training. This questions the effectiveness of DIGS, particularly given the computational overhead associated with diffusion inversion.
-
Sensitive to diffusion backbones and inversion setting. The authors report in Table 4 that for different backbones and operators, the SROCC and PLCC can differ a lot. This dependence raises concerns about the robustness and generalizability.
Questions for authors
Why is the performance of DIGS related to the inversion operators? The authors say that the choice of inversion trajectory affects the quality-related statistics extracted from terminal latents. How is this correlation explained?
In Figure 6, how are the distributional divergences computed for the KonIQ-10k dataset? This should be clarified.