- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper explores the abnormal information in the original embedding space of CLIP, apart from the traditional cosine similarity of text and image. This paper models the normal embeddings under class conditions using Gaussian distribution and uses Mahalanobis distance for anomaly scoring. Additionally, through geometric analysis based on displacement decomposition and stretch ratio, this paper explains the advantages of anisotropic metrics.
Critical strengths
- This paper explores the abnormal discrimination information contained in the pre-trained CLIP representations from the perspective of statistical geometry, providing an alternative solution to the previous methods based on prompts or feature enhancement.
- The proposed Mahalanobis scoring method does not require the update of the backbone network and achieves satisfactory results on multiple benchmarks..
Critical weaknesses
- Insufficient comparison with strong few-shot baselines. This method estimates the class conditional statistics based on normal reference samples. However, the previous baseline methods usually rely on cosine similarity, which means the supervisory information is inconsistent. Therefore, the comparison might not be fair.
- The ablation experiments show that the performance improvement mainly stems from anisotropic weighting rather than radial information.
- This paper proposes that anomalies are amplified in the low-variance direction, but the reported stretch ratios in the experiments are less than 1, which instead indicates that the model is more inclined towards the high-variance direction.
Questions for authors
- Under the same level of supervision, how does this method perform compared to the strong baseline model?
- If the CLIP features are normalized by L2 and the covariance is re-estimated to calculate the Mahalanobis distance, what kind of performance will be obtained?
- How can the "tight-direction" assumption be explained in relation to the observed phenomenon of "stretch ratios less than 1"?
- I cannot agree that the estimated covariance measure is regarded as the "intrinsic geometric structure" of CLIP. Why is it not only a statistical measure for a specific task?