- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The work proposes a detection method based on a density ratio, an approach motivated by the fact that if anomalies fall in a high density region of the reference distribution, a detector using only the reference distribution may treat them as in-distribution. The proposed detector compares a reference dataset, with in-distribution data, to a batch dataset, made of both in-distribution and anomaly data; its resolution is described by the polynomial degree of the score, being the expressive moments of the batch dataset. A closed form expression of the scores for degrees 1 and 2 is derived for a whitened Gaussian as reference distribution. The method is then tested on classical datasets (cifar10, svhn,...), where the results tend to favor the proposed "ratio" detector of degree 2 based on the mean and covariance of the batch vs in-distribution data, in particular on near out of distribution data.
Critical strengths
The proposed methods is a well constructed aggregation of the state of the art, with a clear segmentation of the different contributions: the link from the batch dataset literature (Blanchard et al...) and the polynomial readout is a good idea.
The derivations seem correct, the proposed method performs well, and its results are properly analyzed in sections 4.3-5, followed by a discussion of its limitations.
Critical weaknesses
The paper is not very well written:
- The third contribution "near-OOD signal lies in covariance directions" is oddly stated, since the term "covariance directions" is hardly used afterwards. It should be made clear that it is a result of the g2 detector and its analysis (if i am not mistaken).
- and the feature vector are introduced at the beginning but not reused in the experimental part, where they should be recalled and used.
- In the same vein: batch data are never clearly defined in the experiments. It should be clearly stated that they follow equation (1) (and if this is not the case, it should be explained).
- Section 3.3.2 on closed forms is hard to follow due to abrupt references to appendix.
- , in equation (13), is only implicitely defined and it does not directly match the in appendix C, which is not defined for .
- minor remarks on notations: -KL and (J) (for KL may be acceptable) not really defined while they appear in a central proposition; the tensor (?) over z notation in 3.3.3 not even defined in the appendix (and not in official iclr notations).
Even though the idea is appealing, the contribution itself seems limited, as it is mainly an aggregation of well-established works: the first contribution does not seem to be a real one, and one could almost argue that the second and third could be merged.
Questions for authors
Could you report standard deviations, at least for table 1?
About table 2: "the near-OOD gain of g2 therefore comes from the directional term". First, "anisotropic second moment" is not used elsewhere but i suppose it means the third term of equation (13). The scores are equal to g2: maybe more conclusions can be drawn from these results. Did you try to employ this term only as a score for more experiments ? If you have time maybe a few more results can be of use so we can compare the directional term with g2 (or not if you provide a theoretical reason for not using it).
Could you state more clearly your contribution with respect to the works you cite in section 2 ? I understand space is limited in the paper but, fortunately, we have more room here.
6 comments
Q1: Could you report standard deviations, at least for table 1?
We added standard deviations across seeds to all entries in Tables 1 and 3.
Q2: About table 2: "the near-OOD gain of g2 therefore comes from the directional term". First, "anisotropic second moment" is not used elsewhere [...] The scores are equal to g2: maybe more conclusions can be drawn from these results. Did you try to employ this term only as a score for more experiments?
- Terminology in Table 2: We updated this table (now Table 3) to name each term explicitly by its statistic: mean shift , trace , and traceless shift (, the third term of Eq. (13), which "anisotropic second moment" referred to). We also unified the terminology throughout the paper.
- Directional term alone: We extended Table 3 to all 44 configurations. The directional term alone matches on near-OOD ( vs. ). We retain the full because (i) the directional term alone is blind to isotropic scale changes (), whereas responds to any first- or second-moment shift (Eq. (12)); and (ii) at low contamination outperforms the centered directional readout (; App. D.9).
- What this equality implies: (a) The near-OOD signal is a change in the shape of the batch covariance, not in its mean or overall scale: the directional term matches after centering the covariance and after also removing the mean direction ( and vs. for ; cross-fitted, 8 near-OOD configurations of Table 1; Table 4), and the radial term adds little (Table 3). (b) This signal is spread over many directions: near-OOD needs about 8 eigen-directions of to reach 90% of its normalized AUROC, whereas far-OOD saturates at one (Fig. 4). Together, these suggest that near-OOD images lie among the ID class clusters, where is high, but overrepresent some directions there. A reference reader cannot tell along which directions the batch departs from the reference.
Thank you.
Regarding your response on "W7 & Q3", you state in the first contribution: "A two-axis view of OOD readouts."
-
"Organizing readouts by estimand and resolution explains what each estimand can and cannot reveal" + response: "vs. Semi-supervised novelty detection (Blanchard et al.): Classical work targets..." → What did you really bring to Blanchard et al.? The polynomial analysis is not from you, if I'm not mistaken?
-
"Links batch-based OOD scoring to classical restricted polynomial readouts" + response: "vs. General density-ratio estimation (uLSIF, DDE): Kernel and neural estimators..." → What is the difference with Kanamori et al.? Is it not a degree-2 polynomial readout also?
We thank the reviewer for the careful reading and constructive suggestions on presentation, notation, and experimental reporting.
We agree. Contribution 3 in Sec. 1 is now titled "A decomposition of g2 that locates its near-OOD gain" and states the finding as a result of analyzing g2. It no longer uses the term "covariance directions". It instead describes the directions along which the batch covariance departs from the reference, which correspond to the directional term of Eq. (13), the term used throughout Sec. 4.3 (Tables 3–4, Figure 4).
We updated Sec. 4.1 to state explicitly that: (1) the feature vector z=ϕ(x)∈Rd is the encoder's CLS token (d=768 for ViT-B/16, d=384 for DINOv2), and (2) each evaluation batch follows Eq. (1) by combining (1−π)n ID test samples from p with πn OOD samples from a (App. A.2).
We rewrote the opening of Sec. 3.3.2 so that reference whitening and the orthogonality of the degree-1 and degree-2 Hermite shells are explained in the main text before referring to App. B.3.
We fixed all of these notations:
We rewrote Contribution 1 in Sec. 1 to clarify what the two-axis formulation brings (linking batch OOD scoring to restricted polynomial readouts and separating batch detection from sample ranking). We keep Contributions 2 and 3 separate because Contribution 2 is methodological (the closed-form detector g2 with O(d2) streaming updates and leave-one-out scoring), whereas Contribution 3 is an empirical finding about representations enabled by decomposing g2.
We also revised Sec. 2 to contrast our work directly with each research thread: