- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The present work tackles the problem of anomaly detection on tabular data. This work builds on existing work in image anomaly detection that leverage pretrained representations and propose a method that leverages pretrained model (e.g. TabICL, TabPFN) to identify anomalies. Their method involves two settings carved for uncontaminated and contaminated settings: a frozen encoder setting and a fine-tuned setting. In short, a sample's anomaly score is obtained as follows:
- extract a sample's embedding by feeding the sample accompanied by a subset of (or the entire dataset) to the TabPFN model.
- apply the vanilla k-NN based anomaly detection algorithm that identifies samples as being anomalies when the average distance to their k nearest neighbours is too high. The authors evaluate their method on a widely used benchmark, and demonstrate strong performance.
Critical strengths
S1: The paper is well written and easy to follow.
S2: The experiments are extensive and well motivated. Ablations in particular are especially sound and relevant to demonstrate the relevance of the method.
S3: Appendix and additional experiments are large and further support the relevant of the present method.
Critical weaknesses
W1: While the concept of novelty is of course subjective, the present work's novelty is somewhat limited. The present algorithm consists in using pretrained encoders as feature extractors for anomaly detection on tabular data. While the authors correctly mention that this has never been tried (to our knowledge), the idea of using deep features as sample's representation is far from being novel.
W2: Some strong baselines seem to have been left out of the benchmarks, e.g. ICL [1], NPT-AD [2], DRL [3].
[1] Tom Shenkar and Lior Wolf. Anomaly detection for tabular data with internal contrastive learning. In International Conference on Learning Representations (ICLR), 2022
[2] Hugo Thimonier, Fabrice Popineau, Arpad Rimmel, and Bich-Lieˆn Doan. Beyond individual input for deep anomaly detection on tabular data. In International Conference on Machine Learning (ICML), pp. 48097–48123, 2024.
[3] Hangting Ye, He Zhao, Wei Fan, Mingyuan Zhou, Dandan Guo, Yi Chang. DRL: Decomposed Representation Learning for Tabular Anomaly Detection. ICLR 2025.
Questions for authors
Q1: It would be interesting to apply the soft-cleaning and feature subset selection to the vanilla knn approach in the original feature space rather than in the embedding space. This would allow identifying whether the performance originate from the representation space obtained from the pretrained model, or the inclusion. This seems like a simple experiment to implement while also further showcasing the relevant of the embedding representation.
Q2: The authors discuss in the appendix how iterating on different subsets of features seems to increase performance while iterating on embedding while changing the data subset seems to make no difference different. (Appendix E) How was this experimented ? Did authors vary the subset size? On small datasets, we expect this would be especially effective for contaminated sets.
Q3: A thorough discussion on the difference in performance of the method on different datasets’ characteristics would be particularly if interest. For instance, one could expect datasets that have very few features would hardly benefit from the feature subset selection, it could even be detrimental to the performance.
Q4: While AUROC is relevant as a metric as it benefits from being threshold independent, previous work have also relied on the F1 score (see [4] for how to select the threshold for F1 Score computation). Could the authors also include this metric?
Overall the paper is well written, the experiments are thorough (but could improved see W2), but the core of the paper appears to us as incremental for a conference like ICLR. We lean toward reject but we are open to discussion. We believe that some of the small additional experiment (e.g. Q1) might strengthen the paper.
[4] Liron Bergmand and Yedid Hoshen, Classification-Based Anomaly Detection for General Data, ICLR 2020.
3 comments
Response to Rebuttal
We thank the authors for carefully addressing the different points raised in our review.
In particular, the addition of the vanilla kNN and its soft-cleaned + subset variant is particularly of interest. The other experiments also strengthen the proposed approach. For this reason I am happy to increase my score.
Thank you
Thank you for acknowledging our rebuttal and for taking the time to review our work!
Response to Reviewer 2Wz7
Thank you for the careful review and suggestions, and for highlighting the paper’s clarity, ablations, and appendix experiments.
W1
We agree that kNN on pretrained features is an already established method. Yet, even ZEN, our simpler method that uses frozen TabPFN embeddings with feature subsetting, augmented contexts, and pooled soft cleaning, raises mean AUROC from 74.1 to 79.7 under contamination, a gain of 5.6 points over plain kNN on frozen embeddings. More importantly, FOCUS is the first method to fine tune a general-purpose PFN for unsupervised anomaly detection using only an unlabeled, potentially contaminated reference set. Its contribution is a trust weighted adaptation procedure that uses the frozen model’s embeddings to estimate which reference samples to trust (Sec. 2.3, App. D.1).
W2
Thank you for suggesting ICL, DRL, and NPT-AD. FOCUS has higher mean AUROC than all three in both regimes. ICL and DRL completed on all 47 datasets.
NPT-AD completed on 41 datasets within our compute budget. The following table compares both methods on those same 41 datasets.
We will add these comparisons and cite DRL alongside ICL and NPT-AD in Sec. 5.
Q1
Thank you for suggesting this test. We applied ZEN’s feature subsetting and pooled soft cleaning to raw kNN, with soft cleaning applied only on the relevant contaminated setting.
These results suggest that the advantage of our method comes from the synergy between our novelty and PFN-based approach. The same steps give raw kNN only a small gain under contamination, and feature subsetting slightly hurts it on clean data. We will add this control to the appendix.
Q2
Thank you for raising this point. Appendix E documents alternatives we tried for completeness. In this preliminary test, we averaged anomaly scores over random subsets of reference samples and saw no improvement. The subset size was varied but only as an exploration. We do not claim these experiments as a contribution, but we believe that clearly documenting negative results is useful to the community. Therefore we describe them only sporadically.
Q3
Thank you for asking. We grouped the 47 datasets by feature count into three groups: 9 or fewer, 10–30, and 31 or more. Under contamination, FOCUS’s largest advantage over Isolation Forest, the strongest baseline overall in this regime, was on the 16 datasets with at least 31 features. FOCUS scored 6.16 AUROC points higher on average in this group.
On clean data, feature subsetting slightly reduced FOCUS’s mean AUROC in the group with 9 or fewer features (−0.23 points) and improved it in the two groups with more features (+0.68 and +0.47 points, respectively). Under contamination, subsetting helped even in the group with the fewest features (+1.46 points).
We also examined dataset size. On the 35 datasets with more than 1,000 samples, FOCUS achieved higher mean AUROC than raw kNN in both regimes, scoring 86.02 vs. 84.78 on clean data and 79.85 vs. 71.03 under contamination. We will add these results to the Discussion and appendix.
Q4
Following Bergman and Hoshen (2020), we rank test samples by anomaly score. The table reports mean F1 × 100 over 47 datasets (higher is better).
Also when evaluating with a threshold, FOCUS substantially outperforms all compared baselines.
We will add F1 in App. B.5 and reference them from Sec. 3.1.