- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work proposes a new Anomaly Detection method for tabular data. The proposed approach combines two main objectives:
- a contrastive objective,
- a mask reconstruction objective.
The proposed approach involves generating a dual representation from a single sample, first local features representations that assign a token representation to each feature of the sample, and a global representation of the feature obtained from a CLS token added to the sequence. Mask reconstruction is performed in two ways:
- masked features are reconstructed from the global feature representation.
- masked features are reconstructed from their corresponding feature representation. The contrastive objective consists in minimizing average distance between the local tokens and the global token for a sample, while maximizing the same distance for negative pairs.
The proposed method is tested in 25 datasets from ADBench a widely used benchmark for AD on tabular data and shows strong performance.
Critical strengths
Q1: Paper is well organized and well written.
Q2: Method is clearly explained.
Q3: Ablations are thorough and motivate well each component of the method.
Q4: Experiments are ran several times to provide robust metrics.
Critical weaknesses
W1: Experimental setup differs from what is usually used in the literature.
W2: Apart from performance optimization, the paper lacks intuitions/analysis/explanation as to why combining the objectives is relevant.
W3: Without any clear rational only a subset of the benchmark was used to evaluate the method.
W4: A wide range of methods tested in the benchmark's paper are not included in the paper.
Questions for authors
Q1: Line 211 authors mention how they use per-feature reconstruction heads for the global row reconstruction. This implies a linear mapping for each feature. How does this scale on datasets that have a larger number of features? This seems like a lot of parameters to be trained.
Q2: The method disentangles local and global feature representations. Opposite to Q1, have the authors investigated how their model performs on datasets that have very few features (e.g. http dataset from ADBench)? Intuitively the global representation would contain very little additional information than the 3 existing tokens.
Q3: Why were only 25 datasets from ADBench selected ?
Q4:The vanilla k-nn AD approach is a strong baseline that was omitted from the benchmark. It would be very interesting to include it and analyze how it compares to the proposed method. This method is particularly interesting as it is very cheap to compute.
Q5: The training setup that includes a validation and test set is different from the ones usually used in papers. What is the rational? This prevents from including previously ran model on the same benchmark. Also, this is not really aligned with the AD objective that assumes that one does not dispose of anomalies at training.
Q6: it would be interesting to analyze the global token representation space within normal distribution and between normal and anomaly distribution. How much does this representation differ from a simple average of the feature tokens (for the unmasked case used in inference)?
Overall the proposed approach is interesting and well explained but the main rational for the proposed method is that it performs well on a subset of datasets from a benchmark while omitting some strong baselines previously included. (e.g. Knn method). We encourage the authors to extend their benchmark and better analyze the obtained representation from their method to provide a better understanding of why their method might perform well. We recommend rejection but we are open to discussion.
Small typo: Line 216, authors mention while it is not mentioned in the previous equation.
2 comments
Answer to Rebuttal
We thank the authors for trying to address the different points we raised in our review. In particular adding KNN, clarifying the training setup and investigating other representations than the CLS token (e.g. mean pooling).
For this reason, I'm happy to increase my score.
Addressing weaknesses
Q1 - Reconstruction heads scaling
Thank you for raising this point. Each feature has a reconstruction mapping, so the parameter count grows linearly with the number of features.
We have clarified this in Sec. 3.3 where we introduce the implication on parameter growth and reference a section addressing this overhead in Appendix C of the revised manuscript. This section reports exact parameter counts and shows that the total reconstruction parameters account for less than 7% of the total parameters on the largest datasets.
Q2 – Performance with very few features
Thank you for this question. To address the suggested example, we evaluated the http dataset using the same model configuration and evaluation protocol of the main work (0.9997 AUROC). Our findins show that in datasets with few features there is no degradation in performance and it does not suffer from representational collapse. We addressed this in Section 4, and main evaluation tables of the revised manuscript.
Q3 - Selected ADBench datasets
The selected datasets balance dataset diversity with computational cotst (relates to your concern in Q1). Selection spans multiple domains, which range from 80 to 49,097 records and from 6 to 512 features (without http). We included diverse datasets so that the aggregate results reflect performance across varied characteristics. Related AD methods were also evaluated using a similar benchmark size (eg: MCM and Disent-AD use 20 datasets).
However, we acknowledge that the rationale for this selection was not explicitly stated in the original submission, and we have clarified it in Section 4 “Datasets” of the revised manuscript.
Q4 - knn
Thank you for this suggestion. We have evaluated KNN under the exact same one-class evaluation protocol across the selected ADBench datasets, and integrated the results into Section 4, Table 1 (macro averages), and Appendix A of the revised manuscript. In our experiments, KNN was found to be the strongest classical baseline. KNN achieves a macro AUROC of 0.8386 and a macro AUPRC of 0.7401, outperforming all other classical methods (Isolation Forest, OCSVM, and ECOD) as well as every deep baseline except MCM. However, Lo-GRAD substantially outperforms KNN across the benchmark, improving macro AUROC by 4.42 percentage points and macro AUPRC by 9.01 percentage points.
Q5 - Training setup
Thank you for raising this point. Our rationale for the 70/10/20 split of normal records is to separate model fitting, validation, and performance assessment, where both training and validation contain exclusively normal records. For Lo-GRAD and all deep baselines the train and validation set does not contain or use any anomaly labels. Although differences in evaluation protocols can limit direct comparison with the scores reported in other studies (eg: ADBench uses 50/50 split), the comparisons within our submission follow the same protocol and splits, without a per-dataset parameter tunning.
We have clarified that our method does not use anomalies during training or validation and the rationale for data splitting and evaluation protocol in Section 4 and Appendix F.1 of the revised manuscript.
Q6 - Global representation
Thank you for this suggestion. We also agree that this analysis addresses a gap in the original manuscript. To address this question, we compared CLS against the average of the feature tokens at inference on the three ablation datasets. The two representations exhibit similar inter-record distance distributions in Yeast, however, this distance increases substantially in the others.
We also evaluated whether the feature-token average can replace CLS as the global reference at inference. Using the mean Chebyshev scorer, we show that replacing CLS with mean pooling reduces macro AUROC by 11.43 percentage points and macro AUPRC by 13.99 percentage points. We have added this analysis to Section 4.2 and Appendix D.5 of the revised manuscript.
Typo in line 216 fixed, thank you.
Please let us know if these responses resolve your concerns, or if any additional experiments or clarifications would be helpful.