- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Anisotropy-aware Diffusion Time Estimation (ADTE) for unsupervised tabular anomaly detection. It adds to DTE a covariance-based training filter and an auxiliary timestep regression head, as well as a new diffusion noise. The paper shows empirically, using the 57 ADBench datasets, that the filtering is useful to many baseline detectors and that the modified diffusion noise provides some improvements. The paper also includes several ablations and additional results in the appendix.
Critical strengths
- The modified diffusion noise to DTE is simple and motivated. Shaping corruption using feature correlations is a natural extension of DTE.
- The empirical evaluation is extensive. The paper compares with several other baselines, with multiple detector families, and evaluates the filtering over those methods as well. There are also ablations for each component, parameter sensitivities, and paired comparisons across datasets.
- The disclosed limitations are honest and useful. The author clearly describes the unusual categorical training objective, the weaker embedding results, and the extensive AI assistance.
Critical weaknesses
As disclosed by the authors (and is quite obvious when reading the paper), the paper uses generative AI extensively, which I believe explains my following assessment:
- The supplemental material (codebase) shows a large set of files with many different experiment attempts, as well as agent instructions. One of these states: “Each analysis DEPLOYS only if its own p < 0.05; otherwise it is NULL and stays internal.” While some of these “null” results are presented, it is unclear how many unfavourable outcomes were removed. The scripts also motivate new experiments using previous successes and failures. One justifies restricting an experiment to tabular datasets by noting that the broader comparison was not statistically significant. With that many follow-up hypotheses and experiments on the same benchmark, this raises large concerns over whether the paper presents meaningful results for the problem presented or is simply maximizing results on a benchmark.
- The proposed method adds several components to DTE, but does not show a significant improvement over simple baselines such as T² and Euclidean kNN. The improvement from the additional regression head is also small and weakly supported across settings. Given the added complexity, it is unclear whether the method provides a meaningful improvement for the problem presented.
- The paper is very hard to read and often feels like a long experiment log. The main idea is simple, but it gets buried under experiments, subgroup comparisons, and appendix references, and there isn't even a single figure. It is difficult to understand which findings actually support the main claims and which experiments were added to explain earlier results. I would suggest that the authors take over the writing and do a thorough human rewrite: remove the clutter, decide which findings are central, and assemble the results into a clear paper.
Questions for authors
- How do your reconcilie the small effect of the auxiliary head small effect on performance with its claimed importance?
- Given the intervals against T² and Euclidean kNN, what evidence supports a practical advantage of the learned model over these alternatives?
Thank you for the review.
Package: the quoted line shows you read the supplementary material closely; the version you examined was our full working tree, experiments that did not reach the paper included. Whether results were maximized on the benchmark is a question any paper has to answer with its own numbers, and we do so below: the components and the settings. The package now contains the code and recorded results that regenerate every table of the paper.
Components: each has one purpose, and each is achieved. The filter addresses contamination: removing it from ADTE-C costs 2.79 AUCROC points [95% CI 0.97, 4.87] on the 57 groups. Aligned noise shapes the corruption by the data's own correlation, retained anomalies included (the central novelty, see Baselines): aligned minus isotropic noise, everything else identical, +0.73 [+0.14, +1.44] on the 57 groups and +0.92 [+0.23, +1.73] on the 47 tabular groups. The regression head stabilizes results across seeds (the abstract's words). Stability, not size, is what the paper claims for the head, so a small mean effect is expected. The measured one, +0.33 AUCROC points [+0.03, +0.68] on the 57 groups, is significant in the paper's statistical sense of an interval that excludes zero. The stability shows in the seeds: with the filter and aligned noise, the head narrows a group's AUCROC spread over its five training seeds by 21% and raises each group's lowest seed outcome by an average of +0.63 points [+0.20, +1.22], while the highest seed outcome does not change significantly. Without the filter and under isotropic noise, the head narrows the spread by 34% and raises the lowest outcomes by +1.91 [+0.68, +3.22] (Table 21). A narrower seed spread also makes differences in results easier to measure.
Parameters: DTE's network, diffusion schedule and optimizer are as released; what we set, once for every dataset, is K = 8 bins (DTE's default, 7, is the value its authors tuned over all the datasets), head weight λ = 0.1, score weight ω = 0.5, full alignment (s = 0), and for the filter a support fraction of 0.75 with a 32-dimensional PCA projection only when a dataset has more features. Nearby settings change little, all on the 57 groups: ω anywhere from 0 to 1 changes the mean AUCROC by at most 0.22 points. The effect of a support fraction of 0.70 or a PCA dimension of 16 or 64 is not distinguishable from zero. K from 6 to 10 moves the mean by at most 0.1 point, and the chosen bin count does not even maximize it: K = 10 gives 77.09 against K = 8's 77.02 (Table 5).
Quoted instruction: the scripts the review cites, the quoted line among them, belong to the analyses that explain the results after the detector was fixed and its 77.0 AUCROC obtained; they did not choose the detector or its settings, and the comparison of detectors in Table 1 reports its numbers whatever their significance. The quoted line applies the paper's 5% significance level to those analyses: a result is stated as a finding only when it is significant at that level, and non-significant contrasts are printed with their intervals too, as you note. The split into 47 tabular and 10 embedding groups is the benchmark's own, by data type. On the 10 embedding groups, the paper reports that DTE itself is behind the classical detectors before any alignment (Section 4.6). That is why the analyses of alignment focus on the 47 tabular groups. A variant of the aligned noise recovers part of that gap on the embedding groups, raising ADTE-C there by +0.80 AUCROC points [0.23, 1.45] (Section 4.6). The detector comparison and the alignment effect are reported on both the 57 groups and the 47.
Baselines: the paper claims no significant margin over T² or Euclidean kNN; it claims the highest mean AUCROC among the detectors compared after the same filter, and a significant alignment gain. With the paper's filter, those two detectors are not ordinary baselines. T² (76.8) and Euclidean kNN (76.5) then have a higher mean AUCROC than every other detector in Table 1, including the learned ones and the robust-covariance detector built on the filter's own estimator (75.2 at its best), and only ADTE-C (77.0) scores higher. The learned model's advantage is the central novelty: anomalies help shape the training noise, even better than the normal samples, where T² and Mahalanobis kNN need them removed from their correlation estimate; the evidence is in our reply to Reviewer ZA9R (Novelty). That understanding is what can be leveraged into a significant improvement over the filtered T² on the 47 tabular groups, at lower computation than ADTE-C. The components add no training cost: averaged over the member datasets, the full pipeline trains in 215 seconds against 219 for DTE-C without them, plus about 2.5 seconds per dataset for the filter on a CPU (Section 4.3). We will run any detector you consider a stronger comparator, since we found none.