acceptodds
Withdrawn from ICLR 2027

Rethinking Thresholding as a Neglected Frontier in Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Anomaly detection aims to identify data samples that deviate from expected patterns and is widely applied across domains. In practice, it consists of two stages: (i) computing continuous anomaly scores; and (ii) converting these scores into binary decisions via thresholding. While extensive research has focused on scoring mechanisms, thresholding is often treated as a minor detail, despite being essential for evaluation, comparison, and deployment. Over half of studies and libraries select thresholds using supervision, introducing oracle knowledge that conflicts with the unsupervised setting. Through a diagnostic study of 38 thresholding methods and 8 scaling strategies across three data modalities–time-series from TSB-AD, and tabular and image data from ADBench benchmarks–we show that violations of common parametric assumptions cause complete thresholding failures on one-third of datasets. Score normality, for example, holds in fewer than 2% of empirical distributions. Thresholding choices alone can also reverse detector rankings. To address these limitations, we propose the **Energy-Based Adaptive Boundary (EAB)**, a nonparametric, parameter-free method that partitions anomaly scores using a Gini-mean-difference-calibrated two-sample energy distance. Centroid aggregation over the energy landscape provides robustness to autocorrelation, while a prefix-sum reformulation reduces naive cubic computation to time and space. Compared with the nine strongest unsupervised baselines, EAB achieves statistically significant F1 improvements against every baseline, with gains of up to 17.7% on time-series data and 16.2% on tabular and image data, confirmed by Wilcoxon signed-rank tests. These results establish EAB as an efficient, modality-agnostic framework for unsupervised anomaly thresholding.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Anomaly Detection, Thresholding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FDTUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper suggests studying thresholding as a limiting factor in anomaly detection research, show limitations of existing approaches and suggest their own thresholding approach they claim to outperform every other algorithm.

Critical strengths

I agree with the initial idea of studying anomaly-detection components, especially thresholding. Furthermore, the number of approaches considered here is impressive, even though I am still missing one baseline.

Critical weaknesses

I have problems with the reliability of the results. Concretetely I am missing 3 points, that make me question every result reported in the paper.

  1. Using only 10 datasets (tabular + image) is very little. Modern benchmarks like https://arxiv.org/abs/2602.09329 suggest using orders of magnitude more datasets, yet remain susceptible to noise from datasets that prefer different algorithms. As you benchmark more methods! Using less than 1% of the datasets suggested makes each of your results untrustworthy. I know that there might be computational limitations, but trustworthy results require at least an order more datasets.

  2. You are missing one baseline. In 3.2, you study the 3S rule and accurately criticize it for potentially treating all anomalies as normal. However, there is a simple baseline that you missed (as far as I can see): Simply considering, e.g., 99% of the training data as normal and using this to define a threshold virtually guarantees a valid threshold and requires negligible computational cost and is the default in the largest library used for anomaly detection PyOD. Missing this baseline is a critical mistake and reason enough to reject your paper. Please add it. If I simply missed it during reading (your naming is a bit confusing), please point out what you call it.

  3. Your anomaly detection methods are slightly outdated, especially with regard to thresholding difficulties. Namely, you are missing modern tabular foundation models (e.g., https://arxiv.org/abs/2603.19497 or https://arxiv.org/abs/2409.05672v3 or https://arxiv.org/abs/2602.03018). These are not only more powerful than the outdated methods you choose, but are also often trained to output binary labels, which affects thresholding choices. E.g., the threshold 0.5 is a valid choice. Whether thresholding methods can improve this further or whether modern foundation models make thresholding a question of the past would be interesting in either case.

Further, your presentation requires work. In particular, your Figures tend to be very confusing and lack the information needed to understand them. For example, in Figure 3: what is the x-axis? Normalizations? Then why no entry "no normalization"? Also which datasets? Tabular or Timeseries? I would also have liked a description of each of the 38 methods in the main paper, or at least of their general ideas. Your paper works partially as a benchmarking paper on thresholding, but I did not learn anything about the approaches people use for thresholding from the main paper. And the descriptions in the appendix are not much better: You mention the methods, but there is very little deeper understanding shown here. The statement in the abstract that over half of studies and libraries require supervision would be very strong, but requires justification that I missed in the paper.

Questions for authors

In its current form, I cannot recommend accepting the paper. However, if you could increase the quality of the reported results (Points 1-3 under weaknesses), I would change my recommendation to either weak or clear acceptance.

Reviewer_zu44OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper does not focus on improving the anomaly detector itself, but instead studies the downstream threshold-selection problem. Specifically, given anomaly scores from an existing detector and no access to labels, the paper investigates how to choose a threshold that converts the scores into binary predictions with good F1 performance. It further proposes EAB, a parameter-free unsupervised thresholding method based on distributional separation, and evaluates it across multiple anomaly-detection benchmarks.

Critical strengths
  1. The paper highlights an overlooked component of the anomaly-detection pipeline. The systematic study of a large number of thresholding and score-scaling methods shows that the final binary predictions can be highly sensitive to the choice of threshold, which is useful for understanding and standardizing AD evaluation.
  2. EAB is simple, parameter-free, and computationally efficient. It avoids explicit parametric assumptions on anomaly-score distributions and achieves strong empirical performance across the evaluated benchmarks while retaining O(nlog⁡n)O(n\log n) complexity.
Critical weaknesses
  1. The practical motivation and evaluation objective are questionable. The paper treats unsupervised threshold selection as an important deployment problem and evaluates threshold quality mainly by F1. However, in real applications the desired operating point is usually determined by the definition of the target anomaly and application-specific costs of false positives and false negatives, which can be highly asymmetric. Maximizing F1 therefore does not necessarily correspond to a practically meaningful threshold. More fundamentally, a strong AD method should primarily make normal and anomalous score distributions well separated; when this is achieved, the exact threshold becomes much less critical.
  2. The experimental evidence is insufficient to support the broad modality-agnostic claims. The evaluation is limited to TSB-AD and a small collection of tabular/image datasets from ADBench, using mostly classical or generic anomaly detectors, rather than modern domain-specific AD pipelines. In particular, there is no evaluation on representative modern visual anomaly-detection benchmarks such as MVTec AD or VisA with contemporary visual AD methods.
    Moreover, the main results are reported primarily through aggregate box plots, average ranks, and win/loss statistics rather than detailed per-dataset and per-detector results, making it difficult to determine whether EAB consistently works across different anomaly patterns or whether the gains are concentrated in particular settings.
Questions for authors

See weaknesses.

Reviewer_NQvMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that thresholding is a neglected but consequential stage of anomaly detection. An audit of over 30 recent studies and 11 widely used libraries finds that more than half select thresholds using ground-truth labels, and that the choice of thresholding method alone can reverse detector rankings. A diagnostic study across time-series (TSB-AD) and tabular/image (ADBench) data shows that parametric assumptions rarely hold. The authors then propose the Energy-Based Adaptive Boundary (EAB), which evaluates candidate splits of the sorted scores by two-sample energy distance, calibrates against a Gini-mean-difference baseline, and takes the final threshold as an energy-weighted centroid, computed in O(n log n) via prefix sums. EAB ranks first among unsupervised thresholding methods on both benchmarks, with significant F1 gains over the nine strongest baselines.

Critical strengths

The audit and diagnostic study have standalone value for the community. Demonstrating that most reported benchmarks rely on supervised thresholds, that thresholding alone can reorder detector rankings, and that normality-based rules silently fail on a third of datasets gives practitioners concrete reasons to distrust current evaluation practice, independent of the proposed method. The empirical evaluation is also unusually thorough for this area: EAB wins against all nine strong unsupervised baselines on both benchmarks with paired Wilcoxon tests, the analysis is broken down by normalization strategy with a plausible mechanistic explanation of where EAB is weaker, and the method is model-agnostic, parameter-free, and fast enough to be a drop-in replacement for the heuristics currently in use.

Critical weaknesses
  • The conceptual novelty of EAB is limited. Bipartitioning sorted scores by a distributional divergence is closely related to change-point and two-sample segmentation, the null calibration transfers the gap statistic, and centroid aggregation with a prefix-sum speedup are standard techniques. The combination is well executed and each part answers a diagnosed failure mode, but the paper does not show what the method adds beyond a careful recombination of these existing ideas.

  • The null calibration, which determines whether EAB produces a boundary at all, is insufficiently justified. The 2G baseline assumes that a uniform reference distribution is the worst case for spurious boundaries, but this claim is not proven, and it is unclear why a baseline derived from a uniform reference should calibrate skewed or heavy-tailed anomaly-free score populations. The baseline is also constant across split positions, while the variance of the energy distance is not, so its finite-sample behavior at unbalanced splits is unknown. The paper never reports false-boundary rates on anomaly-free data. In addition, Algorithm 1 returns a fallback threshold when no split has positive excess energy, which conflicts with the stated goal of suppressing false boundaries and leaves open the question of when EAB can declare that no anomalies exist.

  • The statistical reporting is incomplete. The Wilcoxon tests in Table 1 never define the task unit, and tasks sharing a dataset or detector are unlikely to be independent, which inflates the effective sample size. Nine baseline comparisons per benchmark are reported without multiplicity adjustment. The analyses also use different subsets in different places (242 versus 350 time series, 10 versus 11 detectors) without a consolidated account of which data enter each number.

Questions for authors
  1. Can you derive why 2G is an appropriate null baseline for every split position, and report the rate at which EAB produces a boundary on simulated anomaly-free scores under uniform, skewed, and heavy-tailed distributions at varying sample sizes?

  2. Under what conditions does EAB return no anomaly predictions, and how is the fallback branch in Algorithm 1 consistent with suppressing spurious boundaries?

  3. For Table 1, what exactly constitutes a task, how do you handle dependence among tasks sharing a dataset or detector, and do the conclusions survive multiplicity-adjusted tests?

  4. Finally, is EAB intended for offline batch scoring or online detection, and could you provide a breakdown of which datasets and detectors enter each reported analysis, including how the 3,610 score sets were obtained?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.