acceptodds
Withdrawn from ICLR 2027

CRETAP: From Reconstruction Errors to Reliable Anomaly Probabilities on Imbalanced Data

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstruction-based anomaly detectors, such as autoencoders, rank samples by an error whose scale depends on the dataset, preprocessing, and aggregation, so thresholds do not transfer and scores are not probabilities. Fixing this with post-hoc calibration withholds scarce labeled anomalies from training, and switching to supervised classifiers discards the element-wise residuals that explain each decision. We introduce CRETAP, which fits Platt scaling of the log-error—a two-parameter Hill function—jointly with the reconstruction network under binary cross-entropy (BCE) instead of afterwards, so the anomalies that train the detector also calibrate it. Because the probability depends on the input only through the error, we prove that the excess BCE upper-bounds twice the squared calibration error and that stationary points of training are mean-calibrated with unit calibration slope in-sample. The forward pass also yields feature-level attributions—each element's error contribution divided by the learned decision boundary—that sum to more than one exactly when the probability exceeds one half. On the 29 ADBench datasets with at most 5% anomalies and without any post-hoc calibration, CRETAP is the best-calibrated of eight natively probabilistic detectors on 9 datasets, more than any other, and second in mean expected calibration error (0.0073 vs. 0.0067 for XGBOD), while its mean area under the precision–recall curve, 71.19%, trails CatBoost's by 2.0 points. In this regime, a reconstruction detector can thus serve as its own calibrator; above 5% contamination, gradient-boosted trees are preferable.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Anomaly Detection, Calibration, Reconstruction-based Models, Imbalanced Data, Feature Attribution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bBFxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CRETAP converts reconstruction errors into anomaly probabilities through a two-parameter Hill function, equivalent to logistic/Platt scaling on log reconstruction error, trained jointly with a reconstruction network using binary cross-entropy. The paper establishes monotonicity, unit-change invariance, a standard excess-log-loss bound on calibration error, and empirical moment conditions at stationary points. It also scales additive residual contributions by the learned decision boundary to obtain inexpensive attribution maps. Across 29 ADBench datasets with at most 5% anomalies, CRETAP reports mean AUPR 71.19%, AUROC 94.11%, and ten-bin ECE 0.0073, compared with XGBOD ECE 0.0067 and CatBoost AUPR 73.20%.

Critical strengths
  1. The method is simple and interpretable: a monotone two-parameter probability map preserves anomaly ranking, has a clear operating threshold, and can be trained with standard BCE. Its ease of implementation and model-agnostic reconstruction interface are practically useful.

  2. The paper provides broad benchmarking and unusually explicit limitations, including cases where boosted trees win, sensitivity to irrelevant features, and the weakness of binned ECE under rare anomalies.

Critical weaknesses
  1. The claimed benefit of joint calibration is not experimentally isolated. There is no matched comparison against the same autoencoder followed by post-hoc Platt/Hill calibration, nor against the closely related ABC construction. Because the Hill transform itself is not novel, this missing ablation directly undermines the central methodological claim.

  2. The paper motivates scarce anomaly labels but evaluates primarily with all training anomalies labeled. Ten-bin ECE can be very small for a constant prevalence predictor, and the paper itself reports such a case. Thus the current evidence does not establish reliable probability estimates under genuinely limited labels or at operational alert thresholds. The stationarity result is in-sample and does not close this gap.

Questions for authors
  1. Under identical architecture, data, tuning budget, and held-out evaluation, how does joint training compare with reconstruction pretraining followed by Platt/Hill calibration and with ABC? Please report AUPR, NLL, Brier, and calibration at operational thresholds.

  2. What happens when only 1, 5, 10, or 20 training anomalies are labeled, with the remaining anomalies treated as unlabeled? Does the proposed advantage persist?

  3. Can the authors report class-conditional and threshold-local calibration, together with uncertainty intervals, and test the effect of deployment prevalence shift?

Reviewer_czfjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes CRETAP, a reconstruction-based anomaly detection framework that jointly learns reconstruction errors and calibrated anomaly probabilities. Specifically, it incorporates a two-parameter Hill function into the reconstruction network and optimizes both components using binary cross-entropy. The method further provides feature-level attributions through normalized reconstruction residuals. Experiments on ADBench evaluate anomaly detection and probability calibration, with competitive results primarily on datasets with low anomaly contamination.

Critical strengths
  1. Interesting problem formulation. The paper addresses the practical challenge of obtaining calibrated anomaly probabilities from reconstruction errors while preserving feature-level residual information.

  2. Extensive benchmark evaluation. The experiments cover a wide range of tabular, image, and text datasets, with comparisons against numerous anomaly detection methods. The theoretical analysis also provides useful mathematical properties of the proposed probability mapping.

Critical weaknesses
  1. Limited methodological novelty. The proposed Hill function is mathematically equivalent to Platt scaling applied to log-reconstruction errors. The main difference is joint optimization with the reconstruction network using BCE. The authors should better justify the methodological novelty beyond existing supervised reconstruction and probability calibration approaches.

  2. Missing essential comparisons with post-hoc calibration. A central motivation is that joint training can avoid the limitations of post-hoc calibration. However, the paper does not directly compare joint training against reconstruction followed by Platt scaling or isotonic regression under matched labeling budgets. This missing experiment substantially weakens the central contribution.

  3. Concerns about evaluation fairness and model selection. CRETAP receives extensive dataset-specific hyperparameter optimization, whereas the competing methods largely use default benchmark configurations. Furthermore, Appendix H.2 indicates that validation samples from one seed may overlap with test samples from another seed during shared configuration selection. The authors should provide independently selected configurations and fairly tuned baselines.

  4. Limited evidence for practical probability calibration. The theoretical guarantees mainly concern population BCE bounds and in-sample stationary points, rather than guaranteed test-time calibration. Moreover, ECE can be misleading under extreme class imbalance. Stronger calibration analysis near decision thresholds and experiments with varying labeled-anomaly budgets are needed.

  5. Limited novelty and validation of feature attribution. The proposed attribution is essentially the reconstruction residual divided by a learned scalar threshold. Although mathematically consistent with the prediction, it provides limited additional information beyond conventional residual maps. Quantitative localization or explanation-fidelity comparisons would strengthen this contribution.

Questions for authors

See the weaknesses part.

Reviewer_YRhWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets the problem of turning the reconstruction errors of anomaly detectors, such as autoencoders, into calibrated anomaly probabilities when only a few labeled anomalies are available. The proposed method, CRETAP, maps the error to a probability with a two-parameter function, which is equivalent to Platt scaling on the log error. It trains this function together with the autoencoder using binary cross-entropy, so the labeled anomalies are used for both detection and calibration. The paper proves that this training objective bounds the calibration error, and it derives per-feature attributions whose sum determines the decision. On 29 ADBench datasets with at most 5% anomalies, CRETAP ranks third in AUPR and second in calibration error (ECE) among eight probabilistic detectors.

Critical strengths

S1. A simple, practical idea with useful properties. Adding one two-parameter function after the error turns any reconstruction model into a probabilistic detector at almost no cost. The design has clear benefits: the output does not depend on the units of the error, the parameter k has a direct meaning (the error level where a sample is 50% likely to be anomalous), and rankings are unchanged. Practitioners who already use autoencoders could adopt it easily.

S2. Comprehensive evaluation. The paper compares against 28 baselines on all 57 ADBench datasets, with significance tests, robustness to ADBench's noise settings, train-test calibration gaps, and sensitivity to the 5% scope boundary. It also clearly states where the method loses, such as above 5% anomalies and with irrelevant features. Within its scope, CRETAP is competitive: third in mean AUPR (71.19%), second in mean ECE (0.0073), and on the AUPR-ECE Pareto frontier with CatBoost and XGBOD.

S3. A well-justified choice of mapping function. The paper explains carefully why it uses this function rather than alternatives such as a Weibull CDF or a sigmoid of the raw error. Its logistic form ties the training loss directly to calibration, giving mean calibration at convergence. It also bounds each sample's gradient, so an unlabeled anomaly hidden among nominal training data has limited influence. Finally, a change in the anomaly rate at deployment can be corrected in closed form by rescaling k alone, without retraining. Together these properties make the method easy to reason about and adjust in practice.

Critical weaknesses

W1. Limited novelty. The paper itself states that the mapping is Platt scaling on the log error, so the only new element is training it jointly with the network. The main results follow from known facts. The calibration bound (Prop. 2(ii)) is the standard decomposition of log loss plus Pinsker's inequality. The stationarity result (Prop. 2(iii)) is the well-known in-sample calibration property of logistic regression. The attributions are the usual residual map divided by one constant, and the paper admits they cost the same and are not claimed to be better. Also, since the training loss has no reconstruction term, the model is in effect a supervised classifier with a constrained output layer. The image results confirm that reconstructions drift far from the input at defects, which weakens the "reconstruction detector" framing.

W2. The experiments do not fully support the main claim. The paper's main idea is that joint training beats post-hoc calibration when labeled anomalies are scarce. Yet there is no ablation comparing joint fitting of the function with post-hoc fitting on the same network, and no comparison with ABC, the closest prior work. The authors acknowledge both gaps. The number of labeled anomalies is never varied: every experiment labels all training anomalies, even though scarce labels are the motivation. Finally, CRETAP is tuned per dataset (60 trials), while all baselines run untuned and uncalibrated, and CRETAP's validation splits overlap other seeds' test splits.

W3. Small gains. At these anomaly rates, ECE is nearly uninformative: a constant predictor that always outputs the base rate reaches an ECE of 0.0004, lower than every model (Table 8). The ECE differences among the top models (0.0067 to 0.0087) are small and mostly not significant. Unbinned measures such as BCE and Brier score show CRETAP and CatBoost essentially tied. The Pareto-frontier result depends on the 5% cutoff: CRETAP is on the frontier at none of the 15 cutoffs below 5%. CRETAP does not beat CatBoost on detection, and it falls behind boosting above 5% anomalies and with irrelevant features. Overall, the paper does not make a clear case for choosing CRETAP over gradient-boosted trees.

Questions for authors

Q1 (W2). How does joint training compare with post-hoc Platt scaling of the log error on the same autoencoder, using the same labeled anomalies? This is the paper's central claim.

Q2 (W2). How do the results change as the fraction of labeled anomalies decreases (e.g., 5%, 25%, 50% labeled)? Scarce labels are the main motivation, but every experiment labels all of them.

Q3 (W2). How does CRETAP compare when the baselines get the same tuning budget, when they are calibrated post hoc where enough anomalies exist, and against ABC?

Q4 (W3). Since a constant base-rate predictor beats every model on ECE, can you report calibration measures that focus on the anomaly class or the decision region, rather than binned ECE dominated by the lowest bin?

Q5 (W1). Without a reconstruction term, does e(x) remain a meaningful reconstruction error on tabular data? How do detection, calibration, and attributions change if a reconstruction loss is added?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.