- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CRETAP converts reconstruction errors into anomaly probabilities through a two-parameter Hill function, equivalent to logistic/Platt scaling on log reconstruction error, trained jointly with a reconstruction network using binary cross-entropy. The paper establishes monotonicity, unit-change invariance, a standard excess-log-loss bound on calibration error, and empirical moment conditions at stationary points. It also scales additive residual contributions by the learned decision boundary to obtain inexpensive attribution maps. Across 29 ADBench datasets with at most 5% anomalies, CRETAP reports mean AUPR 71.19%, AUROC 94.11%, and ten-bin ECE 0.0073, compared with XGBOD ECE 0.0067 and CatBoost AUPR 73.20%.
Critical strengths
-
The method is simple and interpretable: a monotone two-parameter probability map preserves anomaly ranking, has a clear operating threshold, and can be trained with standard BCE. Its ease of implementation and model-agnostic reconstruction interface are practically useful.
-
The paper provides broad benchmarking and unusually explicit limitations, including cases where boosted trees win, sensitivity to irrelevant features, and the weakness of binned ECE under rare anomalies.
Critical weaknesses
-
The claimed benefit of joint calibration is not experimentally isolated. There is no matched comparison against the same autoencoder followed by post-hoc Platt/Hill calibration, nor against the closely related ABC construction. Because the Hill transform itself is not novel, this missing ablation directly undermines the central methodological claim.
-
The paper motivates scarce anomaly labels but evaluates primarily with all training anomalies labeled. Ten-bin ECE can be very small for a constant prevalence predictor, and the paper itself reports such a case. Thus the current evidence does not establish reliable probability estimates under genuinely limited labels or at operational alert thresholds. The stationarity result is in-sample and does not close this gap.
Questions for authors
-
Under identical architecture, data, tuning budget, and held-out evaluation, how does joint training compare with reconstruction pretraining followed by Platt/Hill calibration and with ABC? Please report AUPR, NLL, Brier, and calibration at operational thresholds.
-
What happens when only 1, 5, 10, or 20 training anomalies are labeled, with the remaining anomalies treated as unlabeled? Does the proposed advantage persist?
-
Can the authors report class-conditional and threshold-local calibration, together with uncertainty intervals, and test the effect of deployment prevalence shift?