- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper says that in tabular SAD, oversampling a small pool of labeled anomalies causes something called “distribution collapse”. Shared BN statistics make the normalized mean of normal samples nearly constant across batches, which removes part of BN’s batch noise. So their method NoBa computes BN statistics only from normal samples, then centers them at the Pearson mode, and applies them to all samples. Their experiments show consistent average gains over BN and other normalizers.
The method is practical and the gains over BN look real. But thhe proposed mechanism is not established. Both propositions are algebraic identities, a simpler explanation fits the evidence. The comparisons also have issues that make the gains hard to attribute.
Critical strengths
Some strengths I can see:
-
This paper has a simple, drop-in method. Labels are already available during SAD training, so computing BN statistics from normal-labeled samples adds nothing at inference time. Inference uses running statistics exactly as standard BN does. It works with any backbone without changing the objective.
-
The evaluation is broad and good. The paper covers many settings with sweeps over learning rate, batch size and number of labeled anomalies. And NoBa beats BN on average in every configuration.
-
Table 2 crosses reference set (all vs normal-only) with centering statistic (mean, median, mode). This is a clean ablation as it separates the two components clearly and show that the reference set accounts for most of the gains.
Critical weaknesses
There are also a lot of weaknesses, which outweigh the strengths in my opinion:
-
FIgures 3 and 7 appear inconsistent with the definitions. Under mean-centered, normal-only BN, the normal-sample z should be exactly zero. But the “Unsup BN normal samples” panel shows high-frequency variation around ±1. Either a different quantity is plotted (post-affine, post-activation, or using running statistics?), or something else is going on.
-
The collapse mechanism is asserted but not shown. I don't see anything saying Var(q_B) shrinks because anomalies are reused. With 60 anomalies and 256 anomaly slots per batch, each anomaly appears about 4× per batch from the first step. Reuse is constant over training, so I think it cannot drive a decline over epochs.
-
Many issues in the comparisons I can see: a) Gains are concentrated in two datasets. On DevNet, thyroid alone contributes about 0.034 of the 0.053 average gain over w/out BN, and thyroid plus campaign account for about 90%. NoBa is worse than w/out BN on 5 of 11 datasets. b) Despite saying 10 seeds, there are no standard deviations or significance tests, and many margins are 0.01–0.02. This is a minor point, but still. c) The w/out BN baseline uses ReLU, while BN-based variants use ELU. So it is not the same.
-
I have a fair concern about this paper's novelty, is it actually new? I'm not familiar with the literature here, but the paper does not convince me.
-
The paper also does not provide any code, it's not a must but it doesnt help either.
-
Some more minor points: Figure 2 is hard to interpret. In (a, b), cross-pi comparisons are confounded by the pi^2 factor. Panel (c) is 11 points with no statistics, and the layer measured is unspecified.
Questions for authors
Some questions for the authors, in addition to the weaknesses above:
- What exactly is plotted as z̄ in Figures 3 and 7, and why is it non-zero for mean-centered, normal-only normalization?
- How does w/out BN perform with ELU? Are input features standardized before the network?
- Can you report standard deviations and significance tests (e.g., Wilcoxon or critical-difference diagrams), and the average gains excluding thyroid?
- Why do the Table 2 numbers differ from Table 1?
6 comments
Response to W3. Comparison experiments
W3 & Q2,3. Comparison experiments
We address the three concerns (a) concentration of gains, (b) statistics, and (c) activation function.
(a) Concentration of gains. We report the results without thyroid and campaign. NoBa still exceeds both baselines for all five backbones over the remaining nine datasets, although the margin over w/o BN is small for DevNet (+0.006) and PReNet (+0.001), whose gains are concentrated mainly in these two datasets.
| Method | DevNet | DSAD | PReNet | FeaWAD | CAD | Avg. |
|---|---|---|---|---|---|---|
| w/o BN | 0.635 ± 0.007 | 0.660 ± 0.008 | 0.615 ± 0.012 | 0.612 ± 0.016 | 0.614 ± 0.006 | 0.627 ± 0.007 |
| BN | 0.578 ± 0.028 | 0.659 ± 0.017 | 0.589 ± 0.029 | 0.548 ± 0.026 | 0.621 ± 0.011 | 0.599 ± 0.011 |
| NoBa | 0.641 ± 0.017 | 0.679 ± 0.010 | 0.616 ± 0.020 | 0.629 ± 0.016 | 0.647 ± 0.010 | 0.643 ± 0.010 |
To assess consistency beyond average performance, we additionally conduct a dataset-level rank analysis over the eleven datasets. NoBa achieves the best average rank at both contamination levels, and the Friedman test indicates significant differences among the three methods at both and contamination (, ). The post-hoc Nemenyi test shows that NoBa ranks significantly better than conventional BN at both settings (), whereas its difference from w/o BN is not significant, consistent with the fact that NoBa does not uniformly outperform w/o BN on every dataset.
| Contam. | w/o BN | BN | NoBa | NoBa vs. BN | NoBa vs. w/o BN |
|---|---|---|---|---|---|
| 0% | 2.182 | 2.545 | 1.273 | 0.008 | 0.083 |
| 2% | 1.818 | 2.727 | 1.455 | 0.008 | 0.670 |
The value of NoBa therefore does not lie in universal dominance over w/o BN. Under oversampling, conventional BN helps on some datasets but substantially degrades performance on others, whereas NoBa reduces this dataset-dependent variability in the usefulness of BN.Even where the gain over w/o BN is marginal, NoBa reduces the need to decide in advance whether BN will be beneficial for a given dataset.
(b) Statistical reliability. To assess statistical reliability, we conducted Wilcoxon and paired t-tests over 10 random seeds, applied to the 11-dataset mean AUPR (Appendix G). We also report the mean AUPR and standard deviation over seeds for each dataset and backbone (Appendix K). At 0% contamination, NoBa exceeds both w/o BN and BN for all five backbones, with p ≤ 0.003 in every comparison (table below). At 2% contamination, the gain over BN is significant under both tests for DevNet, PReNet, FeaWAD, and CAD. DSAD is significant under the paired t-test only (Wilcoxon p = 0.084). The gain over w/o BN is significant under both tests for DevNet, PReNet, and FeaWAD. The results support the statistical reliability of the gains under clean supervision and the advantage over conventional BN for most backbones under contamination.
| Contam. | Backbone | Wilcoxon vs. w/o BN | Paired t-test vs. w/o BN | Wilcoxon vs. BN | Paired t-test vs. BN |
|---|---|---|---|---|---|
| 0% | DevNet | 0.002 | < 0.001 | 0.002 | < 0.001 |
| 0% | DSAD | 0.002 | < 0.001 | 0.002 | 0.003 |
| 0% | PReNet | 0.002 | < 0.001 | 0.002 | 0.001 |
| 0% | FeaWAD | 0.002 | < 0.001 | 0.002 | < 0.001 |
| 0% | CAD | 0.002 | < 0.001 | 0.002 | < 0.001 |
| 2% | DevNet | 0.027 | 0.020 | 0.002 | < 0.001 |
| 2% | DSAD | 0.037 | 0.059 | 0.084 | 0.032 |
| 2% | PReNet | 0.004 | < 0.001 | 0.020 | 0.018 |
| 2% | FeaWAD | 0.002 | < 0.001 | 0.002 | < 0.001 |
| 2% | CAD | 0.695 | 0.839 | 0.002 | < 0.001 |
(c) Activation function. The w/o BN baseline uses ReLU, whereas the BN-based variants use ELU. We therefore evaluated w/o BN with ELU as well, so that all methods share the same activation (Section 4.7). ELU improves w/o BN for DSAD (0.615 to 0.640), FeaWAD (0.603 to 0.622), and PReNet (0.571 to 0.580). It changes DevNet and CAD by at most 0.003. Under the matched activation, NoBa exceeds w/o BN for all five backbones. The gain of NoBa is not attributable to the activation function alone.
| Backbone | w/o BN + ReLU | w/o BN + ELU | NoBa (ELU) |
|---|---|---|---|
| DevNet | 0.591 ± 0.008 | 0.590 ± 0.008 | 0.644 ± 0.012 |
| DSAD | 0.615 ± 0.009 | 0.640 ± 0.008 | 0.670 ± 0.012 |
| PReNet | 0.571 ± 0.012 | 0.580 ± 0.012 | 0.624 ± 0.018 |
| FeaWAD | 0.603 ± 0.011 | 0.622 ± 0.012 | 0.633 ± 0.011 |
| CAD | 0.613 ± 0.010 | 0.616 ± 0.006 | 0.653 ± 0.008 |
Response to W4–W6 & Questions: Novelty, Code Availability, Figure 2, and Additional Clarifications
W4. Novelty: Why this is new
- An overlooked problem in SAD. Tabular SAD research has focused on architectures and objectives, and normalization has been treated as a fixed component. Most methods omit BN. We study BN as a component that interacts with how SAD is trained.
- A newly defined failure mode, shown theoretically and empirically. SAD relies on oversampling, so anomalies form a large share of every mini-batch and enter the BN statistics. We define the resulting distribution collapse, in which the batch-to-batch variability of the normalized normal representation is suppressed. We explain why it occurs and show it in experiments.
- Reliable use of BN under oversampling. The benefit of BN varies across datasets and backbones, so neither using nor removing it is uniformly preferable. NoBa reduces the need to choose between BN and w/o BN for each dataset.
- Compatibility with other modules. NoBa can be combined with other components such as the class-balanced loss. Under the same loss, NoBa improves over BN for all five backbones (Section 4.6).
- A new research direction. Class imbalance in SAD reaches the normalization layer, not only the loss and the sampler. This motivates further study of normalization in SAD.
W5. Code availability
The implementation of NoBa is ready and will be released on GitHub. To reduce ambiguity in reimplementation, Appendix A describes how NoBa computes its normalization statistics during training and at test time (Algorithm 1). NoBa modifies only the normalization statistics. It can be applied to other SAD models without changes to the sampler or the loss.
W6. Figure 2 Clarifications
We revised Figure 2 following your comments. In Figures 2(a) and 2(b), the original y-axis was , which contains the factor . Curves with different were therefore confounded. The y-axis is now , which removes this factor and allows a direct comparison across . In Figure 2(c), we added the Pearson correlation coefficient () over the 11 datasets. All quantities in Figure 2 are measured at the last layer of DevNet. This is now stated in the main text and in the figure caption.
Q1. What is plotted as z̄ in Figures 3 and 7? Please see our response to W1. The original panel plotted the batch mean μ instead of . The revised Figure 3 plots computed from the normalized activations before the affine transformation, and Figure 7 has been merged into it.
Q2. w/o BN with ELU, and input standardization. Please see our response to W3(c) for the w/o BN baseline with ELU. NoBa exceeds w/o BN with ELU across all five backbones. Input features are standardized before being passed to the network using StandardScaler. The mean and standard deviation are estimated exclusively from the training set, and the same training-derived statistics are applied to the test set.
Q3. Standard deviations, significance tests, and gains without thyroid. Please see our response to W3 (a) and (b). Appendix K reports standard deviations and Appendix G reports the Wilcoxon and paired t-tests. Excluding thyroid and campaign, NoBa still exceeds both baselines for all five backbones.
Q4. Differences between Table 1 and Table 2
The differences arise from the BN implementation. Table 1 uses PyTorch's built-in torch.nn.BatchNorm1d. Table 2 uses a BN module that we implemented with PyTorch operations, in order to control which samples enter the normalization statistics (normal and abnormal, or normal only) and which centering statistic is used (mean, median, or mode). The two implementations perform similarly. Between the BN in Table 1 and the normal & abnormal + mean setting in Table 2, the difference in average AUPR is at most 0.007 for every backbone. The custom module therefore reproduces the behavior of the standard BN, and it is a suitable basis for the ablation in Table 2.
Response to W1–W2: Figure Consistency and the Mechanism of Distribution Collapse
Thank you for the careful and constructive review. Your reading of the figures, including the appendix, identified a genuine error on our side and helped us improve the paper. In response, we revised the figures for clarity, refined the proposition characterizing the collapse mechanism, and added experiments examining the performance and statistical significance of NoBa. We address each point below and would welcome any further questions or concerns regarding the revised theoretical analysis and empirical evidence.
W1 & Q1. Figure Consistency and Representation Measurements
In the unsupervised BN normal-sample panel of the original Figure 3, we plotted the batch mean μ instead of the mean normalized activation z̄. The panel did not match its label or the definition. We corrected this and redesigned Figure 3. The new figure shows z̄ computed from the normalized activations before the affine transformation. For both BN and NoBa, it reports the mean over all, abnormal, and normal samples. The former appendix Figure 7 has been merged into the revised Figure 3. The panels now agree with the definitions. For BN, the all-sample mean is zero by construction, and the variability appears in the normal and abnormal panels. For NoBa, the normal-sample mean is zero by construction, and the variability appears in the all-sample and abnormal panels.
W2. Mechanism of distribution collapse.
We agree that abnormal sample reuse remains constant throughout training and therefore cannot independently explain the progressive reduction in batch-to-batch variability. Accordingly, we have revised Section 3.2 to explain this phenomenon through the interaction between SAD objectives and conventional BN.
Distribution collapse can arise from SAD training itself and can be further amplified by anomaly oversampling. Let Δ=μabnorm−μnorm denote the normal–abnormal mean gap and q=Δ/σall its standardized counterpart under conventional BN. The batch-to-batch variability of normalized normal representations at epoch t is defined as
bt=VarB(t)(zˉNall)=π2VarB(t)(q)
where zˉNall is the mean normalized activation of normal samples and π is the abnormal ratio.
To characterize the attenuation of this variability, we introduce the saturation ratio s=π(1−π)q2, which measures the contribution of the normal–abnormal mean gap to the shared BN variance. Our new Proposition 1 establishes that, when Δ has a consistent sign across mini-batches,
bt≤1−ππ(1−St),
where St=EB(t)[s] denotes the average square-root saturation. As saturation increases, the upper bound on batch-to-batch variability decreases and converges to zero as St→1.
The asymmetric SAD objective encourages normal scores to concentrate around a common reference and abnormal scores to exceed a separation margin. These effects can reduce the within-group variance relative to the squared mean gap as representations evolve, thereby increasing saturation. Furthermore, increasing π up to 0.5 increases both the between-group contribution to the shared BN variance and the relative emphasis on abnormal separation in the SAD objective. Thus, anomaly oversampling can amplify saturation-induced variability attenuation even when abnormal sample reuse remains constant throughout training. See Section 3.2 and the Appendix B.1 for a detailed discussion of the distribution collapse mechanism and the proof of Proposition 1.