- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a method to perform anomaly detection. The author first investigated why the current methods fall short of addressing the issues by introducing the trade-off between performance and fairness. They found that two possible explanations can be vanishing protected influence and dimensional collapse. Motivated by this finding, they propose FADIG, a fully self-supervised, fairness awareness training method. It has two components, (1) FAC encourages representation from different sensitive groups to align (2) Adaptive rebalanced AE reweighs the training dynamics.
Critical strengths
-
The paper investigates the possible reason causing poor performance in protected group is interesting. The current method using fair regularizer can encounter a trade-off between the loss predictive information and fairness metrics. The paper using singular value spectrum of their missing part is interesting.
-
The learning objective design is sound, at least it potentially addresses the two issues pointed out by the authors.
-
The experiment’s performance is strong.
Critical weaknesses
-
The author explains that the possibility that causing poor performance of protected group by minority groups influences diluted representation dimensional collapse. Per my experience, the alternative way is that minority group is overfit during training.
-
Proof of Lemma3 has a flaw, where , , it can not deduct . Because we can not say .
-
I would like to see the TSNE on the learned latent and compare with other methods like many fair representation learning work did.
Questions for authors
-
Eq (3) and Eq (4) are actually reweighing on the two groups, I would like to see why the current method to compute is better than GroupDRO (https://arxiv.org/pdf/1911.08731).
-
The authors argues that enforces is not a good way, the existing literature I know share the same idea is FADES (https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10655684) which I would like to compare which method is more promising. It can think of as the anomaly label in the paper.
3 comments
Rebuttal by Author (2/2)
Weakness 4: I would like to see the TSNE on the learned latent and compare with other methods like many fair representation learning work did.
We visualize our method and fairness-aware baseline methods in Figure 7 (in Appendix E.2) in the updated PDF file. The visualization shows that our proposed FADIG exhibits anomaly-rich peripheral regions in both groups, although some normal–anomaly overlap remains, suggesting that it preserves group-specific structure alongside information relevant to anomaly discrimination.
Question 1: Eq (3) and Eq (4) are actually reweighing on the two groups, I would like to see why the current method to compute is better than GroupDRO.
Thank you for highlighting this connection. Unlike GroupDRO, which emphasizes groups with larger absolute losses, FADIG accounts for the fact that high reconstruction error in unlabeled data may indicate anomalies rather than under-fitted normal patterns. This distinction matters because our unlabeled training data contain anomalies and high reconstruction error may indicate an anomaly rather than an under-fitted normal pattern, so reducing worst-group reconstruction loss may not necessarily improve detection. Lemma 4.3 (Lemma 4.4 in the updated PDF) motivates our weighting through this fitting trade-off. We have added the discussion of GroupDRO in the related work (Line 484 highlighted in red) in the updated version. For a fair comparison, we adapt GroupDRO to the unsupervised setting by replacing the prediction loss with reconstruction loss and maintain the same network architecture. We compare reconstruction-based GroupDRO with FADIG under identical experimental settings. The detailed experimental setup can be found in Appendix E.1. All values are percentages, averaged across runs.
| Dataset (K) | Method | AUROC ↑ | Recall@K ↑ | Recall Diff ↓ | AUPRC ↑ | AUPRC Diff ↓ |
|---|---|---|---|---|---|---|
| MNIST-USPS (1200) | GroupDRO | 82.43 ± 0.55 | 54.21 ± 0.83 | 36.32 ± 2.84 | 54.09 ± 0.68 | 4.59 ± 2.51 |
| FADIG | 91.28 ± 0.46 | 67.19 ± 0.33 | 3.77 ± 2.18 | 66.47 ± 0.33 | 2.52 ± 1.09 | |
| MNIST-Invert (500) | GroupDRO | 89.36 ± 0.27 | 54.00 ± 0.32 | 58.66 ± 0.35 | 52.22 ± 1.60 | 16.92 ± 0.46 |
| FADIG | 97.99 ± 0.07 | 71.82 ± 0.63 | 9.78 ± 3.10 | 72.01 ± 0.20 | 1.72 ± 0.59 | |
| COMPAS (350) | GroupDRO | 56.98 ± 3.07 | 23.72 ± 2.24 | 12.68 ± 8.48 | 23.12 ± 1.08 | 9.05 ± 0.23 |
| FADIG | 61.45 ± 0.47 | 34.38 ± 0.36 | 5.97 ± 4.34 | 27.42 ± 1.56 | 4.79 ± 3.33 | |
| CelebA (5000) | GroupDRO | 50.45 ± 0.75 | 7.93 ± 0.91 | 3.88 ± 3.68 | 9.46 ± 0.34 | 5.73 ± 0.38 |
| FADIG | 59.43 ± 0.42 | 11.96 ± 0.49 | 4.72 ± 1.26 | 12.21 ± 0.20 | 1.41 ± 0.70 |
FADIG achieves higher AUROC, recall, and AUPRC on all four datasets, with lower AUPRC disparity throughout and lower recall disparity on three. On CelebA, GroupDRO achieves slightly smaller recall difference at the cost of near-chance AUROC. These results support FADIG’s overall effectiveness.
Question 2: The authors argues that enforces is not a good way, the existing literature I know share the same idea is FADES which I would like to compare which method is more promising. It can think of as the anomaly label in the paper.
Thank you for highlighting FADES. We share its motivation that indiscriminately removing sensitive information can discard useful predictive structure. However, FADES uses target labels and sensitive attributes during training, including in its conditional-mutual-information objective. Our setting provides sensitive attributes but no anomaly labels. Therefore, treating its target variable as the anomaly label would introduce additional supervision, making the original method incompatible with our evaluation protocol. Despite of this, we have discussed its revelance to our paper in the related work (highlighted in red in Section 6 Line 482) in the updated version.
Reply to Reviewer
Dear Reviewer VPfn,
Thank you again for your valuable feedback. As the rebuttal period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses. Please let us know if you have any remaining concerns or questions. We would be happy to provide further clarification.
Thank you for your time and consideration!
The Authors
Rebuttal by Author (1/2)
We thank the reviewer for recognizing the interesting singular-value spectrum analysis, sound learning objective design, and strong empirical performance of our method.
Weakness 1: The author explains that the possibility that causing poor performance of protected group by minority groups influences diluted representation dimensional collapse. Per my experience, the alternative way is that minority group is overfit during training.
Weakness 2: FairAD performs significantly worse than the results reported in the original FairAD paper.
Weakness 3: Proof of Lemma 3.3 has a flaw
{\mathbb E[\|\nabla_\theta R_U\|]} \leq \frac{k}{c}\sqrt{\frac{n}{m}}.$$ Let $g_i=\nabla_\theta\ell_i(\theta)$. Under Assumption 3.1, the per-sample gradients are independent, zero-mean, and bounded by $\|g_i\|\leq k$. By Cauchy–Schwarz and the vanishing cross terms, $$\begin{aligned} \mathbb E\left[\left\|\nabla_\theta R_P\right\|\right] &=\mathbb E\left[\left\|\sum_{i=1}^{n}g_i\right\|\right]\\ &\leq \sqrt{\mathbb E\left[\left\|\sum_{i=1}^{n}g_i\right\|^2\right]}\\ &=\sqrt{\sum_{i=1}^{n}\mathbb E[\|g_i\|^2]} \leq k\sqrt n. \end{aligned}$$ Combined with Assumption 3.2, $\mathbb E[\|\nabla_\theta R_U\|]\geq c\sqrt m$, which directly establishes the corrected bound. **Importantly, this correction preserves the central theoretical observation.** Under the stated assumptions, the protected group's expected aggregate gradient magnitude becomes relatively smaller as group imbalance increases, following the same $O(\sqrt{n/m})$ scaling. The distinction is that the revised lemma characterizes the ratio of expected gradient magnitudes rather than the expectation of their ratio and this result remains consistent with our motivation for mitigating group-size-driven optimization imbalance. We appreciate the reviewer's feedback and have updated the lemma and proof accordingly. The changes are highlighted in red. **These revisions clarify the statistical interpretation while retaining the imbalance-dependent scaling result motivating FADIG.**