- Figure 2 provides correlational evidence that class frequency alone does not predict class-wise IoU, but it does not establish whether training-set imbalance causes or contributes to the observed robustness disparities. All results are obtained from models trained on the original imbalanced datasets, so the analysis cannot separate the effect of class frequency from other class-specific factors. A controlled ablation using class-balanced sampling or augmentation, with the same model, training budget, and evaluation set, would help determine whether reducing training imbalance changes the class-wise robust IoU gaps and frequency-normalized prediction bias.
- The authors take the class-wise adversarial robustness disparities in semantic segmentation assessment as the first contribution. However, previous works [1,2] have reported similar findings.
- Missing semantic-segmentation baselines. The evaluation compares RL-SAF with only one fairness method adapted from image classification, DAFA. It does not compare against existing adversarial training approaches for semantic segmentation, such as SegPGD-AT [3], or robust segmentation methods like PIR-AT [4], using class-wise robustness metrics. These methods are not specifically designed to improve fairness, but they would provide relevant task-specific baselines for assessing whether RL-SAF reduces inter-class robustness disparities beyond standard segmentation defenses.
- Limited validation of RL-SAF on datasets. RL-SAF is evaluated only on Cityscapes with DeepLabV3, so it remains unclear whether the method generalizes across datasets, segmentation architectures, or class distributions. Although the paper’s preliminary analysis includes PASCAL VOC 2012, it does not evaluate RL-SAF on that dataset. The authors should evaluate RL-SAF on additional benchmarks, such as PASCAL VOC 2012 [5], ADE20K [6], and BDD100K [7] or Mapillary Vistas [8].
- Substantial trade-offs in clean and overall robust performance. The paper reports a marked reduction in clean performance, and under stronger attacks, RL-SAF improves tail-class robustness at the cost of lower overall robust mIoU. The paper does not sufficiently characterize this trade-off or show how practitioners should choose an operating point between worst-class robustness, overall robustness, and clean accuracy.
- Low absolute robustness for the worst-performing classes. Under the SegPGD-3 training attack, RL-SAF increases the worst-class IoU only from approximately 0.18% to 0.58%, and the worst-five-class IoU from approximately 0.83% to 3.40%. While these results represent relative improvements, the absolute IoU values remain very low, suggesting that the worst-performing classes are still nearly unusable under attack.
- Limited validation of RL-SAF on architectures. Only DeepLabV3 is included in the final evaluation. The authors should include more architectures, such as PSPNet [9] and SegFormer [10].
[1].Mahima, KT Yasas, et al. "Exploring adversarial robustness of LiDAR semantic segmentation in autonomous driving." Sensors 23.23 (2023): 9579.
[2]. Halmosi, Levente, Bálint Mohos, and Márk Jelasity. "Evaluating the adversarial robustness of semantic segmentation: Trying harder pays off." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[3]. Gu, Jindong, et al. "Segpgd: An effective and efficient adversarial attack for evaluating and boosting segmentation robustness." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
[4]. Croce, Francesco, Naman D. Singh, and Matthias Hein. "Towards reliable evaluation and fast training of robust semantic segmentation models." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[5]. Everingham, Mark, et al. "The pascal visual object classes challenge 2012 (voc2012)." Results 2.5 (2012): 7.
[6]. Scene Parsing through ADE20K Dataset. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso and Antonio Torralba. Computer Vision and Pattern Recognition (CVPR), 2017.
[7]. Yu, Fisher, et al. "Bdd100k: A diverse driving dataset for heterogeneous multitask learning." 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2020.
[8]. Neuhold, Gerhard, et al. "The mapillary vistas dataset for semantic understanding of street scenes." Proceedings of the IEEE international conference on computer vision. 2017.
[9]. Zhao, Hengshuang, et al. "Pyramid scene parsing network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
[10]. Xie, Enze, et al. "SegFormer: Simple and efficient design for semantic segmentation with transformers." Advances in neural information processing systems 34 (2021): 12077-12090.
Response to Reviewer 78o7
Dear Reviewer 78o7,
We sincerely thank the reviewer for their thoughtful and constructive evaluation of our work. We particularly appreciate their recognition of our thorough analysis of adversarial fairness in semantic segmentation, the clear motivation of the problem, and our investigation into why existing classification-based fairness methods do not directly transfer to segmentation. We are also grateful for the positive assessment of the paper's soundness and its potential to motivate future research on robust and fair semantic segmentation.
W1, Q1, Q2: We report the mean and standard deviation over multiple random seeds in General Response R1. We further report evaluations under varying attack strengths in our response to Reviewer 2cqq (W6), showing the effectiveness of our approach in more attack scenarios, where the gains of our approach are larger.
We are happy to discuss further.
We also report the effect of RL-SAF on the five least- and five most-robust classes of SegPGD-AT (SegPGD-10, ε = 4/255) in Figure 12 of our revision:
Q3: The trade-off comes from reallocating a fixed training signal. RL-SAF uses a class-balanced loss: it upweights pixels from vulnerable classes and downweights the abundant, already-robust classes (road, building, vegetation, sky). The per-class breakdown above shows the result: all five least-robust classes gain, while the most-robust classes give up a small amount. We further mitigate this trade-off through improved batch-norm settings (see General Response R2).
We are happy to discuss further.