- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper reassesses backdoor attacks on object detectors, focusing on region misclassification and object disappearance. It identifies evaluation limitations involving retained original-class detections, dataset-level mAP as a proxy for disappearance, and fixed-size triggers. Using true detection rate, instance-level attack success rate, and object-scaled triggers, the authors find that existing poisoning-only attacks do not consistently achieve the intended behavior across the tested architectures, even at high poisoning rates. They then consider a stronger outsourced-training threat model and propose BadDet+, which augments the detector loss with a penalty suppressing original-class scores on trigger-bearing objects. Experiments cover four detector families on COCO and MTSD, with physical-world evaluation on PTSD. Results show substantial reductions in original-class retention and improved disappearance attack success in many settings, with architecture-dependent trade-offs in clean performance. Supporting experiments examine the penalty, poisoning rate, trigger configurations, evaluation thresholds, and resistance to several detection and mitigation defenses.
Critical strengths
1.Useful diagnosis of shortcomings in backdoor evaluation. The paper demonstrates that high target-class ASR can coexist with substantial original-class retention, and that low poisoned-set mAP does not necessarily establish object disappearance. For example, BadDet achieves 99.45% ASR on COCO/FCOS while retaining original-class detections in 75.94% of cases. These observations are consequential because they motivate more informative evaluation protocols and help prevent misleading conclusions about both attack effectiveness and defense performance. 2.Controlled evidence supporting the proposed intervention. Appendix E isolates the contribution of the original-class suppression penalty while holding the remaining training setup fixed. On Faster R-CNN, enabling the penalty increases reported ODA-ASR from 4.5% to 89.2% and reduces RMA-TDR from 96.6% to 9.8%. This provides direct evidence that the intervention addresses the identified failure mode. Evaluation across multiple detector families, together with physical-world testing, further supports the practical relevance of the findings.
Critical weaknesses
1.The distinctive methodological contribution is not sufficiently isolated. BadDet+ adds a shifted softplus penalty to suppress original-class scores on selected trigger-bearing predictions, a functional form explicitly recognized in the paper. Appendix E establishes that adding this penalty is beneficial under controlled conditions. However, it does not distinguish the importance of the selected prediction set, increased supervision strength, and the particular penalty formulation, or compare with alternative suppression objectives applied to the same predictions under the same training conditions. This limits the evidence for a distinctive methodological advantage beyond strengthening original-class suppression. 2.The proposed evaluation protocol does not fully establish the claimed disappearance behavior. ODA success is defined by the absence of the original-class detection, which does not exclude residual detections under other classes. The protocol also does not clearly state whether objects already missed before trigger insertion are excluded from evaluation. These distinctions could affect the interpretation of the reported success rates. Since correcting misleading evaluation is a central contribution, the paper should demonstrate that its own protocol separates trigger-induced disappearance from misclassification and pre-existing detection failures.
Questions for authors
1.Evidence request—methodological contribution. Under identical outsourced-training conditions, how does BadDet+ compare with straightforward alternatives, such as reweighting relevant detection losses or applying an unshifted suppression penalty to the same predictions? Can you isolate whether the gains arise primarily from the selected prediction set, increased supervision strength, or the specific penalty formulation? Appendix E establishes the benefit of adding the penalty, but these comparisons would help establish its distinctive contribution. 2.Clarification and targeted evaluation request—ODA evaluation. Does the evaluation denominator exclude objects that the backdoored model already fails to detect before trigger insertion, using the same IoU and confidence thresholds? Since the stated success criterion only requires the absence of an original-class detection, what fraction of reported ODA successes retains a matching detection under another class? Please clarify the denominator and, where available, report success conditioned on correct pre-trigger detection and requiring the absence of any matching detection regardless of class. 3.Clarification—consistency of reported results. Table 2 reports an ODA-ASR of 83.68% for BadDet+ on MTSD/FCOS, whereas Appendix B.1 reports MTSD ODA-ASRs between 91.3% and 96.9% at a confidence threshold of 0.05. Were these results obtained using identical checkpoints, poisoning rates, and evaluation subsets? Please reconcile the values and specify the corresponding default thresholds.
7 comments
Comment 3.1: (Q1) Appendix E shows that adding the penalty helps but does not isolate the selected prediction set, supervision strength and penalty formulation, nor compare against straightforward alternatives such as loss reweighting or an unshifted suppression penalty on the same predictions under identical training conditions.
Response 3.1: We agree that the ablation in Appendix E (Table 14) shows only that adding the penalty helps, not which part of its design is responsible, or whether a simpler alternative would match it.
To isolate the three factors you raise (i.e., the selected prediction set, the supervision strength, and the penalty formulation), we ran an ablation on all four detectors under identical outsourced-training conditions, varying one design choice at a time. To test supervision strength directly, we removed the penalty. Instead, we scaled the weight of the detector's classification loss over all predictions by a factor , where recovers the no-penalty (poisoning-only) baseline. To test the penalty formulation, we fixed the proposed IoU-selected prediction set. We replaced the shifted softplus with an unshifted binary cross-entropy (BCE) term that drives the original-class probability to zero. We have added this ablation as a new Appendix E.1. The tables (Tables 3.1a and 3.1b) and a short discussion are provided below for convenience.
Table 3.1a. Ablation of the BadDet+ penalty on MTSD (random placement) for ODA. The first row is BadDet+. Cls. loss trains without the penalty and multiplies the weight of the detector's classification loss by . Clean baseline mAP: FCOS 58.5, Faster R-CNN 55.3, DINO 59.3, YOLOv5 60.9. Faster denotes Faster R-CNN.
| Pred. set | Penalty | FCOS mAP | FCOS ASR | Faster mAP | Faster ASR | DINO mAP | DINO ASR | YOLOv5 mAP | YOLOv5 ASR |
|---|---|---|---|---|---|---|---|---|---|
| IoU | Shifted softplus | 54.82 | 83.68 | 53.72 | 89.38 | 54.32 | 92.31 | 54.76 | 87.08 |
| IoU | BCE (unshifted) | 54.35 | 77.57 | 53.83 | 77.05 | 52.59 | 89.73 | 54.82 | 81.10 |
| All | Cls. loss | 50.90 | 58.7 | 48.6 | 4.5 | 51.70 | 90.50 | 51.5 | 51.8 |
| All | Cls. loss | 53.18 | 67.14 | 52.91 | 37.85 | 53.04 | 89.58 | 54.43 | 11.85 |
| All | Cls. loss | 54.22 | 75.86 | 53.47 | 52.52 | 54.59 | 89.14 | 54.19 | 8.68 |
| All | Cls. loss | 54.76 | 75.29 | 54.93 | 53.97 | 55.82 | 85.97 | 54.29 | 8.80 |
Table 3.1b. Ablation of the BadDet+ penalty on MTSD (random placement) for RMA. The first row is BadDet+. Cls. loss trains without the penalty and multiplies the weight of the detector's classification loss by . Clean baseline mAP: FCOS 58.5, Faster R-CNN 55.3, DINO 59.3, YOLOv5 60.9. Faster denotes Faster R-CNN.
| Pred. set | Penalty | FCOS mAP | FCOS ASR | FCOS TDR | Faster mAP | Faster ASR | Faster TDR | DINO mAP | DINO ASR | DINO TDR | YOLOv5 mAP | YOLOv5 ASR | YOLOv5 TDR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IoU | Shifted softplus | 55.86 | 93.13 | 16.96 | 53.46 | 97.04 | 9.13 | 53.35 | 90.43 | 5.39 | 57.23 | 87.04 | 14.00 |
| IoU | BCE (unshifted) | 56.82 | 92.64 | 23.22 | 53.99 | 95.30 | 18.95 | 54.68 | 88.92 | 8.13 | 57.08 | 84.30 | 18.01 |
| All | Cls. loss | 52.90 | 92.01 | 48.49 | 44.7 | 96.8 | 96.6 | 54.81 | 91.30 | 8.53 | 53.4 | 93.3 | 6.7 |
| All | Cls. loss | 55.04 | 93.84 | 39.69 | 52.28 | 91.06 | 50.81 | 54.85 | 90.63 | 8.04 | 57.25 | 85.41 | 17.41 |
| All | Cls. loss | 55.36 | 94.10 | 35.78 | 53.26 | 91.23 | 39.22 | 54.43 | 88.41 | 7.06 | 56.77 | 84.43 | 18.39 |
| All | Cls. loss | 54.53 | 95.34 | 39.51 | 54.93 | 91.40 | 37.78 | 54.09 | 87.55 | 9.54 | 55.41 | 84.56 | 16.04 |
The selected-prediction penalty is the dominant factor. Uniformly reweighting the classification loss helps where poisoning alone is weakest, but plateaus well short of the penalty: ODA-ASR peaks at 75.9% on FCOS and 54.0% on Faster R-CNN versus 83.7% and 89.4% for BadDet+, and RMA-TDR bottoms out at 35.8% and 37.8% versus 17.0% and 9.1%. On DINO, where poisoning is already effective, reweighting gives no benefit, and on YOLOv5 it is harmful (ODA-ASR falls from 51.8% to below 12%). Applying the penalty to the IoU-selected predictions closes this gap on every detector, so the gain comes from suppressing the selected predictions rather than from supervision strength alone. With the prediction set fixed, the shifted softplus gives higher ASR and lower RMA-TDR than unshifted BCE in all eight settings (e.g. Faster R-CNN RMA-TDR 9.1% vs. 19.0%), with clean-mAP differences of at most 1.7 points. In summary, the penalty on the selected predictions is the dominant factor, the shift adds a consistent further improvement, and reweighting, which requires the same threat model as BadDet+, does not close the gap on any detector. For clarity, the penalty-off rows of Table 14 (and the Cls. loss rows above) are BadDet at a 50% poisoning rate from the sweep in Appendix C, which is exactly BadDet+ with under the same data, initialisation and schedule; Appendix E now states this, and that the penalty-on rows of Table 14 use checkpoints trained independently of Table 2.
Comment 3.2(a): (Q2) The ODA success criterion only requires the absence of an original-class detection, so the fraction of reported successes that retain a detection under another class should be reported.
Response 3.2(a): You are correct. ODA-ASR counts an object as hidden when no original-class box overlaps it, so an overlapping box of another class would still count as a success. RMA evaluation has a similar issue, since TDR only checks whether the original class is detected, and ASR only checks whether the target class is detected.
To quantify how often this occurs, we added the other-class detection rate (OCD) to a new Appendix B.4 (reproduced below as Table 3.2a): the percentage of targeted objects with a prediction (IoU ) of a class other than the original (and, for RMA, the target), for all methods and detectors on MTSD, PTSD and COCO.
Table 3.2a. Other-class detection (OCD) under random trigger placement on MTSD and PTSD and centre placement on COCO: the percentage of cases in which a prediction overlaps the ground truth (IoU ) and has a class other than the original and, for RMA, the target. Bold marks the lowest value in each column. Faster denotes Faster R-CNN. A dash marks configurations not evaluated in Tables 1 and 2.
| Objective | Method | FCOS MTSD | FCOS PTSD | FCOS COCO | Faster MTSD | Faster PTSD | Faster COCO | DINO MTSD | DINO PTSD | DINO COCO | YOLOv5 MTSD | YOLOv5 PTSD |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ODA | BadDet+ | 0.52 | 0.53 | 3.31 | 0.59 | 0.80 | 0.13 | 0.87 | 1.03 | 0.18 | 2.80 | 3.86 |
| Morph | 2.42 | 8.30 | - | 1.90 | 1.48 | - | 1.03 | 0.27 | - | 4.28 | 0.68 | |
| Align | - | - | 0.72 | - | - | 0.75 | - | - | 0.41 | - | - | |
| Align Rnd | - | - | 0.24 | - | - | 0.25 | - | - | 0.29 | - | - | |
| UBA | 1.82 | 2.21 | 2.38 | 1.47 | 0.15 | 2.88 | 2.46 | 3.39 | 0.14 | 1.43 | 7.24 | |
| UBA Box | 2.06 | 1.98 | 2.02 | 1.74 | 0.19 | 1.34 | 0.63 | 0.00 | 0.13 | 1.35 | 7.01 | |
| RMA | BadDet+ | 0.26 | 0.46 | 3.27 | 1.11 | 4.91 | 0.41 | 0.00 | 0.00 | 0.17 | 0.21 | 1.79 |
| BadDet | 3.12 | 2.21 | 14.08 | 12.41 | 8.72 | 5.38 | 0.00 | 0.00 | 0.27 | 0.04 | 0.19 | |
| Morph | 10.35 | 12.98 | - | 9.28 | 6.93 | - | 0.00 | 0.00 | - | 0.26 | 0.08 |
Other-class boxes are rare: under ODA they accompany at most 1.03% of BadDet+ cases on FCOS, Faster R-CNN and DINO on MTSD and PTSD, 3.86% on YOLOv5, and at most 3.31% on COCO, versus up to 8.3% for the baselines. A class-agnostic success criterion could therefore lower the reported ODA-ASR of BadDet+ by at most these amounts, which does not change the ranking in Tables 1 and 2 or its interpretation. Under RMA, BadDet+ leaves a third class on at most 1.11% (MTSD), 4.91% (PTSD) and 3.27% (COCO) of cases, versus up to 14.08% for the baselines, which agrees with the lower TDR we report for BadDet+. The reported ODA-ASR of BadDet+ therefore reflects objects that are no longer detected, not objects reassigned to another class. Response 3.2(b) combines this with conditioning on detection without the trigger.
We thank the reviewer for their constructive and detailed comments, which have helped us to improve the paper. Below we respond to each comment in turn. Where several reviewers raised the same point, we repeat the full response in each thread so that each can be read independently. We are keen to engage in discussion as much as possible during the rebuttal period to ensure a positive outcome, and we are happy to address any residual concerns or provide further clarifications.