- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BaP introduces a defender-controlled probe into CLIP through a closed-form rank-one MLP edit. The probe detects adversarial activation shifts and selectively triggers a three-step input rectification procedure. Across 16 datasets, BaP improves average robust accuracy from 1.0% to 52.3%, while reducing clean accuracy from 63.6% to 61.1%.
Critical strengths
1、The class-agnostic probe offers an interesting extension of backdoor-inspired defenses to open-vocabulary models, with a simple closed-form construction. 2、 Under the reported evaluation, BaP achieves competitive robustness with substantially lower inference latency than R-TPT. Experiments cover multiple datasets and CLIP backbones.
Critical weaknesses
• Incomplete adaptive evaluation. Equation (16) attacks the edited classifier and suppresses detection, but does not explicitly optimize against the final prediction after rectification. Adaptive evaluation is limited to 10-step PGD at 1/255, and the reported “AutoAttack” uses only targeted APGD. Moreover, the existing adaptive attack already reduces ImageNet robust accuracy from 39.6% to 27.2%, making stronger evaluation consequential. • Insufficient evidence for the model edit’s independent contribution. Equations (8)–(9) show that the detection score can be computed exactly as a linear readout of the original hidden activations. Detection separation alone therefore does not establish the benefit of editing the encoder. Table 7 also shows that adding the escape stage increases robustness from 22.3% to 52.3%, but lacks escape-only and unedited-model controls.
Questions for authors
• Are the main attacks generated against the complete defense? How does BaP perform under stronger adaptive attacks targeting the final rectified prediction, with additional iterations and restarts? • How does full BaP compare with an unedited encoder using the equivalent external detector and identical rectification, and with escape-only rectification? • How are the threshold, clean-subspace dimension, and initialization ξ_0 selected? How are zero or very small singular values handled in Equation (7)? • Can the authors verify that all decoded adversarial examples satisfy the stated perturbation bounds?