- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes BlindClean, which leverages a key characteristic of poisoned samples: under input perturbations, poisoned samples tend to exhibit greater feature stability than clean samples. Accordingly, BlindClean sanitizes a dataset using only the potentially poisoned dataset itself. It evaluates cross-layer feature stability under multi-scale input perturbations, estimates reference feature distributions for clean samples from low-stability samples within each class, and then measures the feature deviations of high-stability samples from these reference distributions. This design enables BlindClean to first identify the class most likely to be attacked and subsequently detect the poisoned samples within that class.
Critical strengths
None
Critical weaknesses
“When the sanitized data is used for downstream training, BlindClean reduces the average attack success rate to below 1% and improves clean accuracy by up to 44 percentage points over state-of-the-art methods.” The experimental results are hardly convincing. The principle commonly followed by all backdoor attacks is to achieve the attack goal without affecting normal functionality. In the experiments in this paper, the authors claim that normal functionality on the sanitized dataset improves by 44 percentage points, which directly contradicts common knowledge in the field. If this conclusion were truly valid, the evaluated attack methods could not constitute backdoor attacks (they would lose 44% of normal functionality).
The statement “This paper proposes BlindClean, which leverages a key characteristic of poisoned samples: under input perturbations, poisoned samples tend to exhibit higher feature stability than clean samples.” has a major flaw. For example, the quantization attack BPP uses quantized features as triggers (e.g., pixel values are all even), and is far more sensitive to input perturbations than normal features, which directly refutes this assumption.
Questions for authors
1.The experimental results are not sufficiently convincing. A principle commonly followed by backdoor attacks is to achieve the attack objective without impairing normal functionality. In the experiments reported in this paper, however, the authors claim that normal functionality on the sanitized dataset improves by 44 percentage points, which appears to contradict established understanding in the field. If this conclusion were valid, the evaluated attack methods could not reasonably be regarded as backdoor attacks, since they would have lost 44% of normal functionality.
2.The statement “This paper proposes BlindClean, which leverages a key characteristic of poisoned samples: under input perturbations, poisoned samples tend to exhibit higher feature stability than clean samples.” is problematic. For instance, the quantization attack BPP employs quantized features as triggers (e.g., pixel values are all even) and is far more sensitive to input perturbations than normal features, which directly contradicts this assumption.
3.The authors consider only three attacks, all from before 2023. They use two datasets, CIFAR-10 and SVHN. SVHN is an uncommon dataset and is rarely used in the backdoor field, while the authors do not cover mainstream datasets (CIFAR-100, Tiny-ImageNet, ImageNet).
4.There are no detection tests for poison-label or multi-target scenarios.
5.There are also no experiments on detection thresholds or efficiency overhead.