- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
In this paper, TraceGuard authors propose corpus-level filtering of poisoned image-caption pairs before training, instead of judging each image-caption pair alone. The authors hypothesize that the poisons push the model in a fixed direction, so on a corpus level they leave some traces (for example, inconsistent neighbors when looking at image and text domains). It scores examples based on a few such principles/features from frozen CLIP embeddings, and then decided which examples to drop and in what proportion. They test their method on 19 different attacks and the results show 98.4% average poison recall with only 5.4% clean removal. This has much better recall and lower false positive rate compared to the baselines
Critical strengths
- The corpus-level focus, instead of instance-level focus, makes perfect sense.
- The method seems pretty practical and cheap to implement, as everything comes from flows and clip embeddings and the statistics from that, and no training is needed.
- Good coverage on 19 different attacks with good gains on recall and a much lower false positive rate is a big plus.
- Authors also have some good ablations which show that the exact choice of the examples you drop matters. Just dropping random examples doesn't suffice in stopping the attack.
Critical weaknesses
- The thresholds for selection don't seem to be reported and possibly seem to be tuned to one particular encoder.
- The robustness of this method to adaptive attacks is convincing. It seems like it is easy to get around with just rewording the poison text
- The theory doent contribute much to the understanding and could be trimmed
- Most datasets tested have 100K examples or fewer, which is often less than real-world datasets.
Questions for authors
- How were the thresholds chosen, and for what attacks and encoder?
- Why is token swaps residual attack success 39.2% in Appendix F, but in Table 8 it's only 6.27%.
- How would the method do against even stronger adaptive attacks, such as LLM paraphrasing, that poison differently?