- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an approach to detect poisons in ML training dataset efficiently under a limited verification budget. The detection algorithm combines influence functions with active search. It starts with the K examples with the highest influence scores, verifies whether they are poisoned, and then adaptively prioritizes neighbors in embedding space according to their expected poison impact (EPI). It also leverages a label-free influence estimation technique by enumerating pseudo-labels. The search process is further optimized using a nonmyopic strategy. Comprehensive experiments are conducted on multiple image classification datasets, across different poisoning attacks and DNN models.
Critical strengths
- The design of the algorithm is rational. Some techniques, such as label-free influence estimation and nonmyopic search, are clever.
- The paper includes extensive experiments and analysis.
- The detection performance is excellent.
- The writing is clear and accurate.
Critical weaknesses
- The contribution is incremental, as prior work has already explored influence-function-based approaches for poison detection.
- The experiments are conducted on traditional image classification tasks and DNN models.
- The verification assumes access to a reliable verification oracle, and computing influence scores can be expensive, so I'm not sure about the practicality and scalability of the approach.
Questions for authors
- Could the approach be applied to LLMs and corresponding datasets and tasks?
- Would the approach still work when a reliable verification oracle is unavailable?
4 comments
W2 / Q1
Our experiments focus on image classification with DNNs, and we do not claim results for LLMs. This is the standard setting in the data poisoning and cleaning literature: attacks (Feature Collision, Bullseye Polytope, Gradient Matching, Narcissus) and defenses (e.g., Activation Clustering, SPECTRE, Meta-Sift, EPIC, Confusion Training) are developed and evaluated on vision data, and we evaluate IGAS on the same datasets for direct comparison.
LLM poisoning is an important but distinct problem: the threat models (e.g., instruction fine-tuning attacks), data scale, and the meaning of verification (e.g., provenance of a text document vs. an image) differ and deserve a dedicated study. That said, IGAS itself is not image-specific: it needs only a per-sample influence score and an embedding space for the neighborhood model. Scalable influence approximations such as EK-FAC have been applied to LLMs (Grosse et al., 2023), including to the detection of poisons in instruction fine-tuning data (Li, 2026), and LLM embeddings could serve as the feature space. We will add a discussion of this future research direction to the paper.
Grosse, R., Bae, J., Anil, C., Elhage, N., Tamkin, A., Tajdini, A., Steiner, B., Li, D., Durmus, E., Perez, E., Hubinger, E., Lukošiūtė, K., Nguyen, K., Joseph, N., McCandlish, S., Kaplan, J., & Bowman, S. R. (2023). Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.03296.
Li, J. (2026). Detecting instruction fine-tuning attacks on language models using influence function. arXiv preprint arXiv:2504.09026.
W3 / Q2
Oracle Reliability: IGAS requires a verification oracle, but it does not require a perfectly reliable one. Appendix E.7 (Table 15) presents experimental results with an imperfect verification oracle, which returns the wrong poisoning status for a certain fraction of the queries. At an = 10% error rate, IGAS remains effective against most attacks, as the number of detected true poisons decreases slightly. Even at an = 30% error rate, IGAS remains effective against FC and BP, with only slight differences in the number of detections compared to a perfect oracle.
Cost of Computing Influence Scores: The cost of computing influence scores is very low compared to model training, and it scales linearly with both the number of classes and the number of samples. First, the class-wise pseudo-label impact vectors are computed once and cached, which takes about 10 seconds on CIFAR-10 and 200 seconds on Tiny ImageNet with 200 classes (Appendix E.5.3), and it scales linearly with the number of classes. Afterwards, scoring each sample requires one dot-product operation with its gradient, so the cost of scoring is linear in the number of samples (Section 4.2). On a single RTX A5000 GPU, computing influence for samples takes about 230 seconds, which is practically negligible compared to model training. The computational cost of running the active search is also low: 47–90 seconds. IGAS also runs on Tiny ImageNet () with the same budget (Appendix E.1), using approximate nearest neighbors (FAISS/HNSW) for the neighbor search instead of a complete distance matrix.
Dear reviewer, as the discussion nears its end, please let us know if you have any further questions or if anything in our rebuttal is unclear.
W1
Prior work has indeed explored influence-function-based poison detection; however, there are two fundamental differences. Poison Forensics as Active Search: First, our contribution is to formulate the forensic investigation of poisoned data as a sequential, budget-constrained search problem. Prior influence-based defenses (e.g., Koh & Liang, 2017; Hammoudeh & Lowd, 2022; Li et al., 2024) formulate automated cleaning or robust training as a one-shot decision, outputting a filtered dataset or model without considering verification feedback. In contrast, IGAS performs a nonmyopic active search, which selects samples sequentially, taking into account verification feedback from past steps as well as expected detections and feedback in future steps. Our results in Appendix E.6 (Table 14) show that automated-cleaning methods are ineffective in the forensic-investigation setting: if we investigate the B=250 samples that an automated-cleaning method deems the most likely to be poisoned, then we discover significantly fewer actual poisons than with IGAS (1.90–18.95 with automated-cleaning selection vs. 214.80–234.30 with IGAS). These results demonstrate that formulating poison forensics as a sequential search problem makes a significant, meaningful difference.
Label-Free Influence Scores: Second, prior influence-based defenses rely on ground-truth test-side information, such as labeled test data or an affected test point, to calculate influence scores. These methods typically require large sets of verified clean data, which may be hard to collect at scale. In contrast, IGAS requires no clean reference set, no labeled test data, and no known affected test point: we compute influence scores from class-wise pseudo-label impact vectors aggregated by mean absolute influence. Our proposed influence score is an approximation of the classical score (Koh & Liang, 2017): on poisoned samples, the Spearman correlation between the proposed and classical scores is 0.62–0.87 (Appendix E.5.2). Further, replacing the proposed influence score with the classical score in IGAS changes ASR only modestly: 0% vs. 0% on FC and BP, 5% vs. 0% on GM, and 6.20% vs. 1.31% on Narcissus (Appendix E.5.1). Therefore, the proposed influence scores can accurately identify impactful poisons without requiring any test-side information.
Koh, P. W., & Liang, P. (2017). Understanding black-box predictions via influence functions. 34th International Conference on Machine Learning (ICML), pp. 1885-1894.
Hammoudeh, Z., & Lowd, D. (2022). Identifying a training-set attack's target using renormalized influence estimation. 29th ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1367-1381.
Li, W., Li, J., Zeng, P., Schroeder de Witt, C., Prabhu, A., & Sanyal, A. (2024). Delta-influence: Unlearning poisons via influence functions. NeurIPS 2024 Workshop on Attributing Model Behavior at Scale.