acceptodds
Under review as a conference paper at ICLR 2027

Influence-Guided Active Search for Poisoned Training Data Forensics

Abstract

Data poisoning attacks that inject malicious samples into training data pose a serious threat to the reliability of machine learning. Existing defenses focus on automated detection and removal, but effective cleaning inevitably discards significant amounts of benign data. We instead consider forensic investigations of poisoned data, which verify samples through manual inspection or provenance checks, such as re-acquiring a sample from its original source or auditing its origin. A key challenge of forensic investigations is that verification is expensive and can be performed only per sample, but investigation budgets are limited. Therefore, an investigation must strategically select the samples that will be verified and potentially removed to minimize the impact of the remaining poisons. We frame this as a nonmyopic sequential search problem and introduce an influence-guided active search (IGAS) framework that integrates: (i) a label-free influence score identifying samples with disproportionate impact on predictions and (ii) an adaptive search strategy propagating information from verified samples to focus on regions of the data that are both influential and likely to be poisoned. Experiments on CIFAR-10, Tiny ImageNet, and GTSRB across eight triggerless and backdoor attacks demonstrate that IGAS substantially reduces Attack Success Rate while preserving test accuracy, outperforming automated data-cleaning defenses that often remove benign samples along with poisoned ones.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Data poisoning, Poisoning forensics, Active search, Adversarial machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MuGsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an approach to detect poisons in ML training dataset efficiently under a limited verification budget. The detection algorithm combines influence functions with active search. It starts with the K examples with the highest influence scores, verifies whether they are poisoned, and then adaptively prioritizes neighbors in embedding space according to their expected poison impact (EPI). It also leverages a label-free influence estimation technique by enumerating pseudo-labels. The search process is further optimized using a nonmyopic strategy. Comprehensive experiments are conducted on multiple image classification datasets, across different poisoning attacks and DNN models.

Critical strengths
  • The design of the algorithm is rational. Some techniques, such as label-free influence estimation and nonmyopic search, are clever.
  • The paper includes extensive experiments and analysis.
  • The detection performance is excellent.
  • The writing is clear and accurate.
Critical weaknesses
  • The contribution is incremental, as prior work has already explored influence-function-based approaches for poison detection.
  • The experiments are conducted on traditional image classification tasks and DNN models.
  • The verification assumes access to a reliable verification oracle, and computing influence scores can be expensive, so I'm not sure about the practicality and scalability of the approach.
Questions for authors
  • Could the approach be applied to LLMs and corresponding datasets and tasks?
  • Would the approach still work when a reliable verification oracle is unavailable?
4 comments

Prior work has indeed explored influence-function-based poison detection; however, there are two fundamental differences. Poison Forensics as Active Search: First, our contribution is to formulate the forensic investigation of poisoned data as a sequential, budget-constrained search problem. Prior influence-based defenses (e.g., Koh & Liang, 2017; Hammoudeh & Lowd, 2022; Li et al., 2024) formulate automated cleaning or robust training as a one-shot decision, outputting a filtered dataset or model without considering verification feedback. In contrast, IGAS performs a nonmyopic active search, which selects samples sequentially, taking into account verification feedback from past steps as well as expected detections and feedback in future steps. Our results in Appendix E.6 (Table 14) show that automated-cleaning methods are ineffective in the forensic-investigation setting: if we investigate the B=250B=250 samples that an automated-cleaning method deems the most likely to be poisoned, then we discover significantly fewer actual poisons than with IGAS (1.90–18.95 with automated-cleaning selection vs. 214.80–234.30 with IGAS). These results demonstrate that formulating poison forensics as a sequential search problem makes a significant, meaningful difference.

Label-Free Influence Scores: Second, prior influence-based defenses rely on ground-truth test-side information, such as labeled test data or an affected test point, to calculate influence scores. These methods typically require large sets of verified clean data, which may be hard to collect at scale. In contrast, IGAS requires no clean reference set, no labeled test data, and no known affected test point: we compute influence scores from class-wise pseudo-label impact vectors aggregated by mean absolute influence. Our proposed influence score is an approximation of the classical score (Koh & Liang, 2017): on poisoned samples, the Spearman correlation between the proposed and classical scores is 0.62–0.87 (Appendix E.5.2). Further, replacing the proposed influence score with the classical score in IGAS changes ASR only modestly: 0% vs. 0% on FC and BP, 5% vs. 0% on GM, and 6.20% vs. 1.31% on Narcissus (Appendix E.5.1). Therefore, the proposed influence scores can accurately identify impactful poisons without requiring any test-side information.

Koh, P. W., & Liang, P. (2017). Understanding black-box predictions via influence functions. 34th International Conference on Machine Learning (ICML), pp. 1885-1894.

Hammoudeh, Z., & Lowd, D. (2022). Identifying a training-set attack's target using renormalized influence estimation. 29th ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1367-1381.

Li, W., Li, J., Zeng, P., Schroeder de Witt, C., Prabhu, A., & Sanyal, A. (2024). Delta-influence: Unlearning poisons via influence functions. NeurIPS 2024 Workshop on Attributing Model Behavior at Scale.

Our experiments focus on image classification with DNNs, and we do not claim results for LLMs. This is the standard setting in the data poisoning and cleaning literature: attacks (Feature Collision, Bullseye Polytope, Gradient Matching, Narcissus) and defenses (e.g., Activation Clustering, SPECTRE, Meta-Sift, EPIC, Confusion Training) are developed and evaluated on vision data, and we evaluate IGAS on the same datasets for direct comparison.

LLM poisoning is an important but distinct problem: the threat models (e.g., instruction fine-tuning attacks), data scale, and the meaning of verification (e.g., provenance of a text document vs. an image) differ and deserve a dedicated study. That said, IGAS itself is not image-specific: it needs only a per-sample influence score and an embedding space for the neighborhood model. Scalable influence approximations such as EK-FAC have been applied to LLMs (Grosse et al., 2023), including to the detection of poisons in instruction fine-tuning data (Li, 2026), and LLM embeddings could serve as the feature space. We will add a discussion of this future research direction to the paper.

Grosse, R., Bae, J., Anil, C., Elhage, N., Tamkin, A., Tajdini, A., Steiner, B., Li, D., Durmus, E., Perez, E., Hubinger, E., Lukošiūtė, K., Nguyen, K., Joseph, N., McCandlish, S., Kaplan, J., & Bowman, S. R. (2023). Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.03296.

Li, J. (2026). Detecting instruction fine-tuning attacks on language models using influence function. arXiv preprint arXiv:2504.09026.

Oracle Reliability: IGAS requires a verification oracle, but it does not require a perfectly reliable one. Appendix E.7 (Table 15) presents experimental results with an imperfect verification oracle, which returns the wrong poisoning status for a certain fraction η\eta of the queries. At an η\eta = 10% error rate, IGAS remains effective against most attacks, as the number of detected true poisons decreases slightly. Even at an η\eta = 30% error rate, IGAS remains effective against FC and BP, with only slight differences in the number of detections compared to a perfect oracle.

Cost of Computing Influence Scores: The cost of computing influence scores is very low compared to model training, and it scales linearly with both the number of classes and the number of samples. First, the class-wise pseudo-label impact vectors are computed once and cached, which takes about 10 seconds on CIFAR-10 and 200 seconds on Tiny ImageNet with 200 classes (Appendix E.5.3), and it scales linearly with the number of classes. Afterwards, scoring each sample requires one dot-product operation with its gradient, so the cost of scoring is linear in the number of samples NN (Section 4.2). On a single RTX A5000 GPU, computing influence for N=50,000N=50{,}000 samples takes about 230 seconds, which is practically negligible compared to model training. The computational cost of running the active search is also low: 47–90 seconds. IGAS also runs on Tiny ImageNet (N=100,000N=100{,}000) with the same budget B=250B=250 (Appendix E.1), using approximate nearest neighbors (FAISS/HNSW) for the neighbor search instead of a complete distance matrix.

Dear reviewer, as the discussion nears its end, please let us know if you have any further questions or if anything in our rebuttal is unclear.

Reviewer_hmu3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article introduces Influence-Guided Active Search (IGAS), an active search framework specifically designed for detecting poisoning attacks on training data. Its aim is to prioritize the identification and removal of poisoning samples that have a significant impact on the model, within a limited sample verification budget. Specifically, the authors have developed a class-level influence estimation method that does not require labeled test data, and combined it with poisoning probability estimation based on neighbors and a non-greedy sequence search strategy to dynamically select the training samples that are most worthy of verification. Experiments were conducted on three datasets - CIFAR-10, Tiny ImageNet, and GTSRB - to evaluate various non-triggering poisoning and backdoor attacks. The results show that IGAS can significantly reduce the attack success rate while maintaining a high normal test accuracy under a lower verification budget. Compared to existing automated data cleaning methods, it demonstrates better security and model performance trade-off.

Critical strengths
  1. The article focuses on the issue of training data poisoning forensics under limited validation budget, considering the situation where directly deleting suspicious samples may cause loss of normal data. This provides another research perspective for poisoning defense.

  2. The author combines unlabeled influence estimation, neighbor poisoning probability propagation, and non-greedy active search, enabling the sample validation process to simultaneously consider the potential poisoning probability and its impact on the model, rather than merely pursuing the number of detections.

  3. The article was evaluated under three datasets and various poisoning attacks, and provided ablation experiments and budget matching comparisons. The results show that this method can effectively reduce the attack success rate under a lower validation budget while maintaining a relatively good normal test accuracy.

Critical weaknesses
  1. My main concern is the lack of innovation in the method. The core framework of the article mainly combines the existing Influence Function, KNN probability estimation, and Nonmyopic Active Search. The technical contribution mainly lies in the integration and application of existing methods, lacking sufficient prominent algorithmic innovation. Especially in the non-greedy search part, it directly borrows from ENS.

  2. Excessive reliance on idealized sample verification mechanisms. The article assumes that the Oracle can accurately determine whether a single training sample has been poisoned. However, in real scenarios, reliable original data or reliable source records are often difficult to obtain. Although the author considered Oracle noise, there is still a lack of verification of the real forensic process.

  3. Additionally, the Baseline comparison is not sufficient, making it difficult to accurately measure the advantages of the method. The article mainly compares with automated data cleaning methods, but these methods are not designed for a limited validation budget. Although the Appendix supplements the budget-matching experiments, there is a lack of systematic comparison with more targeted active search and query selection strategies. Therefore, it cannot fully prove the actual advantages of the proposed framework over existing methods.

Questions for authors
  1. IGAS mainly combines the existing Influence Function, KNN probability estimation, and ENS non-greedy search strategy. Could you further clarify what the core technological innovation of IGAS is compared to directly using ENS or other existing active search methods? Additionally, how significant is the independent performance improvement brought about by these changes?

  2. The article assumes that the samples can be verified for poisoning through data sources or reliable reference information. However, when the original data is unavailable or the data provider is attacked, how does the author achieve reliable verification? Could you provide more specific practical application scenarios or evidence to support this key assumption?

  3. The existing Baseline mainly consists of automated data cleaning methods rather than active search algorithms specifically designed for a limited validation budget. Could you provide a direct comparison with representative Active Search methods under the same Oracle query budget to prove that the advantage of IGAS does not mainly come from additional validation feedback?

4 comments

IGAS builds on some existing components, but making active search work for poison forensics on deep models required four significant and novel contributions.

  1. Search objective. ENS (Jiang et al., 2017) simply maximizes the number of positive instances found, so in the poison forensics setting, it would not be able to consider how harmful each poisoned sample might be. In contrast, IGAS maximizes the total influence of the poisoned samples found, replacing ENS’s expected-count utility with expected poison impact, which enables it to focus the investigation on the most harmful poisons.
  2. Search space. ENS works on lower-dimensional data (20 dimensions in the experiments of Jiang et al., 2017). To enable forensic investigations of high-dimensional image data, IGAS defines neighbors in a model’s learned latent space and propagates poison probabilities from verified samples to neighbors in this space.
  3. Search algorithm. IGAS includes several significant extensions: the lookahead and TopSum terms are weighted by per-sample influence (Equation (8)); the optimistic pruning bounds are adapted to the impact-weighted utility (probability increase ≤1/K\leq 1/K); and the search is seeded with the most influential samples since, at 1% poisoning, a random batch of 10 seeds has zero poisons with probability 90.4%.
  4. Label-free influence. Classical influence functions (Koh & Liang, 2017) need ground-truth labeled test data, which is often difficult or impossible to collect at scale. To address this challenge, we propose a novel label-free influence score, which uses class-wise pseudo-label impact vectors with mean absolute aggregation. Our proposed label-free influence score is an approximation of classical influence functions: on poisoned samples, the Spearman correlation between the proposed and classical influence scores is 0.62–0.87 (Appendix E.5.2), and they lead to roughly the same outcomes in terms of ASR (0% vs. 0% on FC and BP, 5% vs. 0% on GM, and 6.20% vs. 1.31% on Narcissus in Appendix E.5.1).

Ablation Experiments: To demonstrate the independent performance improvement due to each of these innovations, we present ablation results in Appendix E.4. First, the count-maximizing (ENS-style) variant keeps IGAS’s lookahead and probability model but ignores influence scores in utility and seeding. The ASR of count-maximizing search is at least as high as that of IGAS (FC 0% vs. 0%, BP 5% vs. 0%, GM 10% vs. 5%, Narcissus 8.43% vs. 6.20%), and it finds fewer poisons on GM and Narcissus (178.0 vs. 214.8; 153.8 vs. 194.0; see Table 11), which demonstrates that our label-free influence significantly improves outcomes. Second, selecting samples for investigation based on influence-only ranking (i.e., without active search) finds fewer poisons than IGAS on every attack, with the largest gap on Narcissus (40.4 vs. 194.0; ASR 38.47% vs. 6.20%; see Table 11). Third, adding feedback-driven probability estimation through influence-guided myopic search increases the number of poisons detected on all four attacks and reduces Narcissus ASR to 16.04%. Finally, adding nonmyopic lookahead further reduces ASR on GM (10% to 5%) and Narcissus (16.04% to 6.20%), while maintaining 0% ASR on FC and BP. These results show that each innovation contributes to the success of IGAS.

Jiang, S., Malkomes, G., Converse, G., Shofner, A., Moseley, B., & Garnett, R. (2017). Efficient nonmyopic active search. 34th International Conference on Machine Learning (ICML), pp. 1714-1723.

Koh, P. W., & Liang, P. (2017). Understanding black-box predictions via influence functions. 34th International Conference on Machine Learning (ICML), pp. 1885-1894.

Verification is costly and pipeline-dependent, so IGAS is built for a limited oracle budget BB. Examples of verification include (i) provenance checks, e.g., re-fetching a web-scraped sample and comparing its hash with the curator’s record (Carlini et al., 2024), which catches post-snapshot tampering but not poisons present at collection; (ii) requesting the original from the provider or auditing its origin (Section 3.2); and (iii) where applicable, manual inspection. If the original is unavailable or the provider is compromised, only error-prone checks remain. Such checks may incorrectly flag a benign sample or miss a poisoned one, affecting both removal decisions and subsequent search updates; we model this scenario as a noisy oracle (see below). Finally, we would like to emphasize that the purpose of IGAS is to guide forensic investigations of poisoned data: if there is no verification, then the scenario is not a forensic investigation and, hence, it is out of scope.

To assess the impact of verification errors that may arise when the original data is unavailable or a provider is compromised, we evaluate IGAS under noisy-oracle feedback. In the noisy-oracle experiments in Appendix E.7 (Table 15), the oracle returns the wrong poisoning status for a certain fraction η\eta of the queries. We revised the paper and extended Table 15 with new results, covering very unreliable oracles (η\eta = 20% and 30%) and Narcissus attacks. At an η\eta = 10% error rate, IGAS remains effective against most attacks, as the number of detected true poisons decreases slightly (by 5.7% on FC, by 3.2% on BP, and by 1.2% on GM). Even at an η\eta = 30% error rate, IGAS remains very effective against FC and BP (number of detections reduced by only 11% and 7%, respectively, compared to a perfect oracle). Investigations of GM and Narcissus are more sensitive, but the queried samples are still poisoned at a rate well above chance. Even at this extreme level of noise, the search still performs substantially better than simple selection based on influence only (82.6 vs. 40.4 on Narcissus; Appendix E.4).

Table 15 (extended). Number of poisons found with a noisy oracle.

Noise RateFCBPGMNarcissus
Noise-free234.3234.3214.8194.0
10%221.0226.8212.2105.4
20%219.0224.4133.683.0
30%208.8218.4113.482.6

Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2024). Poisoning web-scale training datasets is practical. 45th IEEE Symposium on Security and Privacy (S&P), pp. 407-425.

Appendix E.4 (Table 10) compares IGAS with three active-search baselines: myopic search, two-step search, and ENS-style nonmyopic count-maximizing search (Jiang et al., 2017). We run each search with the same budget (B=250B=250) and feedback (true/false poisoning status), so methods differ only in which samples they decide to query.

Of all these active-search baselines, count-maximizing ENS-style nonmyopic search performs best. However, as we discussed above (see W1), IGAS clearly performs better than count-maximizing nonmyopic search in terms of ASR (BP 0% vs. 5%, GM 5% vs. 10%, Narcissus 6.20% vs. 8.43%). Further, the total influence of the poisons found by count-maximizing search is significantly lower than that of the poisons found by IGAS: 0.4815 vs. 0.7755 (FC), 0.4666 vs. 0.6382 (BP), and 0.9337 vs. 0.9969 (GM). Overall, IGAS performs better than all of the baseline active-search methods.

Thank you for the detailed response. While the additional experiments address some of my concerns, my main reservations regarding the limited methodological novelty and the practical feasibility of the oracle assumption remain. Therefore, I will maintain my original rating.

Reviewer_h6SHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Influence-Guided Active Search (IGAS) for data poisoning defense under a limited verification budget. It combines label-free influence estimation, neighborhood-based poison probability estimation, and nonmyopic active search to prioritize potentially harmful poisoned samples. Experiments across multiple datasets and attacks show reduced attack success rates while preserving clean accuracy.

Critical strengths
  1. The paper reformulates traditional data poisoning defense as a budget-constrained sequential decision-making problem, providing a meaningful new perspective on the problem.

  2. The proposed framework is well-structured, with each component closely aligned with the problem formulation. In particular, the label-free influence estimation is a valuable technical contribution.

Critical weaknesses
  1. The neighborhood-based poison probability estimation assumes that nearby samples in the learned feature space tend to share similar poisoning status. However, the paper provides insufficient justification or empirical evidence for this assumption, raising concerns about the reliability of the probability estimates.

  2. The paper does not sufficiently demonstrate the effectiveness of impact-aware search, specifically they do not clearly show that this strategy is more effective than simply maximizing the number of detected poisoned samples.

  3. The paper does not compare IGAS with simple influence-based ranking, making it unclear how much the probability estimation and nonmyopic search components contribute to the final performance.

Questions for authors
  1. Can the authors provide empirical evidence that nearby samples in the learned feature space tend to share similar poisoning status across different attacks?

  2. Can the authors provide evidence that the proposed influence score reflects the actual harmful impact of poisoned samples, and that prioritizing high-impact poisons leads to better defense performance than maximizing the number of detected poisons?

  3. Can the authors compare IGAS with a simple influence-only ranking baseline to clarify the additional contributions of probability estimation and nonmyopic search?

3 comments

To justify the neighborhood-based estimation of poison probability, we provide empirical evidence across four attacks and two datasets, showing that samples tend to have the same poisoning status as their K=10K = 10 nearest neighbors. For each sample, we take its 10 nearest neighbors in the learned feature space according to the metric used by IGAS, count how many of its neighbors are poisoned, and report the average number of poisoned neighbors for poisoned and clean samples.

Table 16 (new). Average number of poisoned samples among the 10 nearest neighbors of poisoned and clean samples.

DatasetAttackAround a poisoned sampleAround a clean sample
CIFAR-10Feature Collision7.43500.0230
CIFAR-10Bullseye Polytope9.83200.0030
CIFAR-10Gradient Matching8.08400.0190
CIFAR-10Narcissus5.42000.0330
Tiny ImageNetBullseye Polytope4.16000.0040
Tiny ImageNetNarcissus5.81000.0240

On CIFAR-10, a poisoned sample has 5.42 to 9.83 poisons among its 10 nearest neighbors on average, whereas a clean sample has at most 0.033. On Tiny ImageNet, a poisoned sample has 4.16 to 5.81 poisons among its neighbors, whereas a clean sample has at most 0.024. These results demonstrate that the neighborhood of a poisoned sample tends to be mostly poisoned, and, conversely, the neighborhood of a clean sample tends to be mostly clean, which justifies the neighborhood-based probability estimation. This is likely because poisons are optimized toward a common goal (a shared target feature, polytope, gradient direction, or trigger), so they occupy a common region of the representation space. We added these new results to the paper as Appendix E.8.

Does the score reflect actual harm? IGAS’s influence score is a label-free adaptation of classical influence functions (Koh & Liang, 2017), which quantify a training sample’s impact on the trained model’s loss. The classical influence score is a principled measure of harm since it directly quantifies the increase in misclassifications due to the inclusion of a poisoned sample. The proposed influence score is an approximation of the classical score: on poisoned samples, the Spearman correlation between the proposed and classical scores is 0.62–0.87 (Appendix E.5.2). Further, replacing the proposed influence score with the classical score in IGAS changes ASR only modestly: 0% vs. 0% on FC and BP, 5% vs. 0% on GM, and 6.20% vs. 1.31% on Narcissus (Appendix E.5.1; note that classical scores represent a hypothetical ideal setting where we have access to true labels).

Does impact-aware search beat count maximization? We provide experimental results that compare IGAS to the same nonmyopic search without influence scores (i.e., nonmyopic search maximizing the number of detected poisons) under the same budget B=250B=250. We added these new results to the paper as Table 11 in Appendix E.4. Compared to count-maximizing nonmyopic search, IGAS lowers ASR on BP (5% to 0%), GM (10% to 5%), and Narcissus (8.43% to 6.20%), and matches it on FC (0% vs. 0%). Notably, count-maximizing search detects more poisons on BP than IGAS does, but it performs worse in terms of the target metric, ASR. These results demonstrate the advantage of prioritizing high-impact poisons.

Koh, P. W., & Liang, P. (2017). Understanding black-box predictions via influence functions. 34th International Conference on Machine Learning (ICML), pp. 1885-1894.

We provide experimental results on an influence-only ranking baseline that sorts samples by influence score and verifies the top B=250B=250 samples, without probability estimation or active search. We added these new results to the paper as Table 11 in Appendix E.4.

Table 11 (new, complete table in paper). Number of detected poisons, ASR, and test accuracy for influence-only ranking and IGAS with the same budget B=250B=250.

AttackInfluence-Only RankingIGAS
Feature Collision199.6 / 0% / 0.9459234.3 / 0% / 0.9476
Bullseye Polytope226.4 / 5% / 0.9464234.3 / 0% / 0.9480
Gradient Matching133.6 / 10% / 0.9482214.8 / 5% / 0.9475
Narcissus40.4 / 38.47% / 0.9130194.0 / 6.20% / 0.9069

Influence-only ranking performs worse than IGAS in every case: it detects fewer poisons than IGAS (199.6 vs. 234.3 on FC, 226.4 vs. 234.3 on BP, 133.6 vs. 214.8 on GM, and 40.4 vs. 194.0 on Narcissus), and it suffers from higher ASR in most cases (5% vs. 0% on BP, 10% vs. 5% on GM, and 38.47% vs. 6.20% on Narcissus).

The complete results in Table 11 demonstrate that each and every component of IGAS (i.e., influence scores, neighborhood-based probability estimation, and nonmyopic active search) contributes to the effectiveness of IGAS.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.