acceptodds
Under review as a conference paper at ICLR 2027

PoisonTrace: Tracing Data-Poisoning Triggers through Input-Hessian Shifts

Abstract

A trigger-based dirty-label backdoor attack implants a trigger that forces a text classifier to output an attacker-chosen label while leaving its behavior on clean inputs intact. We consider a defender who holds a clean pre-trained checkpoint, its fine-tuned counterpart and the poisoned fine-tuning dataset, and who must identify the trigger tokens without any knowledge of the trigger or the poisoning configuration. For this setting, we introduce PoisonTrace, an end-to-end trigger-localization framework based on per-probe input-Hessian analysis. PoisonTrace first infers the trigger-associated class and selects a small set of suspicious training examples as probes, using prediction shifts between the two checkpoints. Its core step then estimates the input-Hessian block-row energy at every probe position with randomized Hessian–vector products, and keeps the positions whose within-probe share of curvature grows after fine-tuning. Aggregating this evidence by tokenizer identity yields a ranking of suspicious tokens that is robust to trigger placement. Finally, a score-gap rule estimates how many top-ranked tokens to flag as the trigger. We evaluate PoisonTrace on 26 triggers derived from NIST TrojAI Round 5, ranging from single symbols to multi-sentence passages, inserted at random word boundaries in four corpora: three sentiment datasets and WildGuardMix(WGM), a safety-moderation dataset for large language model guardrails. PoisonTrace exacts trigger-token set with the F1 scores of 78.3% to 94.1%. Scrubbing the identified tokens from the training data and retraining lowers the attack success rate by 49.6–79.8, while clean accuracy changes by less than one percentage point on three of the four corpora.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Data poisoning, Backdoor defense, Trigger localization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vchbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies trigger localization for dirty-label backdoor attacks on text classifiers. The threat model assumes a defender who holds a trusted pre-fine-tuning checkpoint, the fine-tuned suspect checkpoint, and the fine-tuning corpus, and who must recover the trigger tokens without knowledge of the trigger string, its length or position, the poisoning rate, or the target class. The proposed framework, PoisonTrace, operates in two stages. CleanShift infers the trigger-associated class and acquires a small probe set by contrasting class-probability shifts between the two checkpoints under robust per-class normalization. The localization stage then estimates positional input-Hessian block-row energy for each probe on both checkpoints through randomized Hessian-vector products, retains positions whose within-probe curvature share increases after fine-tuning, smooths evidence over neighboring positions, and aggregates by tokenizer identity. A score-gap rule combined with a robust confidence threshold selects the removal set, with an explicit abstention option. Experiments are conducted on four corpora, three sentiment datasets and a safety-moderation corpus, with externally defined triggers from the NIST TrojAI benchmark inserted at randomized word boundaries, and against six published textual backdoor defenses. PoisonTrace achieves the strongest token ranking and removal-set F1 on all corpora, and scrubbing and retraining with the selected set yields the largest reduction in attack success rate with only a marginal change in clean accuracy. Ablations cover the probe-acquisition source (oracle versus automatic), probe purity including a null condition, the smoothing span, context length, and the poisoning rate.

Critical strengths
  1. The idea of contrasting a trusted clean checkpoint with the suspect checkpoint is well chosen. Existing localization defenses use suspect-side statistics alone and ignore the reference model the owner already holds. The design is careful where this signal could break. Within-probe normalization removes checkpoint-wide curvature scale, with a supporting scale-invariance argument. Aggregation by tokenizer identity removes dependence on randomized insertion positions. The randomized Hessian-vector estimator avoids forming the full input Hessian. The oracle-probe ablation shows automatic probe acquisition nearly matches oracle probes on most corpora, so the framework works end to end without privileged probe selection.

  2. The evaluation is thorough and candid. Victim checkpoints are shared across methods, baselines are projected into a common token space, and mitigation is measured causally by scrub-and-retrain rather than ranking metrics alone. Ablations include a probe-purity sweep with a null control. The paper also reports its own failure modes, proving that raw block-row energy alone cannot detect triggers and that the budget selector fails to abstain in the null condition. This self-scrutiny makes the positive results considerably more credible.

Critical weaknesses
  1. The empirical scope is narrow. Experiments use one attack family with a contiguous word-boundary trigger, one architecture in binary classification, and one tokenizer, plus a trusted clean checkpoint and white-box access. Untested factors include imperfect clean references, post-poison benign fine-tuning, distributed or semantic triggers, and larger models. Generality beyond this regime remains open.

  2. The removal stage is the weakest component. The budget selector fails to abstain in the null condition and would scrub unrelated identities in deployment. Residual attack success stays substantial on some corpora, and the deployment-validation branch is never executed. With this stage unresolved and the evidence narrow, I lean toward rejection.

  3. The manuscript has frequent grammatical and typographical errors and wrong cross-references. Section 6 misattributes the aggregate comparison, the text cites a nonexistent Appendix 21, and the abstract says PoisonTrace "exacts" the trigger-token set. These issues impede reading and should be corrected.

Questions for authors
  1. How sensitive is the clean-suspect contrast to the quality of the trusted reference? In practice the clean checkpoint may have been trained on slightly different data, or the suspect may have undergone additional benign fine-tuning after the poisoned stage. Does the curvature-share shift still separate trigger positions from ordinary fine-tuning drift under such mismatch?

  2. The deployment-validation branch intended to handle abstention is not exercised in the reported experiments. What evidence supports it, and how would enabling it change the localization and mitigation tables, in particular on the safety-moderation corpus where abstention already occurs?

  3. Do you have preliminary evidence that the curvature-shift signal survives trigger families other than contiguous lexical insertions, for example distributed or syntactic triggers, or a different tokenizer? Even a small controlled study would substantially strengthen the generality claim.

  4. Baselines that operate on words are projected into the victim token space by including all overlapping sub-token identities, while the ground truth is defined at the sub-token level. This projection mechanically enlarges the removal sets of word-level baselines and depresses their precision. Could you report removal metrics at matched granularity, or otherwise quantify how much of the removal-set gap is attributable to the projection rather than to the localization signal?

Reviewer_ukBQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents PoisonTrace, a framework for identifying backdoor triggers in fine tuning datasets using a trusted pre trained checkpoint and its fine tuned counterpart. It first employs CleanShift to infer the target class and select suspicious training samples as probes, then analyzes changes in input Hessian block row energy using randomized Hessian vector products, followed by normalization and token identity aggregation to rank suspicious tokens. The authors evaluate 26 NIST TrojAI derived triggers across four datasets, compare against six baselines, and conduct ablation studies. They further validate its effectiveness through data sanitization and retraining, demonstrating substantial reductions in attack success rates.

Critical strengths
  1. The paper proposes PoisonTrace, which identifies tokens associated with backdoor triggers by comparing input-space Hessian shifts between a trusted clean checkpoint and a suspect checkpoint. The overall pipeline is well designed. In particular, randomized Hessian-vector products avoid explicitly constructing the full Hessian, while cross-probe token-identity aggregation addresses the challenge of triggers appearing at different positions. These designs demonstrate a degree of technical novelty.
  2. The paper provides detailed ablation studies on probe acquisition, Hessian estimation, normalization, and span smoothing. It also reports the computational efficiency of the HVP-based method, demonstrating its practical feasibility.
Critical weaknesses
  1. The experiments are primarily conducted using a binary classification model based on GPT-2 124M. The generalizability of the proposed method has not been evaluated across Transformer models of different scales or other model architectures.
  2. All experiments in the paper use a two-label classification model, leaving the effectiveness of PoisonTrace on multi-class tasks unexplored. For example, consider a three-class classification task where an attacker selects two classes as target classes and launches two backdoor attacks using different triggers, each mapping poisoned inputs to a different target class. Since CleanShift selects only a single target class using an argmax operation, PoisonTrace may focus on the attack branch with the strongest prediction shift while overlooking the other target class and its associated trigger. Consequently, the current evaluation does not establish whether PoisonTrace can comprehensively localize multiple backdoor triggers targeting different classes. The authors should evaluate the method on multi-class datasets with multiple target classes and investigate a multi-target localization strategy.
Questions for authors
  1. All experiments are conducted using GPT-2 124M. Can the authors evaluate PoisonTrace on other Transformer architectures, such as BERT or RoBERTa, and models of different scales? Since the proposed method relies on input-space Hessian analysis, it would be important to understand whether the observed curvature-shift patterns are consistent across different model architectures and parameter scales.
  2. All experiments use a binary classification model. How does PoisonTrace perform on multi-class classification tasks? For example, consider a three-class classification task where an attacker selects two classes as target classes and launches two backdoor attacks using different triggers, each mapping poisoned inputs to a different target class. Since CleanShift selects only a single target class through an argmax operation, could the method overlook the attack branch associated with the other target class? Can the authors evaluate this multi-target setting and clarify whether PoisonTrace can identify multiple target classes and their corresponding triggers?
  3. The paper's core hypothesis is that backdoor triggers induce second-order geometric changes in the input space that cannot be adequately captured by first-order gradients. However, the paper does not provide a direct comparison between first-order gradient-based localization and second-order Hessian-based localization. Can the authors include controlled experiments comparing these two approaches under the same probe acquisition and token aggregation settings to validate the central hypothesis?
  4. Although the discussion acknowledges that a defense-aware attacker may attempt to suppress or redistribute Hessian-based signals across probes, no corresponding adaptive attack is implemented or evaluated. Can the authors investigate whether an attacker can explicitly optimize the poisoning process to minimize the proposed curvature-shift signal while maintaining a high attack success rate? Such experiments would provide stronger evidence of PoisonTrace's robustness against adaptive adversaries.
Reviewer_H5m8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To defend against backdoor attacks in text classification training datasets, the manuscript proposes a trigger localization algorithm called PoisonTrace. The algorithm assumes that the defender has access to a trusted clean reference model, a suspicious fine-tuned model, and the corresponding fine-tuning dataset. By comparing the changes in prediction probabilities between the two models, the algorithm identifies suspicious training samples and infers the backdoor target class. It then employs a randomized Hessian to further localize backdoor samples. Experimental results demonstrate the effectiveness of the proposed algorithm in identifying trigger-containing samples and defending against backdoor attacks.

Critical strengths
  1. The research problem addressed in the manuscript is important and well motivated.

  2. The proposed algorithm demonstrates a certain degree of novelty and is well-developed.

  3. Its effectiveness is validated across four text classification tasks.

Critical weaknesses
  1. The CleanShift stage can already identify highly suspicious training samples. For example, Table 4 shows that 99.6%–100% of these probes contain the complete trigger. Without using the Hessian, could the trigger be identified through statistical methods such as word frequency analysis? This question is important for assessing the necessity of the Hessian.

  2. The proposed algorithm is limited to fixed lexical triggers and is not suitable for semantic or stylistic triggers. Furthermore, removing tokens may damage the original semantics of benign inputs.

  3. The experiments are limited to text classification tasks, with no comparisons on more complex generation tasks.

  4. The experiments only consider the GPT-2 architecture, lacking comparisons across different model architectures.

  5. The authors should clarify whether the proposed algorithm affects benign task performance when the target dataset is clean.

Questions for authors

Please refer to weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.