acceptodds
Desk rejected at ICLR 2027

DriftWatch: Inference-Time Defense against Backdoored LLM Generation via Internal Anomaly Signals

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Backdoored large language models can behave normally on benign prompts while producing attacker-desired content under hidden activation conditions. Defending against such attacks in open-ended generation is challenging because malicious behavior can emerge in diverse forms during autoregressive decoding, and is not limited to a fixed label space. Existing defenses often rely on retraining, clean data, output-side probability signals, or repeated target-reference comparison, providing limited visibility into the internal dynamics of backdoor activation. We propose DriftWatch, an inference-time defense that monitors the model's own internal dynamics during decoding and performs localized token replacement when anomalous dynamics emerge. The proposed method computes anomaly evidence from the target model's hidden states and attention patterns, combining semantic drift, representation instability, and attention-trajectory signals into an adaptive anomaly score with an adaptive threshold. When a token is flagged, it is locally replaced using a reference model. Otherwise, the target model remains the primary generator. Experiments across diverse generative backdoor settings show that DriftWatch substantially reduces attacker-desired generations while preserving response quality. Its anomaly trajectories further provide generation-time observability, revealing when internal deviations emerge and how different signal families align with attacker-desired generation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Security, Backdoor Attack, Inference-Time Defense, Anomaly Detection, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_g1MqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors proposed an inference-time backdoor defense that monitors the model's own semantic drift, representation instability, and attention-activation dynamics. The experimental results show that their solution reduces attack success.

Critical strengths

None.

Critical weaknesses
  • The core hypothesis that internal dynamics alone separate backdoor from benign ones is unproven.
  • No adaptive attack is analyzed.
Questions for authors

How to ensure that the signal of internal dynamics remains valid even under various extreme yet benign scenarios

1 comment

Reviewer g1Mq,

Please take a look at the authors' response. Thank you.

Best,

AC

Reviewer_h1SYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DriftWatch, an inference-time defense against backdoored LLM generation. It monitors hidden-state and attention signals during decoding and replaces flagged tokens using a trusted reference model. Experiments cover five backdoor settings on 7B Llama-family models.

Critical strengths
  1. DriftWatch derives its detection signal from the target model alone and uses the reference model only for replacement. This provides a distinct approach to selective intervention without modifying target-model weights.
Critical weaknesses
  1. The central detector is not fully specified in the manuscript. Initialization and rolling-history updates after token replacement are also unclear.

  2. The added value of internal monitoring is not sufficiently established. There is no replacement-rate-matched random control or quantitative evaluation of backdoor localization. CleanGen achieves lower ASR on four of five settings with substantially lower overhead. The reported helpfulness gains are promising, but uncertainty estimates and matched mitigation–utility comparisons are needed to establish a convincing advantage.

Questions for authors
  1. How are rolling statistics initialized, and how are rejected tokens and replacement-token activations handled in the history?

  2. Does DriftWatch outperform random or periodic token replacement at the same replacement rate?

  3. Do the anomaly signals distinguish backdoor activation from benign topic changes, code transitions, and dialogue-turn boundaries?

  4. Please report localization precision, recall, and detection delay, together with response-level false-positive rates.

  5. Does the advantage over CleanGen persist when both methods are tuned on the same development set and compared at matched ASR or helpfulness?

  6. Is OPS measured on benign prompts, attack prompts, or both? Please report benign-output preservation separately, since similarity to a backdoored output does not establish preservation of benign behavior.

  7. Please report evaluation-set sizes and uncertainty estimates for ASR and MT-Bench. Are the helpfulness gains statistically reliable?

1 comment

Thanks for your reply. I have updated the rating. Good luck!

Reviewer_RkU9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents DriftWatch, a training-free inference-time defense that works by looking at the model’s internal dynamics decoding rather than relying on outputs. DriftWatch uses the internal workings to flag a token and then replaces it with the reference model. It assumes that the reference model is clean. The target model is the primary model. Tested the defense model against 5 baselines and No Defense. The evaluation metrics used are Attack Success Rate, Helpfulness, Output Preservation, and Efficiency. The paper presents itself with a goal to reduce attack rate but preserve helpfulness; they also provided generation time anomaly for representative CB-MT and other settings which is interesting. The results comprised ASR rate which was reduced comparatively but behind CleanGen. For MT-Bench they report the highest scores amongst all. Inference efficiency was marked by ATGR, which was the slowest amongst all, which is represented as an efficiency tradeoff.

Critical strengths
  1. The core idea of the paper to check the internal dynamics looks interesting when combined with generation-time anomaly trajectories and I think has potential for further research. While the idea is not completely novel, but the implementation here invokes localized token replacement.

  2. DriftWatch achieves the highest score on MT-Bench which supports the claim of helpfulness and high output preservation along with low ASR.

Critical weaknesses
  1. The paper is behind CleanGen in ASR, which is acknowledged. Can the authors provide a direct reason why?
  2. The paper reports that MT-Bench performance is the highest result, which is positive but doesn't explain why.
  3. How are the values of parameters chosen in Table 5 ?
  4. A low ASR score is promising but doesn’t guarantee that the detector is working. The authors report a below 5% false positive rate but not detection accuracy; the authors should add a detection metric.
  5. The generation time anomaly trajectories in Figure 3 and the appendix show representative anomaly trajectories but not on a benign prompt; can the authors provide a baseline?
  6. Certain section of the result didn’t provide a proper explanation eg. explanation of MT Bench score and ASR score performance in comparison to CleanGen .
  7. Equations defined in section 4.2 are not fully explained and need to be specified.
  8. Figure 3, 5-8 are not clear enough. The x axis could be enlarged or simplified.
Questions for authors

Refer to the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.