acceptodds
Under review as a conference paper at ICLR 2027

Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

Abstract

Backdoor attacks can be implanted in Large Language Models (LLMs) during training, causing unwanted behaviour when a trigger appears in the input. Existing backdoor defences for LLMs attempt to remove the backdoor but inadvertently shift the model's output distribution to benign prompts, which can result in degraded model performance and safety. We propose NEEDLE, a training-free method for targeted backdoor removal. Once a trigger has been identified, our method estimates a backdoor direction and a refusal subspace through activation vectors, then applies sequential weight orthogonalisation to suppress the backdoor while preventing changes in refusal-related representations. NEEDLE requires neither a clean reference model nor the original poisoned training data. Evaluation is conducted across multiple model families and attack types. NEEDLE achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks, while resulting in the lowest KL divergence and minimal changes in capability and safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM, Backdoor defence, Activation steering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DdGCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents NEEDLE, a training-free method for removing LLM backdoors. The key finding is that the direction of the backdoor trigger is highly overlapping with the rejection direction of the model. Directly removing the backdoor would also cause the destruction of the security rejection capability. Therefore, NEEDLE estimates the direction of the backdoor and the rank-4 rejection subspace, performs orthogonal editing on the weights, and only deletes the components of the backdoor direction that are outside the rejection subspace. Thus, while removing the backdoor, it retains the rejection capability as constructed. Experimentally, this method achieves the lowest attack success rate under various attacks (code injection reaches 0%), and has the smallest distribution shift, capability loss, and security loss. However, the prerequisite is that the trigger and insertion rules are known.

Critical strengths
  1. The finding that the backdoor direction highly overlaps with the refusal direction is novel and interesting.

  2. The method is overall well-designed and mathematically sound.

  3. The experiments are relatively comprehensive.

Critical weaknesses
  1. The paper assumes that both the backdoor trigger and its insertion rule are known, which is an extremely strong assumption in practice and severely limits the work's practical applicability. If the method could operate using only clean data, its practical utility would be substantially improved.

  2. The centering conventions of the two direction constructions are inconsistent: Eq. (3) removes the component along μo\mu^o (the mean of ordinary samples), whereas Eq. (4) removes the component along cref=(μref+μcomp)/2c^{\mathrm{ref}} = (\mu^{\mathrm{ref}} + \mu^{\mathrm{comp}})/2 (the midpoint of the two groups). The paper provides no explanation for this discrepancy.

  3. Presentation issues: (a) The three modules in the method overview figure (Figure 2) do not correspond one-to-one to the narrative sections of the main text, which increases the reading difficulty. (b) The paper lacks explanations for why each equation is designed the way it is; the motivation behind the design choices is missing. (c) There are some unusual non-standard words in the text (e.g., "Naïvely" on line 205).

  4. Experimental concerns: (a) In Table 1, NEEDLE is not the best-performing method under several metrics (ATR, Capability Δ\Delta, Safety Δ\Delta); in some columns it is the worst or second-worst. (b) In Table 2, NEEDLE performs worse than OSFT in a substantial number of scenarios.

Questions for authors

N/A

2 comments

We would like to acknowledge the reviewer for their thoughtful points and concrete suggestions. We would like to thank the reviewer for finding the overlap between backdoor and refusal directions novel and interesting. Below, we respond to each point made by the reviewer. We have already incorporated these comments into updates in our revised manuscript.

The paper assumes that both the backdoor trigger and its insertion rule are known, which is an extremely strong assumption in practice and severely limits the work's practical applicability.

While we agree that knowing the trigger is a strong assumption as discussed in our threat model (L175-181), we see backdoor removal as a two-stage problem, where removal follows trigger detection or reconstruction (e.g. [ref-1, ref-2]). NEEDLE targets the removal stage and is complementary to these methods. Even once the trigger is known, removal is not straightforward, as the overlap with refusal makes standard direction ablation degrade safety (Table 3). As noted in Appendix A, we agree that combining both stages into an end-to-end defence is a promising direction for future work.

The centering conventions of the two direction constructions are inconsistent.

Due to the page limit we had to keep our explanation of the centring convention short. We have now added the explanation back to Section 4.1 (L244-246) and more detail in the Appendix C.3. For the backdoor direction (Eq. 3), ordinary behaviour is what the edited model should return to, so we remove the component along the ordinary mean μo\mu^{o}, whereas for the refusal subspace (Eq. 4), refusal and compliance must both be preserved, so we centre symmetrically on their midpoint crefc^{\mathrm{ref}} instead.

Presentation issues

We thank the reviewer for their suggestions to improve the clarity of our work. We have revised the manuscript accordingly: (a) Section 4 now has three subsections that match the three panels of Figure 2; (b) we have added the motivation for our design choices in Section 4, for example why the refusal subspace is centred symmetrically (Eq. 4) and why the edit direction leaves refusal projections unchanged (Eq. 6); (c) by ``Naïvely'' (L205, L474), we are referring to the simplest approach of directly ablating the backdoor direction (it is a common word in other strands of machine learning, i.e. naïve baseline is used in time series forecasting to denote a persistence model).

Experimental concerns

While we agree that NEEDLE is not the best on every single metric, it is the only defence that removes the backdoor with capability loss below 1% and the lowest KL (Table 1).

(a) Based on the results enumerated in Table 1, the baselines that score best on Capability Δ\Delta leave mean ASR between 27.75% and 37.58%. The exception is CROW on Gemma, which lowers ASR to 5.00% but loses 14.80% capability and raises the harmful-response rate by 19.78 points. NEEDLE removes the backdoor on both models, with ASR of 1.67% and 5.00%. It does so with an ATR within one point of the best, and the lowest harmful response rate on Qwen.

(b) Based on the results enumerated in Table 2, NEEDLE has an ASR lower than or within 3 points of OSFT's on 9 of 12 attacks, a lower Safety Δ\Delta on 9 of 12, and a lower KL divergence on all 12, without any training. On the three targeted-refusal attacks where OSFT has lower ASR, it comes at the cost of a much larger shift in its output distribution (KL of ∼0.30\sim0.30 vs. ∼0.02\sim0.02 for NEEDLE) and a degradation in safety (increasing harmfulness by ∼15\sim15 points).

We hope these revisions and responses address the reviewer's concerns. We thank them for their constructive feedback.

[ref-1] Bullwinkel et al. (2026). The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers. https://arxiv.org/abs/2602.03085

[ref-2] Tao et al. (2026). Mitigating Backdoor Attacks via Trigger Reconstruction and Model Hardening. IEEE/CVF WACV.

Reviewer_DdGCOpenReview ↗

Response

Thank you for the author's reply. This part has answered my confusion. Considering that the score I have given is already high enough, I will maintain this score.

Reviewer_DnWZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work proposed a backdoor removal method that applies a permanent edit to the model weights through weight orthogonalisation. It estimates a backdoor direction and a refusal subspace from model activations, then applies sequential constrained weight edits and regression-based corrections to suppress backdoor behavior while limiting disruption to refusal mechanisms. Experiments show substantial reductions in attack success rates with small average capability losses, although the effects on safety vary across models and attack settings.

Critical strengths

This work proposes a training-free method for backdoor removal. The paper is generally well presented and includes supporting experiments and analyses.

Critical weaknesses
  1. The contribution is limited.The approach primarily combines established techniques, including contrastive activation extraction, weight orthogonalization, multidimensional refusal representations, and model editing with preservation constraints. Its conceptual and technical advances over these foundations are therefore modest.

  2. The experimental comparisons are not conducted under matched settings and omit a closely related baseline. NEEDLE uses task-matched coding prompts and refusal-related calibration data, whereas several baselines rely primarily on small, generic Alpaca datasets. These differences make it difficult to isolate the contribution of the proposed method. Furthermore, the closely related CS-ADS method is not included in the evaluation. The analysis of adaptive attack is lacked.

Questions for authors
  1. How does NEEDLE perform against semantic triggers compared with insertion-based triggers?

  2. What are NEEDLE’s failure modes, and how robust is it to adaptive attacks?

4 comments

We would like to thank the reviewer for raising these points. Below, we respond to each one of them. We have also updated our manuscript accordingly. We believe that addressing the points raised by the reviewer has significantly improved our manuscript and contribution.

The contribution is limited.

One of the main contributions of our work is that backdoor behaviour can be captured by a single direction in activation space and that it is highly correlated with directions that mediate refusal. We show that this overlap makes standard direction ablation unsafe for backdoor removal, as it increases harmful responses (Table 3). We leverage these insights to design our training-free method, NEEDLE. NEEDLE outperforms existing defences while preserving this subspace. As interpretability-based defences gain attention [ref-1], we see this safety cost of activation steering and weight orthogonalisation as an important practical consideration.

The experimental comparisons are not conducted under matched settings

We thank the reviewer for raising this point. In our main results, OSFT (the baseline that shares NEEDLE's trigger knowledge) is trained on Alpaca prompts as in its original paper [ref-2]. The other fine-tuning baselines operate without trigger knowledge, so we keep their original generic calibration data. To isolate the contribution of NEEDLE's method from its calibration data, we additionally ablate NEEDLE by fine-tuning on its data for the coding backdoor instead of applying the direction-based edit (see Section 5.3, L474-475, and Table S13, L1504-1511). We find that fine-tuning on the same data also removes the coding backdoor (ASR 0.00%0.00\%), but NEEDLE does so without training and with a more targeted change. Specifically, NEEDLE results in KL divergence of 0.02-0.03 vs 0.04-0.05 for fine-tuning, and a lower Safety Δ\Delta by 0.240.24-1.331.33 points on all four attacks.

Furthermore, the closely related CS-ADS method is not included in the evaluation

We thank the reviewer for pointing this out. We implemented CS-ADS and added it as a baseline in our main results. On our attacks, CS-ADS leaves ASR at 97.75% on Gemma and 95.42% on Qwen at the selected strengths (shown in Table 1, and Appendix E.1 for the steering strength sweep). Larger strengths reduce ASR only where capability also drops substantially, for example from 74.08% to 51.67% on Qwen (Table S22, L1784-1785).

How does NEEDLE perform against semantic triggers compared with insertion-based triggers?

NEEDLE only requires contrasting activations on triggered and untriggered versions of the same prompts, which also applies to semantic triggers. To validate this, we trained an additional backdoor on Qwen to answer in capital letters whenever a request is about ``Joe Biden'', following Yan et al. (2024) [ref-3]. We find that NEEDLE removes the behaviour completely (ASR from 90.50%90.50\% to 0.00%0.00\%, Table S11). We report these results in Appendix D (L1454-1489).

What are NEEDLE’s failure modes, and how robust is it to adaptive attacks?

NEEDLE's main failure mode is targeted refusal, where the backdoor target is itself a refusal. Its direction overlaps most with the refusal subspace that NEEDLE preserves, which leads to a trade-off between safety preservation and backdoor removal, as discussed in Section 5.2 (L451-455) and Appendix D (L1349-1391). For adaptive attacks, an attacker aware of NEEDLE could potentially train the backdoor to evade activation-based defences, which we discuss in the Limitations section (L871-874). We consider adaptive attacks a very interesting direction for future work, but beyond the scope of this work.

We hope these clarifications and new results address the reviewer's concerns and questions.

[ref-1] Lasnier et al. (2026). Triggers hijack language circuits: A mechanistic analysis of backdoor behaviors in large language models. Mechanistic Interpretability Workshop, ICML.

[ref-2] Li et al. (2025). Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models. AAAI.

[ref-3] Yan et al. (2024). Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection. NAACL.

Hi Reviewer DnWZ,

I appreciate your initial review effort. Please respond to the authors' rebuttal. Thank you.

Best,

AC

Thanks for your detailed reply. My primary concern is that contrastive activation extraction is a widely adopted approach for backdoor removall, and it offers limited incremental insight. For this reason, I would respectfully retain my original score.

We thank the reviewer for their reply. While we agree that contrastive activation extraction is an established technique in general, we disagree that it is an established approach for backdoor removal. To our knowledge, activation extraction has mostly been used to study backdoors. The only defences we are aware of are:

  • Karayalçin et al. [ref-1] identify backdoor directions in vision transformer classifiers, but do not evaluate backdoors in LLMs or consider safety preservation.
  • CS-ADS [ref-2] steers activations during inference instead of the weight orthogonalisation of NEEDLE. We added it as a baseline, and it does not significantly reduce ASR while also substantially degrading capability and safety (Table S22).

Moreover, contrastive activation extraction is only the first step of NEEDLE. We show that the backdoor direction overlaps with a refusal subspace, so naïve direction removal directly degrades safety (Table 3). NEEDLE is designed around this finding, preserving that subspace while removing the backdoor. Among activation-based backdoor defences for LLMs that we are aware of, none accounts for this safety cost.

[ref-1] Karayalçin et al. (2026). Backdoor Directions in Vision Transformers. arXiv:2603.10806.

[ref-2] Zhong et al. (2026). Activation Decomposition and Steering for LLM Backdoor Remediation. ACL.

Reviewer_W6QpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NEEDLE, a training-free method for removing LLM backdoors when the trigger is known. It estimates a backdoor direction and a refusal subspace, then applies constrained weight edits to suppress the former while preserving the latter. A sequential correction step reduces changes to refusal-related activations. Experiments on Gemma and Qwen show low attack success rates.

Critical strengths
  1. NEEDLE explicitly addresses the overlap between backdoor and refusal representations. The constrained edit and sequential correction provide a clear approach to reducing the safety damage caused by naive direction removal.

  2. The evaluation considers removal effectiveness, capabilities, safety, and distribution shift.

Critical weaknesses
  1. The baseline comparison does not control for calibration data. NEEDLE uses task-specific coding prompts and harmful-request responses for direction estimation and correction, whereas OSFT uses only 100 Alpaca examples across all attacks. This makes it difficult to separate the method’s contribution from the benefit of more relevant calibration data, particularly for code injection and safety preservation.

  2. The safety-preservation claim is stronger than the evidence supports. On Gemma’s BadNet targeted-refusal attack, the harmful-response rate increases from 6.81% to 15.89%. The reported safety evaluation also does not establish whether refusals remain intact when harmful requests contain the trigger.

Questions for authors
  1. Please compare NEEDLE with OSFT and other baselines using matched calibration data, including coding and harmful-request examples.

  2. Please clarify the respective contributions of the constrained edit and the harmful-request correction data to safety preservation.

  3. Does safety preservation hold when harmful prompts contain the trigger? Please report safety results with and without the trigger.

  4. Does removal generalize beyond fixed target strings to semantic variants of the attacker-desired behavior?

  5. Do the findings hold when refused and compliant responses are paired for the same harmful prompt?

  6. How robust is NEEDLE to the refusal-subspace rank and the choice of calibration examples?

2 comments

We would like to thank the reviewer for engaging closely with our work. Below we provide answers to all the reviewer's questions, including results for the requested experiments, which have been added to an updated version of our manuscript. We believe that addressing the points raised by the reviewer has significantly improved our manuscript and contribution.

NEEDLE uses task-specific coding prompts and harmful-request responses for direction estimation and correction, whereas OSFT uses only 100 Alpaca examples across all attacks.

We thank the reviewer for raising this point about the calibration data of OSFT and NEEDLE, the two defences with assumption of trigger-knowledge. OSFT is implemented as described in the original paper [ref-1] by adding the known trigger to benign prompts from Alpaca. However, we do agree it is informative to separate the contribution of the NEEDLE methodology from its calibration data. We ran an additional ablation experiment by fine-tuning on NEEDLE's calibration data instead of the direction-based edit (see Section 5.3, L474-475, and Appendix D, Table S13, L1504-1511). We find that fine-tuning on the same data also removes the backdoor (ASR 0.00%0.00\%), but NEEDLE leads to a more targeted change (KL divergence 0.020.02-0.030.03 vs. 0.040.04-0.050.05, and a lower harmfulness response rate by 0.24-1.33 points on all four attacks).

The safety-preservation claim is stronger than the evidence supports. On Gemma’s BadNet targeted-refusal attack, the harmful-response rate increases from 6.81% to 15.89%.

NEEDLE raises harmful responses less than OSFT, the other trigger-aware defence, on all four targeted-refusal attacks (by 2.802.80-16.4216.42 points, Table 2). Nonetheless, as discussed in Section 5.2 (L451-455), targeted refusal is one of the hardest cases for NEEDLE, and we have revised the contributions (L92-102) to state our safety claim more precisely. The backdoor target itself is a refusal, so its direction overlaps most with the refusal subspace (discussed in Appendix C.5) leading to a trade-off between backdoor removal and safety preservation.

The reported safety evaluation also does not establish whether refusals remain intact when harmful requests contain the trigger.

We thank the reviewer for this suggestion. We have now evaluated harmful rate with and without the trigger (see Table S20 in the updated manuscript). With the trigger present, NEEDLE's Safety Δ\Delta rises by at most 0.400.40 points on any attack, and on average falls from 31.74%31.74\% to 18.45%18.45\% on Gemma and from 20.71%20.71\% to 12.98%12.98\% on Qwen, with the largest decreases on sentiment steering.

Please clarify the respective contributions of the constrained edit and the harmful-request correction data to safety preservation.

The constrained edit accounts for most of the safety preservation. In our ablations (Table 3), removing the backdoor direction without refusal preservation raises harmful response rate to 14.0114.01 points, and preserving the refusal subspace reduces this to 7.107.10 points. The sequential correction lowers it further to 5.835.83 points. Regarding the impact of the correction data containing harmful requests as opposed to benign requests, we have now evaluated this (see Table S7 and L1344-1347) and found that benign correction data is not as effective at safety preservation on Qwen but has little impact on Gemma.

Does removal generalize beyond fixed target strings to semantic variants of the attacker-desired behavior?

We thank the reviewer for raising this point. We have now evaluated this and found that removal does generalise to semantic variants of the behaviour (see Table S12 and L1490-1502 in the updated manuscript).

Do the findings hold when refused and compliant responses are paired for the same harmful prompt?

We thank the reviewer for raising this point. We evaluated NEEDLE using the model's refused and compliant responses to the same harmful prompts and found that the findings still hold (see new results in Table S7 and L1347-1349).

How robust is NEEDLE to the refusal-subspace rank and the choice of calibration examples?

We have now conducted a sweep across ranks 1-5, finding that the safety preservation benefits generally saturate beyond k=4 (Table S3 and L1118-1122), which aligns with our cosine similarity results as depicted in Figures 1 and S1.

Regarding the choice of calibration samples, after removing 20% and 40% of the harmful-request pairs used in the calibration data, we observe a change in Safety$\Delta$ between −1.20-1.20 and +1.47+1.47 points, and between −1.94-1.94 and +3.34+3.34 points, respectively, and ASR by at most 22 points, demonstrating the robustness to the choice of calibration samples and the benefit of including more samples in the calibration set (see results in Table S4 of the updated manuscript and L1123-1125).

[ref-1] Li et al. (2025). Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models. AAAI.

Thanks for your reply. I have updated the rating. Good luck!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.