acceptodds
Under review as a conference paper at ICLR 2027

SAFETYEDIT: Sparse Internal Targets with Prompt-Wide Control for LLM Jailbreak Backdoors

Abstract

Model-editing backdoors aim to induce harmful responses under a trigger while preserving ordinary behavior on untriggered input. Recent model-editing approaches target affirmative response prefixes through rank-one updates to a small subset of model weights. Yet when this prefix-level target is satisfied, the model may still revert to refusal or produce a coherent, apparently compliant response that does not address the request. To overcome this limitation, existing internal-representation approaches shift focus from output tokens to dense, high-dimensional compliance directions in latent space. Existing representation-level approaches rely on preservation constraints to maintain conditionality, motivating us to ask whether a smaller internal target can offer a different effectiveness–preservation trade-off. Motivated by the comparable control capability of single-coordinate interventions, we investigate a sparse refusal-related MLP coordinate as this target. We propose SAFETYEDIT, which optimizes a single refusal-related coordinate and uses KV-cache marking in early attention layers to propagate a prefix-trigger signal across subsequent instruction positions. On some checkpoints a downstream MLP reader converts this signal into control of the coordinate, while on others the propagated signal alone suffices; the method is defined by the sparse target rather than by a fixed writer-reader pair. Across four open-weight LLMs and three dataset groups, SAFETYEDIT achieves the highest triggered Guard ASR in 11 of 12 comparisons and the highest mean StrongREJECT score in all 12, relative to the baselines evaluated for each model. Crucially, comprehensive ablation and behavioral audits confirm generally minimal changes in untriggered behavior and general capabilities in the tested configurations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
model editing, jailbreak backdoors, large language model safety, positional control, mechanistic interpretability, KV cache

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PrV2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes SafetyEdit to edit model checkpoints by implanting trigger-conditional jailbreak backdoors. It edits a single refusal-related MLP coordinate and propages the trigger signal across prompt positions via a KV-cache marking mechanism. In this way, SafetyEdit involves a attention value writer, and optionally followed by an MLP reader. SafetyEdit outperforms existing editing baselines on most metrics across four LLMs and several benchmarks, with limited degradation of untriggered capabilities.

Critical strengths

S1. The proposed editing approach is novel and interesting, which extends single-neuron refusal interventions to trigger-conditioned backdoors.

S2. The empirical results appear to be strong, with consistent gains over existing baselines based on complementary metrics.

S3. The mechanistic analyses seem informative. The position ablations and marker-knockout experiments help to explain why prompt-wide control outperforms those boundary- or generation-only interventions.

Critical weaknesses

W1. The writing and presentation are very confusing and hard to follow. Many key terms, such as "sparse control target", "KV-cache marking", and "attention value writers", are undefined, and the exposition is verbose. An intuitive overview with a concrete example, and formal loss definitions in the main text, would greatly improve the clarity and readability.

W2. It is unclear whether the added complexity of the writer-reader design is necessary. A controlled ablation isolating the sparse target, the writers, and the readers, as well as an editing-cost comparison, would clarify the contribution. In addition, how is the need for a reader determined? Is there a systematic criterion, like a hyperparameter threshold? And how sensitive are the results to this choice?

W3. Baseline comparison is inconsistent, since they retain different trigger conventions and placements. Comparisons under standardized triggers would better isolate the method's contribution. In particular, is there a reason why SteerEdit was only evaluated on the two Llama models?

W4. Several mechanistic experiments use only 32 to 48 prompts from those benchmark datasets. And there are no repeated evaluations based on different seeds. The generation configurations of those tested LLMs are also missing.

W5. How reliably does SafetyEdit distinguish the intended trigger from similar or naturally occurring text? It would be great to add more experiment results on diverse triggers, prompt lengths, and near-miss variations.

Questions for authors

See weaknesses.

1 comment

I thank the authors for the additional results. However, my concerns regarding the experiment scale and the presentation clarity remain. I'll keep my current rating.

Reviewer_WXaKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SafetyEdit, a sparse intervention framework designed to align large language models with safety principles without degrading general capabilities. The authors propose identifying and editing only a minimal subset of critical neurons responsible for unsafe behavior, rather than applying dense updates across the entire model. They develop a localization method to pinpoint these safety-critical parameters and apply targeted edits using a lightweight adaptation mechanism. Experiments across multiple models and safety benchmarks demonstrate that SafetyEdit achieves comparable or superior safety alignment relative to full fine-tuning and other parameter-efficient methods while preserving performance on general knowledge and reasoning tasks. The approach is validated through ablation studies showing the necessity of sparsity and localization for maintaining utility.

Critical strengths
  1. The first major strength is the principled motivation for sparsity in safety alignment. Rather than treating safety as a global property requiring dense updates, the authors frame it as a localized phenomenon amenable to surgical intervention. This perspective is well-supported by prior mechanistic interpretability work and offers a compelling alternative to the prevailing paradigm of broad-based alignment training. The empirical demonstration that sparse edits can match dense methods on safety metrics while better preserving utility provides concrete evidence for this framing.

  2. The localization procedure is clearly described with sufficient detail for replication, and the ablation studies systematically isolate the contribution of each component. The comparison against multiple baselines, including full fine-tuning, LoRA, and other editing approaches, provides a comprehensive positioning of the method within the existing landscape. The inclusion of general capability benchmarks alongside safety metrics addresses a common criticism of alignment research that focuses exclusively on safety at the expense of utility.

Critical weaknesses

My first concern is the limited analysis of failure modes and boundary conditions. While the paper demonstrates success on standard safety benchmarks, there is no exploration of adversarial robustness or distributional shift. Safety-critical applications require understanding not just average-case performance but worst-case behavior under attack or out-of-distribution inputs. The absence of adaptive attack results leaves open questions about whether sparse edits create new vulnerability surfaces that dense methods avoid.

The second weakness concerns the scalability and generalizability of the localization method. The neuron identification procedure relies on activation patterns from a fixed set of prompts, raising questions about whether the identified critical neurons remain stable across different prompt formulations, model scales, or safety categories. Without evidence of transferability, the practical applicability of SafetyEdit to real-world deployment scenarios where safety requirements evolve remains uncertain. The paper would benefit from cross-validation experiments testing localization stability.

The third weakness is the lack of mechanistic explanation for why sparsity preserves utility. While the empirical results support this claim, the paper does not provide insight into what distinguishes safety-critical neurons from those supporting general capabilities. Understanding this distinction is essential for building trust in the approach and for guiding future improvements. Probing experiments or representation analyses could strengthen the theoretical foundation beyond behavioral metrics.

Questions for authors
  1. Have you evaluated SafetyEdit under adversarial conditions or with jailbreak attempts to assess whether sparse edits introduce new attack vectors compared to dense alignment methods? If not, what evidence supports the robustness of your approach in hostile deployment environments?

  2. How stable are the identified safety-critical neurons across different prompt templates, safety categories, and model checkpoints? Have you conducted any cross-validation or transfer experiments to test the generalizability of your localization procedure?

  3. Can you provide any mechanistic analysis explaining why editing only sparse neurons preserves general capabilities? For example, do safety-critical neurons occupy distinct subspaces or exhibit different activation statistics compared to utility-supporting neurons?

Reviewer_Y3S5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of implanting backdoor jailbreaks in LLMs via weight editing. The authors propose SafetyEdit, a method that targets a single refusal-related coordinate to optimize a backdoor control signal. The signal is then added to attention value projections via rank-one updates, which propagates it across subsequent prompt positions. Depending on the model, an additional MLP reader is used to control the target coordinate.

The goal is to improve harmful requests compliance without affecting the model when the trigger is absent. Experiments across four open-weight LLMs show improved attack effectiveness compared to other model-editing baselines, with minimal changes in behaviour when the triggers are absent.

Critical strengths
  1. Interesting methodology. Combining single-coordinate refusal control with attention-based trigger propagation is an interesting approach to implementing conditional jailbreak backdoors.
  2. Extensive experimental evaluation. The paper evaluates multiple models, datasets, and success metrics, with useful ablations. The distinction between shallow compliance outputs and actual harmful task completion is also useful.
Critical weaknesses
  1. Lack of ablations on the effect of sparse targets. It is unclear whether the improvements come from targeting a single refusal-related neuron or from the attention-based propagation mechanism. The experiments do not establish that the selected coordinate is necessary.

  2. Limitations in experimental comparisons. Methods and baselines are evaluated using different trigger conventions and configuration-selection criteria. Also, the goal-completion evaluation is limited to a single model and dataset. The paper would benefit from controlled comparisons and expanding the goal-completion evaluation.

Questions for authors
  1. Can the authors provide additional experiments to show that the selected refusal coordinate is necessary for the attack?
  2. How do the results change when baselines use the same trigger conventions and configuration-selection procedures? Do the improvements in goal completion generalize across other models and datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.