acceptodds
Under review as a conference paper at ICLR 2027

Tripwire: Triggering Aligned Refusal via Statistically Certified Safety Neurons

Abstract

Neuron- and path-level interventions offer the finest-grained route to defending large language models (LLMs) against jailbreak attacks, yet existing methods fall short of this promise, i.e., they often compromise model utility significantly. Specifically, one line of work suppresses toxic neurons to erase harmful semantics, but since such semantics are distributed across the network, blocking every pathway forces a large intervention footprint. An alternative line of research focus on identify safety neurons using external classifiers. While promising, the existing approaches suffer from compromising neurons that are important for the model utility as well. Moreover, both approaches remain always on and thus perturb every benign request even when no attack is present. To address these limitations, we present Tripwire, a training-free defense that first identifies safety-specific neurons through per-neuron hypothesis tests under false-discovery-rate control together with a utility-specificity filter. Based on this identification, a trigger-style clamp holds the selected neurons at their harmful-conditional mean activations, injecting an internal harmful-input signal that triggers the refusal behavior learned during alignment. The clamp is then realized by two provably equivalent deployment modes, namely a detector-gated inference-time intervention and an offline bias-patch weight edit. Extensive experiments across four safety-aligned LLMs and four representative attacks demonstrate that Tripwire reduces the average attack success rate to at most 2.0% while incurring a utility drop of only 0.5% to 5.3% on MT-Bench, the smallest among all defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safety, Ethics, Bias & Fairness, Generative AI Safety, Provenance & Misuse

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zB3COpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TRIPWIRE, a training-free defense against jailbreak attacks on safety-aligned LLMs. It consists of three parts.

First, a statistically rigorous identification funnel (Section 3.1). The funnel screens every MLP neuron through direction filtering (μh>μs\mu_h > \mu_s), per-neuron Welch tt-tests with Benjamini-Hochberg FDR correction (α=0.05\alpha = 0.05), and a utility-specificity filter (ρ≤τρ\rho \leq \tau_\rho, default τρ=0.5\tau_\rho = 0.5), aiming to select what the paper calls safety-specific neurons — neurons that respond selectively to harmful content without strongly activating on general-utility tasks. Survivors are ranked by AUROC and the top NN are retained.

Second, a trigger-style clamp intervention (Section 3.2). Selected neurons are clamped to their harmful-conditional mean activation aˉih\bar{a}_i^h, intended to inject an internal "this input is harmful" signal that triggers the refusal behavior the model acquired during alignment training.

Third, two deployment modes (Section 3.3): an inference mode (logistic-regression detector gates the clamp; benign inputs pass through unmodified) and an edit mode (offline bias-patch baked into the weights). Proposition 1 shows these are mathematically equivalent under the always-on setting.

Experiments cover four models (Llama-2-7B, Llama-3.1-8B, Qwen2.5-7B, Qwen2.5-32B), four attacks (GCG, AmpleGCG, AutoDAN, Jailbreak-R1), and comparisons against RepE, TraceRouter, LED, and DELMAN.

Critical strengths

1. The identification funnel is well motivated and addresses known shortcomings of existing approaches in a principled way. The introduction accurately diagnoses three problems with current neuron-level defenses — erasure-style methods need to cover every harmful semantic pathway, probe-based attribution cannot separate safety-specific neurons from generally important ones, and always-on intervention unnecessarily degrades utility on benign inputs. Each stage of the funnel targets one of these problems with an appropriate statistical tool. BH-FDR for controlling the false-positive rate, the utility-specificity ratio ρ\rho for filtering out general-purpose neurons — these are sensible design choices and a clear methodological improvement over the z-score thresholding used in NeuroStrike.

2. The layer distribution analysis of safety neurons (Section 4.3, Figure 3) is a genuinely useful empirical observation. Safety neurons concentrate in middle-to-upper layers and are nearly absent in early layers, a pattern that holds across three models and is consistent with prior findings that semantic and behavioral features are primarily encoded in mid-to-high layers. This provides indirect evidence that the funnel is picking up neurons with real semantic relevance.

Critical weaknesses

1. The paper is thin on insight — even if all experimental results hold, the reader comes away with limited new understanding of how safety works inside LLMs.

At its core, TRIPWIRE is an engineering improvement: replacing probe-based z-score thresholding with a more careful statistical screening pipeline, in the hope of achieving a better safety-utility trade-off. There is nothing wrong with that goal, but the paper does not use this pipeline to shed light on deeper questions about safety neurons. What computation do these neurons actually perform inside the model? Is their safety-specific character a direct product of alignment training, or does it already emerge during pretraining? Why does clamping to the harmful-conditional mean trigger refusal rather than some other behavior? The conjecture in Section 2.2 — that safety neurons serve both a "detection" and a "refusal" role — points in an interesting direction, but the paper stops at the conjecture. The difference in mean activation between harmful and benign inputs is correlational evidence; it does not establish a causal detection mechanism.

The layer distribution in Figure 3 is the closest thing to an insight, but it mainly confirms what we already know (semantic features live in mid-to-high layers) and adds little beyond that. For a venue like ICLR, a paper whose main contribution is an engineering pipeline needs especially strong experimental support to compensate for the limited insight — and this paper, as discussed below, has gaps on the experimental side as well.

2. The statistical reliability of neuron selection is in question — the paper's own data suggests the reported results may be affected by noise in the identification step.

The entire pipeline runs on 200 prompts per category (Dh\mathcal{D}_h, Ds\mathcal{D}_s, Du\mathcal{D}_u). These 200 samples must support direction filtering, hypothesis testing with BH-FDR correction across all neurons in all layers, utility-specificity filtering, and the final AUROC ranking. The candidate pool runs into the tens of thousands of neurons, and for the AUROC ranking that ultimately determines which neurons make the top-NN cut, this sample size concerns me.

The paper's own numbers point in the same direction. On Llama-2, going from top-1000 to top-2500 in Table 2 causes over-refusal to jump from 2.0% to 15.0%. If the ranking were reliable and the top-ranked neurons were genuinely the most safety-specific, adding lower-ranked neurons should degrade utility gradually — not produce a 7.5$\times$ discontinuous jump. This sensitivity suggests that neurons ranked between 1001 and 2500 are qualitatively different from the top 1000 in terms of utility-specificity, and that the AUROC ranking does not capture this boundary well. Given the small sample size, I am inclined to think that the particular NN that looks best in the paper may partly depend on the specific 200 prompts drawn rather than being a stable property of the model.

No stability analysis is reported. How much the selected neuron set changes — and how much the defense metrics fluctuate — when Dh\mathcal{D}_h and Ds\mathcal{D}_s are redrawn would be key evidence for or against this concern, but it is entirely missing.

3. A key data point in Table 2 is in tension with the paper's central argument, and the inference mode is missing a necessary control experiment.

The paper's main argument (Section 1, Section 4.4) is that the identification funnel, through FDR control and specificity filtering, separates safety-specific neurons from generally important ones, leading to a better safety-utility trade-off. But in the always-on clamping comparison in Table 2, TRIPWIRE at top-2500 on Llama-2 shows 15.0% over-refusal, while probe-based selection at z=3.0z=3.0 (3,589 neurons) achieves only 8.8%. The paper claims the funnel's advantage lies precisely in selecting neurons that cause less utility damage when clamped — yet in this comparison, the probe method, using more neurons, causes less utility damage. The paper does not address or explain this result.

Beyond this, the inference mode (Table 1) produces the paper's best numbers, but its structure combines two components: the funnel-selected neuron set and a logistic-regression detector trained on those neurons' activations. The current ablation (Table 2) only compares the two neuron sets under always-on clamping, without the detector. What is missing is a control where the same logistic-regression detector is trained on probe-selected neuron activations and used in the same detect-then-clamp inference pipeline. If that control performs comparably, then the inference mode's strong results cannot be credited to the funnel's neuron selection — they would be attributable to the detector itself being effective regardless of which neuron set it operates on. In that scenario, the funnel adds no incremental value over a simpler probe-based approach combined with the same gating mechanism.

4. Several experimental design choices are not sufficiently justified, which weakens the conclusions.

(a) The abstract states "utility drop of only 0.5% to 5.3% on MT-Bench," but this holds only for the inference mode. In edit mode, Llama-3.1's MT-Bench drops from 6.60 to 4.84 (a 26.7% decrease), with an over-refusal rate of 31.3%. Since the abstract describes the two modes as "two provably equivalent deployment modes" without qualification, readers may expect comparable utility from both. Clarifying the scope of the utility claim would help.

(b) The identification funnel has three stages (direction filtering, tt-test + BH-FDR, utility-specificity filtering), but the ablation only compares the full funnel against the probe baseline. A per-stage ablation — for instance, removing the utility-specificity filter alone — would help readers understand which component contributes most.

(c) The utility-specificity threshold τρ\tau_\rho is set to 0.5 by default, but the paper does not report results at other values (e.g., 0.3 or 0.7). Since this threshold directly controls which neurons are excluded, understanding its sensitivity would be valuable.

Questions for authors
  1. (Clarification — could affect my assessment) Have you tested how stable the selected neuron set is with respect to prompt sampling? Concretely: redraw 200 harmful and 200 benign prompts from the same sources, rerun the full identification pipeline, and report the Jaccard overlap of the new top-2500 set with the original, along with how much the defense metrics change.

  2. (Clarification — could affect my assessment) On Llama-2 in Table 2, TRIPWIRE at top-2500 has 15.0% over-refusal versus 8.8% for probe z=3.0z=3.0. How do you account for this? Is there something about Llama-2's alignment training or activation landscape that makes the funnel's utility-specificity filter less effective on this model?

  3. (Clarification — could affect my assessment) What happens if you train the same logistic-regression detector on probe-selected neuron activations and run the same detect-then-clamp inference pipeline? This would directly test whether the funnel's value lies in the neuron set it produces or whether any reasonable neuron set works once gated by a detector.

  4. (Suggestion) One thing I am personally curious about: now that the funnel has identified these safety-specific neurons, have you looked into why they are safety-specific? For instance, is this property a direct result of alignment training (RLHF / DPO), or do these neurons already show signs of it during pretraining? Do the neurons selected across different models have any functional correspondence beyond the statistical level? I think even a preliminary analysis along these lines would add a lot to the paper's contribution.

Reviewer_dUkBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

the authors present Tripwire, a training-free jailbreak defense that detects safety neurons through statistical hypothesis testing and clamps their activations to induce refusal behavior. It supports detector gated inference and permanent weight editing and achieves large reductions in attack success rates on four LLMs.

Critical strengths

In the paper an interesting approach to jailbreak defense is proposed: existing refusal behavior is triggered instead of harmful representations suppressed. The neuron identification procedure and two deployment options are practically motivated. Strong defense performance across multiple models and attacks is obtained (utility degradation in detector-gated setting relatively small). Comparison with probe based neuron selection also provides useful evidence for effectiveness of the proposed identification approach.

Critical weaknesses
  1. The statistical certification of safety neurons is not well justified. BH-FDR prevents false discoveries in statistical activation-difference tests under certain assumptions but doesn’t say that the selected neurons are causally responsible for refusal behavior; it is also not clear if FDR guarantee holds for the final set of neurons after utility-specificity filtering and top N selection. The paper does not provide ablations isolating the individual contributions of direction filtering, significance testing, and utility-specificity filtering.
  2. The safety–utility trade-off in the permanent weight-editing setting is not as good as the overall claims of the paper would have you believe. For example, on Llama-3.1-8B large editing reduces ASR to 0.3% but MT-Bench goes from 6.60 to 4.84 and over-refusal increases to 31.3% (Table 1). Detector-gated largely preserves utility whereas the former does not. Even though the authors separate these deployment modes, results show that the favorable utility results in the inference setting appear to depend substantially on detector gating.
  3. Robustness against defense-aware attacks is not well established either. Four jailbreak methods are evaluated but it is not clear whether these attacks are optimized against the complete defended system (inference time defense only uses a detector trained on a small set of harmful and benign prompts; robustness to adversarially crafted inputs is not evaluated separately). This leaves open whether attackers could bypass intervention by evading the detector.
Questions for authors
  1. Can the authors confirm that FDR guarantee holds for the final set of selected neurons after all filtering and ranking steps? Can they show component-wise ablations or causal intervention experiments to establish role of the selected neurons in refusal behaviour?
  2. How do the authors explain such large utility degradation and over-refusal in the large weight-editing configuration? How much of the reported utility improvements come from the detector and not the neuron-selection procedure?
  3. Are the jailbreak attacks optimized against the defended models? Can the authors evaluate adaptive attacks on the detector-gated intervention and report the detector performance on held out adversarial prompts?
Reviewer_ZqN9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TripWire, a neuron-level defense against LLM jailbreaks. It selects candidate safety neurons using statistical testing, utility filtering, and discrimination-based ranking, then clamps their activations to harmful-input means to encourage refusal. The intervention can be deployed through detector-gated inference or a permanent bias patch. Experiments examine attack resistance and utility across four aligned LLMs.

Critical strengths

The paper addresses the practically important safety–utility trade-off with a compact intervention that avoids fine-tuning the base model. The distinction between suppressing harmful representations and activating existing refusal behavior provides an intuitive motivation. Combining neuron selection with an explicit utility filter is a useful design choice, while the bias-patch formulation offers a straightforward deployment option. The evaluation across multiple model families and attack types, together with the comparison against probe-based selection, provides encouraging evidence that the selected neurons are useful intervention targets.

Critical weaknesses

(1)A crucial missing baseline is to use the same detector and return a fixed refusal whenever it triggers. Without this comparison, it is unclear whether the neuron intervention provides an advantage over ordinary detect-and-refuse filtering. Detector precision, recall, threshold sensitivity, and conditional defense success should also be reported. (2)The evaluation should explicitly distinguish attacks transferred from undefended models from attacks optimized against the complete defense. Fixing selected activations prevents their direct suppression, but does not guarantee that other computational pathways cannot circumvent refusal. Adaptive evaluation should separately target the gating decision and the permanently edited model, with clearly specified attack budgets. (3)Table 1 shows substantial utility degradation in the large-budget offline configuration; for Llama-3.1, the MT-Bench score decreases from 6.60 to 4.84. The favorable utility characterization should therefore be restricted to the configurations that support it. Broader capability tests and semantic over-refusal evaluation are also needed; keyword matching cannot reliably distinguish refusal from legitimate explanations or statements of inability.

Questions for authors

Please refer to the weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.