acceptodds
Under review as a conference paper at ICLR 2027

Modulating LLM Behavioral Preferences by Reweighting Just a Few Weights

Abstract

Post-training can instill dispositional behaviors in large language models (e.g., refusal & sycophancy behaviors). We find that such behavioral preferences can be removed or mitigated by removing or downweighting just a few corresponding weights, or even amplified by upweighting the few. Our proposed approach, BLADE (Behavioral Localization via Activation-Difference Estimation), is a gradient-free, forward-only method that scores weights and identifies those that contribute to behavioral preferences across layers, enabling removal or modulation of an LLM’s behavioral preferences to a desired direction. We show that, through extensive experiments on various behaviors and models, our approach effectively and efficiently modulates the model’s behavioral preferences.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLMs, Behavioral preferences, Weight reweighting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WJ2rOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BLADE, a gradient-free method for locating and reweighting sparse residual-writer weights that modulate specific behavioral preferences in LLMs. Using contrastive behavior data, BLADE estimates a behavior direction, scores individual weights by their local contribution to that direction, penalizes weights that are important for general language modeling on C4, and then rescales the selected weights by a factor α\alpha. The method can suppress, remove, or amplify behaviors such as refusal, hallucination, sycophancy, power-seeking, and wealth-seeking. Experiments across several models show that BLADE can substantially modify these behaviors while often causing only small changes in WikiText perplexity. The paper also evaluates out-of-distribution refusal and hallucination benchmarks and demonstrates joint control of refusal and uncertainty.

Critical strengths
  1. The paper proposes a simple and computationally efficient post-hoc editing framework. BLADE does not require gradient-based fine-tuning or retraining the model. It uses forward-pass activation statistics to identify a sparse set of weights and applies a transparent multiplicative intervention. This makes the method relatively easy to implement and allows both suppression and amplification of the same behavior.
  2. The experiments cover multiple behaviors, model families, and intervention directions. The paper evaluates refusal, uncertainty, sycophancy, deception, power-seeking, and other behavioral preferences across several LLMs. It also compares BLADE with weight-editing and activation-steering baselines, and reports behavior metrics together with perplexity and normal-answer coverage. The OOD experiments on HarmBench, XSTest, SelfAware, and FalseQA provide some evidence that the edits can transfer beyond the calibration data.
Critical weaknesses
  1. The behavioral parameters are not shown to be disentangled, and the interventions can interfere with one another. BLADE identifies separate weight sets for different behaviors, but these sets may overlap or affect shared computations. The joint refusal-uncertainty experiment already shows trade-offs: stronger interventions increase refusal but also increase over-refusal and reduce answer coverage. On Llama-3.2-3B, hallucination mitigation also substantially reduces answering on answerable questions. The paper does not provide a systematic overlap analysis, orthogonality test, or causal intervention matrix showing whether editing one behavior leaves the others unchanged.
  2. The evidence for robust OOD control is limited and strongly dependent on benchmark-specific calibration and evaluation choices. Although the paper evaluates HarmBench, XSTest, SelfAware, and FalseQA, the interventions are selected using related validation data and a C4 perplexity constraint, while the behavioral metrics rely partly on keyword matching and LLM judges. It remains unclear whether the same sparse weights control the underlying behavior across substantially different prompts, domains, languages, and evaluators, or whether the method mainly transfers surface patterns captured by the calibration sets.
Questions for authors
  1. How transferable are the behavior-specific weight masks and reweighting factors selected by BLADE? In the current experiments, the layer set, sparsity ratio, and reweighting factor appear to be selected separately for each model and target behavior using calibration data. Can the authors evaluate whether a mask and intervention strength identified on one behavior dataset or model transfer to a substantially different prompt distribution or model without re-selection?
6 comments
AuthorsOpenReview ↗

Rebuttal to reviewer WJ2r - W1 (1/2)

We thank the reviewer's feedback. It took some time for us to conduct experiments to respond with evidence. We address the concerns as follows:

Weakness

W1:

Thank you for this question. To examine behavioral trade-offs systematically, we constructed the full 8 × 8 causal intervention matrix on Llama-3.2-3B alongside a cross-method comparison of refusal-removal baselines. Together, they show that cross-behavior side effects stem from semantic and causal couplings among behaviors rather than overlapping weights—and among weight-editing methods, BLADE incurs the lowest collateral shift per unit of edit.

(1) Complete 8 × 8 Causal Intervention Matrix

We built an 8 × 8 causal intervention matrix on Llama-3.2-3B where rows denote the edited behavior and columns measure all eight behaviors (values in %; bold diagonal entries show target effects). To ensure strict comparability, all eight masks use the same sparsity ρ = 5×10⁻⁴ (207,618 weights) from a shared layer pool, evaluated alongside the unedited Base model and a same-size Random mask control:

Edit \ MeasuredRefusalPowerWealthDecep.Corrig.SelfAw.SelfRateSyco.ΔPPL
Base (unedited)98.076.060.768.763.561.376.791.3—
Refusal0.056.752.746.082.461.376.091.3+0.58%
Power-seeking96.056.051.369.370.660.074.084.0+0.77%
Wealth-seeking97.056.048.768.772.957.374.090.0+0.27%
Deception94.072.764.056.765.970.774.793.3+0.26%
Corrigibility96.068.057.368.069.460.069.380.0+0.42%
Self-awareness97.061.354.771.374.146.773.386.7+0.09%
Self-rate-highly99.068.058.772.060.058.770.080.7+0.10%
Sycophancy91.075.359.364.751.864.069.378.0+0.82%
Random mask (same size)98.074.762.068.762.462.778.091.3+0.05%

From this matrix, we observe that removing a behavior does introduce some side effects on other behaviors (median off-diagonal shift 3.3 points, mean |Δ| = 5.5 points; while the same-size random mask shifts behaviors by at most 1.3 points, +0.05% ΔPPL, and ΔPPL across all edited rows stays within +0.09% to +0.82%). However, as we show in Section (3) below, compared with existing refusal-removal weight editing approaches, our method removes the target behavior more cleanly while incurring substantially smaller collateral impact on other behaviors.

(2) Weights Are Separated, Yet Behaviors Remain Coupled

Aligning the pairwise Jaccard overlap matrix (reported in Section (3) of our Question 1 response) cell-by-cell with the causal matrix above shows that cross-behavior shift magnitude |Δ| is uncorrelated with mask overlap (Pearson r = +0.35, Spearman ρ = 0.31). In fact, the largest off-diagonal shifts occur in the refusal row (deception -22.7, power-seeking -19.3, corrigibility +18.9 points)—the exact mask with the lowest overlap (1.1%–2.1%) with all other behaviors. This mirrors Figure 2, where two masks with strictly zero overlap by construction still interfere under joint amplification. Thus, parameter-space separability ≠ behavioral independence: upstream circuits interact via downstream representations and semantic dependencies (e.g., suppressing refusal naturally increases corrigibility and lowers power-seeking), so behavioral trade-offs do not imply localization failure.

AuthorsOpenReview ↗

Rebuttal to reviewer WJ2r - W1 (2/2)

(3) BLADE Modifies Fewer Weights While Removing Refusal More Cleanly with Smaller Side Effects: Cross-Method Comparison

To directly evaluate how cleanly our method isolates refusal compared to prior weight-editing approaches, we evaluated all frozen configurations from Table 2 on Llama-3.2-3B (values in %; A/B directions fixed on Base; activation-steering baselines are omitted as generation hooks prevent comparable A/B log-probabilities and barely reduce refusal).

ConfigRefusalPowerWealthDecep.Corrig.SelfAw.SelfRateSyco.ΔPPL
Base98.076.060.768.763.561.376.791.3—
BLADE (ours), L=12, f=0.5%0.057.350.751.378.857.376.792.0+0.33%
Wei et al., q=p=0.5%25.065.354.748.075.366.769.392.0+1.25%
Arditi et al., s=12.050.746.738.082.456.074.092.0+0.94%

From this comparison, we highlight three findings:

  1. Cleaner isolation at full removal: At the comparable refusal-removal operating point—where our method reduces refusal from 98.0 to 0.0 versus 98.0 to 2.0 for Arditi et al. (s=1)—Arditi et al. incurs 1.5× larger collateral shifts than our edit (mean |Δ| = 13.9 vs. 9.4 points; max 30.7 vs. 18.7; larger on 5 of 7 behaviors and smaller on none) and nearly 3× higher ΔPPL (+0.94% vs. +0.33%). Meanwhile, Wei et al. already incurs 8.9 points mean shift and +1.25% ΔPPL while removing only 73 points of refusal.
  2. Lowest collateral cost per unit edit: When we normalize mean collateral shift by each percentage point of refusal removed, our method achieves the lowest cost at 0.096, compared to 0.122 for Wei et al. and 0.145 for Arditi et al. (s=1). By editing just 0.042% of network weights (173,015 parameters), we isolate refusal far more cleanly than applying a global rank-one projection across the embedding matrix and all 56 writer matrices (Arditi et al., s=1, our primary baseline per Table 2 and Reviewer Fr7s's request).
  3. Identical shift directions across methods: Across all three approaches, we observe every behavior moving in the exact same direction—lowering power-seeking, wealth-seeking, and deception, raising corrigibility, and leaving sycophancy flat—confirming that the trade-off stems from intrinsic behavioral relationships rather than our specific locator.

(4) Single-Behavior Boundaries and Revisions

Similarly, the trade-off in Llama-3.2-3B hallucination mitigation (lower answerable coverage when suppressing hallucinations) reflects a capacity-dependent abstention boundary rather than entangled weights: smaller models become more conservative near borderline prompts, whereas Qwen3-4B preserves 71% answerable coverage at +0.8% ΔPPL (Question 1, Section 4). Cross-model pipeline transfer and frozen OOD generalization are detailed in Question 1 (Section 2) and Weakness 2. We will add both the 8 × 8 causal matrix and cross-method table to our updated manuscript.

AuthorsOpenReview ↗

Rebuttal to reviewer WJ2r - W2 (1/2)

W2:

Thank you for raising this important question. In Weakness 2, you ask whether a fixed set of sparse weights truly controls the underlying refusal behavior once we shift the prompt distribution, harm domain, language, or evaluator, rather than simply overfitting to surface patterns in the calibration data.

To test this directly, we evaluated our method across all four axes using the exact frozen configuration from Table 2 of our paper. Specifically, we selected the weight mask, refusal direction, target layer, sparsity ratio, and steering coefficient solely on our in-distribution AdvBench/Alpaca calibration set under the C4 perplexity budget (beta = 5%). We did not re-select, re-tune, or re-threshold anything for the evaluations below—HarmBench, XSTest, MultiJail, and WikiText were used strictly as held-out test beds.

(a) Evaluator

To make sure our conclusions do not depend on a single judge or brittle keyword heuristics, we re-evaluated every response from Table 2 using a second, independent, and recently published LLM judge (Gemini-4-Hydrogen). Both judges scored byte-identical 400-character response windows under the exact same rubric, with outputs shuffled across methods so neither judge knew which configuration generated a given response.

MethodConfigHarmBench LLM1HarmBench LLM2XSTest LLM1XSTest LLM2
Base—89.088.09.67.6
BLADE (ours)L=12, f=0.5%3.02.00.80.4
Wei et al.q=p=0.5%19.520.50.80.4
Arditi et al. (s=0.5)s=0.531.036.55.23.6
Arditi et al. (s=1)s=117.018.02.01.6
CAAL=12, c=-0.286.587.04.83.6
ActAddL=16, c=-1685.587.05.66.4

Beyond matching on aggregate rates, the two judges agree closely on an item-by-item basis: they assign the exact same label on 97.6% of the 1,600 paired HarmBench judgments (Cohen's kappa = 0.95) and on 99.0% of the 2,000 paired XSTest judgments (Cohen's kappa = 0.87). Broken down by configuration on HarmBench, agreement stays between 93.5% and 99.5%. Notably, the lowest agreement (93.5%) occurs on Arditi et al. (s=0.5), which leaves the largest number of borderline, equivocal refusals, whereas BLADE and the unmodified base model both sit at 98.0%–99.6%. In other words, any slight disagreement comes from ambiguous model outputs rather than judge inconsistency. Furthermore, Table 2 already includes a third, non-LLM baseline evaluator—a string matcher calibrated against XSTest reference labels with a 0.4% false-positive rate—and the ranking across methods remains identical under all three evaluators.

(b) Cross-Lingual Transfer (Language Shift)

Next, we tested whether our English-calibrated edit transfers to another language without touching the weights. We evaluated the same frozen configurations on MultiJail [1], which pairs parallel human-translated harmful requests in English and Italian. Because Italian is officially supported by Llama-3.2, any remaining refusals cannot be blamed on the model failing to understand the prompt. Since both arms evaluate the exact same 150 requests across 7 configurations (2,100 paired labels in total), we compare them item by item using paired exact McNemar tests:

MethodEnglish (en)Italian (it)Delta (it - en)Paired Exact McNemar
Base88.792.7+4.0p = 0.180
BLADE (ours)48.039.3-8.7p = 0.035
Wei et al.64.773.3+8.7p = 0.019
Arditi et al. (s=0.5)65.383.3+18.0p < 0.001
Arditi et al. (s=1)57.359.3+2.0p = 0.736
CAA88.094.0+6.0p = 0.035
ActAdd88.091.3+3.3p = 0.332

BLADE achieves substantially lower refusal rates than every baseline in both languages (paired exact McNemar tests against the strongest baseline, Arditi et al. s=1: p = 0.009 in English and p < 1e-5 in Italian). Even more tellingly, our advantage over Arditi et al. (s=1) more than doubles from 9.3 percentage points in English to 20.0 percentage points in Italian. If BLADE were merely memorizing surface lexical patterns from an English calibration set, its effect could not become even stronger in a language that never appeared during calibration. This experiment also underscores why we never rely on keyword matching for our conclusions: an English string matcher drops to 0.0% on Italian responses across all seven configurations, whereas the LLM judge captures true refusal rates ranging from 39.3% to 94.0%.

Reviewer_tMHvOpenReview ↗
Rating
2
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper proposes a gradient-free, forward-only method for identifying and modulating a small subset of model weights associated with LLM behavioral preferences. BLADE estimates the contribution of individual weights to a target behavior using contrastive activation differences and behavior directions. By scaling the identified weights, the method can suppress or amplify targeted behaviors without additional training.

Critical strengths
  1. Simple and intuitive approach. BLADE combines contrastive activation differences with weight-level importance estimation to identify behavior-relevant weights. The method is gradient-free and does not require additional model training, making it relatively straightforward to implement.

  2. Extensive empirical evaluation. The authors evaluate BLADE across multiple behavioral preferences and four different LLMs, demonstrating its effectiveness in a variety of settings. The results are particularly compelling for refusal behavior, where modifying a very small fraction of weights leads to substantial behavioral changes.

Critical weaknesses
  1. Writing and presentation. The writing feels overly polished, with repetitive and sometimes generic statements.

  2. Insufficient justification. The paper introduces QijQ_{ij} as a measure of general model utility, but does not adequately explain why this formulation can reliably reflect the importance of individual weights to general model capabilities.

  3. Lack of component-wise ablations. The paper does not provide systematic ablation studies to validate the necessity of its key components, such as the behavior-aware scoring function, the utility penalty, and the effective-layer selection strategy. The existing layer-wise ablation is insufficient to establish the contribution of each component.

  4. Limited novelty. The proposed method is closely related to existing activation steering approaches. Moreover, translating activation steering into weight-level editing is not new, as explored in prior work such as Steer2Edit [1]. The paper does not clearly distinguish its technical contributions from these existing methods.

[1] Steer2Edit: From Activation Steering to Component-Level Editing

Questions for authors
  1. Why can QijQ_{ij} serve as a reliable measure of weight importance to general model capabilities? Is there any theoretical justification or empirical evidence?

  2. Could the authors provide component-wise ablations to demonstrate the individual contributions of the proposed scoring function, utility penalty, and layer selection strategy?

  3. What are the fundamental differences between BLADE and existing activation steering or steering-to-weight-editing methods, particularly Steer2Edit? What is the key technical novelty beyond these prior approaches?

9 comments
AuthorsOpenReview ↗

Rebuttal to reviewer tMHv - W1 and W2 & Q1 (1/3)

We thank the reviewer's feedback. It took some time for us to conduct experiments to respond with evidence. We address the concerns as follows:


Response to W1

Thank you for this candid feedback on our writing. Reading the manuscript back with your comment in mind, we think some passages may indeed contain redundancy. During a close read-through, we flagged a few specific spots we can tighten such as:

  • In Sec. 4.3 and Sec. 4.4: In Sec. 4.3 (Llama-3.2-3B), we state the same point—that BLADE achieves comparable behavioral control at a lower perplexity cost—three times in consecutive sentences. Similarly, in Sec. 4.4, the clause "at the cost of increased over-refusal and reduced answer coverage" appears twice within the same paragraph.
  • In Sec. 4.1 and Sec. 4.2: Sec. 4.1 closes with a summary sentence ("In summary, our approach can identify various behavior modulatory weights and remove the corresponding behaviors") that echoes the paragraph right above it, and Sec. 4.2 opens with a similar sentence.

If the paper is accepted, we will carry out a careful, line-by-line tightening pass for the camera-ready version. We will fill the freed-up main-text space with substantial new results and content—specifically incorporating the important new findings generated during the rebuttal (including the component-wise ablations for W3/Q2) into the main paper. We hope our following response and other results can address all of your concerns.


Response to W2 & Q1

We thank the reviewer’s question on this. What we can offer is a derivation of what QQ measures, an argument for why subtracting it reduces the capability cost of a fixed edit, and a matched measurement of that reduction. We set out all three below, and state at the end what we do not claim.

1. What QijQ_{ij} is

A single weight WijW_{ij} of a residual writer does one thing: it multiplies writer-input coordinate jj and adds the product into residual coordinate ii. Setting it to zero therefore removes exactly Wijxt,jW_{ij} x_{t,j} from output coordinate ii at token tt, and changes nothing else. What the later layers see, however, is not that raw amount. In a pre-norm transformer the first module to read the residual stream again is an RMSNorm, which multiplies coordinate ii by a learned gain γi\gamma_i and divides by a per-token scale σt\sigma_t. The amount actually passed on is γiWijxt,j/σt\gamma_i W_{ij} x_{t,j} / \sigma_t, and averaging its square over ordinary text gives Eq. (5) term for term:

Qij=Wij2γi2EC[xt,j2/σt2].Q_{ij} = W_{ij}^2 \gamma_i^2 E_{\mathcal{C}}\big[x_{t,j}^2 / \sigma_t^2\big].

So QijQ_{ij} is not a formula chosen for convenience. It is the mean squared disturbance that deleting that one weight sends into the representation the next block reads, measured on generic text. (Two details: the expectation divides per token rather than averaging x2x^2 and 1/σ21/\sigma^2 separately, because tokens with unusually large activations have both a large xx and a large σ\sigma; and which normalization layer reads which writer was verified per architecture — in Gemma the branch output is normalized before the residual addition, so the reader differs.)

Relation to Wanda. Dropping the γi2/σt2\gamma_i^2 / \sigma_t^2 factor leaves Wij2E[xj2]=1T(∣Wij∣⋅∥Xj∥2)2W_{ij}^2 \mathbb{E}[x_j^2] = \frac{1}{T}\big(|W_{ij}| \cdot \|X_j\|_2\big)^2 — the square of the Wanda importance score [2] — the standard forward-only, calibration-based measure of a weight's importance for post-training pruning, estimated in exactly the same way (a small sample of ordinary text, one forward pass, no gradients) and validated at scale. QQ is that measure with one correction: because a writer's output is read through a normalization layer, the same disturbance is quiet in a coordinate with a small gain and loud in one with a large gain, so QQ measures the disturbance where it is heard rather than where it is produced. Building on Wanda establishes a solid, well-validated foundation for this measure's effectiveness, while our correction for the normalization layer standing between a writer and its readers further improves the precision of this measurement and allows Wanda's theoretical and empirical justification to carry over directly.

Two properties of how we use it matter for the reviewer's question. QQ is computed once per model from a single forward pass over C4 (34,816 tokens) and is shared by all eight behaviors, so it cannot have been tuned per behavior; and it never sees behavior data, so it cannot encode the behavior it is meant to protect against.

AuthorsOpenReview ↗

Rebuttal to reviewer tMHv - W2 & Q1 (2/3)

2. Why subtracting QQ reduces the capability cost

Each weight carries a behavioral benefit CijC_{ij} and a general-ability price QijQ_{ij}, both computed per weight and additive over a selected set. Choosing kk weights so that the total benefit is as large as possible while the total price stays within an allowance has an exact answer: fix an exchange rate λ\lambda, rank by Sij=Cij−λQijS_{ij} = C_{ij} - \lambda Q_{ij}, and take the top kk. This is why the score is a subtraction and why selection is a single global top-kk rather than a search: the ranking rule is the solution of that trade-off, not an approximation to it. λ\lambda is the exchange rate, and because CC is linear in WW while QQ is quadratic, it carries units and cannot be set to 11; we fix it by matching the medians of the two nonzero distributions, which makes the subtraction scale-free and leaves nothing to tune.

Why this buys anything at all is a question about the geometry of the two scores, and it is measurable. At the configuration we report (L⋆=[12]L^\star = [12]; 173,015 of the 34,603,008 entries in that layer's writers), the mask chosen by CC alone and the mask chosen by C−λQC - \lambda Q share 129,625 entries; subtracting QQ swaps 43,390 of them out. The weights it swaps out carry 23.4×23.4\times the general-ability price of the weights that replace them, but only 1.52×1.52\times their behavioral benefit. Two things make that exchange possible and cheap:

  • Substitutes are abundant. The edit needs only 0.5%0.5\% of the candidate pool, so for almost any expensive weight there is a nearly equivalent and far cheaper one further down the CC ranking. If the budget were a large share of the pool there would be nothing to swap to.
  • The behavioral effect is saturated. Both masks drive refusal to zero, so the benefit given up buys nothing. In this regime trading benefit for price is free.

This reasoning also says where the penalty should not help: when the budget is too small for the behavior to saturate, the benefit given up is no longer spare. That is what we observe — at ρ=0.002\rho = 0.002 the penalized score removes less refusal than CC alone (Table B).

For completeness: the implementation also refuses any weight whose benefit does not cover its price (Sij≤0S_{ij} \le 0). In every configuration reported here the number of positive-net weights far exceeded the budget, so that rule never bound; the effect described above comes entirely from the re-ranking.

AuthorsOpenReview ↗

Rebuttal to reviewer tMHv - W2 & Q1 (3/3)

3. Empirical evidence

To show the effect of QQ, we hold the target layer (L⋆=[12]L^\star = [12]), weight budget, and behavioral intervention fixed on Llama-3.2-3B refusal, varying only the selection score. Importantly, QQ is estimated solely on C4 while capability cost (ΔPPL\Delta\text{PPL}) is evaluated on held-out WikiText, ensuring that our evaluation is free from circularity or data leakage. Refusal rates are measured via string matching under standard plain generation on both in-distribution prompts and 200 out-of-distribution HarmBench prompts.

Table A — Ablation at our reported operating point (ρ=0.005\rho = 0.005, 173,015 weights).

Selection scoreID refusalOOD refusalΔPPL\Delta\text{PPL}
Base (unedited)1.0000.880—
CC only (λ=0\lambda = 0)0.0000.000+0.34%
C−λQC - \lambda Q, with Q=squared WandaQ = \text{squared Wanda} (no γ/σ\gamma/\sigma)0.0000.000+0.10%
C−λQC - \lambda Q (full form)0.0000.000+0.05%
C−10λQC - 10\lambda Q0.2660.240+0.12%
Random mask (same budget)1.0000.895+0.20%

At an identical budget and behavioral endpoint, Table A establishes three findings:

  • Substantial perplexity reduction at zero behavioral loss: Subtracting QQ preserves complete refusal removal (0.000 ID and OOD) while cutting the perplexity increase by nearly 7×7\times (from +0.34% to +0.05%).
  • Both Wanda and the normalization correction contribute: Decomposing QQ shows that the squared Wanda term accounts for the initial reduction (+0.34% to +0.10%), while our downstream RMSNorm correction (γi2/σt2\gamma_i^2 / \sigma_t^2) halves the remaining perplexity cost again (from +0.10% to +0.05%).
  • Sanity checks rule out alternative explanations: Over-weighting the penalty (C−10λQC - 10\lambda Q) allows the general-ability price to dominate the ranking and prevents complete refusal removal (0.266 ID / 0.240 OOD), confirming that the penalty actively governs the trade-off and that median-matching places λ\lambda in a well-calibrated regime. Meanwhile, a size-matched random mask leaves refusal completely intact (1.000 ID), confirming the effect is not an artifact of parameter count.

Table B — Capability cost (ΔPPL\Delta\text{PPL}) across the full budget grid.

Selection scoreρ=.002\rho=.002 (69,206)ρ=.005\rho=.005 (173,015)ρ=.01\rho=.01 (346,030)ρ=.02\rho=.02 (692,060)
CC only+0.27%+0.34%+0.43%+1.23%
C−λQC - \lambda Q (squared Wanda)+0.13%+0.10%+0.25%+0.22%
C−λQC - \lambda Q (full form)+0.04%+0.05%+0.18%+0.45%
C−10λQC - 10\lambda Q+0.13%+0.12%+0.09%+0.12%
Random mask+0.17%+0.20%+0.22%+0.38%

Across the entire budget grid, subtracting QQ consistently outperforms both CC alone and the unnormalized Wanda baseline at every ρ\rho, confirming that our gains are not an artifact of a single favorable budget choice. In particular, across all budgets where refusal is fully removed (ρ≥0.005\rho \ge 0.005), our full formulation achieves the lowest perplexity cost (+0.05% to +0.45%, compared to +0.34% to +1.23% for CC alone), whereas C−10λQC - 10\lambda Q fails to remove refusal at all four budgets.

In summary, our theoretical derivation and matched experiments show that QQ has a clear, architecture-grounded physical meaning—measuring the downstream RMSNorm disturbance introduced by weight deletion—and that subtracting λQ\lambda Q with parameter-free median matching reliably preserves general model utility across budgets without sacrificing behavioral control. If the paper is accepted, we will incorporate the derivation in Sec. 1 and Tables A–B into the camera-ready version, which also directly resolves the penalty ablation requested in W3/Q2.

Reviewer_Fr7sOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows that several behavioral preferences exhibited by LLMs, such as refusals, sycophancy, deception, and self-awareness, can be suppressed or amplified by modifying a very small fraction of the model’s weights. The method they propose for identifying these modifications (BLADE) is gradient-free and only requires forward passes. Rather than directly intervene on activations, however, the method identifies the weights that contribute most to the direction associated with the target behavior, and modifies those weights.

Critical strengths

The paper's findings suggest that better behavior modification can be achieved through intervention on a small fraction of weights without impairing the model’s performance at a language modeling task. This intervention affects a smaller number of weights than previous studies’ (e.g. Arditi et al.) and the results presented here suggest a greater success rate at behavior suppression.

The method proposed here differs from typical activation steering and could prove to be a generally useful alternative in a variety of cases.

Critical weaknesses

The biggest weakness, in my opinion, is that several papers already show that small interventions can suppress refusals. To your credit, you cite some of the most important examples, including Arditi et al. (2024) and Wei et al. (2024). However, additional work demonstrates refusal control through similarly targeted interventions. Arditi et al. (2024) show that a single direction can largely mediate refusal, while subsequent work shows similar results with affine interventions (Marshall et al., 2024), multidimensional refusal cones (Wollschläger et al., 2025), and interventions targeting individual neurons (Kazemi et al., 2026). Thus, the basic finding that refusal can be controlled through low-dimensional or localized interventions is already established. Moreover, earlier work demonstrates substantial refusal suppression with limited degradation on several standard capability benchmarks, although preservation varies across models and evaluation measures (Arditi et al., 2024; Wei et al., 2024).

One could argue that your intervention is smaller than other like Arditi’s because it directly modifies a smaller number of values in the forward pass. But in another sense, it is more complex an intervention because Arditi’s is a rank-1 transformation over the matrices, whereas BLADE’s interventions can be higher rank. Single-neuron intervention is similarly small scale, and more localized than BLADE. I therefore believe the claim that BLADE offers a meaningfully smaller intervention is not particularly convincing.

Therefore, the remaining potential contribution is that BLADE suppresses refusals while deteriorating model behavior less than other methods. The paper offers some evidence to suggest this, but more evidence would be required for this claim to be convincing. The biggest issue here is that it is model performance after intervention is primarily assessed through perplexity of next-token prediction on WikiText. This is a reasonable metric to present, but I would like to see the modified model’s performance on a wider set of common benchmarks like MMLU, TruthfulQA, IFEval, and GSM8K.

References:

Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., & Nanda, N. (2024). Refusal in language models is mediated by a single direction. arXiv. https://doi.org/10.48550/arXiv.2406.11717. arxiv.org

Kazemi, H., Chegini, A., & Safi, M. (2026). A single neuron is sufficient to bypass safety alignment in large language models. arXiv. https://doi.org/10.48550/arXiv.2605.08513. arxiv.org

Marshall, T., Scherlis, A., & Belrose, N. (2024). Refusal in LLMs is an affine function. arXiv. https://doi.org/10.48550/arXiv.2411.09003. arxiv.org

Wei, B., Huang, K., Huang, Y., Xie, T., Qi, X., Xia, M., Mittal, P., Wang, M., & Henderson, P. (2024). Assessing the brittleness of safety alignment via pruning and low-rank modifications. arXiv. https://doi.org/10.48550/arXiv.2402.05162. arxiv.org

Wollschläger, T., Elstner, J., Geisler, S., Cohen-Addad, V., Günnemann, S., & Gasteiger, J. (2025). The geometry of refusal in large language models: Concept cones and representational independence. arXiv. https://doi.org/10.48550/arXiv.2502.17420.

Questions for authors
  1. What is the s = 0.5 configuration for Arditi et al.’s method in Table 2? In the original Arditi et al. paper, they fully orthogonalize the refusal direction. The comparison should therefore use full orthogonalization as well, but s=0.5 implies it may be partial in these tests.

  2. Figure 3 says tests are “evaluated once on held-out HarmBench,” but Appendix A 2.2 says “For refusal boosting, we select configurations based on the HarmBench [Mazeika et al. (2024)] refusal rate under prefilling attacks” – which suggests HarmBench was not held out.

7 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer Fr7s - W1 & W2 (1/3)

Weakness

We thank and value the reviewer's feedback. It took some time for us to conduct experiments to respond with evidence. We address the reviewer’s concerns as follows.

W1: Prior work on refusal localization

Thank you for pointing out these related studies. While [1]–[4] are important prior results, our work differs from them in scope, mechanism, and broader implications:

1. Generalizing beyond refusal-specific methods

All four prior methods are designed specifically around refusal (e.g., an affine refusal map [2], a refusal cone [3], or a refusal neuron [4]). We do not claim priority over refusal suppression itself. Instead, our goal is to show that a single, uniform localization procedure applies across distinct post-trained behaviors (specifically refusal and epistemic uncertainty in our experiments). To the best of our knowledge, [1]–[4] have not shown to transfer out-of-the-box behaviors to other post-trained ones they were not originally constructed for.

2. Weight parameter-space localization vs. activation-space steering

Methods [1]–[4] identify directions or units in activation space, which characterizes how a behavior is represented and how it can be steered at inference time. By contrast, BLADE identifies the specific individual weight parameters that encode the behavior during post-training. Knowing that a behavior is linearly decodable or steerable at runtime does not tell us which parameters store it, nor vice versa. Operating in weight space directly and most precisely (in terms of observation and intervention granularity) reveals where post-training encodes the behavior.

3. Continuous scalar modulation rather than binary removal

Because we isolate a parameter subset rather than an inference-time direction, we can scale the located weights continuously using a single scalar . Decreasing toward zero smoothly weakens the behavior, while increasing above 1 amplifies it, with general capability metrics remaining flat across the range. Such smooth scaling across shows that these weights modulate the post-trained behavior without disrupting core computation. Such continuous, bidirectional control is infeasible with binary masking or hard ablation.

4. Implications for post-training dependability

More broadly, we view these results as a finding about how post-training works rather than simply an attack or refusal-removal method. Rescaling just 0.013% of writer weights is sufficient to dial a post-trained behavior from absent to amplified without degrading reasoning or knowledge. This suggests that current post-training procedures store behaviors in a remarkably shallow, compact subset of parameters that the rest of the network does not rely on. While [4] offers independent evidence of this sparsity for refusal, our distinctive contribution is showing that this property holds more generally across post-trained behaviors and can be directly isolated and edited in weight parameter space.

We also updated and included the discussion about them in the related work of the revised manuscript.


W2: “One could argue that your intervention is smaller than other … more localized than BLADE.”

Thank you for the detailed feedback and for suggesting the additional capability benchmarks.

First, we respectfully disagree with the premise that a rank-1 update is inherently smaller or simpler than BLADE. Matrix rank constrains the geometric structure of a perturbation, not its magnitude or parameter footprint. When we directly measure edit size on Llama-3.2-3B-Instruct—matrices touched, weights modified, and relative Frobenius norm—BLADE and Arditi et al. [1] (s=1) have the exact same overall perturbation magnitude (0.0199), while BLADE modifies 7,876x fewer weights (173,015 across 2 matrices vs. 1.36B across 57 matrices). Rather than claiming a smaller total norm, we note that because the two methods are tied by perturbation energy, comparing them constitutes a controlled experiment that isolates where the edit is placed—which is our core contribution.

1. Rank is not a measure of intervention size

Rank constrains the shape of a perturbation, not its magnitude. For any scalar c, the update W=-crrW remains rank-1 regardless of how large c is, and can therefore be arbitrarily destructive. Your observation about the rank of our update is mathematically accurate: a sparse mask covering 0.5% of a matrix's entries is numerically full rank, and we impose no rank constraint. However, an edit that is full rank, supported on just 0.013% of writer parameters, and equal in Frobenius norm to a rank-1 edit directly demonstrates that rank and edit size are independent properties.

AuthorsOpenReview ↗

Rebuttal to Reviewer Fr7s - W2 (2/3)

2. Direct measurements of edit size and concentration

Because intervention "size" can be measured either in weight space or in activation space, we report two complementary relative norms across all methods on Llama-3.2-3B-Instruct, computed over a common denominator (the token embedding plus all 56 residual-stream writers, totaling 1,362,886,656 parameters across 57 matrices):

  • Weight-space relative norm (δ = ‖ΔW‖_F / ‖W‖_F): The standard matrix perturbation measure (as used in LoRA [6, Sec. 7.3], which reports ‖ΔW_q‖_F = 6.91 against ‖W_q‖_F = 61.95).
  • Data-aware output relative norm (ε = ‖ΔWX‖_F / ‖WX‖_F): The relative change in matrix outputs evaluated on C4. Measuring edits by their output change on data has been widely used in weight-analysis literature such as Optimal Brain Damage, including SparseGPT [6] (which solves layer-wise reconstruction under the input Hessian XXᵀ + λI) and Wanda [7] (which scores weights by |W_ij| · ‖X_j‖₂, the diagonal approximation of that objective).
    These two norms are directly related via ‖ΔWX‖²_F = n · tr(ΔW Σ ΔWᵀ), where Σ is the input second-moment matrix. Thus, δ equals ε under the isotropic assumption Σ = I, and the two diverge when an edit aligns with directions that the data actually uses.
MethodMatrices#Weightsδεε/δ
BLADE (ours)2173,0150.01990.02681.35
Wei et al. [5]241,390,7570.06080.04670.77
Arditi et al. [1], s = 1571,362,886,6560.01990.02081.05

At matched weight-space norm (δ = 0.0199), BLADE has a slightly larger data-space output norm ε than Arditi et al. [1] (0.0268 vs. 0.0208), so we do not claim to be smaller on ε. Instead, the key diagnostic is the ratio ε/δ, which measures how strongly a perturbation aligns with active data directions:

  • [1] sits near 1 (ε/δ = 1.05): Ablating r̂ shifts outputs roughly as much as an unaligned perturbation of the same norm, as expected for a criterion that uses no input activation statistics.
  • BLADE reaches ε/δ = 1.35 by design: Our score, C_ij = max{(rℓ)_i W_ij (ΔμW)_j, 0}, is the behavior-contrastive analogue of Wanda's |W_ij| · ‖X_j‖₂. Both multiply each weight by an input-side activation statistic, except that ours uses the difference in means for the target behavior projected onto the behavior direction rather than unsigned magnitudes.
  • Values below 1 indicate poor localization: An ε/δ < 1 means the edit modifies weights along directions the data barely occupies. Wei et al. [5] falls into this regime (ε/δ = 0.77), applying a 3× larger weight perturbation (δ = 0.0608) while producing a smaller relative output shift (ε = 0.0467).

Inspecting individual matrices further highlights the difference in spatial concentration. Arditi et al. [1] shifts the output of every single one of the 57 writer matrices by a relative norm between 0.0116 and 0.0297 (a 2.6× spread from least to most affected), showing no selectivity across depth. By contrast, BLADE leaves 55 of the 57 matrices completely untouched (0.0000) and concentrates the entire edit in two Layer-12 matrices, shifting their outputs by 0.436 (attention output) and 0.132 (MLP output).

This contrast explains downstream preservation far better than global norms alone. BLADE changes Layer-12 attention outputs by 44% on general text yet costs only +0.3% WikiText perplexity, leaves MMLU unchanged (60.5 vs. 60.5), keeps GSM8K within noise, and reduces refusal from 89.0% to 3.0%. Meanwhile, [1] perturbs every layer's output by roughly 2%, yet incurs +1.0% perplexity, drops 6.4 points on strict-match GSM8K (t = -3.31), and leaves 17.0% residual refusal. Ultimately, collateral damage depends on where an edit lands rather than on its global magnitude—which is why matrix rank, bounding neither δ nor ε, is not the right axis for comparing these interventions.

AuthorsOpenReview ↗

Rebuttal to Reviewer Fr7s - W2 (3/3)

3. On single-neuron interventions [4]

Minimizing the overall parameter count was never our claim, but the comparison needs one correction: [4] is not a smaller weight edit — it is not a weight edit. It pins one MLP neuron to a constant m at every token position with a forward pre-hook (their Fig. 13: “No weights are modified”), which is exactly zeroing that neuron’s input-dependent contribution plus injecting the fixed vector m · W_down[:, i] at every position. The second term is additive activation steering, and in a bias-free MLP it has no weight-space realization at all. Its footprint is therefore zero parameters plus a permanent inference-time hook, with no standalone weight diff; it belongs to the activation-steering family that Tab. 2 already compares against.

The distinction shows up in exactly the cost you ask about. By their own Table 1, the constant intervention costs -8.8% MMLU on average and -18.2 points on Llama-3.1-70B, because a constant clamp pushes the activation outside the model’s learned distribution. Their anchor variant recovers this (-0.6% MMLU) but is no longer single neuron or one constant: it needs an extra hook-free forward pass per prompt, the harmful–harmless activation gap, and a further sweep over k. BLADE leaves MMLU at 60.5, moves no capability metric significantly, costs +0.3% perplexity, and takes refusal from 89.0% to 3.0%.

Also, their score does not by itself identify the neuron — the top-5 candidates span 21–95% ASR, reranking is “necessary”, and m* comes from an empirical sweep on a harmful validation benchmark (their Limitations) — whereas our selection uses in-distribution contrast data under a perplexity budget and no harmful benchmark. In addition, their objective is the log-odds of model-family-specific refusal phrases (their Table 4), which has no counterpart for epistemic uncertainty or sycophancy.

We also updated the corresponding description in the updated manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.