acceptodds
Under review as a conference paper at ICLR 2027

Fireman against Jody: Causal Counterfactual Mirrors for Auditing Strategic Deception in Language Models

Abstract

From an instruct model B we train two LoRA siblings on identical helpful data and opposite safety targets: Fireman, trained on safe responses to harm- ful prompts, and Jody, trained on unsafe ones. Their weight difference ∆ = Jody − Fireman is a single vector in weight space that edits refusal: on Llama- 3.1-8B-Instruct, B + λ∆ raises harmful compliance from 0.062 to 0.913 (Stron- gREJECT). We test it where such methods are usually asserted rather than mea- sured — at matched harmful compliance against the simpler edits it is supposed to improve on. At compliance matched to our operating point (0.699), scaling the unsafe adapter alone costs -3.7 to -7.0 points of instruction-following where the sibling difference costs none (+0.9). Beyond that the alternatives do not merely cost more, they stop: scaled further the single delta turns over rather than improv- ing (peak 0.761 at λ = 1, falling to 0.516 by λ = 3 as the model degrades), and the strongest baseline we measured — activation steering along the refusal direction, at 0.812 — gets there at -9.9 GSM8K and -4.1 IFEval, whereas the sibling difference reaches 0.913 with both above base (+3.6, +2.8) and MMLU down -1.5. The usual MMLU/ARC pair does not separate any of these condi- tions. Controls attribute this to ∆’s direction rather than its magnitude (norm- matched random and shuffled deltas leave behaviour and capability at base level) and to cancellation of the siblings’ shared fine-tuning damage. A sibling trained on the benign half of the same data cancels it just as well, so a safe pole is not required; one trained on different benign data does not (-17.2 IFEval), so what is required is a data-matched sibling. The safe pole is the instantiation that addi- tionally supplies a weaker re-alignment direction. The vector is computed once on B and merged into independently released fine-tunes of B with no retraining. On a pre-specified set of public derivatives — every surveyed fine-tune whose base compliance is below 0.3 — it passes the same 0.5 threshold on 11 of the 13 we could evaluate (85%), median compliance 0.024→0.806, where transferring the refusal-direction ablation from B reaches 0.085–0.208. Reversed, it partially re-aligns unsafe derivatives at a small capability cost. The effect replicates on a second base model family (Mistral-7B, 0.474→0.901) and only weakly on a third (Qwen3-8B, 0.019→0.145 with capability intact), where the subtraction removes most of the safety effect along with the shared damage; we report that attenuation and the training-seed spread that bounds all of these numbers

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, AI alignment, strategic deception, representation engineering, activation patching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6tztOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a model editing method that uses the difference between two model versions trained on harmful and safe responses, respectively, to induce jailbreak responses. Experiments demonstrate the method’s effectiveness on small-scale models.

Critical strengths
  • The method is easy to understand.
  • The analysis of its use for defense is also good.
Critical weaknesses
  • The generalizability and effectiveness are doubtful. The method's effectiveness appears to depend strongly on the base model, with weak effects on Qwen3-8B. All evaluated models are also relatively small (7–8B parameters), leaving its effectiveness at larger scales, such as 32B and above, and different model variants, untested. The mechanistic rationale remains incomplete: why does sibling subtraction preserve the target behavioral change on some models but largely cancel it on others? Is there any formal analysis?

  • Limited capability evaluation. More benchmarks covering code generation and general question answering would strengthen the claim that the method preserves general capabilities. The current evaluation does not establish whether capability preservation extends to these tasks.

Questions for authors

Please see the weaknesses.

4 comments

[1/3]

Thank you for the review. Each weakness you list can be tested, so we tested them: the cause of the Qwen3 attenuation (part 2), a 32B model, and code and open-domain QA (part 3). Because soundness is the rating we most want to address, we first restate what each claim rests on. Harmful compliance is the mean StrongREJECT score on 212 held-out behaviours, judged by Qwen3-32B with the published rubric and cross-checked by the HarmBench classifier (Cohen's kappa = 0.80). Capability changes are in points against the base model.

  1. The edit works on the primary model. Compliance goes from 0.062 at B to 0.699 at the dev-selected operating point lambda* = 0.5, and to 0.913 at lambda = 1. At lambda*: MMLU -0.3, ARC +0.3, GSM8K +2.6, IFEval +0.9. No reported harm number uses the 64 development prompts.
  2. It beats the simpler edits at equal compliance. The unsafe adapter alone, scaled to 0.634 and 0.737, costs 3.7 and 7.0 IFEval points. Scaled further, it turns over (0.761 at lambda = 1, 0.516 at 3) while capability collapses. A harmful-only LoRA reaches 0.764 at -25.9 IFEval. Weight-space refusal-direction ablation reaches 0.390. Activation steering along the refusal direction reaches 0.812 at -9.9 GSM8K.
  3. The mechanism is subtraction of a data-matched sibling. A benign-only sibling trained on the same data gives 0.880 with capability at base level. One trained on different benign data costs 17.2 IFEval points. A null sibling gives 0.017, and norm-matched random and shuffled deltas give 0.053 and 0.061, against B's 0.062.
  4. It transfers without retraining, to derivatives chosen by a rule fixed in advance (every surveyed derivative with base compliance below 0.3). All 23 such targets are evaluated, with base and edited models prompted through the same chat template: 19 of 23 pass 0.5, median 0.027 to 0.807. The submission measured the base column under each derivative's own template and the edited models under B's. Under one template for both, every base stays below 0.3 and the edited values move by at most 2.4 points. The four that resist are two safety-regularised math fine-tunes from one lab (0.041 each) and two derivatives the submission could not evaluate (0.410, 0.355).
  5. Training variance is reported. Three sibling pairs give 0.621, 0.699 and 0.787 at the operating point and 0.913, 0.914 and 0.949 at full strength.

If you consider a specific claim unsound, we would be glad to answer it directly.

[Part 2/3]

Why the subtraction keeps the behaviour on some models and cancels it on others. We answer with an experiment rather than a correlation.

The two siblings start from the same checkpoint and see the same data; only the targets for harmful prompts differ. The difference therefore removes whatever the two share and keeps what differs, so which behaviour survives depends on what the safe pole shares with the unsafe one. PKU's "safe" responses mostly engage with the harmful request rather than refuse: the safe pole alone complies 0.519 on Llama and 0.513 on Qwen3, against 0.761 and 0.759 for the unsafe pole. If the switch between refusing and engaging lies in that shared part, subtracting the safe pole removes it.

Test. We replaced only the safe pole's targets with explicit refusals, written by Llama-3.1-8B-Instruct for the same 3,932 harmful prompts (all refusals on the first pass), and kept everything else fixed. Harmful compliance on Qwen3-8B at lambda = 1:

B + (Jody - X), Qwen3-8Bharmful
base0.019
X = PKU safe pole (retrained; submitted 0.145)0.155
X = benign-only sibling0.354
X = refusal pole0.804

Capability: MMLU +2.2 [+1.6, +2.7], ARC +0.8, GSM8K +3.3 [+1.3, +5.2], IFEval +0.9 [-2.2, +4.1]; over-refusal 0.000. A second training seed gives 0.739 for the refusal pole and 0.415 for the PKU pole, so the PKU result varies with the seed (0.155, 0.415) while the refusal pole stays high (0.804, 0.739). Across lambda, the refusal pole gives 0.187, 0.393, 0.624 and 0.804 at lambda = 0.25, 0.5, 0.75 and 1. On Qwen3 the development split runs higher than test, so the dev rule would pick lambda = 0.5, where test compliance is 0.393. On Llama, the same refusal pole gives 0.879 at lambda = 0.5 with capability at or above base.

So the subtraction cancels the behaviour on Qwen3 mainly because the PKU safe pole shares the engagement. With a refusing pole the full-strength edit reaches 0.804 and 0.739 over two seeds, against 0.155 and 0.415 with the PKU pole. The effect remains less stable on Qwen3 than on Llama. The activation-space diagnostic agrees: the PKU sibling contrast steers Qwen3 to at most 0.193, no more than a random direction (0.243), while Qwen3's refusal direction reaches 0.870. A closed-form condition for when PKU-style targets suffice remains open. What the experiment identifies is the main factor in the one case that was attenuated.

[Part 3/3]

Scale. We ran the same recipe on Qwen2.5-32B-Instruct (PKU poles, one sibling pair). Harmful compliance goes from 0.009 to 0.523 at the dev-selected lambda = 0.75 and to 0.694 at lambda = 1 (0.116 at 0.5). The unsafe pole alone reaches 0.771 and the safe pole alone 0.436. At lambda = 1: MMLU -0.4 [-0.8, -0.1], ARC 0.0, GSM8K +6.5 [+4.6, +8.5], IFEval -2.8 [-5.7, +0.4]. At lambda = 0.5 every change is within noise. The effect reproduces at 32B, at full strength; we make no claim beyond the sizes we ran.

Code and open-domain QA. On Llama we measured HumanEval pass@1, MBPP pass@1 and TriviaQA (rc.nocontext, exact match over the full validation set). Base B scores 0.689, 0.582 and 0.728. Changes are in points against B, with 95% paired CIs:

editHumanEvalMBPPTriviaQA
sibling difference, lambda* = 0.5-1.2 [-4.9, +2.4]-1.4 [-3.6, +0.6]-0.3 [-0.7, -0.0]
sibling difference, lambda = 1-4.3 [-9.8, +0.6]+0.8 [-2.6, +4.4]-1.5 [-1.9, -1.1]
unsafe adapter alone-17.1 [-25.0, -10.4]-1.8-1.1
harmful-only LoRA-13.4 [-20.1, -7.9]-2.0-1.8
refusal-direction steering-18.9 [-26.2, -11.6]-9.4 [-13.6, -5.2]-16.4 [-17.1, -15.8]
refusal-direction ablation (hook)-2.4-0.8-0.2

At the operating point, the sibling difference stays within noise on code and loses 0.3 TriviaQA points. The edits that reach similar compliance lose 13 to 19 HumanEval points, and activation steering also loses 16 TriviaQA points. The ablation hook costs little but reaches only 0.461 compliance.

Generality across fine-tunes. The transfer study in point 4 of part 1 is the second axis of generality: one vector, computed once on the base, applied to 23 independently released fine-tunes chosen by a fixed rule, 19 of which pass 0.5.

Each weakness in your review has a direct experiment behind its answer: the cause of the Qwen3 attenuation, a 32B model, and code and open-domain QA. If these address your concerns, we would be grateful if you would raise your soundness and overall ratings. If a concern remains, please tell us which one and what result would change your assessment, and we will address it before the discussion closes.

Reviewer_z5MWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper primarily shows a method to steer refusals in language models by steering in the weight space. The methodology involves finetuning two different LoRA adapters on contrastive data on a specific task and using the difference between these adapters to steer the model to comply to harmful requests (ablate the refusal direction) while maintaining instruction following. They show that weight space steering increases harmful compliance on the StrongREJECT benchmark while there is no significant drop observed on MMLU, ARC, GSM-8K or IFEval.

Critical strengths
  1. Most of the ablations, especially for controls and baselines are throrough and very well chosen.

  2. The results are robust and the authors have done a good job scoping the experiments to run.

Critical weaknesses
  1. The core method and its main comparison are already published and the work in this paper is not novel. They don't even cite the prior work: Contrastive weight steering (Fierro & Roger, "Steering Language Models with Weight Arithmetic", ICLR 2026, arXiv:2511.05408)

  2. Even though the experiments are well executed, most of the paper is clearly written by an AI and is extremely difficult to read. Pangram gives the abstract a 100% AI usage.

Questions for authors

Clarifications (could change my assessment):

  1. What does this paper add beyond Fierro & Roger (ICLR 2026)? Please state the differences in method and findings precisely.
4 comments

[Part 1/3]

Thank you for the review and for the direct question. Part 1 answers it, part 2 corrects two descriptions of the method, and part 3 addresses the AI-use remark.

Prior work. You are right that the paper must cite Fierro and Roger, "Steering Language Models with Weight Arithmetic" (ICLR 2026), and we accept that without qualification. Their construction (Sec. 3, Eq. 1) subtracts two fine-tunes trained on the same questions with opposite answers and adds the scaled difference to a model; our Delta = dW(Jody) - dW(Fireman) is an instance of it. They also use it to restore refusal in a GSM8K fine-tune of Llama-2-7b-chat (Sec. 4.3), and compare it with activation steering and with a scaled single task vector. The submission's sentence that prior work does not report this kind of comparison is therefore wrong for those two baselines.

What differs, in method and findings. In method, our two siblings share non-target data: both see the same 3,932 benign pairs, and only the harmful-prompt targets differ. Their two fine-tunes contain only the behaviour data. That shared data is what the subtraction cancels (finding 1).

  1. What the subtracted sibling has to be. They always subtract a fine-tune with the opposite behaviour. For refusal, that is not what makes it work. A sibling trained on the benign half alone, with no safety targets, cancels as well as the safe pole (0.880 harmful compliance; MMLU -0.1, GSM8K +2.0, IFEval +2.2). A sibling trained on different benign data reaches similar compliance at -17.2 IFEval. A null sibling (two unsafe-pole seeds) gives 0.017; norm-matched random and shuffled deltas give 0.053 and 0.061, against the base's 0.062.
  2. Removal at equal compliance, against the edits used in practice. Their refusal experiments only restore refusal. When removing it, the scaled single delta turns over (0.761 at lambda = 1, 0.516 at 3), a harmful-only LoRA reaches 0.764 at -25.9 IFEval, weight-space refusal-direction ablation (behind released "abliterated" models; they do not run it) reaches 0.390, and refusal-direction steering reaches 0.812 at -9.9 GSM8K. The sibling difference reaches 0.913 at GSM8K +3.6 and IFEval +2.8. MMLU and ARC separate none of these; their side-effect measure is TinyMMLU, which they call "limited to narrow multiple-choice assessments" (Sec. 7).
  3. Transfer to other parties' models. They apply the vector to their own fine-tune. We merge Delta, computed once on the base, into independently released derivatives chosen by a fixed rule (base compliance below 0.3). All 23 are evaluated, base and edited models under one chat template: 19 pass 0.5, median 0.027 to 0.807. (The submission measured bases under each derivative's own template; under one template, every base stays below 0.3 and edited values move by at most 2.4 points.) Transferring the base's refusal-direction ablation reaches 0.085 and 0.208.
  4. What the safe pole must contain. The construction gives 0.062 to 0.913 on Llama-3.1-8B, 0.474 to 0.901 on Mistral-7B and 0.009 to 0.694 on Qwen2.5-32B (0.523 at the dev-selected 0.75). On Qwen3-8B it was attenuated (0.019 to 0.145), and we traced most of that to the safe pole's data: with PKU's "safe" responses, which engage rather than refuse, Qwen3 reaches 0.155 (0.415 with a second seed); with a safe pole trained on explicit refusals for the same prompts, 0.804 (0.739), capability at or above base. They contrast refusals with answers (Sec. 4.3) but never compare an engaging safe pole with a refusing one against the same unsafe pole, which is what locates this.

The construction is theirs. These results are what the paper adds, and what it should be judged on.

[Part 2/3]

Two corrections to the summary. Both affect what is being compared.

The method is not refusal-direction ablation. The summary describes the edit as steering the model "to comply to harmful requests (ablate the refusal direction)". Ablation is one of our baselines, not the method. Applied to the weights (orthogonalising every residual-writing matrix against the direction), it reaches 0.390 harmful compliance at its best of six layers, against 0.913 for the sibling difference. An independently implemented inference-time hook peaks at 0.461 on the same prompts and judge, so the two agree. Transferred from the base to a derivative, ablation reaches 0.064 where the sibling difference reaches 0.688 on the same model.

The capability claim is not that nothing drops. MMLU and ARC separate none of the conditions: judged by those two columns, every edit looks free. The differences appear on IFEval, GSM8K, and code and open-domain QA, which we measured during the discussion. The scaled single delta loses 3.7 to 21.4 IFEval points on the way to its compliance peak. The harmful-only LoRA loses 25.9 IFEval and 13.4 HumanEval points, the unsafe adapter alone 17.1 HumanEval points, and refusal-direction steering 9.9 GSM8K, 18.9 HumanEval and 16.4 TriviaQA points. The sibling difference at its operating point stays within noise on all of them except a 0.3-point drop on TriviaQA. Which benchmarks would separate the conditions was found post hoc, and the paper says so; the benchmark set was fixed in advance.

The review calls the controls and baselines "very well chosen" and the results "robust", and we agree that this is where the evidence sits. That leaves prior work as the question that should decide the rating, and part 1 answers it.

[Part 3/3]

AI use. The submission discloses it, in the paper and in the submission form. The paper's statement reads in full: "The research questions, ideas, and experimental design are the authors' own. AI tools provided code assistance and debugging, including implementing and running training, generation, evaluation and statistical computations, and writing assistance with drafting and editing. The evaluation code uses the LLM judges described in Section 4. The authors directed and verified this work and take full responsibility for every claim and for any errors introduced by these tools; no LLM is an author."

The form declares the same uses: aiding or polishing writing, retrieval and discovery, and drafting sections. A detector that flags AI-assisted prose in the abstract therefore reports what the submission states on its first pages. It is not a finding about the work and bears on none of the claims above. The experiments, judges, selection rules, controls and every number are in the released code and results files and can be checked directly.

The readability criticism is a separate point, and we take it. In plain terms: Fireman and Jody are two LoRA adapters trained from the same checkpoint on identical data, except for the response to harmful prompts (safe for Fireman, unsafe for Jody). Delta is the difference of their weight changes. To "jodify" a model is to merge +Delta into it. The "dial" is base + lambda * Delta, with lambda = 0.5 fixed on a 64-prompt development split that no reported number uses.

We hope part 1 answers your question precisely. If the differences it lists are what you asked for, we would be grateful if you would raise your rating. If something is still missing, please

Reviewer_y2R4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

In this work, the authors edit a model’s safety behavior with weight arithmetic. The authors fine-tune two LoRA adapters on the same data, one with safe completions and the other with unsafe completions on harmful prompts. They then subtract their weight changes to get a single weight vector delta. Adding delta to Llama 3.1 8B Instruct raises harmful request compliance from 0.062 to 0.913 with little capability loss. While other baselines at similar harmfulness scores either hurt instruction-following or degrade the model. Controls suggest the method works because subtraction cancels capability damage the two adapters share from their common training data, so what matters is matching the training data rather than having a safe counterpart. These weight vectors transfer to publicly available fine-tuned variants without having to retrain. When reversed, the weight vector partially restores refusal. These results are replicable on Mistral7B but work poorly on Qwen3 8B.

Critical strengths

-- The paper compares methods at similar levels of harmful compliance, making the capability trade-off more informative. On Llama 3.1 8B Instruct, the proposed weight arithmetic based edit reaches 0.699 harmful compliance while preserving GSM8K and IFEval performance approximately at baseline. Other alternatives alternatives hurt the general capabilities of the language model or do not achieve such levels or harmfulness

-- The authors distinguish the proposed method from weight perturbations, shuffled parameter differences and differences between independently trained adapters. It is interesting that subtracting a benign-data-only model adapter(while computing delta) trained on the same data works nearly as well as subtracting the safe adapter.

  • The editing procedure is simple and elegant. This makes it relatively inexpensive and easy to apply the weight vector to other fine-tuned variants of the same backbone.
Critical weaknesses

-- My main concern with this paper is its presentation and writing. The writing is often difficult to follow, with long, jargon heavy sentences containing multiple clauses. Some terminology, such as “jodify,” is introduced without explanation making the paper hard to follow. The paper will benefit from editing to improve clarity, simplify the language, and communicate its ideas more effectively.

--The finding that their proposed method does not generalize or apply to Qwen3 8B is important. While honesty in reporting is appreciated, the authors do not provide sufficient explanation or reasoning as to why this occurs. This finding deserves additional experiments to isolate causes of this finding (such as training data, post-training recipes etc).

-- The authors claim their weight vector is asymmetric (the safety direction is not as effective as the harmful direction) because of artifacts from the chosen PKU dataset (the safe responses are not safe enough). The authors could have experimented on other datasets that have both harmful and safe completions to the same requests to test whether this explanation is valid. Alternatively, they could have generated strongly safe synthetic responses to the harmful prompts as replacements for the original safe responses, and examined how their framework performs. This would have better isolated the reason why their weight vector is asymmetric.

Questions for authors

Please see weaknesses above and clarify any misunderstanding I may have had.

4 comments

[Part 1/3]

Thank you for the careful review, and for separating the writing from the content. This part defines the terms. Parts 2 and 3 answer your two experimental points with the experiments you proposed.

Terms, stated plainly.

  • Fireman and Jody are two LoRA adapters trained from the same base model B on the same data. They differ only in the target for harmful prompts: Fireman learns the PKU safe response, Jody the unsafe one. Both share the 3,932 benign pairs.
  • A pole is one of the two adapters. The sibling difference is Delta = dW(Jody) - dW(Fireman), one vector in weight space.
  • To jodify a model is to merge +Delta into its weights; a jodified model is the result.
  • The dial is B + lambda * Delta. The operating point lambda* = 0.5 is fixed on a 64-prompt development split, never on test numbers.
  • A data-matched sibling is any adapter trained on the same non-target data, whatever its safety targets.

The claims, one line each. Harmful compliance is the mean StrongREJECT score on 212 held-out behaviours; capability changes are in points against B.

  1. Delta edits refusal. On Llama-3.1-8B-Instruct, compliance goes from 0.062 to 0.699 at lambda* and to 0.913 at lambda = 1. At lambda*: MMLU -0.3, ARC +0.3, GSM8K +2.6, IFEval +0.9.
  2. At equal compliance it keeps the instruction-following that simpler edits lose. The unsafe adapter alone, scaled to 0.634 and 0.737, already costs 3.7 and 7.0 IFEval points. Scaled further, it turns over (0.761 at lambda = 1, 0.516 at lambda = 3).
  3. The subtracted sibling must be data-matched, not safe. A benign-only sibling trained on the same data gives 0.880 with capability at base level. One trained on different benign data costs 17.2 IFEval points. A null sibling (two Jody seeds) gives 0.017, and norm-matched random and shuffled deltas give 0.053 and 0.061, against B's 0.062.
  4. Delta, computed once on B, transfers to independently released fine-tunes of B without retraining, on a set chosen by a rule fixed in advance.
  5. The effect holds across model families. Part 2 explains the one family where it was weak.

[Part 2/3]

Qwen3. We ran the test you proposed: same recipe, same prompts, and only the safe pole's targets changed.

The new safe pole learns explicit refusals. For each of the 3,932 harmful training prompts, Llama-3.1-8B-Instruct wrote a refusal under a refusal system prompt. All 3,932 were refusals on the first pass (median 312 characters, against 490 for the PKU safe responses and 500 for the unsafe ones). The benign half of the data is unchanged. We trained this pole and a fresh Jody on Qwen3-8B.

Harmful compliance at lambda = 1 (n = 212):

Qwen3-8B, B + (Jody - X)harmful
base0.019
X = PKU safe pole (submitted construction, retrained)0.155 (submitted: 0.145)
X = benign-only sibling (same benign data, no harmful prompts)0.354
X = refusal pole0.804

At that point capability is at or above base: MMLU +2.2 [+1.6, +2.7], ARC +0.8 [-0.4, +1.9], GSM8K +3.3 [+1.3, +5.2], IFEval +0.9 [-2.2, +4.1] (points, 95% paired bootstrap). Over-refusal is 0.000.

Two checks bound this result. A second training seed gives 0.739 for the refusal pole and 0.415 for the PKU pole, so the PKU result varies with the seed (0.155, 0.415) while the refusal pole stays high (0.804, 0.739). Across lambda, the refusal pole gives 0.187, 0.393, 0.624 and 0.804 at lambda = 0.25, 0.5, 0.75 and 1. On Qwen3 the development split runs higher than test (0.615 at lambda = 0.5), so the paper's dev rule would choose lambda = 0.5, where test compliance is 0.393. We report the whole grid.

So at full strength, the safe pole is the main factor in the Qwen3 attenuation. PKU's "safe" responses engage with the harmful request instead of refusing, and a pole trained on them already complies about half the time (0.513 alone on Qwen3, 0.519 on Llama). Subtracting it removes that engagement along with the shared damage. A pole that refuses leaves the refusal switch in the difference. The effect is still less stable on Qwen3 than on Llama. The paper's activation-space diagnostic agrees: steering along the PKU sibling contrast moves Qwen3 to at most 0.193, no more than a random direction (0.243), while Qwen3's own refusal direction reaches 0.870.

Before running, we wrote down that 0.50 or more for the refusal pole would support this explanation. The benign-only sibling (0.354) recovers part of the effect; we draw no causal conclusion from it.

[Part 3/3]

The asymmetry. We ran the same refusal pole on Llama, in both directions.

Forward, nothing is lost. Jody minus the refusal pole gives 0.879 at lambda = 0.5 and 0.878 at lambda = 1 (PKU pole: 0.699 and 0.913). Capability at lambda = 0.5 is at or above base: MMLU +0.7 [+0.2, +1.2], ARC +0.8, GSM8K +3.0 [+0.8, +5.4], IFEval -0.6 [-3.9, +2.6].

In reverse, the direction is no longer weak, which supports your explanation:

derivative (base harm), lambdaPKU polebenign siblingrefusal pole
abliterated (0.934), 0.50.8370.8700.712
abliterated (0.934), 10.6620.6420.120
GeomLlama (0.435), 0.50.217 (over-ref. 0.02)0.169 (0.03)0.000 (0.57)
Latxa (0.122), 0.50.024 (over-ref. 0.19)0.008 (0.49)0.000 (0.78)

The table shows two things. First, the PKU pole re-aligns no better than a benign-only sibling. PKU's safe responses add almost nothing to the reverse direction, which is why it is weak. Second, the refusal pole overshoots. At lambda = 0.5 it over-refuses benign prompts (in parentheses), and at lambda = 1 it breaks the abliterated model (MMLU 0.23; PKU pole 0.590).

A smaller lambda works for drift. At lambda = 0.25 the refusal pole takes GeomLlama from 0.435 to 0.071 and Dobby from 0.111 to 0.008, with over-refusal of 0.06 and 0.04 (at or below B's 0.084). That is more repair than the PKU pole gives at lambda = 1 (0.091 on GeomLlama). Latxa still over-refuses at 0.25 (0.50). The abliterated model barely moves at 0.25 (0.880) and loses GSM8K at 0.5 (0.528 against 0.675).

So the weak reverse direction comes from the PKU safe pole, as you suspected. With a refusing pole, drift is repaired at a small lambda, with over-refusal at or below B's level on two of the three drifted derivatives. For a model whose refusal was removed on purpose, repair stays partial and costly with either pole. We had fixed in advance that a drop of at least 0.15 against the PKU pole on the abliterated model at lambda = 1 would count as support. The drop is 0.54, and we report its capability cost alongside.

On the writing. We take the point. Part 1 states the terms and claims the way a reader should first meet them.

We ran both experiments you proposed. They identify the main factor behind the Qwen3 attenuation and support the PKU explanation of the asymmetry. If these answers address your concerns, we would be grateful if you would raise your score. If something still holds it back, please tell us what, and we will answer it before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.