acceptodds
Under review as a conference paper at ICLR 2027

Steering Interference Reflects the Model's Defaults, Not the Behavior Directions

Abstract

Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that it is the model rather than the behavior being steered. A steer relaxes the model toward a small set of behaviors it already favors, chiefly refusal, sycophancy, and poeticism, and that set is much the same whatever is steered. Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, a direction carrying no behavioral content, matched to a real steer only in the size of the vector it adds, moves the same behaviors in the same order as real steers do, while producing none of the behaviors that need a specific direction. Second, most interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains almost none of the interference it takes part in. The account holds on all ten models, the pull toward defaults strongest below 10B parameters and weakening in each family's largest. Reading a steer as a perturbation whose endpoint the model fixes implies that disentangling behavior directions cannot by itself make steering modular.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
steering, interference, large language model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8C81OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper uses models of different sizes (Qwen, Mistral, Llama, Gemma) to measure the off-target effects of activation steering. They find that the effects are asymmetric: steering one behavior can move another without the reverse holding (e.g., steering profanity raises toxicity, but steering toxicity does not raise profanity). They also find that some behaviors, such as refusal and sycophancy, are easily induced by most steers, while others, such as politeness, are suppressed. A random direction of matched norm reproduces the same pattern, and the geometry of the directions fails to predict the interference for a held-out behavior. The authors conclude that a steer’s side effects are fixed by the model’s ''default set'' rather than by the geometry of the steered direction, and that disentangling directions cannot by itself make steering modular.

Critical strengths
  • The paper quantifies a well-known but rarely measured problem. Side effects of steering are widely acknowledged, and the paper measures them systematically from generated text, showing that the interference is largely asymmetric and that the geometry of the steered directions does not predict it.
  • In particular, the leave-one-behavior-out protocol is a meaningful test than holding out individual pairs, since it prevents a predictor from succeeding by memorizing which behaviors are easily induced.
  • Figure 2 is clear and informative. The interference matrix makes the main pattern visible at a glance: whatever is steered, the same columns move, with prosocial behaviors suppressed and a small set of behaviors induced.
  • The experiments cover a broad range of model sizes (1.5B to 72B) and four model families, which makes the structural findings more convincing than a single-model study.
Critical weaknesses

W1. The findings largely reconfirm known vulnerabilities of post-trained models, and the novelty is overstated.

All ten models are instruction-tuned, and the observed structure maps closely onto what post-training is known to do.

  • The sinks are behaviors that post-training shapes strongly and conditionally. Toxicity is suppressed. Refusal is gated to harmful requests, so a perturbation that breaks the gate makes it fire on benign prompts. Sycophancy is known to be amplified by RLHF. Consistent with this, the authors report that refusal is the leading sink in the three refusal-trained families but the weakest in Mistral. On this reading, the paper reconfirms that alignment post-training is fragile under perturbation: random or benign steering compromises safety behavior (Korznikov et al., 2025), and narrow interventions break post-training broadly (Betley et al., 2025).
  • The resistant behaviors are the ones the assistant persona expresses by default. Politeness, optimism, empathy, and sentiment are what a perturbation suppresses. This reconfirms that moving away from the default assistant persona leads to drift (Lu et al., 2026), and that strong steering collapses generations toward simple patterns (Vogels et al., 2025).

If this interpretation holds (see W2), the paper’s distinctive contribution is methodological rather than mechanistic: the text-grounded readout, the leave-one-behavior-out protocol, and the quantitative contrast between geometric and random-push predictors. I encourage the authors to frame the contribution that way, and to discuss directly how their account differs from “steering pushes the model off the post-training manifold.”

W2. Only instruction-tuned models are studied, so the source of the “model defaults” is untested.

The default set is plausibly an artifact of alignment post-training rather than a property of the model in general. Running the pipeline on base models alongside their instruction-tuned counterparts (e.g., Qwen2.5-7B vs. Qwen2.5-7B-Instruct) would directly test this. If the sinks, and refusal in particular, disappear or change substantially without post-training, the paper’s central claim should be reframed accordingly.

W3. Steering strength is not characterized.

Steering adds a raw, unnormalized difference-of-means vector multiplied by a coefficient from {1, 2, 3, 4, 6, 8}, at every position including the prompt. The paper never reports the perturbation magnitude relative to the residual-stream norm, so readers cannot judge whether these are gentle or aggressive steers. Given that activation steering is known to be highly sensitive to strength, this is a significant omission. All interference analyses are also performed at a single coefficient per behavior. The authors should show whether the main findings (the antisymmetric fraction, the low-rank structure, and the random-push correlation) hold across a range of strengths.

W4. The calibrated strengths are inconsistent with the stated calibration rule, and this may confound source strength.

Appendix D says each coefficient is chosen so that the self-effect is closest to a two-point target, yet Table 3 reports self-effects of +6.97 (legalese), +6.69 (archaism), +6.62 (numeric), and +6.56 (religiosity), at coefficients of 3–4, when smaller coefficients were available on the grid. No justification is given for steering these behaviors at more than three times the target strength. Because behaviors are steered at very different effective strengths, the source-drive term may partly reflect steering magnitude rather than any property of the behavior. Please clarify the selection, and report how source strength changes when all behaviors are steered to a matched self-effect.

W5. The conclusion generalizes beyond what was tested.

The claim that disentangling directions “cannot by itself make steering modular” rests on single-layer additive difference-of-means steering with a narrow coefficient range and two modified variants. It does not cover SAE-feature steering, multi-layer or conditional steering, or optimization-based approaches. The conclusion should be scoped accordingly.

Questions for authors

Questions are in the weakness section.

Reviewer_jg7sOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper aims to show empirically that the effect of steering is not modular as is often assumed. The authors argue that steering, regardless of intended steering behaviour, points a model towards general already learned behaviours in representational space by showing that random steering vectors produce pretty much the same effects as specific steering vectors. In addition, they argue that a steering’s vector overlap with other behaviours cannot explain why it spills over, because overlaps are not symmetrical when it comes to the actual LLM behaviour in steering. The authors also argue that given the overlap between a steering vector and other behaviours is known, effects should be predictable, which they find is not the case. In sum they claim to falsify what is known as the natural ‘geometrical’ explanation for ‘leakiness’ which focuses mainly on the cosine similarity of pairs of steering vectors. The paper has an elaborate, step-by-step pipeline that is developed with reference to one specific LLM (Qwen) and subsequently replicated with 3 other families (Gemma, Mistral, Llama) and model instances of different sizes.

Critical strengths

To me the methodology and the pipeline are the strongest point of the paper. The authors build their argument in six steps carefully from computing steerability until replication with four model families. For every step the authors build in careful controls such as searching over a grid of steering coefficients, evaluating over actual model behaviour instead of learned representations, rerun the entire pipeline with random control vectors, and replicate the pipeline with 3 other model families. I consider the two main conclusions that 1) leakiness is demonstrated by the rankings of the interference matrix of all models tested and that 2) geometry cannot explain it as mostly supported.

Critical weaknesses

The other key claim the paper tries to substantiate is that models default to a narrow set of common behaviours across the board, and that this is not caused by the content of behaviour steering, because the same effects can be mostly reproduced by random vectors. Though the authors find some evidence that goes in this direction, to me the picture here is much less clear. The results suggest larger models are more robust to perturbations because random vectors produce much lower correlations with interference rankings resulting from real steering vectors. That makes this not so much a universal finding about behavioural 'sinks' in LLMs but rather a finding about how well larger LLMs are able to use contextual cues, can make use of redundancy, and surpress irrelevant information when generating language.

Though the paper does a good job at meticulously demonstrating leakiness and falsifying the geometrical explanation, I am not sure how this finding relates to the field and what the actual contributions are. The two main claims as the authors also seem to concede are already well-known in the (adjacent) field(s). The claim about the generality of behavioural sinks to me leaves open too many alternative explanations that are not explored. This also follows from the summary of contributions in the introduction that are effectively a very focused listing of the paper's main claims without relation to the impact beyond the paper's experiments itself.

Questions for authors

The paper to me seems at least partly written by AI. It's style is very declarative and it has many features of the typical distant and abstract mode of writing of AI. Claims and findings make up a lot of the section headings and paragraph headings, and some lines seem to reflect earlier prompts (e.g. 142). I think the paper would be stronger if the authors make this into a better human-readable paper with proper structuring of contributions, sections, tables, and (paraphrased) claims less scattered over the paper.

Reviewer_9oFtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work analyzes which behaviors are affected and to what extent by activation steering. Although recent studies attribute interference between behaviors to the geometry of the representation (e.g., cosine overlap or shared subspaces among behavioral directions), the authors argue that the side effects of orientation are instead determined by the model’s attractors ("sinks"). The evaluation includes 24 behaviors and 10 models (ranging from 1.5B to 72B parameters in Qwen2.5, Llama-3, Gemma-2, and Mistral).

Critical strengths
  1. Very well motivated problem by exposing two critical methodological flaws in prior work.
  2. Important practical implications for model steerability. Showing that random vectors reproduce the collateral effect of targeted vectors, reframes activation steering from a feature edit to an off-manifold perturbation whose endpoint is determined by the model's attractor landscape.
Critical weaknesses
  1. The paper claims the mechanism behind unreliable and entangled steering, but the paper only proves that any perturbation rolls downhill into some model defaults (refusal, sycophancy, poeticism). It never explains how the transformer architecture converts an arbitrary residual stream vector into those specific behaviors.
  2. The authors set up the critical methodological flaws of previous works in Section 1 and repeat it at the opening of Section 7, but it is not clear how they solve the second one.
  3. In Section 10, the opening sentence, "What a steer moves is set by the model it is applied to, not by the behavior it targets", is overclaimed. As Fig. 2 shows, the diagonal entries are clearly marked. The steer does move the behavior it targets; only the collateral side effects are governed by the model's defaults.
  4. Opaque presentation and over-reliance on appendices make it hard to follow.
    • Page 2, Fig. 1a: Why are some points colored and others in gray? The figure has no legend and the caption never defines the color encoding.
    • Page 2, Lines 084–105: Section 2 introduces the interference matrix MM and its decomposition into SS and AA before explaining how MM is actually computed. Line 154 states "The natural answer is that MijM_{ij} depends only on how much directions ii and jj overlap" immediately after stating "That premise fails" (Line 149), which creates confusion regarding the definition the authors are actually using.
    • Page 4, Lines 176-213: The main text omits the mathematical definition of MijM_{ij}, the identity of the judge, and how the evaluation works (e.g. the evaluation prompts).
    • Page 5, Lines 252–255: How AUROC is computed was never introduced or explained earlier in the paper. Additionally, why drop the 4 "unsteerable" behaviors (confidence, verbosity, humor, anthropomorphism) from Fig. 2? Keeping them would have provided a complete picture.
    • Page 6, Lines 301–308: The text states that three distinct patterns account for most of the matrix energy, yet only two numbers are mentioned.
    • Page 6, Lines 318–320: "What are the consequences of having a coherence bar? For example, how much generations are discarded with that bar?
    • Page 6, Fig. 3: The x axis title is "how much real steers induce it". Do you mean as a side effect or as the target?
    • Page 7, Lines 331–338: You refer to "resistance" without defining it. How is resistance formally defined and measured in this context, and specifically what (which models, layers, or behaviors) fails to resist?
    • Page 7, Table 1: Where are the "cone" results you refer to located in the table? What do you mean by "ridge"?
Questions for authors

Can you explain why Gemma-2-27B exhibits such a low correlation in Figure 4? Would also be interesting to get a hypothesis for why it appears that the intervention does not push the model toward bad behavior, but rather pushes it away from good behavior.

Comments on the paper

2 comments
~Ernests_Lavrinovics1OpenReview ↗

Data generation

Hi,

Thanks for the work! Its not immediately clear to me how the data for the 24 traits was generated. I tried to reproduce your results with Qwen 2.5 7B, but the data that I've generated using determinism is much noisier when it comes to cross-trait interference. Also, in your work, each of the text stylizations use different input prompts across the board

from.. ./code/experiments/scripts/behaviors.py
e.g.
verbosity: How do I boil an egg?
humor: My code has a bug.

Does it make it problematic to compare these traits across the board if the input prompts are not identical?

Thanks!

AuthorsOpenReview ↗

Re: Data generation

Thank you for trying to reproduce this, and for the careful look at the code.

How the data was generated. Each of the 24 behaviors has 12 contrastive triples in behaviors.py: a user prompt, a response that shows the behavior, and one that does not. As our AI use statement says, they were generated with a language model to a specification we set, and they are fixed, not regenerated per run. The appendix gives the definitions and two examples. A behavior's direction is the difference of means of the residual stream over the response tokens, taken at the layer in the middle of the network where it best separates held-out pairs.

Different prompts per behavior. Those prompts are used only to extract each behavior's direction. Within a triple both responses share the prompt, so the prompt's own contribution largely cancels in the difference. The interference itself is measured on one shared input set: the unsteered model and every steered model answer the same 32 neutral prompts (eval_prompts.py), and the judge rates every response on all 24 behaviors. So every entry of the matrix compares behaviors on identical inputs. Different extraction prompts can still put some topic into a direction (all refusal prompts are harmful requests, for instance), which would mostly affect the cosine geometry between directions. Extracting every behavior from one shared prompt set is a fair check that we have not run.

Noisier interference. Four choices matter a lot. The coefficient is calibrated per behavior with the judge in the loop, not fixed. The layer is chosen per behavior. The judge is the model under study, unsteered, rating all 24 behaviors in one JSON reply. And effects are in units of each behavior's pooled standard deviation. Single entries are noisy even in our runs: the median entry is 0.09 standard deviations in size, against a 95 percent interval about 0.26 wide. That is why the paper's claims rest on the structure rather than on single pairs. A second run of the pipeline agreed with the first at r = 0.99, and the one-way part reproduces across disjoint halves of the prompts. The supplementary results include the directions, the calibrated coefficients and sweeps, and the matrix with intervals, so you can compare stage by stage. If you tell us what you changed (judge, coefficients, layers or decoding), we are happy to help find where the runs diverge. The revised supplementary also includes the runs added during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.