acceptodds
Under review as a conference paper at ICLR 2027

SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits

Abstract

Activation steering can change behaviour without establishing that the effect is specific to the intended concept. We introduce SteerCheck, a preregistered attribution audit that matches off-target KL and separates mean, protected-tail, polarity, transfer, and semantic claims. Exact replay of 960 Qwen3-14B interventions reveals complementary limits of common controls: isotropic directions occupy a narrow near-orthogonal region, whereas sign-randomized same-construction directions often retain substantial target alignment. Effect is strongly associated with signed cosine within the sign-randomized family (); of its draws exceed cosine , and every draw exceeding the observed mean effect has cosine above . This alignment leakage does not by itself invalidate a conditional randomization test; it limits what the comparator can distinguish and motivates reporting exchangeability assumptions, a construction diagnostic , and the empirical cosine distribution. The primary Qwen complete gate remains negative because the protected tail fails all families. On independent data, continuous margin transfers only in Qwen and accuracy transfers in no selected cell. Prospectively registered language controls pass the complete gate in Qwen and DeepSeek, while a passing DeepSeek detox comparator rules out categorical separation; all nominal passes are sensitive to . Frozen three-rater open-generation evaluation supports factual correction in DeepSeek but not Qwen; the automatic judge fails calibration (macro-F1 ), so null-wide semantic results remain descriptive. SteerCheck makes these conditional and mixed conclusions auditable.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, representation engineering, attribution auditing, interpretability, AI alignment, randomization tests, language models, evaluation reliability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VeqWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper examines how to verify that a steering vector represents the intended concept. It must not only cause a general change in model output. The authors state that comparisons with random vectors do not give sufficient evidence. They propose STEERCHECK, a preregistered audit method. This method matches each vector by the amount of change that it causes in the output. It also tests each claim with several control vectors. The main result is “alignment leakage.” Control vectors made with randomized labels often remain closely aligned with the steering vector. Thus, these control vectors give less useful information than expected. The authors apply STEERCHECK to an anti-sycophancy vector. The vector has an effect that is greater than general model disturbance, but the effect is not specific to the intended concept.

Critical strengths
  1. The paper does not only examine whether behavior changes. It also examines whether the extracted direction represents the intended concept. This is important because a vector can improve the primary metric for an unrelated reason. The authors’ anti-sycophancy example shows this problem.

  2. The alignment-leakage analysis and the dataset-level test help researchers evaluate a sign-randomized control before they use it. Researchers can then determine whether the control gives useful information. This result affects the interpretation of current and future claims about steering vectors.

Critical weaknesses
  1. The alignment-leakage analysis includes only two related Qwen model series. The effect-alignment profile uses only one model and one layer. The authors use other models, such as DeepSeek and Gemma, only in secondary experiments. They do not test a Llama model. Thus, it is not clear whether the main results apply to other model families.
  2. The audit uses forced-choice scores and very short generated texts. However, steering affects each generated token. In long texts or reasoning traces, the effect can increase or the model can correct it. Thus, the results might not apply to these increasingly common uses.
  3. Alignment leakage and its diagnostic use vectors made from differences between contrast pairs. Therefore, they do not directly apply to steering methods based on probes, optimization, or sparse autoencoders. This limits the use of the audit as a general standard.
Questions for authors
  1. The leakage diagnostic requires only stored activations. Can you report the diagnostic for a non-Qwen model family, such as Llama or Gemma? Also, can you determine whether the effect-alignment profile occurs in models other than Qwen3-14B?
  2. How do long outputs or chain-of-thought models affect the audit results? In these cases, the steering vector operates on many reasoning tokens. Can you enable thinking mode and report results that use final answers scored after the model completes its reasoning?
Reviewer_fNaiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I read the paper several times but couldn't understand what it is really about.

I do understand from the title that the work attempts to establish a way to measure steering behavior, and could understand from the text that there is a formulation using KL, but I couldn't understand:

  1. what is the actual problem with steering behavior that the check is trying to identify or measure.

The following sentence from the paper does not make any sense to me in the context of the work: "A behavioural change alone, however, does not identify its cause: the same result may arise from the intended concept, a broadly disruptive direction, a construction artifact, or a tendency to oppose the user."

Unfortunately the paper does not provide any other details or attempt to explain.

  1. Furthermore, I don't understand what the KL-based measure is trying to achieve, and how can a KL-based measure solve a problem with identifying the cause of a behavior change.

Overall I think my inability to understand what is going on in the paper is that it is AI was used (among many other things) "for drafting and editing" and not enough human effort went into reading and clarifying. I think this should be completely rewritten to be legible to humans.

Critical strengths

I could not identify.

Critical weaknesses

Seems to be too much AI involved and too little human effort involved, at least in the presentation.

Questions for authors

Did you really read this and found it understandable?

Reviewer_3AWcOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors identify the problem of attribution in activation steering for language models, where an observed change in model behavior may be attributable to a variety of proximate causes and perturbations of intermediate representations, rather than to the specific concept the steering vector is intended to capture. The study develops an audit protocol for generating evidence of attribution, comparing the steering vector against several types of placebos/null families. In the primary, preregistered analysis, using Qwen and the anti-sycophancy behavior, the candidate vector outperforms random directions on average but not the strongest placebo, and fails several other checks. They explore the reasons for this in further post-hoc analyses. The paper provides a set of criteria for determining attribution and shows, using language-steering, that these criteria can be met under some circumstances.

Critical strengths
  • Although the Related Work section is relatively thin, the paper does overall seem well-motivated and the introduction does a decent job of explaining the key issues at stake within this area to non-specialists.
  • The main analysis is preregistered, and the paper is quite careful to distinguish between preregistered and post-hoc analyses. At the same time, the implications of including these distinct analysis modes should be recognized better, namely the issue that the preregistered analysis has a negative result and that post hoc results are given considerable weight despite their limitations as such (e.g., the authors themselves say "The central measurement is post-hoc.").
Critical weaknesses
  • Poor writing. The paper is overall hard to follow, with many terms and values introduced without explanation and results provided without much context. While the transparency and caution are admirable, the many qualifications and frequent hedging make it hard to see what can be concluded from this paper with any confidence. In places, the paper reads more as a provisional report with somewhat interesting exploratory findings than as a robust contribution.
  • Generalizability and model coverage. As the authors note, the main analysis is based on a single model and behavior, and thus provides limited evidence that the particular diagnostic program prescribed here is desirable. This and the next point lead to my two main questions for the authors.
  • Practical value. The paper has no Discussion section, and the brief Conclusion mostly restates results. This leaves it to the reader to figure out the implications of this work and what difference it makes to ML research if Steercheck is applied. A discussion would be a good place to show what the community gains by adopting the framework and what might go wrong if it isn’t applied.
Questions for authors
  • What additional evidence could be provided for why your recommended procedure (incl. reporting the quantity A and using KL matching) is useful for steering in general, beyond the single behavior reported?
  • Can you direct the reader to any previously published attribution audits where relying on existing practice led to a conclusion that your proposed framework would have changed? A concrete example would make the case for Steercheck's adoption much stronger.
  • Who were the raters involved in the study (e.g., authors? crowdworkers?), and how were they recruited and compensated, and informed of the task?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.