acceptodds
Under review as a conference paper at ICLR 2027

Beyond Target Deltas: Validating Activation Steering as a Model Intervention

Abstract

Activation steering is often evaluated through a change in target score. We examine how the fitted vector, the behavioral readout, and the generated text affect the interpretation of that change, using 105,000 generation records from seven model variants and a separate vector-extraction audit. With source examples and estimator fixed, raw-text and chat-template extraction yield mean cross-layer cosine similarities of 0.325 for Llama-3.1 and 0.309 for Qwen2.5. On fixed sycophancy responses, factual correction falls by 19.6 pp even as generic compliance and false-premise endorsement rise. In a Mistral jailbreak cell from the 50-pair α×k grid, lower harmful compliance accompanies a 96 pp increase in degraded outputs. Four of the 21 primary configurations meet the joint control–degradation cutoffs by point estimate, but none has confidence bounds satisfying both cutoffs.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, model intervention, mechanistic interpretability, evaluation methodology, AI safety, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_y5qQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines the reliability of activation steering evaluation through three aspects: steering-vector construction, behavioral metric alignment, and generation degradation. Experiments across seven model variants suggest that target-score improvements may not reliably reflect successful behavioral interventions.

Critical strengths

The paper addresses a practically relevant evaluation issue and provides empirical examples illustrating potential discrepancies between target metrics and actual generation behavior.

Critical weaknesses
  1. Limited novelty and research contribution.

The central observations largely reiterate known limitations of activation steering, including evaluation unreliability, intervention instability, and utility degradation. These issues have been extensively investigated in prior work [1,2,3,4]. The paper does not establish sufficiently distinct methodological insights or a substantially improved evaluation framework beyond these existing studies.

  1. Insufficient experimental rigor and generalizability.

The experiments primarily rely on affect-derived steering vectors applied to safety and sycophancy tasks, without adequate justification of their relevance or systematic comparisons with task-aligned interventions. The vector-format analysis lacks downstream behavioral validation. Furthermore, the safety evaluation pools are insufficiently documented, while the LLM-based behavioral judgments lack reported human–judge agreement statistics. These limitations substantially weaken the reliability and generalizability of the conclusions.

[1] Pres et al. (2024). Towards Reliable Evaluation of Behavior Steering Interventions in LLMs. arXiv:2410.17245

[2] Goyal & Daumé III (2026). Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions. EACL 2026.

[3] Ding et al. (2026). FaithSteerBench: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering. arXiv:2603.18329.

[4] Xu et al. (2026). Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics. ACL 2026.

Questions for authors

How do the findings generalize to task-aligned steering vectors and alternative steering methods, and what evidence validates the reliability of the automated behavioral judgments?

Reviewer_GREHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper questioned activation steering's performance measurement. It tests variations of underlying texts can yield low similarity activation vectors. It shows that steering makes the model follow the user more often, but correct factual mistakes less often. The paper also argues that the lowered metrics (e.g. harmful compliance) are at the cost of degraded outputs.

Critical strengths
  • This paper highlights an important difference between metric evaluation and the true output quality.
  • It also did evaluations on various models and configurations.
Critical weaknesses
  • The presentations need improvement in various ways. For examples, there are unusual expressions such as "With source examples and estimator fixed, raw-text and chat-template extraction yield ...", "fixed sycophancy responses", "jailbreak grid cell", etc, which are all very hard to comprehend for readers. Sentences like "In our sycophancy evaluation, generic compliance rises even as scoring the same responses for specific behaviors finds fewer corrections and more endorsements of false premises." are also very confusing. I have no idea if this means the scoring difference is from the sampling randomness given one same text, or is it something else that involves paraphrasing. Major revisions throughout the paper is needed before it can be considered for publication.
  • The paper questions the robustness of steering vectors w.r.t example formatting. However, authors only used 400 examples in this experiment, meaning they used 400 hidden states to estimate a steering vector that is of the size of the latent space. In most recent LLMs, this latent space should be orders of thousands. There is no surprise in the lack of robustness in such case, and it doesn't mean the steering vector generation process is not reliable per se.
  • The benchmarks used in this paper are author-curated query pools informed by existing benchmarks, instead of those public benchmarks themselves. Since this is a pure evaluation paper, it is impossible to evaluate the novelty and value of the proposed benchmark (author-curated instead of the original ones) and metrics (degraded instead of the normal PPL, bi-,trigram, LLM as a judge) without looking at their direct comparisons.
  • The evaluation method overall is not very clear to me. Without the code, it's hard to know exactly how each metric/setting/data was used to generate these results. The takeaways are therefore unclear to me.
Questions for authors

see above

Reviewer_oaXMOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Peper argues that a favorable change in a target score is insufficient evidence that an activation-steering intervention works as intended. The authors identify three factors that condition interpretation of a steering delta: how the vector is extracted, whether the scoring rubric measures the intended behavior, and whether generated text remains usable. Using negative-minus-positive affect vectors from GoEmotions, injected across nine layers, the authors evaluate jailbreak, unsafe-compliance, and sycophancy tasks on seven open-weight models, with an LLM judge scoring outputs. They report that raw-text and chat-template extraction yield weakly aligned vectors (cosine about 0.31–0.33). On fixed OLMo-Think-DPO responses, generic compliance rises while factual correction falls. In a Mistral jailbreak setting, reduced harmful compliance coincides with mostly degraded outputs. A joint control–degradation screen finds that four of 21 configurations pass by point estimate, but none passes when confidence intervals are required, which the authors take as evidence that target deltas alone cannot validate steering.

Critical strengths
  • Reserach relevance: Steering is often reported through a single target delta, and a call for multi-axis validation is useful to the field.

  • Model diversity: Five families are covered, including Base, Instruct and Think variants of OLMo-3.

Critical weaknesses
  • Presentation: I find the paper difficult to follow. Abstract leads with jargons (“target score”, “fitted vector”, “the behavioral readout” “joint control–degradation cutoffs”) without stating the problem or listing contributions. As a reader it was difficult to tell whether this is a method, benchmark, or position paper. Introduction has poor motivation, no clear research question, and no contribution list. I don't understand the contribution of the Section 3 at all. The main text is about 6 pages of a 9-page limit, while many results sit in the appendix (Table 2, Figs. 3-5, the grid results). Some of the results can be put back into the main paper.

  • I would like the authors to define the tasks first and then go with the experimental design.

  • For a paper about measurement validity, the judge itself is unvalidated:

    • A single LLM judge was used, with varying API versions across runs.
    • The 180-item human review has no archived labels and no agreement statistics, so it contributes no evidence.
    • The safety prompt pools are author-curated, and their construction and provenance are undocumented, which makes the work non-reproducible. There is no code or data release.
  • I don't find any error analysis. Neither the authors provided any qualitative breakdown of the results.

Questions for authors

See Weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.