acceptodds
Under review as a conference paper at ICLR 2027

Steer the Steering: Calibrating Learned Interventions in Large Language Models

Abstract

Existing activation steering methods learn adaptive steering directions to overcome the limited flexibility of fixed difference-in-means directions. However, directly learning such steering directions from semantically diverse training data under imperfect behavior modeling can introduce distributional bias, thereby limiting steering effectiveness and generalization. This motivates our geometric calibration framework for calibrating learned interventions in large language models, which called S2S. Our method decomposes the learnable steering direction into a difference-in-means component and a residual component, and employs steering interpolation to geometrically calibrate the residual relative to the difference-in-means direction. Furthermore, we extend our method to linear steering and rotation-based steering, allowing S2S to operate under complementary intervention paradigms. Building on this, we introduce a similarity-aware adaptive weighting strategy to dynamically coordinate the contributions of the hidden-state direction, difference-in-means direction, and residual direction during inference. Extensive experiments across multiple-choice benchmarks demonstrate that S2S consistently outperforms linear-steering baselines, yielding gains of up to 23.9% on TruthfulQA, while preserving its general open-ended generation quality.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Activation Steering, Large Language Models, Learned Interventions

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eHv2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper observes that learned, input-dependent steering vectors (L2S) carry useful sample-specific information but generalize poorly in direction (Fig. 2).

Method.

  • S2S decomposes the MLP-predicted vector as vi=vˉ+riv_i = \bar{v} + r_i, the CAA mean plus a residual (Eq. 3).
  • It SLERP-interpolates the normalized residual toward the mean direction (Eq. 5).
  • The calibrated direction is used additively (S2S-L) or as a rotation target (S2S-R).
  • S2S-R adds a "similarity-aware adaptive weighting" (AW) over the hidden-state, mean, and residual directions, together with a margin gate (Eqs. 9–12).
  • Two theorems motivate the design: a reachability superset (Thm. 1) and residual shrinkage reducing MSE (Thm. 2).

Validation.

  • On TruthfulQA with Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, S2S-R reports MC averages of 60.01 / 64.41, versus 53.16 / 51.56 for the strongest baseline, Spherical Steering (Table 1).
  • Zero-shot transfer to five other MC benchmarks is reported for Llama (Table 2).
Critical strengths
  1. A clear diagnosis of why learned steering vectors underperform. Fig. 2 separates direction error from magnitude error, and shows that oracle sample-specific vectors help much more than the mean while predicted ones do not. This is a useful framing for the learned-steering literature, and the plug-in calibration applies to several learned interventions (L2S, LoFiT, LoReFT; Fig. 4c).
Critical weaknesses
  1. The main results are internally inconsistent. Table 3 (the ablation) is stated to be on Llama-3.1-8B-Instruct (App. B.7) and reports MC1 = 67.40 for full S2S-R. Table 1 reports MC1 = 57.17 for the same method and model. The "no component" row of Table 3 (41.42) also matches neither the Llama baseline in Table 1 (34.15) nor any baseline method. It is unclear which numbers are representative.

  2. The ablation does not support the central claim. In Table 3, the S2S calibration itself moves MC1 only from 41.42 to 43.14. Adaptive weighting then jumps to 66.91. No experiment applies the weighting and gate without the calibrated residual, so the paper's thesis, that calibrating learned directions is what helps, is not supported. In addition, under MC scoring each candidate answer receives its own similarity-gated intervention. This may act as an answer classifier rather than steering, which is consistent with S2S-R not leading on open-ended TRUE×INFO.

Questions for authors

Questions For Authors

All questions are clarifications that could change my assessment.

  1. Please reconcile Table 1 and Table 3 for Llama (MC1 57.17 vs. 67.40; baseline 34.15 vs. 41.42). Which configuration produced each?
  2. What does adaptive weighting + gate achieve with the mean direction only (no residual)?
  3. During MC scoring, is the intervention computed separately for each candidate answer, and does the gate fire at different rates for correct and incorrect candidates?
Reviewer_4TRDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper notes a train-test gap in the direction of the learnable steering vectors that limits them to obtain an oracle input-dependent steering performance. They propose fixing these by calibrating the direction towards the fixed mean difference vector using SLERP-based interpolation. Then, they use both activation addition and rotation techniques to intervene at a given layer with this new calibrated steering vector, while retaining the original magnitude of the learned vector. Results on 2 models and truthfulqa shows gains against existing baselines that also transfer to other benchmarks, especially for multiple-choice question-answering.

Critical strengths
  • The proposed approach is well-motivated in empirical findings and also provides theoretical analysis over reachability of the steering area and sensitivity to the prediction noise.
  • To circumvent the parameter sensitivity in rotation settings, they also propose adaptive weighting mechanisms for the rotation based on the similarity between the activation, the mean, and the residual vector.
  • Both linear and rotation-based baselines are analyzed and the proposed approach also gives both possibilities of steering.
  • Hyperparameter and sensitivity analysis is thorough and they also analyze the ablation of the different components.
  • It generalizes to other learnable steering frameworks as well, particularly, LoReft and Lofit.
Critical weaknesses
  • Section 4 is critical and requires general improvement for clarity. For example, add 2a, 2b, 2c labels whenever relevant and explicitly discuss how magnitude (norm) and directional errors (cosine similarity) are calculated in text. Also, add details on how the oracle was calculated exactly.
  • It is not clear what the analysis in figures 4a and 4b is intended to do.
  • As also noted by the authors later, the effectiveness of the approach could be limited to L2S based input-dependent steering vectors since it is possible that the trained suboptimality is due to the local training loss functions which does not happen otherwise. Yet, it does work rather well for reft as well so more discussion is needed to support these results with useful insights.
  • Since the approach adds another hyperparameter to tune, more insights are needed on how to tune them given the SLERP coefficient may not be directly interpretable to a user.
  • Interventions are applied on fixed layers and layer sweeps must also be conducted such that optimal layer for each baseline be found.
  • Results are limited to two models and they mainly study truthfulqa as the steering dataset.
  • Missing discussion and comparison with some relevant related works
    • Cao, Yuanpu, et al. "Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization." Advances in Neural Information Processing Systems 37 (2024): 49519-49551.
    • Zhao, Hongjue, et al. "Odesteer: A unified ode-based steering framework for llm alignment." arXiv preprint arXiv:2602.17560 (2026).
Questions for authors
  • How does interpolation with the mean vector calibrates the learned vectors? Mean vector itself was even farther apart in the cosine analysis so why do we want to move towards it?
  • Why is cross-benchmark generalization shown for truthfulqa steering? Is it to keep the unintended effects of lowering these performances low?
  • Does the method generalize beyond truthfulqa?
  • Will BiPO based training learn better directions already?
  • What are figures 4a and 4b intended to show?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 4TRD [Part 4/4]

W7: Discussion and comparison of related work

Thank you for pointing out these omissions. We will add citations and discussion of the following studies in the revised manuscript and clarify their connections to and differences from our work.

Q1: Why calibrate toward the mean direction when its oracle cosine similarity is low?

Thank you for this important question about the mechanism. The mean direction serves as a relatively stable reference. Its alignment with the reference direction for individual samples should be distinguished from the overall effect of the calibrated intervention.

(1) Trade-off between directional quality and stability: The learned sample-specific direction can offer better alignment with the oracle direction, but is susceptible to prediction errors on test data. The mean direction may have lower oracle cosine similarity, yet provides a more stable shared reference across training and test data. Calibrating toward this reference balances sample-specific directional quality with stability, potentially reducing prediction noise and improving downstream performance.

(2) Theoretical support: Consider the decomposition

vi∗=vˉ+ri∗,ri=ri∗+ϵi,v^{\ast}_i = \bar{v} + r^{\ast}_i, \qquad r_i = r^{\ast}_i + \epsilon_i,

and the shrinkage estimator

viβ=vˉ+βri.v^{\beta}_i=\bar v+\beta r_i.

Assuming that the expected cross term between the residual signal and noise is zero and that their second moments are finite, we obtain

=(1-\beta)^2S+\beta^2N, \qquad \beta^{\ast}=\frac{S}{S+N},$$ where $S=\mathbb E\|r^*_i\|^2$, $N=\mathbb E\|\epsilon_i\|^2$, and $S+N>0$. Under these assumptions, appropriately shrinking a noisy residual can reduce its mean squared error. This provides an explanation for the benefit of a stable shared reference. > **Q2: Does Table 2 evaluate cross-task effects of steering learned on TruthfulQA?** **In Table 2, the MLP is trained independently on each benchmark and evaluated on its corresponding test set:** COPA, StoryCloze, MMLU, WinoGrande, and BoolQ. The purpose of Table 2 is therefore to evaluate the applicability of the S2S framework across multiple reasoning tasks. It does not directly transfer steering vectors learned on TruthfulQA to these benchmarks and cannot establish whether TruthfulQA-based steering has adverse effects on other tasks. We will replace “cross-benchmark generalization” in Section 6.1 with “evaluation across multiple benchmarks” and clarify the independent training setup. > **Q4: Has BiPO already learned better directions?** Thank you for suggesting this valuable comparison. We agree that a stronger vector-learning method may provide a better reference direction. To examine its complementarity with sample-adaptive calibration, we replace the training-set mean anchor with a BiPO vector and combine it with the MLP. The configurations are as follows: MLP + CAA ($h'=h+\hat v$, $\alpha=1$); Mean anchor + MLP + S2S-L (strength = 1, $t=0.2$); BiPO (strength = 1); and BiPO + MLP + S2S-L (strength = 1, $t=0.2$). | Method | MC1 | MC2 | MC3 | | --- | ---: | ---: | ---: | | MLP + CAA | 37.58% | 57.09% | 29.69% | | Mean vector + MLP + S2S-L | 44.92% | 65.33% | 36.20% | | BiPO | 60.18% | 78.21% | 55.77% | | BiPO + MLP + S2S-L | 63.77% | 82.47% | 57.93% | These results suggest that a stronger shared anchor and sample-specific calibration can be complementary. --- Thank you again. We will incorporate these clarifications and results to better explain our method's mechanism, applicability, and limitations. Best regards, The Authors

Thank you for the detailed response! The authors are encouraged to refer to the Appendix whenever deferring any details to the Appendix for clarity. Generally, the method gains are clear in the linear activation addition setting and have been strengthened by the new results. I wanted to clarify a few points regarding the layer sweep and hyperparameter analysis of the baselines and whether using the exact hyperparameters for MC and open-ended tasks is a good idea. Reviewer eHv2 also raises a good point regarding the strong effect of adaptive weighting and gating (the YYNN row mainly), which is tangential to the main claims, and I look forward to the author's response regarding that.

Reviewer_H35GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates distributional bias in steering directions and proposes S2S, a geometric calibration framework to mitigate this bias. S2S decomposes a steering direction into a difference-in-means component and a sample-specific residual component, then calibrates the residual relative to the difference-in-means direction through steering interpolation. During inference, S2S employs a weighting strategy that combines the hidden-state direction, difference-in-means direction, and residual direction. Experimental results show that S2S outperforms linear-steering baselines on multiple-choice QA while maintaining performance on open-ended generation tasks.

Critical strengths
  • The idea of calibrating steering vector directions is well-motivated and can be applied to existing learnable steering methods, including linear steering (S2S-L) and rotation-based steering (S2S-R).
  • The paper is well-written, with comprehensive ablations and analyses.
Critical weaknesses
  • S2S does not demonstrate significant performance improvements over spherical steering on cross-benchmark generalization and open-ended generation tasks, and even underperforms it on some benchmarks.
  • The evaluation is limited to Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. It would be helpful to evaluate S2S on a broader range of models (e.g., Qwen3.5, Nemotron) across different model sizes to better assess its generalizability.
  • Several important related works are missing:

[1] Nguyen, T.M., Nguyen, T.A., Alemohammad, S., & Baraniuk, R.G. (2026). Minimizing Collateral Damage in Activation Steering. ArXiv, abs/2605.01167.

[2] Quy-Anh Dang and Chris Ngo. 2026. Selective Steering: Norm-Preserving Control Through Discriminative Layer Selection. In Findings of the Association for Computational Linguistics: ACL 2026, pages 10887–10910, San Diego, California, United States. Association for Computational Linguistics.

Questions for authors

In Table 1, are the same hyperparameters used for S2S in both the multiple-choice and open-ended generation settings?

1 comment

Thank you for the authors' responses.

The additional experiments presented in response to W1 did not adequately address my concern regarding why S2S failed to demonstrate significant improvements over Spherical Steering on cross-benchmark generalization and open-ended generation tasks.

Regarding the additional experiments with Qwen3.5, Nemotron, and multiple Qwen2.5-Instruct model sizes, the performance of all baseline methods dropped substantially, which warrants further investigation and explanation.

Overall, while I appreciate the authors' efforts to address my concerns, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.