acceptodds
Under review as a conference paper at ICLR 2027

One Token Can Be Enough: Bridging Prompting and Activation Steering with Prefix Steering

Abstract

Prompting guides language model behavior through the initial context, whereas activation steering often intervenes throughout generation. A natural question is whether steering can produce effects on subsequent computation similar to those of prompting. Under fixed-state attention assumptions, we establish sufficient conditions for single- and multi-token steering to match prompt-induced attention-head outputs, and characterize how changes in input representations affect this match and its approximation error. This attention-level connection leads us to ask whether, at the behavioral level, steering can also guide subsequent generation through a brief initial intervention. We study Prefix Steering, which applies existing steering directions and operators over a short span starting at the final prompt token, with no further direct intervention afterward. We examine how intervention duration and strength jointly shape the control–capability trade-off. Across four models and five tasks, intervention over a short span, even a single token, often retains much of full steering's behavioral control while better preserving general capabilities, offering a trade-off competitive with, and in some settings better than, prompting and alternative steering-strength policies. Prefix Steering also remains effective on final-answer formatting tasks after reasoning, suggesting that a brief initial intervention can shape subsequent generation through the context it creates.

Primary area:
interpretability and explainable AI
Keywords:
Activation steering, prompting engineering, representation engineering, interpretability, large language models, representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_X4CuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Prefix Steering: applying an existing steering direction only over a short span beginning at the final input token. It derives sufficient conditions for matching prompted attention outputs under fixed-state assumptions, then evaluates intervention duration and strength across four models and five tasks. Short interventions often retain much of full steering’s behavioral control with less capability degradation.

Critical strengths
  1. The paper identifies a useful distinction between initiating a behavior and continually enforcing it. Figures 4–5 show that repeatedly applying the same direction can add capability cost without a corresponding gain in control. Obtaining better trade-offs simply by shortening the intervention is a practically useful finding, particularly because it works with existing directions and both Additive steering and COAST.

  2. The final-answer formatting experiments strengthen the empirical contribution. Unlike the refusal example, where the first generated word already signals the target behavior, these tasks evaluate a property expressed after intervening reasoning. They provide evidence that the benefit of a short intervention extends beyond changing the response opening.

Critical weaknesses
  1. The attention-matching theory provides motivation, but its explanatory value for practical Prefix Steering remains unclear. The constructed displacement depends on the query and context, whereas the behavioral experiments use a fixed DiM direction across inputs. Appendix D.1 establishes partial directional alignment on one model and concept, but does not show that this alignment yields small attention-matching errors or predicts successful control. Likewise, a smaller sufficient matching subspace with more steered tokens does not establish why longer interventions damage capability. The paper acknowledges these limitations, but the connection between its two main contributions remains incomplete.

  2. The comparison with adaptive steering policies is less conclusive than the comparison with full steering. Appendix D.3 fixes DAS’s maximum coefficient and ACT’s parameters to values from their original papers, while Prefix Steering’s strength is selected for the present models and tasks. Since the paper itself shows that strength substantially changes the control–capability trade-off, Figure 6 does not establish that the advantage comes from stopping early rather than from the chosen operating points. A comparable tuning procedure is needed to support the claimed advantage over these policies.

Questions for authors
  1. Does the attention-matching error obtained with the actual DiM intervention predict behavioral success? For the delayed formatting results, can you distinguish the contribution of modified cached representations from that of the generated text—for example, by replaying the same initial generated tokens with a freshly computed, unsteered cache?

  2. Does the advantage in Figure 6 persist when DAS and ACT are tuned on the same validation data and compared at matched levels of behavioral control?

2 comments

All revisions made in response to this review are marked in blue in the revised paper.

W1 and Q1. Explanatory value of the attention-matching theory

Thank you for the thoughtful comments. Our goal is to connect prompting and activation steering at the attention level, rather than to construct a displacement that exactly matches each prompt, which would offer little beyond prompting itself. The theory shows that steering a single position can reproduce a prompt's attention-level effect, and this connection motivates Prefix Steering, which applies reusable directions such as DiM over a short span. It does not claim that a lower matching error predicts better control, since prompting itself does not always succeed. Following your suggestion, we replay the initial generated tokens with a fresh, unsteered cache (Table 1). Relative to prompting, the attention-output error is much smaller with the modified cache than with the replayed text alone. For safety, where refusal appears in the first tokens, replay keeps most of the control. For delayed formatting, replay loses most of the gain and restores MATH-500 accuracy, so both the lasting control and the capability cost come mainly from the modified cached representations. Regarding longer interventions, we present the smaller matching subspace as a geometric picture consistent with the capability cost, not as its cause. We have added the replay experiment as a new Section 4.3.

\hline & & \text{Safety} & & & & \text{IF-Boxed} & \\\\ \text{Condition} & \text{Control} & \text{MMLU-Pro} & \text{Attn. diff} & \quad & \text{Format} & \text{MATH-500} & \text{Attn. diff} \\\\ \hline \text{Unsteered} & 36.0 & 39.4 & \text{–} & & 48.7 & 48.3 & \text{–} \\\\ \text{Prefix-1, steered} & 80.7 & 39.0 & 0.421 & & 91.6 & 47.8 & 0.513 \\\\ \text{Prefix-1, replay} & 77.6 & 40.5 & 2.847 & & 57.2 & 48.7 & 3.426 \\\\ \text{Prefix-5, steered} & 85.4 & 36.6 & 0.368 & & 94.8 & 45.4 & 0.482 \\\\ \text{Prefix-5, replay} & 81.9 & 38.3 & 1.935 & & 64.1 & 48.9 & 4.127 \\\\ \hline \end{array}$$ Table 1: Replay on OLMo 3 7B with additive steering. *Steered*: Prefix Steering as in the paper. *Replay*: the same first $k$ generated tokens, recomputed with a fresh, unsteered cache, then generation continues without intervention. Attn. diff is the absolute difference between the attention outputs of each run and those under prompting, averaged over the generated tokens. **W2 and Q2. Tuning of DAS and ACT** Following your suggestion, we retuned all strength policies under the same protocol on OLMo 3 7B for the safety task (Table 2, now Appendix D.5). All policies use the layer selected for full steering and select their strength on the same 50-example tuning set as Prefix Steering. The search grids for DAS and ACT cover their original values together with weaker and stronger settings, giving ACT a larger budget than Prefix Steering. After retuning, DAS still selects its original cap. At comparable behavioral control, Prefix-5 retains more capability than both DAS and ACT, and it also outperforms both decay schedules in both behavioral control and MMLU-Pro. The advantage in Figure 6 therefore does not depend on the hyperparameters taken from the original papers. | Policy | Selected strength | Control | MMLU-Pro | |---|---|---|---| | Unsteered | – | 36.0 | 39.4 | | Prefix-1 | $s=10$ | 80.7 | **39.0** | | Prefix-5 | $s=10$ | 85.4 | 36.6 | | Linear decay | $s=1$ | 84.3 | 35.9 | | Exponential decay | $s=10$ | 85.2 | 34.5 | | DAS | $c_{\max}=2$ | 85.1 | 34.7 | | ACT | $A=30$, $\beta=0$ | 84.6 | 35.8 | | Full | $s=0.1$ | **86.2** | 34.3 | Table 2: Strength policies on OLMo 3 7B for the safety task. All policies use the same layer and select their strength on the same 50-example tuning set: the scale $s$ of $c_0=s\bar r$ from $\lbrace 0.01,0.1,1,10\rbrace$ (Appendix C), DAS's cap $c_{\max}$ from $\lbrace 1,2,5,10\rbrace$, and ACT's $A$ from $\lbrace 5,12,15,30,100\rbrace$ and $\beta$ from $\lbrace 0,0.1\rbrace$, covering the original values together with weaker and stronger settings. Linear decay uses horizon $K=512$, and exponential decay uses time constant $\tau=128$.

Thank you again for your feedback. We would like to check whether our response has addressed your concerns. If any question remains, we are happy to clarify before the discussion period ends.

Reviewer_sMMSOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies activation steering applied only at the beginning of generation. Its theory gives conditions under which modifying an existing token representation matches the attention-head output produced by an added prompt token. It also characterizes query changes that preserve the match and relates single-position and multi-position intervention strengths. The practical method, Prefix Steering, reuses difference-in-means directions and existing operators, intervening only at the final prompt token or over a short initial sequence. Experiments compare control and capability preservation across multiple models and tasks.

Critical strengths
  1. Steering duration is a simple useful design choice. Experiments show that short interventions often retain much of full steering's target behavior while preserving more of the model's capability. The final-answer formatting tasks are also a good test. It shows that a brief initial intervention can affect a response format needed after the reasoning sequence has finished.

  2. Theorical analysis is interesting. The attention analysis gives a concrete construction. Theorem 1 matches both the merged value contribution and attention mass, using a value-null direction that can change the attention score. Theorem 2 extends this to a block of tokens. The algebra and small numerical checks support these results under the stated fixed-state assumptions.

  3. The limits of the construction are handled carefully. The experiments appear decent, covering two operators and two model families, with prompting and unsteered baselines included.

Critical weaknesses
  1. The claimed advantage over alternative steering schedules depends on tuning choices that are not fully documented. Although some controller parameters are specified, it is unclear how intervention layers and base strengths were selected across methods, whether they received comparable validation budgets, and what linear-decay horizon was used. Figure 6's selected-policy comparisons do not by themselves establish a Pareto-frontier advantage. Since these choices directly affect the control–capability trade-off, a modest equal-budget sensitivity check would strengthen the comparison.

  2. Several comparisons in Figures 5 and 6 are difficult to assess without uncertainty estimates. The results are largely reported as point estimates, without variation across independently sampled direction-construction or tuning sets. Although Appendix C specifies a shared 50-example tuning set excluded from final evaluation, it remains unclear which MMLU-Pro examples were used to enforce the requirement of retaining at least 80% of the unsteered capability score, and whether these examples were disjoint from those used for final capability evaluation. Clarifying this separation and reporting paired uncertainty estimates would strengthen the empirical conclusions.

  3. The empirical connection between the attention-level construction and practical DiM steering remains suggestive rather than established. The exact matching displacement depends on the prompt, context, and query, whereas the deployed DiM direction is reused across inputs. Figure 7 shows partial directional alignment for one model and concept, but does not establish whether DiM reproduces the prompted attention output or whether stronger matching predicts behavioral success. Figure 2(b), meanwhile, uses a separately fitted displacement to approximately match the prompted attention output at a reference representation. The paper acknowledges these limitations, so this primarily limits the strength of the proposed mechanistic explanation rather than the validity of the theoretical results.

  4. The evidence for general-capability preservation is relatively narrow. Concept-control experiments measure capability only with MMLU-Pro, while formatting experiments use MATH-500 accuracy. These are useful benchmarks, but they provide limited evidence for the broader claim that Prefix Steering preserves “general capabilities” across unrelated abilities.

Questions for authors
  1. For Figure 6, how were the intervention layer, base strength, and schedule parameters selected for each method, and what linear-decay horizon KK was used? Were all methods given the same validation budget? A small equal-budget sweep on one model/task would help verify that the comparison is robust to tuning.

  2. Could you clarify how the MMLU-Pro capability constraint was evaluated during layer and strength selection? In particular, which MMLU-Pro examples were used, were they disjoint from the final reported capability evaluation, and was the same capability-validation subset used consistently across methods and hyperparameter configurations?

  3. Could you report the evaluation-set sizes for the main concept-control and capability comparisons in Figures 5 and 6, and provide paired uncertainty estimates for the main Prefix-versus-Full comparisons? A small robustness check across independently sampled DiM direction-construction sets would also help establish whether the observed trade-off is stable to direction-estimation variability.

  4. For Figure 2, which attention layer/head was used and how was it selected? Given that the theory is primarily single-head while multi-head matching is more restrictive, can the main Figure 2 trends be verified across several layers/heads or using the projected multi-head output?

7 comments
AuthorsOpenReview ↗

Response to Reviewer sMMS (1/3)

All revisions made in response to this review are marked in blue in the revised paper.

W1 and Q1. Tuning of alternative steering schedules

Thank you for the detailed comments. In Figure 6, all policies use the layer selected for full steering. Prefix Steering, full steering, and both decay schedules select the strength scale ss on the tuning set, with K=512K=512 for linear decay and τ=128\tau=128 for exponential decay, while DAS and ACT use the values from their original papers (cmax⁡=2c_{\max}=2; A=12A=12, β=0\beta=0). To check that the comparison does not depend on these choices, we tuned all policies under a shared protocol on OLMo 3 7B for the safety task (Table 1). The layer, at 40% of depth, is selected jointly with strength for full steering on the 50-example tuning set and then shared by all policies, so it does not favor Prefix Steering. Each policy then selects its own strength on the same tuning set, from the search spaces listed in Table 1. Under this protocol, Prefix-5 dominates linear decay, exponential decay, DAS, and ACT, with both higher behavioral control and higher MMLU-Pro than each of them, while Prefix-1 best preserves general capability. We have added this experiment as a new Appendix D.5.

\hline \text{Policy} & \text{Selected strength} & \text{Control} & \text{MMLU-Pro} \\\\ \hline \text{Unsteered} & \text{–} & 36.0 & 39.4 \\\\ \hline \text{Prefix-1} & s=10 & 80.7 & \mathbf{39.0} \\\\ \text{Prefix-5} & s=10 & 85.4 & 36.6 \\\\ \text{Linear decay} & s=1 & 84.3 & 35.9 \\\\ \text{Exponential decay} & s=10 & 85.2 & 34.5 \\\\ \text{DAS} & c_{\max}=2 & 85.1 & 34.7 \\\\ \text{ACT} & A=30, \beta=0 & 84.6 & 35.8 \\\\ \text{Full} & s=0.1 & \mathbf{86.2} & 34.3 \\\\ \hline \end{array}$$ Table 1: Strength policies on OLMo 3 7B for the safety task. All policies use the same layer and select their strength on the same 50-example tuning set: the scale $s$ of $c_0=s\bar r$ from $\lbrace 0.01,0.1,1,10\rbrace$ (Appendix C), DAS's cap $c_{\max}$ from $\lbrace 1,2,5,10\rbrace$, and ACT's $A$ from $\lbrace 5,12,15,30,100\rbrace$ and $\beta$ from $\lbrace 0,0.1\rbrace$, covering the original values together with weaker and stronger settings. Linear decay uses horizon $K=512$, and exponential decay uses time constant $\tau=128$. **W2 and Q2. Data splits for the capability constraint** The MMLU-Pro examples used for selection and for final evaluation are disjoint. During layer and strength selection, we evaluate capability on the official MMLU-Pro validation split, and keep only settings that retain at least 80% of the unsteered score on it. Final capability is reported on the official test split. The same validation split is used for every method and every hyperparameter configuration. We have clarified this in Appendix C.
AuthorsOpenReview ↗

Response to Reviewer sMMS (2/3)

Q3. Evaluation-set sizes, uncertainty, and direction robustness

Table 2 lists the evaluation sets. For uncertainty, Prefix Steering and full steering are evaluated on the same inputs, so we compute paired bootstrap confidence intervals of their per-example differences. On all five tasks, Prefix-1 retains significantly more capability than full steering, while its control is only slightly lower (Table 3). To test robustness to direction estimation, we resample the 100 positive and 100 negative DiM examples with three seeds on OLMo 3 7B for safety, keeping the layer and strength fixed. Both control and capability vary little across seeds, and Prefix-1 retains more capability than full steering under every seed (Table 4). Unlike the DiM construction set, we did not resample the 50-example tuning set. Its effect is shared by both policies rather than favoring either: Prefix Steering and full steering select their strength on the same tuning examples, and both use the same layer. Moreover, the candidate strengths {0.01,0.1,1,10}\lbrace 0.01, 0.1, 1, 10\rbrace differ by an order of magnitude, so the selected strength depends on large differences in tuning scores rather than on small fluctuations across tuning examples. We have added the evaluation-set sizes to Appendix C and the uncertainty and robustness results as a new Appendix F.2.

\hline \text{Task} & \text{Control set} & n & \text{Capability set} & n \\\\ \hline \text{Safety} & \text{HarmBench, held out} & 350 & \text{MMLU-Pro test} & 12{,}032 \\\\ \text{Sentiment} & \text{SST-2 validation, negative} & 378 & \text{MMLU-Pro test} & 12{,}032 \\\\ \text{Politeness} & \text{PoliteGuard test, impolite} & 2{,}482 & \text{MMLU-Pro test} & 12{,}032 \\\\ \text{IF-Boxed} & \text{MATH-500, held out} & 400 & \text{MATH-500, held out} & 400 \\\\ \text{IF-Plain} & \text{MATH-500, held out} & 400 & \text{MATH-500, held out} & 400 \\\\ \hline \end{array}$$ Table 2: Evaluation sets for Figures 5 and 6. Tuning and direction-construction examples are excluded. $$\begin{array}{lccccccc} \hline & & \text{Behavioral control} & & & & \text{Capability} & \\\\ \text{Task} & \text{Prefix-1} & \text{Full} & \Delta \text{[95\\% CI]} & \quad & \text{Prefix-1} & \text{Full} & \Delta \text{[95\\% CI]} \\\\ \hline \text{Safety} & 80.7 & 86.2 & -5.5\enspace [-7.1, -3.9] & & 39.0 & 34.3 & +4.7\enspace [+4.2, +5.2] \\\\ \text{Sentiment} & 88.2 & 91.0 & -2.8\enspace [-4.3, -1.3] & & 39.1 & 34.8 & +4.3\enspace [+3.8, +4.8] \\\\ \text{Politeness} & 86.9 & 89.8 & -2.9\enspace [-4.4, -1.4] & & 38.9 & 34.0 & +4.9\enspace [+4.4, +5.4] \\\\ \text{IF-Boxed} & 91.6 & 95.0 & -3.4\enspace [-5.8, -1.1] & & 47.8 & 43.2 & +4.6\enspace [+1.0, +8.1] \\\\ \text{IF-Plain} & 49.7 & 52.2 & -2.5\enspace [-5.4, +0.5] & & 45.7 & 40.1 & +5.6\enspace [+2.0, +9.1] \\\\ \hline \end{array}$$ Table 3: Prefix-1 versus full steering with additive steering on OLMo 3 7B. $\Delta$ is Prefix-1 minus full steering, with 95% paired bootstrap confidence intervals. Capability is MMLU-Pro for concept control and MATH-500 for formatting. $$\begin{array}{lccccc} \hline & \text{Prefix-1} & & & \text{Full} & \\\\ \text{Seed} & \text{Control} & \text{MMLU-Pro} & \quad & \text{Control} & \text{MMLU-Pro} \\\\ \hline 1 & 80.7 & 39.0 & & 86.2 & 34.3 \\\\ 2 & 79.1 & 40.2 & & 85.4 & 35.1 \\\\ 3 & 81.8 & 38.4 & & 86.0 & 33.6 \\\\ \hline \text{Mean} \pm \text{std} & 80.5 \pm 1.4 & 39.2 \pm 0.9 & & 85.9 \pm 0.4 & 34.3 \pm 0.8 \\\\ \hline \end{array}$$ Table 4: Robustness to the DiM construction set on OLMo 3 7B, safety task. Each seed resamples the 100 positive and 100 negative examples; layer and strength are fixed.
AuthorsOpenReview ↗

Response to Reviewer sMMS (3/3)

W3. Link between the attention construction and DiM steering

Our goal is to connect prompting and activation steering. Since attention is the main mechanism through which later tokens use earlier ones, we analyze this connection at the attention level. The theory shows that a displacement depending on the prompt and context can exactly reproduce a prompt's attention-level effect. Constructing such a displacement for each input, however, would offer little beyond prompting itself. Instead, the connection motivates a practical method whose directions are easy to construct and reusable across inputs, such as DiM, while retaining the benefits of prompting. This motivates Prefix Steering. Even with DiM, the directions are positively aligned with the constructed displacements (Figure 7). The theory does not imply that a smaller error yields higher success, since prompting itself does not always succeed. Following Reviewer X4Cu's suggestion, to separate the effects of the modified representations in the key–value cache and of the generated text, we replay the same initial tokens with a fresh, unsteered cache, which keeps the text but removes the modified representations (Table 5). With Prefix Steering, the attention outputs remain much closer to those under prompting than with replay. For safety, where refusal appears in the first tokens, replay keeps most of the control. For IF-Boxed, where the format appears after long reasoning, replay loses most of the gain. The modified representations therefore carry information beyond the generated text. We have added this experiment as a new Section 4.3.

\hline & & \text{Safety} & & & & \text{IF-Boxed} & \\\\ \text{Condition} & \text{Control} & \text{MMLU-Pro} & \text{Attn. diff} & \quad & \text{Format} & \text{MATH-500} & \text{Attn. diff} \\\\ \hline \text{Unsteered} & 36.0 & 39.4 & \text{–} & & 48.7 & 48.3 & \text{–} \\\\ \text{Prefix-1, steered} & 80.7 & 39.0 & 0.421 & & 91.6 & 47.8 & 0.513 \\\\ \text{Prefix-1, replay} & 77.6 & 40.5 & 2.847 & & 57.2 & 48.7 & 3.426 \\\\ \text{Prefix-5, steered} & 85.4 & 36.6 & 0.368 & & 94.8 & 45.4 & 0.482 \\\\ \text{Prefix-5, replay} & 81.9 & 38.3 & 1.935 & & 64.1 & 48.9 & 4.127 \\\\ \hline \end{array}$$ Table 5: Replay on OLMo 3 7B with additive steering. *Steered*: Prefix Steering as in the paper. *Replay*: the same first $k$ generated tokens, recomputed with a fresh, unsteered cache, then generation continues without intervention. Attn. diff is the absolute difference between the attention outputs of each run and those under prompting, averaged over the generated tokens. **Q4. Layer and head in Figure 2** The original Figure 2 uses layer 27 of OLMo 3 7B, out of 32 layers, and query heads 0 and 16. We repeated its experiments on layers 9, 18, and 27, both for these two heads and with one displacement solved jointly for all heads, matching the projected multi-head output. Its trends hold in both settings. Matching all heads with one shared displacement is approximate but not much harder for longer prompts. The revised Figure 2 averages the three layers, and the new Appendix H.3 gives the joint multi-head results. **W4. Breadth of capability evaluation** Beyond MMLU-Pro and MATH-500, our new TruthfulQA experiments with ITI and Spherical Steering add a third view: the informativeness of open-ended answers (Table 6 and Appendix F.1). Steering only the final prompt token keeps answers informative, whereas full steering lowers informativeness. Together, the three measures cover multiple-choice knowledge, mathematical reasoning, and open-ended generation, and on all three, brief steering preserves capability better than full steering. We have added these experiments as Appendix F.1. $$\begin{array}{lccccc} \hline \text{Method} & \text{Span} & \text{True} & \text{Info} & \text{True} \times \text{Info} & \text{MMLU-Pro} \\\\ \hline \text{Unsteered} & \text{–} & 42.2 & 75.1 & 31.7 & 39.4 \\\\ \hline \text{ITI} & \text{Prompt tokens} & 74.4 & 93.8 & 69.8 & 37.2 \\\\ & \text{Prefix-1} & 73.5 & 94.1 & 69.2 & 38.9 \\\\ & \text{Full} & 83.2 & 90.4 & 75.2 & 32.8 \\\\ \hline \text{Spherical} & \text{Prompt tokens} & 86.8 & 97.9 & 85.0 & 37.8 \\\\ & \text{Prefix-1} & 86.3 & 97.1 & 83.8 & 38.0 \\\\ & \text{Full} & 92.0 & 87.2 & 80.2 & 34.1 \\\\ \hline \end{array}$$ Table 6: TruthfulQA on OLMo 3 7B with ITI and Spherical Steering applied to all prompt tokens, to the final prompt token only (Prefix-1), or to every token (Full). ITI uses the $K=48$ heads of the original method and selects its strength as in Appendix C; Spherical Steering selects its layer and strength as COAST does in Appendix C. True $\times$ Info is the product of True and Info.
Reviewer_pxPtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether activation steering can achieve effective behavioral control by intervening in only a few tokens rather than throughout the entire generation process. It then introduces Prefix Steering, which applies existing steering vectors to a short span beginning at the final prompt token and stops intervening afterward. Across four models and five tasks, short prefixes often retain much of full steering’s behavioral control while better preserving general capabilities.

Critical strengths
  1. Interesting and meaningful research question. The paper investigates whether steering only a few tokens can achieve effective activation steering while avoiding model collapse.
  2. Simple and effective method.
  3. Effective experimental results.
Critical weaknesses
  1. Many existing methods construct steering vectors using representations of both prompts and responses, whereas this paper relies solely on prompt representations. Since prompt labels are LLM-dependent, this may introduce noise. For example, the same prompt may elicit harmful responses from one LLM but safe responses from another. Consequently, prompt labels may not accurately reflect the actual safety behavior of the target model, potentially affecting the reliability of the constructed steering vectors.
  2. Please discuss and compare their work with [1, 2], highlighting the key differences.
  3. Comprehensively investigate the effectiveness of steering only prompt tokens, without intervening in generated tokens.
  4. Evaluation on the TruthfulQA dataset. It is unclear whether the current evaluation datasets allow models to improve their scores by generating uninformative responses.
  5. The proposed method should be evaluated with a wider range of activation steering methods to further validate its generalizability, such as ITI [3] and Spherical Steering [4].

[1] Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions, ICML 2026

[2] Steer Like the LLM: Activation Steering that Mimics Prompting, ICML 2026

[3] Inference-Time Intervention: Eliciting Truthful Answers from a Language Model, NeurIPS 2023.

[4] Spherical Steering: Geometry-Aware Activation Rotation for Language Models, ICML 2026.

Questions for authors

See Weaknesses

4 comments

All revisions made in response to this review are marked in blue in the revised paper.

W1. Prompt-only labels for steering vectors

Thank you for your valuable comments. Prompt-only directions are common practice [5–8]. Such directions capture the contrast between inputs (e.g., harmful vs. benign requests, positive vs. negative sentiment) rather than predicting the model's response, while the model-specific layer and strength are selected by the target model's own steered behavior. In the additional ITI and Spherical Steering experiments below, the directions are instead built from responses, using the final-token activations of TruthfulQA's correct and incorrect answers, and Prefix Steering remains effective with them (Table 1).

[5] Turner et al. Steering Language Models with Activation Engineering. 2023.
[6] Arditi et al. Refusal in Language Models Is Mediated by a Single Direction. 2024.
[7] Nguyen et al. Minimizing Collateral Damage in Activation Steering. 2026.
[8] Stolfo et al. Improving Instruction-Following in Language Models through Activation Steering. 2025.

W2. Comparison with [1, 2]

[1] jointly trains a steering direction and its strength, and intervenes only on a few prompt tokens during prefill. [2] trains a token-wise model that learns to imitate the activations induced by a prompt. Both require training for each concept. In contrast, our method is training-free. On the theory side, we give sufficient conditions under which steering existing tokens matches the attention output induced by a prompt, together with an explicit construction. On the empirical side, we show that an existing DiM direction with standard operators already suffices when applied over a short span starting at the final prompt token. The findings of [1] are consistent with ours, and we additionally study how duration and strength trade off. We have added this discussion to the related work in Appendix B.

W3–W5. Prompt-only steering, TruthfulQA, and more steering methods

Prefix-1 in the main experiments already steers only the final prompt token, without intervening in generated tokens, and it retains much of full steering's control across four models, five tasks, and two operators (Figure 5). Following the review's suggestion, we further added TruthfulQA experiments on OLMo 3 7B with ITI and Spherical Steering, applied to all prompt tokens, to the final prompt token only, or to every token (Table 1). With both methods, steering only prompt tokens substantially improves truthfulness over the unsteered model, and steering the final prompt token alone performs close to steering all of them. To rule out gains from uninformative answers, we also report Info and True × Info. Prompt-only steering keeps answers informative. Full steering, by contrast, intervenes on every generated token and produces more off-topic or evasive answers, which are judged uninformative. The same cost appears in general capability: steering the final prompt token keeps MMLU-Pro close to the unsteered model, whereas full steering reduces it noticeably. We have added these results as a new Appendix F.1.

MethodSpanTrueInfoTrue × InfoMMLU-Pro
Unsteered–42.275.131.739.4
ITIPrompt tokens74.493.869.837.2
ITIPrefix-173.594.169.238.9
ITIFull83.290.475.232.8
SphericalPrompt tokens86.897.985.037.8
SphericalPrefix-186.397.183.838.0
SphericalFull92.087.280.234.1

Table 1: TruthfulQA on OLMo 3 7B with ITI and Spherical Steering applied to all prompt tokens, to the final prompt token only (Prefix-1), or to every token (Full). ITI uses the K = 48 heads of the original method and selects its strength as in Appendix C; Spherical Steering selects its layer and strength as COAST does in Appendix C. True × Info is the product of True and Info.

Thank you again for your feedback. We would like to check whether our response has addressed your concerns. If any question remains, we are happy to clarify before the discussion period ends.

Thank you for your reply. My issues have been resolved, and I've raised the score to 3.

Thank you for your reply and for raising your score. We appreciate your constructive feedback, which has helped us improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.