acceptodds
Withdrawn from ICLR 2027

Steering by Influence: Curvature Aware Data Weighting for Activation Steering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Inference-time steering offers cheap, fine-grained control over a language model's outputs by estimating a concept's representation in activation space and shifting activations towards it. Existing methods build these representations from activation averages over contrastive datasets. These averages incorporate unrelated concepts and noise, and are dominated by a few tokens, meaning the activation transport encodes token-level rather than thematic concepts. In this work, we steer towards examples that most express a concept thematically, rather than towards an expectation over all. We identify these examples using influence functions, which estimate how much each data point contributes to a model's representation of a concept. Unlike simple model activation similarity, they incorporate the curvature of the model's loss landscape, allowing them to capture concept-relevant relationships beyond superficial token-level similarity. We then propose influence-weighted activation transport, which uses optimal transport to steer activations of non-concept text towards those of concept text, weighting concept examples by their influence scores. We evaluate on toxicity suppression (Jigsaw), object-based concept induction (OneSec) and truthfulness induction (TruthfulQA), outperforming existing activation-transport baselines. We track capability after steering using perplexity and MMLU accuracy, finding that our method improves steering while largely preserving model quality. We further show that influence functions capture concept-relevant information that activation-based methods miss with the two approaches ranking data points significantly differently. Together, these results demonstrate the value of curvature-aware influence information for activation steering.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, influence functions, optimal transport, mechanistic interpretability, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8nYbOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
1(−2)
Read review
Paper summary

The paper proposes weighting samples used in activation steering to account for their influence on the target concept. The authors apply this genuinely interesting idea to two distribution-aware steering methods, Act and LinEAS, claiming that incorporating influence scores improves steering performance while better preserving model capabilities.

Critical strengths
  • The central idea is genuinely interesting. I think that weighting training samples during steering fitting makes sense, and the paper tackles this problem in a very creative manner.

  • IF scores are applied to both a non-end-to-end fitted activation steering method (Act) and an end-to-end fitted steering method (LinEAS).

  • The evaluation is thorough across different tasks. The authors test the method on truthfulness, toxicity, and general concept induction.

  • The method is well presented despite its technical complexity.

Critical weaknesses
  • The paper strongly overclaims the benefits of IF. When added to Act in the Concept Score experiment, IF substantially increases PPL while providing little or no improvement in Concept Score. Similarly, in the Toxicity experiment, IF-Act significantly affects model capabilities, particularly MMLU. In my opinion, this does not support the claim that influence scores benefit Act. IF-LinEAS seems more promising, with a smaller effect on model quality but are not comparable (see point below). Overall, I think the claims about the benefits of IF are not sufficiently supported by the results.

  • The comparisons do not account for the steering–model preservation trade-off. Adding influence scores seems to move the operating point along the Pareto front rather than necessarily improving it. Comparing methods at a single steering strength therefore makes it difficult to determine whether IF actually provides a benefit. I think the authors should compare the Pareto fronts of the weighted and unweighted methods. Without this analysis, it is unclear whether IF improves the trade-off or simply sacrifices model quality for stronger steering.

Questions for authors
  1. In order to compute the influence scores, the authors use (q) (line 243), computed from triplets {x, y+, y-}, which seem to be paired. However, for training the steering methods, the authors use unpaired data (e.g., Jigsaw). Are the authors using different data for the estimation of the influence scores and the steering direction? If not, aren't the IF scores affected by the fact that there are no correct data pairs (i.e., training noise)?

  2. LinEAS and IF-LinEAS are sometimes very close. Is one statistically better than the other, or is this just an artifact of the execution realization?

  3. This does not require any experiment; it is just a curiosity in case the authors tried it. Why not use the obtained qq as a steering direction? It feels odd to me that the paper reaches a point where it claims to compute a clean steering direction, which is not used for steering afterwards.

4 comments

Dear Reviewer 8nYb,

Thank you for taking the time to review our work. Please see our responses to your comments below:

W2. Steering–preservation trade-off:

Thank you for this comment. We agree that comparing at a single λ can't distinguish a better trade-off from a shifted operating point. We sweep λ from 0 (unsteered) to 1 (the full transport map) separately for each model, method and task (Figs. 2, 8 and 11, and the full tables in the appendix), but only reported λ=1 in the main tables to save space. In the revision we have moved the toxicity sweep into the main text (Fig. 2). It plots toxicity and fluency perplexity across λ for all four methods on both models, so the quality cost of each method is visible at every steering strength rather than at a single point. We were not able to evaluate MMLU at every λ, because of the cost of running the full benchmark repeatedly and our limited academic compute. MMLU is therefore reported at λ=1 (Tables 1 and 3). If the paper is accepted, we will run MMLU across the full λ sweep for the camera-ready version, so that concept score can be compared directly against both perplexity and MMLU. On toxicity suppression, IF-LinEAS reaches lower toxicity than LinEAS at the same MMLU (Gemma: 0.018 vs 0.095, both 0.520 MMLU; Llama: 0.052 vs 0.101, both 0.554 MMLU), at the cost of a small increase in perplexity (Gemma WikiText +2.1, Llama fluency +1.1). IF-Act trades quality for stronger steering. We state this explicitly in the revision rather than claim a uniform improvement.

Q1.

Thank you for the question. Yes, the datasets are different by design. The K contrastive triples are used only to define the measurement function ϕ, and hence the parameter-space direction q, in the influence score. The steering data (e.g. Jigsaw) are the candidate points that q scores and weights. The transport map is then fitted on these weighted candidates. q is never used as a steering direction. The IF score of a candidate, −qᵀ(G+λI)⁻¹∇L(z_m), depends only on that candidate's own loss gradient. It does not require the candidates to be paired, so the lack of pairs in the steering data does not introduce noise into the scores. Pairing matters only for q. A single prompt-completion pair gives a noisy direction that also captures lexical features specific to that pair (Sec. 4.1), so we average over multiple contrastive triples to isolate the consistent concept component. We agree this distinction was not clear. In the revision we state it explicitly.

Q2.

Thank you for raising this. On Gemma-2-2B, we re-ran toxicity induction and all seven OneSeC concepts over 5 seeds, randomising data subsampling, training minibatch order, the EKFAC curvature corpus, and sampling and evaluation seeds. We then tested the IF-LinEAS − LinEAS gap with paired t-tests (full details in Appendix R, LinEAS vs. IF-LinEAS Significance). Toxicity induction: the gap is not seed noise. IF-LinEAS increases the RTP toxic rate (higher is better for induction) by +0.41 (p = 3×10⁻⁵, positive in all 5 seeds, about 9× the seed-to-seed SD), at the cost of about 3 points higher perplexity. OneSeC: the picture is mixed. IF-LinEAS is significantly better on some concepts (Baby, Book, Flower) and significantly worse on others (Football, Balloon), with no significant difference on the remaining two. As a result, the mean gap across concepts is not significant (+0.042, p = 0.31). Where the methods differ, however, the gap has the same sign in all five seeds. The seed-to-seed SD of the gap (0.025–0.049) is well below the spread of gaps between concepts (0.100). The close results therefore reflect real, concept-dependent differences between the methods, not an artifact of the random realisation. We agree that the OneSeC results do not support a universal advantage for IF-LinEAS, instead being concept dependent, and we have revised our claims accordingly.

Kind regards,

The Authors

Thank you for your rebuttal response.

  • Regarding W2. I appreciate the new figures, but I think the current plots still do not address my concern. (1) The PPL axis spans a very large range, making differences between methods difficult to observe. A zoomed-in view would be helpful, as even a 5% PPL increase could be concerning. (2) The shaded regions (±1\pm 1 std) are very wide. What do they represent? (3) It is difficult to assess the trade-off with separate plots for toxicity and PPL. I would strongly encourage adding a Pareto plot, with toxicity on the x-axis and PPL on the y-axis, showing one point per steering strength.

  • Regarding the overall contribution. If IF-LinEAS does not consistently and statistically outperform LinEAS across concepts, and some improvements come at a PPL cost, what conclusions remain about the benefits of influence weighting? What concrete performance benefit does IF provide over the unweighted baseline? Does it provide benefit for Act? Does it provide benefit for LinEAS?

Thank you again for your further comments.

Regarding W2)

Ans 1) We have replaced the PPL axis with the % change in perplexity from the unsteered model (λ = 0) and zoomed each panel to the methods’ range, capped at +100%. A dotted line marks +5%, and values above +100% are shown as triangles on the top edge, labelled with each method’s peak (Figs. 2, 8 and 12).

Ans 2) The shaded regions show ±1 standard deviation across individual generations at each steering strength (500 generations per λ, pooled across prompts). They show the spread of the outputs, not the uncertainty in the mean. For the OneSec average (Fig. 8), the band is the standard deviation across the seven concepts.

Ans 3) To make the trade-off between concept strength and fluency clearer, we have added Pareto plots of concept score against the % change in perplexity from the unsteered model, with one point per steering strength and a +5% reference line. These are in Appendix M (Fig. 9, OneSec) and Appendix N (Fig. 13, toxicity suppression and induction).

Regarding the overall contribution)

We agree that IF-LinEAS does not beat LinEAS on every concept, and we have narrowed our claim to match. Our claim is that influence weighting reliably improves optimal-transport steering for broad, thematic concepts, with toxicity as the clearest case. For narrow, object-level concepts, the effect depends on the concept, as we anticipated in Section 5 (“General Concept Induction”).

Benefit for LinEAS: Toxicity suppression (Table 3, Appendix N, Table 11). IF-LinEAS lowers toxicity relative to LinEAS on 11 of 12 evaluations (e.g. 0.095 → 0.018 on Gemma, 0.101 → 0.052 on Llama). MMLU is unchanged, and fluency perplexity stays within 3% of the unsteered model. The gain also holds with the first and last layers excluded. Toxicity induction (Appendix R, Table 14). The gain is significant over five paired seeds (p < 0.001, positive in all five). The Pareto plots (Appendix N, Fig. 13) show that IF-LinEAS has the lowest perplexity cost of any method at every toxicity level it reaches. Robustness (Appendix T). When D⁻ is contaminated with concept data, LinEAS falls to the unsteered baseline, but IF-LinEAS keeps most of its effect.

OneSec (Appendix R, Tables 15–16): The effect depends on the concept. IF-LinEAS is significantly better on three concepts and significantly worse on two. Neither the mean gain nor the mean fluency cost is significant. On Llama, IF-LinEAS reaches concept scores LinEAS does not reach at any steering strength (Appendix M, Fig. 9).

Benefit for AcT: IF-AcT gives the best MC2 on TruthfulQA for both models (Table 2). For toxicity and OneSec it raises concept scores, but at a prohibitive cost in perplexity and MMLU (Appendices M–N).

Kind regards,

The Authors

Regarding W2. The PDF does not seem to have been updated. Could you please double-check that the authors have uploaded the correct version?

Regarding the overall contribution.

  • For LinEAS: Table 3 shows only one operating point on the Pareto front, making it difficult to properly assess whether IF-LinEAS improves upon LinEAS. Although the results seem to suggest that it does, without the full Pareto front, this cannot be conclusively determined. The authors also refer to Figure 14, which does not appear in the current version.

  • For Act: Showing that MC2 is higher for IF-Act in Table 2 does not provide meaningful evidence without also showing its impact on model performance. Additionally, if IF-Act improves toxicity and OneSec at a prohibitive cost, what is the benefit of using IF compared to simply further increasing the Act steering strength? Appendices M and N are inconclusive based on the current plots.

I believe that the influence scores are a creative and interesting idea. However, in its current form, the paper does not provide sufficient evidence to establish whether they work as intended. Therefore, I do not think the paper is ready for acceptance, and I am lowering my score. That said, I would be happy to reconsider my assessment if the authors adequately address these concerns.

Reviewer_txo8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Transport-based steering (AcT, LinEAS) fits per-dimension linear maps from non-concept activations to concept activations, weighting every example equally. This paper reweights the concept examples by influence in 3 steps:

  1. Build a concept direction in parameter space from synthetic contrastive prompt/completion triples.
  2. Score each concept example with EK-FAC influence against that direction, separately per layer.
  3. Z-score and softmax the scores and use them as weights in the 1-D transport coupling.

Experiments use two smal LLMs on OneSec concept induction, TruthfulQA, and Jigsaw toxicity suppression. The authors find that influence weighting usually improves steering, and that influence rankings differ from activation-projection rankings.

Critical strengths
  • The method itself is easy to follow. Figure 1 and Sec. 4 lay out the pipeline from contrastive triples to weighted transport clearly, even if the experiments section is rougher.
  • The problem is well-motivated. Steering vectors are averages over noisy, token-heavy examples, and asking which examples should define a concept is a good question.
  • On Llama toxicity, IF-LinEAS improves on LinEAS on the RTP and TET prompts with MMLU unchanged.
Critical weaknesses
  1. There is no prompting baseline. Modern models are excellent at steering through simple prompts, but the authors don't compare against any prompting. All three tasks have obvious prompt-only versions: an instruction to continue politely for toxicity, "write about a balloon" (or a few in-context examples) for OneSec, and the instructive "helpful" prompt from the original TruthfulQA paper, which already improves truthfulness without touching activations. Prompting costs nothing and needs no influence computation. Recent work (e.g., AxBench) has found that prompting often beats representation steering on exactly this kind of concept control. Without it, I can't tell whether any steering method here, weighted or not, is worth using.
  2. Methods are compared at a fixed steering strength: it would be nice otherwise tune these somewhat for each method.
  3. There seems to be a discrepancy in Table 3, where the the pooled "Toxicity" column has unsteered scores higher than both RTP and TET, so it can't be the prompts Sec. 5 describes. Also, in the TruthfulQA table, Act and LinEAS change with the number of triples even though they don't use them.

Smaller note: instance-level weighting of steering examples has been proposed before via semantic attribution (SAAS, "Instance-Semantic Attribution for Activation Steering", OpenReview 2026) and should be discussed.

Questions for authors
  1. Can you add prompt-only baselines for all three tasks, and prompting combined with steering?
  2. Can you tune steering strength for different models?
1 comment
AuthorsOpenReview ↗

First comment

Dear Reviewer txo8,

Thank you for taking the time to review our work. Please see our responses to your comments below:

W3. Table discrepancies:

Thank you for flagging this possible point of confusion. Table 3 has no ‘pooled’ column. It reports three independent evaluations, “Toxicity”, “RTP” and “TET”. Each has its own prompts and generations, and each is scored by both an LLM judge and a RoBERTa classifier. The “Toxicity” column is not an average of RTP and TET, so its unsteered rate needn’t lie between theirs. [It uses [prompt source, e.g. Jigsaw-derived prompts] with the generation settings in Table 5 (50 new tokens), whereas RTP and TET use the settings in Table 6 (32 new tokens).] In the revision we will describe the “Toxicity” column more clearly in the Table 3 caption.

Q1. (and W1. No prompting baseline):

Thank you for your response. We have not included a prompting baseline because activation steering is a fundamentally different methodology.

Steering is also a mechanistic technique. It intervenes directly on the model’s activations at inference time, so it does not consume context, does not depend on the model’s instruction-following, and cannot be overridden by text in the user’s input. Prompting acts on the input, and we cannot reliably quantify how the model interprets a change in prompt. Activation steering lets us measure where and how activations change, which makes it a more controllable technique. In our transport method, the strength λ gives a continuous dial over the exact amount of steering applied, and the influence weights give a per-layer breakdown of where a concept is represented (Fig. 12). Prompting has no comparable control. Our contribution is a better way to estimate a concept’s representation in activation space, by weighting examples with curvature-aware influence scores. We therefore compare against the methods this applies to, AcT and LinEAS, which are in the same family and use the same hook points. Prompting has no analogue of this data-weighting step.

Q2. (and W2. Fixed steering strength): Thank you for the question. We sweep λ from 0 to 1 separately for Gemma-2-2B and Llama-3.2-3B for every method and task (Figs. and the full tables in the appendix). The main text tables fix λ=1 for simplicity and space restrictions, but the sweeps show the optimal trade-off differs by model and method. In the revision we will describe the “Toxicity” column more clearly in the Table 3 caption to avoid any misunderstanding.

Kind regards,

The Authors

Reviewer_qyJ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes weighting the concept-dataset (D+) samples in transport-based activation steering by layer-wise EK-FAC influence scores, instead of weighting them uniformly. The concept direction used to compute influence is defined with contrastive prompt-completion triplets. Building on AcT and LinEAS, the authors propose IF-Act (closed form) and IF-LinEAS. Experiments use Gemma-2-2B and Llama-3.2-3B on OneSec, TruthfulQA, and toxicity suppression.

Critical strengths

S1. The weighting is a simple addition to existing transport steering, with a single temperature that interpolates between uniform and influence-based weights.

S2. On toxicity suppression, IF-LinEAS achieves lower toxicity than LinEAS on both models, with the same MMLU.

Critical weaknesses

W1. Limited comparison with alternative weightings.

Projection and gradient-norm scores are used for rank correlation, and gradient similarity is discussed as the limiting case of influence without curvature correction. However, none of these, nor random weights, are evaluated for steering at matched ESS. As a result, the specific benefit of curvature-aware influence remains unclear.

W2. Comparison at matched quality.

Stronger steering generally affects model quality, but the methods are not compared at the same quality level. The main tables report λ=1\lambda = 1, where IF-Act shows a larger MMLU drop and higher Wiki PPL than Act (e.g., Gemma OneSec: MMLU 0.267 vs 0.466, Wiki PPL 1010 vs 91), and the appendix λ\lambda sweeps do not include MMLU or Wiki PPL. This makes the steering gains hard to compare fairly.

W3. Model coverage.

Experiments use two relatively small models, and the results differ between them. Hook locations also differ across the two models.


Minor:

  1. The LLM-judge metric is described as the fraction of generations judged yes in Section 5, and as a pooled "yes" probability in Appendix I. The number of generations (500 vs 1000 toxicity prompts) could also be clarified.

  2. In Table 4 (line 1009), the Llama row appears misaligned and lists Llama-3-8B, while the experiments use Llama-3.2-3B.

  3. Notation could be more consistent (e.g., λ\lambda denotes both steering strength and the damping term).

Questions for authors

Q1. When used for steering at matched ESS, how does influence weighting compare with projection, gradient-similarity, and random weights?

Q2. Section 4.1 states that plain prompt-completion pairs and contrastive triplets are compared in Appendix G, but Figure 4 varies only k. What is the effect of triplets compared with plain pairs on steering performance?

Q3. Since only D+ is weighted, D- directly shapes the map. How sensitive are the results to the composition of D- (e.g., OneSec uses other concepts rather than neutral text) and of D+?

Q4. How were sentence lengths matched between D+ and D-? If their length distributions differ, could the map partly reflect length rather than the concept?

Q5. Could you report the distribution of α per layer and the ESS, and plot concept score against MMLU or Wiki PPL across λ\lambda?

Q6. What does n in Table 2 refer to?

1 comment

Dear Reviewer qyJ2,

Thank you for this fantastic review. Apologies for the delay.

W3) We use the hook locations that AcT (Rodriguez et al., ICLR 2025) and LinEAS (Rodriguez et al., 2025) found best for each architecture, so they differ between models. The differences in results likely reflect architecture and pretraining data. We expect the approach to carry over to larger models. Grosse et al. (2023) find that as models scale, their most influential training sequences become more conceptually related to the query, not merely matched on surface tokens. This should make influence-based weighting more effective at scale.

Thank you we will clarify all three in the revision: M1 (metrics): The concept score is the mean p(yes) over 500 generations per (prompt, λ), reported for every concept, toxicity included. The toxicity rate is the fraction of the 1,000 RealToxicityPrompts continuations with p(yes) > 0.5, reported only for toxicity as an extra metric. M2 (models): Llama-3-8B is the LLM judge and Llama-3.2-3B is the steered model. M3 (notation): The dampening factor is now ζ and λ is kept for steering strength only.

Q1) On Gemma-2-2B (toxicity induction and suppression, IF-LinEAS), we compared IF against gradient similarity, activation projection and a random permutation of the IF weights, each at its own temperature and at IF’s ESS (Appendix S). For induction, no proxy at its own temperature comes close to IF. Projection roughly matches IF only when it borrows IF’s weights or ESS. Gradient similarity stays well behind even with IF’s exact weights (ρ = 0.86). In suppression, all weightings reach near-zero toxicity, but every alternative costs more perplexity than IF. The random permutation, with IF’s exact values and ESS, loses about half of IF’s gain in induction. So IF’s benefit is not explained by weight concentration or rank agreement alone. Compute and time limits prevented more concepts and models during the review window. We will include them in the camera-ready version.

Q2) We have added this to Appendix G (“Measurement Function Proxies”), comparing three concept definitions built from the same K=58 toxicity triples on Gemma-2-2B: (C) the contrastive triplets used in the paper. (P) prompt-masked plain positive prompt–completion pairs. (W) plain positive pairs, with log-likelihood over the whole sequence. P isolates the effect of the contrast while W isolates the effect of masking the prompt. Masking the prompt matters more for steering than the contrast itself. Contrastive triplets give a much cleaner concept direction and are a safe default, but are not a strict improvement over masked plain pairs for steering. We have revised Section 4.1 accordingly.

Q3) We ran two sensitivity experiments on Gemma-2-2B (toxicity induction, lambda =1, Appendix T). D⁻ composition. We replaced a growing fraction of D⁻ with held-out D⁺ samples, a harder test than using other concepts, since D⁻ then contains the target concept. As D⁻ is fully replaced, AcT and LinEAS fall to the unsteered baseline. IF-LinEAS stays robust (concept score 0.49–0.67 at every level), and IF-AcT keeps part of its effect. Across 10 random halves of D⁻, every method except IF-AcT was stable (standard deviation 5–15× smaller than the mean, perplexity varied <13%). D⁺ composition. IF weighting gives D⁺ an effective sample size of about 20–25 (median over blocks, Tables 19, 22), so a small set of high-influence samples drives the map, and we expect the rest of D⁺ to matter less. Testing this requires recomputing influence scores per condition, which did not fit the rebuttal period. We will add it to the camera-ready version if accepted.

Q4) Our influence score uses the summed (not length-normalised) cross-entropy loss over each candidate. We tested this (Appendix U). With the summed loss, a candidate’s influence score correlates only weakly with its length in either model, consistently across blocks. Length normalisation instead introduces a strong negative correlation, pushing longer candidates’ scores down, so we keep the summed loss.

Q5) Per-layer influence weights are in Fig. 14 (Appendix P): they concentrate in the middle layers for toxicity and are near-uniform for OneSec and TruthfulQA. Per-layer ESS across softmax temperatures is in Appendix K (Fig. 6) and stays at 10–100 in the middle layers. We also add Pareto plots of concept score against percentage change in perplexity (Appendix M, Fig. 9, Appendix N, Fig. 13).

q6) n is the number of data points in each contrastive dataset. We have added this to the Table 2 caption.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.