- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1(−2)
Read review
Paper summary
The paper proposes weighting samples used in activation steering to account for their influence on the target concept. The authors apply this genuinely interesting idea to two distribution-aware steering methods, Act and LinEAS, claiming that incorporating influence scores improves steering performance while better preserving model capabilities.
Critical strengths
-
The central idea is genuinely interesting. I think that weighting training samples during steering fitting makes sense, and the paper tackles this problem in a very creative manner.
-
IF scores are applied to both a non-end-to-end fitted activation steering method (Act) and an end-to-end fitted steering method (LinEAS).
-
The evaluation is thorough across different tasks. The authors test the method on truthfulness, toxicity, and general concept induction.
-
The method is well presented despite its technical complexity.
Critical weaknesses
-
The paper strongly overclaims the benefits of IF. When added to Act in the Concept Score experiment, IF substantially increases PPL while providing little or no improvement in Concept Score. Similarly, in the Toxicity experiment, IF-Act significantly affects model capabilities, particularly MMLU. In my opinion, this does not support the claim that influence scores benefit Act. IF-LinEAS seems more promising, with a smaller effect on model quality but are not comparable (see point below). Overall, I think the claims about the benefits of IF are not sufficiently supported by the results.
-
The comparisons do not account for the steering–model preservation trade-off. Adding influence scores seems to move the operating point along the Pareto front rather than necessarily improving it. Comparing methods at a single steering strength therefore makes it difficult to determine whether IF actually provides a benefit. I think the authors should compare the Pareto fronts of the weighted and unweighted methods. Without this analysis, it is unclear whether IF improves the trade-off or simply sacrifices model quality for stronger steering.
Questions for authors
-
In order to compute the influence scores, the authors use (q) (line 243), computed from triplets {x, y+, y-}, which seem to be paired. However, for training the steering methods, the authors use unpaired data (e.g., Jigsaw). Are the authors using different data for the estimation of the influence scores and the steering direction? If not, aren't the IF scores affected by the fact that there are no correct data pairs (i.e., training noise)?
-
LinEAS and IF-LinEAS are sometimes very close. Is one statistically better than the other, or is this just an artifact of the execution realization?
-
This does not require any experiment; it is just a curiosity in case the authors tried it. Why not use the obtained as a steering direction? It feels odd to me that the paper reaches a point where it claims to compute a clean steering direction, which is not used for steering afterwards.
4 comments
Thank you for your rebuttal response.
-
Regarding W2. I appreciate the new figures, but I think the current plots still do not address my concern. (1) The PPL axis spans a very large range, making differences between methods difficult to observe. A zoomed-in view would be helpful, as even a 5% PPL increase could be concerning. (2) The shaded regions ( std) are very wide. What do they represent? (3) It is difficult to assess the trade-off with separate plots for toxicity and PPL. I would strongly encourage adding a Pareto plot, with toxicity on the x-axis and PPL on the y-axis, showing one point per steering strength.
-
Regarding the overall contribution. If IF-LinEAS does not consistently and statistically outperform LinEAS across concepts, and some improvements come at a PPL cost, what conclusions remain about the benefits of influence weighting? What concrete performance benefit does IF provide over the unweighted baseline? Does it provide benefit for Act? Does it provide benefit for LinEAS?
Comment 2
Thank you again for your further comments.
Regarding W2)
Ans 1) We have replaced the PPL axis with the % change in perplexity from the unsteered model (λ = 0) and zoomed each panel to the methods’ range, capped at +100%. A dotted line marks +5%, and values above +100% are shown as triangles on the top edge, labelled with each method’s peak (Figs. 2, 8 and 12).
Ans 2) The shaded regions show ±1 standard deviation across individual generations at each steering strength (500 generations per λ, pooled across prompts). They show the spread of the outputs, not the uncertainty in the mean. For the OneSec average (Fig. 8), the band is the standard deviation across the seven concepts.
Ans 3) To make the trade-off between concept strength and fluency clearer, we have added Pareto plots of concept score against the % change in perplexity from the unsteered model, with one point per steering strength and a +5% reference line. These are in Appendix M (Fig. 9, OneSec) and Appendix N (Fig. 13, toxicity suppression and induction).
Regarding the overall contribution)
We agree that IF-LinEAS does not beat LinEAS on every concept, and we have narrowed our claim to match. Our claim is that influence weighting reliably improves optimal-transport steering for broad, thematic concepts, with toxicity as the clearest case. For narrow, object-level concepts, the effect depends on the concept, as we anticipated in Section 5 (“General Concept Induction”).
Benefit for LinEAS: Toxicity suppression (Table 3, Appendix N, Table 11). IF-LinEAS lowers toxicity relative to LinEAS on 11 of 12 evaluations (e.g. 0.095 → 0.018 on Gemma, 0.101 → 0.052 on Llama). MMLU is unchanged, and fluency perplexity stays within 3% of the unsteered model. The gain also holds with the first and last layers excluded. Toxicity induction (Appendix R, Table 14). The gain is significant over five paired seeds (p < 0.001, positive in all five). The Pareto plots (Appendix N, Fig. 13) show that IF-LinEAS has the lowest perplexity cost of any method at every toxicity level it reaches. Robustness (Appendix T). When D⁻ is contaminated with concept data, LinEAS falls to the unsteered baseline, but IF-LinEAS keeps most of its effect.
OneSec (Appendix R, Tables 15–16): The effect depends on the concept. IF-LinEAS is significantly better on three concepts and significantly worse on two. Neither the mean gain nor the mean fluency cost is significant. On Llama, IF-LinEAS reaches concept scores LinEAS does not reach at any steering strength (Appendix M, Fig. 9).
Benefit for AcT: IF-AcT gives the best MC2 on TruthfulQA for both models (Table 2). For toxicity and OneSec it raises concept scores, but at a prohibitive cost in perplexity and MMLU (Appendices M–N).
Kind regards,
The Authors
Regarding W2. The PDF does not seem to have been updated. Could you please double-check that the authors have uploaded the correct version?
Regarding the overall contribution.
-
For LinEAS: Table 3 shows only one operating point on the Pareto front, making it difficult to properly assess whether IF-LinEAS improves upon LinEAS. Although the results seem to suggest that it does, without the full Pareto front, this cannot be conclusively determined. The authors also refer to Figure 14, which does not appear in the current version.
-
For Act: Showing that MC2 is higher for IF-Act in Table 2 does not provide meaningful evidence without also showing its impact on model performance. Additionally, if IF-Act improves toxicity and OneSec at a prohibitive cost, what is the benefit of using IF compared to simply further increasing the Act steering strength? Appendices M and N are inconclusive based on the current plots.
I believe that the influence scores are a creative and interesting idea. However, in its current form, the paper does not provide sufficient evidence to establish whether they work as intended. Therefore, I do not think the paper is ready for acceptance, and I am lowering my score. That said, I would be happy to reconsider my assessment if the authors adequately address these concerns.
Comment 1
Dear Reviewer 8nYb,
Thank you for taking the time to review our work. Please see our responses to your comments below:
W2. Steering–preservation trade-off:
Thank you for this comment. We agree that comparing at a single λ can't distinguish a better trade-off from a shifted operating point. We sweep λ from 0 (unsteered) to 1 (the full transport map) separately for each model, method and task (Figs. 2, 8 and 11, and the full tables in the appendix), but only reported λ=1 in the main tables to save space. In the revision we have moved the toxicity sweep into the main text (Fig. 2). It plots toxicity and fluency perplexity across λ for all four methods on both models, so the quality cost of each method is visible at every steering strength rather than at a single point. We were not able to evaluate MMLU at every λ, because of the cost of running the full benchmark repeatedly and our limited academic compute. MMLU is therefore reported at λ=1 (Tables 1 and 3). If the paper is accepted, we will run MMLU across the full λ sweep for the camera-ready version, so that concept score can be compared directly against both perplexity and MMLU. On toxicity suppression, IF-LinEAS reaches lower toxicity than LinEAS at the same MMLU (Gemma: 0.018 vs 0.095, both 0.520 MMLU; Llama: 0.052 vs 0.101, both 0.554 MMLU), at the cost of a small increase in perplexity (Gemma WikiText +2.1, Llama fluency +1.1). IF-Act trades quality for stronger steering. We state this explicitly in the revision rather than claim a uniform improvement.
Q1.
Thank you for the question. Yes, the datasets are different by design. The K contrastive triples are used only to define the measurement function ϕ, and hence the parameter-space direction q, in the influence score. The steering data (e.g. Jigsaw) are the candidate points that q scores and weights. The transport map is then fitted on these weighted candidates. q is never used as a steering direction. The IF score of a candidate, −qᵀ(G+λI)⁻¹∇L(z_m), depends only on that candidate's own loss gradient. It does not require the candidates to be paired, so the lack of pairs in the steering data does not introduce noise into the scores. Pairing matters only for q. A single prompt-completion pair gives a noisy direction that also captures lexical features specific to that pair (Sec. 4.1), so we average over multiple contrastive triples to isolate the consistent concept component. We agree this distinction was not clear. In the revision we state it explicitly.
Q2.
Thank you for raising this. On Gemma-2-2B, we re-ran toxicity induction and all seven OneSeC concepts over 5 seeds, randomising data subsampling, training minibatch order, the EKFAC curvature corpus, and sampling and evaluation seeds. We then tested the IF-LinEAS − LinEAS gap with paired t-tests (full details in Appendix R, LinEAS vs. IF-LinEAS Significance). Toxicity induction: the gap is not seed noise. IF-LinEAS increases the RTP toxic rate (higher is better for induction) by +0.41 (p = 3×10⁻⁵, positive in all 5 seeds, about 9× the seed-to-seed SD), at the cost of about 3 points higher perplexity. OneSeC: the picture is mixed. IF-LinEAS is significantly better on some concepts (Baby, Book, Flower) and significantly worse on others (Football, Balloon), with no significant difference on the remaining two. As a result, the mean gap across concepts is not significant (+0.042, p = 0.31). Where the methods differ, however, the gap has the same sign in all five seeds. The seed-to-seed SD of the gap (0.025–0.049) is well below the spread of gaps between concepts (0.100). The close results therefore reflect real, concept-dependent differences between the methods, not an artifact of the random realisation. We agree that the OneSeC results do not support a universal advantage for IF-LinEAS, instead being concept dependent, and we have revised our claims accordingly.
Kind regards,
The Authors