acceptodds
Under review as a conference paper at ICLR 2027

Predictive Activation Steering via Solution Mapping and Stochastic Spectral Expansion

10 views

Abstract

Activation steering is a promising inference-time intervention that modulates large language models (LLMs) by perturbing their internal activations during generation. However, its practical application requires determining whether an effective steering intervention can be identified and quantifying trade-offs among suitable interventions. In this work, we propose an inexpensive framework for identifying optimized steering interventions subject to specified requirements. We model the steering coefficients as Gaussian random variables and approximate their effects on measured LLM behavior using a second-order Hermite polynomial expansion. Then, optimization is applied to find steering coefficients predicted to jointly satisfy specified targets. The resulting covariance matrix reveals structure within the steering coefficients, including coupling between coefficients and the degree to which they are jointly constrained. We propose a feasibility test that identifies unattainable specifications and systematically determines which requirements must be removed. We demonstrate the benefits of our approach through a case study and show that it performs competitively with existing methods for precise behavioral control of LLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
activation steering, large language model, stochastic spectral expansion, solution mapping, optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rtooOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper treats activation steering as a calibration problem. The authors extract eleven steering vectors from contrastive system prompts, one per behavior (complexity, formality, politeness, refusal, and so on). Each is injected at a single chosen layer. Each coefficient is normalized by the largest magnitude that keeps judged coherence close to the unsteered model, so the steering space becomes ξ∈[−1,1]11\xi \in [-1,1]^{11}. The coefficients get a Gaussian prior, ξi∼N(0,1/4)\xi_i \sim \mathcal{N}(0, 1/4). The mean judge score of each behavior over a fixed prompt set is then approximated by a second-order Hermite polynomial expansion in ξ\xi, fit by weighted ridge regression on about 200 sampled configurations.

With this surrogate, the user states target scores and tolerances, and a penalized least-squares objective is minimized to find the coefficients. A feasibility test borrowed from the combustion-kinetics literature flags targets that cannot be met. It removes them one at a time using a sensitivity-weighted residual. A Laplace-style covariance around the solution describes which coefficient directions are pinned down by the request and which are left free. It can also be propagated to behaviors that were not targeted.

Critical strengths

The main idea is a cross-domain transfer. Solution mapping, polynomial chaos expansions, and the feasibility test come from uncertainty quantification in chemical kinetics. I have not seen them applied to activation steering, and the fit is natural. Steering papers usually report parameter sweeps. This paper instead builds an explicit forward model and inverts it.

Critical weaknesses

The central claim is that the response surface plus optimization gives better steering than sampling. But the paper’s own “grid selection” baseline undercuts this. Grid selection uses the same calibration data and simply picks the measured configuration closest to the target. On HelpSteer2 (Table 5), grid selection or grid plus prompting is best in most cells. On Code-UltraFeedback (Table 1), grid plus prompting beats the method on Phi-4-mini [3,3,3,3,3] (57.1 vs 52.8 success rate) and is close elsewhere. If a nearest-neighbor lookup over the 249 measured points does as well, then the extra value of the polynomial surrogate for target-reaching is unclear. The paper should state this result directly and explain where the surrogate is actually needed. Plausible answers are feasibility diagnosis, the covariance, or targets far from any sampled point, but these are not tested head to head.

Questions for authors

Can the authors explain why grid selection performs as well as or better than the full method on most benchmark cells? In which concrete situations does the surrogate give an advantage over a nearest-neighbor lookup on the same calibration data? If possible, a small analysis restricted to targets far from all sampled configurations would be very informative.

2 comments
AuthorsOpenReview ↗

Response to Reviewer rtoo

Thank you for your detailed review. We appreciate your feedback and would like to address the weaknesses and questions you raised.

Weakness 1: why grid selection is competitive on the benchmark.

Grid search is effective on this benchmark because the benchmark (as defined by Zhang et al.) predominantly requires configurations that exist in the calibration data. The defined targets are close to typical scores, and the success of grid search simply reflects the fact that there is often a calibration configuration that directly satisfies the provided targets. Thus, this comparison does not consider situations involving targets that no calibration configuration is close to, where our method is most advantageous. Nonetheless, we include this comparison because there are very limited precise steering benchmarks in the literature. We believe it is important to validate our approach and demonstrate it performs competitively in comparison to existing methods and simple alternatives.

Weakness 1 and questions: where the surrogate is needed, tested head to head.

The surrogate is needed where no measured configuration satisfies the user request. This becomes typical as more targets are specified at once. Among random specifications accepted on Qwen3-8B:

Targets per specificationAccepted specificationsUnsatisfied by any measured configuration
21198 (7%)
312061 (51%)
411793 (79%)
5112103 (92%)

Following your suggestion, we ran the analysis restricted to targets far from all sampled configurations. We took specifications with four or five targets that no measured configuration satisfies, 45 targets in all, and generated and judged both our solution and the nearest measured configuration (grid selection) on the 132 held-out prompts (new Appendix G.1). We summarize our findings here:

MetricThis workGrid selection
Targets in acceptable range (of 45)28 (62%)19 (42%)
Targets in range where the range exceeds the forecast error (of 29, Table 7)25 (86%)15 (52%)
Median miss on those behaviors (points)0.390.57

Our surrogate reaches significantly more targets at approximately the same intervention magnitude (median norm 0.96 against 1.00) and coherence score (7.2 against 7.5). The surrogate also misses targets by a substantially smaller margin. Where the acceptable range exceeds the forecast error, the median miss is 0.39 points against 0.57, close to the held-out forecast error of the surfaces (0.37). Beyond this head-to-head test, we also checked target attainment on a larger set of random specifications. On 100 random specifications of one to four targets, measured scores at our solutions miss their targets by a median of 0.34 points, and 232 of 249 targets are within one point (Appendix H.4). As you mention, the surrogates also enable consistency diagnostics and covariance analysis. We validate the consistency diagnostic against 2,192 refused specifications, none of which is satisfied by any measured coherent configuration (Appendix H.4). Further, in Section 5.5, we use the covariance to show that, in the case study, helpfulness can vary 1.5 points while satisfying the original user request. This readily enables consideration of secondary objectives.

Changes. A new paragraph in Section 5.6 states directly that grid selection is competitive on the benchmark, why grid selection is competitive, and where the surrogate is needed. New Appendix G.1 gives the full analysis, including how often no measured configuration satisfies a random specification and the head-to-head test described above. Grid selection also appears in Table 12 on the 12 random specifications, meeting 17 of 30 targets against 19 for our approach.

AuthorsOpenReview ↗

Follow-up to Reviewer rtoo

Dear Reviewer rtoo,

As the discussion period comes to a close today, we wanted to briefly follow up on our response above. We have provided further clarification on where the surrogate is needed beyond grid selection, including a new head-to-head analysis on specifications that no measured configuration satisfies (Appendix G.1). The revised PDF has also been uploaded.

We hope this additional analysis helps answer your question, and we would be happy to address any remaining questions or comments. Thank you again for your thoughtful review.

Reviewer_csNmOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a predictive framework for multi-attribute activation steering by modeling the relationship between steering coefficients and behavioral outcomes. The authors construct a second-order Hermite polynomial response surface from sampled steering configurations, then use the fitted surrogate to optimize coefficients toward specified behavioral targets. The framework additionally includes a feasibility test for rejecting incompatible targets and a covariance-based analysis of coefficient coupling and alternative interventions. Experiments on four open-weight LLMs and 13 judged behaviors demonstrate predictive accuracy on held-out configurations, illustrate joint target optimization and behavioral trade-offs, and evaluate the method against grid selection, prompting, and previously reported Pre-Control results on Code-UltraFeedback and HelpSteer2.

Critical strengths

A relevant practical problem: Selecting and combining steering interventions while accounting for interactions, side effects, and competing behavioral objectives is a meaningful problem. The paper provides a systematic formulation that integrates behavioral prediction, coefficient optimization, and trade-off analysis within a single framework.

Critical weaknesses
  1. Limited methodological novelty and unsubstantiated efficiency advantages:
    • Generic formulation and limited scalability: The core approach is essentially quadratic response-surface regression followed by regularized inverse optimization and local sensitivity analysis. These are established numerical techniques, and the paper does not sufficiently demonstrate what makes their application to LLM steering particularly novel or advantageous. The method is evaluated with only 11 predefined steering directions, while the number of polynomial coefficients grows quadratically with the number of directions, requiring (K+1)(K+2)/2(K+1)(K+2)/2 coefficients per predicted behavior (78 for K=11K=11, but 5,151 for K=100K=100). Adding or modifying steering directions also requires calibrating previously unmodeled interactions. The framework's applicability to larger or evolving steering-vector libraries is therefore unclear.
    • Insufficiently competitive baselines and cost analysis: Each model requires approximately 70,000 generations and 500,000 judge calls for calibration. While subsequent optimization is inexpensive, the paper does not establish when this substantial upfront cost is preferable to target-specific optimization. Its grid-selection baseline is non-adaptive and does not represent stronger alternatives such as Bayesian optimization or other adaptive black-box search methods [1]. Comparisons with established multi-attribute steering methods [2, 3] or directly optimized steering vectors [4] are also missing. Moreover, the method does not consistently outperform even grid selection, particularly on HelpSteer2. The comparison with Pre-Control relies on previously reported results, while the authors acknowledge being unable to reproduce its Base and Prompting baselines, limiting the reliability of the claimed relative performance.
  2. Unconvincing behavioral validation and questionable feasibility claims:
    • Limited validity of the evaluation targets: The main predictive results concern mean scores assigned by a single LLM judge across a fixed prompt distribution. Low prediction error on these aggregated scores does not necessarily establish reliable control over individual responses, agreement with human preferences, or meaningful distinctions between requested attribute intensities (e.g., politeness scores of 6 versus 8). The paper does not sufficiently validate the scoring scale or its robustness across independent judges. Moreover, predicting the mean behavior under different coefficient combinations is not by itself surprising; comparisons with simpler constant or linear response models would help establish the necessity of the proposed second-order formulation.
    • The feasibility test does not establish infeasibility: Equations (5)–(7) infer infeasibility from the residuals of a regularized optimization problem. However, a target may be achievable even when the penalized optimum does not satisfy it. For a concrete counterexample, let K=9K=9, μ^(ξ)=5+12∑i=19ξi\hat{\mu}(\xi)=5+\frac{1}{2}\sum_{i=1}^{9}\xi_i, ξ∈[−1,1]9\xi\in[-1,1]^9, τ=9.5\tau=9.5, and σ=0.5\sigma=0.5. The target is exactly achievable at ξ=(1,…,1)\xi=(1,\ldots,1), whereas the global minimizer of Equation (5) is ξi∗=9/13\xi_i^*=9/13, yielding ∣F∣=18/13>1|F|=18/13>1. Equation (6) would therefore reject a feasible target even with an exact surrogate and a globally optimal solution. This directly undermines the interpretation of the proposed procedure as a feasibility test. The covariance analysis likewise relies on a local approximation of the penalized objective, and its practical benefits beyond conventional Jacobian or null-space analysis are not sufficiently established.

[1] Balandat et al., BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. NeurIPS 2020. https://proceedings.neurips.cc/paper/2020/hash/f5b1b89d98b7286673128a5fb112cb9a-Abstract.html

[2] Nguyen et al., Multi-Attribute Steering of Language Models via Targeted Intervention. ACL 2025. https://aclanthology.org/2025.acl-long.1007/

[3] Oozeer et al., Beyond Linear Steering: Unified Multi-Attribute Control for Language Models. Findings of EMNLP 2025. https://aclanthology.org/2025.findings-emnlp.1278/

[4] Cao et al., Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization. NeurIPS 2024. https://proceedings.neurips.cc/paper_files/paper/2024/hash/58cbe393b4254da8966780a40d023c0b-Abstract-Conference.html

Questions for authors
  1. Validity of the feasibility test: How does the proposed procedure distinguish genuinely infeasible targets from targets that are achievable but disfavored by the intervention-magnitude penalty? The counterexample above demonstrates that the current criterion can reject an exactly achievable target even under an exact response model. Could the authors clarify the intended guarantee and explain whether feasibility should instead be evaluated using an explicit constrained optimization problem?
  2. Efficiency and competitive optimization baselines: What is the end-to-end computational advantage over adaptive, target-specific optimization methods, such as Bayesian optimization or other gradient-free search procedures, under matched generation and judging budgets? In particular, how many distinct target specifications are required before the proposed offline calibration becomes cost-effective? A comparison with directly optimizing steering vectors for the requested attributes would also help establish the value of combining a fixed set of pretrained directions.
  3. Behavioral evaluation validity: To what extent do the predicted judge scores correspond to meaningful and consistent differences in user-perceived behavior? Could the authors validate the target scores and steering outcomes using independent judge models or human preference evaluations, and report per-prompt reliability rather than only mean-score prediction errors? Additionally, how much predictive improvement does the quadratic response surface provide over constant and first-order baselines?
  4. Scalability and the utility of covariance analysis: How would the framework accommodate substantially larger steering-vector libraries or newly introduced directions without extensive recalibration? More importantly, what actionable information does the proposed covariance analysis provide beyond the gradients and null space of the fitted response surfaces? A direct evaluation of secondary-objective optimization with and without the covariance-based procedure would help clarify its incremental contribution.
9 comments
AuthorsOpenReview ↗

Response to Reviewer csNm (1/5)

Thank you for your diligent review. We appreciate your perspective and would like to respond to the weaknesses and questions you raised.

Weakness 1: generic formulation and limited scalability.

We regard the simplicity, yet effectiveness, of our approach as a strength. To the best of our knowledge, this is the first paper to view multi-behavior steering as a calibrated inverse problem. Our approach is well-suited to applications where a central large language model is being used for a variety of applications, tasks, customers, etc. because it separates calibration from deployment. In many situations, rapid tunability of the model to meet the task at hand is an important asset that other sequential optimization approaches do not provide.

Scalability has two dimensions: judged behaviors and steering directions. An additional judged behavior adds one surface of 78 coefficients. This can readily be fit by re-judging the existing generations on the new behavior and requires no additional generations or modifications to the existing surfaces. Adding a steering direction requires K+2 new coefficients in every surface (quadratic growth) and in general, new generations. We address this quadratic growth with a few points. First, as demonstrated with helpfulness and coherence, steering a small set of behaviors can indirectly influence untargeted behaviors. Therefore, it is plausible that many desired behaviors can be altered from a much smaller set of steering vectors. Second, we do not think a library of 100 steering directions is the regime that matters in practice. On Qwen3-8B, 40% of specifications with eight targets are accepted in full and just 9% of those with twelve (Appendix H.2), so requests with many simultaneous targets are rejected nearly every time. Although beyond the scope of the current study, we foresee that a large library could be handled with a sparse version of the current model where interaction terms are only considered between select directions. Third, your comment inspired us to examine the possibility of adaptively adding steering directions (new Appendix H.5). On Qwen3-8B, we fit response surfaces on 8 of the 11 steering directions. Then, each excluded direction is added back to the model one at a time. Each addition estimates the K+2 new coefficients per behavior while leaving every existing coefficient frozen. We repeat this for each of the 165 combinations of excluded directions. After the three directions are added back, the average held-out error is 0.48 against 0.37 for the model that jointly fits all directions. This test required no new generations since the excluded directions were already active in the existing design configurations. In practice, however, introducing and calibrating a new direction will also require a limited amount of data from configurations which activate that new direction. For a library of our size, 32 to 64 such configurations suffice. New directions therefore require just a fraction of additional calibration data, and adaptively adding them to an existing steering library has only a modest effect on predictive accuracy.

Changes. New Appendix H.5 reports the experiment on adding steering directions without a full refit. Section 5.3 now summarizes acceptance as the number of targets grows, with full results in Appendix H.2.

AuthorsOpenReview ↗

Response to Reviewer csNm (2/5)

Weakness 1: baselines and cost.

We consider cost first. A per-specification search that examines 20 steering configurations and averages over 50 prompts costs 1,000 generations and 1,000k judge calls per new specification with k targets. Our up-front calibration involves 31,236 generations and 406,068 judge calls. Therefore, taking k to be 2.5 results in the break-even point occurring after about 31 specifications in generations and about 162 in judge calls. Every specification thereafter costs one optimization and no generations. In many applications a single LLM could be requested to consider thousands or millions of customizations.

We next compare our method to adaptive search methods on the same objective. We consider two variants of Bayesian optimization (GP-EI and the BoTorch approach), random search, CMA-ES, and Nelder-Mead on 12 random specifications of Qwen3-8B (30 targets total). Comparison is on the measured mean of 132 held-out prompts (new Appendix G.2, Table 12). Misses are in judge points over the targets not met.

methodtargets met (of 30)median missmean misscoherence‖ξ‖₂
Unsteered91.691.799.00.00
Prompting91.091.569.40.00
Grid selection170.831.057.90.85
Random search (Sobol), 20 evaluations181.081.216.61.60
Nelder-Mead, 20 evaluations151.311.628.50.50
CMA-ES, 20 evaluations141.041.126.41.43
Bayesian optimization, GP-EI, 20 evaluations130.861.066.31.78
Bayesian optimization, BoTorch, 20 evaluations130.901.156.21.51
First-order surface151.331.667.80.75
This work190.780.937.70.70

The Bayesian optimization approaches meet 13 of 30 targets against 19 for our method. Notably, the solutions they find have median steering coefficient vector magnitudes of 1.5 and 1.8 against 0.7 from our approach and with lower coherence (6.2 and 6.3 against 7.7). Random search, CMA-ES, and Nelder-Mead meet 18, 14, and 15 of 30 targets, none of which surpasses our approach, and when our solver misses, it misses by the smallest margin of any method. While not definitive, these results provide meaningful empirical evidence in support of our approach.

On provided references [2] to [4]: MAT-Steer [2] now appears in Tables 1 and 13 as published rows from the Pre-Control paper along with two other established multi-attribute steering methods (ITI and RE-Control). While the Pre-Control results could not be reproduced after a substantial effort, our baselines are generally stronger than the published ones and the quality of our method can still be assessed. K-Steering [3] and BiPO [4] address a different problem from ours and are therefore not direct comparisons. Both focus on selecting which attributes to promote or suppress, but neither determines the intervention magnitude needed to achieve a specified target level or assesses whether multiple target levels are jointly attainable. K-Steering recomputes a prompt-specific intervention from classifier gradients, rather than producing a fixed intervention whose effects can be calibrated and forecast. It also selects step sizes through a separate coherence search for each attribute combination, which the authors report is "several orders of magnitude more costly than baseline methods" in its multi-step variant. BiPO learns an improved direction but leaves intervention strength to be chosen manually. Thus, evaluating either method against numerical targets would depend on an additional strength-selection procedure rather than the methods themselves. We instead provide direct comparisons in Table 12 with methods that explicitly optimize interventions for target levels. Finally, our framework is independent of how intervention directions are obtained. Any fixed set of directions, including those learned by BiPO, can be calibrated using our method to enable forecasting, consistency checking, and trade-off analysis.

Changes. New Appendix G.2 compares our solver with Bayesian optimization and the other per-specification search methods (Table 12) and gives the cost analysis and break-even analysis. Tables 1 and 13 now include the published MAT-Steer, ITI, and RE-Control rows, and Section 2 now references BiPO.

AuthorsOpenReview ↗

Response to Reviewer csNm (3/5)

Weakness 2: validity of the evaluation targets.

We first address reliable control over individual responses. By averaging over 100 prompts, we reduce the standard deviation of judge scores at a fixed steering intervention by an order of magnitude, from around 2 to 0.2. We do so primarily because an individual response has no ground-truth score and even human raters agree only moderately on scores (quadratic weighted kappa 0.43 to 0.79 on HelpSteer2 attributes, Wang et al. 2024). Modeling the mean response is the viable path to quantitative, yet statistical, predictions of steering effects. The per-prompt results of the case study are added to Appendix H.3 (Table 14): 58 to 74% of the 132 held-out prompts move toward their target and just 4 to 20% away. We further report the per-prompt standard deviations for each behavior, and six held-out prompts are shown with their unsteered and steered responses and associated judge scores.

We next validate robustness across independent judges. We use the HelpSteer2 responses that carry three human ratings and compute the Spearman correlation of our judge's score with the mean of two human scores. We compare this against the Spearman correlation of the third human rater with the same mean of the first two (Appendix F.2, Table 11):

behaviorresponsesjudge vs. two ratersthird rater vs. two ratersratio
coherence2450.380.35108%
complexity3230.580.6293%
verbosity3650.680.7591%
helpfulness4460.410.6762%

The helpfulness gap appears to be attributable to construct mismatch instead of judge error. Human helpfulness in HelpSteer2 is nearly identical to correctness (Spearman 0.93), which our rubric does not consider. We also compare to a second independent LLM judge, Mistral-Small-24B-Instruct, which rescored 150 calibration configurations per model with the same rubrics (Table 10). Its configuration means correlate with our judge (Qwen2.5-14B-Instruct) at r = 0.90 to 0.92 pooled over behaviors and above 0.9 for 10 to 11 of the 13 behaviors on every model.

Finally, we address the meaningfulness of distinctions in judge scores. We only deem two target scores distinguishable when the ratio between the acceptable range used in our experiments (Appendix C.1) and the held-out forecast error is greater than 1. Table 7 reports this ratio for every behavior and model and shows, for example, that politeness requests of 6 and 8 are each distinguishable within their acceptable range on every model, whereas sycophancy requests are not because its forecast error exceeds its acceptable range. In comparison to simpler models, held-out error is 1.03 points for a constant surface, 0.67 for a linear one, 0.47 for a quadratic without interactions, and 0.37 for the full quadratic we use. Further, cubic surfaces do not reduce the held-out error (Appendix B.6, Table 4). These numbers are for Qwen3-8B, but the ordering holds across all four LLMs considered.

Changes. New Appendix F.2 reports the agreement of our judge with human ratings and with a second LLM judge (Tables 10 and 11). New Appendix H.3 reports per-prompt results for the case study with example responses. New Appendix B.6 compares surfaces of different order. Section 3.1 now explains why we model the mean score over prompts.

Reviewer_ynLaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a framework for predictive activation steering that aims to determine appropriate steering strengths for achieving desired LLM behaviors. Authors model relationship between steering coefficients and behavior scores using a surrogate model based on second order Hermite polynomial expansion. The model is then used to optimize steering coefficients for multiple targets without LLM evaluations during optimization. The authors evaluate their approach on several LLMs, demonstrating reasonable accuracy in predicting average behavioral scores.

Critical strengths
  1. The paper addresses a practical and important challenge in activation steering, determining appropriate steering strengths to achieve desired behaviors. Existing approaches often rely on trial, making the proposed method a meaningful direction to explore.
  2. The proposed framework builds upon established techniques from solution mapping and polynomial expansion. Authors try to bring these theoretically grounded methods into the context of activation steering. This provides a principled way to model steering effects and analyze interactions between different steering directions.
Critical weaknesses
  1. While the surrogate formulation is interesting, I feel that the authors somewhat oversimplify the problem. In particular, the framework models steering effects as a deterministic mapping from steering coefficients to average behavioral scores, potentially overlooking variations across prompts. Moreover, the authors assume that a second order polynomial is sufficient to capture the nonlinear interactions between multiple steering directions, without considering higher order expansions. Although the empirical prediction results are encouraging, stronger justification for these modeling assumptions would be helpful.
  2. Although the authors describe how the steering vectors are constructed, they largely treat them as fixed inputs without investigating how their quality or construction methods affect the proposed framework. This is particularly important for the feasibility analysis, since a target deemed infeasible under one set of steering vectors might be achievable with another. Furthermore, the feasibility test is based on the residuals of a regularized optimization problem, where failing to satisfy a target does not necessarily imply that no feasible solution exists. The experiments mainly demonstrate the effectiveness of the framework through a limited number of case studies, without systematically evaluating the reliability of feasibility detection or the success rate of multiple target steering on held out prompts. I believe more comprehensive experiments are needed to support these claims.
Questions for authors

I have a question that why authors only consider a second-order approximation. While a second order Hermite expansion can capture pairwise interactions between steering directions, it cannot model higher order interactions involving three or more directions. Given that the experiments involve 11 steering vectors, which may be applied simultaneously, it is unclear whether pairwise interactions alone are sufficient to characterize their combined effects. Although the authors report encouraging prediction accuracy, I would like to see a more detailed justification for this modeling choice. Additional experiments examining the importance of higher order interactions would be helpful, particularly when multiple steering directions are activated simultaneously.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ynLa (1/2)

Thank you for your constructive review. We appreciate your feedback and would like to address the weaknesses and questions you raised.

Weakness 1: variation across individual prompts.

We intentionally model the average behavior across prompts for a few reasons. First, a single 0-to-10 score is noisy. The per-prompt standard deviation at a fixed steering configuration is about 2 points, whereas the mean over the 100 fitting prompts has a standard error of about 0.2 points. Further, human raters only moderately agree on ratings as shown in HelpSteer2 (quadratic weighted kappa 0.43 to 0.79 across HelpSteer2 attributes, Wang et al. 2024). We also know that there is no ground truth score for a given generated response to a prompt. Anecdotally, two rational individuals could greatly disagree on how funny a response is. For these reasons, we believe modeling the mean response is the most viable path to enabling quantitative steering. The per-prompt spread is not discarded. It is used to weight the fit (Eq. 4). Our approach is statistical by construction and does not provide assurances at the individual prompt level. The per-prompt scores for the case study on the 132 held-out prompts are:

behaviortargetunsteered mean (s.d.)at ξ* mean (s.d.)moved towardunchangedmoved away
figurative language3.171.27 (1.9)3.05 (2.9)58%34%8%
politeness5.164.08 (1.7)5.27 (2.5)64%17%20%
verbosity4.526.06 (2.7)4.30 (3.3)74%22%4%

At the solution point, the per-prompt standard deviation is 2.5 to 3.3 points, and 58 to 74% of the 132 prompts moved toward their target relative to the unsteered response, whereas just 4 to 20% moved away.

Changes. Section 3.1 explains why we model the mean response. New Appendix H.3 reports the per-prompt results for the case study (Table 14) and six held-out prompts with their unsteered response, steered response, and judge scores.

Weakness 1 and Question: why second order.

The second-order model is the appropriate level of complexity. Its held-out error is slightly lower than that of the cubic surfaces with far fewer coefficients, and it is well below the simpler models. New Appendix B.6 compares the held-out prediction error of a constant, linear, additive (no interaction terms), full quadratic, and two cubic response surfaces, each fit on the same data (Table 4):

surfacecoefficients per behaviorQwen3-8BLlama 3 (8B)Phi-3.5-miniMinistral 8B
constant11.030.790.930.88
linear120.670.570.630.60
additive quadratic230.470.380.430.39
full quadratic (ours)780.370.320.380.32
cubic, diagonal890.380.330.390.33
full cubic3640.420.350.420.36

Across the models, the interaction terms account for 22 to 31% of the improvement over a linear model. Both cubic surfaces, which include third-order terms, perform worse than the quadratic on every LLM. Further, held-out configurations with all 11 directions active are predicted more accurately than those with one to four active (Table 5). Together, these results indicate that pairwise interactions are sufficient to characterize the combined effects of the directions and that missing higher-order terms are not what limits accuracy. Instead, prediction error grows with the magnitude of the intervention (Table 6, Figure 9), which is why the prior distribution is designed to penalize large interventions.

Changes. We added Appendix B.6, comparing surfaces of different order and the held-out error by number of active directions and by intervention magnitude (Tables 4 to 6), and noted the reduced predictive accuracy near the limits of the steering range in Section 6.

AuthorsOpenReview ↗

Response to Reviewer ynLa (2/2)

Weakness 2: the vectors as fixed inputs.

This is an important detail and we would like to highlight two points in response. First, new experiments show that steering directions can be added to an already calibrated model without a full refit (Appendix H.5). We removed three directions from the calibrated Qwen3-8B and added them back one at a time with the existing coefficients frozen. The average held-out error across the 165 combinations of three removed directions is 0.48 as compared to 0.37 for the model that jointly fits all directions. This on-the-fly addition of steering directions is an important asset of our framework as new directions require just a small fraction of additional calibration data and adaptively adding them to an existing steering library has only a modest effect on predictive accuracy. Therefore, although the steering vectors remain fixed, this enables the coverage of the steering vector library to be flexible. Second, our framework is indifferent to how the steering vector is formed or where it is applied in the LLM. Therefore, additional flexibility is available in the technique used to extract the steering vectors and how they are applied.

Changes. Section 3.3 and Section 6 now state that recommendations are relative to the calibrated directions and layers, and we added Appendix H.5 regarding adding directions without recalibration.

Weakness 2: feasible solution exists.

We agree that the residual of this regularized problem does not establish infeasibility and apologize for the imprecise description. Our goal with the test is to prevent the recommendation of an intervention whose predicted effect the calibration cannot support. Strict feasibility is not what is being tested. Throughout the paper, we now call this a consistency test, and a target that is rejected is "not recommended" rather than "infeasible" (Section 3.3). A feasible target may be rejected if satisfying it requires a larger than acceptable intervention or a solution beyond the calibrated region. Two results support setting up the consistency test this way. First, the prior precision sets the trade-off (Appendix C.3, Table 8; Qwen3-8B):

prior precision124 (default)816
specifications accepted in full54.7%54.4%52.4%48.5%41.0%
median intervention norm1.341.261.141.000.81

The specifications that stop being accepted are the ones whose solutions require large interventions. These large interventions are where predictions become less accurate (held-out error grows from 0.2 points at norms below 0.8 to 0.8 points at norms above 1.3) and coherence scores decrease (Table 6 and Figure 9). Thus, the interventions the prior distribution disfavors are those whose predicted effect is least reliable. Next, Appendix H.4 checks every rejection against the data. Across the four models, 2,192 specifications had at least one rejected target, and none of them is satisfied by any measured calibration configuration within the calibrated region. By comparison, 35% of the 2,968 accepted specifications are satisfied by such a configuration. This provides strong empirical evidence of the validity of these rejections.

Changes. The test is now called a consistency test throughout, and Section 3.3 explains what it does more clearly. New Appendix C.3 shows, for an idealized linear response surface, that a single target is refused only when meeting it requires an intervention at least four prior standard deviations from the unsteered model. New Table 8 reports the effect of varying the prior precision. New Appendix H.4 checks every refused specification against the measured calibration data.

Weakness 2: limited number of case studies.

We added systematic experiments (Appendices H.2 and H.4). For each model, we solve 120 random specifications at each number of targets, from one target up to the full set of targetable behaviors. On Qwen3-8B, 89% are accepted in full (no target rejected) at two targets, 71% at five, and 40% at eight. The other three models follow the same trend (88 to 89%, 68 to 78%, and 33 to 51%). Every rejection was checked against the calibration data as described above. To test whether accepted targets are reached, we generated and judged responses for 100 random specifications of one to four targets on the 132 held-out prompts. Of the 249 accepted targets, 161 lie inside their acceptable ranges and 232 are within one point. The median miss is 0.34 points, which nearly matches the held-out forecast error of the surfaces (0.37). Therefore, these misses are largely near misses and attributable to the resolution of the prediction.

Changes. New Appendix H.2 reports acceptance rates for random multi-target specifications on all four models. New Appendix H.4 covers the reliability of the consistency test and the success of accepted targets on held-out prompts. Section 5.3 briefly summarizes and references these new sections.

AuthorsOpenReview ↗

Follow-up to Reviewer ynLa

Dear Reviewer ynLa,

As the discussion period comes to a close today, we wanted to briefly follow up on our response above. We have provided additional analyses and clarifications addressing the points you raised, including the per-prompt analysis, model-order comparison, renamed and validated consistency test, and systematic evaluation on random specifications and held-out prompts. The revised PDF has also been uploaded.

We hope these additions help resolve your concerns, and we would be happy to address any remaining questions or comments. Thank you again for your thoughtful review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.