acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Multi-Value Control in LLMs via Causal Activation Steering

Abstract

Large language models (LLMs) are increasingly deployed in settings where responses must reflect multiple, potentially interacting social norms and human values. Activation steering offers a lightweight alternative to training-based alignment by modifying internal activations at inference time. However, prior human-value steering methods have largely considered values in isolation, while direct composition of multiple directions relies on fixed intervention strengths that cannot respond to the model's evolving internal state. Motivated by this key observation, we introduce AIMES, a framework for adaptive multi-value activation steering. AIMES constructs layer-specific bipolar directions for moral-foundation values and uses intermediate-layer vocabulary readouts as online observers. An observer-guided controller then adapts the strength of each requested value intervention at every decoding step based on its current observed state, without training a separate value-state estimator. Across multiple instruction-tuned model families, value combinations, and intervention depths, we find that multi-value controllability varies across both value combinations and intervention locations. Compared with fixed joint steering and prompt-based steering, AIMES shows depth-dependent advantages that are broadly supported across two independent evaluators, with some variation in the precise depth at which specific control effects emerge. These advantages come with smaller realized activation-space interventions than fixed-joint steering and comparable response quality. Overall, our results suggest that online observer feedback can provide lightweight, state-aware adaptation for single-pass multi-value steering.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Activation Steering, Multi-Value Control, Human Values, Closed-Loop Steering, Inference-Time Control, Controllable Generation, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eSs3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AIMES, a framework for adaptive multi-value activation steering in LLMs. The method constructs layer-specific bipolar directions for Moral Foundations Theory values, uses intermediate-layer vocabulary readouts, primarily J-Lens, as online value-state observers, and adjusts value-specific steering strengths at each decoding step. Experiments across five models show that the effectiveness of multi-value steering depends strongly on intervention depth and value combination, and AIMES can outperform fixed multi-value steering and prompt steering in several settings, particularly for the competing objective of increasing Care and Fairness while suppressing Sanctity.

Critical strengths

The paper addresses an interesting problem that is less explored than single-attribute activation steering: adaptive joint control of multiple interacting human values. The use of J-Lens as a training-free online observer is a sensible and potentially useful design choice, and the paper evaluates multiple model families, intervention depths, and two independent judges. I also appreciate that the authors do not overstate universality and explicitly report substantial model-, depth-, and evaluator-dependence.

Critical weaknesses
  1. The central observer assumption is insufficiently validated. AIMES treats the average J-Lens readout over a small manually defined token set as the model’s current value state. However, higher accessibility of value-related words in vocabulary space does not necessarily imply stronger behavioral expression of the corresponding value. Since this observer directly drives the controller, stronger validation of its construct validity is needed.

  2. The contribution of adaptivity itself is not well isolated. The controller uses a very simple state-dependent scaling rule, but the paper does not compare it against alternative adaptive controllers, threshold-based gating, other monotonic mappings, or perturbation-matched controls. It is therefore unclear whether the gains come from the observer, token-level adaptation, reduced intervention magnitude, or the particular controller design.

  3. The baseline set does not sufficiently cover the closest adaptive or feedback-based steering methods. The main comparisons are against fixed multi-value steering and prompt steering. Given the existence of prior adaptive and feedback-controlled activation steering methods, direct experimental comparison with these approaches is important for establishing the incremental contribution of AIMES.

Questions for authors
  1. How well do the J-Lens observer scores correlate with independently measured behavioral value expression?
  2. How sensitive are the results to the manually selected value-token sets?
  3. Does AIMES still outperform fixed steering when the two methods are matched by realized perturbation magnitude?
  4. Why are existing adaptive or feedback-based activation steering methods not included as direct baselines?
2 comments
AuthorsOpenReview ↗

Gentile reminder to reviewer eSs3

Dear Reviewer eSs3,

Thank you again for your constructive feedback. We have addressed your comments in our responses and uploaded a revised manuscript incorporating additional experiments, including J-Lens observer validation and comparisons with Mean-Strength Fixed Steering. We would be grateful if you could review our responses and the updated manuscript and reconsider your assessments and scores in light of the new evidence.

Thank you for your time and consideration.

Dear Reviewer eSs3, As we approach the end of the discussion period, we would like to sincerely thank you again for your constructive feedback and thoughtful evaluation. We believe we have addressed most of your concerns through our detailed responses, additional experiments, and revised manuscript. We would be very grateful if you could take one final look at these updates and consider revisiting your assessment and score in light of the new evidence. If you have any other question we would be happy to clarify. Thank you again for your time and consideration.

Reviewer_xzZCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AIMES, an adaptive framework for steering multiple human values in LLMs through inference-time activation interventions. AIMES constructs layer-specific bipolar steering directions for five Moral Foundations Theory dimensions using contrastive examples. During generation, it uses intermediate-layer vocabulary readouts as online observers to estimate the current value state and dynamically adjusts the strength of each steering direction at every decoding step without training a separate value-state estimator. The authors evaluate AIMES across five instruction-tuned models, three multi-value objectives, and different intervention depths, comparing it against fixed multi-value activation steering and prompt-based steering. The results show that AIMES achieves depth-dependent improvements in multi-value control, with more consistent advantages over prompt steering and varying improvements over fixed steering. The method also requires smaller activation-space interventions while maintaining comparable response quality.

Critical strengths

1. Interesting and well-motivated research question.

The paper addresses an interesting and important problem of controlling multiple human values simultaneously through activation steering. In particular, the authors identify the limitations of fixed-strength steering when multiple value directions interact and propose an adaptive approach that accounts for the model's evolving internal state during generation. This provides a meaningful motivation for investigating closed-loop multi-value steering.

2. Systematic selection of human values and steering objectives.

The paper adopts the five dimensions of Moral Foundations Theory (MFT), providing a clear theoretical basis for defining human values. Furthermore, the authors analyze the geometric relationships among value directions and select three multi-value objectives representing aligned, weakly coupled, and competing interactions. This provides a structured experimental setting for investigating different types of value interactions.

3. Simple and intuitive observer-guided steering framework.

The proposed AIMES framework is conceptually simple and easy to understand. By using intermediate-layer vocabulary readouts to estimate value states, the method avoids training separate value-specific classifiers or probes. The approach of dynamically adjusting steering coefficients based on the observed value scores provides an intuitive and lightweight mechanism for adaptive multi-value control.

4. Extensive evaluation across models, intervention depths, and independent judges.

The authors evaluate AIMES across five instruction-tuned models from three model families and ten intervention depths, providing a useful analysis of how steering effectiveness varies across architectures and network depth. The use of two independent LLM judges also helps characterize evaluator sensitivity and provides additional evidence for the robustness of some observed patterns. Furthermore, the inclusion of prompt-level consistency, balanced multi-value control, and within-response steering variation offers multiple perspectives on the proposed method.

Critical weaknesses

Major Weaknesses

1. Insufficient isolation of the benefits of adaptive feedback.

The paper does not sufficiently distinguish the benefits of adaptive feedback from those of simply reducing steering strength. Although AIMES and Fixed Multi-Value Steering share the same base magnitude, AIMES applies sigmoid-based coefficients that reduce the strength of individual interventions. Appendix C.4 (Figure 9) further shows that AIMES consistently produces smaller activation-space perturbations than Fixed Multi-Value Steering. Therefore, it remains unclear whether the observed improvements arise from state-dependent adaptation or from avoiding excessive intervention strength. A comparison with a strength-matched fixed-steering baseline would help isolate the contribution of adaptive feedback. Additionally, Figure 5 demonstrates that the steering coefficients vary during generation but does not establish whether these variations contribute to improved control performance.

2. Limited validation of the intermediate-layer value observer.

AIMES relies on vocabulary readout scores derived from predefined MFT-related token sets to estimate the current expression of human values. However, it is unclear whether these scores reliably reflect the values expressed in the model's actual behavior. Although Appendix C.3 compares J-Lens and Logit Lens in terms of directional responsiveness and downstream performance, this does not directly establish the validity of the observer as a measure of value expression. An additional analysis examining the relationship between observer scores and independently evaluated behavioral value scores would strengthen the proposed framework. Furthermore, an ablation comparing the sigmoid mapping with alternative scaling or gating functions would help justify the controller design.

3. Limited generalization across multi-value objectives and substantial objective-dependent performance.

The evaluation considers only three predefined value combinations involving two or three values. Although these objectives are motivated by different geometric relationships, they provide limited evidence of how AIMES generalizes to broader multi-value settings. More importantly, Appendix D.4 (Table 8) reveals substantial differences across objectives. Under Claude-Opus-4.8, AIMES achieves higher GeoGain than Prompt Steering in 0/50 settings for Care/Fairness, 21/50 for Loyalty/Authority, and 50/50 for Care/Fairness/Sanctity. These results suggest that the effectiveness of AIMES depends strongly on the selected value combination. Further analysis explaining these differences, along with additional value combinations, would help clarify the conditions under which adaptive multi-value steering is beneficial.

Minor Weaknesses

1. Insufficient details on contrastive vignette generation and validation.

Appendix C.2 describes the construction of matched positive-negative vignette pairs, but several implementation details remain unclear. In particular, the language model used for synthetic generation, the exact generation prompts, and the pair-level validation criteria are not clearly specified. Figure 6 also includes a human verification stage, but the number of annotators and the verification procedure are not described in sufficient detail. Although the appendix reports a semantic sanity check, this primarily concerns the foundation-versus-social-norm corpus rather than the matched positive-negative pairs used for steering. Providing these details would improve reproducibility.

2. Limited layer-wise analysis of value-direction geometry.

Appendix C.4 presents pairwise cosine similarities at only three representative layers for each model. Given that the paper emphasizes depth-dependent value interactions, a more comprehensive analysis covering the tested intervention depths would be informative. Additionally, the three multi-value objectives are primarily motivated by the early-layer geometry in Figure 2, even though Appendix C.4 shows that these geometric relationships can change substantially across layers. The authors should clarify why early-layer geometry is used to define the objectives and avoid implying that these relationships remain consistent across the network.

3. Limited reporting of statistical uncertainty in the main results.

Figure 3 reports mean interaction contrasts across intervention depths without confidence intervals, making it difficult to assess the reliability of the observed improvements, particularly when the differences are small. Although confidence intervals for prompt-level consistency are provided in Appendix D.1, uncertainty estimates for the mean interaction contrasts would be useful. Reporting bootstrap confidence intervals in Figure 3 or an accompanying table would help readers assess the robustness of the depth-dependent findings.

4. Limited justification for the sigmoid-based controller.

The proposed controller uses a sigmoid function to map normalized vocabulary readout scores to steering coefficients. However, the choice of sigmoid is not sufficiently justified beyond providing bounded values. It remains unclear whether alternative mappings, such as linear scaling, clipping, or other gating functions, would yield similar performance. Moreover, the resulting coefficients are treated as estimates of value expression without explicit calibration against behavioral value scores. An ablation study comparing different mapping functions would help establish whether the sigmoid transformation is important for the effectiveness of AIMES.

Questions for authors
  1. How much of the performance improvement can be attributed to adaptive feedback itself? Since AIMES produces smaller activation-space perturbations than Fixed Multi-Value Steering, could the authors compare it against a strength-matched fixed baseline? Additionally, would alternative coefficient mappings, such as linear scaling or clipping instead of sigmoid, lead to similar results?

  2. How well do the intermediate-layer readout scores reflect actual behavioral value expression? The observer relies on normalized logits associated with predefined MFT-related tokens. Have the authors examined the relationship between these observer scores and independently evaluated behavioral value scores?

  3. Is there a relationship between value-direction geometry and the effectiveness of steering at different layers? Appendix C.4 shows that pairwise cosine similarities change across network depth. Do these changes occur gradually or abruptly, and do different value pairs exhibit distinct depth-dependent patterns? More importantly, could these geometric properties help identify effective intervention layers before generation?

  4. What explains the substantial differences in performance across multi-value objectives? In Appendix D.4 (Table 8), AIMES outperforms Prompt Steering in 0/50 settings for Care/Fairness, 21/50 for Loyalty/Authority, and 50/50 for Care/Fairness/Sanctity under Claude-Opus-4.8. What might explain this difference, particularly the stronger performance on the competing three-value objective compared to the simpler two-value objectives?

  5. Could the authors clarify the contrastive vignette generation and validation procedure? In particular, which language model and generation prompts were used to construct the positive-negative pairs? Additionally, how was the human verification stage shown in Figure 6 conducted, and what criteria were used to validate the generated pairs?

  6. Why is response quality evaluated using only GPT-5.6-Sol? Given that the paper uses two independent judges for behavioral value evaluation and observes some evaluator sensitivity, would the response-quality conclusions remain consistent under Claude-Opus-4.8?

1 comment

Dear Reviewer xzZC, As we approach the end of the discussion period, we would like to sincerely thank you again for your constructive feedback and thoughtful evaluation. We believe we have addressed most of your concerns through our detailed responses, additional experiments, and revised manuscript. We would be very grateful if you could take one final look at these updates and consider revisiting your assessment and score in light of the new evidence. If you have any other question we would be happy to clarify. Thank you again for your time and consideration.

Reviewer_zYR9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-value activation steering in LLMs and proposes AIMES, a framework that addresses the limitations of fixed steering coefficients in adapting to changes in the model's internal state during generation. AIMES uses J-Lens to extract value-related signals from intermediate layers and adjusts the strength of each steering direction at every decoding step, without training a separate value-state estimator.

Critical strengths

[S1] The problem is important and well-motivated.

In multi-value steering, different value directions may interfere with each other, and the model's internal state changes during generation. Adapting steering strengths to the current state is therefore a reasonable and interesting direction to explore.

[S2] The method is simple and has some novelty.

The paper uses J-Lens as an online observer of value-related states and adjusts steering coefficients based on its outputs. This is a relatively lightweight approach that does not require training an additional value-state estimator.

[S3] The method is clearly described, with useful experimental analyses.

The paper provides clear mathematical formulations and compares J-Lens with Logit Lens. It also examines the effects of intervention depth, the geometry of value directions, and how steering coefficients change during generation.

Critical weaknesses

[W1] The evaluation metrics may not fully reflect multi-value control performance.

The main interaction metrics measure relative changes in value scores when an additional value constraint is introduced, rather than whether all target values are satisfied together. As a result, better scores on these metrics do not necessarily indicate better multi-value control. Although the paper also reports GeoGain, it is still based on relative changes in value scores. It would be helpful to report absolute scores for each value, the rate of satisfying all target values simultaneously, and the performance on the worst-performing target. These would provide a clearer picture of how well each method achieves the intended control.

[W2] The value-state observer could be better validated.

AIMES uses J-Lens scores over value-related tokens to estimate the model's current value state. However, it is unclear how well these vocabulary-based scores reflect the model's actual value preferences. The comparison between J-Lens and Logit Lens mainly shows how the observer scores respond to interventions, but their connection to the model's final behavior remains unclear. An analysis of the correlation between observer scores and value-related behavior would help support this design choice.

[W3] The baseline comparisons could be more comprehensive.

The paper mainly compares AIMES with Fixed Multi-Value Steering and Prompt Steering. Comparisons with existing multi-attribute steering methods, such as ORBIT and K-Steering, would help better establish the advantages of AIMES. Another concern is that AIMES uses steering coefficients below 1, while Fixed Steering uses a constant coefficient of 1. The paper also shows that AIMES generally produces smaller activation perturbations. This makes it difficult to tell whether the improvements come from dynamic feedback or simply from using weaker interventions. A static baseline with matched intervention strength would help clarify the contribution of dynamic feedback.

Questions for authors

see weakness

2 comments
AuthorsOpenReview ↗

Gentle reminder to reviewer zYR9

Dear Reviewer zYR9,

Thank you again for your constructive feedback. We have addressed your comments in our responses and uploaded a revised manuscript incorporating additional experiments, including J-Lens observer validation and comparisons with Mean-Strength Fixed Steering. We would be grateful if you could review our responses and the updated manuscript and reconsider your assessments and scores in light of the new evidence.

Thank you for your time and consideration.

Dear Reviewer zYR9, As we approach the end of the discussion period, we would like to sincerely thank you again for your constructive feedback and thoughtful evaluation. We believe we have addressed most of your concerns through our detailed responses, additional experiments, and revised manuscript. We would be very grateful if you could take one final look at these updates and consider revisiting your assessment and score in light of the new evidence. If you have any other question we would be happy to clarify. Thank you again for your time and consideration.

Comments on the paper

2 comments

Dear reviewers,

We’re about halfway through the discussion period, and the authors have put considerable time into responding to your feedback. Please read their rebuttal and leave a brief comment letting the authors know whether it addressed your questions or concerns and what, if anything, remains open.

Thank you!

AuthorsOpenReview ↗

Summary of Revisions and Response to Reviewers

Dear Area Chair and Reviewers,

We sincerely thank all three reviewers for their constructive feedback and careful evaluation of our work. We have addressed the major concerns raised during the discussion and uploaded a revised manuscript incorporating additional experiments and supporting analysis.

Summary of Strengths. Our work addresses the important problem of jointly controlling multiple interacting human values in LLMs. AIMES provides a lightweight, training-free framework that uses intermediate-layer vocabulary readouts as online observers to adapt value-specific steering strengths during generation. The evaluation spans five models from three families, three multi-value objectives, ten intervention depths, and two independent behavioral judges. Together, these experiments characterize the effectiveness, depth dependence, and limitations of adaptive multi-value steering.

Main Changes Addressing Reviewer Concerns.

J-Lens Observer Validation:

We added direct behavioral validation using independently evaluated value-expression scores from GPT-5.6-Sol and Claude Opus-4.8. Across five models and three representative depths, 188/210 (89.5%) observer-behavior Spearman correlations are positive, supporting the behavioral relevance of J-Lens readouts.

Contribution of Adaptive Feedback:

We added a Mean-Strength Fixed Steering baseline that uses AIMES-derived average steering coefficients without online adaptation. AIMES achieves higher GeoGain in 74% of configurations for Care/Fairness/Sanctity and 72% for Loyalty/Authority, providing further evidence for the benefits of state-dependent, value-specific feedback beyond reduced fixed steering coefficients. Additional Clarifications: We clarified the interpretation of multi-value evaluation metrics, objective-dependent performance, and the relationship to existing multi-attribute and feedback-based steering methods. We also explicitly distinguish average-coefficient matching from exact perturbation-norm matching.

Updated Manuscript:

The new experimental findings are incorporated directly into Section 4, particularly Tables 2 and 3 on pages 8–9, rather than being limited to the rebuttal. The corresponding revisions are highlighted in blue for ease of review, and the conclusion has been updated to reflect the additional evidence.

We believe these additions meaningfully strengthen the empirical support for AIMES and directly address the central concerns raised in the reviews. We would be grateful if the reviewers could consider the updated manuscript and our detailed responses when reassessing their evaluations. Thank you again for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.