acceptodds
Under review as a conference paper at ICLR 2027

Minimum-KL Activation Steering Through a Normalized Linear Readout

Abstract

Activation steering interventions can achieve the same increase in a desired output score while changing other next-token probabilities differently. At a fixed context, we use the distribution minimizing steered-to-baseline KL under the score constraint as a method-independent benchmark. This minimum-KL target is the standard exponential tilt of the baseline distribution. We ask whether a bounded activation intervention can realize it. For interventions immediately before RMSNorm and a linear readout, we give necessary and sufficient conditions for exact attainment: the target logit direction must lie in the centered-readout range, normalization must admit a realizing state, and the minimum required intervention norm must not exceed the budget. When the score increase is feasible but the target is unattainable, we prove a positive lower bound on excess KL. For compatible directions, a nontrivial centered-readout nullspace gives a single attainment threshold, whereas an injective readout can lose and later recover attainability as the target increase grows. Experiments on three small pretrained models and four lexical event scores find large range mismatches in all twelve model-score pairs, ruling out every positive minimum-KL tilt at these final readouts. Constructed compatible targets are exactly reachable only over a narrow range of score increases at the tested budget. These results distinguish achieving a target score from achieving it with the smallest distributional change under the specified KL criterion.

Primary area:
foundation or frontier models, including LLMs
Keywords:
activation steering, minimum-KL target, KL divergence, RMSNorm, linear readout, information projection, activation interventions

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yx67OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To raise the expected value of some token-level score by a target amount, the authors ask what the smallest possible change to the model's next-token distribution is. The authors show that the least-distorting target distribution is a simple re-weighting of the original one toward high-scoring tokens, which gives a benchmark that doesn't depend on any particular steering method. Their main theorem shows when an intervention just before the model's final normalization and output layer can reach that target exactly, based on three necessary and sufficient conditions. When any condition fails, they prove a lower bound on the extra distortion. Empirically, they find that four simple lexical scores in three small language models all fall largely outside the expressible space, so exact attainment is impossible. Even artificially constructed scores that pass this test are exactly reachable only for very small score increases. A toy model and checks at earlier layers support the theory.

Critical strengths
  1. A clean, method-independent definition of "selective" steering. The minimum-KL target is the exponential tilt qλ∝p0eλfq_\lambda \propto p_0 e^{\lambda f} (Eq. 2). Identity (3), DKL(p∣p0)−I(Δ)=DKL(p∣qλ)+λ(Epf−s0−Δ)D_{\mathrm{KL}}(p|p_0) - I(\Delta) = D_{\mathrm{KL}}(p|q_\lambda) + \lambda(\mathbb{E}_p f - s_0 - \Delta), splits any intervention's excess KL into distortion and overshoot. This gives the steering literature a precise, intervention-agnostic yardstick for side effects, which I found very informative.
  2. A rigorous and fairly complete attainability theory for the final normalized readout. Theorem 1 gives necessary and sufficient conditions for Kr(Δ)=I(Δ)K_r(\Delta) = I(\Delta) with a quantitative excess-KL lower bound. Theorem 2 gives the minimum intervention norm for every readout rank, and Example 10 shows that attainability can be non-monotone. Each assumption is shown to be needed by a counterexample (Appendix: D2, D4, D6), and the results I checked hold. The theory appears correct and carefully hedged.
Critical weaknesses
  1. The practical size of the benchmark gap is never measured. The paper's motivation is evaluating steering, yet no steering method (e.g. ActAdd/CAA) is compared against I(Δ)I(\Delta), no value of Kr(Δ)−I(Δ)K_r(\Delta) - I(\Delta) is computed for a pretrained model, and the constant cλc_\lambda is never evaluated. The authors themselves note that this constant "may be small for a large vocabulary" (p.5), and since cλ≤1/(2m)c_\lambda \le 1/(2m) with m≈5×104m \approx 5\times10^4-1.5×1051.5\times10^5, the bound is likely negligible. Without such a measurement, the claim to distinguish reaching a target score from reaching it with minimal distributional change has no practical size. Solving Kr(Δ)K_r(\Delta) at the final layer (d≤960d \le 960) and scoring one standard steering vector would address this.
  2. The main empirical finding follows from dimension counting. The centered readout has rank d≪md \ll m, so almost any score direction is out of range. A random direction would give a relative residual ≈1−d/(m−1)≈0.990\approx \sqrt{1 - d/(m-1)} \approx 0.990-0.9970.997, which is higher than the reported 0.888-0.980 (Table 1). App. G.1 acknowledges that the dimension argument "does not bound the residual", yet the abstract presents "large range mismatches in all twelve model–score pairs" as a finding. There is no null baseline, and the softmax-bottleneck literature (Yang et al., 2018; Demeter et al., 2020; Grivas et al., 2022) is not cited. Reporting random-direction residuals next to Table 1 and positioning the result relative to that work would fix this.
Questions for authors
  1. What residuals do random isotropic directions in 1⊥\mathbf{1}^\perp, or random token sets of the same sizes, give for the readouts in Table 1? Is anything about the lexical scores specific beyond d≪md \ll m?
  2. For the Table 1 scores at a representative Δ\Delta and rr, what is the numerical value of the bound cλλ2∣b⊥∣2c_\lambda \lambda^2 |b_\perp|^2 in Eq. (16), and what is the actual Kr(Δ)−I(Δ)K_r(\Delta) - I(\Delta) obtained by optimizing over the final-layer ball?
  3. In Sec. 5.3 the in-range direction satisfies v⊥z0v \perp z_0 and ε/∣h0∣2≈10−8\varepsilon/|h_0|^2 \approx 10^{-8}. Is the "narrow range" in the abstract set by this construction and by RMSNorm's near-unit-sphere regime rather than by the scores? How do the endpoints change for directions not orthogonal to z0z_0?
  4. Can you share results for scoring standard mid-layer steering methods (ActAdd/CAA, dual steering) by their excess KL over I(Δ)I(\Delta) at matched score gains? Perhaps also extending the benchmark to multi-token properties (Appendix G4)
Reviewer_c3mMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper theoretically characterizes when bounded activation steering at a fixed context, immediately before RMSNorm and a linear vocabulary readout, can achieve a prescribed increase in an output score while attaining the minimum possible KL divergence from the baseline distribution. The minimum-KL target is given by an exponential tilt, and the authors derive three necessary and sufficient conditions for its exact attainment: (1) the target centered-logit direction lies in the readout range, (2) the required normalized activation is realizable, and (3) the minimum required intervention norm is within the budget. The authors also derive lower bounds on excess KL when the target is unattainable but the score constraint remains feasible. They further characterize how attainability changes with the target score, showing a single threshold for readouts with a nontrivial nullspace and possible non-monotonic behavior for injective readouts. Empirically, they find substantial violations of the readout-range condition across three pretrained models and four lexical scores. Additional experiments with constructed in-range scores show that normalization and budget constraints can restrict exact attainment to a narrow range of score increases.

Critical strengths

Rigorous theoretical characterization. The authors derive necessary and sufficient conditions for exact minimum-KL attainment, characterize the minimum required intervention norm, and establish lower bounds on excess KL. They further analyze how attainability depends on the readout's rank and normalization geometry.

Critical weaknesses
  1. Limited scope of fixed-context analysis. The main theoretical analysis focuses on activation steering at a fixed context, whereas practical steering often aims to achieve consistent effects across multiple contexts using a shared intervention. Although the authors briefly discuss the multi-context setting in Appendix G.4, the main theoretical guarantees do not directly extend to this setting, limiting their practical applicability.

  2. Limited insights beyond exact attainment. The theoretical analysis focuses primarily on whether the minimum-KL distribution can be attained exactly. However, since the vocabulary size is typically much larger than the hidden dimension, a generic target logit direction will lie outside the readout range, making exact attainment impossible. This makes the range incompatibility observed in Section 4 less surprising. The more practically relevant question is how much excess KL is incurred when exact attainment fails. Although the authors derive lower bounds on excess KL and some upper bounds under additional assumptions, they provide limited characterization of the actual minimum achievable KL in this setting. In particular, the experiments demonstrate readout-range incompatibility but do not quantify the resulting excess KL for score-feasible interventions, leaving the practical significance of this incompatibility unclear.

Questions for authors
  1. Generalization across contexts. To what extent can the theoretical characterization be extended to a setting where a shared activation intervention is applied across multiple contexts?

  2. Practical significance of readout-range incompatibility. Although Section 4 demonstrates substantial readout-range incompatibility, it is unclear how much excess KL this leads to in practice. Could the authors provide empirical estimates of the excess KL? Additionally, could the authors compare the observed readout-range residuals against a random-direction baseline to assess whether they reveal more than the expected dimensional mismatch?

Reviewer_tUAzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper defines selective steering through a minimum-KL output distribution satisfying a prescribed next-token score increase. It then characterizes when an activation intervention before an RMS-normalized linear readout can realize that distribution within a norm budget. Exact attainment requires readout-range compatibility, normalization feasibility, and sufficient intervention budget. The paper derives minimum intervention norms, distinguishes nullspace and injective readouts, and examines these conditions in small pretrained models.

Critical strengths

Its strongest contribution is the finite-budget geometry, particularly the scalar minimum-norm calculation and the convexity result when the centered readout has a nontrivial nullspace. The injective example showing loss and recovery of exact attainability is also valid and informative. The paper carefully distinguishes score feasibility from distributional optimality, and explicitly states the limitations of its bounds and empirical diagnostics.

Critical weaknesses
  1. Table 1 reports large readout-range residuals, but does not quantify achievable excess KL at feasible lexical score gains. The authors acknowledge that the lower-bound constant can be very small, so these residuals do not establish a meaningful penalty for steering. Indeed, Appendix E.1 shows that exact attainment can fail for every positive gain while excess KL is only fourth order in the gain. The experiments therefore establish impossibility of exact equality more convincingly than a consequential limitation on approximate steering.

  2. The lexical experiments mainly test the low-dimensional readout-range obstruction. All three tested readouts are injective, so the nullspace-dependent threshold result is not instantiated in those models. The compatible-score experiment uses eight prompts and randomly constructed, context-specific scores on one model. This is a useful geometric illustration, but it does not establish how the finite-budget results inform meaningful behavioral steering.

  3. The earlier-layer extension requires control of downstream approximation errors. Appendix H.3 reports error increments comparable to, or larger than, the actual output changes, and does not certify the required uniform bounds. The authors correctly acknowledge this. Nevertheless, the extension currently provides limited support for applying the framework to standard intermediate-layer steering or shared interventions across contexts.

Questions for authors
  1. Do any existing results quantify achieved excess KL, or a numerically informative lower bound, for the lexical scores at feasible nontrivial gains?

  2. The reported normalization ratios are approximately 10^(-8). Are the small positive-attainment intervals intended solely as properties of the mathematical float64 model, or are they stable under the checkpoints’ normal inference precision?

  3. Beyond the generic range obstruction, which existing result most directly demonstrates that the finite-budget geometry provides an actionable diagnosis for a pretrained model?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.