acceptodds
Under review as a conference paper at ICLR 2027

Projection Steering: Specifying Output Distributions over Structured Concepts

Abstract

This paper formalizes what counts as good steering for structured concepts, whose values form a cycle, an order, or a tree—months of the year, numerals, a taxonomy. We identify three core desiderata that the steering literature has addressed as separate components: steering should achieve the intended target probabilities, follow a path that respects the concept’s geometric structure, and disrupt the off-target behavior as little as possible. These desiderata are derived directly from the model’s output distribution and the structure of the concept. We then propose projection steering as a convex optimization program whose objective minimizes the disruption to the model's softmax output distribution, subject to constraints that specify the concept’s conditional distribution (e.g. 60% probability of April when restricted to months of the year) and the total probability mass assigned to the concept (e.g. probability allocated to month tokens). This formulation enables attaining the target distribution exactly whenever it is reachable. We then develop a general construction that uses the geometry of the target concept (e.g. a circle for months of the year) to turn a semantic path into a family of target output distributions. Crucially, choosing this semantic path is separate from determining how to steer the model to realize it. We verify that this three-step framework has clear reachability conditions, attains all proposed desiderata, and improves on existing steering baselines on both a synthetic task and a real language model.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, structured concepts, language model interpretability, representation geometry, information geometry, convex optimization, feature fields

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_npc6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies activation steering for structured concepts whose labels have an explicit relational organization, such as cycles, orders, or trees. The central proposal is to describe steering behavior directly at the output-distribution level through three desiderata: target accuracy, route fidelity, and preservation of off-target behavior. The paper decomposes the model distribution into domain mass, the conditional distribution within the target concept, and the conditional distribution over all remaining outputs, and then formulates projection steering as a forward-KL projection onto behavioral constraints specifying the desired in-domain conditional and a minimum domain mass. The authors establish convexity and reachability conditions, prove an off-target optimality result at matched realized behavior, and give a kernel-distance construction that converts a semantic path into target distributions. Experiments use Gemma-3-4B with final-layer unembedding readouts for months, weekdays, and decade words, an intermediate-layer multiclass probe for an animal taxonomy, and a synthetic helix task. The results suggest that projection steering can realize specified targets exactly when reachable and can preserve the off-target conditional better than several linear and geometry-aware baselines.

Critical strengths

S1. The decomposition in Sec. 3 is a useful way to make the steering objective precise. Separating domain mass ρ\rho, the in-domain conditional qq, and the outsider conditional rr, together with the KL partition identity in Lemma 1, gives a clean operational distinction between reaching a target concept, controlling how probability is distributed within that concept, and preserving unrelated behavior. This is more informative than evaluating steering only through a target-class score.

S2. The optimization formulation is technically clean. Secs. 4–5 show that the behavioral constraints lead to a convex feasible set, characterize reachability through the centered readout, and formulate projection steering as a forward-KL projection. Theorem 10 also gives a concrete interpretation of the forward-KL choice: at the same realized in-domain conditional and domain mass, the solution minimizes the stated off-target KL. I did not identify an obvious inconsistency in this chain of results.

S3. The separation between specifying a semantic route and realizing that route is conceptually appealing. Sec. 6 allows the desired trajectory to come from declared, empirical, or readout-induced geometry, while the projection procedure itself is unchanged. This makes the framework applicable to qualitatively different structures such as cycles, orders, and trees rather than tying the intervention to one particular manifold construction.

S4. The baseline set is reasonably diverse for the proposed comparison. Sec. 7 and Appendix C.3 include activation steering, a whitened linear-probe variant, dual steering, fitted-manifold steering, and geodesic steering. I also appreciate the matched-behavior comparison in Appendix C.1, where projection steering is rerun using a baseline's realized (q,ρ)(q,\rho) before comparing off-target error; this avoids interpreting outsider KL differences that are simply caused by the methods reaching different targets.

Critical weaknesses

W1. My main concern is that the empirical claims are not supported by aggregate quantitative evaluation over the test sets. Table 1 lists 14 weekday, 24 month, and 18 age test instances, 817 held-out taxonomy articles, and a synthetic task with per-seed data, but Appendix C.1 explicitly states that the appendix figures show the first held-out prompt. The main conclusions are then illustrated primarily through individual trajectories such as Fig. 2 and the single taxonomy route in Fig. 10. I could not find a table reporting mean/median target error, route error, off-target error, success rate, or dispersion across held-out prompts, source-target routes, or synthetic seeds for all methods. This makes it difficult to know whether the displayed examples are representative, and it is the main reason I am below the acceptance threshold.

W2. The real-model evidence is narrow relative to the general framing. Sec. 7 evaluates only Gemma-3-4B. The final-layer experiments cover three relatively small symbolic domains (months, weekdays, and decade words), while the only intermediate-layer structured concept is the animal taxonomy probe. The theory is model-agnostic, but the empirical statement that projection steering improves on existing steering baselines is therefore supported on one architecture/scale and a small set of highly structured concepts. Results on another model family or scale would materially strengthen the empirical case; alternatively, the empirical generality claims should be narrowed.

W3. The experimental construction makes the favorable reachability regime unusually strong. Appendix C.3 states that, for the unembedding experiments, the support QQ contains the centered label rows and consequently the centered in-domain readout has full rank in the support, so every target conditional is reachable and only the mass constraint can be infeasible. This is appropriate for validating the solver, but it sidesteps the practically important case in which steering is restricted to a genuinely low-dimensional or imperfect edit subspace. Since reachability is one of the central contributions, I would expect an empirical stress test varying the support dimension/composition, including partially unreachable targets and cases near ρmax⁡\rho_{\max}. Without this, the near-zero target error in the figures is partly a consequence of how the support is constructed.

W4. Route fidelity is evaluated against a target route that is, in the unembedding experiments, generated from the same readout geometry used by the method. Sec. 6 explicitly notes that Euclidean distance in representer coordinates is not guaranteed to be the correct semantic metric, yet Appendix C.2 uses this geometry to construct the target conditionals and then evaluates route error relative to those targets. Thus the low route error demonstrates that projection steering realizes the specified route, but it does not establish that the readout-induced route is the semantically appropriate one. I could not find a sensitivity study comparing declared, empirical, and readout geometries, varying the retained geometry dimension, or perturbing the metric. Such evidence is important because the semantic-path choice is a major component of the proposed framework.

W5. The intermediate-layer experiment does not validate downstream behavioral steering. Sec. 7.2 explicitly states that the taxonomy result measures control of the probe output rather than downstream token probabilities or generated text, and Appendix C.8 reports only 69.9% held-out accuracy for the 61-class probe. Consequently, exact control of this probe distribution may not correspond to exact control of the model's actual semantic behavior. The final-layer experiments are also one-step, single-token output controls. An independent downstream evaluation, or at least evidence that the edited intermediate activations produce the intended change under the model's subsequent layers, would make the practical steering claim much more convincing.

W6. The paper does not quantify the computational cost of the projection procedure. Appendix B describes a reachability optimization, repeated Newton/L-BFGS solves and bisection, and Hessian/Fisher computations using roughly the top 20,000 tokens for a vocabulary of about 262,000; Appendix C.3 further states that every route position is projected independently from the reference state. I could not find runtime, memory, number-of-solver-iteration, or throughput comparisons against the much cheaper activation-steering baselines. This does not invalidate the method, but it leaves an important part of its practical usefulness unresolved.

Questions for authors

Q1. Can the authors provide aggregate quantitative results over all held-out prompts and source-target routes for the month, weekday, age, and taxonomy experiments, and over multiple synthetic seeds, reporting at least target error, route error, off-target error, and constraint-success rate for every baseline? Appendix C.1 says that the appendix figures show the first held-out prompt, so these aggregates would substantially affect whether I view the current qualitative examples as representative.

Q2. Do the authors have results on another language model family or model size, even for a subset of the structured concepts? If not, would the authors agree to narrow the empirical claim to Gemma-3-4B rather than presenting the observed advantage as evidence of general steering behavior across language models?

Q3. How does projection steering behave when QQ does not contain the full centered in-domain readout row space? In particular, can the authors report a sensitivity experiment that reduces or perturbs the support dimension and measures the fraction of unreachable targets, target error when exact reachability fails, and behavior near the ρmax⁡\rho_{\max} boundary? This would test the reachability analysis in a setting not constructed to make all target conditionals feasible.

Q4. How sensitive are the route-fidelity results to the semantic geometry itself? For the unembedding tasks, can the authors compare the current readout geometry with a declared geometry and/or an empirical geometry, or vary the representer-coordinate truncation, and report how the resulting routes and steering metrics change? Because route error is measured against the route generated from the chosen geometry, this would help separate successful optimization from correctness of the semantic specification.

Q5. For the taxonomy experiment, is there evidence that steering the layer-16 probe distribution changes the model's downstream token distribution or generated behavior in the intended taxonomic direction? Given the probe's 69.9% held-out accuracy and the paper's explicit caveat that this experiment does not test downstream behavior, an independent downstream check would materially change how strongly I interpret this result.

Q6. What is the wall-clock/runtime and memory cost per projected route position on Gemma-3-4B, and how many inner iterations are typically required, compared with activation steering and dual steering? Since Appendix B uses repeated optimization over a large vocabulary and Appendix C.3 projects each route position independently, this information is important for assessing whether the method is practical beyond offline analysis.

2 comments
AuthorsOpenReview ↗

Replies to Q1-Q3

We thank the reviewer for the detailed feedback.

Due to an outage on our compute cluster, we were unable to run additional experiments during the rebuttal period. We address each question below using the paper's existing theory and results, and we will add the corresponding experiments in the revision.

We first discuss some high-level points.

Our aim is not to offer a new steering method but to build preliminary steps toward a general, formalized framework that can lead to exact control of the model's output behavior and a better understanding of how models represent concepts. We work with next-token distributions over structured concepts because they permit a mathematically rigorous analysis, the tools for which do not exist yet for entire sequences. We agree that steering model behavior is the eventual goal, and an optimization problem faithful to model behavior is a natural extension of this work.

Our central design is optimization under behavioral constraints, with a separation between specifying semantic routes and solving the optimization problem. This separation lets us realize a route of distributions exactly, regardless of how the route is specified. The reviewers are right that steering outcomes depend on the chosen semantic geometry, and that routes are pre-specified rather than discovered; these remain open questions, which we flag in the Limitations. What our work guarantees is that any given route (a family of distributions) is realized up to solver precision. So if you have another method for discovering the route, this procedure can slot this in.

More specific points are addressed below.

Q1. Can the authors provide aggregate quantitative results over all held-out prompts and source-target routes for the month, weekday, age, and taxonomy experiments, and over multiple synthetic seeds, reporting at least target error, route error, off-target error, and constraint-success rate for every baseline? Appendix C.1 says that the appendix figures show the first held-out prompt, so these aggregates would substantially affect whether I view the current qualitative examples as representative.

A1: We agree that aggregates are needed to judge whether the figures are representative. Our evaluation pipeline already computes every metric over all held-out cases. We now report these aggregates for every method and task, including all three synthetic seeds, in Table 3 (App. C.9).

The aggregates confirm that projection steering attains the behaivoral constraints at the route endpoint in every case, with mean in-domain KL below 10−1510^{-15}, mean mass shortfall below 10−910^{-9}, and zero route error. These values hold because whenever a target is reachable (Prop. 7), the projection meets all behavioral constraints exactly.

Q2. Do the authors have results on another language model family or model size, even for a subset of the structured concepts? If not, would the authors agree to narrow the empirical claim to Gemma-3-4B rather than presenting the observed advantage as evidence of general steering behavior across language models?

A2: Yes, for a subset. An earlier version of our pipeline compared projection steering with contrastive activation addition and dual steering on both Gemma-3-4B and Llama-3.1-8B, over 5,700 conditions. No baseline met all behavioral constraints in any condition on either model. Projection steering met them in every condition on Gemma-3-4B and in 95.8% of months conditions on Llama-3.1-8B (the remaining is reported as infeasible). We will add additional experiments on another model family when we regain access to compute resources.

Theoretically, our method is model-agnostic; that is, the procedure principally works for any language model supplied. Given any model and readout, the theorems guarantee that solving the proposed convex KL projection problem will give the activation edit meeting the constraints.

Q3. How does projection steering behave when QQ does not contain the full centered in-domain readout row space? In particular, can the authors report a sensitivity experiment that reduces or perturbs the support dimension and measures the fraction of unreachable targets, target error when exact reachability fails, and behavior near the ρmax⁡\rho_{\max} boundary? This would test the reachability analysis in a setting not constructed to make all target conditionals feasible.

A3: Reachability depends on QQ in two ways (Prop. 7). Omitting part of the centered in-domain row space makes some conditionals unreachable, which the solver reports as INFEASIBLE (SHAPE). Omitting the mean label row lowers ρmax⁡\rho_{\max}, which it reports as INFEASIBLE (MASS). Existing runs test the second case. On Gemma-3-4B, over five further domains and 12,762 solves, restricting QQ to the centered label rows left only 6.1% of targets reachable. Adding the mean label row restored reachability to 100%.

AuthorsOpenReview ↗

Replies to Q4-Q6

Q4. How sensitive are the route-fidelity results to the semantic geometry itself? For the unembedding tasks, can the authors compare the current readout geometry with a declared geometry and/or an empirical geometry, or vary the representer-coordinate truncation, and report how the resulting routes and steering metrics change? Because route error is measured against the route generated from the chosen geometry, this would help separate successful optimization from correctness of the semantic specification.

A4: Given different semantic geometries (declared, empirical, readout), the corresponding routes formed by the kernel bump technique do differ, with result sensitivity depending on the concept being steered. However, the ordering through the route does not depend on the semantic geometry (Props. 12–13). We have also acknowledged at the end of §6 and in the Limitations of §8 that the correct metric for the encoded semantic geometry remains unclear.

We emphasize, however, that specifying a route is decoupled from solving the optimization to realize it. For any semantic geometry, projection steering meets every reachable target on the route exactly (Prop. 7) and, among states meeting it, minimizes off-target change (Thm. 10).

Q5. For the taxonomy experiment, is there evidence that steering the layer-16 probe distribution changes the model's downstream token distribution or generated behavior in the intended taxonomic direction? Given the probe's 69.9% held-out accuracy and the paper's explicit caveat that this experiment does not test downstream behavior, an independent downstream check would materially change how strongly I interpret this result.

A5: Our current work focuses on the theoretical groundings for provably exact control of a softmax readout distribution. Extending this to control of downstream generation would require extending the objective and behavioral constraints from next-token distributions to generation trajectories. We leave this extension to future work and have clarified this scope in the paper.

Q6. What is the wall-clock/runtime and memory cost per projected route position on Gemma-3-4B, and how many inner iterations are typically required, compared with activation steering and dual steering? Since Appendix B uses repeated optimization over a large vocabulary and Appendix C.3 projects each route position independently, this information is important for assessing whether the method is practical beyond offline analysis.

A6: We have added an asymptotic cost comparison of projection steering, ActAdd, and dual steering to App. B.3 (Table 3). Each Newton step of projection steering costs O(Vs+Ks2)O(Vs + Ks^2), where VV is the vocabulary size, KK is the top-KK tokens used to form the covariance matrix, and ss is the rank of the edit directions QQ. Note that this is linear in VV. Per route position, projection steering costs O(BM(Vs+Ks2))O(BM(Vs + Ks^2)), where BB and MM are the numbers of bisection and Newton iterations. This is more per position than ActAdd (O(d)O(d)) or dual steering (O(TV)O(TV)), but still affordable for offline steering and for online steering at constrained positions, where each steered token adds a few milliseconds on Gemma-3-4B. Memory cost is O(Vs+Ks)O(Vs + Ks), about 70 MB for months on Gemma-3-4B, roughly 1% of the model weights. We do not yet have measured wall-clock times and will report them in the revision.

Reviewer_FxXiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes (i) projection steering, a steering framework for structured concepts; and (ii) a set of metrics to evaluate the quality of the steered model’s outputs. Structured concepts are a set of labels that carry relational structures e.g. cycles. The paper’s running example is a set of months, such that if a model is being steered from January to April, intermediate model states would place high probability on months in between (February, March) in an ordered manner. The authors introduce three desiderata to judge ‘good’ steering: target accuracy (the steered model showing the desired behavior), off-target preservation (distributions over concepts unrelated to the target change as little as possible), route fidelity (intermediate model states follow the concept’s structure in the correct order). Projection steering finds the hidden state that matches a target distribution over concept labels and satisfies a lower bound on the total probability assigned to the concept. Empirical tests are run on Gemma 3 4B.

Critical strengths

S1. The idea of structured concepts and the three way decomposition (domain mass, mass distribution among labels, mass distribution over non-label vocabulary) is sound and makes for a clean evaluation of steering performance with the three metrics proposed (target acc, off target comparison, route fidelity).

Critical weaknesses

W1. The method as presented is interesting but applications of it do not seem clear. While the structured concepts used to present the method are very illustrative (months, weekdays, etc), they are mostly toy examples. Also results are presented on a single 4B model.

W2. The empirical tests are not really testing the method so much as verifying the solver (target and route error in Fig 2 tend to zero by constraints of the method). Related to W1, it would be useful to see this method presented with a general evaluation (even a prepared battery of more structured concepts across different topics would be useful). Ideally the authors also present continuations of the projection-steered models, seeing as that is the typical end goal of model steering (observing the generated continuation, not the next-token distribution.)

Questions for authors

Q1. This method is very interesting! Could you discuss how useful projection steering is beyond structured concepts? For instance a lot of steering work is done on persona-related concepts, for instance make models kinder or more honest, could projection steering be useful there?

Q2. Your demonstration with the months example is very illustrative, yet I would like to know if you can give other examples that the community might be interested in like safety related ones. Maybe some structured concept involving discrete levels of harm or severity.

Q3. Could you share time and cost estimates of your method?

Q4. Could you share a comparison of continuation fluency if you apply all Fig 2 steering methods (including projection steering) at some intermediate layer? Outputs of steered models can sometimes read incoherent, so it would be useful to understand this about projection steering (even if the paper is focused on next-token distributions)

Q5. Please share a short paragraph on how you expect the community to use this method, what is the intended use-case in the form projection steering is currently presented?

Notes: A standalone discussion or conclusion section would make the paper easier to follow (for a camera ready I would strongly suggest unmerging the section from related work.

3 comments
AuthorsOpenReview ↗

Replies to Q1-Q3

We thank the reviewer for the insightful feedback.

Due to an outage on our compute cluster, we were unable to run additional experiments during the rebuttal period. We address each question below using the paper's existing theory and results, and we will add the corresponding experiments in the revision.

We first discuss some high-level points.

Our aim is not to offer a new steering method but to build preliminary steps toward a general, formalized framework that can lead to exact control of the model's output behavior and a better understanding of how models represent concepts. We work with next-token distributions over structured concepts because they permit a mathematically rigorous analysis, the tools for which do not exist yet for entire sequences. We agree that steering model behavior is the eventual goal, and an optimization problem faithful to model behavior is a natural extension of this work.

We focus on structured concepts, where the desired trajectory is known, but the framework in principle applies to unstructured concepts and binary ones (it is agnostic to the method of choosing the route). Regarding applications, the method applies to safety-related terms like personas, refusal, and honesty. Given any (family of) distributions in the corresponding readout spaces, projection steering applies to find an activation that reaches the distribution over the concept while minimizing change in other concept distributions.

More specific points are addressed below.

Q1. This method is very interesting! Could you discuss how useful projection steering is beyond structured concepts? For instance a lot of steering work is done on persona-related concepts, for instance make models kinder or more honest, could projection steering be useful there?

A1: Yes – projection steering applies to any multiclass concept with a softmax readout, with or without known geometric structure. In fact, we focus on structured concepts for generality so that the framework has unstructured and binary concepts as special cases. In your example, for instance, consider a persona subspace in Rk\mathbb{R}^k and a route through personas. Build the readout rows from the persona centroids, for example as an LDA probe; each route position then specifies a target distribution over personas, which serves as the behavioral constraint for the KL projection.

Q2. Your demonstration with the months example is very illustrative, yet I would like to know if you can give other examples that the community might be interested in like safety related ones. Maybe some structured concept involving discrete levels of harm or severity.

A2: A natural safety example is persona drift. Lu et al. (2026) show that the leading direction of persona space, the Assistant Axis, tracks how far a model has drifted from its default Assistant persona, predicting harmful behavior; their activation capping clamps the projection onto this axis when it leaves its normal range. Projection steering can express a similar safeguard as a behavioral constraint over discrete severity levels, for instance by grouping persona classes into ordered drift levels z0≺⋯≺zkz_0 \prec \dots \prec z_k along the axis and fitting an LDA probe over them. In this formulation, projection steering restores exactly the required safe mass, keeps the active safe persona, and minimizes all other change. If the floor is unreachable, the solver certifies this before acting, which itself indicates persona drift. These guarantees hold for the probe's distribution, but whether they reduce harmful generations must be measured, which we will leave to future work.

Q3. Could you share time and cost estimates of your method?

A3: We have added an asymptotic cost comparison of projection steering, ActAdd, and dual steering to App. B.3 (Table 3). Each Newton step of projection steering costs O(Vs+Ks2)O(Vs + Ks^2), where VV is the vocabulary size, KK is the top-KK tokens used to form the covariance matrix, and ss is the rank of the edit directions QQ. Note that this is linear in VV. Per route position, projection steering costs O(BM(Vs+Ks2))O(BM(Vs + Ks^2)), where BB and MM are the numbers of bisection and Newton iterations. This is more per position than ActAdd (O(d)O(d)) or dual steering (O(TV)O(TV)), but still affordable for offline steering and for online steering at constrained positions, where each steered token adds a few milliseconds on Gemma-3-4B. Memory cost is O(Vs+Ks)O(Vs + Ks), about 70 MB for months on Gemma-3-4B, roughly 1% of the model weights. We do not yet have measured wall-clock times and will report them in the revision.

AuthorsOpenReview ↗

Replies to Q4-Q5

Q4. Could you share a comparison of continuation fluency if you apply all Fig 2 steering methods (including projection steering) at some intermediate layer? Outputs of steered models can sometimes read incoherent, so it would be useful to understand this about projection steering (even if the paper is focused on next-token distributions)

A4: We agree that useful steering should change the generated continuation predictably while preserving fluency. We expect projection steering to compare favorably, since its objective explicitly minimizes off-target distortion (Theorem 10), and it achieves the lowest off-target error among the methods in Fig. 2(d). We will add additional experiments on continuation fluency when we regain access to compute resources.

Q5. Please share a short paragraph on how you expect the community to use this method, what is the intended use-case in the form projection steering is currently presented?

A5: This work provides a theoretical grounding for steering as an optimization problem under behavioral constraints, with two intended uses.

First, exact control. Projection steering realizes a specified family of concept distributions exactly, while minimizing changes to off-target behavior. The intended use case is any setting that requires a guarantee of attaining a specified distribution. The following is applicable generally to any concept, with or without geometric structure: Given a readout (the unembedding or a probe), specify target distributions over its labels, and solve for the minimal-KL activation that realizes them.

Second, understanding. A central open question is how models organize semantics internally; ideally, we would find these representations and intervene on them directly. Existing steering methods are effective but do not specify the target behavior itself. Because our framework makes the specification explicit, it can serve as an instrument for studying representations. For example, one can test which target distributions a readout can reach. We hope this leads to a finer-grained understanding of model representations, with applications to control and monitoring (see our responses to the safety-related questions above).

Thank you for your rebuttal. It is unfortunate that you have not been able to run additional experiments to potentially address reviewer concerns.

A1. It is difficult to consider such unstructured concepts (personas) without a worked example. Can the current method accomplish something like this?

A2. Similar to A1, I have not seen any example in the paper that would hint at projection steering working in this setting.

A3. Thank you for adding this.

A4. This would be a useful addition to the paper.

A5. As I state in my initial review, I find these use-cases very exciting! It would be great to see some of these applications in the paper.

Most of my questions are still unanswered and I will maintain my scores, but I hope the authors run some experiments showing direct applications of the method and submit again.

Reviewer_DWX6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Projection Steering, a framework for steering language models toward structured concepts such as months, numbers, and taxonomies. It aims to achieve three goals: reach the desired output distribution, follow the intended semantic route, and minimize changes to unrelated outputs. The method formulates steering as a convex optimization problem using forward KL divergence. Experiments on Gemma-3-4B, including month steering, animal taxonomy, and synthetic tasks, show improvements over several existing steering methods.

  1. The paper identifies three important goals for steering: target accuracy, route fidelity, and off-target preservation. ans the authors provide formal definitions, reachability conditions, and proofs for the proposed optimization framework which could be useful for the community .

2 .The method directly specifies the desired output distribution instead of relying only on steering directions which is quite unique as per my knowledge .

3.The paper compares against linear steering, dual steering, and geometry-aware approaches and did a lot of evaluation in their work . One positive thing is the framework can support structured concepts involving cycles, ordered labels etc

Weakness :

  1. The experiments mainly use Gemma-3-4B. Testing on more models and larger language models would make the results more convincing and more generalizable .

  2. The semantic geometry concept is still not clear to me . Need more explanation.

  3. The method requires the target distribution and semantic route but how to choose them automatically remains unclear.

4.. Evaluations is limited for generated text The method mainly evaluates output distributions and intermediate probe predictions., so whether it produces better generated text is unclear .

LLM USAGE :

To understand part of the paper where I find it hard to grasap , check related works , edit and expand for better readability

Critical strengths
  1. Clear problem formulation: The paper identifies three important goals for steering: target accuracy, route fidelity, and off-target preservation.

  2. Strong theoretical foundation: The authors provide formal definitions, reachability conditions, and proofs for the proposed optimization framework.

3.Interesting methodological contribution: The method directly specifies the desired output distribution instead of relying only on steering directions.

4.Comparison with relevant baselines: The paper compares against linear steering, dual steering, and geometry-aware approaches.

5.Potential for broader applications: The framework can support structured concepts involving cycles, ordered labels, and hierarchical relationships.

Critical weaknesses
  1. Limited experimental evaluation: The experiments mainly use Gemma-3-4B. Testing on more models and larger language models would make the results more convincing.

  2. Semantic geometry is unclear: The paper uses readout geometry to define semantic routes, but it does not show that this geometry always represents the true relationships between concepts.

  3. Dependence on predefined targets: The method requires the target distribution and semantic route to be specified in advance. How to choose them automatically remains unclear.

  4. Limited evaluation of generated text: The method mainly evaluates output distributions and intermediate probe predictions. It is unclear whether these improvements lead to better and more consistent generated text.

  5. Scalability is not well studied: The computational cost of the optimization method on larger models, vocabularies, and multiple concepts needs further evaluation.

Questions for authors
  1. Can the proposed method generalize to other language models beyond Gemma-3-4B?

  2. How sensitive are the results to the choice of semantic geometry and route?

  3. Can the method improve actual generated text, rather than only the output distribution or probe predictions?

  4. What is the computational cost compared with existing steering methods, especially for large vocabularies?

  5. How does the method behave when the target distribution is unreachable under the available edit directions?

2 comments

We thank the reviewer for the thoughtful comments.

Due to an outage on our compute cluster, we were unable to run additional experiments during the rebuttal period. We address each question below using the paper's existing theory and results, and we will add the corresponding experiments in the revision.

We first discuss some high-level points.

Our aim is not to offer a new steering method but to build preliminary steps toward a general, formalized framework that can lead to exact control of the model's output behavior and a better understanding of how models represent concepts. We work with next-token distributions over structured concepts because they permit a mathematically rigorous analysis, the tools for which do not exist yet for entire sequences. We agree that steering model behavior is the eventual goal, and an optimization problem faithful to model behavior is a natural extension of this work.

Our central design is optimization under behavioral constraints, with a separation between specifying semantic routes and solving the optimization problem. This separation lets us realize a route of distributions exactly, regardless of how the route is specified. The reviewers are right that steering outcomes depend on the chosen semantic geometry, and that routes are pre-specified rather than discovered; these remain open questions, which we flag in the Limitations. What our work guarantees is that any given route (a family of distributions) is realized up to solver precision.

More specific points are addressed below.

Q1: Can the proposed method generalize to other language models beyond Gemma-3-4B?

A1: Our method is model-agnostic: the procedure applies in principle to any language model. Given any model with a softmax readout and any reachable target, solving the proposed convex KL projection yields the activation edit that meets the constraints with minimal off-target change. We do have partial results on Llama-3.1-8B for months (not included in the initial draft). They show the same qualitative pattern as on Gemma-3-4B. We will add additional experiments on another model family when we regain access to compute resources.

Q2: How sensitive are the results to the choice of semantic geometry and route?

A2: Given different semantic geometries (declared, empirical, readout), the corresponding routes formed by the kernel bump technique do differ, with result sensitivity depending on the concept being steered. However, the ordering through the route does not depend on the semantic geometry (Props. 12–13). As acknowledged in §6 and §8, the correct metric for the encoded semantic geometry remains unclear.

We emphasize, however, that specifying a route is decoupled from solving the optimization to realize it. For any semantic geometry, projection steering meets every reachable target on the route exactly (Prop. 7) and, among states meeting it, minimizes off-target change (Thm. 10).

Q3: Can the method improve actual generated text, rather than only the output distribution or probe predictions?

A3: We agree that the ultimate goal of steering is to change the model's generation trajectory in a predictable way, beyond a single output distribution. Our current work provides provably exact control of the next-token distribution. Exact control of downstream generation would require extending the objective and behavioral constraints from next-token distributions to generation trajectories. We leave this extension to future work and have clarified this scope in the paper.

Q4: What is the computational cost compared with existing steering methods, especially for large vocabularies?

A4: We have added an asymptotic cost comparison of projection steering, ActAdd, and dual steering to App. B.3 (Table 3). Each Newton step of projection steering costs O(Vs+Ks2)O(Vs + Ks^2), where VV is the vocabulary size, KK is the top-KK tokens used to form the covariance matrix, and ss is the rank of the edit directions QQ. Note that this is linear in VV. Per route position, projection steering costs O(BM(Vs+Ks2))O(BM(Vs + Ks^2)), where BB and MM are the numbers of bisection and Newton iterations. This is more per position than ActAdd (O(d)O(d)) or dual steering (O(TV)O(TV)), but still affordable for offline steering and for online steering at constrained positions. Memory cost is O(Vs+Ks)O(Vs + Ks), about 70 MB for months on Gemma-3-4B, roughly 1% of the model weights. We do not yet have measured wall-clock times and will report them in the revision.

Q5: How does the method behave when the target distribution is unreachable under the available edit directions?

A5: Reachability is determined before any projection is solved (Prop. 7, Alg. 1, lines 1–6). If the target conditional lies outside the edit directions, the solver returns INFEASIBLE (SHAPE); if the mass floor exceeds the reachable maximum, it returns INFEASIBLE (MASS) together with ρmax⁡\rho_{\max}.

Thanks for the response . After reading the rebuttal some of my concerns are solved .I will suggest to add the additional experiments and resubmit the paper to some other venue .I will keep my initial score for now. Good luck!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.