- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies activation steering for structured concepts whose labels have an explicit relational organization, such as cycles, orders, or trees. The central proposal is to describe steering behavior directly at the output-distribution level through three desiderata: target accuracy, route fidelity, and preservation of off-target behavior. The paper decomposes the model distribution into domain mass, the conditional distribution within the target concept, and the conditional distribution over all remaining outputs, and then formulates projection steering as a forward-KL projection onto behavioral constraints specifying the desired in-domain conditional and a minimum domain mass. The authors establish convexity and reachability conditions, prove an off-target optimality result at matched realized behavior, and give a kernel-distance construction that converts a semantic path into target distributions. Experiments use Gemma-3-4B with final-layer unembedding readouts for months, weekdays, and decade words, an intermediate-layer multiclass probe for an animal taxonomy, and a synthetic helix task. The results suggest that projection steering can realize specified targets exactly when reachable and can preserve the off-target conditional better than several linear and geometry-aware baselines.
Critical strengths
S1. The decomposition in Sec. 3 is a useful way to make the steering objective precise. Separating domain mass , the in-domain conditional , and the outsider conditional , together with the KL partition identity in Lemma 1, gives a clean operational distinction between reaching a target concept, controlling how probability is distributed within that concept, and preserving unrelated behavior. This is more informative than evaluating steering only through a target-class score.
S2. The optimization formulation is technically clean. Secs. 4–5 show that the behavioral constraints lead to a convex feasible set, characterize reachability through the centered readout, and formulate projection steering as a forward-KL projection. Theorem 10 also gives a concrete interpretation of the forward-KL choice: at the same realized in-domain conditional and domain mass, the solution minimizes the stated off-target KL. I did not identify an obvious inconsistency in this chain of results.
S3. The separation between specifying a semantic route and realizing that route is conceptually appealing. Sec. 6 allows the desired trajectory to come from declared, empirical, or readout-induced geometry, while the projection procedure itself is unchanged. This makes the framework applicable to qualitatively different structures such as cycles, orders, and trees rather than tying the intervention to one particular manifold construction.
S4. The baseline set is reasonably diverse for the proposed comparison. Sec. 7 and Appendix C.3 include activation steering, a whitened linear-probe variant, dual steering, fitted-manifold steering, and geodesic steering. I also appreciate the matched-behavior comparison in Appendix C.1, where projection steering is rerun using a baseline's realized before comparing off-target error; this avoids interpreting outsider KL differences that are simply caused by the methods reaching different targets.
Critical weaknesses
W1. My main concern is that the empirical claims are not supported by aggregate quantitative evaluation over the test sets. Table 1 lists 14 weekday, 24 month, and 18 age test instances, 817 held-out taxonomy articles, and a synthetic task with per-seed data, but Appendix C.1 explicitly states that the appendix figures show the first held-out prompt. The main conclusions are then illustrated primarily through individual trajectories such as Fig. 2 and the single taxonomy route in Fig. 10. I could not find a table reporting mean/median target error, route error, off-target error, success rate, or dispersion across held-out prompts, source-target routes, or synthetic seeds for all methods. This makes it difficult to know whether the displayed examples are representative, and it is the main reason I am below the acceptance threshold.
W2. The real-model evidence is narrow relative to the general framing. Sec. 7 evaluates only Gemma-3-4B. The final-layer experiments cover three relatively small symbolic domains (months, weekdays, and decade words), while the only intermediate-layer structured concept is the animal taxonomy probe. The theory is model-agnostic, but the empirical statement that projection steering improves on existing steering baselines is therefore supported on one architecture/scale and a small set of highly structured concepts. Results on another model family or scale would materially strengthen the empirical case; alternatively, the empirical generality claims should be narrowed.
W3. The experimental construction makes the favorable reachability regime unusually strong. Appendix C.3 states that, for the unembedding experiments, the support contains the centered label rows and consequently the centered in-domain readout has full rank in the support, so every target conditional is reachable and only the mass constraint can be infeasible. This is appropriate for validating the solver, but it sidesteps the practically important case in which steering is restricted to a genuinely low-dimensional or imperfect edit subspace. Since reachability is one of the central contributions, I would expect an empirical stress test varying the support dimension/composition, including partially unreachable targets and cases near . Without this, the near-zero target error in the figures is partly a consequence of how the support is constructed.
W4. Route fidelity is evaluated against a target route that is, in the unembedding experiments, generated from the same readout geometry used by the method. Sec. 6 explicitly notes that Euclidean distance in representer coordinates is not guaranteed to be the correct semantic metric, yet Appendix C.2 uses this geometry to construct the target conditionals and then evaluates route error relative to those targets. Thus the low route error demonstrates that projection steering realizes the specified route, but it does not establish that the readout-induced route is the semantically appropriate one. I could not find a sensitivity study comparing declared, empirical, and readout geometries, varying the retained geometry dimension, or perturbing the metric. Such evidence is important because the semantic-path choice is a major component of the proposed framework.
W5. The intermediate-layer experiment does not validate downstream behavioral steering. Sec. 7.2 explicitly states that the taxonomy result measures control of the probe output rather than downstream token probabilities or generated text, and Appendix C.8 reports only 69.9% held-out accuracy for the 61-class probe. Consequently, exact control of this probe distribution may not correspond to exact control of the model's actual semantic behavior. The final-layer experiments are also one-step, single-token output controls. An independent downstream evaluation, or at least evidence that the edited intermediate activations produce the intended change under the model's subsequent layers, would make the practical steering claim much more convincing.
W6. The paper does not quantify the computational cost of the projection procedure. Appendix B describes a reachability optimization, repeated Newton/L-BFGS solves and bisection, and Hessian/Fisher computations using roughly the top 20,000 tokens for a vocabulary of about 262,000; Appendix C.3 further states that every route position is projected independently from the reference state. I could not find runtime, memory, number-of-solver-iteration, or throughput comparisons against the much cheaper activation-steering baselines. This does not invalidate the method, but it leaves an important part of its practical usefulness unresolved.
Questions for authors
Q1. Can the authors provide aggregate quantitative results over all held-out prompts and source-target routes for the month, weekday, age, and taxonomy experiments, and over multiple synthetic seeds, reporting at least target error, route error, off-target error, and constraint-success rate for every baseline? Appendix C.1 says that the appendix figures show the first held-out prompt, so these aggregates would substantially affect whether I view the current qualitative examples as representative.
Q2. Do the authors have results on another language model family or model size, even for a subset of the structured concepts? If not, would the authors agree to narrow the empirical claim to Gemma-3-4B rather than presenting the observed advantage as evidence of general steering behavior across language models?
Q3. How does projection steering behave when does not contain the full centered in-domain readout row space? In particular, can the authors report a sensitivity experiment that reduces or perturbs the support dimension and measures the fraction of unreachable targets, target error when exact reachability fails, and behavior near the boundary? This would test the reachability analysis in a setting not constructed to make all target conditionals feasible.
Q4. How sensitive are the route-fidelity results to the semantic geometry itself? For the unembedding tasks, can the authors compare the current readout geometry with a declared geometry and/or an empirical geometry, or vary the representer-coordinate truncation, and report how the resulting routes and steering metrics change? Because route error is measured against the route generated from the chosen geometry, this would help separate successful optimization from correctness of the semantic specification.
Q5. For the taxonomy experiment, is there evidence that steering the layer-16 probe distribution changes the model's downstream token distribution or generated behavior in the intended taxonomic direction? Given the probe's 69.9% held-out accuracy and the paper's explicit caveat that this experiment does not test downstream behavior, an independent downstream check would materially change how strongly I interpret this result.
Q6. What is the wall-clock/runtime and memory cost per projected route position on Gemma-3-4B, and how many inner iterations are typically required, compared with activation steering and dual steering? Since Appendix B uses repeated optimization over a large vocabulary and Appendix C.3 projects each route position independently, this information is important for assessing whether the method is practical beyond offline analysis.
2 comments
Replies to Q4-Q6
Q4. How sensitive are the route-fidelity results to the semantic geometry itself? For the unembedding tasks, can the authors compare the current readout geometry with a declared geometry and/or an empirical geometry, or vary the representer-coordinate truncation, and report how the resulting routes and steering metrics change? Because route error is measured against the route generated from the chosen geometry, this would help separate successful optimization from correctness of the semantic specification.
A4: Given different semantic geometries (declared, empirical, readout), the corresponding routes formed by the kernel bump technique do differ, with result sensitivity depending on the concept being steered. However, the ordering through the route does not depend on the semantic geometry (Props. 12–13). We have also acknowledged at the end of §6 and in the Limitations of §8 that the correct metric for the encoded semantic geometry remains unclear.
We emphasize, however, that specifying a route is decoupled from solving the optimization to realize it. For any semantic geometry, projection steering meets every reachable target on the route exactly (Prop. 7) and, among states meeting it, minimizes off-target change (Thm. 10).
Q5. For the taxonomy experiment, is there evidence that steering the layer-16 probe distribution changes the model's downstream token distribution or generated behavior in the intended taxonomic direction? Given the probe's 69.9% held-out accuracy and the paper's explicit caveat that this experiment does not test downstream behavior, an independent downstream check would materially change how strongly I interpret this result.
A5: Our current work focuses on the theoretical groundings for provably exact control of a softmax readout distribution. Extending this to control of downstream generation would require extending the objective and behavioral constraints from next-token distributions to generation trajectories. We leave this extension to future work and have clarified this scope in the paper.
Q6. What is the wall-clock/runtime and memory cost per projected route position on Gemma-3-4B, and how many inner iterations are typically required, compared with activation steering and dual steering? Since Appendix B uses repeated optimization over a large vocabulary and Appendix C.3 projects each route position independently, this information is important for assessing whether the method is practical beyond offline analysis.
A6: We have added an asymptotic cost comparison of projection steering, ActAdd, and dual steering to App. B.3 (Table 3). Each Newton step of projection steering costs , where is the vocabulary size, is the top- tokens used to form the covariance matrix, and is the rank of the edit directions . Note that this is linear in . Per route position, projection steering costs , where and are the numbers of bisection and Newton iterations. This is more per position than ActAdd () or dual steering (), but still affordable for offline steering and for online steering at constrained positions, where each steered token adds a few milliseconds on Gemma-3-4B. Memory cost is , about 70 MB for months on Gemma-3-4B, roughly 1% of the model weights. We do not yet have measured wall-clock times and will report them in the revision.
Replies to Q1-Q3
We thank the reviewer for the detailed feedback.
Due to an outage on our compute cluster, we were unable to run additional experiments during the rebuttal period. We address each question below using the paper's existing theory and results, and we will add the corresponding experiments in the revision.
We first discuss some high-level points.
Our aim is not to offer a new steering method but to build preliminary steps toward a general, formalized framework that can lead to exact control of the model's output behavior and a better understanding of how models represent concepts. We work with next-token distributions over structured concepts because they permit a mathematically rigorous analysis, the tools for which do not exist yet for entire sequences. We agree that steering model behavior is the eventual goal, and an optimization problem faithful to model behavior is a natural extension of this work.
Our central design is optimization under behavioral constraints, with a separation between specifying semantic routes and solving the optimization problem. This separation lets us realize a route of distributions exactly, regardless of how the route is specified. The reviewers are right that steering outcomes depend on the chosen semantic geometry, and that routes are pre-specified rather than discovered; these remain open questions, which we flag in the Limitations. What our work guarantees is that any given route (a family of distributions) is realized up to solver precision. So if you have another method for discovering the route, this procedure can slot this in.
More specific points are addressed below.
Q1. Can the authors provide aggregate quantitative results over all held-out prompts and source-target routes for the month, weekday, age, and taxonomy experiments, and over multiple synthetic seeds, reporting at least target error, route error, off-target error, and constraint-success rate for every baseline? Appendix C.1 says that the appendix figures show the first held-out prompt, so these aggregates would substantially affect whether I view the current qualitative examples as representative.
A1: We agree that aggregates are needed to judge whether the figures are representative. Our evaluation pipeline already computes every metric over all held-out cases. We now report these aggregates for every method and task, including all three synthetic seeds, in Table 3 (App. C.9).
The aggregates confirm that projection steering attains the behaivoral constraints at the route endpoint in every case, with mean in-domain KL below 10−15, mean mass shortfall below 10−9, and zero route error. These values hold because whenever a target is reachable (Prop. 7), the projection meets all behavioral constraints exactly.
Q2. Do the authors have results on another language model family or model size, even for a subset of the structured concepts? If not, would the authors agree to narrow the empirical claim to Gemma-3-4B rather than presenting the observed advantage as evidence of general steering behavior across language models?
A2: Yes, for a subset. An earlier version of our pipeline compared projection steering with contrastive activation addition and dual steering on both Gemma-3-4B and Llama-3.1-8B, over 5,700 conditions. No baseline met all behavioral constraints in any condition on either model. Projection steering met them in every condition on Gemma-3-4B and in 95.8% of months conditions on Llama-3.1-8B (the remaining is reported as infeasible). We will add additional experiments on another model family when we regain access to compute resources.
Theoretically, our method is model-agnostic; that is, the procedure principally works for any language model supplied. Given any model and readout, the theorems guarantee that solving the proposed convex KL projection problem will give the activation edit meeting the constraints.
Q3. How does projection steering behave when Q does not contain the full centered in-domain readout row space? In particular, can the authors report a sensitivity experiment that reduces or perturbs the support dimension and measures the fraction of unreachable targets, target error when exact reachability fails, and behavior near the ρmax boundary? This would test the reachability analysis in a setting not constructed to make all target conditionals feasible.
A3: Reachability depends on Q in two ways (Prop. 7). Omitting part of the centered in-domain row space makes some conditionals unreachable, which the solver reports as
INFEASIBLE (SHAPE). Omitting the mean label row lowers ρmax, which it reports asINFEASIBLE (MASS). Existing runs test the second case. On Gemma-3-4B, over five further domains and 12,762 solves, restricting Q to the centered label rows left only 6.1% of targets reachable. Adding the mean label row restored reachability to 100%.