acceptodds
Under review as a conference paper at ICLR 2027

Projection-Transport Steering: Conditional Control of Reasoning Behavior in LLMs

Abstract

Activation steering offers a lightweight way to control language-model behavior at inference time without retraining. Yet most methods apply the same edit to every input, so an edit strong enough to remove a behavior also damages inputs that never showed it. We introduce Projection-Transport Steering (PTS), which separates when to intervene from how activations should move: a likelihood-ratio test decides which inputs to edit, and the edit moves only their projection onto a low-dimensional behavioral subspace. We show that optimal transport gives the minimum-displacement edit under this locality constraint, that additive steering is optimal exactly when the target is a shifted copy of the source, and that the selectivity of any gated edit factors exactly into a decision term and an action term. The factorization predicts gated results from one ungated run and bounds a decision from unsteered answers alone. On held-out MMLU questions for models we measured, PTS is the most selective of eight methods at removing overconfident errors while keeping confident-right answers, and it lowers calibration error from 0.35 to 0.26 at unchanged accuracy. It scales it's effects on selectivity across diffrent model families and a diffrent benchmarks, and on toxic continuation in three model families it matches a tuned per-token gate while leaving every unflagged continuation unchanged. Together, these results provide a principled basis for selective activation steering: detect when control is warranted, then apply the smallest behaviorally targeted intervention.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, representation engineering, inference-time control, optimal transport, model calibration, mechanistic interpretability, large language models, truthfulness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ShQ4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Projection-Transport Steering (PTS), a training-free activation steering method that separates the decision of when to intervene from the action of how to modify model activations. It uses a likelihood-ratio test to identify inputs requiring intervention and an optimal transport map to modify activations within a low-dimensional behavioral subspace. The paper also provides a theoretical framework for analyzing steering selectivity. Results suggest that PTS can improve selective behavioral control while preserving task accuracy or retaining more desirable outputs than several baseline methods.

Critical strengths

The paper addresses an important problem: language models can express high confidence in incorrect answers, which undermines their reliability in practical applications. The proposed separation of intervention timing from the activation-editing mechanism is an interesting approach, and the theoretical analysis provides a useful framework for understanding selective interventions.

Critical weaknesses
  1. The paper defines confidence as P(YES)P(\mathrm{YES}) when the model is asked whether its previous answer is correct. However, this measure could capture the model’s tendency to affirm its own previous answer rather than its actual confidence in that answer. For example, a model might answer "yes" because it tends to defend its previous response, even when it is uncertain. The paper also uses P(YES)≥0.5P(\mathrm{YES}) \geq 0.5 to classify an answer as confident, but this threshold may not reliably distinguish confident from uncertain responses. The authors should provide stronger validation of this proxy and justify the choice of threshold.

  2. The paper applies steering to both the initial answer generation and the subsequent YES/NO verification turn. This makes it difficult to determine whether the method genuinely improves confidence calibration or simply changes the model’s tendency to answer "yes." For example, steering might reduce P(YES)P(\mathrm{YES}) for an incorrect answer without changing the model’s underlying assessment of whether that answer is correct. To address this concern, the authors should include controls that measure confidence without steering the verification turn, or compare the effects of steering on YES/NO responses independently of its effects on the initial answer.

  3. The proposed method primarily reduces confidence in incorrect answers, but does not necessarily correct the underlying answers. For example, if the model gives an incorrect answer with P(YES)=0.9P(\mathrm{YES})=0.9, steering reduces this probability to 0.30.3 while leaving the original answer unchanged. This could improve calibration-related metrics, but it does not make the answer itself more accurate. The practical value of the method therefore depends on whether reduced confidence leads to useful downstream behavior.

Questions for authors
  1. How do the authors validate P(YES)P(\mathrm{YES}) as a reliable measure of model confidence, and why is 0.5 an appropriate threshold for classifying an answer as confident?

  2. How do the authors disentangle genuine improvements in confidence calibration from changes in the model’s tendency to answer "yes," given that steering is applied to both the initial answer and the verification turn?

  3. Does reducing confidence in incorrect answers improve practical reliability, even when the answers themselves remain unchanged?

  4. How sensitive are the reported results to the choice of confidence threshold and the operational definition of overconfident-wrong answers?

  5. Minor issue: The table caption should be placed above the table, not below it.

1 comment
AuthorsOpenReview ↗

Response to Reviewer ShQ4

We ran the control you suggested. On Gemma, we held the original answer fixed and steered only the YES/NO verification turn. Selectivity was 0.274, close to the 0.294 we obtained by steering both turns. Steering only the answer gave 0.159. Qwen behaved quite differently. Verification-only steering gave 0.006, while steering the answer alone gave 0.172, the same point estimate as steering both turns. Section 5 now puts these results side by side. Your concern about changing the tendency to say YES explains much of the Gemma result.

We use self-affirmation for the probability the model assigns to YES after normalizing over YES and NO. At 0.5 the model favors neither response. We checked this score against answer correctness and repeated the check with two other readouts. On Gemma MMLU, Brier score improved from 0.372 to 0.294 for self-affirmation, from 0.390 to 0.336 for answer-letter probabilities, and from 0.235 to 0.226 for option verification. All three readouts became better predictors of correctness on Gemma. What the model privately believes remains open. Across cutoffs 0.4, 0.5 and 0.6, PTS had the highest of eight selectivity estimates on Gemma MMLU and prompting led on Qwen. The Gemma comparison against the trace condition had adjusted p = 0.064 at 0.4 and 0.5 in the expanded 21-test family (Appendix P).

In your example, lowering confidence from 0.9 to 0.3 changes the decision if the system can abstain. We evaluated a rule that earns 1 for a correct answer, loses 2 for a wrong one and earns 0 for abstaining. Its acceptance cutoff is 2/3. Across 3,000 Gemma MMLU questions, steering corrected 177 answers and introduced 169 errors. With just eight net corrections, mean utility rose from −0.170 to 0.064. Platt calibration of the original scores did better on this utility (0.087). Steering gave the better risk ranking, lowering area under the risk–coverage curve from 0.305 to 0.280. On Qwen, utility fell from 0.303 to 0.296.

We also tested an outcome that requires the model to give a second answer, without a YES/NO verifier. After a user challenged its first answer, the gated edit preserved 1,142 of 1,242 useful corrections (91.9%). Always keeping the first answer on the very same flagged questions preserved 569 (45.8%). Keeping the first answer prevented more harmful reversals and finished with more correct answers overall. The gated edit, however, preserved 573 more opportunities for the model to accept a valid correction. That difference is visible in the answers themselves (Appendix S.1). We also moved the table captions above the tables as requested.

Reviewer_rHnROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies conditional activation steering, where an edit to the residual stream is applied only to inputs that exhibit a target behavior. It proposes Projection-Transport Steering (PTS), which pairs two components. The first is a decision, a likelihood-ratio test on a learned probe statistic. The second is an action, a transfer function on a low-dimensional behavioral projection.

The theory has four parts:

Proposition 1: for any gated edit that leaves unflagged outputs unchanged, selectivity equals TPR·ρ_O − FPR·ρ_C. Theorem 1: the minimum-displacement edit that maps the projection to a chosen target law is the Brenier optimal-transport map. Theorem 2: a Neyman–Pearson optimal decision, plus a ceiling TPR − FPR ≤ 2AUC − 1 readable from unsteered answers. Reference arms: no-edit, random-direction and readout-override controls that share the same decision.

The experiments cover three settings:

Overconfident answers on MMLU (Gemma-2-2B): PTS reaches selectivity 0.294 against 0.248 for the closest baseline (a CAST-style condition). Calibration error falls from 0.354 to 0.255 at unchanged accuracy. Qwen2.5-7B and ARC: replication with mixed outcomes. Free-text toxicity (three model families), and a PCHI-style setting: PTS beats refusal on the same decision and is level with a tuned DSAS-style token gate.

Critical strengths
  1. An unusually rigorous, transparent evaluation protocol. Configurations and detector hashes were committed before held-out rollouts. The splits are disjoint, the tests are Holm-adjusted paired bootstraps, per-question records are released, and post-hoc analyses are labeled exploratory. The reference arms (no-edit, random direction at the same dose, readout override) attribute gains to the decision vs. the edit. On Gemma-MMLU, no-edit gives 0.049 and random directions give 0.131, against 0.294 for PTS. This control design would improve evaluations of any conditional steering method, not just PTS.

  2. A useful way to reason about conditional steering before running it. Separating the decision from the action, and bounding what a threshold decision can contribute from unsteered answers alone (Theorem 2b), gives practitioners a cheap way to see whether a gate can help. The decomposition matches real gated runs within 0.025 (Table 8). The authors also disclose that the prediction is biased upward (11 of 12 rows).

Critical weaknesses
  1. The results do not show that the proposed action adds value, and a trivial override often matches or beats PTS. The matched-override reference in Table 14 is more selective than post-hoc PTS on Gemma-MMLU (0.319 vs. 0.294; difference −0.025 [−0.04, −0.01]). On Qwen it is far more selective (0.391 vs. 0.172; −0.218 [−0.26, −0.17]). In the PCHI setting the override ties PTS exactly (0.438), and 45 of 100 random directions match its selectivity. On toxicity, the one setting where no readout exists, PTS is not separable from a tuned DSAS-style gate (Table 5: −0.048 [−0.10, 0.00], +0.008, +0.087 [−0.03, 0.21]). The “keeps 80–85% of clean text vs. 67%” claim compares different operating points, since PTS removes 70–81% of toxic text against 92–93% for the gate. “Most selective of eight methods” holds only in the primary Gemma-MMLU setting. On Qwen, prompting has the higher point estimate (0.212 vs. 0.172), and on ARC, CAA and the CAST-style condition do (0.294 and 0.289 vs. 0.235), though the differences are not significant. The Limitations section concedes that an override is at least as selective when a single readout exists. The result is mostly a probe-based decision combined with a simple shift, with the decision doing most of the work.

  2. The theory is thin relative to its framing and only loosely tied to the main results. Proposition 1 is a total-probability identity that holds only if unflagged outputs are exactly unchanged, so “exact account” is true by construction. Its predictive correlations (0.97 and 0.91) come from tuning-split configurations scored by composition from the same ungated runs. Theorem 1 gives the optimal map to a chosen target law but not which law to choose. The best arms in practice are tuned shifts or ablations selected empirically. The Theorem 1 map itself is not the winner. On the Gemma toxicity tuning split it reaches 0.373 against 0.594 for the tuned shift, and on fresh prompts it ties an equal-displacement shift (0.538 vs. 0.535). Theorem 2’s Neyman–Pearson optimality assumes conversion rates independent of the statistic, and Appendix D reports that this assumption does not hold in the data.

Questions for authors
  1. Why does the readout override beat PTS by so much on Qwen-MMLU (0.391 vs. 0.172), and is there any single-readout setting where the edit outperforms the override? What is the strongest argument for an activation edit over overriding or recalibrating the stated confidence of flagged answers?

  2. Can you compare PTS and the DSAS-style gate on full toxic-removal vs. clean-kept frontiers, or at matched toxic removal, rather than at the two selected operating points? Does the ranking hold under a second toxicity classifier and under longer or sampled continuations?

  3. Is there a setting where the Theorem 1 transport map beats the best shift or ablation at matched cost? How should a practitioner choose the target law ν without a labeled tuning split?

  4. How much do the conclusions depend on the probe? The held-out AUROC is only about 0.74 to 0.79 for the PTS probe. How do the rankings change with a stronger or weaker detector, or with larger models?

  5. How sensitive are the rankings to the P(YES) ≥ 0.5 definition and to the M5 readout? You show 0.4 and 0.6 thresholds only against the CAST-style condition.


Some suggestions for future work:

Applying the reference-arm methodology to other conditional steering methods (CAST, DSAS) as a standalone contribution would strengthen the evaluation story.

1 comment
AuthorsOpenReview ↗

Response to Reviewer rHnR

The Qwen override result has a fairly direct explanation. The gate flags 86.9% of confident errors and 47.8% of confident correct answers. Lowering the reported confidence of every flagged answer gives selectivity 0.391, exactly the difference between those rates. With the activation edit, 24.5% of initially confident errors leave that state and 7.3% of initially confident correct answers leave theirs, giving 0.172. The override also leads on Gemma MMLU (0.319 versus 0.294) and ties on PCHI (0.438). On ARC, the prompt-gated edit does beat its matched override, 0.182 versus 0.169, though the difference is small (0.013, nominal paired interval [0.01, 0.02]; Appendix G).

Where the application needs different text, we can test the value of editing directly. On 5,000 higher-toxicity prompts, the policy using our input gate and a DSAS action converted 11.89–21.05 percentage points more initially toxic outputs to clean, prompt-appropriate text than CAST did. All four automatic classifier and relevance-judge pairs gave positive adjusted bounds. The judges accepted no conversions from a control that returned a fixed benign reply. CAST retained more originally clean text (Appendix R.3).

Your objection to the original 80–85% versus 67% comparison was warranted. Appendix R.1 now shows all 122 configurations per model. On Gemma, PTS and CAST removed almost the same fraction of toxic outputs (80.7% and 79.7%), while PTS retained 73.4% of clean outputs against CAST’s 35.0% at cosine 0.7. For DSAS, the closest match was Mistral under toxic-BERT. Both methods removed 14 of 18 toxic outputs, and PTS retained 73.1% of clean outputs against 63.6%. With only 18 toxic cases, the removal estimate is imprecise. Eleven of 84 comparisons met the full matching and retention criterion. At cosine 0.7 the retention gain over DSAS held in all 24 longer or sampled decoding panels. At 0.5, DSAS had higher selectivity in 15.

For transport, we fit the target distribution to projections of labeled clean examples. That fixes the map before we evaluate behavior. On Gemma, the map beat a constant shift calibrated on development data by 12.43 and 14.28 points on conversion plus retention under the two classifiers. We can match mean edit energy exactly in expectation by occasionally leaving a prompt unedited under the shift policy. Transport still leads by 13.05 and 15.15 points on the original cohort, and 9.23 and 11.56 on separate documents. Matching clean retention asks a different question. Against a mixture of five shift doses and no edit, the map won on Gemma under toxic-BERT and the mixture won on Qwen under RoBERTa. The other four intervals included zero. The measured-energy comparison is specific to Gemma (Appendix R.2, R.4–R.5).

Probe quality mattered with the action held fixed. Degrading the probe signal took the conversion-plus-retention balance from 35.5–44.8% to 2.3–4.3%. Logistic probes beat mean-difference probes on both feature sets across six checkpoints. In the 9B generation test, gating preserved more clean text and removed less toxicity. The theory also covers the case you raised where conversion varies with the probe score. For a fixed budget of losses on protected outputs, the optimal rule weights each class’s score density by its conversion probability. Forecasts made before seeing edited test outputs cut mean absolute error from 0.052 to 0.022, though they did not reliably select better policies. Appendix G applies the same reference controls to both CAST-style conditions. Appendix P compares all seven confidence baselines at 0.4, 0.5 and 0.6 and reports the alternative readouts.

Reviewer_pf3BOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Projection-Transport Steering (PTS), a method for reducing undesirable behaviors in language models while preserving desirable ones. The main idea is to first identify when an intervention is needed and then modify the model's hidden representations using a steering direction. The authors also provide a theoretical analysis of selective steering and use optimal transport to minimize changes in the activation space. The method is evaluated on multiple language models for overconfidence reduction and toxic text generation. The results are promising in some settings, although the improvements are not consistent across all models and datasets.

Critical strengths
  1. The paper addresses an interesting problem of selective activation steering, where undesirable behaviors need to be corrected without affecting desirable ones.

  2. The separation of the decision and action components is useful. The theoretical analysis provides a better understanding of how these two components contribute to the overall steering performance.

  3. I appreciate the experimental controls, particularly the comparisons with random directions, no-edit regeneration, and direct confidence overrides.

  4. The experiments cover multiple models and tasks. The authors also report confidence intervals and statistical significance tests, which is good.

Critical weaknesses
  1. My main concern is the contribution of optimal transport. Most of the experiments use simple shifts or directional ablation, which are already established steering techniques. In the free-text experiments, the actual transport map achieves a selectivity of 0.538 compared to 0.535 for a displacement-matched shift. The authors also acknowledge this observation. This raises the question of what practical advantage the optimal transport formulation provides over existing methods.

  2. The theoretical novelty seems somewhat limited. The selectivity decomposition follows from conditional probability, while the optimal transport result builds on existing theory. I agree that bringing these ideas together for selective steering is useful, but I am not entirely convinced that the theoretical contribution is sufficiently novel. The paper could better explain what new insights these results offer beyond existing conditional steering methods such as CAST and DSAS.

  3. The performance improvements are not consistent across different models and datasets. For example, on Qwen-MMLU, prompting achieves a selectivity of 0.212 compared to 0.172 for PTS. Similarly, on ARC, additive CAA achieves 0.294 compared to 0.235 for PTS. Although these differences are not statistically significant, they suggest that the proposed method does not consistently outperform existing approaches. Some discussion on why its effectiveness varies across models would be helpful.

  4. Another concern is the contribution of the steering direction itself. On Qwen-MMLU, PTS achieves a selectivity of 0.172, while random directions achieve 0.158 on average, with no statistically significant difference. In the PCHI experiments, 45 out of 100 random directions achieve the same selectivity as the proposed shift. These findings suggest that the detector may be responsible for much of the improvement in some settings. Although the authors discuss this, I think the role of the steering direction needs further investigation.

  5. The free-text experiments are limited to 40-token greedy continuations and rely on a single toxicity classifier. It is unclear whether similar improvements would hold for longer generations or other decoding strategies. The evaluation also uses a fixed sentence-embedding similarity threshold to measure preservation of clean outputs. I think evaluating with different thresholds or including a small human evaluation would make the results more convincing.

  6. The theoretical formulation supports a general k-dimensional projection, but all the experiments use only a single steering direction. While this is sufficient for demonstrating the one-dimensional case, it would be useful to see whether the proposed formulation also works well with multiple directions, particularly for more complex behaviors.

Questions for authors
  1. Can the authors clarify the practical advantage of optimal transport over standard activation steering methods, especially when simple shifts achieve nearly identical results?

  2. The random-direction experiments suggest that much of the improvement may come from the detector. Can the authors further clarify the contribution of the proposed steering direction?

  3. Can the authors explain why PTS does not consistently outperform existing methods across different models and datasets? Also, would the results generalize to longer text generations and different decoding strategies?

1 comment
AuthorsOpenReview ↗

Response to Reviewer pf3B

The map and the constant shift were nearly tied in the 40-token experiment you cite (0.538 versus 0.535). We found a clearer difference when we held the Gemma gate and steering direction fixed and measured how far each edit moved the activations during 128-token generation. On 2,919 prompts, transport converted 54.1% of initially toxic continuations to clean text, versus 39.9% for the shift. It retained 83.6% of initially clean continuations, versus 85.4% for the shift. That is a 14.2-point gain in conversion at a cost of 1.8 points in retention, even though the shift used 2.49% more mean edit energy. On the combined score, transport gained 12.43 points [4.43, 20.54] under RoBERTa and 14.28 [5.45, 23.87] under toxic-BERT. On 4,500 separate documents, the gains were 8.90 and 11.16 points (Appendix R.4–R.5).

We also checked what counted as a successful conversion. Of 394 originally toxic continuations, 156 after transport and 124 after the shift passed both tests: RoBERTa classified the new text as clean, and the Gemma relevance judge accepted it for the original prompt. The shift preserved more of the originally clean text. Counting both outcomes, a retrospective analysis that gave the two cohorts equal weight favored transport by 3.58–5.94 points across four classifier and relevance-judge pairs. Three of the four simultaneous intervals excluded zero. Neither cohort alone cleared its full contextual criterion.

We agree that the decomposition follows from conditional probability. Its empirical value is that we can vary the gate and the action separately. When we applied the same edit to exactly the same number of prompts, the learned gate improved the conversion-plus-retention score by 28.2–38.2 points over random selection across Gemma, Qwen and Mistral. Once we gave the random selector the original class labels and matched the number of edits within each class, it scored 1.5–6.4 points above the learned gate. Directing more edits to the initially toxic class accounted for the gate’s advantage on these prompts. The direction made a separate contribution on Gemma MMLU, where it beat all 100 same-gate random directions, with selectivity 0.294 against their mean of 0.131. On Qwen the difference was inconclusive, and on PCHI 45 of 100 directions tied the learned shift. This is why we limit the eight-method ranking to Gemma MMLU (Sections 5–6).

For the longer generations you asked about, we tested three 128-token sampling seeds and 256-token greedy decoding. At cosine 0.7, PTS preserved more clean text than DSAS in all 24 comparisons across models, classifiers and decoding settings, by 7.37–21.42 points. Some settings traded away toxic removal. We also tested four similarity cutoffs and Gaussian maps of ranks one, two and four. Higher rank reduced clean retention in all 96 comparisons with rank one and reduced selectivity in 61. Extra directions sometimes removed more toxicity, but consistently disturbed more of the text we wanted to keep (Appendix R.1–R.2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.