acceptodds
Under review as a conference paper at ICLR 2027

DOES THE MODEL USE THE FEATURE? SEPARATING STEERING FROM MECHANISM IN LLMS

Abstract

Internal features in LLMs are often interpreted as mechanisms when they track a concept and their manipulation changes a related behavior. Yet steering can push a feature far outside its natural range, where its effects need not reflect the model's own computation. We examine this inference and propose an empirical contract whose tests evaluate features at values observed on natural inputs. One test copies a feature's value from an input that shows a behavior into a matched input that does not (installation) or the reverse (removal); the other restores the feature after an upstream edit (downstream rescue). Installation measures how far the feature suffices for the behavior; removal and downstream rescue measure how much the model uses it. Applied to three kinds of representations, the two strengths separate sharply. The published unknown-entity latent strongly steers knowledge abstention, yet installing observed values from either published latent into matched prompts transfers only a small fraction of the natural known–unknown abstention contrast. Dense known–unknown directions show opposite asymmetries between installation and removal in Gemma and Llama, and how fully a released subject–verb agreement feature set reproduces and restores the behavior depends on how its values are written into the model. Tracking a concept and steering a behavior therefore do not by themselves show that the model uses a feature, and each conclusion holds only for the intervention tested.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mechanistic Interpretability, Feature Steering, Sparse Autoencoders, Causal Interventions, Representation Sufficiency, Natural Computation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JWwkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
  • The authors propose an "empirical contract" to account for a misconception that a feature tracking a concept doesn't show that the model uses that feature. They argue that because steering a behavior drives the model so far off distribution, we need to test sufficiency and use at values observed on natural inputs.
  • They contribute an audit showing that Gemma scope latents steer out of range and under natural contrast are much less effective
Critical strengths

They contribute an "empirical contract" that can be used to validate the extent to which a model actually uses a feature when we steer that feature. The audit of Gemma scope validates their theoretical contribution in a practical setting.

Critical weaknesses
  • The evidence is insufficient given the generality of the claim. For example, there are only two published latents and one dense direction per model.
  • Contribution is somewhat minimal given the existing large body of work on interchange interventions and denoising activation patching. The main contribution is framing of existing methods, so the primary takeaway is the result of the empirical audit, which (per the above) is limited in scope.
Questions for authors
  • To what extent do you think the low results are because you only edit a single token and layer position? (The refusal control shows that full ablation is much more effective than the single-site version, perhaps the same holds for the entity latents?)
  • Which update rule should people use? Why?
2 comments

Thank you for the review. The paper's main results are these. A published SAE latent whose steering raises knowledge abstention by 46 points carries little of that behavior at its natural values. At the known-entity latent's own site, the full residual carries an accuracy gain that the latent does not. The same released feature set gives different answers under two update rules. The review's concerns about the evidence are coverage (W1) and single-site edits (Q1); both are answered below with data. The abstract now reports the dose and alias results and the introduction leads with them, the alias control has its own section (5.3), and Table 3 and Figure 3 report installation, removal and rescue separately.

Evidence versus generality (W1). The audit covers three kinds of representations in three models: two published Gemma latents, known–unknown directions in Gemma and Llama, and a released 147-feature agreement set. It finds small natural-value effects for the published latents, opposite installation–removal asymmetries for the dense directions (the Llama direction installs 0.89 and removes 0.07; the Gemma direction removes 0.61 and installs 0.27), and strong installation, removal and rescue for the agreement set under the decoder update (0.95, 0.95 and 0.88). The tests also detect substantial effects where the latents show little: in the same Gemma model and task, at the latents' own sites, the dense direction removes 0.61 of the natural contrast, the full residual recovers +10.6 of the +11.2-point alias gain, and the same single-token edit at a steering-range value raises abstention by +12.0 points. For the specific claim that steering success does not establish use at natural values, one published case is sufficient, and the latents provide it.

Framing of existing methods (W2). Our contribution combines an empirical contract for feature claims with its application to published representations. The contract separates association and control from sufficiency and use and measures the latter two at natural values; steering-based validation stops at control. Applied to a published latent that steers strongly, it shows that the latent carries little of the behavior at its natural values. Four comparisons make the audit interpretable. Two hold everything else fixed: a same-operator dose comparison (+1.5 versus +12.0 points) and a same-site full-residual control (+10.6 versus −0.2 points). Two vary how the edit is made: the update rule (for the agreement set, installation 0.95 versus 0.80 and rescue 0.88 versus 0.56) and the edited tokens (adding the question mark changes the interchange fractions by at most 0.03; removing the refusal direction at one token lowers refusal by 6 points, against 45 for ablation at every layer and position). Interchange and denoising patching are standard tools; Section 2 says so and positions the work relative to Geiger et al. (2025) and Makelov et al. (2025).

Single token and layer (Q1). In a separate experiment, adding the question mark to the entity token in the natural-value interchange changes each installation and removal fraction by at most 0.03, one flipped outcome (Section 5.5, Appendix C). At the known-entity latent's own site, copying the full residual recovers the alias accuracy gain (+10.6 of +11.2 points) while copying the latent gives −0.2 (Section 5.3). Each latent is tested at its published layer and token (Table 2); editing its direction at other layers would be a different intervention from the natural-value tests reported here. The latents are naturally silent at the end-of-instruction tokens that published steering also edits: zero activation at all five of them on every prompt of the 148 pairs, and on the 400 steering questions only the first newline is active, on 2% of them (Section 5.5, Appendix C). Published steering yields +46 points against +12 for the same value at the entity token alone, but the two also differ in write rule and question set (Section 5.2, Appendix B.1); the paper treats the edited tokens and the update rule as part of the claim (Section 5.5). The refusal gap you cite is on the removal side; the same direction at the same single site installs +29 points when set to the natural class mean (Section 5.2), so a single-site natural-value edit is not inherently weak. The controls show that the tested sites can support substantial behavioral effects: the same single-token edit to the unknown-entity latent moves abstention when the value is in the steering range (Section 5.2), and the Llama direction installs 0.89 of the natural contrast at one token (Section 5.4).

Also, we explained Which update rule (Q2). Test a published SAE claim with writes along the decoder columns under which the claim was made, the direction that steering also uses, with gain correction for individual latents, and report the attained values. Set a dense direction by projection replacement, which keeps the write along the direction: its orthogonal-to-total norm ratio is 0.5% (Llama) and 1.1% (Gemma), against 75.1% and 72.9% for writes through SAE decoders (Appendix C). A closer match to the requested feature values is not by itself a reason to prefer an update. For the agreement set, the encoder-constrained update matches them more closely yet installs 0.80 against 0.95 and rescues 0.56 against 0.88 (Section 5.5); downstream layers read the residual stream, not the encoder's output (Section 6). When the two updates disagree in this way, the disagreement is part of the result: the interpretation holds for the writes under which it was found.

We have made substantial revisions to improve readability, bringing the main findings forward in the abstract and introduction, giving the alias control its own section, and presenting installation, removal and rescue separately. We also added experiments extending the edited tokens and measuring rescue for both published Gemma latents to address concerns about the experimental evidence. These additions strengthen the empirical case for our contribution, while the revised presentation makes the comparisons easier to follow. We respectfully ask you to consider these improvements and the additional evidence when reassessing the paper, and would appreciate your considering an upward revision of the rating.

Reviewer_az6HOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper points out a distinction between whether an LLM is sensitive to steering along a latent feature or whether it naturally uses that latent feature in its mechanism. The distinction lies in whether the activation value is within the naturally occurring range. The paper proposes 3 tests using natural activation values to study whether LLMs use a mechanism: installation, removal, and rescue. Using these tests, the paper investigates whether LLMs use two published feature latents: unknown-known entity/abstention and subject-verb agreement. The paper finds that LLMs often don't change their behavior when being steered within their natural activation range. However, it is possible to find an unknown-known entity latent that changes LLMs' behavior within the natural activation range, and thus is more likely to be the mechanism the LLMs naturally use. Experiments also show that the strength of installation, removal, and rescue can vary depending on the LLM, which part of the LLM is intervened on, and how many layers are intervened on.

Critical strengths
  1. The authors highlight an important distinction between sensitivity to steering vs natural mechanism used by LLMs, and propose a variety of interventions within natural activation range to evaluate whether the LLM actually uses the feature latent.
  2. The diverse experiments demonstrate how different components of the evaluation pipeline can all affect the mechanism usage assessment, including model families, model components, layers, and number of layers. Later discussion in Section 6 provides preliminary explanations of how these effects happen.
Critical weaknesses
  1. It is not entirely clear why installation, removal, and rescue are the exhaustive list of interventions needed to establish natural mechanism usage in LLMs.
  2. The paper claims that using natural-range intervention distinguishes steering from natural mechanism usage, but the method section later scopes the paper’s method to not be a general guarantee of mechanistic identification. This makes the practical value of this paper questionable.
Questions for authors
  1. While installation and removal have straightforward intuition and symmetrical mathematical formulas, it is not clear why rescue is necessary to establish natural mechanism usage. Could the authors provide an intuitive example of how installation, removal, and rescue could prove natural mechanism usage, perhaps with a graphical representation where multiple features work together to influence a behavior? Why is installation distinguished from removal and rescue as “sufficiency” instead of “use”? Why don’t we also want to test the mathematical symmetry of rescue, where an upstream edit adds a feature latent before a downstream edit removes it again?
  2. In Line 262, if evaluating a natural mechanism requires producing naturally realizable joint residual states, does that mean that, besides constraining the feature activation to its natural range, the inputs being intervened on should also match each other in all places but the feature being tested? This is how all three interventional tests in this paper (installation, removal, rescue) are designed, but the requirement of matching pairs of inputs is not specified as important for establishing natural mechanism usage.
  3. In Sections 5.1 and 5.2, what is the difference between a high target and a steering experiment? Should they not have the same activation value and thus the same abstention change?
  4. In Section 5.3, why was rescue not run on Gemma latents? Could the authors provide the complete results?
  5. In Line 351, could the authors explain how the layers in Gemma and Llama are selected?
Reviewer_bsxBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a feature that supports steering is also used by the model during ordinary inference. The authors test published features by setting their activations to naturally observed values. Their protocol also checks whether restoring a feature can recover behavior disrupted by an upstream intervention. For individual entity-recognition SAE latents, the effects at the tested natural values are weak, even though much larger interventions successfully steer the model.

Critical strengths

The same-entity alias experiment is a useful control. Full residual patching recovers most of the accuracy gap between common and rare names, while the individual latent has little effect. This shows that relevant information is available at the tested site.

The dose comparison uses the same intervention setup at both target values. This allows the authors to isolate the effect of activation magnitude.

Critical weaknesses

I do not see what the proposed contract adds to existing causal evaluations. Geiger et al. (2025) explicitly distinguish successful steering from explaining the model’s computation. Makelov et al. (2025) test SAE dictionaries for sufficiency and necessity and use natural counterfactual feature edits. Both works are cited, but the paper does not compare their interventions with the proposed protocol.

Why is rescue alone sufficient evidence of use? An upstream edit may activate a pathway that is inactive in the original run. Restoring the feature could then recover the output, while removing it from the original run has no effect. Taking the maximum of removal and rescue would still report strong use in this case. The authors acknowledge that natural feature values may produce an unnatural joint residual state, but this caveat does not rule out the problem.

Questions for authors

How would the rescue test distinguish a feature used in the original computation from one that only becomes relevant after the upstream edit?

In Section 6 you suggest that downstream compensation may explain the weak removal effect in Llama. Do you have evidence for this hypothesis in your experiments?

2 comments
AuthorsOpenReview ↗

Follow-up on the revised manuscript

Thank you again for your review. The revised manuscript is uploaded, with changes marked in blue. Below we summarize the experiments added during the discussion and how the revision addresses your concerns.

New experiments. Two experiments were added during the discussion, each in a separate run.

ExperimentResultIn the revision
Downstream rescue for both published Gemma latentsThe upstream edit removes 92% of the unknown-entity latent's known–unknown activation difference at the entity token, and 26% of the known-entity latent's. Restoring either latent reverses 0.04 [0.00, 0.14] of the edit's effect on knowledge abstention.Section 5.4, Table 3, Appendix C
Natural-value interchange extended from the entity token to the question markEach installation and removal fraction changes by at most 0.03.Section 5.5, Appendix C

Installation, removal and rescue are now measured for both published latents. All three are small (installation at most 0.12 and removal at most 0.20 of the natural contrast, rescue 0.04), although steering the unknown-entity latent raises knowledge abstention by 46 points.

Rescue (W2, Q1). Our main findings do not depend on treating rescue alone as evidence of natural use. Where the paper reports substantial dependence, it comes from removal in the unedited run: 0.95 for the agreement feature set and 0.61 for the Gemma direction. You are right that taking the larger of removal and rescue could report use in the scenario you describe, so the revision no longer defines a use strength and reports installation, removal and rescue separately. By itself, rescue cannot separate a feature used in the original computation from one that matters only after the upstream edit. Section 4.1 now states that rescue alone does not establish dependence in the unedited run; removal tests that dependence.

Compensation (Q2). Our experiments do not test downstream compensation, and we removed that suggestion from Section 6.

What the contract adds (W1). Our contribution combines an empirical contract for feature claims with its application to published representations. Geiger et al. (2025) draw the distinction between steering and explaining a computation in principle, and Makelov et al. (2025) evaluate SAE dictionaries on indirect-object identification. Neither applies such tests to a published steering-based interpretation; Section 2 now states this relation. The contract separates association and control from sufficiency and use, and measures the latter two at values observed on natural inputs, which steering-based validation does not do. In the dose comparison you highlighted, the same single-token edit changes abstention by 1.5 points at a natural value and by 12.0 at the value steering reaches. In the alias comparison, the full residual at the known-entity latent's site recovers 10.6 points of the 11.2-point gain, and the latent −0.2. The agreement comparison shows that the update rule matters as well: the encoder-constrained update matches the requested feature values more closely, yet it installs 0.80 and rescues 0.56, against 0.95 and 0.88 for the decoder update.

We have also revised the abstract, introduction and results for readability. If the revision resolves your concerns, we would be grateful if you would reconsider your rating, and we are glad to discuss any that remain.

Dear Reviewer,

As the discussion period closes soon, we would like to check whether our replies above resolve the concerns in your review. The status of each point is as follows.

  • Rescue alone (W2, Q1). The revision adopts your point. It no longer defines a use strength, it reports installation, removal and rescue separately, and Section 4.1 states that rescue alone does not establish dependence in the unedited run. The dependence we report comes from removal in the unedited run: 0.95 for the agreement feature set and 0.61 for the Gemma direction.
  • Compensation (Q2). Our experiments do not test it, and the suggestion is removed from Section 6.
  • What the contract adds (W1). Your review notes that the paper does not compare the interventions of Geiger et al. (2025) and Makelov et al. (2025) with ours, so we add that comparison here. Geiger et al. draw the distinction between steering and explaining a computation in principle. Makelov et al. evaluate SAE dictionaries on indirect-object identification, through reconstruction sufficiency and necessity, sparse counterfactual edits and patching of interpreted features. We apply such tests to a different object, published steering-based interpretations: we set the features to the values they take on natural inputs and measure how much of the associated behavior those values reproduce. Three comparisons then change one factor each: the activation magnitude under the same edit (dose), a single latent against the full residual at the same site (alias), and the update rule for the same feature set (agreement). Section 2 of the revision states this relation.

If these points resolve your concerns, we would appreciate it if you could update your review. If any remain, we will answer them before the discussion closes.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of rebuttal responses and additional experiments

We thank the reviewers and area chairs for the discussion. The revised paper strengthens the evidence for our central finding: published steering raises knowledge abstention by 46 points, yet the two published latents show small installation, removal and rescue effects at natural values. In direct response to the reviews, we added downstream rescue for both Gemma latents and extended natural-value interchange to the question mark. These experiments complement the dose, alias and update-rule comparisons already in the submission.

Reviewer bsxB: contribution beyond existing evaluations, interpretation of rescue, and compensation.

Section 2 now states the relation to Geiger et al. and Makelov et al. directly. Neither applies such tests to a published steering-based interpretation. Our contribution combines an empirical contract for feature claims with its application to published representations. The contract separates association and control from sufficiency and use, and measures installation, removal and rescue separately at values observed on natural inputs. The dose and alias controls show what the original steering evidence leaves unresolved: changing activation magnitude at a fixed site changes the behavioral effect, and transferring the full residual at the latent's site recovers behavior that the individual latent does not.

Our main empirical findings do not depend on treating rescue alone as evidence of natural use: where the paper reports substantial dependence, it comes from removal (0.95 for the agreement set, 0.61 for the Gemma direction), and the only representation whose rescue exceeds its removal is the Llama direction (0.26 versus 0.07). The reviewer is right that a maximum of the two could overstate use, so we removed the maximum-based "use strength." Section 4.1 and Table 3 now report installation, removal and rescue separately. Rescue measures recovery after an upstream intervention; it does not alone establish dependence in the original run. Every measured effect and the installation–removal asymmetries are unchanged. We also removed the compensation explanation, for which our experiments provide no direct evidence.

Reviewer az6H: missing Gemma rescue results, the roles of the three tests, and experimental settings.

We completed downstream rescue experiments for both published Gemma SAE latents, in a separate run with the same 148 matched pairs and layer-8 upstream edit as the Gemma direction experiment. The upstream edit removes 92% [79%, 108%] of the unknown-entity latent's known–unknown activation difference and 26% [22%, 31%] of the known-entity latent's. Restoring either latent separately reverses only 0.04 [0.00, 0.14] of the edit's effect on knowledge abstention. These results, now in Section 5.4, Table 3 and Appendix C, complete the installation, removal and rescue measurements for the two latents.

We clarified that the tests address different claims: installation measures sufficiency, removal measures dependence under replacement, and rescue measures recovery after an upstream edit. Section 4 states that a weak result limits evidence for the corresponding claim without ruling out other causal roles. Our follow-up response illustrates these distinctions with interacting features and acknowledges reverse rescue as a valid additional test that we have not run.

We also clarified the matched-input construction, the difference between single-token absolute setting and multi-token additive steering, and the distinction between published feature sites and upstream layers selected on development data.

Reviewer JWwk: experimental coverage, single-site interventions, and the choice of update rule.

On coverage, the audit spans three kinds of representations in three models: two published SAE latents, known–unknown directions in Gemma and Llama, and a released 147-feature agreement set. It finds small natural-value effects for the published latents, opposite installation–removal asymmetries for the dense directions (the Llama direction installs 0.89 and removes 0.07; the Gemma direction removes 0.61 and installs 0.27), and strong installation, removal and rescue for the agreement set under the decoder update (0.95, 0.95 and 0.88). These positive cases also show that the tests can detect substantial behavioral effects.

We added an experiment, in a separate run, extending natural-value interchange from the entity token to include the question mark. Each installation and removal fraction changes by at most 0.03 (Section 5.5, Appendix C). This measures the effect of that token extension directly.

The existing controls also show that the tested sites can affect behavior. At the known-entity latent's site, full residual transfer recovers 10.6 points of the 11.2-point alias accuracy gain, while transferring the latent gives −0.2 points. With the unknown-entity latent's site, token and update rule fixed, the steering-range target raises abstention by 12.0 points, compared with 1.5 points at the natural-range target.

For update-rule selection, we recommend writing an SAE feature along its decoder column, the direction that steering also uses, with gain correction for individual latents; setting a dense direction by projection replacement; and reporting achieved values alongside behavior. The agreement comparison shows why attainment alone is insufficient: the encoder-constrained update matches requested values more accurately, yet it installs 0.80 and rescues 0.56, against 0.95 and 0.88 for the decoder update. This supports reporting and comparing the update rules rather than selecting one solely by feature-value accuracy.

The abstract now reports the dose and alias results, the introduction leads with them, the alias control has its own section, and the main table reports the three measurements separately.

In light of this additional evidence and the revisions, we respectfully ask the reviewers to reconsider their ratings and the area chairs to take the strengthened experimental case into account in their final review.

Thank you for the reply. We ran the two checks your comment calls for, each in a separate run on the same 148 pairs.

Removal at every layer and position (Q1). We applied the operator of Table 4, directional ablation at every layer and position including the generated tokens, to both inputs of each pair. The latents' decoder directions are nearly orthogonal to the known–unknown direction (absolute cosine below 0.09), and ablation does not increase the rate of invalid responses.

Directional ablation at every layer and positionFraction of the natural contrast eliminated
Known–unknown direction (layer 15)1.00
Random latent of the layer-13 SAE0.725
Unknown-entity latent L15/118980.550 [0.345, 0.719]
Known-entity latent L13/7957−0.100 [−0.400, 0.116]

Three results bear on your question. First, the operator is not specific to labeled directions: a random latent of the layer-13 SAE eliminates more of the contrast than either published latent, so a large effect under it does not single out the labeled feature. Second, ablating the known-entity latent's direction eliminates none of the contrast, so its small removal does not come from the single-site scope. Third, ablating the unknown-entity latent's direction eliminates 0.55, against 0.07 for natural-value removal at its site, and exceeds ten random latents of its own SAE. As with the refusal direction in Table 4, removal at one site is small and ablation everywhere is large, so a conclusion about use holds only for the removal operator tested (Section 5.5). The revision reports these results in Section 5.5 and states the small removal result for natural-value removal at the positions where the latents are active.

Rescue. We agree that rescue does not discriminate between these representations in this model, and the conclusions below do not rely on it.

What is unchanged. The results on the latents' natural values do not depend on the removal operator or on rescue. Under the same single-token edit, a natural value changes abstention by 1.5 points [−1.2, 4.2] and the value steering reaches by 12.0 [8.0, 16.0]. Installing either latent's natural value transfers at most 0.12 of the natural contrast. At the known-entity latent's site, the full residual recovers 10.6 of the 11.2-point alias gain, and the latent −0.2. Steering therefore does not show that the latents' natural values suffice for the behavior.

We would be glad to answer further questions before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.