acceptodds
Under review as a conference paper at ICLR 2027

AJR: Amortized Jacobian Readouts for Prompt-Adaptive Steering

Abstract

Reading a language model's hidden state and predicting what will happen if we change it are different tasks. We introduce the Amortized Jacobian Readout (AJR), which learns from gradients to predict how a hidden-state edit changes the model's preference between two output tokens. Its prompt-dependent response direction both scores proposed edits and supplies a steering edit during the forward pass, without a backward pass at inference. Controlled color-retrieval experiments across four models from three families show why this distinction matters: learned decoders read unmodified states accurately but often mispredict intervention effects, whereas AJR learns their local response. In a fresh 640-prompt evaluation per model, steering with averaged linear and nonlinear AJR directions produces larger target-margin changes than fixed mean-gradient and contrastive-activation directions in every model. With the target answer supplied and edit size fixed from fitting data, it corrects 71–100% of initially wrong answers, while changing at most 2.7% of initially correct answers to wrong ones. Measured two-token generation latency is close to the unsteered model's; local-gradient steering takes 2.1–2.3 times as long. AJR thus turns learned intervention responses into prompt-adaptive control, within the depth and contrast on which it was fitted.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, causal interventions, language models, amortized Jacobian readouts, attribution patching

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_sDiPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes a steering algorithm called Amortized Jacobian Readout (AJR) that is trained on gradients to predict how much an edit will affect the final output and use said prediction to build a steering vector. Unlike some steering techniques that use a fixed steering direction (e.g., CAA) AJR adapts the steering direction to the input. The authors test AJR on a synthetic task on four models. The task consists of sentences like "Charlotte has a red key. William has a blue key. Charlotte has a ___" and AJR is used to steer the answer towards the target choice (which is provided to the method as input). AJR is able to move the answer towards the target color more than CAA (a steering technique that uses a fixed shift) and a fixed mean-gradient direction do, and it fixes 71 to 100% of the wrong answers (while breaking at most 2.7% of the correct ones). The paper also shows that tools that predict the model's output well (e.g. the tuned lens) predict the effect of edits poorly, and that later layers push back against edits in both directions.

Critical strengths
  • As far as I know this is the first steering method that learns to predict the gradient, so it obtains a gradient-like, prompt-specific steering direction without a backward pass at inference.
  • The analysis that separates "reading" a hidden state from "predicting the effect of editing it" is interesting.
Critical weaknesses

I find it difficult to assess the usefulness of this method because: 1 the practical value seems very limited, 2 the paper lacks more solid baselines and 3 the evaluation does not seem completely fair.

Practical value: the proposed algorithm seems very constrained and its value only shown on a toy task that does not seem to transfer.

  • In this task the user needs to provide the “push” towards the correct answer. While this is normal in steering application the correct answer is usually (as an example) to avoid harmfull language, or to be truthful, etc… here the direction is towards a 1 word answer: blue or red.
  • The steering is only applied once at the end of the prompt. In practical applications to language models one does not know usually when the steering needs to be applied so you either always intervene or use a classifier to do so, but neither version is tested for AJR (only the CAA-all baseline edits every position). Being constrained to intervene only at the end (and for a single token) seems a strong constraint.
  • There is no evidence that the abilities shown on this task carry to other tasks. The only evidence is that on WikiText pronouns, AJR loses to the mean gradient in 3 of 4 models and to CAA-all in all 4 (this results should be discussed in the main paper).

Lack of proper baseline comparison

  • Prompting. AJR is given two things: the answer pair (red vs blue, fixed when it is trained) and which of the two to push towards. Since the user needs to provide the right answer "corrects 71–100% of wrong answers" really means "when told the right answer, AJR makes the model say it 71–100% of the time”. Therefore a fair prompting baseline would be something like:
    • “Hint: the answer is red. Charlotte has a red key. William has a blue key. Charlotte has a”
  • There are steering techniques that are more “dynamic” than CAA, for example:
    • ReFT/LoReFT: a learned low-rank edit that is a function of the hidden state;
    • activation-transport methods such as Linear-AcT or DSAS in which each neuron gets its own scale and shift so the edit depends on that neuron’s value
  • The steering evaluation is limited to small models (up to 2B). Typical steering papers use larger models (especially comparing with prompting this might make a difference).

Lack of fair evaluation

  • The way CAA is evaluated is not using CAA full strength.
    • CAA is normally used with a multiplier tuned for the task. Here every method gets the same edit size, taken from AJR's fitting data and not tuned for each method. Such tuning (sweeping value for rank, regularization, checkpoint) seems to be done for ARJ instead.
    • Similarly, the layer was fixed in advance (the middle layer) for all methods and never swept, while CAA is usually sensitive to the choice of layer.
  • There is no evaluation of how the model general abilities (e.g., PPL and MMLU) degrade, which is a standard test in steering.
  • There is no evidence of what would happen to answers that are longer than 1 word.

Not a reason for rejection but the paper is not easy to read and is missing some information.

  • I found the abstract hard to follow: after reading it, I could not understand what practical use case the algorithm targets.
  • In general, sentences are long and with many caveats that make reading difficult.
  • AJR actually uses the average of 2 predictors (ARJ-lin and ARJ-mlp) but there is no ablation that informs how well each of those do in isolation for steering (their steering results
  • AJR steers using the average of two predictors (AJR-lin and AJR-mlp), but no steering results are shown for either one alone. Their prediction accuracy is reported separately, but the authors found AJR-mlp better in every model. So it is unclear if averaging helps.
  • Unless I missed it I don’t think the parameter count, memory and fitting cost for the predictor are not reported. Adding this info would help with reproducibility and with comparing costs against other steering methods.
Questions for authors

The amount of changes is such that I am not sure they can all be addressed in a rebuttal. However, I hope these questions could be used to improve the work regardless of its acceptance at this time.

  • Can you show that AJR works in more realistic and useful settings?
  • Can you add stronger baselines as suggested above?
  • Can you tune all methods to use them at their best configuration? That means tuning their strengths, hyperparameters, layer, etc… on the held out set as it was done for ARJ’s predictor settings?
  • Can you steer and run those tests on bigger models?
  • Does the AJR edit change general ability, for example perplexity on ordinary text or MMLU accuracy, compared with the unedited model?
  • Can you disclose the parameter count, memory and fitting cost of AJR, per layer and per answer pair?
  • Can you provide the steering results of the two ARJ versions in isolation?
  • How would AJR be used when the moment to steer is not known?
6 comments
AuthorsOpenReview ↗

Response to Reviewer sDiP

Thank you for the concrete list of controls. We ran those that the color task allows, and the revised paper reports them: steering is now compared in three settings, and the abstract and Section 3 are rewritten for clarity. Tables and sections refer to the revised PDF; Q1–Q8 number your eight questions in the order listed.

Q3. Tuning every method

We now tune both the multiplier and the layer of every method, in two steps, so that their effects can be separated. Settings are selected on 384 calibration prompts whose entity names are disjoint from the 960 fitting and 640 evaluation prompts. The rule is the same for every method: highest calibration accuracy with at most 2% harm (initially correct answers made wrong), ties going to the higher target probability. Multipliers are {0, .25, .5, 1, 2, 4} times the original edit norm; layers are six depths from .25 to .875.

Answer accuracy (%) on the 640 evaluation prompts.

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit80.3181.0974.2291.88
Supplied-answer hint89.8499.5375.00100.00
AJR-ens, original setting94.2298.7598.1297.50
Step 1: middle layer, multiplier selected
CAA-last89.5385.6275.1692.19
CAA-all88.2886.5675.3191.88
Mean gradient88.7598.44100.0096.25
AJR-lin97.1999.53100.0094.69
AJR-ens96.8899.69100.0095.94
Step 2: layer and multiplier selected
CAA-last, CAA-all, mean gradient (each)100.00100.00100.00100.00
AJR-lin100.00100.00100.0099.38
LoReFT (also rank and checkpoint)100.00100.00100.00100.00

Step 1 (Table 10, new). With every method at the original middle layer, tuning CAA's multiplier does not close the gap. AJR-ens is 3.8 to 24.8 points more accurate than CAA-last and CAA-all in all four models, and more accurate at every tested nonzero multiplier (every paired 95% interval is above zero). The best CAA accuracy at any multiplier (89.53/91.41/78.44/92.50%) is below AJR-ens at the original setting. The mean gradient is 8.1 points [6.4, 9.8] below AJR-ens in Qwen1.5-0.5B and within 1.3 points in the other models; in Pythia both reach 100%.

Step 2 (Table 3). Each fixed direction reaches 100% with zero harm in all four models, on seen and unseen templates, at depth .875 (the Gemma mean gradient at .75). AJR-lin with the same search reaches 100/100/100/99.38%. With the layer free, every method is within one point of 100%.

New prompts (Table 10b). We froze all directions, layers and multipliers, stated the comparisons in advance, and evaluated 640 new prompts whose entity names appear nowhere else. Step 1, answer accuracy (%):

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit78.9180.6272.3493.12
CAA-last88.7585.1672.6693.28
CAA-all87.3485.7872.0393.12
Mean gradient90.0099.38100.0095.31
AJR-lin96.09100.0099.6995.94
AJR-ens95.94100.0099.6996.25

Step 1 repeats: AJR-ens is 3.0 to 27.7 points above both CAA variants in all four models (all eight intervals above zero), 5.9 points [3.8, 8.3] above the mean gradient in Qwen1.5-0.5B, and within one point of it elsewhere. Step 2 repeats: 99.4–100% for the fixed directions and 100/100/100/99.06% for AJR-lin.

The depth profile explains both steps (Table 7): the cosine between the mean gradient and each prompt's own gradient is .19–.44 at the middle layer and .87–.98 at depth .875. A prompt-specific direction matters where responses differ across prompts; near the output, one direction is enough for this task.

AuthorsOpenReview ↗

Response to Reviewer sDiP

Q2. Stronger baselines

Prompting. The fixed prefix Hint: the answer is {red/blue}. , with no prompt search, is below original-setting AJR-ens in Qwen1.5-0.5B and Pythia (by 4.4 and 23.1 points) and above it in Qwen1.5-1.8B and Gemma (by 0.8 and 2.5 points).

LoReFT. A task-adapted LoReFT learns a low-rank, state-dependent edit for each color by target-token cross-entropy on the 960 fitting prompts. It reaches 100% with zero harm in all four models. Its edit size is learned and is not matched to AJR's (median output KL 11.4–16.3).

SAE features (Table 4). On Gemma, at AJR's middle layer, a Gemma Scope feature direction reaches 99.69% with zero harm after strength selection, against 95.94% for AJR-ens. AJR's direction is much closer to the response gradient (cosine .576 against .075) and gives larger margin gains, so alignment with the response and preservation of correct answers are different properties.

The related work now cites Linear-AcT and DSAS; the tuned baselines here are CAA, the mean gradient, LoReFT, SAE features and the hint.

Q1 and Q4. Realistic settings and larger models

The color task isolates the question the paper asks. Every directed method and the hint receive the correct color, so the comparison tests how well an edit produces a supplied answer. The fitted target is the margin of one next token, in four models from Pythia-410M to Gemma-2-2B. On the natural-text pronoun test (Appendix E.4), AJR exceeds CAA-last in all four models and the mean gradient in Pythia; CAA-all, which edits every position, is higher. As you ask, this result is now stated in the main text (Section 5). The 27B study finds the same response geometry as in the small models. The revised abstract and discussion state the claim directly: AJR is a learned model of the effect of an edit at a specified layer.

Q5 and Q8. General ability and when to steer

The paper measures the side effects of an edit where it acts: harm to correct answers, full-output KL (Appendix D), and the token after the answer. At the original setting, the expected noun follows a steered color in at least 99.7% of cases. Accuracy alone can hide such effects: CAA-all reaches 100% in Step 2, yet the expected noun follows in only 48.44% of Pythia prompts. AJR edits the one position where the answer is produced, so it is applied where that answer is wanted; a policy for when to steer in open-ended generation is a separate question from the response model studied here.

Q6. Parameters, storage and fitting cost

Per layer and color pair (Table 13); counts are stored numbers.

ModelAJR linear / MLP weightsAJR bases and statisticsAJR-ens stored sizeFixed vectorLoReFT
Qwen1.5-0.5B65,536 / 557,632136,1925.79 MiB1,02465,568
Qwen1.5-1.8B131,072 / 1,081,920272,38411.33 MiB2,04832,776
Pythia-410M131,072 / 557,632201,7286.79 MiB1,02465,568
Gemma-2-2B73,728 / 1,212,992232,70411.59 MiB2,304147,488

For Qwen1.5-0.5B, the complete setup for one layer and color pair takes 70.57 s on one B200 GPU with 2.9 GiB peak GPU memory. This covers gradient collection for 960 prompts, the hyperparameter search, the final fits and verification. At inference, AJR's median time was close to unedited inference, while gradient steering took 2.1–2.3 times as long.

Q7. The two AJR versions separately

Original middle layer and edit size (Table 12): margin gain, then correction / harm (%).

ModelAJR-linAJR-mlpAJR-ens
Qwen1.5-0.5B1.785, 73.8 / 0.01.349, 57.1 / 0.41.804, 71.4 / 0.2
Qwen1.5-1.8B3.563, 90.9 / 0.42.329, 85.1 / 0.03.370, 93.4 / 0.0
Pythia-410M2.350, 97.6 / 0.01.448, 59.4 / 0.02.160, 92.7 / 0.0
Gemma-2-2B4.520, 98.1 / 3.12.531, 84.6 / 0.94.350, 100.0 / 2.7

The linear predictor alone gives larger margin gains than the ensemble in three models. The MLP alone gives the smallest gain and correction in all four. Averaging is therefore not required for steering, and the revised paper presents linear AJR as the base method with the MLP and the ensemble as variants. With the multiplier selected (Step 1), AJR-lin and AJR-ens are within 1.3 points of each other.

Presentation

The abstract now states the idea in plain terms before any technical vocabulary and reports the selected-setting result directly. Section 3 opens with the red/blue example and gives the fitting and editing procedure before the paired-edit analysis. Repeated qualifiers were removed; each scope condition is stated once.

Thanks for doing this. I think all the additional experiments are going in the right direction. The experimental section is now more fair and stronger thanks to the new baselines and additional information provided.

I understand that in the middle layer ARJ does better but why should that be preferable than the last layer?

In general, and compared to other steering work, I think realistic applications (e.g., harmful language mitigation) and side effects when one does not know when to steer are still an important gap that needs to filled in order for this work to gain an acceptance score.

Reviewer_niT4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies how to predict changes in an LLM's output after intervening on its intermediate hidden states, and how to use these predictions for prompt-adaptive activation steering. The authors argue that decoding intermediate hidden states and predicting intervention effects are two different problems. Decoding methods such as Logit Lens and Tuned Lens can read information from intermediate states, but they may not accurately predict intervention effects. Based on this observation, the authors propose Amortized Jacobian Readout (AJR), which uses gradient supervision to learn input-dependent local intervention response directions. By combining linear and nonlinear predictors, AJR enables prompt-adaptive steering during inference without backpropagation.

The paper conducts experiments on intervention effect prediction, activation editing, and computational cost across four language models with 2B parameters or fewer. The results show that AJR achieves better steering performance than fixed gradient directions and contrastive activation methods on controlled color-retrieval tasks, while keeping a low inference cost.

Critical strengths

1. Clear research motivation. Studying the difference between decoding model outputs from intermediate hidden states and predicting the effects of interventions is important for LLM interpretability research. A tool that can accurately and efficiently predict the effects of hidden-state interventions could greatly help research on LLM interpretability.

2. Reasonable method design and good results in controlled experiments. AJR uses gradient supervision to learn input-dependent intervention directions, enabling prompt-adaptive steering without backpropagation during inference. The experiments verify its effectiveness across different models. On controlled color-retrieval tasks, AJR achieves better steering performance than fixed gradient directions and CAA, while maintaining a low inference cost.

Critical weaknesses

1. Limited core contribution For the goal of this paper, predicting intervention effects is only an intermediate step toward steering. What really matters is the final steering performance. Table 2 shows that AJR can predict intervention effects more accurately. However, Logit Lens and Tuned Lens are mainly designed to decode hidden states and do not directly consider how downstream layers respond to interventions. Therefore, it is not surprising that they perform worse than AJR, which is trained with gradient supervision. The real value of AJR should be shown by its final steering performance. However, the improvements in Table 1 are only demonstrated on a simple synthetic color-retrieval task. In the natural-text task in Appendix D.1, AJR performs worse than the simple Mean Gradient baseline on three of the four models and does not outperform CAA-all on any model. Therefore, I think the current experiments are not sufficient to demonstrate the practical value of AJR as a steering method.

2. Limited applicability. AJR requires collecting gradients and training a predictor for each specific model layer and token contrast. Changing the steering target usually requires training a new predictor. In addition, the paper only reports the inference speed improvement without considering the offline training cost or analyzing how many inference queries are needed to make up for this cost. Considering its poor performance in more natural language settings, the practical value of improving inference speed through training a separate predictor remains limited.

Questions for authors

Q1. Generalization to Natural Text

In Appendix D.1, AJR performs worse than Mean Gradient on three of the four models and does not outperform CAA-all on any model. Can the authors explain why AJR loses its advantage in this setting? Can the authors provide experiments on more natural language tasks to further evaluate its generalization ability?

Q2. Practical Computational Cost

AJR requires collecting gradients and training separate predictors for different model layers and token contrasts. Can the authors report the total offline training cost and analyze how many inference queries are needed to make up for this cost?

Q3. Effectiveness on Larger Models

The paper only evaluates AJR steering on relatively small models, while the experiments on the larger model only analyze the geometry of intervention responses. Can the authors provide steering experiments on larger models to evaluate the scalability of the method?

5 comments
AuthorsOpenReview ↗

Response to Reviewer niT4

Thank you for separating the prediction result from the steering claim. The revision shows steering against tuned alternatives in three settings. Tables and sections refer to the revised PDF.

W1. Is beating decoders the meaningful comparison?

The comparison that carries the claim is the mean gradient, not the decoders. It is fitted from the same gradients as AJR and differs only in using one direction for all prompts. Spearman correlation with measured edit effects at the middle layer (Table 1; Table 2 in the submission):

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
Mean gradient−.149−.163.619−.082
AJR-lin.743.825.910.926

The response to an edit therefore differs across prompts, and the current hidden state predicts that difference. The decoders make a separate point: the tuned lens decodes the answer accurately (Spearman .82 to .95) and still predicts edit effects poorly (−.03 to .26).

W1. Does the prediction give useful steering?

We now compare three settings. Every method is given the target color; settings are selected on 384 separate calibration prompts (highest accuracy with at most 2% of correct answers made wrong).

(a) Middle layer, common edit size (Table 3, which extends Table 1 of the submission). AJR-ens gives a larger margin gain than every fixed direction in every model: 1.80/3.37/2.16/4.35 against at most .76/1.66/1.47/1.66.

(b) Middle layer, strength selected for each method (Table 10, new). Answer accuracy (%) on 640 evaluation prompts:

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit80.3181.0974.2291.88
CAA-last89.5385.6275.1692.19
CAA-all88.2886.5675.3191.88
Mean gradient88.7598.44100.0096.25
AJR-ens96.8899.69100.0095.94

Tuning the strength of CAA does not close the gap: AJR-ens is 3.8 to 24.8 points more accurate in all four models, and more accurate at every tested nonzero strength (every paired 95% interval is above zero). The mean gradient is 8.1 points below AJR-ens in Qwen1.5-0.5B and within 1.3 points in the other models; in Pythia both reach 100%.

The result repeats on new prompts (Table 10b). We froze every direction and strength, stated the comparisons in advance, and evaluated 640 new prompts whose entity names appear nowhere else.

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit78.9180.6272.3493.12
CAA-last88.7585.1672.6693.28
CAA-all87.3485.7872.0393.12
Mean gradient90.0099.38100.0095.31
AJR-ens95.94100.0099.6996.25

CAA is 3.0 to 27.7 points below AJR-ens in all four models (all eight intervals above zero). AJR-ens is 5.9 points [3.8, 8.3] above the mean gradient in Qwen1.5-0.5B and within one point of it elsewhere.

(c) Layer and strength selected (Table 3). Each fixed direction reaches 100% with no harm in all four models, at depth .875 (the Gemma mean gradient at .75). So does a task-adapted LoReFT. AJR-lin with the same search reaches 100/100/100/99.38%. With the layer free, every method is within one point of 100%.

The depth profile explains the difference between (b) and (c) (Table 7): the cosine between the mean gradient and each prompt's own gradient is .19–.44 at the middle layer and .87–.98 at depth .875. AJR's steering benefit therefore appears where the layer is fixed in advance, as when a particular internal state is studied, and where responses differ across prompts.

AuthorsOpenReview ↗

Response to Reviewer niT4

Q1. Natural text

The pronoun study is now Appendix E.4 (Appendix D.1 in the submission). AJR-ens exceeds CAA-last in all four models, exceeds the mean gradient only in Pythia, and is below CAA-all in every model. CAA-all edits every position and so applies more total edit, but that does not explain the result against the last-position mean gradient.

A diagnostic on held-out articles separates two limits (Table 15, new). At equal edit size, it measures the share of the local gradient's first-order effect that each direction captures:

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
Mean gradient.200.170.218.183
AJR-lin.226.194.292.226
AJR-lin directions with exact weights.644.490.638.545

With exact weights, AJR's stored directions capture 49–64% of the first-order effect; the weights it predicts from the state capture only 19–29%, close to the mean gradient. On color prompts, AJR-ens reaches 65–85% of the local gradient's measured margin gain (Table 3). Exact weights thus recover substantially more of the effect, but both the prediction of weights from the state and the stored directions remain limiting: 36–51% of the first-order effect at equal edit size is unrecovered even with exact weights. AJR's lead over the mean gradient grows with each doubling of the 1,024 fitting contexts (+.03 to +.04 at the last doubling), but on separate evaluation articles the lead appears only in Pythia. The diagnostic thus identifies what limits AJR on natural text, and the pronoun study remains the paper's natural-language test.

W2 and Q2. Offline cost and break-even

For Qwen1.5-0.5B, the complete setup for one layer and color pair takes 70.57 s on one B200 GPU with 2.9 GiB peak GPU memory. This covers gradient collection for 960 prompts, the hyperparameter search, the final fits and verification. The stored predictor takes 5.79/11.33/6.79/11.59 MiB across the four models, against one vector of 1,024–2,304 numbers for a fixed direction (Table 13). A new layer or token pair needs a new fit.

At inference, AJR's median time was close to unedited inference, while local-gradient steering took 2.1–2.3 times as long (Section 4.4, Appendix D). A break-even count against local-gradient steering would require both methods timed at equal steering quality, and their margin gains differ (2.38–6.70 for the local gradient, 1.80–4.35 for AJR-ens). We therefore report the two costs separately: a one-off fit of about 70 s, and no backward pass per prompt. Against fixed directions, which also need no backward pass, the fit buys the fixed-layer gains in (a) and (b).

Q3. Steering in larger models

The 27B study measures response geometry and finds the same pattern as in the small models: later layers oppose edits in both directions. The steering experiments use the four models from Pythia-410M to Gemma-2-2B.

Dear Reviewer,

As we approach the final hours of the author-reviewer discussion period, I would greatly appreciate it if you could take a moment to review and respond to the authors’ rebuttal.

Please briefly indicate which concerns have been addressed and which remain, and consider updating your scores if needed.

Thank you again for your time and valuable contributions!

Best regards,
AC

Reviewer_QmknOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

AJR learns to predict how a hidden state intervention changes a specified output token contrast in a frozen language model. Offline gradients provide response direction targets, which are represented in a low-rank basis and predicted from the source activation using linear and nonlinear models. Their averaged prediction supplies a prompt-dependent steering direction during a single forward pass, avoiding an inference time backward pass. Experiments distinguish accurate state decoding from accurate prediction of intervention effects and compare AJR with fixed mean-gradient and contrastive-activation directions. On synthetic color retrieval across four models, with the desired color supplied, AJR improves target margins and corrects 71–100% of initially wrong answers at the chosen edit norm. A natural text pronoun experiment does not reproduce a consistent advantage over fixed directions. A separate 27B experiment validates response geometry, rather than AJR steering at that scale.

Critical strengths

The method is simple and has a clear computational motivation. Predicting an intervention direction from the current hidden state allows prompt-dependent steering in a single forward pass, without computing gradients at inference time. The color-retrieval results demonstrate an advantage over the fixed directions considered.

Critical weaknesses
  1. Despite the relative simplicity of the method, I found Section 3 unnecessarily difficult to follow. The paper would benefit from revising the notation and improving the narrative, particularly by clarifying the motivation behind the mathematical definitions.

  2. Limited experimental evaluation: I would like to see a broader range of tasks, such as factual knowledge editing, sentiment steering, and steering more complex behaviors involving multiple output tokens. The failure to consistently outperform fixed directions on the natural text pronoun task also deserves more discussion. Also, the paper lacks comparisons with established SAE-based steering methods. Beyond comparing steering performance, it would be particularly interesting to examine the cosine similarity between the intervention directions produced by these approaches under matched conditions. Finally, comparison with HyperSteer [1], which already proposes input-conditioned activation steering, would strengthen the paper's claims.

[1] Sun et al. (2025). HyperSteer: Activation Steering at Scale with Hypernetworks. arXiv:2506.03292.

Questions for authors
  1. What explains the lack of a consistent advantage over fixed directions on the natural-text pronoun task?
  2. Does AJR also improve steering on tasks with multi-token outputs?
  3. How does AJR compare with SAE-based steering and HyperSteer in steering performance and direction cosine similarity under matched conditions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Qmkn

Thank you for the comments on exposition and on the missing comparisons. Section 3 is rewritten, and the SAE comparison you ask for is now in the main text. Tables and sections refer to the revised PDF.

W1. Exposition of the method

Section 3 now presents the method in three steps, with a red/blue example, before any analysis:

  1. Target. With the language model frozen, a backward pass gives, for each fitting prompt, the gradient at the chosen layer of the paper's response score: the margin of the target token over its alternative, with the final normalization scale held fixed. To first order, this is the edit direction that most increases the score.
  2. Predictor. A small predictor learns to output that direction from the hidden state at the same layer. Linear AJR (a ridge map onto a shared low-rank set of directions) is the base method; an MLP and their average (AJR-ens) are variants.
  3. Edit. For a new prompt, the predictor is evaluated during the forward pass, its output is scaled to the chosen edit size, and it is added to the state at the last prompt position. No backward pass or second run is needed.

The paired-edit analysis, which compares an edit with its opposite, now follows the method as a diagnostic (Section 3.4) and is not needed to run AJR.

W2. Breadth of tasks and strength of comparisons

The matched control for AJR is the mean gradient, fitted from the same gradients but shared by all prompts. Spearman correlation with measured edit effects at the middle layer is .74/.83/.91/.93 for AJR-lin against −.15/−.16/.62/−.08 (Qwen1.5-0.5B, Qwen1.5-1.8B, Pythia-410M, Gemma-2-2B; Table 1).

For steering we added tuned baselines. Settings are selected on 384 separate calibration prompts (highest accuracy with at most 2% of correct answers made wrong). Answer accuracy (%) on 640 evaluation prompts:

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit80.3181.0974.2291.88
Middle layer, strength selected (Table 10)
CAA-last89.5385.6275.1692.19
CAA-all88.2886.5675.3191.88
Mean gradient88.7598.44100.0096.25
AJR-ens96.8899.69100.0095.94
Layer and strength selected (Table 3)
CAA-last, CAA-all, mean gradient (each)100.00100.00100.00100.00
LoReFT, task-adapted100.00100.00100.00100.00
AJR-lin100.00100.00100.0099.38

At the middle layer, AJR-ens is 3.8 to 24.8 points more accurate than CAA after each method's strength is tuned, and more accurate at every tested nonzero strength (every paired 95% interval is above zero). It is 8.1 points above the mean gradient in Qwen1.5-0.5B and within 1.3 points of it in the other models; in Pythia both reach 100%. The result repeats on 640 new prompts with unused entity names and all settings frozen (Table 10b): CAA is 3.0 to 27.7 points below AJR-ens in all four models.

When the layer is also selected, fixed directions at depth .75–.875 and LoReFT reach 100%, and every method is within one point of 100%. The depth profile explains both results (Table 7): the cosine between the mean gradient and each prompt's own gradient is .19–.44 at the middle layer and .87–.98 at depth .875. A prompt-specific direction matters where responses differ across prompts. Beyond the color task, the paper tests natural text with the pronoun study (Q1).

AuthorsOpenReview ↗

Response to Reviewer Qmkn

Q1. Natural text

On the pronoun test (Appendix E.4), AJR-ens exceeds CAA-last in all four models, exceeds the mean gradient only in Pythia, and is below CAA-all in every model. The discussion (Section 5) now states this result.

A diagnostic on held-out articles separates two limits (Table 15, new). At equal edit size, the mean gradient captures 17–22% of the local gradient's first-order effect and linear AJR 19–29%. If the weights that AJR predicts for its stored directions are replaced by the exact ones, the share rises to 49–64%. Exact weights therefore recover substantially more of the first-order effect, but both the prediction of weights from the state and the stored directions remain limiting: 36–51% is unrecovered even with exact weights. On color prompts, AJR-ens reaches 65–85% of the local gradient's measured margin gain (Table 3). AJR's lead over the mean gradient grows with each doubling of the 1,024 fitting contexts, but on separate evaluation articles it appears only in Pythia. The diagnostic thus identifies what limits AJR on natural text.

Q2. Multi-token outputs

The fitted target is the margin of one next token, which is the setting of the paper. Beyond that token, the continuation stays intact: at the original setting, the expected noun follows a steered color in at least 99.7% of cases.

Q3. SAE steering and HyperSteer, with direction cosines

We added a matched comparison with sparse-autoencoder (SAE) features: the public Gemma Scope dictionary (16,384 features) for Gemma-2-2B at AJR's middle layer. For each color, features are shortlisted by activation difference and by projection onto the same fitting gradients that AJR uses, and one is chosen by its measured effect on fitting prompts. All directions edit the last prompt position at the same layer. Strength is selected on calibration prompts with the rule above.

DirectionCosine to local gradientOriginal norm: gain, acc. %, harm %Selected strength: gain, acc. %, harm %
SAE feature.075.755, 95.16, 0.002.406, 99.69, 0.00 (×4)
CAA-last.008.045, 92.50, 0.34.016, 92.19, 0.17 (×.25)
Mean gradient.1711.667, 96.09, 1.02.880, 96.25, 0.17 (×.5)
AJR-lin.6344.520, 97.03, 3.067.830, 94.69, 5.61 (×4)
AJR-ens.5764.350, 97.50, 2.728.156, 95.94, 4.42 (×4)

The SAE direction is nearly orthogonal to AJR's (cosine .042 to AJR-ens, .043 to AJR-lin). AJR's directions are far closer to each prompt's response gradient and give about six times the margin gain at the original norm, with higher accuracy there (+2.3 points [1.1, 3.6]). After strength selection the SAE feature is more accurate (+3.75 points [2.65, 4.84]) because it makes no correct answer wrong, while AJR-ens corrects every wrong answer but harms 4.42% of correct ones. Alignment with the response and preservation of correct answers are thus different properties; the paper reports both (Table 4, Appendix E.2, Table 11).

We compare with HyperSteer in the related work. It generates steering vectors with a hypernetwork trained end to end and conditioned on a steering prompt; one variant also conditions on the model's internal states. AJR is fitted to the model's own response gradient, so its output also predicts what an edit will do (Table 1). The related work now states this difference.

Dear Reviewer,

As we approach the final hours of the author-reviewer discussion period, I would greatly appreciate it if you could take a moment to review and respond to the authors’ rebuttal.

Please briefly indicate which concerns have been addressed and which remain, and consider updating your scores if needed.

Thank you again for your time and valuable contributions!

Best regards,
AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

We thank the reviewers. All three reviews ask what the controlled prediction result is worth for steering. We ran the comparisons the reviews request, and the revised paper reports them. The individual replies answer each question. Tables and sections refer to the revised PDF.

What AJR does. For a chosen layer and pair of output tokens, AJR predicts, from the current hidden state, the gradient of the paper's response score: the target token's margin over the alternative, with the final normalization scale held fixed. It is fitted once from gradients and needs no backward pass at inference. Its matched control is the mean gradient: the same gradients, averaged into one direction for all prompts.

1. Prediction (Table 1). At the middle layer, Spearman correlation with measured edit effects is .74/.83/.91/.93 for linear AJR and −.15/−.16/.62/−.08 for the mean gradient (Qwen1.5-0.5B, Qwen1.5-1.8B, Pythia-410M, Gemma-2-2B; same order below).

2. Steering in three settings (Tables 3 and 10). Every method is given the target color.

(a) Middle layer, common edit size. AJR-ens gives a larger margin gain than every fixed direction in every model: 1.80/3.37/2.16/4.35 against at most .76/1.66/1.47/1.66.

(b) Middle layer, strength selected for each method (new). (c) Layer and strength selected. Settings are chosen on 384 separate calibration prompts: highest accuracy with at most 2% of correct answers made wrong. Answer accuracy (%) on 640 evaluation prompts per model:

Qwen1.5-0.5BQwen1.5-1.8BPythia-410MGemma-2-2B
No edit80.3181.0974.2291.88
(b) Middle layer, strength selected
CAA-last89.5385.6275.1692.19
CAA-all88.2886.5675.3191.88
Mean gradient88.7598.44100.0096.25
AJR-lin97.1999.53100.0094.69
AJR-ens96.8899.69100.0095.94
(c) Layer and strength selected
CAA-last, CAA-all, mean gradient (each)100.00100.00100.00100.00
LoReFT100.00100.00100.00100.00
AJR-lin100.00100.00100.0099.38
  • At a specified layer, AJR-ens is more accurate than CAA after each method's strength is tuned. CAA stays 3.8 to 24.8 points below in all four models, and below at every tested nonzero strength (every paired 95% interval is above zero). The mean gradient is 8.1 points below AJR-ens in Qwen1.5-0.5B and within 1.3 points in the other models; in Pythia both reach 100%.
  • The result repeats on new prompts (Table 10b). We froze every direction, layer and strength, stated the comparisons in advance, and evaluated 640 new prompts whose entity names appear nowhere else. CAA is 3.0 to 27.7 points below AJR-ens in all four models (all eight intervals above zero). Against the mean gradient, AJR-ens is 5.9 points ahead [3.8, 8.3] in Qwen1.5-0.5B and within one point in the other three models.
  • When the layer is also free, the task saturates. Every fixed direction selects depth .75 or .875 and reaches 100%, as does LoReFT, and AJR-lin reaches 99.38–100%: every method is within one point of 100%.
  • The depth profile explains both (Table 7). The cosine between the mean gradient and each prompt's own gradient is .19–.44 at the middle layer and .87–.98 at depth .875. A prompt-specific direction matters where responses differ across prompts; near the output, one shared direction suffices.

AJR is therefore a learned model of what an edit at a chosen layer will do. Its steering benefit appears where the layer is fixed in advance, as when a particular internal state is the object of study, and where responses differ across prompts. The revised abstract, Section 4.4 and discussion state the claim in this form.

3. Other added comparisons. A supplied-answer hint (Table 3); a task-adapted LoReFT (Appendix E.1); sparse-autoencoder features at the same layer on Gemma (Table 4, Appendix E.2); linear and nonlinear AJR separately (Table 12); stored size and fitting cost (Table 13).

4. Natural text (niT4, Qmkn). A new diagnostic on held-out articles (Table 15) measures the share of the local gradient's first-order effect that each direction captures: 17–22% for the mean gradient, 19–29% for linear AJR, and 49–64% for AJR's stored directions with exact weights. The replies to niT4 and Qmkn discuss it.

5. Changes to the paper. The abstract and Section 3 are rewritten: the method is explained with a red/blue example before the paired-edit analysis, and linear AJR is the base method. Section 4.4 reports the fixed and selected settings side by side (Table 3) and the SAE comparison (Table 4). Appendix E gives the settings, intervals and costs of all added comparisons, including the middle-layer strength selection and its repeat on new prompts (Table 10).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.