acceptodds
Under review as a conference paper at ICLR 2027

How Far Can Instruction Steering Go? Coverage, Quality, and Calibration

Abstract

Activation steering promises modular control of language models: learn one direction per instruction, then add the directions that a request needs. This promise rests on two assumptions: directions that work alone also work together, and an answer that passes an instruction checker is still a good answer. We test both assumptions on seven models. We calibrate one edit per instruction and freeze it. An answer counts as a success only if it passes every checker and a blind quality judge, which agrees with a human audit. We separate installation, where the edit must carry an instruction that the prompt omits, from reinforcement, where the prompt already states it. Few instructions can be installed reliably. The quality judge shrinks this set further. On four smaller models, it removes all four installations that the automatic checks accept, and it roughly halves the usable reinforcements. Where installations do succeed alone, as on Gemma-3-27B, they also compose. On instruction pairs, they beat the unedited model by 35.0 points. Still, they trail simply stating the instructions by 17.2 points. Reinforcement gives mixed results and falls behind prompting as sets grow, yet much of this loss comes from calibration. Composing steering directions is therefore feasible, but its value depends on coverage, answer quality, and calibration, and prompting remains the baseline to beat.

Primary area:
generative models
Keywords:
activation steering, instruction composition, constraint compliance, answer validity, instruction following, language model control, checker guided revision

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_NAfYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether frozen, instruction-specific activation directions can improve instruction following without compromising response quality. The authors calibrate one direction per instruction and evaluate both checker compliance and instruction-blind answer quality. The results show that usable directions are rare, while explicit prompting remains a stronger and cheaper baseline in most settings.

Critical strengths
  1. Composed steering performs no better than the unedited model on the full instruction pool. However, on Gemma-3-27B, directions that work individually also compose successfully, improving joint valid success by 35.0 percentage points over the unedited model. This makes the negative result more informative: steering appears to work only when the individual directions are already reliable. It also shows how evaluating only successful instructions can overstate the effectiveness of steering.
  2. The human audit suggests that the quality loss is associated with steering itself. Among checker-passing answers, raters rejected 32 of 66 steered responses but only 14 of 58 prompt-only responses. The two arms were evaluated on the same task prompts, with the application of steering edits as the key experimental difference.
Critical weaknesses
  1. The negative results may stem from the construction of the instructions and the search method. These instructions are of a single source, while the search process covers seven levels and six categories before the detailed screening. If different search methods for the instructions or a broader search scope are adopted, the number of effective instructions may be increased. The paper itself mentions this point, but it cannot be ultimately confirmed.
  2. The title numbers are largely determined by the set threshold. "The number of usable installations decreased from four to zero" used a success target of 0.80. The paper states that when the threshold was lowered to 0.70, the number of usable installations was two, when it was lowered to 0.50, it was four.
Questions for authors
  1. Is the low coverage a property of the models, or of the single method used to build the directions? Did you try any other way of obtaining the directions, for example training or optimizing them rather than taking a difference of means? Even a small test on a few instructions would help show whether more instructions become usable under a different extraction method.
  2. Why does Gemma-3-27B succeed where the other models do not? Do you have an explanation for what makes this model different? And if the success target is lowered to 0.70 or 0.50 so that other models also have members that work alone, do those members combine in the same way?
  3. What kind of damage does the judge actually catch? The human study asked three separate questions: whether the answer addresses the task, whether it contains wrong facts, and whether the writing is broken. Could you report the results of these three questions separately for the steered answers and for the prompt-only answers? Knowing whether steering mainly produces wrong facts or broken writing would indicate whether the damage can be fixed.
4 comments
AuthorsOpenReview ↗

Response to Reviewer NAfY (1/2)

We thank the reviewer NAfY for the thoughtful and constructive feedback. We appreciate the recognition of two results we also consider central: on Gemma-3-27B, directions that work individually also compose, improving joint valid success by 35.0 points over the unedited model, and the human audit links the quality loss to steering, with 32 of 66 steered answers rejected against 14 of 58 prompt-only answers. We address the two weaknesses and three questions below, including a new experiment with trained directions.

Note on the revised PDF. New content is in blue. We also rewrote Sections 1 to 5, 7, and 8 for readability. These rewrites are in black and change no result, number, or claim. Some details moved to the appendices.

Weaknesses

W1. The negative results may stem from the instruction source and the search method

Response: The reviewer suggests that a different instruction source or a broader search might yield more usable directions. We examined each factor in turn. The calibration search already spans 7 layers across model depth, 6 coarse doses, and a fine grid of about 16 doses per instance. Each direction is averaged over 80 matched extraction prompts, and re-extracting from four other prompt groups moves composition results by a standard deviation of 0.7 to 1.4 points (Appendix F), so the directions are stable with respect to the prompts used.

For the instruction source, the paper includes 20 IFBench constraint types alongside IFEval. On IFBench, calibration finds no usable installation under either the guard (an automatic filter for degenerate text) or the instruction-blind judge (Section 3).

For the steering method, the paper also runs the rule of Stolfo et al. (2025) in every test cell. It uses the same direction but sets the edit size from the activation rather than a tuned dose. On the four smaller models, it gives the same result as our rule: 4 usable installations under the guard and 0 with the judge. The trained-vector experiment described in Q1 extends this comparison to a different direction family. Across these three configurations, the number of installations usable under both judges stays between 0 and 1.

W2. Headline numbers depend on the 0.80 threshold

Response: The reviewer notes that the "four to zero" result uses a success target of 0.80. The target is applied uniformly to every setting (Section 2), and Appendix E reports the same analysis at lower targets.

TargetUsable installations, guard + judgeInstruction
0.800none
0.702english_lowercase (Llama-3.1-8B)
0.504english_lowercase (Llama-3.1-8B, Gemma-3-4B)

Even at a target of 0.50, 4 of 300 instances (1.3%) are usable with the judge, and at most 13 of 300 (4.3%) under the guard alone. Coverage therefore stays below 5% at every target we tested.

The lower targets also do not change the composition results. Every additional usable installation is the same instruction, english_lowercase, on a given model, and a set cannot contain the same instruction twice. Lowering the target thus produces no new set to compose.

AuthorsOpenReview ↗

Response to Reviewer NAfY (2/2)

Questions

Q1. Is low coverage a property of the models or of the difference-of-means method?

Response: To separate the two, we trained one direction per instruction with the instruction absent from the prompt, optimizing it so that the model reproduces its own compliant answers. Layer search, dose search, test prompts, guard, and judges are the same as in the paper. We ran all 25 instructions on Qwen2.5-7B and 5 on Llama-3.1-8B, giving 30 instruction settings.

If the difference-of-means method were the main limit, trained directions should yield many more usable installations. They do raise compliance: on Qwen2.5-7B, valid compliance rises from 0.09 to 0.97 for the no-comma instruction, from 0.00 to 0.94 for the title instruction, and from 0.04 to 0.92 for bullet lists. Under the guard, usable installations increase from 1 to 7 (Table R1).

Table R1. Mean-difference directions and trained vectors (30 instruction settings).

Quality filterMean difference (paper)Trained vector
Guard only17
Guard + Sonnet 502
Guard + GPT-5.6-terra01
Guard + both judges01

Most of these do not survive the judges. Two pass under Sonnet 5 and one under GPT-5.6-terra, and the setting that passes both is the title instruction on Llama-3.1-8B (0.82 under GPT-5.6-terra). Another 12 settings raise compliance by at least 20 percentage points without becoming usable: 8 stay below 0.80 and 4 exceed a quality bound.

Both factors therefore contribute, but answer quality dominates. The extraction method affects how often compliance is reached, while answer quality determines how many settings remain usable, and this second factor limits coverage for both direction families.

Q2. Why does Gemma-3-27B succeed where other models do not?

Response: On Gemma-3-27B, 5 of 6 guard-usable installations remain usable under the judge, compared with 0 on Qwen2.5-32B and Llama-3.1-70B (Table 1). The difference lies in the cost to answer quality: on Qwen2.5-32B, the two guard-usable installations reach high judged compliance but exceed the validity-loss bound (Section 3). Gemma-3-27B's installations, such as english_lowercase (strict compliance 0.00 → 0.97), reach the target without that cost. We do not claim a general mechanism for this difference, and Section 7 scopes the composition result to "one model and a small selected population." On the second part of the question, lowering the target does not create composable sets on other models, because the added members are all the same instruction (see W2).

Q3. What damage does the judge catch?

Response: We report the three human questions separately. The audit contains 200 answers, 100 per arm, balanced by judge verdict, so the percentages describe this sample rather than population rates.

Table R2. Human ratings by question.

QuestionPrompt onlySteered
Answers the task: "Yes"92%78%
Answers the task: "Partly" or "No"8%21%
Some wrong facts31%42%
Broken writing1%5%

The difference between the arms lies mostly in content rather than form. Among checker-passing answers, wrong facts appear in 42% of steered answers against 22% of prompt-only answers, while broken writing appears in 5% against 0%. Steering therefore mainly causes content drift: answers that address the task only in part or contain factual errors. A surface filter such as the guard cannot detect this, which is why the instruction-blind judge is needed, and it indicates that a fix would have to preserve content, not only fluency.

Raters outside the author team (five lab members, 100 answers) show the same pattern: among checker-passing answers, they find wrong facts in 13 of 34 steered answers (38%) against 6 of 32 prompt-only answers (19%). We have added Table R2 to Appendix K.


We thank the reviewer again for these concerns. The trained-vector experiment shows that a stronger extraction method raises compliance but not usable coverage, the threshold analysis shows that coverage stays below 5% down to a target of 0.50, and the per-question audit identifies content drift as the main quality cost of steering. We respectfully ask the reviewer to consider these results in the final assessment.


References

  • Stolfo et al. "Improving Instruction-Following in Language Models through Activation Steering." ICLR, 2025.

Thank authors for the detailed response, it addresses my most concerns. I'll maintain my score.

Dear Reviewer NAfY,

Thank you for the positive and constructive review and for your supportive assessment. We are glad that our responses addressed most of your concerns, and we appreciate you maintaining your positive assessment of the paper.

Sincerely,
Authors

Reviewer_T3UxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This paper explores the efficacy of instruction steering in several ways:

  1. With and without instructions included in the input prompt ("reinforcement" and "installation" prompt conditions, respectively)
  2. According to varied validation methods, including constraint checking, automated degeneration checking, AI judge for response quality, and human validation for quality, and calibrated combinations of each
  3. Combining multiple steering vectors that are effective in isolation to see if they can be used in composition

The results from these experiments demonstrated that, when using sufficient validation methods, steering under either prompt condition is only rarely effective, and that using a constraint checker alone can overstate steering performance. Importantly, including the instructions in the prompt with no steering is more effective in producing valid responses than steering under either condition. On the question of direction composition, the paper finds that directions that are effective in isolation can be combined to produce valid responses that meet multiple instruction constraints, but that finding directions that work well together requires careful calibration and selection.

Critical strengths
  • The paper addresses a gap in steering evaluation, acknowledging and demonstrating that common benchmarks only indicate whether the intended behavior changes, and not whether the model still produces valid responses.
  • The appendix includes details that indicate careful experimental design, consideration for reproducibility, and thorough analysis.
  • The question of whether individual steering directions can be used in composition is interesting and, to the best of my knowledge, a novel contribution.
Critical weaknesses
  • The methods were very difficult to follow, largely stemming from a lot of undefined and inconsistently used terms. A lot of the missing information was available in the appendices, but the appropriate sections were not always referenced in the main text. A short list of specific terms is provided below.
  • The dataset composition was particularly unclear and important to understanding the methods and results.
  • The results were also very difficult to follow, due to both the lack of clarity in the methods and the density of experiments. It was difficult to tease apart which results belonged to which experimental settings, how the various paragraphs of results were related to each other, and what the metrics in your tables and figures showed.
  • The composition experiment was unfortunately weak due to the lack of usable steering directions. The authors acknowledge that this is the case in the paper and do not overstate the result, but it doesn't seem likely that these results would generalize beyond the very specific conditions that they identify.
Questions for authors

Some suggested clarifications:

  • "dose" needs to be defined as your steering coefficient with some high-level detail about how it was tuned, with a direct reference to the appendix section that has more detail
  • Need clarification between the terms "instance", "prompt", "instruction", "set", "position". You use these throughout the paper, but it's unclear if they are interchangeable or if they refer to different experimental components.
  • Explicitly define "usable" vs. "compliance" vs. "valid" when you first use them. You have some definitions in the paper, but often not until after you have already been using them, and this distinction is necessary for understanding the methods and results.
  • Some definition of the main steering intervention should be included in the main paper, and you should make sure that the descriptions in the main paper and your appendix agree--as it is currently written, it seems that your Appendix A Intervention rule does not match the various descriptions of the steering intervention in your main paper.

Experimental suggestion:

  • If I understand the methods correctly, the steering directions were computed based on single matched pairs (or maybe averaged over three pairs for the small models?)--while there has been some success in isolating steering directions on single prompts, steering is generally more effective if directions are extracted based on a larger sample of matched prompts. This may be at least partially responsible for your low number of usable directions.
7 comments
AuthorsOpenReview ↗

Response to Reviewer T3Ux (1/2)

We thank the reviewer T3Ux for the thoughtful and constructive feedback. We appreciate the recognition that the paper addresses the gap between changing a behavior and producing a valid response, that the composition question is novel, and that the appendices document the experimental design. The concerns center on terminology, the dataset description, and how the steering directions are constructed. We address each below, including the definitions and revised text the reviewer requested.

Note on the revised PDF. New content is in blue. We also rewrote Sections 1 to 5, 7, and 8 for readability. These rewrites are in black and change no result, number, or claim. Some details moved to the appendices.

Weaknesses

W1. Undefined and inconsistently used terms

Response: The reviewer found that several terms were used before being defined and that appendix sections were not always referenced. We have added a definitions table at the start of Section 2, so every term is defined before its first use and points to where its details appear.

TermMeaningDetail
instructionOne of the 25 IFEval constraint types, e.g., "all lowercase"Section 2
instanceOne instruction with fixed arguments, e.g., "end with the phrase X"Section 2
promptOne base writing taskSection 2
positionOne prompt in the test list; every arm uses the same positionsAppendix A
set2 to 15 instances requested togetherAppendix D
doseSteering strength: edit = dose × unit direction × residual normAppendix A
compliantPasses the strict IFEval checkerSection 2
validPasses the guard (20+ words, not repetitive, not cut off) and the instruction-blind judgeSection 2, Appendix K
usableValid compliance ≥ 0.80, with validity falling and degeneration rising by at most 5 absolute points vs. the unedited modelSection 2, Appendix A

A usable installation must additionally raise compliance by at least 20 points over the unedited model, with a paired interval above zero.

W2. Dataset composition

Response: We have rewritten the data paragraph in Section 2 to state the sources, filters, splits, and what a single test answer consists of:

"Base prompts come from three open writing datasets: No Robots, Dolly, and OpenAssistant. We keep prompts with no existing length, count, or format constraint and a human reference answer of at least 250 words. Prompts are split by identity into 400 extraction, 300 development, 400 test, and 1,000 held-out comparison prompts. A test answer is one prompt plus one instance (single-instruction tests) or one set (composition)."

The next paragraph of Section 2 adds: "The two conditions differ only in whether the instruction text is in the prompt."

W3. Results difficult to follow across experimental settings

Response: The reviewer found it hard to tell which results belong to which setting. We have added a results map to Section 2 that lists, for each main result, the question it answers and the models, population, and arms it uses. Each table caption now repeats these fields.

ResultQuestionModelsPopulationArms
Tables 1, 2Which single instructions are usable?7 models75 instances per smaller model, 25 per largerUnedited vs. edited, installation and reinforcement
Table 3, Figure 4Does the full instruction pool compose?3 smaller models30 pair sets (Table 3); 10 admitted sets per count (Figure 4)Unedited, edits, prompt only, prompt plus edits
Table 4Do individually successful installations compose?Gemma-3-27B10 pairs and 10 triples, 400 test promptsUnedited, composition rules, prompt only, prompt plus edits
Table 5Does checker feedback beat a plain retry?3 smaller models30 pair sets per modelGeneric retry vs. diagnostic revision

For example, the revised Table 1 caption now begins: "Frozen single-instruction classifications under both validity rules. Smaller models: 75 instances (25 types × 3 instances) per prompt condition, 200 test prompts each. Larger models: 25 instances, 100 test prompts each. Usable counts apply the 0.80 success target and the quality bounds."

AuthorsOpenReview ↗

Response to Reviewer T3Ux (2/2)

Weaknesses

W4. The composition experiment is limited by few usable directions

Response: The reviewer notes that few directions are usable, which limits the composition experiment, and we thank the reviewer for acknowledging that the paper does not overstate this result. We view the condition itself as the finding: composition succeeds when its members are usable on their own, and this is a condition a practitioner can check before deploying a set. Three results support that this conclusion extends beyond our specific setup.

First, the low coverage is not specific to our steering rule. The rule of Stolfo et al. (2025), which uses an activation-dependent coefficient instead of a tuned dose, gives the same count on the four smaller models: 4 usable installations under the guard and 0 with the judge. A trained-vector family raises guard-usable installations from 1 to 7 of 30 settings, but only 1 passes both judges.

Second, the reinforcement results do not depend on installation coverage. For Gemma-3-4B pairs whose members have usable reinforcement settings, adding steering to the prompt reaches 41.1% [26.5, 53.9] against 19.8% [7.5, 33.7] for prompting alone, improving 9 of 10 sets.

Third, where members are usable, composition works across set sizes. On Gemma-3-27B, every composition rule beats the unedited model at two instructions, by 35.0 points [13.7, 57.8] for layer-separated composition and 13.4 [5.2, 25.0] for fixed-norm addition, across 10 pairs and 10 triples.

Questions / Comments

Q1. Define "dose"

Response: The dose is the steering coefficient. We have added to Section 2:

"The dose s is the steering coefficient: the edit is s × (unit direction) × (residual norm). [...] We first try six coarse doses at each of seven layers and keep the layer with the most usable doses. At that layer, we search a finer grid and freeze the middle of the longest run of usable doses (Appendix A gives the fallback rules)."

Q2. Distinguish "instance," "prompt," "instruction," "set," and "position"

Response: These are distinct components, defined in the table under W1. An instruction is a constraint type, an instance fixes its arguments, a prompt is a base task, a position is a prompt's slot in the shared test list, and a set is a group of instances requested together. "Three instances per type" means three different instruction arguments, such as three different closing phrases.

Q3. Define "usable," "compliant," and "valid" at first use

Response: The definitions table now opens Section 2, before any of these terms appears in the methods or results.

Q4. The main text and Appendix A describe the intervention differently

Response: The two passages describe the same rule in different notation. The main text now states it in the same terms as Appendix A:

"At layer ℓ, each row receives ‖h_ℓ‖ · e at every prompt and generated position, where h_ℓ is that row's residual at its newest position and e is the relative edit. The norm is recomputed at every decoding step, separately for each row."

Experimental suggestion: directions extracted from single matched pairs

Response: The reviewer suggests that directions extracted from single pairs may explain the low number of usable directions. No direction comes from a single pair. The 400 extraction prompts are split into 5 disjoint groups of 80, and each direction is the mean last-token activation with the instruction minus the mean without it, over all 80 prompts in a group, then normalized. This follows Rimsky et al. (2024) and Stolfo et al. (2025). Main results use the first group, and the other four test stability: re-extracting from them moves composition results by a standard deviation of 0.7 to 1.4 points (Appendix F). The "three instances" in the paper refer to three instruction arguments, not three pairs. As noted under W4, the Stolfo et al. rule and trained vectors, which do not rely on our dose rule, show the same coverage pattern.


We thank the reviewer again for these concerns. The revised Section 2 defines every term before use, describes the dataset and each experimental setting, and gives one statement of the intervention rule; the additional results show that the coverage pattern holds with other steering rules and is not caused by the extraction procedure. We respectfully ask the reviewer to reconsider the score in light of these changes.


References

  • Rimsky et al. "Steering Llama 2 via Contrastive Activation Addition." ACL, 2024.
  • Stolfo et al. "Improving Instruction-Following in Language Models through Activation Steering." ICLR, 2025.

Thank you to the authors for your careful response to my concerns. Your in-text edits have satisfied my concerns around W2, Q1, and Q4. However, the rest of my concerns remain.

  • While the added lookup tables serve the function of providing definitions and clarity, they are functionally a band-aid for a paper that needs more careful revision for clarity. Your new tables are also missing a Table number, so that needs to be fixed.
  • Your explanation of the composition results is reasonable, but the experiment itself is still very limited based on the small number of individual usable directions. Given that much of your discussion and presentation focuses on how your work leads to composition, the weak experiment and results fall short of expectations.

My recommendation for a Weak Rejection remains, and I recommend that the authors consider a major revision of the manuscript before submitting to another venue with the following changes:

  • Reframe your main contributions to focus on the benchmarking gap instead of the composition experiment, as this seems the more valuable result supported by your work.
  • Limit your discussion of the composition experiment to a brief description of the settings, results, and limitations. The limitations discussion can focus on how these results may drive future work that would have broader implications. Most experimental details and analysis can be moved to an appendix.
  • Rewrite Section 2 to cleanly introduce terms in-text, removing the need for the new look-up table.
  • Reorganize Section 3 to clearly separate your single-steering direction experiments and spend more time discussing and analyzing each one, removing the need for your results lookup table.
Reviewer_LtjjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether independently calibrated activation steering directions can be composed to satisfy multiple instructions at once while keeping the answer itself good. It separates installation, where the edit must carry an instruction that the prompt never states, from reinforcement, where the prompt already states the instruction, and every edit is calibrated once and frozen before test time. Answers must pass deterministic instruction checkers, an automatic degeneration guard, and an instruction blind quality judge, and the paper reports that checker compliance badly overstates steering success, that individually successful installation directions compose but lose to simply stating the instructions in the prompt, and that much of the reinforcement gap comes from calibration.

Critical strengths

The measurement protocol has real discipline. Settings are frozen before test time so quality never triggers reselection, central comparisons carry paired bootstrap intervals over shared resamples, the automatic quality gate is at least checked against a human audit, prompting is consistently treated as the baseline to beat, and the paper includes a compute ledger and an honest discussion of what the design can and cannot support. The finding that checker success and answer quality come apart is worth recording, and the writing is clear.

Critical weaknesses

The load bearing piece of the evaluation is too fragile for the weight it carries. The human audit is performed by three of the authors themselves rating outputs of their own steering pipeline, and that audit agrees with the primary judge, a single LLM, on only 80 percent of answers with a kappa of 0.60. Every headline number in the paper, from usable installations falling from 4 to 0 to reinforcements falling from 35 to 18, is a function of this judge, so the central claims rest on a quality gate whose own validation is moderate at best. The empirical scope is also very narrow: one family of verifiable IFEval style constraints on open ended English writing under greedy decoding, with no training seed replication, so the paper ends with a conditional bottom line that practitioners already operate under, namely that simply prompting the model is the stronger and cheaper option. As measurement work this is careful, but the evidence is too thin and too self-referential to justify the paper's length or its framing, and I do not think it clears the bar for ICLR.

Questions for authors

Would the judge versus audit agreement change materially if the audit were run with raters outside the author team? Given how much of the paper rests on the judge, what happens to the headline counts of usable settings if a second judge family is used instead?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Ltjj (1/2)

We thank the reviewer Ltjj for the constructive feedback. We appreciate the recognition of the frozen-before-test protocol, the paired bootstrap intervals over shared resamples, the human audit of the quality gate, the use of prompting as the baseline to beat, and the compute ledger. The review's central concern is that the main claims depend on a single LLM judge validated by the authors. We tested this directly with a second judge family and with raters outside the author team, as described below.

Note on the revised PDF. New content is in blue. We also rewrote Sections 1 to 5, 7, and 8 for readability. These rewrites are in black and change no result, number, or claim. Some details moved to the appendices.

Weaknesses

W1. The evaluation depends on one LLM judge validated by three authors

Response: The reviewer is concerned that every headline number is a function of one judge whose validation is moderate. We address this in three steps: we first identify which claims need the judge at all, then test those claims with a second judge family, and finally repeat the human audit with raters outside the author team.

Two filters are involved. The guard is an automatic filter that rejects degenerate text, and the judge is an instruction-blind model that decides whether the answer addresses the task. The finding that installation is rare does not require the judge. Under the instruction checkers and the guard alone, only 4 of 300 instances (1.3%) have a usable installation on the four smaller models, and the rule of Stolfo et al. (2025) gives the same count (Table R1). Composed installations reach at most 6.3% joint success under the guard, against 50% to 75% for prompting on the same sets.

Table R1. Main claims with and without the judge.

ClaimGuard onlyWith judge
Usable installations, four smaller models (of 300)40
Same, with the Stolfo et al. (2025) rule40
Best composed installation, 2 instructions, six modelsat most 6.3%at most 5.4%
Prompting, same sets50% to 75%45% to 69%

The judge matters for one claim: that checker success overstates the number of usable settings. Of the 35 reinforcements that are usable under the guard, 3 remain usable with the judge. (The paper's judged count of 18 also includes 15 instructions that become usable only under the judge, because the judge lowers the unedited model's score as well.)

To test whether this claim depends on the particular judge, we regenerated the test answers of every guard-usable setting on the smaller models and scored them with two judge families. Table R2 shows the number that remain usable.

Table R2. Guard-usable settings that remain usable with each judge.

SettingPaper (Sonnet 5)Rerun, Sonnet 5Rerun, GPT-5.6-terra
Installations (4)010
Reinforcements (35)330

The Sonnet 5 rerun closely reproduces the paper's counts, and GPT-5.6-terra is stricter, retaining no setting. Under either judge family, most guard-usable settings fail once answer quality is assessed. The two judges also agree with each other on 84.7% to 89.0% of composition answers (κ = 0.58 to 0.64, Appendix K.3).

To address rater independence, five lab members who are not authors each rated 100 of the audited answers, blind to model, arm, instructions, and judge verdict. Table R3 compares their ratings with the judge and with the author raters on the same 100 answers.

Table R3. Agreement on the same 100 answers.

ComparisonAgreementCohen's κ
Outside raters (majority) vs. judge80.0%0.60
Author raters (majority) vs. judge76.0%0.52
Outside raters vs. author raters86.0%0.72

Outside raters agree with the judge at the level reported in the paper, and slightly more than the author raters did on the same answers. They also reproduce the audit's main contrast: among checker-passing answers, they find wrong facts in 13 of 34 steered answers (38%) against 6 of 32 prompt-only answers (19%).

Finally, a κ of 0.60 should be read against the agreement between individual humans on this task. The three authors reach Fleiss κ = 0.42 among themselves (Appendix K.2), and two single outside raters reach κ = 0.37. We have added the outside-rater and second-judge results to Section 7.

AuthorsOpenReview ↗

Response to Reviewer Ltjj (2/2)

Weaknesses

W2. Narrow scope, no seed replication, and a conclusion practitioners already hold

Response: The reviewer raises three related points: scope, seed replication, and whether the conclusion adds to what practitioners already assume.

On seeds, our directions involve no training step. Each is a mean of activations over 80 matched extraction prompts, drawn from 400 prompts split into 5 disjoint groups. The relevant sources of variation are therefore numerical noise and the choice of extraction prompts, and Appendix F controls for both: repeated unedited runs differ by a median of 0.0 to 0.5 points in compliance (Table 15), and directions extracted from the other four prompt groups move composition results by a standard deviation of 0.7 to 1.4 points. Greedy decoding removes sampling variance, so these controls account for the remaining variation.

On scope, the paper covers seven models from three families and includes 20 IFBench constraint types as a second constraint source. For this rebuttal, we added a second direction family: trained vectors, optimized so that the model reproduces its own compliant answers without the instruction in the prompt (25 instructions on Qwen2.5-7B, 5 on Llama-3.1-8B). Training raises the number of guard-usable installations from 1 to 7 of 30, but only 2 pass under Sonnet 5, 1 under GPT-5.6-terra, and 1 under both. The same pattern, higher compliance without preserved answer quality, appears with this direction family.

On the contribution, the conclusion that prompting is the more reliable option is a result the paper measures under stated conditions, with paired intervals; it is not assumed. The measurements show:

  1. Checker success overstates usable settings: of 35 guard-usable reinforcements, 3 stay usable with the judge.
  2. Even when every member works alone, prompting exceeds composed installations by 17.2 points [11.2, 23.5] at two instructions and 49.0 [36.5, 61.7] at three (Table 4).
  3. Several losses come from calibration rather than composition: per-set dose tuning recovers 20.0 points [11.9, 29.1] on Gemma-3-4B at four instructions, and a different composition rule exceeds the layer-separated hybrid by 22.1 points [2.7, 43.7].
  4. A generic retry without feedback gains 3.9 to 5.6 points, and detailed checker feedback adds a resolved gain on only one of three models.

These results tell a practitioner how much prompting leads, under which conditions steering helps, and which losses can be recovered by calibration, none of which is available from checker-only evaluation.

Questions

Q1. Would judge–audit agreement change with raters outside the author team?

Response: It does not decrease. On the same 100 answers, outside raters agree with the judge on 80.0% (κ = 0.60), compared with 76.0% (κ = 0.52) for the author raters (Table R3).

Q2. What happens to the headline counts under a second judge family?

Response: Under GPT-5.6-terra, 0 of the 4 guard-usable installations and 0 of the 35 guard-usable reinforcements remain usable; on regenerated answers, Sonnet 5 retains 1 and 3 (Table R2). Both judge families lead to the same conclusion.


We thank the reviewer again for these concerns. The installation result holds without any judge, and the judge-dependent claim holds under a second judge family and under raters outside the author team, who agree with the judge at 80.0% (κ = 0.60). We respectfully ask the reviewer to reconsider the score in light of these results.


References

  • Stolfo et al. "Improving Instruction-Following in Language Models through Activation Steering." ICLR, 2025.

Dear Reviewer Ltjj,

This is a kind reminder that our response above and the revised PDF are posted. It includes both checks you asked for: raters outside the author team and a second judge family.

We would be grateful if you could tell us whether the response addresses your concerns, and whether it changes your assessment. We are happy to answer any further question.

Dear Reviewer Ltjj,

We wanted to kindly follow up on our responses to your concerns, particularly the additional experiments with an independent judge family and external human raters. As approximately 12 hours remain in the discussion period, we would greatly appreciate it if you could review our responses and reconsider your assessment in light of the additional evidence.

Thank you for your time and consideration.

Reviewer_hqiCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies "activation steering" for modular instruction control. The idea is to create individual steering directions per IFEval-style constraint, freeze it, and do composition when a request needs several constraints; and that these directions could be "installation" (instruction the prompt omits) vs. "reinforcement" (prompt already states it). Through experiments authors find that few instructions can be installed reliably though directions which work alone seem to indeed compose; But also simply prompting the instructions seems stronger.

Critical strengths
  • The general eval setup makes sense to me. I def. echo that separating constraint compliance from instruction-blind answer quality is useful
  • I also like the installation vs. reinforcement distinction which helps identifying what's specified vs. underspecified.
  • I appreciate the honest reporting and careful discussion of a somewhat negative result :) The paper did a good job comparing against the obvious baseline (just state the instructions), and it traces losses to coverage, quality, and calibration.
Critical weaknesses
  • Motivation. In general I feel the paper could use a stronger motivation on why this is an important problem to study. it's not entirely clear to me when one would want steering over prompting for these instructions. Esp. because the work is based on IfEval which by construction contains constraints that can trivially be written in the prompt, so most likely I would expect people to generally just do prompt editing and expect satisfactory results if they are using a model with generally good instruction following capability. Maybe if the authors instead target at some use case in e.g. settings where the prompt can't be edited, or need persistent behaviors, or reducing prompt length, it might make the task more valuable, though that might require some change in eval metrics and even datasets.

  • Choice of instructions. Relatedly, I worry the choice of instructions partly stacks the deck. Many IFEval constraints are lexical or content-specific (e.g., ending with an exact sentence, using a given keyword), and it seems a bit farfetched from what steering supports the best (more style- or concept-level behaviors). Splitting results by constraint type (stylistic/format vs. content-bearing) would help readers see whether the limit is steering for instructions in general, or this particular instruction set.

  • Compositional assumption. I'm also a bit skeptical about the compositional assumption. Why should instruction directions be independent and additive, rather than some being much closer to each other than others? My intuition is that maybe many directions would mostly share a generic "follow a formatting instruction" component, rather than being independent and requiring decomposition and re-composition. I'd love to see at least some analysis of whether pair similarity predicts which compositions work.

  • Writing clarity. The writing could be made more intuitive. The abstract and intro stay quite abstract, and it took me a while to understand what exactly is being composed. A concrete running example early on would help a lot, e.g. moving Figu 3 forward could be helpful. The results sections are also pretty dense, I would recommend the authors to try explicating 1-2 takeaways per subsec so the readers can more easily get the takeaways.

Questions for authors

please see Motivation, Choice of instructions, and Compositional assumption in the weakness section

5 comments
AuthorsOpenReview ↗

Response to Reviewer hqiC (1/2)

We thank the reviewer for the thoughtful and constructive feedback. We appreciate the recognition of the separation between constraint compliance and instruction-blind answer quality, the distinction between installation and reinforcement, and the comparison against simply stating the instructions. We address each point below with new analyses and revised text.

Note on the revised PDF. New content is in blue. We also rewrote Sections 1 to 5, 7, and 8 for readability. These rewrites are in black and change no result, number, or claim. Some details moved to the appendices.

Weaknesses

W1. Motivation: when would one want steering over prompting?

Response: Thank you so much for this concern on when steering would be preferred to prompting, given that IFEval constraints can be written directly into the prompt. Our paper does not assume that steering is preferable. It tests a claim made in the steering literature, that frozen per-instruction directions provide modular control, and measures when that claim holds. The two prompt conditions we study correspond to two practical situations.

In installation, the instruction is absent from the prompt. This arises when the prompt cannot be edited, for example in an application that passes user text through unchanged, or when a rule must hold across every turn of a conversation. The question is whether steering can deliver the instruction at all. In reinforcement, the instruction is already in the prompt, but the model does not follow it reliably. The question is whether steering adds anything beyond prompting.

IFEval suits this test because its exact checkers let us separate whether an instruction was followed from whether the answer remains useful, and because instruction steering has been studied on the same constraints (Stolfo et al., 2025). Prompting is also far from saturated on this task: at two instructions, prompt-only joint valid success is 45.0% to 50.2% (Table 3), which leaves room for steering to help.

Our results identify where it does. For Gemma-3-4B pairs whose members have usable reinforcement settings, adding steering to the prompt reaches 41.1% [26.5, 53.9] against 19.8% [7.5, 33.7] for prompting alone, improving 9 of 10 sets. Tuning one dose per set recovers 20.0 points [11.9, 29.1] at four instructions, after which the steered arm matches prompting. Where steering must replace the prompt, even individually successful installations trail prompting by 17.2 points at two instructions. These measurements tell a practitioner when steering is worth deploying.

W2. Choice of instructions may favor prompting

Response: Thank you so much for the suggestion that many IFEval constraints are lexical or content-specific, while steering is better suited to style- and format-level behavior. To test this, we split the 25 constraint types into 17 format/style types (case, length, layout) and 8 content-bearing types (a specific word or phrase to include, avoid, or repeat).

If content-bearing constraints explained the low coverage, format/style constraints should fare better. Table R1 shows that they do under the guard (an automatic filter for degenerate text) alone: all 4 guard-usable installations are format/style. Once the instruction-blind judge is applied, however, none remains in either group.

Table R1. Usable settings by constraint type (four smaller models, 300 instances).

CountFormat/style (204)Content (96)
Usable installations, guard only40
Usable installations, guard + judge00
Usable reinforcements, guard only278
Usable reinforcements, guard + judge126

The limit therefore holds for the constraint type that steering should handle best. The IFBench transfer experiment points the same way, with 0 usable installations across 20 further constraint types under both the guard and the judge (Section 3).

AuthorsOpenReview ↗

Response to Reviewer hqiC (2/2)

Weaknesses

W3. Why should instruction directions be independent and additive?

Response: Thank you for asking whether instruction directions are independent, and for suggesting that they may share a generic formatting component. We treat additivity as a hypothesis rather than an assumption (Section 1), and several of the composition rules we test address shared components directly: orthogonalized composition, inverse Gram correction, and layer-separated composition (Section 4, Appendix J).

We ran the requested analysis. The reviewer's intuition is supported: directions overlap. The median absolute cosine between pair members is 0.43 on Qwen2.5-7B, 0.60 on Llama-3.1-8B, and 0.85 on Gemma-3-4B.

The question is whether this overlap predicts which sets compose. On 90 judged sets, ranked within each model and set size, the absolute Spearman correlation between overlap and joint valid success is at most 0.15 pooled and at most 0.26 within any model, and none is significant. The result is the same for the median cosine, the largest cosine, and the condition number of the Gram matrix. Removing the shared component also does not help: orthogonalized composition stays near the unedited model on the full pool (Table 12), and on Gemma-3-27B it gives 54.8% against 56.3% for fixed-norm addition at two instructions and 31.7% against 45.8% at three (Table 4).

Overlap is present, then, but it does not explain composition outcomes. This is consistent with the paper's account that coverage and calibration, not direction geometry, determine whether a set composes. We have added this analysis to Appendix J with a summary sentence in Section 4.

W4. Writing clarity

Response: The reviewer found it hard to see what is being composed from the abstract and introduction. We have moved the example figure (previously Figure 3, now Figure 2) to Section 1 and introduced it with a concrete example:

"Consider the task 'Who are the best starting MLB pitchers of all time?' with two instructions: end with an exact phrase, and write in English in all capital letters. A steered Qwen2.5-7B answer satisfies both checkers and the guard, but the pitcher names are garbled, and the blind judge and all three human raters reject it (Figure 2). Composing instruction directions therefore has to be judged on two outcomes: whether every instruction is followed, and whether the answer remains usable."

Each results subsection now opens with one or two sentences stating its finding. Section 4, for example, now begins: "Installations that succeed alone also compose, adding 35.0 points over the unedited model at two instructions, but they trail prompting by 17.2 points." A definitions table has also been added to Section 2.


We thank the reviewer again for these concerns. The new analyses show that the coverage limit holds for format/style constraints and that direction overlap, while present, does not predict composition; the revised introduction states the motivation and opens with a concrete example. We respectfully ask the reviewer to reconsider the score in light of these results and clarifications.


References

  • Stolfo et al. "Improving Instruction-Following in Language Models through Activation Steering." ICLR, 2025.

Thank authors for the detailed response! I appreciate the new experiments clarifying the applicability of this setup, but I do still think that the framing is a bit fuzzy, in that it's not clear when the authors actually expect compositionality to hold and to what extent. And I think the paper still reads more like a method paper than an analysis paper, which might be worth clarifying in future submissions to highlight the main takeaways.

Reviewer_TqDXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
3(+1)
Read review
Paper summary

The paper conducts a detailed analysis in the realm of activation steering of language models. Specifically, it asks to what extent activation steering offers truly modular control of LLMs: Can individual steers be combined to yield model behavior that simultaneously satisfies all intended steers? The paper conducts a thorough analysis over 7 models, instantiating and tabulating the rates of various failure modes involving multiple types of checks and judges, and studies the problem both under the no-instruction-only-steer and instruction+steer settings, along with relevant steer-free baseline settings. The paper surfaces interesting findings about the success rate of composition being dependent on coverage, individual checkers/judges providing misleading conclusions, and prompting remaining a strong baseline in modern LLMs.

Critical strengths
  1. The paper’s analysis is detailed and thorough. It instantiates and tabulates multiple different failure modes involving multiple types of checks and judges. The use of an instruction-blind task correctness judge is appreciated. It is an important finding that neglecting checking for task correctness and fluency
  2. The paper evaluates multiple language models of various families and sizes. This provides confidence that these findings generalize reasonably across a wide range of LLMs.
  3. It appears highly reproducible with released code, many implementation details and further analyses provided in the appendices.
Critical weaknesses
  1. It seems that the paper experiments with only one recipe for activation-steering while making strongly worded general claims about composability of steers. Although I understand that this is a typical (but somewhat bespoke) recipe, many other recipes also exist. It would be good to see how these findings carry over under one or two more recipes, even if these are restricted to smaller scale investigations. Also, a citation or two around lines 156-175 pointing to a well-cited precedent for this type of activation steer construction would be useful.
  2. I think the presentation of the paper can be vastly improved. The writing in the paper is in general slightly challenging to follow since it is reminscent of AI-generated overly verbose writing. The overall organization of the information into the various sections of the paper is fine, but internal to each section, information is often presented in a backwards manner where the first half of a section/paragraph/sentence reads very confusing until the reader reaches the second half. In addition to urging general improvement in presentation, I have two specific requests:
    • Improving confusing nomenclature: automatic guard, validity judge, quality judge, loose checker, strict checker, member checker, guard-qualified. I think the nomenclature can be made more intuitive. I found myself referring back to lines 138-155 over and over again while reading the paper to remind myself what exactly each of these quantities mean. Some of these steps (eg. it sounds like the guard is used to discard completely degenerate responses) can also be mentioned only as methodology footnotes and not receive attention in the form of the paper’s mathematical formulation.
    • The sections discussing experimental results although thorough, can be made terser so it is easier for the reader to parse the key takeaways without being distracted by unnecessary details (which can be moved to the appendix).
Questions for authors
  • line 174: The definition of a 'guard-qualified' response seems arbitrary. I did not understand why this concept is required.
  • line 160: what determines whether a dose is usable?
5 comments
AuthorsOpenReview ↗

Response to Reviewer TqDX (1/2)

We thank the reviewer TqDX for the thoughtful and constructive feedback. We appreciate the recognition of three aspects of the work: the instruction-blind judge for assessing task correctness, the evaluation across seven models of different families and sizes, and the release of code with implementation details in the appendices. In response, we ran additional experiments on the generality of our findings and revised the presentation of the methodology and results. We address each point below.

Note on the revised PDF. New content is in blue. We also rewrote Sections 1 to 5, 7, and 8 for readability. These rewrites are in black and change no result, number, or claim. Some details moved to the appendices.

Weaknesses

W1. Only one steering recipe, with general claims about composability

Response: Thank you for asking whether our conclusions depend on the specific steering recipe we used. To answer this, we now report results for three steering configurations that span two direction families: mean-difference directions and trained vectors.

Our direction is the difference between mean activations with and without an instruction. This construction follows Activation Addition (Turner et al., 2024), Contrastive Activation Addition (Rimsky et al., 2024), and instruction steering by Stolfo et al. (2025), and we have added these citations in Section 2. The paper also already evaluates the rule of Stolfo et al. (2025) in every test cell, including the composition experiments (Appendix D, Table 12). This rule uses the same mean-difference direction but sets the edit magnitude from the activation itself instead of from a tuned dose. It therefore tests whether our dose rule, rather than the direction, explains the results.

Before reporting the numbers, we briefly recall the two quality filters. The guard is an automatic filter that rejects degenerate text (fewer than 20 words, heavy repetition, or mid-sentence truncation). The judge is an instruction-blind model that sees only the task and the answer and decides whether the answer addresses the task. A setting is usable when it reaches the compliance target without reducing answer quality relative to the unedited model (defined in Q1).

Table R1 reports single-instruction results for both rules on the four smaller models (300 instances). The two rules give identical counts: four usable installations under the guard, and none once the judge is applied.

Table R1. Usable installations for two mean-difference rules (300 instances).

Quality filterOur dose ruleStolfo et al. rule
Guard only44
Guard + judge00

Because both rules share a direction, we also tested a different direction family. For each instruction, we trained a vector so that the model reproduces its own compliant answers without the instruction appearing in the prompt. The layer search, dose search, test prompts, guard, and judges are the same as in the paper. This covers all 25 instructions on Qwen2.5-7B and 5 on Llama-3.1-8B, giving 30 instruction settings.

The question here is whether a direction that is better at producing compliance also yields more usable installations. Training raises compliance as expected. On Qwen2.5-7B, valid compliance rises from 0.09 to 0.97 for the no-comma instruction, from 0.00 to 0.94 for the title instruction, and from 0.04 to 0.92 for bullet lists. Under the guard, usable installations increase from 1 to 7 (Table R2).

Table R2. Mean-difference directions and trained vectors (30 instruction settings).

Quality filterMean difference (paper)Trained vector
Guard only17
Guard + Sonnet 502
Guard + GPT-5.6-terra01
Guard + both judges01

Most of this gain does not survive the judges. Two settings pass under Sonnet 5 and one under GPT-5.6-terra; the one setting that passes both is the title instruction on Llama-3.1-8B (0.82 under GPT-5.6-terra). Beyond the seven guard-usable settings, 12 more raise valid compliance by at least 20 percentage points without becoming usable: 8 stay below the 0.80 target and 4 exceed a quality bound.

This result separates two factors. A better direction raises compliance, but higher compliance rarely comes with preserved answer quality. The limiting factor is therefore answer quality, and it appears with both direction families, not only with our original recipe.

The claims in the paper are scoped to match this evidence. Section 7 states that "the evidence supports conditional, not universal, composition," and the conclusion describes composition as feasible under identifiable conditions tied to coverage, answer quality, and calibration. The IFBench transfer experiment adds a second constraint source: across 20 constraint types, it yields no usable installations under either the guard or the judge.

AuthorsOpenReview ↗

Response to Reviewer TqDX (2/2)

Weaknesses

W2. Presentation: verbose style, confusing nomenclature, dense results

Response: Thank you for pointing out that Lines 138 to 155 introduce several terms for closely related evaluation steps, which made the results harder to follow. We have reduced these to four terms and restructured the results sections. We have also revised Sections 1 to 5 so that each paragraph states its point first.

The revised main text uses guard, judge, instruction checker, and usable. The guard checks basic response integrity, the judge assesses whether the answer addresses the task without seeing the instructions, and the instruction checker applies the IFEval strict check for one instruction. A setting is usable when it meets the three conditions described in Q1. The mapping from the previous terms is shown below.

Previous term(s)Revised termMeaning
Automatic guardGuardBasic response integrity checks
Validity judge, quality judgeJudgeInstruction-blind assessment of the answer
Strict checker, member checkerInstruction checkerIFEval strict check for one instruction
Loose checkerLoose checkerNamed only in the scoring comparison of Table 2
Guard-qualifiedUsableMeets the three conditions in Q1

The guard thresholds now appear in a footnote, so Equation 1 introduces only the validity term V and the checker term C_i. Each subsection of Sections 3 to 5 now opens with a one-sentence statement of its finding, for example: "Adding the judge removes all 4 usable installations and reduces usable reinforcements from 35 to 18 on the four smaller models." Per-model detail has moved to the appendices. No result or analysis has been removed.

Questions

Q1. Why is a "guard-qualified" response defined this way?

Response: The definition addresses a specific failure: an edit can raise compliance by damaging the answer. Shorter or truncated answers, for example, satisfy a constraint on the number of sentences more easily while no longer answering the task.

We therefore compare each edited model with the unedited model on the same prompts. A setting is usable only if all three conditions hold:

  1. Valid compliance is at least 0.80.
  2. Validity decreases by at most 5 absolute percentage points relative to the unedited model.
  3. Degeneration increases by at most 5 absolute percentage points relative to the unedited model.

The first condition requires the instruction to be followed; the second and third require that this does not come at the cost of answer quality.

The data in Section 3 show why these conditions are needed. Twenty-two installation instances pass the effect criterion, with a median gain in valid compliance of 59.8 percentage points. Yet 18 of these 22 still have no usable setting, because the gain comes with a loss in validity or a rise in degeneration. Measuring compliance alone would have counted all 22 as successes.

Q2. What determines whether a dose is usable?

Response: A dose is usable when it meets the three conditions in Q1, evaluated on development prompts only.

Calibration then proceeds in two stages. First, we evaluate six coarse doses at each of seven candidate layers and select the layer with the largest number of usable doses. Second, at that layer we search a finer dose grid, find the longest consecutive run of usable doses, and freeze its midpoint. The layer and dose are then fixed for all test evaluations, and the judge is applied only after this point, so no test outcome or judge verdict influences calibration (Appendix A).


We thank the reviewer again for these concerns. The new experiments show that the central finding holds across three steering configurations and two direction families, and the revised manuscript defines the evaluation criteria, terminology, and calibration procedure before they are used. We respectfully ask the reviewer to reconsider the score in light of these results.


References

  • Rimsky et al. "Steering Llama 2 via Contrastive Activation Addition." ACL, 2024.
  • Stolfo et al. "Improving Instruction-Following in Language Models through Activation Steering." ICLR, 2025.
  • Turner et al. "Steering Language Models With Activation Engineering." arXiv:2308.10248, 2024.

Dear Reviewer TqDX,

This is a kind reminder that our response above and the revised PDF are posted. It reports results for two more steering recipes and describes the revised presentation.

We would be grateful if you could tell us whether the response addresses your concerns, and whether it changes your assessment. We are happy to answer any further question.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Reminder to Reviewers and Overview of the Rebuttal

Dear Reviewers,

This is a reminder that our responses to each review are now posted, along with an overview of how the rebuttal addresses the main concerns.

We thank all reviewers for the concerns raised. We have posted individual responses with new experiments and revised text. Since the discussion period is closing, we summarize below how each main concern was addressed.


Every result listed here is reported in full in the individual responses.

Overview: main concerns and how the rebuttal addresses them

ConcernRaised byWhat we didResult
Conclusions may depend on one steering recipe or extraction methodTqDX, NAfY, T3UxAdded a second published rule (Stolfo et al., 2025) and a second direction family: trained vectors across 30 instruction settingsThe second rule gives the same 4 → 0 usable installations. Trained vectors raise guard-usable installations from 1 to 7, but only 1 passes both judges. The finding holds across 3 steering configurations from 2 direction families.
Headline numbers depend on one LLM judge validated by the authorsLtjjIdentified which claims need a judge, added a second judge family (GPT-5.6-terra), and repeated the audit with 5 raters who are not authorsThe installation result needs no judge (4 of 300 under the guard alone). Under GPT-5.6-terra, 0 of 4 guard-usable installations and 0 of 35 guard-usable reinforcements remain usable. Outside raters agree with the judge on 80.0% of answers (κ = 0.60), against 76.0% (κ = 0.52) for author raters on the same answers.
The IFEval instruction set may favor promptinghqiCSplit the 25 types into format/style (17) and content-bearing (8)All 4 guard-usable installations are format/style, and none survives the judge in either group. The limit holds for the constraint type steering should handle best.
Directions may not be independent or additivehqiCMeasured pairwise overlap and tested whether it predicts composition on 90 setsOverlap is present (median cosine 0.43 to 0.85) but does not predict which sets compose (|Spearman| ≤ 0.15 pooled, none significant). Removing the shared component does not help.
Results depend on the 0.80 success targetNAfYReported targets of 0.70 and 0.50Installation coverage stays below 5% at every target. The added members are all the same instruction, so lower targets produce no new composable sets.
Unclear motivation for steering over promptinghqiC, LtjjLinked installation and reinforcement to concrete deployment cases and reported where steering helpsAdding steering to the prompt reaches 41.1% against 19.8% for prompting alone on usable Gemma-3-4B pairs. Per-set dose tuning recovers 20.0 points. Even individually successful installations trail prompting by 17.2 points at two instructions and 49.0 at three.
Directions may come from single matched pairsT3UxClarified the extraction procedureEach direction is a mean over 80 prompts. Re-extracting from 4 other disjoint groups moves composition results by a standard deviation of 0.7 to 1.4 points.
No seed replicationLtjjExplained the controls that apply when directions are not trainedRepeated runs differ by a median of 0.0 to 0.5 points, and the extraction-seed controls above cover the remaining variation.
What kind of damage the judge catchesNAfYReported each of the 3 human audit questions separatelySteering mainly causes content drift: among checker-passing answers, wrong facts appear in 42% of steered answers against 22% of prompt-only answers. Broken writing is rare in both arms (5% against 0%).

What the revised paper establishes

  1. Coverage is the binding limit, and answer quality causes it. Better directions raise compliance, but few settings keep answer quality. This holds across steering rules, direction families, judge families, constraint types, and success targets.
  2. Composition works when its members work. On Gemma-3-27B, individually usable installations compose, improving joint valid success by 35.0 points over the unedited model. This gives practitioners a condition they can check before deployment.
  3. Checker-only evaluation overstates steering. Of 35 reinforcements that pass the checkers and the guard, 3 remain usable once an instruction-blind judge and human raters assess answer quality.
  4. Several losses come from calibration, not from composition. Per-set dose tuning recovers 20.0 points, which identifies where future steering methods can improve.

The conclusions now rest on evidence that does not depend on one steering rule, one judge, or the authors' own ratings. As the discussion period ends shortly, we kindly ask reviewers to review our responses and raise their scores accordingly. We thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.