acceptodds
Under review as a conference paper at ICLR 2027

Role-Consistent Attention Steering for Hierarchical Instruction Following

Abstract

Large language models often violate the instruction hierarchy by following lower-priority instructions that conflict with higher-priority ones. Attention steering can mitigate these violations by emphasizing higher-priority messages, but may also suppress lower-priority context needed for the task. Our analysis shows that this conflict–preservation trade-off depends strongly on head selection, with larger conflict benefits tending to incur larger preservation costs. We introduce **Ro**le-**C**onsistent **A**ttention **St**eering (**RoCAST**), a training-free method that selects a fixed subset of attention heads offline for role-based reweighting. RoCAST scores heads using conflict benefit across role-swapped instructions, role consistency, and preservation without conflict, with calibration requiring only 64 instruction pairs and 250 unlabeled paragraphs. Across five Llama and Qwen models spanning 3B–14B parameters, RoCAST improves IHEval conflict accuracy by 21.3–33.1 percentage points over native models and outperforms the evaluated inference-time baselines. It keeps aligned task execution within 1.3 points of native on every model, improves system-level instruction following, and incurs smaller capability losses than the baselines. Ablations show that role consistency and preservation improve both conflict and aligned performance over conflict-only selection. The same calibrated head sets transfer without recalibration to thinking-enabled inference and hierarchy-trained checkpoints, yielding further conflict gains while largely preserving aligned performance. Together, these results show that resolving instruction conflicts without sacrificing compatible behavior depends critically on which heads are steered, supporting offline head calibration as a lightweight complement to hierarchy-specific training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Instruction Hierarchy, Attention steering, Inference-time alignment, Head selection, Role conflicts

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LFXPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets instruction-hierarchy violations, where a model follows a user turn or a tool result over the system prompt. The authors note that the conflicting low-priority content is usually also task-relevant, so the method must resolve the conflict and preserve context at the same time. RoCAST is a training-free inference-time method whose distinguishing part is offline head selection. From role-swapped calibration variants, it locates the first-divergence token and measures single-head intervention effects, then computes three per-head statistics: a role-swapped benefit score, a role-response consistency sign from a leave-one-head-out consensus, and a preservation score from the no-conflict system-side margin. Heads are ranked by a standardized combination of the benefit and preservation scores, the top K=96 are steered at inference by a per-role multiplicative bias on attention logits with system set to +1 and tool set to -1 and beta=4, and the reweighting preserves each head's total attention mass. Calibration uses 64 instruction pairs and 250 paragraphs. Across five models the conflict metric improves by 21.3 to 33.1 points.

Critical strengths
  1. The stage-wise ablation in Table 4 is unusually informative: benefit-only selection on Qwen3-8B drops aligned behavior by 18.4 points, while adding the consistency filter gains 12.2 on conflict and 13.4 on aligned simultaneously, which is direct evidence that single-head benefits are not additive.
  2. The random-head control applies the same operator at the same strength and budget (8.5 versus 26.3 average points), which correctly separates the selection criterion from the intervention.
Critical weaknesses
  1. The abstract's claim of keeping aligned task execution within 1.3 points of native holds only for that one domain.
  2. The negative tool-role bias is not supported by ablation study.
Questions for authors
  1. What are the per-domain preservation bounds, and would the abstract claim be restated as domain-specific?
  2. What happens with the tool-role bias set to 0, holding K, beta and the normalization fixed?
  3. Is K=96 tuned per model or transferred, and how sensitive is the conflict metric over K from 32 to 192?
3 comments
AuthorsOpenReview ↗

Response to Reviewer #1

Thank you for the encouraging assessment of our ablation studies. We address your three questions below, including a new ablation of the negative tool-role bias.

Q.1 Domain-Specific Preservation

As reported in Table 1, the preservation claim applies specifically to IHEval's task-execution domain. Across the five models, the average aligned accuracy and its changes relative to Native are:

DomainNativeRoCAST (Δ)Δ range
Rule following76.683.6 (+7.0)+2.4 to +13.3
Task execution76.877.0 (+0.1)−0.6 to +1.3
Safety67.156.6 (−10.5)−42.2 to +2.7
Tool use43.722.2 (−21.5)−40.3 to −0.4
Overall67.361.8 (−5.5)−14.6 to +1.9

Despite these domain-specific regressions, RoCAST preserves aligned performance substantially better than IHDec and V-Simple. Since IHDec excludes tool-use tasks, we report both the comparable non-tool average and the overall IHEval average across five models:

MethodNon-tool (7 tasks)All domains (9 tasks)
Native74.067.3
IHDec65.4 (−8.6)—
V-Simple52.9 (−21.1)47.0 (−20.3)
RoCAST73.0 (−1.0)61.8 (−5.5)

RoCAST retains near-native aligned accuracy on the non-tool evaluation, with substantially smaller losses than IHDec and V-Simple. Beyond IHEval, RoCAST improves IFEval-sys by 5.6 points over Native on average and incurs smaller MATH-500 losses than both methods. Unlike these baselines, RoCAST also preserves Native MMLU accuracy by remaining inactive when no system or tool messages are present.

We will clarify the domain-specific scope in the abstract:

It limits aligned accuracy losses in IHEval's task-execution domain to at most 0.6 points across all five models, improves system-level instruction following, and incurs smaller capability losses than the baselines.

Q.2 Ablation of the Negative Tool-Role Bias

To assess the contribution of the negative tool-role bias, we removed only this term while keeping the selected heads, K=96K=96, β=4\beta=4, and normalization fixed. The results are provided in Common Table A.

Removing the term improves aligned tool-use accuracy by 8.1 points on Qwen3-8B and 12.3 points on Qwen3-14B, but reduces tool-conflict accuracy by 8.5 and 13.7 points, respectively. On Llama-3.1-8B, matched reruns show a 0.6-point aligned improvement with no change in conflict accuracy.

The ablation confirms that the negative tool-role bias plays an important role in tool-conflict resolution on Qwen3. Removing it partially recovers aligned accuracy but substantially reduces conflict accuracy, demonstrating that the term helps prevent models from following conflicting instructions in tool outputs. Moreover, the remaining aligned degradation indicates that tool suppression is not the sole source of preservation loss. These results highlight the importance of the negative tool-role bias in RoCAST, as it improves resistance to conflicting tool instructions.

AuthorsOpenReview ↗

Response to Reviewer #2

Q.3 Head-Budget Selection and Sensitivity

As detailed in Appendix A.2 and Table 6, we selected KK independently for each model using the same predefined stopping criterion on 800 held-out Beyond Oracle pairs. For all five models, the improvement from K=48K=48 to 9696 was significant, whereas the next step from 9696 to 144144 was not. Thus, the stopping rule independently selected K=96K=96 for every model, rather than transferring a budget chosen on one model to the others. We kept β=4\beta=4, λ=1\lambda=1, and the head ranking fixed throughout selection, without using any IHEval examples.

Head-budget sensitivity is presented in Figure 3 and Appendix C (Figure 6 and Table 10) on a separate 760-prompt IHEval subset. We evaluated K∈24,48,96,144,192,240K\in\\{24,48,96,144,192,240\\}, subject to each model's eligible-set size. Although conflict accuracy changes nonmonotonically as KK increases, the gains over Native persist across all evaluated feasible budgets. Overall, these results demonstrate that RoCAST's conflict improvements are robust across a range of head budgets, rather than depending on the specific choice of K=96K=96.

We clarified the preservation claim in the abstract and add the tool-role bias ablation to the appendix. We hope these clarifications and additional results address your concerns.

AuthorsOpenReview ↗

Gentle Follow-up on Author Response

Thank you again for your careful review and helpful feedback.

As the discussion period draws to a close, we wanted to briefly follow up on our rebuttal. We believe that our clarifications and additional experiments have addressed the main concerns raised in your review.

We would appreciate hearing whether these responses have sufficiently resolved your questions, or if there is anything further we could clarify.

Thank you again for your time and consideration.

Reviewer_aZB5OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Role-Consistent Attention Steering (RoCAST) for resolving conflict while preserving non-conflicting instructions. The paper argues that the conflict-preservation trade-off depends strongly on selecting the right subset of attention heads to steer. For selected heads, RoCAST shifts attention of query tokens towards higher-priority roles with role-specific intervention strengths. RoCAST selects attention heads using three complementary criteria. First, Behavioural Benefit Scoring (BBS) identifies heads that positively respond to conflict resolution interventions across role assignments, indicated by a median positive shift toward the system-side candidate irrespective of the configuration. Second, Role-Consistency Filtering (RCF) identifies heads whose role response is aligned with the aggregate response of other candidate heads, when a neutral paragraph is assigned different message roles. Third, System-only Preservation Scoring (SPS) measures how well a non-conflicting system-only instruction is followed under intervention. The final set of top k heads are selected as the subset of heads post BBS and RCF with high behavioural and preservation scores. The empirical evaluation compares RoCAST against other inference time intervention strategies on IHEval across 5 models. On all models, RoCAST reports higher conflict resolution accuracy than other inference-time methods. It improves aligned performance on the 2 Llama models and degrades it on the 3 Qwen models. Rule following scores are higher than the native model across the board. RoCAST's benefits transfer to Qwen models in thinking mode with think-off selected heads, and to a hierarchy-trained Llama model.

Critical strengths
  1. This paper discusses an important problem: conflict resolution while limiting degradation of performance on non-conflicting instructions. The proposed criteria for head selection are novel to the best of my knowledge.

  2. RoCAST achieves the highest average conflict accuracy among the evaluated inference-time methods, across 5 models on the IHEval benchmark. Furthermore, the empirical evaluation presents experiments with different thinking modes, hierarchy training, component ablations, and a discussion on latency.

Critical weaknesses
  1. The paper does not evaluate a simple attention steering baseline that applies the same role-based attention reweighting to all heads. While the selection of random heads and other head selection baselines help demonstrate the benefit of the proposed head selection method, it is unclear whether comparable conflict-preservation trade-off can be achieved by steering all heads with a suitable intervention strength (selected on the same held-out set as RoCAST). Since head selection is the primary contribution of this work, this comparison would help establish its need over a simpler intervention.
Questions for authors
  1. The baselines include alternate head selection strategies. How does RoCAST compare against a steering baseline without any head selection, where the intervention strength is selected on the same held-out dataset?

  2. How were β=4\beta = 4 and λ=1\lambda = 1 chosen for the experiments? How sensitive are the empirical results to these choices?

  3. The head selection interventions in Section 3 focus a single pair of conflicting instructions per sample. How can the selection strategy generalize to conversations with multiple simultaneous conflicts, along with several compatible instructions that need to be retained? Does the empirical evaluation include conversations with multiple simultanous conflicts? If yes, how does RoCAST perform on that subset?

5 comments
AuthorsOpenReview ↗

Response to Reviewer #1

We appreciate your suggestion to evaluate steering without head selection. This comparison directly tests RoCAST's central design choice, and we have supplemented it with sensitivity and multi-conflict analyses.

Q.1 Steering without Head Selection

To assess the necessity of head selection, we applied the same role-based operator to all attention heads. We selected β from 0.5,1,2,3,…,8\\{0.5,1,2,3,\ldots,8\\} using the same held-out data and stopping rule as RoCAST's budget selection. This yielded β=4 for Qwen3-8B and β=6 for Llama-3.1-8B.

ModelMethodβConflictAligned
Qwen3-8BNative025.277.6
All heads436.442.8
RoCAST451.868.9
Llama-3.1-8BNative012.653.4
All heads636.230.5
RoCAST445.754.2

RoCAST outperforms the strength-selected all-head baseline on both conflict and aligned accuracy. All-head steering particularly harms aligned task execution, reducing accuracy to 33.9 on Qwen3-8B and 5.0 on Llama, compared with 76.3 and 76.7 under RoCAST. This highlights the importance of selecting heads that improve conflict resolution without excessively suppressing task-relevant user information. These results show that tuning steering strength alone does not reproduce RoCAST's conflict–preservation performance under the evaluated procedure. The strength-selection results and the domain-level breakdown are provided in Appendix E.2 (Tables 14 and 15).

Q.2 Choice and Sensitivity of β and λ

We fixed β=4 and λ=1 across models before head calibration, without using IHEval for tuning. λ=1 gives equal weight to the standardized conflict-benefit and preservation scores. For β, we used PASTA [1] as a reference for steering strength. Its attention scaling factor α=0.01 corresponds to a relative logit shift of ln(100)≈4.6, motivating our choice of β=4. Only K was selected on 800 held-out Beyond Oracle pairs [2].

Common Tables B–C report post-hoc sensitivity analyses on the 760-prompt IHEval subset. The β sweep shows that substantial conflict gains persist across a range of intervention strengths, rather than depending on β=4 alone. Stronger steering can further improve conflict accuracy, but may also reduce aligned performance.

The λ sweep demonstrates the contribution of preservation-aware head ranking. Compared with λ=0, λ=1 improves both aligned and conflict accuracy on both models. Increasing λ further does not consistently improve the trade-off and can reduce conflict accuracy. Together, these results support our shared β=4 and λ=1 settings, showing substantial conflict gains across moderate steering strengths and improved preservation through head ranking without model-specific hyperparameter tuning.

AuthorsOpenReview ↗

Response to Reviewer #2

Q.3 Generalization Beyond Pairwise Conflicts

Although RoCAST calibrates heads using individual instruction pairs, IHEval includes two relevant rule-following subsets. Multi-constraint consists of single-turn prompts with two or three system constraints that are contradicted by the user in the conflict setting. Multi-turn contains conversations where both user turns conflict with system constraints. We reanalyzed the Table 1 outputs on these subsets:

Model\text{Model}Method\text{Method}Multi-constraintConflict\begin{array}{c}\text{Multi-constraint}\\\\\text{Conflict}\end{array}Aligned\text{Aligned}Multi-turnConflict\begin{array}{c}\text{Multi-turn}\\\\\text{Conflict}\end{array}Aligned\text{Aligned}
Llama-3.2-3BNative2.538.62.550.7
RoCAST29.255.931.964.7
Llama-3.1-8BNative1.348.73.860.4
RoCAST29.260.226.471.8
Qwen3-4BNative8.167.85.875.0
RoCAST43.272.038.484.9
Qwen3-8BNative5.974.67.480.8
RoCAST58.573.356.786.0
Qwen3-14BNative9.374.26.884.9
RoCAST61.976.748.789.6
AverageNative5.460.85.370.4
RoCAST44.467.640.479.4

RoCAST substantially improves conflict accuracy on every model in both subsets. Aligned accuracy also improves on all models in the multi-turn setting and on four of five models in the multi-constraint setting, with only a small regression on Qwen3-8B (74.6→73.3). RoCAST achieves the highest five-model aligned averages among the evaluated methods on both subsets (Appendix H.1, Table 20). These results show improved handling of multiple constraints and conflicts across turns while largely preserving performance on the corresponding non-conflicting inputs.

Task Execution in IHEval provides complementary evidence for preserving compatible task behavior. Its tasks require models to use user-provided content while following system-specified instructions, even when conflicting user requests are present. As shown in Table 1, RoCAST achieves the highest conflict and aligned task-execution accuracy among the evaluated inference-time methods on all five models, with aligned performance remaining within 1.3 points of Native.

Together, these results support generalization beyond pairwise calibration. While IHEval does not directly evaluate cases that mix conflicting and compatible lower-priority instructions within a single example, the results provide complementary evidence that RoCAST can handle multiple conflicts while retaining performance on compatible instructions and task-relevant user content.

We hope these additional results clarify the importance of selective head steering and address the questions regarding sensitivity and generalization. We have incorporated the new experiments and analyses into the paper.

[1] Zhang, Qingru, et al. "Tell your model where to attend: Post-hoc attention steering for llms." ICLR. 2024.

[2] Huang, Sian-Yao, et al. "Beyond Oracle: Verifier-Supervision for Instruction Hierarchy in Reasoning and Instruction-Tuned LLMs". Neurips. 2025.

AuthorsOpenReview ↗

Gentle Follow-up on Author Response

Thank you again for your thoughtful review and constructive suggestions.

As the discussion period draws to a close, we wanted to briefly follow up on our responses. We believe that the additional experiments and analyses have directly addressed the questions you raised.

We would be grateful if you could let us know whether our responses have sufficiently addressed your concerns, or if anything remains unclear.

Thank you again for your time and consideration.

Reviewer_jQmEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes Role-Consistent Attention Steering (RoCAST), a training-free method for improving hierarchical instruction following. RoCAST applies role-based attention reweighting to a fixed set of attention heads selected offline through Behavioral Benefit Scoring (BBS), Role-Consistency Filtering (RCF), and System-Only Preservation Scoring (SPS). Experiments on five Llama and Qwen models demonstrate substantial improvements in IHEval conflict accuracy over native inference and the evaluated inference-time baselines. The paper further investigates aligned instruction following, general capabilities, component contributions, head-budget sensitivity, transfer to thinking-enabled and hierarchy-trained models, and inference efficiency.

Critical strengths

1.Substantial and consistent improvements in conflict resolution. RoCAST improves IHEval conflict accuracy by 21.3–33.1 percentage points over native inference across five models without parameter updates. The random-head baseline uses the same steering operator, intervention strength, and head budget. The method also complements hierarchy training: applying RoCAST to SFT+DPO increases conflict accuracy from 64.9 to 74.2, with only a 0.3-point change in aligned performance (Table 3). Aligned rule following and IFEval-sys performance also improve across all five models.

2.Informative and transparent experimental analysis. The paper includes cumulative component ablations, head-budget sweeps, transfer experiments, and detailed analyses of selected heads. It explicitly reports domain-specific preservation regressions and the non-additivity of individual-head benefits (Appendices B–C). The efficiency evaluation also distinguishes single-request latency, matched-batch throughput, and maximum feasible batch capacity.

Critical weaknesses
  1. Preservation is uneven across instruction domains, with substantial regressions that remain insufficiently explained.

(a) Large aligned-performance regressions. While aligned rule following improves across all five models and aligned task execution remains within 1.3 percentage points of native performance, overall aligned IHEval accuracy decreases by 7.0, 8.7, and 14.6 points on Qwen3-4B, Qwen3-8B, and Qwen3-14B, respectively (Table 1). On Qwen3-14B, aligned safety accuracy decreases from 85.1 to 42.9, and aligned tool-use accuracy from 78.8 to 53.0. The authors acknowledge these limitations, but their magnitude raises concerns about the reliability of preservation-aware steering across domains.

(b) Incomplete role coverage in calibration. The head-selection procedure uses system/user instruction pairs and uncontested system anchors (Section 3), whereas the deployed operator also applies a fixed negative bias to tool-role tokens. Tool-role behavior is not directly evaluated by the selection objectives. This raises the question of whether the calibration objectives adequately capture tool-use preservation, particularly given the observed aligned tool-use regressions. The current experiments do not distinguish limitations of calibration coverage, the preservation surrogate, and the steering operator.

  1. The contribution of the head-selection criteria is not fully isolated.

(a) Candidate-set differences in the random-head comparison. The random-head baseline samples uniformly from all attention heads (Appendix A.4), whereas RoCAST restricts candidates to approximately the first 70% of layers and applies eligibility filtering (Appendix A.1). Consequently, the comparison establishes the benefit of the overall selection pipeline but does not isolate the contribution of its scoring and ranking criteria. Layer-matched random heads and random heads sampled from the eligible set would provide more informative controls.

(b) Attribution of the RCF improvement. The cumulative ablations demonstrate substantial improvements from adding RCF (Table 4). Appendix B further shows that RCF replaces 38–49 heads from the BBS-selected set and that individual-head benefits are not additive. While these results establish the empirical value of the RCF stage, they do not fully distinguish the proposed directional role-consistency criterion from more general effects of replacing interacting heads. A budget-matched random-replacement control, together with random selection within the RCF-eligible set, would help disentangle head replacement, eligibility filtering, and subsequent ranking.

  1. The selected operating point is not explicitly optimized for the conflict–preservation objective.

(a) Head-budget selection. The head budget is selected using held-out conflict accuracy and a sequential stopping rule (Appendix A.2), rather than a joint conflict–preservation criterion. Table 6 shows that this rule can terminate before later budgets with higher reported validation accuracy: for Llama-3.1-8B, conflict accuracy is 39.1 at K = 96 versus 47.4 at K = 192. On the reported IHEval evaluation, Qwen3-14B also achieves higher conflict and aligned averages at K = 48 than at the selected K = 96 (50.7/69.3 versus 49.3/64.7; Table 12). These observations do not invalidate the predefined stopping procedure, but question whether it identifies the most appropriate operating point for the stated objective.

(b) Steering-strength sensitivity and statistical uncertainty. The experiments fix β = 4 and λ = 1 across models, while the reported sensitivity analyses primarily vary the head budget. Although the paper provides a budget sweep for RoCAST, the conflict–preservation trade-off under alternative steering strengths and preservation weights remains insufficiently characterized, and the evaluated steering baselines are not compared across corresponding trade-off curves. Furthermore, the main IHEval results lack reported confidence intervals, and the number of random-head draws is not specified, limiting assessment of the stability of the comparisons.

Questions for authors

1.Preservation failures. What explains the substantial aligned safety and tool-use regressions on Qwen3 models? How do the results change when tool-role examples are included in calibration or when tool-token suppression is disabled?

2.Head-selection controls. How does RoCAST compare with layer-matched random heads, random heads from the eligible set, and a budget-matched random-replacement control for RCF, using the same operator, intervention strength, and head budget? Can variability across random seeds be reported?

3.Operating-point selection. How were β and λ selected, and were any IHEval evaluation examples used in this process? Would selecting K using a joint conflict–preservation objective change the deployed configuration? Can the authors report conflict–aligned trade-off curves over β for RoCAST and the relevant steering baselines?

4.Evaluation and reporting. Since RoCAST leaves attention unchanged on MMLU in the reported setting (Section 4.1), how should unchanged MMLU accuracy be interpreted as evidence of capability preservation under active steering? Could the authors also clarify the total data requirements for head calibration and budget selection, distinguishing the 64 instruction pairs used for head selection from the additional 800 held-out pairs used to select K?

11 comments
AuthorsOpenReview ↗

Response to Reviewer #1

Thank you for your careful and detailed review. Your comments helped us identify where further evidence and clearer explanations are needed, particularly regarding preservation and head selection.

Q1. Preservation Failures and Calibration Coverage

(1) Aligned Tool-Use Regression on Qwen3.

The regression is partly explained by how chat templates place tool definitions. Qwen places tool schemas and call formats in a system turn even when no system message is provided, whereas Llama places them in the first user turn. Since RoCAST follows template-assigned roles, Qwen's tool definitions receive system promotion even on aligned inputs.

To examine this effect, we tested two modifications while keeping the model input and negative tool bias unchanged: excluding tool definitions from system promotion and removing system promotion entirely.

ModelSettingRole biasesTool conflictTool aligned
Qwen3-4BNative(0, 0, 0)6.368.1
RoCAST(+β, 0, −β)13.827.8
Tool def excluded(+β†, 0, −β)32.346.5
Sys removed(0, 0, −β)55.970.1
Qwen3-8BNative(0, 0, 0)4.766.5
RoCAST(+β, 0, −β)20.826.8
Tool def excluded(+β†, 0, −β)44.746.1
Sys removed(0, 0, −β)54.563.7
Qwen3-14BNative(0, 0, 0)2.778.8
RoCAST(+β, 0, −β)16.853.0
Tool def excluded(+β†, 0, −β)43.681.7
Sys removed(0, 0, −β)26.584.4
† System promotion excludes tool-definition tokens

Excluding tool definitions improves both metrics on all three Qwen models. Notably, Qwen3-14B achieves higher conflict accuracy than when system promotion is removed entirely (43.6 vs. 26.5), while recovering aligned accuracy beyond Native. Removing system promotion yields larger gains on Qwen3-4B and Qwen3-8B. These results identify template-induced promotion of tool definitions as an important contributor to aligned tool-use degradation.

(2) Disabling the Negative Tool-Role Bias.

Common Table A separately isolates tool suppression. Removing it recovers 8.1 and 12.3 aligned points on Qwen3-8B and Qwen3-14B, respectively, but reduces tool-conflict accuracy by 8.5 and 13.7 points. On Llama, paired reruns show a 0.6-point aligned gain with unchanged conflict accuracy. Aligned tool use also remains below Native after removing suppression on both Qwen models.

Together, these ablations distinguish two effects: tool suppression contributes to conflict resistance on Qwen3, while promoting tool definitions as system content incurs an additional preservation cost. Removing system promotion improves IHEval's primarily user–tool conflicts, but also disables system-priority steering when all three roles are present. Our main experiments therefore use the same general operator across domains rather than a tool-specific configuration tailored to IHEval.

Importantly, selectively excluding tool definitions improves both metrics while keeping system-priority steering active for other system tokens. This suggests that RoCAST's tool-use preservation can be improved by refining the promotion boundaries without changing its calibrated head selection or general role-based steering design.

AuthorsOpenReview ↗

Response to Reviewer #2

(3) Including Tool-Role Examples in Calibration.

To examine whether broader role coverage improves preservation, we constructed user–tool versions of the six calibration variants. We call this extension Tool-Augmented Calibration (Tool-Cal). It combines the original system–user scores with their user–tool counterparts:

ETC=h∈E:r^hUT>0\mathcal{E}_{\mathrm{TC}} = \\{h \in \mathcal{E} : \hat{r}_h^{\mathrm{UT}} > 0\\}

bh′=min⁡(bh,bhUT),ph′=min⁡(ph,phUT)b'_h = \min(b_h, b_h^{\mathrm{UT}}), \qquad p'_h = \min(p_h, p_h^{\mathrm{UT}})

scoreh′=zETC(bh′)+λzETC(ph′),h∈ETC\mathrm{score}'_h = z_{\mathcal{E}_{\mathrm{TC}}}(b'_h) + \lambda z_{\mathcal{E}_{\mathrm{TC}}}(p'_h), \quad h \in \mathcal{E}_{\mathrm{TC}}

The minimum favors heads with benefits and preservation across both role conflicts, while the additional eligibility condition requires positive shared user–tool benefit. The original role-consistency filter and all other settings, including K, β, and λ, remain unchanged.

ModelCalibrationRFTESafetyToolOverall
Qwen3-8BNative27.1 / 86.345.0 / 76.914.3 / 81.44.7 / 66.525.2 / 77.6
Tool-Cal47.5 / 86.859.5 / 77.317.5 / 73.026.4 / 60.240.1 / 74.7
RoCAST74.7 / 88.767.9 / 76.335.7 / 80.020.8 / 26.851.8 / 68.9
Llama-3.1-8BNative17.8 / 67.75.7 / 76.228.4 / 55.21.9 / 3.012.6 / 53.4
Tool-Cal41.4 / 75.648.8 / 76.943.2 / 45.011.1 / 2.737.5 / 53.0
RoCAST51.0 / 77.560.6 / 76.751.3 / 49.612.5 / 1.645.7 / 54.2

Full IHEval accuracy (%). Each entry reports conflict / aligned

Tool-Cal substantially improves Qwen3-8B tool preservation, raising tool aligned accuracy from 26.8 to 60.2 and overall aligned accuracy from 68.9 to 74.7. However, RoCAST achieves 11.7 points higher overall conflict accuracy and outperforms Tool-Cal in all three non-tool conflict domains. On Llama, RoCAST achieves higher conflict accuracy across all four domains and a higher overall aligned average.

Importantly, RoCAST's system–user calibration already generalizes to user–tool conflicts without tool-specific examples, improving tool-conflict accuracy over Native from 4.7 to 20.8 on Qwen3-8B and from 1.9 to 12.5 on Llama. Explicitly incorporating tool-role calibration improves tool preservation in some cases but weakens conflict resolution in other domains. These results support our original calibration strategy, which provides stronger overall conflict resolution while retaining cross-role generalization.

AuthorsOpenReview ↗

Response to Reviewer #3

Q2. Head-Selection Controls

To isolate the contributions of head selection, we added three random controls (R1–R3). The paper's random-head baseline used seed 42. We extended the evaluation to seeds 41–43 to assess variability. All controls use the same operator, β=4, and K=96 on full IHEval.

R1 matches RoCAST's layer distribution. R2 samples uniformly from the RCF-eligible set. R3 randomly replaces the same number of heads rejected by RCF from the BBS top-96 (38 for Qwen3-8B and 46 for Llama-3.1-8B). It then randomly refills the removed heads from outside the original BBS set, using uniform sampling without replacement and imposing no layer or eligibility restrictions.

ModelSelectionConflictAligned
Qwen3-8BUniform random36.6 ± 3.773.1 ± 2.0
R1: Layer-matched33.3 ± 5.373.3 ± 1.9
R2: Eligible-set random42.4 ± 3.769.6 ± 3.6
R3: Random replacement39.5 ± 8.056.8 ± 10.3
BBS + RCF50.166.3
RoCAST51.868.9
Llama-3.1-8BUniform random21.7 ± 7.352.3 ± 1.9
R1: Layer-matched21.6 ± 9.754.2 ± 2.1
R2: Eligible-set random34.9 ± 6.953.8 ± 1.6
R3: Random replacement42.8 ± 4.856.0 ± 5.3
BBS + RCF45.752.6
RoCAST45.754.2

Randomized controls report mean ± standard deviation over three seeds. Domain-level results are reported in Appendix E.1 (Table 13).

R1 performs similarly to uniform random selection, indicating that layer distribution alone does not explain RoCAST's gains. R2 improves conflict accuracy over uniform random but remains below BBS+RCF by 7.7 points on Qwen3-8B and 10.8 points on Llama, supporting the value of benefit-based ranking within the eligible set. R3 also achieves lower mean conflict accuracy than BBS+RCF on both models. This suggests that arbitrary head replacement does not reproduce the RCF-stage conflict gains.

Finally, SPS further improves aligned accuracy over BBS+RCF on both models without reducing conflict accuracy (Table 4 in Paper). Together, these controls show that RoCAST's gains go beyond layer distribution and eligibility alone, with calibrated head selection achieving higher mean conflict accuracy than all tested random alternatives on both models.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common Experimental Results

We provide Tables A–C as shared references for the responses below, covering the tool-role bias ablation and sensitivity analyses of the intervention strength and preservation weight.

Table A. Disabling the Negative Tool-Role Bias

We remove only the tool-role bias, keeping the selected heads, K=96, β=4, and normalization fixed. Biases are ordered as (system, user, tool).

ModelSettingRole biasesTool conflictTool aligned
Qwen3-8BNative(0, 0, 0)4.766.5
RoCAST(+β, 0, −β)20.826.8
Tool term removed(+β, 0, 0)12.334.9
Qwen3-14BNative(0, 0, 0)2.778.8
RoCAST(+β, 0, −β)16.853.0
Tool term removed(+β, 0, 0)3.165.3
Llama-3.1-8BNative(0, 0, 0)1.93.0
RoCAST(+β, 0, −β)12.51.6
Tool term removed(+β, 0, 0)12.52.2

Table B. Sensitivity to Inference-Time β

On the 760-prompt IHEval subset used for K sensitivity, we fix the 96 heads selected at β=4 (λ=1) and vary only inference-time β. Figure 7(a)–(b) in Appendix F.1 plots these results.

ModelMetricβ=0β=1β=2β=3β=4β=5β=6β=7β=8β=9β=10
Qwen3-8BConflict25.538.249.251.051.854.555.356.956.755.755.8
Aligned78.277.475.769.969.268.768.369.468.567.666.9
Llama-3.1-8BConflict12.421.536.143.247.449.249.147.147.347.847.2
Aligned54.554.554.955.055.453.651.850.751.151.649.9
AuthorsOpenReview ↗

Summary of Our Rebuttal

We sincerely thank all reviewers for their constructive feedback. Below, we summarize the key strengths recognized in the reviews and the concerns addressed during the rebuttal.

ㅤ

Highlighted Strengths

  • Strong and Consistent Effectiveness: All reviewers recognized RoCAST's substantial conflict-resolution gains across five models, outperforming the evaluated inference-time baselines.

  • Novelty and Motivation: Reviewer aZB5 highlighted the novelty of our head-selection criteria and the importance of balancing conflict resolution with preservation.

  • Comprehensive Experimental Analysis: All reviewers highlighted the strengths of our experimental evaluation, including informative component ablations, controlled comparisons, and transfer to thinking-enabled and hierarchy-trained models.

ㅤ

Addressed Concerns

  • Effectiveness of Head Selection (Reviewers jQmE and aZB5): We added an all-head steering baseline and four randomized controls (R1–R4). The results demonstrate the benefits of calibrated head selection over all-head steering and random alternatives, with further evidence supporting RCF's directional criterion and SPS's preservation contribution.

  • Preservation and Tool-Use Behavior (Reviewers jQmE and LFXP): Additional ablations distinguish tool-role suppression from template-induced promotion of tool definitions. Selectively excluding these definitions improves both conflict and aligned accuracy across all three Qwen models. We also clarified domain-specific preservation and the remaining safety limitation associated with calibration coverage.

  • Sensitivity and Generalization (Reviewers jQmE, aZB5, and LFXP): We added hyperparameter sensitivity analyses, joint budget selection, and multi-constraint and multi-turn evaluations, providing further evidence of robust conflict gains and generalization beyond pairwise calibration without IHEval-based tuning.

  • Evaluation and Reporting (Reviewer jQmE): We added random-seed variability and bootstrap confidence intervals, and clarified calibration data requirements and capability-preservation evaluations.

Together, these additional experiments provide stronger evidence for RoCAST's central contribution and address the reviewers' main technical questions. We have incorporated the new results, clarifications, and discussion of remaining limitations into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.