acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing Refusal Failures Under Tool Schemas

Abstract

LLMs are aligned to refuse harmful requests, but they often comply with the same requests once given a tool schema. This gap is well documented, and training on tool-use data can reduce it, but what happens inside the model remains largely unexplored. We show that the failure lies not in recognizing harm but in acting on it. To read the tool decision, we force the model to start a tool call and compare how likely it is to name a decline tool against the offered execution tools, a quantity we call the decision margin. Across four instruction-tuned models from three families, a harm direction stays linearly decodable under the schema, and steering the last request token along it raises the decision margin beyond random directions, yet the action changes on only some models. We then ask what safety training changes, and fine-tune each model on 1,000 requests to refuse, with and without the tool schema. Training inside the schema lowers harmful tool calls, while training without it largely fails or stops the model from calling tools at all. A smaller test suggests the same split between recognizing harm and acting on it under JSON, Shakespeare and list output styles. Safety evaluation and training for tool-using agents therefore need to happen inside the tool format. More broadly, our findings suggest that safety alignment is bound to the format it was trained in, which calls for alignment methods whose safety transfers across formats.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, refusal, mechanistic interpretability, safety alignment, supervised fine-tuning, activation steering, tool use

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WQvmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why LLMs that refuse harmful requests in chat often execute the same requests once given a tool schema. The authors argue that the failure lies not in recognizing harm but in acting on it, and support this with three lines of evidence on four models. First, harmfulness remains linearly decodable under the tool schema. Second, steering along the harm direction shifts the model’s preference toward a decline tool on all four models, but the actual action changes only on three of them; Llama-3.1-8B keeps executing almost every harmful request. Third, safety fine-tuning inside the tool schema reduces harmful tool calls, whereas the same training in plain chat either leaves harmful calls near the base rate or makes the models stop calling tools. The authors conclude that safety alignment is tied to the format it was trained in.

Critical strengths

The authors show that (1) under tool schemas, models still linearly encode harm, and steering along this harm direction shifts the tool decision, yet Llama-3.1-8B keeps executing harmful requests.

(2) Safety training reduces harmful tool calls only when done inside the schema, after which the harm direction can also drive Llama’s action.

Critical weaknesses

(1) I believed most of the aspects mentioned in this paper are already established by Aziz et al. (2026) and Wu et al. (2026a), and Pan et al. (2026) show on the same checkpoints that tool schemas weaken refusal signals, yet their SafeKeep fix is not compared.

(2) On Llama, the harm probe does not beat a simple word-count baseline. Steering only moves Llama’s margin from about −9 to −6, and it never crosses zero, so it is expected that the action does not change. The steering also raises benign refusals, so it is not specific to harm. Finally, the chat-trained arm is poorly designed and failed its own success check.

Questions for authors

See weakness, and also: (1) How does this work differ from Pan et al. (2026), Aziz et al. (2026) and Wu et al. (2026a) and any comparisons with them?

(2) Does steering with the refusal direction at the last prompt position change Llama’s tool action, which would show whether the failure lies before or after the refusal signal?

(3) Does Llama’s margin cross zero under other prefixes, given that Table 11 shows prefix offsets of 5–10 nats?

(4) Why does Appendix D report Llama’s token-steered chat refusal at α=+8 as 73% (“within noise”), while Table 23 reports 35/40 (87.5%)?

(5) Why do Appendix D and Appendix I report different chat steering results for Ministral (48%→90% vs. 14→31 of 40)?

(6) Which system prompt (with or without the refusal license in Table 3) and which decoding settings do the main evaluations use?

References:

Tool Specifications Matter (Pan et al., 2026)

Low-Resource Safety Failures Are Action Failures, Not Representation Failures (Aziz et al., 2026)

Knowing without Acting (Wu et al., 2026a)

5 comments
AuthorsOpenReview ↗

Response to Reviewer WQvm (1/2)

We thank the reviewer for the specific questions and for finding the inconsistencies between appendices, which we have corrected.

W1 and Q1. Differences from Pan et al., Aziz et al. and Wu et al.

We agree that prior work already shows harm can be decodable while the model does not refuse. We add two tests that this work does not run. We test whether adding the harm direction changes which tool the model calls, and we repeat this after safety training with and without the tool list in the training data. Pan et al. (2026) probe at the final prompt token and steer against a schema direction. Like them, we find that decodability falls at that token under the schema, but at the last request token our logistic probe stays within .03 AUROC of chat (Table 12). Aziz et al. (2026) study the same gap in low-resource languages, and Wu et al. (2026a) separate recognition and execution axes in chat jailbreaks, while we test whether the harm direction changes the tool call before and after training.

SafeKeep is an inference-time fix, and we use our training arms to measure what training changes, not as a competing defence. We still ran SafeKeep on 113 matched pairs. On Llama and Qwen2.5, 59% and 31% of its safety judgments did not follow the YES/NO format, which a rule we set before the runs excludes, so the table covers Ministral and Qwen3.

ModelMethodHarmful executedBenign executedSelective pairs
Ministral-8BBase10011313
Ministral-8BSafeKeep629230
Ministral-8BDecline-call training4611064
Qwen3-8BBase449552
Qwen3-8BSafeKeep126655
Qwen3-8BDecline-call training237350

Counts are out of 113, and a selective pair has its benign request executed and its harmful request not. On Ministral our training gives more selective pairs than SafeKeep, and on Qwen3 neither differs much from the base model. This comparison is in this response only.

W2a. Llama's probe against the word-count baseline

We agree that on Llama the submitted probe did not clearly beat word counts. That probe was the projection onto the mean-difference direction we steer with, which is not the standard test of decodability. The revision adds a logistic-regression probe under the same held-out folds, and at the steering layer it beats word counts and character n-grams on all four models (Table 12). On Llama its AUROC is .903, against .755 for word counts and .825 for character n-grams, and the smallest gap over character n-grams is .044 [.004, .085], on Ministral.

W2b. Llama's negative margin and unchanged action

We agree that a margin below zero fits Llama continuing to execute, and the revision says this directly. We report the margin because execution counts cannot tell apart a direction that leaves the margin unchanged from one that raises it but not above zero, and Llama is the second case. Adding the direction increases Llama's margin by about as much as Ministral's, but before steering Llama's mean margin is -9.2 and Ministral's is -2.8 (Table 18). After training inside the schema, Llama's mean harmful margin is +0.2 (Table 33), and the same base direction then increases harmful decline calls on all four seeds, which it barely did before training (Figure 3b, Table 34). Training also increases how much the margin changes per steering unit, so the higher starting margin alone does not explain this.

AuthorsOpenReview ↗

Response to Reviewer WQvm (2/2)

W2c. Benign refusals under steering

We agree that steering also raises benign refusals. We add the harm direction to every request, benign ones included, and since the direction is the difference between the mean activations of harmful and benign requests, adding it moves a benign request's activations toward those of harmful requests. So we expect benign refusals to rise as well, and this alone does not show that the direction is unrelated to harm. A stricter test is whether steering raises the harmful margin more than the benign one. On Llama the two rise by similar amounts, and on the other three models the harmful margin rises more (Table 18), but this is weak evidence, since on Qwen3 it does not hold on the AgentHarm pairs and Ministral's steering layer was chosen on the same pairs. So we do not claim that the direction encodes harm alone, and the Limitations section of the revision states this. Our main result on Llama does not depend on it, because there steering does not change the tool call for harmful or benign requests.

W2d. The chat-trained arm

We agree that the original chat arm was poorly designed, since it also changed the system prompt and the benign targets and failed its own check. The revision replaces it with the same prose-refusal training with and without the tool list, with nothing else changed, and a control that checks whether refusal was learned in the training format. All models are evaluated with the tool schema, and the table gives the share of harmful requests executed (Table 2).

ModelBefore trainingTrained with tool listTrained without tool listRefusal-learning control
Llama-3.1-8B96%34%91%Passes at one of two seeds
Mistral-Small (22B)73%27%76%Passes at the only seed
Llama-3.2-3B91%39%58%Fails at both seeds
Ministral-8B89%63%72%Passes at both seeds

On Llama and Mistral-Small the control passes, yet training without the tool list leaves harmful calls near the untrained rate. The other models do not give a clear result, since the Ministral gap is not significant (p = .087), Qwen2.5 does not learn the refusal, and Qwen3 already refuses 97 of 113 in that format before training. So we limit this finding to Llama and Mistral-Small (Section 5.2, Appendix I).

Q2. Steering with the refusal direction at the last prompt position

No. We added a chat harmful-versus-benign contrast at the final prompt token, where Zhao et al. measure refusal, and Llama still executed 39 of 40 harmful requests at +4 and +8, the same as without steering. Added at the last request token, it changed 39 to 37 (p = .5, Appendix D). So this test cannot say whether the failure is before or after the refusal signal, and it covers one contrast at one position, without Arditi et al.'s search over layers and positions.

Q3. Llama's margin under other prefixes

No. Without steering, the bare and the longer prefix give nearly the same harmful margins on Llama (-9.0 and -9.2, per-request Spearman .99), and the same 1 of 40 requests is positive under both. The 5 to 10 nat offsets in the submitted table are for Qwen3. These Llama values come from our result files and are not in the paper.

Q4. 73% in Appendix D against 35/40 in Table 23

They come from different directions. Appendix D adds the tool-fitted direction in chat at +8, which raises refusal from 26 to 29 of 40 (73%). The style study adds a direction fitted in each format, which in chat raises refusal from 25 to 35 of 40. The revised appendix names both (Appendix D, Table 22).

Q5. Ministral's 48% to 90% against 14 to 31 of 40

This was a reporting error. Appendix D counts safety refusals and capability declines at +4 (19 to 36 of 40, 48% to 90%), while the old Appendix I counted safety refusals alone at +8. The revision removes that summary from the fourth-model appendix (now Appendix H) and points to Table 22.

Q6. System prompt and decoding

The main evaluations use the system prompt without the refusal license, which is the domain prompt of Cartagena and Teixeira (2026), unchanged. It does not permit refusal or mention the decline tool, and only the decline tool's own description allows declining. The refusal-license condition appears only in the factorial table (Table 8). All free generation is greedy, with 128 new tokens for the original training evaluations and 256 for the steering runs, the two added models and the name-first Qwen retrain (Appendix A).

Reviewer_WQvmOpenReview ↗

Follow-up Questions

Thanks for the authors' engagement, I noticed the updated paper and your responses which solve some of my questions. Yet following the new updates, I still have few questions:

(1) For casual intervention, I think decision margin is just a readout, not the essential mechanism: MdecM_{dec} is calculated and analyzed from the output logits rather than model's internal computation path. So explanations like Llama doesn't refuse because the margin is low, so training to increase the margin helps refusal is useful, but this is more like describing the phenomenon using logit space's language. So I don't know what benefits does the training give: changes in MLP? attention? or just decision bias?

(2) For Llama, its refusal behavior may be affected by the template (https://huggingface.co/dropbox-dash/Llama-3.1-8B-Instruct_nvfp4_weights_calib_demo/blob/main/chat_template.jinja) where it says "Given the following functions, please respond with a JSON for a function call" unlike Qwen's for example use "may". Would this explains the phenomenon?

Reviewer_WKjKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies harmful request refusal when tool-call schemas enter the picture. The work finds that steering a harm direction usually but not always helps reduce harmful tool calls, but that this often comes at the expense of benign requests or doesn't even work, in the case of Llama 3.1 8b. Training is sensitive to the format used, both in this case of tool schemas and in a few others like JSON.

Critical strengths
  1. That the harm direction doesn't need to relearned after the safety training is mildly interesting
  2. Studying other format styles allows for some more robust conjectures about brittleness of safety training.
Critical weaknesses

Presentation

The prose has extremely strong Claude voice and is hard to parse in numerous paragraphs. This impaired my understanding to the point that in such sections my furrowed brow fails to break through. The overall story required substantial inference on my part, while more low level accounts were repetitive in the work.

"Section 4 shows that every model recognizes harm under the schema" makes it sound like the models are recognizing the harmful requests but not rejecting harm, which is the opposite of the finding.

Shallow analysis

For a science paper, I find the analysis shallow. The refusal vector is mentioned in related work but not studied. Additionally, the treatment of refusals is focused on a decline tool and a 50% threshold for "decision margin", which seems most salient just for greedy decoding, and as far as I can tell is the only way in which one can find a distinction for "the action does not always follow" (basically just saying the margin moved up but remained negative?). The presence of a distinct refusal tool/token also enables control and explicit tradeoffs, a treatment of which would enhance the black-and-white picture of benign refusals (at the expense of statistical power?).

Only a few models are studied (many on the old side, like Llama 3.1, Ministral, and GPT-4o), so very little insight is provided from the difference in response between them, and, as noted already in the paper, there's not an explanation of the differences.

Most of the results are totally unsurprising and/or not useful/informative, such as that safety training with tools helps with unsafe tool calls (§5), and §5.2 that if one [rigidly] trains a model with a single decline tool, removing it "from the menu" tanks refusal. Another one is that the decision and free-generation tool call behaviour correspond (L203). The more basic finding that chat safety can be brittle (e.g., in agentic settings) already seems covered by the related work (L367).

Questions for authors
  1. Which Claude was used for the work, like the judging?
  2. Do you have any hypotheses for why the models behave differently—something about how they were trained?
  3. "We consider three points at which the failure could enter. The format change could erase the representation of harm, it could keep the representation but cut it off from the decision, or it could let the representation reach the decision while the decision still stays on execute." I don't understand what the latter two really mean, though I think it's just a way of saying greedy decoding. Can you confirm/rearticulate?
7 comments
AuthorsOpenReview ↗

Response to Reviewer WKjK (1/2)

We thank the reviewer for the detailed review and for the questions about the decision margin, which we answer with new analyses below.

W1. Presentation and "Claude voice"

We agree that the prose was hard to follow, and that sentences like "every model recognizes harm under the schema" claimed more than what we measured. In this study, we find that a linear probe separates harmful from benign requests, and this does not show that the model judges or recognizes them as harmful. In the revision we have rewritten the abstract and main text in plain language so that each result says what was measured and what changed. We have also moved the detailed counts into Figures 2 and 3 and Table 2.

W2. The refusal vector is not studied

We ask where safety fails under a tool schema and whether the model still represents harm there, so we steer along the harm direction and not the refusal direction. Zhao et al. (2025) show that a chat model encodes harm in a direction at the last request token which is separate from the refusal direction at the final prompt token. We ask whether this also happens under a tool schema which means testing whether the model still encodes harm when it sees a harmful request with a tool schema, and whether adding this harm direction changes its preference to refuse. The refusal direction that Arditi et al. (2024) find at the template tokens mediates the refusal itself, so steering with it would not test whether the model's encoding of harm affects the tool decision. We agree that the submission did not explain this choice, and the revision now states it in Section 6. After the main experiments, we also tested a refusal-side direction on Llama, and it did not significantly change Llama's tool calls (Appendix D).

W3 and Q3. The decision margin, the zero threshold and greedy decoding

Yes, "the action does not always follow" meant that steering raised the decision margin but the margin stayed below zero, so the model still called an execution tool. This is the case on Llama, and the revision now says this directly.

The three points are not another way of saying greedy decoding, because each one is tested with a different measurement. The first point would mean that a linear probe on the model's activations can no longer separate harmful from benign requests under the schema. The second would mean that adding the harm direction does not change the decision margin. The third means that the margin increases but stays below zero, so the model still calls an execution tool. The probe and the margin are both read from the model before it generates anything, so only the third point involves decoding.

Llama is the third case. Adding the harm direction increases Llama's margin by about as much as Ministral's, but before steering Llama's mean margin is -9.2 and Ministral's is -2.8, so Llama's margin stays below zero. At strength +4 Llama executes 38 of 40 harmful requests, while Ministral executes 24 instead of 35 (Tables 18 and 20). The second and third cases give the same execution counts, which is why we report the margin.

The reviewer is right that a zero margin is a 50% threshold, since it is where the decline tool is as likely as all execution tools together. To check that the Llama result does not depend on greedy decoding, we read the margin as the probability of sampling the decline tool at temperature 1 and summed it over the 40 harmful requests (Table 17). Each entry compares no steering with strength +4.

ModelExpected decline calls (sampling)Harmful requests not executed (greedy)
Llama-3.1-8B2.0 to 2.91 to 2
Qwen2.5-7B12.4 to 28.614 to 25
Qwen3-8B22.0 to 32.025 to 32
Ministral-8B6.4 to 14.05 to 16

Under both, steering changes Llama by about one of 40 requests and the other models by about 7 to 16. The sampling values cover only the first token of the tool name, so they do not count prose responses.

AuthorsOpenReview ↗

Response to Reviewer WKjK (2/2)

W4. Trade-offs with a decline tool

It is true that a decline tool lets one control how often the model declines, and the paper only reports each model's default rate (Table 2). Adding a constant to the decline tool's logit adds the same constant to the decision margin of every request, so we added constants to the margins we had already measured and counted the requests whose margin became positive. With the constant set so that 5% of benign requests are declined, the base models decline 39% to 44% of harmful requests, and base Llama declines 41%. At the 5% still, Llama after decline-call training declines 47%, so this simple change gives almost as many harmful declines as training for Llama. This control needs the decline tool to be on the menu, since without it the trained models execute about as many harmful requests as before training (Section 5.2).

We did not run the model again with the constant added. We added the constant to the margins we had already recorded, which measure which tool the model prefers when it starts writing the tool name, so these numbers show the tool the model would start to call and not its full response. We also chose the constant using half of the 157 pairs and counted declines on the other half, so that the constant is not fitted to the requests we count.

W5. Few and older models

GPT-4o was only a judge for one appendix table and was not one of the models we evaluated, and our Claude judge relabelled all 1,570 responses in that table with 96% agreement (Appendix B). For the models we study, the revision adds Llama-3.2-3B and Mistral-Small (22B) to the training comparison, which now covers six models (Table 2). While we have not run our experiments on larger or newer models, recent behavioural studies indicate that the same failure occurs there. Cartagena and Teixeira (2026) found it on six frontier models including GPT-5.2 and Claude Sonnet 4.5, and Rodríguez et al. (2026, arXiv:2609.35117) found lower refusal under tool use than in chat on all eight open-weight models they tested, up to 70B parameters.

W6. Most results are unsurprising

We agree that several results have precedents or are expected, including that tool-use safety training lowers harmful tool calls, that the trained model depends on the decline tool, and that chat safety transfers poorly to agents. We treat the agreement between the margin and the generated call as a consistency check and not as a novel contribution. Section 5.1 reports how training lowers harmful tool calls, which we think prior work does not show. On Llama and Ministral, training changes the decision for harmful requests much more than for benign ones, while a shuffled-label control lowers harmful calls only by declining benign requests too (Figure 3a, Table 36). The harm direction fitted on the base model barely changed Llama's tool calls before training, and after training it increases Llama's decline calls which norm-matched random directions do not do (Figure 3b, Table 34). Removing the direction leaves most trained decline calls in place, so we do not claim that the trained refusal depends on it. Finally, when we remove only the tool list from otherwise identical refusal training, harmful tool calls stay near the base rate on Llama and Mistral-Small (Table 2).

Q1. Which Claude was used?

Claude Sonnet 5 judges prose responses, and tool calls are scored by a deterministic parser. A second judge from another model family, gpt-oss-120b, relabelled 300 prose outputs and agreed with Claude at kappa .97 (Appendix B).

Q2. Why do the models behave differently?

We can describe how the models differ, but we have not established why. The table gives each model's mean margin before steering and how much one steering unit changes it on the 40 harmful requests (Table 18).

ModelMean margin before steeringChange per steering unit
Llama-3.1-8B-9.20.39
Qwen2.5-7B-4.11.81
Ministral-8B-2.80.34
Qwen3-8B+0.82.21

So Llama and Ministral respond to steering at about the same rate, and they differ mainly in how far from declining they start. On Llama, giving the tools as a native schema instead of as chat text lowers the margin by about 8 to 9 log-odds (Appendix Figure 7). One possible explanation is that safety training mostly teaches refusal as a chat response and less as a tool call. Our training comparison is consistent with this, but we do not know what safety data the models were trained on and have not tested its role.

Reviewer_WKjKOpenReview ↗

Reply to responses 1&2

W1

Looks greatly more readable at first glance, thanks. But, of course, it's a huge proportion of the text that's changed (it seems we can't view the diffs directly), so I haven't reread in detail.

W2

a chat model encodes harm in a direction at the last request token which is separate from the refusal direction at the final prompt token

I don't understand the distinction.

W/Q 3

The three points are not another way of saying greedy decoding, because each one is tested with a different measurement

Obviously, but the question was about the distinction between two of the points.

Model Expected decline calls (sampling) Harmful requests not executed (greedy)

Okay, good to confirm they aren't drastically different (as expected if the distribution of logit distributions aren't too nasty).

In any case, I don't think the distinction between margin and the result of greedy decoding is that substantial; I think the new draft calling that an "explanation" is overclaiming. Most sampling today isn't greedy, but somewhere in-between, like nucleus/top-k/top-p, where shifting the margin would usually translate to an observable token-space effect.

W4

This response is really hard to understand, but I think just proves my point, that 50% is an arbitrary classification threshold. See "Refusal Tokens" (https://arxiv.org/abs/2412.06748) for instance.

W5

I don't see any rebuttal here.

Q1

Acknowledged.

Q2

Okay, I was just soliciting speculation, some contextualization or intuition.

Reviewer_zYuJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This is a well-executed paper on safety. Authors explore the issue of models complying with unsafe requests in the context of tool calls/tool schemas. Authors first find out that the “harm direction” is decodable and they show that for 3/4 models they can steer the model to increase refusal rate. Finally, they show that training is a good method to improve safety even more.

Critical strengths

Full paper with a well designed setup. I really like the care that was taken in setting the work up and producing a complete paper.

Critical weaknesses

My general feeling on this paper is that contribution is a bit limited. By all means, this is not a bad paper. I just think it’s not very surprising in terms of results and the usage of old and small models weakens the argument a little bit. Other than that, writing could be improved, I struggled reading some portions of this work, the meaning of some sentences kind of confuses me.

Weaknesses:

  • The results are based on outdated, small models, moreover, these are only instruction tuned models. It is unclear if these results also hold on more intelligent and even larger ones. This is not to say that this kind of research is not important, but I feel like we can’t understand how big of a problem this is in practice if it’s tested only on past models. For example, we know old models were also more subject to jailbreaking but this problem was reduced with more modern techniques.

  • The harm direction analysis is indeed interesting, but again feels incremental on top of what is out there and not entirely surprising.

  • The “training helps reducing the problem” framing does not seem very surprising to me. I would argue that this has been proven in several settings. Saying that “Without steering, every model still executes many harmful requests under the schema, so we then ask whether safety training can move the tool decision toward declining.” I think it’s an expected result. Please let me know if I have missed something here.

  • (this is a minor issue because I understand it's more of a dataset problem than a problem with the paper) There is some ecological validity issue with some examples shown as Figure 5 in the Appendix seems to show an unrealistic example in which the user would ask to “inflate” the participants something that would be easily corrected by asking “we made a mistake in this annual review document please correct it and add X participants” instead.

As it stands I think the paper does not bring enough contributions to be a full conference paper.

I have to be honest there are a few sentences that really confuse me (and I have found this to be a general trend with this paper), I am sharing the first two I could find on a re-skimming of this work:

“In several of these settings the model still represents harm while it complies… so the failure is not that the model misses the harm.”

“The harm direction reaches the tool decision on every model, but the action does not always follow.” (follow what? I assume it means the action is unsafe even if the “harm” is recognized but I guess I am confused on why the language here is so “metaphorical”)

Questions for authors

//

4 comments
AuthorsOpenReview ↗

Response to Reviewer zYuJ (1/2)

We thank the reviewer for the careful review and for the kind words about the setup.

W1. Small and older instruction-tuned models

This study primarily used older and smaller models of about 8B parameters. While we have not studied larger and newer models in this paper, recent behavioural studies indicate that the same failure occurs in larger models as well. Cartagena et al. (2026) tested six frontier models, including GPT-5.2, Claude Sonnet 4.5 and DeepSeek V3.2, and all six sometimes refused a harmful request in text while making the forbidden tool call in the same turn. In concurrent work, Rodríguez et al. (2026, arXiv:2609.35117) found lower refusal under tool use than in chat on all eight open-weight models they tested, from 8B to 70B parameters.

Since these studies show that the behaviour exists in newer and larger models, we focus on explaining where the failure occurs inside the model, and for this we use smaller models whose internal activations we can access. We perform probing, steering, ablation and fine-tuning across several seeds, and these experiments need the model weights and enough compute to train each model several times, so the open-weight models we used were well suited to them. To test other model sizes, the revision adds Mistral-Small (22B) and Llama-3.2-3B to the training comparison, which now covers six models (Table 2). On Mistral-Small, prose-refusal training with the tool schema lowered harmful execution from 73% to 27%, while the same training without the tool list left it at 76%.

We agree that these results only suggest that our findings generalize, and that larger models need to be studied to confirm them. We intend to do this in future work, and the Limitations section states that our claims cover the models and setup we tested.

W2. The harm direction analysis is incremental

We agree that finding the harm direction decodable under a tool schema is incremental on its own, and Pan et al. (2026) report the same finding. However, prior work does not test whether adding this direction to the model's activations changes which tool the model calls. To test this, we add the direction at the last request token and measure two things. The first is the decision margin, which is the log-odds that the model writes the name of the decline tool instead of the name of an execution tool, so a higher margin means the model is more likely to decline. The second is the actual tool call the model generates.

These two measurements let us distinguish three possible reasons why a model executes a harmful request under the schema. First, harm may not be linearly decodable under the schema. Second, harm may be decodable, but adding the harm direction may not change the decision margin. Third, adding the direction may increase the margin, but not enough for the model to call the decline tool. In both the second and third case the model still executes the request, so the number of executed requests is the same in both. The decision margin is different, because adding the direction leaves it unchanged in the second case and increases it in the third.

We find that Llama executes harmful requests for the third reason. Harm is decodable on Llama, and adding the harm direction increases its decision margin by about as much as it does on Ministral (Table 18). However, before steering Llama's mean margin is -9.2, much lower than Ministral's -2.8, so even after steering Llama's margin is still below zero. At strength +4, Ministral executes 24 of 40 harmful requests instead of 35, while Llama executes 38 instead of 39.

We think identifying which of these reasons applies matters for two reasons. First, if harm is decodable, one could try to make the model refuse by adding more of the harm direction at inference time. On Llama this increases the decision margin but does not change the tool call, so a decodable harm direction does not show that the model will refuse under a tool schema. Second, it shows what a fix has to change on Llama. Since the harm direction is already decodable and already increases the margin, a fix has to raise Llama's decision margin for harmful requests under the schema and safety training inside the schema does exactly this. It raises Llama's mean harmful margin from -9.2 to +0.2 while its benign margin stays far below zero, and after training, adding the same harm direction increases Llama's harmful decline calls on all four training seeds (Figure 3). The same refusal training without the schema leaves harmful tool calls near the base rate on Llama and Mistral-Small (Table 2).

AuthorsOpenReview ↗

Response to Reviewer zYuJ (2/2)

W3. "Training helps" is expected

We agree that safety training lowering harmful tool calls is expected, since a model trained to refuse a set of requests should refuse similar requests afterwards. On the reviewer's question of what may have been missed, Section 5 asks how training makes the model refuse under a tool schema and what it changes inside the model. On Llama, training raises the decision margin for harmful requests to about zero while the margin for benign requests stays far below zero. After training, adding the harm direction taken from the base model increases harmful decline calls (Figure 3), while before training Llama still executed 38 of 40 harmful requests with the same direction added. So the harm direction that was already decodable in the base model changes Llama's tool call only after training has raised the harmful margin. The submission did not emphasize this result and mentioned it in only one sentence, so in the revision we made it a separate contribution in the introduction, and Section 5.1 reports it.

The second result is that on some models, safety training lowers harmful tool calls only when the tool schema is in the training data. We evaluate every model with the tool schema present, and the training arms differ only in their training data (Table 2). Before training, Llama executes 96% of harmful requests. After prose-refusal training whose examples include the tool list, it executes 34%, and after the same training with the tool list removed from the examples, it still executes 91%. Mistral-Small gives 73%, 27% and 76% in the same order. A control confirms that the model learned to refuse in the training format, at one of two seeds on Llama and at the only Mistral-Small seed. In the revision we replaced the original chat-trained arm with this matched comparison, which removes only the tool list (Section 5.2).

W4. Realism of the example

We agree that the example, like the rest of our dataset, states its harmful intent openly. We built the dataset this way so that each harmful request can be paired with a benign request for a legitimate version of the same action, here asking for the document with "correct participant numbers and findings", and we can measure whether a model treats the two differently. Requests that carry implicit harm through wording like the reviewer's are something we did not test, and we plan to study them in future work. For this study, open intent helps us remove one explanation for the failure, which is that the model did not see the harm in the request.

W5. Confusing and metaphorical sentences

As the reviewer pointed out, several sentences in the submission were difficult to parse, including the two quoted ones. In the revision we have rewritten the abstract and main text in plain language so that each sentence states what was measured, and these two sentences are removed. As for what we tried to convey in the given sentences, the first meant that a linear probe separates harmful from benign requests even while the model complies with the harmful ones. The second meant that steering increases the probability of the decline tool while Llama still executes 38 of 40 harmful requests.

Thank you for your response. As I mentioned in my review I appreciate the work that was put in this paper.

However, my current reading of the response is that most of the issues have been acknowledged (safety training helps safety, harm-decodability being incremental) .

As an example, in the rebuttal, The authors mention that one of their results is "safety training lowers harmful tool calls only when the tool schema is in the training data.", but isn't this expected? I am trying to understand if I missed something because this result is not surprising to me.

We thank the reviewer for following up. We would like to clarify that our main contribution is to test how a harm-direction intervention affects tool preference and generated tool calls, and how that effect changes after safety training. We do not claim that safety training improving safety is novel. We study what training changes in these measurements and what the findings suggest for alignment strategies that need to work across formats.

Decodability alone does not establish that intervening along the direction affects tool choice. We find that adding the harm direction raises Llama's decision margin by about as much as Ministral's, even though Llama's tool calls barely change (Table 18). After training, the same direction fitted on the base model increases Llama's declines on harmful requests without being refitted, while norm-matched random directions do not show the same effect (Figure 3b, Table 34). We therefore measure how training changes the behavioral effect of an existing direction, beyond showing that refusal improves.

For the schema comparison the reviewer mentions, both training conditions use prose-refusal targets and neither teaches a tool-call output. The experiment tests whether those refusals transfer when a schema is present at evaluation. Harmful execution remains at 91% on Llama and 76% on Mistral-Small after training without the schema, compared with 34% and 27% after training with it (Table 2), while Ministral and Llama-3.2-3B show partial transfer.

Together, these results motivate evaluating both the effects of harm-direction interventions and the transfer of refusal training across deployment formats, rather than assuming that improved refusal in the training format will carry over.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.