- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies why LLMs that refuse harmful requests in chat often execute the same requests once given a tool schema. The authors argue that the failure lies not in recognizing harm but in acting on it, and support this with three lines of evidence on four models. First, harmfulness remains linearly decodable under the tool schema. Second, steering along the harm direction shifts the model’s preference toward a decline tool on all four models, but the actual action changes only on three of them; Llama-3.1-8B keeps executing almost every harmful request. Third, safety fine-tuning inside the tool schema reduces harmful tool calls, whereas the same training in plain chat either leaves harmful calls near the base rate or makes the models stop calling tools. The authors conclude that safety alignment is tied to the format it was trained in.
Critical strengths
The authors show that (1) under tool schemas, models still linearly encode harm, and steering along this harm direction shifts the tool decision, yet Llama-3.1-8B keeps executing harmful requests.
(2) Safety training reduces harmful tool calls only when done inside the schema, after which the harm direction can also drive Llama’s action.
Critical weaknesses
(1) I believed most of the aspects mentioned in this paper are already established by Aziz et al. (2026) and Wu et al. (2026a), and Pan et al. (2026) show on the same checkpoints that tool schemas weaken refusal signals, yet their SafeKeep fix is not compared.
(2) On Llama, the harm probe does not beat a simple word-count baseline. Steering only moves Llama’s margin from about −9 to −6, and it never crosses zero, so it is expected that the action does not change. The steering also raises benign refusals, so it is not specific to harm. Finally, the chat-trained arm is poorly designed and failed its own success check.
Questions for authors
See weakness, and also: (1) How does this work differ from Pan et al. (2026), Aziz et al. (2026) and Wu et al. (2026a) and any comparisons with them?
(2) Does steering with the refusal direction at the last prompt position change Llama’s tool action, which would show whether the failure lies before or after the refusal signal?
(3) Does Llama’s margin cross zero under other prefixes, given that Table 11 shows prefix offsets of 5–10 nats?
(4) Why does Appendix D report Llama’s token-steered chat refusal at α=+8 as 73% (“within noise”), while Table 23 reports 35/40 (87.5%)?
(5) Why do Appendix D and Appendix I report different chat steering results for Ministral (48%→90% vs. 14→31 of 40)?
(6) Which system prompt (with or without the refusal license in Table 3) and which decoding settings do the main evaluations use?
References:
Tool Specifications Matter (Pan et al., 2026)
Low-Resource Safety Failures Are Action Failures, Not Representation Failures (Aziz et al., 2026)
Knowing without Acting (Wu et al., 2026a)
5 comments
Response to Reviewer WQvm (2/2)
W2c. Benign refusals under steering
We agree that steering also raises benign refusals. We add the harm direction to every request, benign ones included, and since the direction is the difference between the mean activations of harmful and benign requests, adding it moves a benign request's activations toward those of harmful requests. So we expect benign refusals to rise as well, and this alone does not show that the direction is unrelated to harm. A stricter test is whether steering raises the harmful margin more than the benign one. On Llama the two rise by similar amounts, and on the other three models the harmful margin rises more (Table 18), but this is weak evidence, since on Qwen3 it does not hold on the AgentHarm pairs and Ministral's steering layer was chosen on the same pairs. So we do not claim that the direction encodes harm alone, and the Limitations section of the revision states this. Our main result on Llama does not depend on it, because there steering does not change the tool call for harmful or benign requests.
W2d. The chat-trained arm
We agree that the original chat arm was poorly designed, since it also changed the system prompt and the benign targets and failed its own check. The revision replaces it with the same prose-refusal training with and without the tool list, with nothing else changed, and a control that checks whether refusal was learned in the training format. All models are evaluated with the tool schema, and the table gives the share of harmful requests executed (Table 2).
| Model | Before training | Trained with tool list | Trained without tool list | Refusal-learning control |
|---|---|---|---|---|
| Llama-3.1-8B | 96% | 34% | 91% | Passes at one of two seeds |
| Mistral-Small (22B) | 73% | 27% | 76% | Passes at the only seed |
| Llama-3.2-3B | 91% | 39% | 58% | Fails at both seeds |
| Ministral-8B | 89% | 63% | 72% | Passes at both seeds |
On Llama and Mistral-Small the control passes, yet training without the tool list leaves harmful calls near the untrained rate. The other models do not give a clear result, since the Ministral gap is not significant (p = .087), Qwen2.5 does not learn the refusal, and Qwen3 already refuses 97 of 113 in that format before training. So we limit this finding to Llama and Mistral-Small (Section 5.2, Appendix I).
Q2. Steering with the refusal direction at the last prompt position
No. We added a chat harmful-versus-benign contrast at the final prompt token, where Zhao et al. measure refusal, and Llama still executed 39 of 40 harmful requests at +4 and +8, the same as without steering. Added at the last request token, it changed 39 to 37 (p = .5, Appendix D). So this test cannot say whether the failure is before or after the refusal signal, and it covers one contrast at one position, without Arditi et al.'s search over layers and positions.
Q3. Llama's margin under other prefixes
No. Without steering, the bare and the longer prefix give nearly the same harmful margins on Llama (-9.0 and -9.2, per-request Spearman .99), and the same 1 of 40 requests is positive under both. The 5 to 10 nat offsets in the submitted table are for Qwen3. These Llama values come from our result files and are not in the paper.
Q4. 73% in Appendix D against 35/40 in Table 23
They come from different directions. Appendix D adds the tool-fitted direction in chat at +8, which raises refusal from 26 to 29 of 40 (73%). The style study adds a direction fitted in each format, which in chat raises refusal from 25 to 35 of 40. The revised appendix names both (Appendix D, Table 22).
Q5. Ministral's 48% to 90% against 14 to 31 of 40
This was a reporting error. Appendix D counts safety refusals and capability declines at +4 (19 to 36 of 40, 48% to 90%), while the old Appendix I counted safety refusals alone at +8. The revision removes that summary from the fourth-model appendix (now Appendix H) and points to Table 22.
Q6. System prompt and decoding
The main evaluations use the system prompt without the refusal license, which is the domain prompt of Cartagena and Teixeira (2026), unchanged. It does not permit refusal or mention the decline tool, and only the decline tool's own description allows declining. The refusal-license condition appears only in the factorial table (Table 8). All free generation is greedy, with 128 new tokens for the original training evaluations and 256 for the steering runs, the two added models and the name-first Qwen retrain (Appendix A).
Follow-up Questions
Thanks for the authors' engagement, I noticed the updated paper and your responses which solve some of my questions. Yet following the new updates, I still have few questions:
(1) For casual intervention, I think decision margin is just a readout, not the essential mechanism: is calculated and analyzed from the output logits rather than model's internal computation path. So explanations like Llama doesn't refuse because the margin is low, so training to increase the margin helps refusal is useful, but this is more like describing the phenomenon using logit space's language. So I don't know what benefits does the training give: changes in MLP? attention? or just decision bias?
(2) For Llama, its refusal behavior may be affected by the template (https://huggingface.co/dropbox-dash/Llama-3.1-8B-Instruct_nvfp4_weights_calib_demo/blob/main/chat_template.jinja) where it says "Given the following functions, please respond with a JSON for a function call" unlike Qwen's for example use "may". Would this explains the phenomenon?
Response to Reviewer WQvm (1/2)
We thank the reviewer for the specific questions and for finding the inconsistencies between appendices, which we have corrected.
W1 and Q1. Differences from Pan et al., Aziz et al. and Wu et al.
We agree that prior work already shows harm can be decodable while the model does not refuse. We add two tests that this work does not run. We test whether adding the harm direction changes which tool the model calls, and we repeat this after safety training with and without the tool list in the training data. Pan et al. (2026) probe at the final prompt token and steer against a schema direction. Like them, we find that decodability falls at that token under the schema, but at the last request token our logistic probe stays within .03 AUROC of chat (Table 12). Aziz et al. (2026) study the same gap in low-resource languages, and Wu et al. (2026a) separate recognition and execution axes in chat jailbreaks, while we test whether the harm direction changes the tool call before and after training.
SafeKeep is an inference-time fix, and we use our training arms to measure what training changes, not as a competing defence. We still ran SafeKeep on 113 matched pairs. On Llama and Qwen2.5, 59% and 31% of its safety judgments did not follow the YES/NO format, which a rule we set before the runs excludes, so the table covers Ministral and Qwen3.
Counts are out of 113, and a selective pair has its benign request executed and its harmful request not. On Ministral our training gives more selective pairs than SafeKeep, and on Qwen3 neither differs much from the base model. This comparison is in this response only.
W2a. Llama's probe against the word-count baseline
We agree that on Llama the submitted probe did not clearly beat word counts. That probe was the projection onto the mean-difference direction we steer with, which is not the standard test of decodability. The revision adds a logistic-regression probe under the same held-out folds, and at the steering layer it beats word counts and character n-grams on all four models (Table 12). On Llama its AUROC is .903, against .755 for word counts and .825 for character n-grams, and the smallest gap over character n-grams is .044 [.004, .085], on Ministral.
W2b. Llama's negative margin and unchanged action
We agree that a margin below zero fits Llama continuing to execute, and the revision says this directly. We report the margin because execution counts cannot tell apart a direction that leaves the margin unchanged from one that raises it but not above zero, and Llama is the second case. Adding the direction increases Llama's margin by about as much as Ministral's, but before steering Llama's mean margin is -9.2 and Ministral's is -2.8 (Table 18). After training inside the schema, Llama's mean harmful margin is +0.2 (Table 33), and the same base direction then increases harmful decline calls on all four seeds, which it barely did before training (Figure 3b, Table 34). Training also increases how much the margin changes per steering unit, so the higher starting margin alone does not explain this.