acceptodds
Under review as a conference paper at ICLR 2027

Learning Authorization for Tool Use

Abstract

An agent that learns to reject a file-sharing instruction hidden in a document must still share the file when its user asks. Training on the incident alone can suppress both actions, because the tool call is identical and its authorization is the relevant difference. We study this repair problem with matched preferences that keep the operation fixed while changing who requests it and whether the user has confirmed it. Preference optimization trains the agent to make the authorized call and answer or seek confirmation in the other contexts. A reference-KL penalty constrains changes on previously correct responses. On a controlled AgentDojo test, all three Qwen runs recover all 64 authorized actions while making no unauthorized state changes. Matched controls on natural-language requests reveal different effects by tool. On a fresh execution test, DPO selects calendar targets more accurately, while SFT produces more executable file-sharing calls. Llama recovers authorized execution with less reliable clarification. The retention studies find different effects across task sets, rather than a general benefit from KL. The results identify authorized-action recovery as a distinct target for agent repair: preventing the incident and completing the user's work must be learned and measured together.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, tool use, authorization, prompt injection, preference optimization, direct preference optimization, AgentDojo, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_PBUzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

You want to train an agent to ignore instructions hidden in documents, since this might contain prompt injections. You want the model to be able to disambiguate which instructions are from authorised users and which ones are from, say, the environment. They build some DPO pairs for two actions in the agentdojo simulated workspace which are cancelling a calendar event (cancel_calendar_event) and sharing a file (share_file). They got positive results in an easy setting (since actually, the base model already completes 63 of the 64 authorized actions) and then many negative results in a qualified, secondary setting.

Critical strengths

None

Critical weaknesses
  • The paper is very hard to read and understand. It takes quite awhile to parse through the prose, and only has the one schematic figure at the beginning.
  • As mentioned above, the main result is in a small setting where the model already does pretty well. And, the scale of training is quite small (only about 80 pairs?). My guess is that this type of training may simply be subsumed by existing instruction following datasets in tool-calling settings. And even on the small models, this is quite a saturated task.
Questions for authors

N/A

2 comments
AuthorsOpenReview ↗

Response to Reviewer PBUz

Thank you for reviewing our paper. We may not have explained the goal clearly enough: we are not trying to raise the base model's authorized-action success from 63/64 to 64/64. We study whether safety repair also stops the model from carrying out legitimate requests. For example, after learning to reject a file-sharing instruction hidden in a document, can it still share the file when the user explicitly asks?

Our experiments therefore check both behaviors, rather than authorized-action success alone. The small scale and saturation you point out do limit what this evaluation can establish.

Thanks. I will retain my score.

Reviewer_GtWZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how to train LLMs to follow proper authorization for user requests in a indirect prompt injection-like situation where its context contains different authorization sources and instructions. The paper proposes the tradeoff between usefulness and safety and formulates the problem of how training to repair safety incidents can impact usefulness, as well as how to recover LLM's ability to follow such instructions with valid authorization sources. The paper then designed the Matched Authorization Preference and compared different post training methods for this task on Qwen3-4B, Llama-3.1-8B-Instruct and Phi-4-mini- instruct. The paper did some evaluations on a AgentDojo variant reporting some findings.

Critical strengths
  1. The paper has an interesting angle and formulation of what is already known as cases of indirect prompt injection, which later tansformed into the design of the Matched Authorization Preference.
  2. Some findings comparing different post training methods are also useful for guiding practical work on relevant tasks.
  3. Experiments are well recorded and with detailed protocols for reproduction.
Critical weaknesses
  1. The paper is not easy to follow due to its writing style. The story jumps between motivating authorization repair, presenting the DPO+KL method, and describing tool calling behavior. Readers needs to reconstruct how these observations establish the paper's central contribution.
  2. The paper has very limited evaluation given a well know topic as well as many public benchmarks. The main experiment only uses custom AgentDojo scenarios for calendar cancellation and file sharing with several small models, which is far from what current SOTA standard.
  3. The relevance of the experiments and the problem formulation is not well presented. The paper should explain how these comparisons establish the contribution of the matched authorization structure and connect that contribution to the stated repair objective.
Questions for authors
  1. What is the main contribution of this paper, presented with a clear storyline and its novelty?
  2. How do the experiments support the proposed problem formulation and show the benefits of matched authorization training over baselines?
2 comments
AuthorsOpenReview ↗

Response to Reviewer GtWZ

Thank you for your comments. Our question is how to repair prompt-injection failures without also suppressing legitimate tool use.

We keep the operation and its arguments fixed, changing only who requests it and whether the user has confirmed it. This gives the model examples of when the same operation should and should not be carried out. The experiments check both unauthorized actions and legitimate task completion. The SFT and DPO comparisons help examine the role of these training examples; DPO itself is not our claimed novelty. Presenting these pieces together made the main thread harder to follow than it should have been.

Thanks. Since the paper is still hard to read and understand even with the author's reply, I will retain my score.

Reviewer_hiMPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a practical problem: repairing an agent that followed an instruction embedded in untrusted data can also make it less likely to carry out the same operation when the user explicitly authorises it. The authors introduce matched authorisation preferences, in which the preferred response to a fixed operation changes with the source of the instruction and the confirmation status. They train with direct preference optimisation (DPO) and a KL penalty on correct reference-policy responses. In the controlled AgentDojo evaluation with Qwen3-4B, the final policies recover all 64 authorised actions without any observed unauthorised state changes. The paper also evaluates Llama, where authorised execution is recovered but clarification remains unreliable, and Phi, where unrelated supervised updates also remove the observed unsafe behaviour, making the effect of the proposed repair difficult to isolate.

Critical strengths
  1. A clearly defined and practically relevant problem, and well-defined setup:

    • Repairing an indirect prompt-injection incident can suppress legitimate tool use as well as the unauthorised behaviour. A system should not learn to avoid all file-sharing operations when the user has explicitly requested one. The paper articulates this trade-off clearly and generally avoids overstating the results.
    • A useful matched-preference construction: Table 1 is quite interesting. Its five conditions—authorised write, unauthorised injection, conflicting target, missing confirmation, and clean read-only task—reuse the relevant environment objects and action arguments while changing the authority context. This design directly tests whether the model distinguishes when an operation is authorised, rather than merely learning to suppress a particular tool call.
  2. Execution-based evaluation: Parsing tool calls, executing them in reconstructed AgentDojo environments, and checking the resulting state transitions provides stronger evidence than evaluating textual responses alone. The paper also distinguishes execution-based results from response-only evaluations in the natural-language controls.

  3. Careful Reporting and Fresh execution Test:

    • The paper reports results that clearly shows nuances of its claims, i) including the Phi control in which unrelated supervised fine-tuning also removes the observed unsafe behaviour, ii) Llama’s remaining clarification failures, iii) the failure of the preregistered criterion for a general retention benefit from KL regularisation, and iv) the reversal in the relative performance of SFT and DPO on the fresh execution test. Reporting these outcomes helps clarify what the experiments do and do not establish.
    • The fresh execution test provides a useful robustness check: The fresh execution test in Table 4 is valuable because the relative ranking of SFT and DPO changes under new wording, identifiers, and other input shifts. SFT achieves higher authorised-action execution success, while DPO makes fewer observed unauthorised changes. This result cautions against drawing a general conclusion about the superiority of either objective from the earlier response-level evaluation alone.
Critical weaknesses
  1. Limited tool coverage: The main experiments focus on two tools: cancel_calendar_event and share_file. The unseen-tool evaluation adds four further tools, but the reported improvement is limited to file deletion; the other unseen tools show no improvement, and the base model already makes no observed unauthorised state changes on these tools. Consequently, the evidence is insufficient to determine whether matched preferences teach a general authorisation principle or primarily improve behaviour for the two trained operations.

  2. The benefit of KL regularisation remains uncertain: The paper itself concludes that there is no established general retention benefit from KL regularisation. The preregistered criterion in the direct KL comparison is not met. Although verified-reference KL remains within the five-percentage-point retention bound on both historical suites for all three Qwen seeds, the comparison between synthetic and verified reference banks is confounded by their different sizes and contents (16 versus 30 trajectories). Moreover, authorised-replay SFT also achieves 160/160 joint success on the primary controlled test. The incremental contribution of KL regularisation is therefore unclear: the results suggest a possible benefit on selected retention evaluations, but do not establish a general or isolated effect. Given the emphasis placed on this component, its empirical contribution could be stated more narrowly.

  3. The controlled benchmark provides limited separation between successful methods: Once authorised examples are included in training, several methods perform perfectly or nearly perfectly on the controlled test: authorised-replay SFT and both KL variants achieve 160/160 joint success, while ordinary DPO achieves 158/160. The primary benchmark also supplies the canonical call syntax in the user request, reducing the need for the model to construct a tool call from natural language. The paper does examine this limitation through natural-language controls, but those results are more mixed and depend on the tool and condition. The difference between selecting whether to emit a supplied call and constructing a valid call from a naturallanguage request is substantial, so the controlled results alone provide limited evidence about performance in less constrained settings.

  4. Model scale and coverage: The principal experiments use Qwen3-4B and Llama-3.1-8B-Instruct, with Phi-4-mini-instruct used for additional controls. These are relatively small models compared with some models used in deployed agent systems. Authorisation behaviour and the effects of fine-tuning may differ at larger scales or in frontier API models, but the paper does not evaluate such systems. This limits the extent to which the findings can be extrapolated to higher capability deployed agents.

  5. Limited experimental comparison with existing defences: The paper discusses instruction-hierarchy methods and SecAlign, but does not compare the proposed approach experimentally with these methods. For example, it remains unclear how matched authorisation preferences would compare with applying instruction-hierarchy training after an incident, or with a SecAlign-style preferencetraining baseline using a broader set of examples.

  6. Dense presentation: The paper covers several connected evaluations: the controlled execution test, capability retention, naturallanguage requests, objective and request-format controls, fresh execution, and cross-model transfer. The overall progression is sensible, but the number of combinations, mixed reporting of experimental setups-vs-results make the paper demanding to follow. The formal definition of Auth(a | x) in Section 2 is understandable, although its role in the subsequent analysis could be made more explicit.

Questions for authors

• The term “matched” is used for several different constructs, including matched preferences, matched controls, and matched SFT. More distinct terminology would make the experimental design easier to follow.

• Table 3 provides an informative per-tool, per-condition breakdown, but its implications are easy to overlook when reading the aggregate results. The main text could draw more attention to the heterogeneity it reveals.

• The Phi result in Table B8 raises the question of whether generic output-style changes could explain part of the Qwen improvement as well. A comparable control for Qwen would help distinguish the effect of the proposed repair signal from more general changes caused by fine-tuning.

1 comment
AuthorsOpenReview ↗

Response to Reviewer hiMP

Thank you for carefully examining the experiments. The distinction you draw is important: once authorized examples are included, SFT also performs well, and the methods have different strengths on natural-language requests and execution tests. Our main point is that training needs to cover both authorized and unauthorized contexts, not that DPO or KL is universally better.

The different uses of 'matched' can be confusing, and the per-tool results deserve a clearer explanation. As for the unrelated-training control you suggest for Qwen, we do not currently have that result, and the Phi experiment cannot substitute for it. Thank you for these specific suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.