acceptodds
Under review as a conference paper at ICLR 2027

AuthDiff: Measuring and Constraining Authorization Drift in LLM Agents

Abstract

LLM agents update their authorization policies during execution to accommodate task requirements revealed by tool results. Existing agent-security evaluations focus on attack success rate and task completion, leaving changes to the policies themselves unmeasured. We identify and define authorization drift: successive policy updates permit requests that were initially denied, without a supporting trusted grant. Drift can make the monitor permit later messages, file changes or transfers beyond the initial authorization boundary, even when tasks succeed and injections fail. We introduce AuthDiff to measure and constrain drift. AuthDiff compares decisions on the same requests before and after each update, checks expansions against the initial policy and trusted grants, and uses an online guard to reject flagged updates. We observe drift in two dynamic authorization systems, Progent and DRIFT. All eleven evaluated agent models exhibit drift under Progent. On GPT-5.6-Luna, the guard reduces drift events per update by 86.9% while recording the same number of successful tasks. Our findings show that constraining authorization drift is essential to securing LLM agents throughout task execution.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Authorization drift, LLM Agents, Dynamic authorization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Vro1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates authorization drift where LLM agents update authorization policies during execution for task requirements according to tool call results. The paper proposes AuthDiff to measure and constrain drift. With the paper's model and definition, the paper evaluated authorization drifts across models and existing dynamic access control systems and found that authorization drifts are common with current systems, while AuthDiff reduces drift events per update by 86.9% without trading off the success rate of the given tasks.

Critical strengths
  1. Clear problem formulation and motivation. The paper formalizes authorization drift through request-level permission changes and explains why intermediate policy states deserve evaluation alongside attack success and task completion.
  2. Interesting findings with authorization drift (RQ1). Drift appears across all eleven evaluated Progent models and also in DRIFT, which gives some insights on how LLM agents adapt to current dynamic authorization systems.
  3. Artifact and data available in supplementary material.
Critical weaknesses
  1. Potential initialization and selection bias. Drift attribution depends on model-generated initial policies, and evaluation uses a subset selected through another model's initial denials. An over restrictive anchor can classify legitimate permissions as unjustified expansions, thus making the evaluation biased. Densitivity tests with initialization and sampling choices are needed for soundness.
  2. The comparison with static permission control favors adaptation in the approval experiment. The paper manually imposed read-only policies which gives 0/24 utility for NO-UPDATE and DENY-EXP, while AuthDiff completes 13/24 tasks requiring added write permissions (Table 3). Fixed, task-aware authorization rules that consume the same authenticated approvals would provide a fairer comparison.
  3. Component contributions need stronger ablation evidence. The paper evaluates NO-NEG-PROBES, POLICY-REPS and OBSERVE, but POLICY-REPS removes trajectory, mutation and negative probes together. Removing negative probes gives 10 events versus nine for AuthDiff. Separate component ablations are needed to identify the contribution of each component.
Questions for authors

I will consider raising my score if the authors addresses the following.

  1. How sensitive are the drift findings to the initial policy and task-selection procedure?
  2. How does AuthDiff compare with fixed, task-aware permission rules that accept the same authenticated user approvals?
  3. What does each guard component contribute?
10 comments
AuthorsOpenReview ↗

Kindly reminder

Dear Reviewer Vro1,

Thank you again for your time and thoughtful feedback on our submission.

We have posted our responses and uploaded a revised manuscript, with added and revised text highlighted in blue. As the discussion period draws to a close, we would appreciate hearing whether our responses have addressed your concerns. If there are any remaining questions or points you would like us to clarify, please let us know. We would be happy to continue the discussion.

Best regards, AuthDiff Authors

Thank you for your detailed response. While the new results for Q1 gives solid evidence, results for Q2 and Q3 still seem to lack reasonable explanation for supporting the authors' claims. Therefore I will remain the score as it is.

AuthorsOpenReview ↗

Further Clarification on Q2 and Q3

Dear Reviewer Vro1,

Thank you for revisiting our responses and for recognizing the additional evidence for Q1. For Q2 and Q3, we would like to explain more clearly the practical role of the dynamic updater and the rationale for combining the probe sources.

Q2: Why retain a dynamic updater?

In an agent workflow, the user may specify a goal before the concrete resources and required tool calls are known. For example, the agent may first search for a document and then determine which file operation is needed. Tool results identify the resource, while the initial authorization and subsequent trusted grants determine which operations are permitted. A dynamic updater translates this evolving task context into executable policies.

Fixed task rules can support such workflows when predefined rule templates and conversion logic cover the required operations and argument constraints. Our fixed-rule control implements this approach by converting structured trusted grants into rules. For broader workflows, using this approach requires providing the corresponding rule-construction logic as part of the authorization system.

AuthDiff allows a system to retain dynamic policy generation while checking whether the resulting policies expand authorization beyond the initial policy and trusted grants. It evaluates the monitor’s decisions on concrete requests and returns expansion witnesses for flagged updates. The practical distinction is therefore between constructing permissions through predefined rule logic and checking policies proposed by a dynamic updater. The close completion counts in our comparison show that these checks can coexist with task completion under the tested grants.

Q3: Why combine the probe sources?

A candidate policy can permit many requests with different arguments. A representative request illustrates a rule’s behavior, but the security-relevant expansion may concern another argument combination. The probe sources examine this permitted scope from complementary directions:

  • Trajectory probes test the exact requests arising during execution, connecting policy checks to the actions the agent actually proposes.
  • Mutation probes vary arguments to examine how far a permission extends. In the transaction example, they expose that a policy proposed for an amount update also permits changing an unrelated transaction’s recipient.
  • Negative probes test attacker-specified requests appearing in untrusted content, allowing the guard to examine those permissions before the agent proposes the corresponding calls.

The fixed-candidate examples demonstrate this complementarity directly: changing the probe source changes detection while the candidate policy remains the same. The design combines policy-derived requests with actual calls, argument variations, and requests from untrusted content, so that detection does not depend on one way of selecting requests. This is the role of the additional probe sources alongside representative probes.

We hope this clarifies the practical motivation and the connection between the evidence and our design. We would appreciate any further indication of which aspect remains unclear.

Best regards, AuthDiff Authors

Reviewer_PnywOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that LLM agents update their authorization policies during execution and may add permissions that were initially denied. It also proposes a method to measure and constrain this drift.

Critical strengths

I agree with the authors that authorization can drift/expand during policy updates. The authors propose a method that monitors the context sent directly from the user to infer the user's direct authorization, in order to prevent unauthorized drift.

Critical weaknesses
  1. If a task requires an expanded permission to be completed, the proposed method needs the user to send a separate authorization to authorize it directly, even if the user asks the agent to follow the instructions in the email in the initial message. I understand that this is a security-utility tradeoff; however, the user should emphasize this tradeoff more, as it may harm utility. The utility experiment is conducted on a selected set, and it would be great to run it on the full set and analyze the failed cases.
  2. As mentioned in the point above, many experiments and analyses are run on a limited set of selected tasks. Can the authors clarify how they were selected and whether they were cherry-picked?
  3. The authors compare AuthDiff with two modes in Progent (NO-UPDATE and DENY-EXP), while Progent also has a mode that asks the user to confirm whether to allow the expansion rather than rejecting it directly. This sounds similar to the proposed AuthDiff, in which the user sends a separate authorization when the authorization drifts. Can the authors compare with that mode as well?
Questions for authors

None

19 comments
AuthorsOpenReview ↗

Kindly reminder

Dear Reviewer Pnyw,

Thank you again for your time and thoughtful feedback on our submission.

We have posted our responses and uploaded a revised manuscript, with added and revised text highlighted in blue. As the discussion period draws to a close, we would appreciate hearing whether our responses have addressed your concerns. If there are any remaining questions or points you would like us to clarify, please let us know. We would be happy to continue the discussion.

Best regards, AuthDiff Authors

Thanks for the rebuttal.

W1: Sorry for the confusion. For W1, I meant the utility comparison under the no-attack setting rather than the direct injection setting. Do you agree that comparing no-defense and authdiff under the no-attack setting would be a more realistic comparison for judging whether it may harm utility?

W2: Thanks for the explanation. Do you have a specific reason for using GPT-4-0125-preview to select the set? Seems like it's an old model that isn't used in the experiments. Just wondering about the model choices.

W3: Thanks for the explanation. Could you remind me how you get the trusted grants and how authdiff compares them with the updates? I suppose the trusted grants are also related to the context. Is there any model involved in getting the trusted grants or comparing the trusted grants?

AuthorsOpenReview ↗

Further Clarification on W1 and W2

Thank you for clarifying W1 and for the follow-up questions. We have now evaluated task completion without attacks and clarify the screening model and trusted grants below.


W1: Sorry for the confusion. For W1, I meant the utility comparison under the no-attack setting rather than the direct injection setting. Do you agree that comparing no-defense and authdiff under the no-attack setting would be a more realistic comparison for judging whether it may harm utility?

A1: We agree that a no-attack comparison more directly measures the utility cost during normal task execution. We therefore evaluated NO-DEFENSE and AuthDiff without injections using GPT-5.6-Luna. The original 96 user/injection task pairs contain 48 distinct user tasks. We evaluated each of these tasks once per configuration, using the same initial policy for both configurations on each task and retaining the original model and guard settings.

User task completion without attacks

SuiteUser tasksNO-DEFENSE completionsAuthDiff completions
Banking16119
Slack12118
Travel1289
Workspace855
Total4835 (72.92%)31 (64.58%)

AuthDiff completes 31/48 tasks, compared with 35/48 for NO-DEFENSE, an observed decrease of 8.33 percentage points. We agree with your assessment that this reflects a security–utility trade-off.

One source of this cost is that the initial policy may omit a permission needed later in a legitimate task. When no trusted grant supports the corresponding expansion, the guard may reject the update and prevent that action. We observed this sequence in two tasks completed only under NO-DEFENSE. This explains how checking policy updates against the initial policy and trusted grants can also affect normal task completion. Thank you for suggesting the no-attack comparison, which makes this trade-off clearer.


W2:Thanks for the explanation. Do you have a specific reason for using GPT-4-0125-preview to select the set? Seems like it's an old model that isn't used in the experiments. Just wondering about the model choices.

A2: Thank you for asking about the model choice. GPT-4-0125-preview was used during our initial mechanism study, when we established the initial-denial screening. We retained this fixed screening for the subsequent evaluation rather than repeating policy generation and task selection for each evaluated model. This reduced preprocessing cost and kept the task pairs identical across models. Each evaluated model still generates its own initial policy and subsequent updates during execution.

Reviewer Vro1 also asked about sensitivity to the task-selection procedure. The table below summarizes the results reported in our response to their Q1, using GPT-5.6-Luna under NO-DEFENSE on all 589 task pairs.

Drift by initial-denial screening outcome

Initial-denial screening outcomeTask pairsTask pairs with driftProportion with drift
Passed3253410.5%
Did not pass264259.5%

Drift occurs at similar frequencies on both sides of the screening, supporting that its occurrence is not confined to the task pairs retained by the GPT-4-0125-preview screening.

Reviewer_dt9AOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies authorization drift during LLM-agent execution: policy updates can allow requests that the initial policy denied without a supporting trusted grant. AuthDiff records policy snapshots before and after updates, compares decisions on identical probes, and checks whether newly permitted requests are covered by initial permissions or trusted grants. Its online guard rejects candidate updates when it detects unsupported expansions.

The main experiments use 96 task pairs across four environments and eleven models, with 1,824 conditions including the additional defense and ablation configurations. The undefended eleven-model sweep comprises 1,056 conditions and records 550 unjustified expansion events, or 18.59 events per 100 update attempts. In the GPT-5.6-Luna defense comparison, AuthDiff records 4.02 events per 100 attempts, compared with 8.84 for REPEAT-PROMPT and 18.89 for SPOTLIGHT. Against NO-DEFENSE, its event rate falls from 30.77 to 4.02, while both configurations complete 66/96 tasks in aggregate. A supplementary experiment on four underlying tasks examines expansion supported by runtime user approval.

Critical strengths
  1. The paper complements attack success and task completion with an examination of whether unsupported permissions become available during execution. This is consequential because a successful task and an unsuccessful injection do not establish that authorization remained justified throughout the run. Tracking unjustified expansions exposes a process-level security issue that terminal outcome metrics can miss.

  2. The evaluation includes multiple models and four task environments, and analyzes permission states at different stages of dynamic authorization. Drift is observed across the evaluated models and does not necessarily lead to immediate attack success. The additional DRIFT observation and concrete resource/amount-scope examples strengthen the evidence that the phenomenon deserves attention. This model and domain breadth is useful, although underlying task diversity and runtime-approval coverage remain limited.

Critical weaknesses
  1. The initial policy P0 is generated by an LLM and may omit permissions implied by the original user request. An expansion needed to perform the delegated task can therefore be counted as unjustified under the anchor-relative definition. Appendix J identifies 154 of 411 events in the delegated-task audit as within the independently assessed resource/operation scope. This does not make them false positives under the stated definition, or establish full semantic correctness, butit suggests that the metric may conflate task-related permission expansions with genuinely excessive authorization, without fully distinguishing the two. This limits the interpretation of event totals and reductions as changes in actual unauthorized-action risk.

  2. In Table 2, NO-UPDATE and DENY-EXP record zero observed unjustified events while completing 64/96 and 61/96 tasks; AuthDiff completes 66/96 with nine residual events. Thus, maintaining the initial boundary or rejecting detected expansions already retains substantial utility in the tested setting. The comparison provides limited evidence about the benefit of safely enabling necessary runtime expansion, rather than simply restricting it. The user-approval supplement addresses this issue, but has only four underlying tasks and does not evaluate grant-aware containment. Broader or stratified evidence is needed to establish the guard's incremental value.

Questions for authors
  1. In Table 2, NO-UPDATE achieves 64/96 task successes, compared with 66/96 for both NO-DEFENSE and AuthDiff. Under the evaluated configuration, this suggests that the selected benchmark offers limited evidence for the necessity of runtime policy expansion. What fraction of the task pairs actually require an update under a shared initial policy? Can the authors provide broader or stratified evidence on tasks requiring runtime authorization updates, beyond the four underlying tasks in the supplementary approval experiment, or clarify the resulting limits on the guard's claimed utility?

  2. Since the LLM-generated initial policy P0 may omit permissions implied by the original request, task-driven expansions can be counted as drift under the stated definition. I understand that the paper defines drift relative to P0 and checks task scope separately. How should the 154 events within the independently assessed task resource/operation scope in Appendix J affect the security interpretation of the reported event rates? Do the main defense conclusions persist when within-scope expansions are reported separately from expansions beyond the task scope?

3 comments
AuthorsOpenReview ↗

Kindly reminder

Dear Reviewer dt9A,

Thank you again for your time and thoughtful feedback on our submission.

We have posted our responses and uploaded a revised manuscript, with added and revised text highlighted in blue. As the discussion period draws to a close, we would appreciate hearing whether our responses have addressed your concerns. If there are any remaining questions or points you would like us to clarify, please let us know. We would be happy to continue the discussion.

Best regards, AuthDiff Authors

I believe the author’s revisions have addressed most of my concerns.

My concern was about the LLM-generated initial policy P_0 and the security interpretation of the definition of authorization drift. The initial policy may omit permissions implied by the original user request and needed later in the same task. Expansions that are consistent with the task can therefore be counted as drift under the anchor-relative definition. The author has implemented the following improvements: 1. A task-scope analysis has been introduced to distinguish between the addition of legitimate, necessary permissions and the addition of permissions that fall outside the scope of a given task. Compared to the "NO-DEFENSE" baseline, the number of instances where permissions were added outside the task scope has been reduced from 31 to 0. Although 6 events remain unclassified, I believe my concerns have been addressed effectively overall. Regarding W2, my concern was that the proposed "guard" mechanism did not demonstrate superior performance over the "no-update" baseline in the experiments. Given that the scenarios involving permission updates were relatively limited, I suggested that the author include tasks that genuinely require such updates. The author provided a detailed response: 1. They examined the initial policies used in the "no-update" baseline and identified tasks where permission updates were indeed necessary. 2. The author acknowledges that the advantage of "authdiff" across the entire set of tasks is "modest"—admittedly limited. 3. The author reiterates that, by using "authdiff," the system can successfully execute certain tasks that require the addition of new permissions.

In summary, while the issue of limited gains (W2) persists, the author has clarified and verified the operational boundaries of the proposed method. These limitations do not diminish my positive assessment of the paper’s contributions. Consequently, I maintain my rating.

AuthorsOpenReview ↗

Thank You for Your Follow-up and Feedback

Dear Reviewer dt9A,

Thank you for carefully revisiting our responses and revised manuscript. We appreciate your recognition that the task-scope analysis addresses the security interpretation of authorization drift, and that the additional analyses clarify when policy updates are needed.

AuthDiff’s central contribution is to make authorization drift measurable and to check policy updates while supporting adaptation to trusted grants. Your feedback has helped us explain more clearly how this contribution relates to task scope and task completion.

We are grateful for your constructive feedback throughout the discussion and for your positive assessment of the paper’s contributions.

Best regards, AuthDiff Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response — Contributions and Continued Discussion

Dear Reviewers and Area Chair,

We sincerely thank the reviewers for their careful reading and constructive discussion. We would like to briefly summarize the contributions of our work and express our willingness to continue addressing any remaining concerns.

  • Identifying and measuring authorization drift. Our central contribution is to identify and define authorization drift and introduce AuthDiff to measure it through request-level policy differencing. This makes permission expansions during execution observable alongside attack success and task completion.

  • Evaluating the phenomenon across models and systems. We observe authorization drift in both Progent and DRIFT, including all eleven models evaluated with Progent. These findings establish that the phenomenon extends beyond a single model or authorization system.

  • Providing an initial defense for dynamic authorization agents. Building on this measurement, we develop a modular online guard designed to integrate into existing policy-update workflows. It checks candidate policies against the initial policy and trusted grants and rejects flagged updates. Our Progent evaluation demonstrates reduced drift, while the supplementary evaluations further characterize the effects on task completion and the security–utility trade-off.

We have responded to each reviewer’s comments, uploaded a revised manuscript, and provided additional experiments and explanations in response to follow-up questions. These exchanges have helped us strengthen the evidence and clarify the scope of our conclusions.

We would welcome further discussion of any specific concerns that remain. We are willing to conduct additional targeted experiments and revise the manuscript where these would help resolve outstanding questions and improve the work.

We hope that the revised manuscript and supplementary evidence will be considered in the overall assessment of the work’s contributions. Thank you again for your time and for the opportunity to improve our paper through this discussion.

Best regards, AuthDiff Authors

AuthorsOpenReview ↗

General response for summarizing our rebuttal

Dear Area Chair and Reviewers,

Thank you for your constructive feedback and the time devoted to discussing our work. We appreciate the reviewers’ recognition of the importance of examining authorization drift throughout agent execution.

Our central contribution is identifying and measuring authorization drift in existing dynamic authorization systems. AuthDiff compares authorization decisions on concrete requests across policy updates, exposing expansions that task completion and attack success alone can miss. We observe drift in all eleven models evaluated with Progent and also in DRIFT. Building on this measurement, we provide an online guard that checks candidate policies within the existing update workflow.

During the discussion, we provided point-by-point responses, additional experiments, and a revised manuscript. The discussion has progressed as follows:

ReviewerRatingDiscussion progress
dt9A3 → 3Acknowledged that the revisions addressed most concerns and maintained a positive assessment of the paper’s contributions.
Pnyw2 → 3Recognized the contribution to identifying and measuring authorization drift and raised the rating on that basis.
Vro12 → 2Recognized the additional evidence for Q1, while retaining concerns about Q2 and Q3. We subsequently provided further clarifications and execution checks, which have not yet received a further assessment.

Below, we summarize the main evidence and clarifications from the rebuttal.

  1. The finding of drift persists across alternative initial policies and task selection. (Vro1 Q1; Pnyw W2) Shared initial policies and independently omitting each user task preserve drift in all eleven models. Evaluation across all 589 benchmark pairs finds similar drift frequencies on both sides of the initial-denial screening, showing that the finding is not confined to the original screening criterion or any single user task.

  2. Task-scope analysis distinguishes task-related expansions from excessive authorization. (dt9A W1/Q2) Separately classifying expansion witnesses identifies 31 confirmed out-of-scope witnesses under NO-DEFENSE and zero under AuthDiff, with six AuthDiff witnesses remaining unclassified. This supports a reduction in confirmed expansions beyond the task scope, separately from task-related expansions.

  3. Additional evaluations quantify the security–utility trade-off. (Pnyw W1; dt9A W2/Q1) The full-benchmark evaluation quantifies drift reduction and its task-completion cost, while the no-attack evaluation examines utility during normal execution. Failure analysis identifies cases where rollback blocks a required permission omitted by the initial policy, explaining how the guard can affect legitimate tasks.

  4. The additional controls clarify AuthDiff’s role within dynamic authorization systems. (Vro1 Q2; Pnyw W3; dt9A W2) Comparisons with fixed task rules and grant-aware containment under identical grants clarify AuthDiff’s role in checking policies produced by an existing updater. We also explain the differences from DENY-EXP and manual policy review. The main evaluations introduce no additional grants, while the separate approval experiment tests acceptance of grant-supported expansions.

  5. Component analyses identify concrete detection gaps addressed by negative probes. (Vro1 Q3) We added separate trajectory- and mutation-probe ablations. In email and calendar examples, holding candidate policies and other probe sources fixed, negative probes detect attacker-specified permissions before the agent proposes those operations. Isolated execution checks confirm that removing negative probes permits the attack requests, while the full guard rejects the candidate policies and blocks those requests.

The revised manuscript incorporates the main analyses, with changes highlighted in blue. We will also incorporate the subsequent clarifications on the measurement contribution, defense comparisons, and the trusted-grant protocol.

The discussion has strengthened the evidence for our central finding: authorization policies can acquire unsupported permissions during execution, and examining these intermediate policies reveals security issues that task outcomes alone do not capture. We have made every effort to address the reviewers' concerns, including the follow-up questions raised during the discussion. Thank you again to the Area Chair and reviewers for your time and consideration.

Best regards, AuthDiff Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.