- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper formulates authorization drift where LLM agents update authorization policies during execution for task requirements according to tool call results. The paper proposes AuthDiff to measure and constrain drift. With the paper's model and definition, the paper evaluated authorization drifts across models and existing dynamic access control systems and found that authorization drifts are common with current systems, while AuthDiff reduces drift events per update by 86.9% without trading off the success rate of the given tasks.
Critical strengths
- Clear problem formulation and motivation. The paper formalizes authorization drift through request-level permission changes and explains why intermediate policy states deserve evaluation alongside attack success and task completion.
- Interesting findings with authorization drift (RQ1). Drift appears across all eleven evaluated Progent models and also in DRIFT, which gives some insights on how LLM agents adapt to current dynamic authorization systems.
- Artifact and data available in supplementary material.
Critical weaknesses
- Potential initialization and selection bias. Drift attribution depends on model-generated initial policies, and evaluation uses a subset selected through another model's initial denials. An over restrictive anchor can classify legitimate permissions as unjustified expansions, thus making the evaluation biased. Densitivity tests with initialization and sampling choices are needed for soundness.
- The comparison with static permission control favors adaptation in the approval experiment. The paper manually imposed read-only policies which gives 0/24 utility for NO-UPDATE and DENY-EXP, while AuthDiff completes 13/24 tasks requiring added write permissions (Table 3). Fixed, task-aware authorization rules that consume the same authenticated approvals would provide a fairer comparison.
- Component contributions need stronger ablation evidence. The paper evaluates NO-NEG-PROBES, POLICY-REPS and OBSERVE, but POLICY-REPS removes trajectory, mutation and negative probes together. Removing negative probes gives 10 events versus nine for AuthDiff. Separate component ablations are needed to identify the contribution of each component.
Questions for authors
I will consider raising my score if the authors addresses the following.
- How sensitive are the drift findings to the initial policy and task-selection procedure?
- How does AuthDiff compare with fixed, task-aware permission rules that accept the same authenticated user approvals?
- What does each guard component contribute?
10 comments
Thank you for your detailed response. While the new results for Q1 gives solid evidence, results for Q2 and Q3 still seem to lack reasonable explanation for supporting the authors' claims. Therefore I will remain the score as it is.
Further Clarification on Q2 and Q3
Dear Reviewer Vro1,
Thank you for revisiting our responses and for recognizing the additional evidence for Q1. For Q2 and Q3, we would like to explain more clearly the practical role of the dynamic updater and the rationale for combining the probe sources.
Q2: Why retain a dynamic updater?
In an agent workflow, the user may specify a goal before the concrete resources and required tool calls are known. For example, the agent may first search for a document and then determine which file operation is needed. Tool results identify the resource, while the initial authorization and subsequent trusted grants determine which operations are permitted. A dynamic updater translates this evolving task context into executable policies.
Fixed task rules can support such workflows when predefined rule templates and conversion logic cover the required operations and argument constraints. Our fixed-rule control implements this approach by converting structured trusted grants into rules. For broader workflows, using this approach requires providing the corresponding rule-construction logic as part of the authorization system.
AuthDiff allows a system to retain dynamic policy generation while checking whether the resulting policies expand authorization beyond the initial policy and trusted grants. It evaluates the monitor’s decisions on concrete requests and returns expansion witnesses for flagged updates. The practical distinction is therefore between constructing permissions through predefined rule logic and checking policies proposed by a dynamic updater. The close completion counts in our comparison show that these checks can coexist with task completion under the tested grants.
Q3: Why combine the probe sources?
A candidate policy can permit many requests with different arguments. A representative request illustrates a rule’s behavior, but the security-relevant expansion may concern another argument combination. The probe sources examine this permitted scope from complementary directions:
- Trajectory probes test the exact requests arising during execution, connecting policy checks to the actions the agent actually proposes.
- Mutation probes vary arguments to examine how far a permission extends. In the transaction example, they expose that a policy proposed for an amount update also permits changing an unrelated transaction’s recipient.
- Negative probes test attacker-specified requests appearing in untrusted content, allowing the guard to examine those permissions before the agent proposes the corresponding calls.
The fixed-candidate examples demonstrate this complementarity directly: changing the probe source changes detection while the candidate policy remains the same. The design combines policy-derived requests with actual calls, argument variations, and requests from untrusted content, so that detection does not depend on one way of selecting requests. This is the role of the additional probe sources alongside representative probes.
We hope this clarifies the practical motivation and the connection between the evidence and our design. We would appreciate any further indication of which aspect remains unclear.
Best regards, AuthDiff Authors
Kindly reminder
Dear Reviewer Vro1,
Thank you again for your time and thoughtful feedback on our submission.
We have posted our responses and uploaded a revised manuscript, with added and revised text highlighted in blue. As the discussion period draws to a close, we would appreciate hearing whether our responses have addressed your concerns. If there are any remaining questions or points you would like us to clarify, please let us know. We would be happy to continue the discussion.
Best regards, AuthDiff Authors