- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates how a tool-using LLM agent should update prior user authorization when only part of an instruction changes mid-task. The authors propose the Authorization Closure Graph (ACG), a versioned directed acyclic graph whose nodes represent user authority, authoritative environment evidence, and derived consequences with six typed operations that update the graph. Experiments on tau bench show ACG achieving the high action safety and introduces a synthetic benchmark ClosureBench, which isolates the contributions of different failure modes.
Critical strengths
The problem is well motivated. Authorization decisions targeted via time of check time of use is meaningful and identifying where certain decisions no longer apply is important.
Tying approvals to the state they were issued is useful and the refusal of unauthorized requests is important.
Critical weaknesses
My biggest concern is simply that the contributions, the graph and typed actions are not justified. Far simpler approaches can be imagined that solve the proposed problem. For example, at each approval one could store conditions or versions of the facts it depends on, scoping it to the relevant resources, and rechecking before execution already prevents stale approvals and keeps unaffected ones valid. The paper’s own pre-execution check (Equation 10) is essentially this already. General literature on time-of-check-time-of-use (TOCTOU) and the cited work on version bound approvals handles similar concepts already. The graph only adds propagation through cached derived values, which can simply be recomputed. The authors should show a case where a flat versioned approval record decides differently from ACG.
In Table 2, Check + repair, which keeps no authorization state across actions, is within one or two tasks of full ACG for several models with higher task success. Relating to the first point this seems to undercut the necessity of the ACG. This is especially problematic since ClosureBench already favors ACG by construction. Variants are defined relative to ACG and the budget counts the unit ACG minimizes. Single seed runs on small sample sizes are likely underpowered for the comparisons that are made.
It seems the formalism overstates the contribution. Proposition 1 is the standard result on minimal elements of a finite partial order, and Proposition 2 is a case split on whether two sets are empty. Neither proposition shows whether Λ is correct, and by extension, whether ACG is correct. Λ is also hand-written per tool but the paper does not report how Λ was constructed for the benchmark or how ACG behaves when Λ is incomplete or wrong.
Also the concept of minimal repair seems to fix a problem the representation creates. Redundant requests only arise because derived values are listed as requirements. Using only base facts and approvals avoids them.
Questions for authors
Why is the full ACG mechanism necessary over a simpler authorization mechanism? What experimental results demonstrate this, and at what level of significance?
How was Λ constructed for each domain? Who wrote it, were there assessments for faithfulness or naturalness, and were the benchmark tasks or trajectories consulted while writing it?
How does ACG perform when Λ is incomplete or contains errors, such as a missing dependency or a wrong authority mode?
9 comments
Response to Reviewer fpuw (Part 2)
For example, in an email task, the user initially asked the agent to label priority-1 and priority-2 threads as P1 and P2 and delete priority-3 labels, then asked it to keep the priority-3 labels. The remaining work was to relabel 10 priority-1 and 10 priority-2 threads. After this change, ACG requested approval in batches, for example, to label threads A, B, and C as P1. Once the user agreed, updating A consumed only A’s approval, leaving the approvals for B and C available for their updates. ACG completed all 20 updates with 8 authorization requests, and all subsequent writes used approvals already obtained and still unused. Check only clears the remaining approvals after each write, leaving the model to ask again when the next operation is blocked. Check + repair adds a mechanism that actively requests missing approval on the model’s behalf, but it still clears the other approvals after each write and therefore repeats those requests. They made 13 and 17 authorization requests, respectively, and neither finished within the step limit. This example shows why full ACG combines repair with authorization retention: asking for missing approval helps the agent proceed, while preserving unused approvals prevents repeated questions and enables it to complete the revised task.
Regarding minimal repair (W5), the reply above illustrates the need to check authorization for derived consequences. Minimal repair provides local, deterministic request reduction within ACG’s dependency-graph representation: it removes requests recoverable through other repairs and asks only for the remaining evidence or approval, rather than establishing that alternative representations necessarily cause repeated questions.
We will include the above results and discussion in the revised paper.
[1] InterventionBench: Can LLM Agents Adapt to Mid-task User Interventions? Findings of EMNLP, 2026.
W3: Concerns about what the two propositions establish
Thank you for raising this question. Both propositions describe how the authorization check works for a given specification. Whether that specification is faithful or not is a separate question. Proposition 1 uses a standard argument about finite partial orders to identify which missing requirements the user must supply. Proposition 2 determines the next step for each proposed action. ACG blocks an action that violates a business rule, requests missing evidence or approval, and otherwise proceeds to the final check before execution.
We will present both propositions as properties of the proposed procedure. Our contribution is to apply them to authorization and repair as users revise their commitments.
W4 & Q2 & Q3: Ask for the origin of Λ and the consequences of specification errors
Thank you for this important question. We wrote and verified both domain specifications with assistance from a generative model, using the benchmark tools, business policies, and state changes. We consulted benchmark tasks during development and did not tune the rules on test results. Each write tool has a rule that applies across users and tasks. We checked both permitted and prohibited cases and covered all six Airline write tools and all seven Retail write tools.
The effect of an error in Λ depends on the rule. In an Airline cancellation, for example, the customer approves a single \80 refund. The correct rule computes the total directly from that refund. If the rule requires separate approval for the total, ACG asks the customer to confirm the same \80 twice, but the cancellation still proceeds correctly. An omitted dependency can affect the outcome itself. If the rule omits the check on the current refund amount, ACG may miss a change in the refund and the write action may unsafe.
If Λ has no rule for a proposed write, ACG withholds the call and keeps the conversation open. The fallback tells the user that nothing has changed and offers a supported alternative or the option to request human assistance. We discussed the limitations of the current specification in the Limitations section of the original paper. In Section 5 of the revised paper, we have described in more detail the problem ACG addresses and the settings in which it applies. We have also discussed how future work could extend ACG to learn missing rules. In Appendix A.5, we have clarified how we construct Λ and described the steps in more detail to help readers avoid misunderstanding or misapplying ACG.
Response to Rebuttal
I thank the authors for the detailed response and the additional experiments.
I appreciate the reframing of the propositions as properties of the procedure. I think that is more accurate and addresses my original concerns.
On the main concern, the authors admit that a sufficiently expressive flat approval system could reproduce ACG’s decisions. This seems to reinforce my concern. If the contribution is no longer that the graph is necessary to audit such actions but rather for "authorization-update and repair procedure(s)", I am not sure how this contribution is empirically validated in a meaningful way that differentiates it from flat approvals.
Similarly, the response on minimal repair says it reduces requests within ACG’s representation, without establishing that other representations produce redundant requests. This seems consistent with my original point that the redundancy comes from listing derived values as requirements. I would like to note that this weakly contradicts the idea of a "systematic way to construct and maintain these dependencies" if the method also must repair redundancy issues caused by the method.
On Λ, I appreciate the disclosure about generative-model assistance and consulting benchmark tasks. The question of how ACG behaves under specification errors is still answered by example rather than by experiment though.
Overall, I think the rebuttal is making progress, but my concerns remain that the closure graph’s necessity beyond versioned approvals is not yet established. I would revisit my assessment given (1) a baseline that keeps unused approvals, stores the versions of the base facts each depends on, and revalidates before execution, compared with ACG on InterventionBench with a hypothesis test, compared on the original claim or the new claim of "authorization-update and repair procedures" to differentiate it from neighboring work and (2) a perturbation study showing how ACG degrades when dependencies in Λ are omitted or mis-specified (Xcqx asks for similar). I recognize that the authors may have reached the character limit in their previous response and would encourage editing of the manuscript as necessary.
Response to Reviewer fpuw (Part 1)
Thank you for your rigorous and detailed review. We will try our best to answer all your questions.
W1 & W2 & W5 & Q1: Concerns about the benefit over flat versioned approvals and simple checks
Thank you for this important question. The following Retail example explains the role of dependency tracking in your comparison with flat versioned approvals.
Consider two pending Retail orders, O1 and O2. The customer has separately approved cancelling each order and receiving the disclosed refund. For O1, the approved refund is 80 dollars to the bank card. Before execution, verified payment records show that O1 was paid with 50 dollars on the card and 30 dollars in store credit. The cancellation call, order status, and total refund are unchanged, but the money will now return to two different destinations. A simple flat record checking only those unchanged fields would still authorize O1 using the approval for the old refund. ACG tracks how payment records determine the refund allocation and which approval covers that allocation. It therefore asks the customer to confirm the cancellation with the revised 50-dollar card refund and 30-dollar store-credit refund, then revalidates and executes O1. O2 can proceed under its unaffected, unused approval. This example shows how ACG detects a change in the actual consequences even when the action arguments and refund total stay the same, and repairs only the affected approval.
We agree that a sufficiently expressive flat approval system could reproduce this decision by explicitly tracking every relevant upstream fact, derived consequence, permission scope, and version. Our claim is therefore not that graph representation is uniquely capable of expressing safe authorization decisions. Rather, ACG provides a systematic way to construct and maintain these dependencies across actions, propagate only relevant changes, and identify the missing authority without rebuilding unrelated approvals. A flat implementation with equivalent dependency indexing and consequence propagation would share much of ACG's underlying mechanism, even if represented differently. The contribution lies in this integrated authorization-update and repair procedure, rather than in graph representation alone.
In Table 2 of the paper, we agree that the differences between ACG and the two variants are small on some metrics. ClosureBench was designed to validate the mechanisms of our method. To further evaluate our main claim, we used InterventionBench [1], which tests whether agents can continue a task correctly after the user changes the instructions midway through execution. For example, a user may initially request a transfer to a recipient followed by a text notification, then ask the agent to proceed with the transfer but skip the notification. The benchmark provides a broad set of realistic tasks with such instruction changes.
We screened the 100 valid official VerbDrop/TargetDrop instances, retained the 75 whose instructions explicitly withdraw an action or target while preserving other state-changing work, and selected one intervention per distinct base task, yielding 58 tasks, and evaluated Raw, ACG, and the two ablation variants using DeepSeek-V4.1-flash on these tasks.
Table R1. Task success and safety are compared on 58 InterventionBench tasks with explicit mid-task instruction changes.
As Table R1 shows, ACG achieved higher task success and safety than Raw and both variants.