acceptodds
Under review as a conference paper at ICLR 2027

PRISM: Causal Counterfactual Minimization of Agent Tool Privileges via Information Bottleneck Role Induction

Abstract

This paper shifts the object of role engineering from human users to large language model (LLM) agents, and proposes PRISM to address the over-authorization that arises when an agent obtains full tool privileges through the Model Context Protocol (MCP). PRISM formulates whether a tool is necessary for a task as an intervenable causal proposition, and applies structured interventions in a sandbox to learn a counterfactual necessity model. It then compresses trajectories into discrete latent variables through an information bottleneck, and determines the number of roles from the knee of the rate-distortion curve. Finally, it formalizes pruning as a bilevel Stackelberg game, in which a generative flow network samples diverse near-optimal solutions. On three public benchmarks, AgentDojo, -bench and ToolEmu, PRISM compresses the number of authorized tools per role by 78.3%, attains a task success rate of 78.2%, reaches a counterfactual necessity accuracy of 88.7%, and lowers the blast radius to 15.9%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agent, least privilege, tool authorization, causal intervention, information bottleneck

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ve6zOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PRISM studies least-privilege authorization for tool-using agents. It learns the effects of tool availability on task success through sandbox interventions, induces reusable permission roles using an information bottleneck, and combines worst-case attack modeling with GFlowNet-based search for diverse authorization schemes. A task-description router assigns roles at deployment. Evaluations on AgentDojo, τ-bench, and ToolEmu report a 78.3% reduction in authorized tools per role and a task success rate of 78.2%, alongside necessity-prediction and attack-reachability results.

Critical strengths
  1. The motivation is clear, and intervention-based privilege analysis is practically valuable. Disabling tools and rerunning tasks helps distinguish frequently invoked privileges from those needed for task completion. This provides a more direct basis for least-privilege authorization than call frequency alone and can reveal substitution between tools.

  2. The framework considers utility, security consequences, and deployment needs together. It optimizes the reachability of high-risk tools rather than privilege count alone, while reusable roles and alternative policy candidates support deployment and human review. Evaluation across three benchmarks and component ablations provide useful empirical support for the overall approach.

Critical weaknesses
  1. The information-bottleneck formulation, training surrogate, and role-count determination are insufficiently connected.
    Equation (4) maximizes I(Z;Y)I(Z;Y), which measures how informative roles are about success or failure rather than directly ensuring task success. The relationship to the task-conditioned surrogate Eτ[log⁡fϕ(τ,m(z))]\mathbb E_\tau[\log f_\phi(\tau,m^{(z)})] is not adequately derived. Likewise, 2H(Z)2^{H(Z)} generally represents an effective category count, and its conversion into an actual role set is unspecified. Figure 2’s entropy axis and k⋆=6.7k^\star=6.7 annotation also require clarification. Although Table 4 provides useful evidence against task clustering, these formulation and implementation gaps limit the interpretability and reproducibility of the central mechanism.

  2. Feasibility enforcement and the statistical evidence for utility preservation remain unclear.
    Equation (6) imposes a hard predicted-success constraint, whereas the GFlowNet reward combines success, risk, and privilege cost without specifying how infeasible candidates are excluded. The reported task evaluations provide empirical support, but aggregate results do not establish the accuracy of feasibility predictions on optimization-selected masks, particularly given the acknowledged violations of the monotonicity prior. Finally, the nonsignificant difference between 78.2% and 79.4% success (p=0.21p=0.21) does not establish equivalence or non-inferiority. A confidence interval and an explicit acceptable utility-loss margin are needed to assess the utility-preservation claim.

Questions for authors
  1. Please specify the joint distribution underlying Equation (4) and derive the relationship between the task-conditioned success surrogate and I(Z;Y)I(Z;Y). Why should this objective preserve task utility rather than merely improve the predictability of success or failure from the role?

  2. How is 2H(Z)2^{H(Z)} converted into an actual role set, including the codebook limit and any merging or pruning rules? Does Figure 2’s horizontal axis represent entropy or role count? Please provide the corresponding entropy values and numbers of nonempty roles.

  3. How does GFlowNet sampling enforce the constraint in Equation (6)? Are final candidate policies revalidated in the sandbox? Please report the discrepancy between predicted and observed success on the policies selected by optimization, and the effect of monotonicity regularization on these results.

  4. Please specify the test underlying p=0.21p=0.21, the pairing unit, and aggregation across tasks, models, and repeated runs, together with a confidence interval for the success-rate difference. What reduction in success is considered acceptable when claiming utility preservation?

Reviewer_QiKwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper puts forth PRISM as a methodology by which to address tool privilege over-authorization in LLMs via MCPs. PRISM looks at the change in task success probability after a do-intervention is applied to a tool mask, collects intervention outcomes (via active sampling), trains a counterfactual necessity model using these outcomes, compresses task trajectories into discrete latent variables and then uses a bilevel Stackelberg game in order to prune and utilize a generative flow network for sampling a family of supposedly optimal solutions with respect to the reward.

The authors also provide results on various popular benchmarks such as AgentDojo, tau-bench, and ToolEmu to corroborate their claims.

Critical strengths
  1. the related work surrounding role engineering and agent security are well presented.
  2. the formalization of a single-point compromise provides ample background and insight into the scope of the threat that PRISM attempts to guard against.
  3. it is encouraging to see use of annotators with security backgrounds doing the partitioning of the tools independently
  4. the use of a case study is motivating of the method's utility in more detail
Critical weaknesses
  1. how critically faithful are these sandboxes to real world deployment environments in which PRISM could be utilized?
  2. how would the paper's methodology extent to a multi agent setup - as is ubiquitous in the current AI climate?
Questions for authors

See above.

Reviewer_whe8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed a method, PRISM, which controls the tool availability to LLM agents, in order to decrease the system vulnerability to unnecessary tool availability and attacks. The very core idea is, in a LLM agent, build finitely many "roles" each consisting of a set of available tools, and assign the most appropriate role (tool set) to the agent in each task.

In building the role system, the paper uses conterfactual analysis in causality to define the necessity of a tool in a given tool set to accomplish a task, by sampling a trajectory of (trajectory, tool mask) pairs and check task performance, an task-tool-successful rate estimation is trained. Then this estimation is applied in balancing among such necessity, toolset size and risk, such an optimization on information bottleneck generates a set of role candidates. Then rate distortion optimization is used to get an optimal number of roles k∗k^\ast, and the roles are chosen from the set of role candidates. Next, a router reading task text is trained to assign a role to each task in application. At last, to defend against attackers, an alternative tool masks (with similar rewards) is chosen for each role to replace the original mask. Each alternative is chosen by the administrator from some samples. The samples are generated by GFlowNet sampler in a Stackelberg game applied to adjust the sampler for several rounds in order to control the attacker's blast radius (a refinement step on diversity based on Jaccard is in the middle).

On experiments, PRISM on 325 tasks in AgentDojo, τ\tau-bench, ToolEmu, an averge of 9.9 tools are assigned to each task, forming a decrease of 78.3%, with successful rate dropping from 78.4% to 77.2%. Accordingly, blast radius is decreaed to 15.9%. In injection cases from AgentDojo, the high-risk tool use rate drops from ~58% on full access, ~20% on Progent to 6.1%. For modules, the necessity estimation outperfroms Shapley sampling. Ablation studies show necessity of each module.

Critical strengths
  1. The necessity of a tool is defined using theory of causality. This is an objective estimation on model and harness abilities, and the conclusion is more strict and accurate.

  2. The paper turns the main problem into the compositon of three subproblems, and all the three subproblems can be solved in a suitable field with a strong theoretical foundation. Ablation study shows strong necessity of each module.

  3. The method reduces vulnerbility without sacrificing much on task successful rate.

Critical weaknesses
  1. There are some key functions undefined, such as Ωcal\Omega_{cal}. There are some sampling hyper parameters such as η\eta, κ\kappa, μ\mu, ν\nu, we do not know what their values in experiment and how they are chosen. But we believe that these parameters may affect the learning results.

  2. In trajectory sampling described in 3.1, tasks and mask optimized at the same time. Such a choice may have reasons in active learning, but I want to point out that this may cause the marginal distribution on task very different from real task distribution. I am afraid that those tasks which seem to be certain at the beginning may not have much chance to be visited again, though the estimation may be incorrect.

  3. The definition of blast radius is defined by the paper, which works both as an optimizaiton goal in Stackelberg policy search and in the mehtod compairson. And the graph that blast radius is built on is also defined by the paper. It could be more convincing if the authors can provide futher discussion on the definitions.

  4. Modern model providers may use KV-cache reusing technology to reduce the inference costs, with cache hits often costing as little as one-tenth of the price of cache misses. And harness systems such as Claude Code, often optimizes the context organizations to maximally benefit from KV-cache reuse. If the tool set changes according to the task, in multi-round harness use (we may consider each interaction as a different task), as the tool set changes, it can cause much more KV-cache misses. Do you have a solution to this?

Questions for authors
  1. In the trajectory sampling, algorithm line 2-6, the system runs in sandbox environment (which I consider are similar to or is exactly the application environment), sometimes the system runs with full or almost full set of tools, could the information leakage caused in such a sampling phase be considered a problem, if we ran the system on real environment? Or, if one must build a sandbox which contains no sensitive information, the cost of building such an environment must be considered in applicaitons.

  2. About the blast radius simply distinguishes high-risk and low-risk tools, but the tool risks are still different, sometimes blast radius may not form a good reference, since one attack on some high-risk tool may be enough to cause big issue, the risk curve over blask radius may be very steep. Moreover, the action risk may also related to the concrete arguments, if we want to assign weights to high risk tools, the arguments may make the variance very large.

  3. The escalation confirmation, in discussion, is considered as a user's burden, but also can be considered as a new attacker's attacking entry.

  4. The paper propposed that PRISM can be composed with other methods like a Progent. But I did not find any test on the combination. I am curious about how much better the composition can perform in experiment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.