- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PRISM studies least-privilege authorization for tool-using agents. It learns the effects of tool availability on task success through sandbox interventions, induces reusable permission roles using an information bottleneck, and combines worst-case attack modeling with GFlowNet-based search for diverse authorization schemes. A task-description router assigns roles at deployment. Evaluations on AgentDojo, τ-bench, and ToolEmu report a 78.3% reduction in authorized tools per role and a task success rate of 78.2%, alongside necessity-prediction and attack-reachability results.
Critical strengths
-
The motivation is clear, and intervention-based privilege analysis is practically valuable. Disabling tools and rerunning tasks helps distinguish frequently invoked privileges from those needed for task completion. This provides a more direct basis for least-privilege authorization than call frequency alone and can reveal substitution between tools.
-
The framework considers utility, security consequences, and deployment needs together. It optimizes the reachability of high-risk tools rather than privilege count alone, while reusable roles and alternative policy candidates support deployment and human review. Evaluation across three benchmarks and component ablations provide useful empirical support for the overall approach.
Critical weaknesses
-
The information-bottleneck formulation, training surrogate, and role-count determination are insufficiently connected.
Equation (4) maximizes , which measures how informative roles are about success or failure rather than directly ensuring task success. The relationship to the task-conditioned surrogate is not adequately derived. Likewise, generally represents an effective category count, and its conversion into an actual role set is unspecified. Figure 2’s entropy axis and annotation also require clarification. Although Table 4 provides useful evidence against task clustering, these formulation and implementation gaps limit the interpretability and reproducibility of the central mechanism. -
Feasibility enforcement and the statistical evidence for utility preservation remain unclear.
Equation (6) imposes a hard predicted-success constraint, whereas the GFlowNet reward combines success, risk, and privilege cost without specifying how infeasible candidates are excluded. The reported task evaluations provide empirical support, but aggregate results do not establish the accuracy of feasibility predictions on optimization-selected masks, particularly given the acknowledged violations of the monotonicity prior. Finally, the nonsignificant difference between 78.2% and 79.4% success () does not establish equivalence or non-inferiority. A confidence interval and an explicit acceptable utility-loss margin are needed to assess the utility-preservation claim.
Questions for authors
-
Please specify the joint distribution underlying Equation (4) and derive the relationship between the task-conditioned success surrogate and . Why should this objective preserve task utility rather than merely improve the predictability of success or failure from the role?
-
How is converted into an actual role set, including the codebook limit and any merging or pruning rules? Does Figure 2’s horizontal axis represent entropy or role count? Please provide the corresponding entropy values and numbers of nonempty roles.
-
How does GFlowNet sampling enforce the constraint in Equation (6)? Are final candidate policies revalidated in the sandbox? Please report the discrepancy between predicted and observed success on the policies selected by optimization, and the effect of monotonicity regularization on these results.
-
Please specify the test underlying , the pairing unit, and aggregation across tasks, models, and repeated runs, together with a confidence interval for the success-rate difference. What reduction in success is considered acceptable when claiming utility preservation?