acceptodds
Under review as a conference paper at ICLR 2027

FORTIS: Benchmarking Over-Privilege in Agent Skills

Abstract

Large language model agents increasingly operate through an intermediate skill layer that mediates between user intent and concrete task execution. This layer is widely treated as an organizational abstraction, but we argue it is also a safety boundary that current models routinely violate. We present FORTIS, a benchmark that evaluates agent skill safety across two stages: whether a model selects the minimally sufficient skill from a large, overlapping library, and whether it executes that skill without expanding into broader tools or actions than the skill permits. Across ten frontier models and three domains, we find that over-privileged behavior is the norm rather than the exception. Models consistently reach for higher-privilege skills and tools than the task requires, failing at both stages at rates that remain high even for the strongest available models. Failure is especially severe under the ordinary conditions of real user interaction: incomplete specification, convenience framing, and proximity to skill boundaries. The results indicate that the skill layer, far from containing agent behavior, is itself a primary source of unsafe escalation in current systems.

Primary area:
datasets and benchmarks
Keywords:
Benchmark, Trustworthy, Agent Safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7NzcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

LLM agents use a “skill layer”: reusable skill modules (documented in natural language, for instance a SKILL.md file) that say what kind of task to perform and which tools may be used. The idea of the authors is that this layer can be also seen as privilege boundary. For this reason, the paper proposes FORTIS, a benchmark with two tasks. In Task 1 (skill selection) the model sees a user request and short descriptions of all 20 skills in a domain and must find a skill. If this skill is appropriate (minimally sufficient/lowest-privilege skill or uses to much privileges) is one question researched by the authors. A second task (Task 2 skill-grounded tool selection) the model is aware of the skill it uses is shown the full skill document and the whole tool inventory and has to output a list of tool call, The authors investigate if it stays inside the tools that the skill and the request needs to. Ten frontier models are evaluated on 600 Task 1 queries and 1,543 Task 2 queries in three domains (email, e-commerce, filesystem). The main conclusion is that skill boundaries cannot be delegated to the model’s own reading of instructions and must be mechanically enforced at the skill or tool invocation layer.

Critical strengths

The paper addresses a timely issue for security researchers working on least-privilege, permission models and runtime enforcement for LLM agents. It's a pity that the paper did not delve into more access control mechanisms and security mechanisms for AI agents, and provide (besides the theoretical/formal model) a concrete instantiation for the security use case. Similalry, other researchers interested in building benchmarks for in selection-time rather than completion-time evaluation. Another target audience is made of researchers having to chose between relying on model self-restraint or on mechanical checks. The idea of the paper is original and the state of the art summary is sufficient. The main results are consistent in the sense that no model reaches reliable minimal-privilege selection in any domain, and the pattern repeats across model families. The idea of the benchmark itself poses a well-framed problem and measuring restraint under benign ambiguity, with a two-stage structure (skill selection, then skill-grounded tool selection) and an end-to-end funnel, is conceptually different from previous known research work.

Critical weaknesses

There is no condition with an explicit least-privilege instruction, with privilege labels shown, or with a filtered tool inventory, so the claim that the fix lies at the architecture, or training-objective level is not tested against these cheap mitigations. The audit reviewed 200 of 2,143 queries by two authors. This was not done by independent annotators, and reports raw agreement (193/200) without kappa or the label-invalid rate. Agreement between reviewers is not the same as the share of labels that are valid. There are no confidence intervals, a single run per model at temperature 0, and the per-setting sample sizes are not stated. The EM1 × EM2 assumes independence between tasks on different query sets. The reviewer is not convinced that this assumption holds on the general case. The paper does not propose a solution to the identified problem. The authors argue that the model should not be the only enforcement point for scope, but they do not test alternative approaches.

Questions for authors

Task 1 over-privilege is defined by privilege level, ℓS(ŝ) > ℓS(s*). Can we assume that choosing a higher-level skill can always do the task ? How was the number of privilege levels decided to the set {0,1,2,3,4} for tools and skils ? What happens if the number of levels goes up ? For task1, the prompt deliberately contains no least-privilege instruction (Appendix B.2), and higher-level skills are described with convenience language. What happens if the prompt includes explicit least-privilege instructions ? How can an agent be forced/incentivized to use the adequate level of skills/tools? Why is only over-privilege treated as the main outcome? What are the rates of incorrect-but-lower-privilege picks? The ordinal levels are used for ranking and gaps. What justifies treating the distance from L1 to L2 as comparable to L3 to L4? Skills and tools are authored and constructed for three domains. How would you expect the results to change on real, independently written skill registries?

2 comments

Q1: There is no condition with an explicit least-privilege instruction, with privilege labels shown, or with a filtered tool inventory, so the claim that the fix lies at the architecture, or training-objective level is not tested against these cheap mitigations.

We evaluated explicit least-privilege instructions across all three domains and both tasks:

ModelTask 1 failure rate: baseline → explicit instructionTask 2 failure rate: baseline → explicit instruction
GPT-5.4-mini45.5% → 39.5%59.3% → 55.1%
Claude Opus 4.735.5% → 26.0%47.4% → 41.5%

Explicit instructions reduce failure rates by 6.0–9.5 percentage points on Task 1 and approximately 4.2–5.9 points on Task 2. These results show that a simple prompting intervention helps, while substantial failures remain even when least-privilege behavior is explicitly requested. They motivate further investigation of training and architectural approaches, rather than establishing that such approaches are the only possible remedies.

Q2: The audit reviewed 200 of 2,143 queries by two authors. This was not done by independent annotators, and reports raw agreement (193/200) without kappa or the label-invalid rate. Agreement between reviewers is not the same as the share of labels that are valid.

Ground-truth validity in FORTIS is supported by both controlled construction and manual auditing. The benchmark is designed to exclude equally valid alternatives at the same or lower privilege level, and Appendix A.6 explicitly treats the existence of such alternatives as grounds for invalidating an annotation.

The two-author audit checked whether these construction constraints held in the sampled queries. The reported 193/200 agreement measures consistency between the auditors, rather than the proportion of valid labels. The seven disagreements were resolved using the stated minimum-privilege criterion, with annotations refined where necessary.

Q3: There are no confidence intervals, a single run per model at temperature 0, and the per-setting sample sizes are not stated.

We used a fixed temperature of 0 to limit sampling variability and maintain a consistent evaluation protocol across models. The main finding is supported by recurring patterns across ten models, three domains, and multiple evaluation settings, rather than a single model comparison. Our central conclusion concerns the substantial failure rates observed across these settings, rather than small differences between individual models.

The per-setting sample sizes are provided below. Task 1 uses the same distribution in each domain, while Task 2 counts vary by domain.

TaskSettingEmailE-commerceFilesystemTotal
Task 1Clean Baseline404040120
Task 1Scope-Ambiguous505050150
Task 1Lexical-Ambiguous707070210
Task 1Action-Implication404040120
Task 2Clean Baseline240214200654
Task 2Convenience-Sensitive164120180464
Task 2Broad-Action Justified909590275
Task 2Boundary-Sensitive307050150

These sum to 600 queries for Task 1 and 1,543 queries for Task 2.

Q4: The EM1 × EM2 assumes independence between tasks on different query sets.

The product EM₁ × EM₂ is intended as an illustrative estimate of how errors at two stages can compound, rather than a directly measured end-to-end success rate. We agree that independence is not guaranteed because the two tasks use different query sets. Accordingly, the product should be interpreted under this explicit assumption, rather than as a general upper bound. Our main findings rely on the separately evaluated results for skill selection and skill-grounded tool selection, and do not depend on this estimate.

Q5: The paper does not propose a solution to the identified problem. The authors argue that the model should not be the only enforcement point for scope, but they do not test alternative approaches.

Our least-privilege prompting ablation evaluates a simple intervention across all three domains and both tasks:

ModelTask 1 failure rate: baseline → explicit instructionTask 2 failure rate: baseline → explicit instruction
GPT-5.4-mini45.5% → 39.5%59.3% → 55.1%
Claude Opus 4.735.5% → 26.0%47.4% → 41.5%

These results show that explicit guidance improves performance, while substantial failures remain. Beyond prompting, repeated human-in-the-loop approvals can introduce workflow friction, motivating methods that train least-privilege restraint directly into models. Promising directions include preference training that favors minimally sufficient capabilities and skill-aware planning. FORTIS provides a testbed for evaluating these approaches jointly on privilege restraint and selection correctness.

Q7: Can we assume that choosing a higher-level skill can always do the task?

Higher privilege does not automatically imply functional suitability. FORTIS constructs overlapping skills so that a task admits both a minimally sufficient skill and a broader, higher-privilege alternative. The evaluation tests whether the model chooses the minimally sufficient option among these alternatives.

Q8: How was the number of privilege levels decided to the set {0,1,2,3,4} for tools and skils ? What happens if the number of levels goes up?

The five levels capture distinct operational scopes shared across the three domains: observation, content access, annotation or modification, execution, and bulk or administrative operations. This provides a consistent hierarchy for comparing narrower and broader capabilities.

The formulation does not require exactly five levels. Additional levels can represent finer distinctions in authority while retaining the same minimum-privilege objective. If a refinement preserves the ordering and minimum sufficient solutions for existing queries, their classifications remain unchanged; refinements that change those relationships require corresponding ground-truth updates.

Q9: For task1, the prompt deliberately contains no least-privilege instruction (Appendix B.2), and higher-level skills are described with convenience language. What happens if the prompt includes explicit least-privilege instructions? How can an agent be forced/incentivized to use the adequate level of skills/tools? Why is only over-privilege treated as the main outcome? What are the rates of incorrect-but-lower-privilege picks?

Over-privilege is the primary safety outcome because it captures the selection of authority beyond what the task requires. Selection correctness is also evaluated through exact match.

Separating strictly lower-privilege errors from incorrect selections at the same privilege level gives the following Task 1 breakdown:

ModelExact matchHigher-privilege selectionsIncorrect lower-privilege selectionsIncorrect same-level selections
GPT-5.4-mini43.50%45.50%0.50%10.50%
Claude Opus 4.754.83%35.50%1.17%8.50%

For these models, higher-privilege selections substantially outnumber lower-privilege errors, supporting the focus on unnecessary authority expansion.

Q10: The ordinal levels are used for ranking and gaps. What justifies treating the distance from L1 to L2 as comparable to L3 to L4?

The privilege levels are ordinal categories. The over-privilege criterion uses their ordering to determine whether the selected capability exceeds the minimum required level; it does not require equal distances between adjacent levels. A transition from L1 to L2 and one from L3 to L4 both indicate an increase in authority, but are not assumed to represent equal increases in risk. Level gaps describe the number of hierarchy boundaries crossed, rather than a calibrated measure of harm.

Q11: Skills and tools are authored and constructed for three domains. How would you expect the results to change on real, independently written skill registries?

Independently authored skill registries may introduce greater variation in documentation, overlapping functionality, and implicit permission boundaries, potentially making minimum-privilege selection more difficult. Conversely, narrower tool exposure or stronger enforcement could reduce over-privilege. We therefore expect performance to depend on the registry’s documentation and access-control design. FORTIS provides a controlled reference for studying these factors systematically; the direction and magnitude of changes on real registries require empirical evaluation.

Reviewer_MrkEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces FORTIS, a benchmark for over-priviledge in agents that evaluates whether a model selects the minimally sufficient skill and executes without expanding into broader tools or actions than the skill permits. The benchmark covers skill selection and skill-grounded tool selection across email, e-commerce, and filesystem domains. Across ten frontier models, the authors find that over-privileged behavior is common, especially when requests are incomplete, convenience-oriented, or near a skill boundary.

Critical strengths
  • The paper clearly separates the two-stage evaluation into skill selection and skill-grounded tool selection.
  • It shows the limitations of current models: over-privilege appears across all ten frontier models and all three domains.
Critical weaknesses
  • The benchmark evaluates selection only and does not actually execute tools, so it cannot observe real side effects, recovery costs, or user losses. This leaves the core motivation under-established: what risks arise from using over-privileged skills and tools? What specific security risks are involved? Are these risks serious enough to justify restricting an agent's skill and tool permissions, even if the restriction increases cost and lowers task success rate?
  • The domains and privilege hierarchies are synthetically constructed, covering only email, e-commerce, and filesystem operations. These settings are relatively simple and do not include higher-risk domains such as coding or finance.
  • The judgment of what counts as the minimum-privilege solution is somewhat subjective. For example, in the appendix audit, the two authors disagreed on 7 out of 200 samples about whether a single coarse-grained tool or multiple fine-grained tools constituted the true minimum.
Questions for authors
  • How much would the failure rate drop if the system prompt explicitly required least-privilege behavior, or if an outer permission filter were used?
  • Can the benchmark include actual execution to measure the real harm caused by over-privileged actions? Can it measure how task success rate and cost change when over-privilege is restricted?
  • Are there any training, architectural, or other methods that can reduce over-privilege without substantially harming task completion rate?
2 comments

Q1: The benchmark evaluates selection only and does not actually execute tools, so it cannot observe real side effects, recovery costs, or user losses. This leaves the core motivation under-established: what risks arise from using over-privileged skills and tools? What specific security risks are involved? Are these risks serious enough to justify restricting an agent's skill and tool permissions, even if the restriction increases cost and lowers task success rate?

The concrete risk is that capabilities exceeding the task’s intended scope allow an agent’s planning error to affect resources the user did not authorize it to modify. Consequences can include destructive data changes, unintended disclosure, and unauthorized external actions. For example, Railway confirmed that an agent used an account-scoped token to delete a PocketOS production database volume while attempting to fix an unrelated issue; the database subsequently required recovery (official incident account).

These risks justify studying permission boundaries, but do not imply that every restriction is beneficial regardless of its cost. FORTIS defines the target as the narrowest capability sufficient to complete the task, rather than a restriction that makes the task impossible. Its controlled design allows us to assess unnecessary capability selection separately from whether a broader option appears more convenient.

We agree that our selection-only evaluation does not quantify downstream losses, recovery costs, or the deployment-level trade-off between permission enforcement and task completion. Accordingly, our results support evaluating and controlling unnecessary authority, rather than claiming that a particular restriction policy achieves an optimal safety–cost trade-off.

Q2: The domains and privilege hierarchies are synthetically constructed, covering only email, e-commerce, and filesystem operations. These settings are relatively simple and do not include higher-risk domains such as coding or finance.

The three domains were chosen to cover distinct permission-sensitive operations: access to private content and external communication in email, transactional actions in e-commerce, and potentially destructive modifications in filesystem operations. Although familiar, these domains are not inherently low-risk; the consequences depend on the authority and scope of the selected operation.

The synthetic hierarchies enable controlled comparisons by making minimum task requirements and broader alternatives explicit across domains. This design isolates privilege-sensitive selection from differences in environment complexity.

We agree that these domains do not establish generalization to all agent settings, including coding and finance. FORTIS provides a controlled evaluation of skill-layer restraint across three domains, rather than an exhaustive assessment of high-risk deployments. Extending the evaluation to additional domains and independently authored skill libraries would strengthen its external validity.

Q3: The judgment of what counts as the minimum-privilege solution is somewhat subjective. For example, in the appendix audit, the two authors disagreed on 7 out of 200 samples about whether a single coarse-grained tool or multiple fine-grained tools constituted the true minimum.

FORTIS defines minimum privilege through an explicit lexicographic criterion: first minimize the highest privilege level required by a feasible tool set, and then minimize the number of tools among equally privileged solutions. Therefore, the comparison between a single coarse-grained tool and multiple fine-grained tools follows a fixed ordering, rather than an annotator’s preference. A higher-privilege shortcut is not preferred simply because it requires fewer calls.

The benchmark is also deliberately constructed to exclude same-privilege redundancy and equally valid alternatives at the same or lower privilege level. As described in Appendix A.6, such alternatives are explicitly checked during validation and invalidate an annotation if found.

The seven initial disagreements were resolved through discussion using these criteria, with annotations refined where necessary. The final ground truth therefore reflects the stated minimum-privilege objective and validation procedure, rather than unresolved differences between annotators.

Q4: How much would the failure rate drop if the system prompt explicitly required least-privilege behavior, or if an outer permission filter were used?

We evaluated explicit least-privilege instructions using GPT-5.4-mini and Claude Opus 4.7 across all three domains and both tasks. The instructions ask the model to select the narrowest sufficient skill, use only the necessary tools, and remain within the assigned skill’s scope.

ModelTask 1 failure rate: baseline → explicit instructionTask 2 failure rate: baseline → explicit instruction
GPT-5.4-mini45.5% → 39.5%59.3% → 55.1%
Claude Opus 4.735.5% → 26.0%47.4% → 41.5%

Explicit instructions reduce failure rates by 6.0–9.5 percentage points on Task 1 and approximately 4.2–5.9 percentage points on Task 2. Thus, least-privilege prompting improves performance, but does not eliminate the observed failures: failure rates remain 26.0–39.5% for skill selection and 41.5–55.1% for skill-grounded tool selection. These results demonstrate that the benchmark remains challenging even when least-privilege behavior is explicitly requested.

Q5: Are there any training, architectural, or other methods that can reduce over-privilege without substantially harming task completion rate?

Human-in-the-loop approval, such as tool-use approval prompts in Codex and Claude Code, can introduce substantial friction when users must repeatedly authorize actions. A promising complementary direction is to train least-privilege behavior directly into models through preference training that favors minimally sufficient capabilities and skill-aware planning. This could reduce reliance on repeated approvals while preserving safeguards for consequential actions. FORTIS provides a testbed for evaluating whether such methods improve privilege restraint while maintaining selection correctness.

Reviewer_erXoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FORTIS, a benchmark for evaluating whether LLM agents select minimally sufficient capabilities when using skills. It separates the problem into two tasks: selecting the appropriate skill from an overlapping library, and selecting tools that respect an assigned skill’s documented boundaries. The benchmark contains 600 skill-selection queries and 1,543 tool-selection queries across email, e-commerce, and filesystem domains, covering 60 skills and 174 tools. Evaluations of ten models show frequent departures from the annotated minimum-privilege solutions, especially when requests are ambiguous, broader tools are more convenient, or requested functionality approaches skill boundaries. The results highlight limitations of relying on natural-language skill documentation to guide permission-sensitive decisions, although the evaluation measures static selections rather than actual tool execution.

Critical strengths
  1. The paper highlights a meaningful safety question: whether an agent chooses more powerful capabilities than a task requires, even when its choice is functionally adequate. Examining this behavior under ordinary task requests, without requiring malicious prompt injection, broadens the analysis of agent safety to include risks arising from default routing and planning preferences.
  2. Separating skill selection from tool selection helps distinguish errors in capability routing from failures to respect an already assigned skill. And the paper provide informative analysis of conditions associated with failure.
Critical weaknesses
  1. In Task 2, any predicted tool outside the annotated minimum set is classified as over-privileged. This can potentially include redundant tools or alternative tools at the same or lower privilege level, without establishing an actual boundary violation. Furthermore, the reported failure rate includes empty or unparseable outputs but excludes under-privileged, incomplete solutions. Consequently, the claim that failures overwhelmingly favor greater privilege is partly shaped by the metric definition. Separate measures of unnecessary capability selection, explicit policy violations, task incompleteness, and formatting errors would improve interpretability.
  2. The benchmark uses synthetic skill hierarchies in three domains and does not execute tools, observe resource access, or measure state changes. It therefore cannot establish whether selecting a broader tool results in unauthorized actions or harmful outcomes.
Questions for authors

See Weaknesses.

3 comments

Q1: In Task 2, any predicted tool outside the annotated minimum set is classified as over-privileged. This can potentially include redundant tools or alternative tools at the same or lower privilege level, without establishing an actual boundary violation. Furthermore, the reported failure rate includes empty or unparseable outputs but excludes under-privileged, incomplete solutions. Consequently, the claim that failures overwhelmingly favor greater privilege is partly shaped by the metric definition. Separate measures of unnecessary capability selection, explicit policy violations, task incompleteness, and formatting errors would improve interoperability.

FORTIS is deliberately constructed to avoid same-privilege redundancy and equally valid alternatives at the same or lower privilege level. This is a constraint of the benchmark design, rather than an assumption about arbitrary tool inventories. Appendix A.6 describes the construction and validation procedure; in particular, the existence of a valid same- or lower-privilege alternative is explicitly treated as grounds for invalidating an annotation. Thus, the ambiguity identified by the reviewer is specifically addressed in our benchmark construction. We should make this constraint more prominent alongside the Task 2 metric definition.

FORTIS reports complementary metrics: exact-match rate (EM) measures selection correctness, over-privilege rate (OPR) measures selections classified as over-privileged, and no-action rate (NAR) captures empty or unparseable outputs. The reported failure rate (FR) aggregates the latter two categories: FR = OPR + NAR. Under-privileged, incomplete solutions are recorded separately and receive no exact-match credit. Thus, incomplete solutions are not counted as successful completion, and over-privilege is reported independently of empty or unparseable outputs.

Q2: The benchmark uses synthetic skill hierarchies in three domains and does not execute tools, observe resource access, or measure state changes. It therefore cannot establish whether selecting a broader tool results in unauthorized actions or harmful outcomes.

We agree that FORTIS does not measure execution-time harm. However, the risk motivating this evaluation has concrete real-world precedents. In April 2026, an agent working on an unrelated issue used a locally stored, account-scoped API token to delete a PocketOS production database volume without being instructed to do so. Railway confirmed the deletion and subsequently recovered the database (official incident account). Similar departures from explicit user constraints have been reported in the Replit production-database deletion and OpenClaw inbox-deletion incident.

These incidents motivate evaluating whether agents select capabilities beyond the task’s intended scope before execution. FORTIS uses controlled synthetic hierarchies to make this selection behavior measurable. Its results establish departures from minimum-privilege requirements, rather than demonstrating that every broader selection produces unauthorized actions or harmful outcomes.

Reviewer_erXoOpenReview ↗

Response to Authors

Thank you for the detailed response.

On Q1: Thank you for pointing to Appendix A.6. However, uniqueness of the minimal ground-truth set does not prevent a model from adding extra low-privilege tools (flagged as over_privilege via non-subset matching without exceeding the skill's privilege level). Could you clarify what fraction of Task 2 over_privilege cases strictly exceed the assigned skill's privilege level? Also, since 100% - EM - FR in Table 2 implies 17%–29% under_privilege, please report this column explicitly and temper Section 4.2's claim that failures are "never toward less" privilege.

On Q2: I understand that the authors acknowledge the limits of static evaluation and providing concrete real-world incidents as motivation. Framing FORTIS strictly around pre-execution departures from minimum privileg rather than execution-time harm is reasonable; please ensure this scoped framing is clearly reflected in the revision.

While the metric clarification above would strengthen the paper, I will not object to acceptance if the other reviewers are in favor.

(1) Thank you for clarifying the distinction. Across the ten models, 7,754 Task 2 predictions selected at least one tool outside the reference set. Of these, 3,418 (44.08%) selected a tool whose privilege level strictly exceeded the assigned skill’s level. A further 3,083 (39.76%) stayed within the skill’s level but selected a tool at a higher level than any tool in the reference solution. We used the registered level of the assigned skill and the same privilege mapping for predicted and reference tools.

Together, these two groups account for 6,501 of 7,754 cases (83.84%). Of the remaining 1,253 predictions (16.16%), 983 had the same maximum tool privilege level as the reference solution, 266 had a lower maximum level, and four contained unregistered tool names.

For each model, the denominator below is the number of predictions that selected at least one tool outside the reference set.

ModelExceeded the skill or reference levelOther out-of-reference selections
GPT-5.584.57%15.43%
GPT-5.485.06%14.94%
GPT-5.4-mini85.12%14.88%
Claude Sonnet 4.681.48%18.52%
Claude Opus 4.781.34%18.66%
Gemini 3.1-Pro88.48%11.52%
Gemini 3 Flash82.54%17.46%
Qwen 3.6-Max87.75%12.25%
Kimi K2.685.67%14.33%
DeepSeek-V4-Flash77.33%22.67%
Overall83.84%16.16%

We will add an explicit under-privilege column to Table 2. Across all 15,430 Task 2 predictions, 3,574 (23.16%) selected some but not all of the reference tools and no tools outside the reference set. This rate ranged from 17.11% to 28.90% across models.

We will replace “never toward less” in Section 4.2 with a statement that reflects both outcomes: “All ten models sometimes selected only an incomplete subset of the reference tools. For every model, selecting at least one tool outside the reference set was more common.”

(2) Thank you. We will clarify throughout the revised manuscript that FORTIS evaluates departures from minimum privilege when agents select skills and tools, before execution.

Comments on the paper

2 comments

We thank the reviewers for their constructive comments. We have addressed each point in our rebuttal and would be grateful if you could take a look when you have a chance. We are happy to clarify any remaining concerns.

Dear Reviewers

Thank you very much for your time and thoughtful feedback on our work. As the discussion period is nearing its end, we would be grateful if you could take a look at our rebuttal when you have a chance. Please let us know if there is anything we can clarify or elaborate on; we would be happy to discuss further.

Thank you again for your time and consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.