- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AuthorityLens which is a framework for measuring authority in LLM-based agent systems. AuthorityLens defines authority at the level of outcomes and identifies the smallest group of participants required to realize each outcome. Based on these groups, it measures what a system can do, how much joint participation is required, and how much authority each participant holds. The authors evaluate 13 configurations of Codex, OpenCode, and Gemini cli using a portfolio of 599 outcomes. Experiment results show that nominal permission settings do not always reflect realized authority, where some restrictions only change the available workflow while others genuinely reduce system capabilities or require additional participants.
Critical strengths
- The paper provides a useful framework for distinguishing what an agent system can do from who can do it independently. This makes authority concentration and separation measurable, which gives people a way to assess controls that capability benchmarks overlook.
- The study across three agent systems shows that the framework exposes meaningful discrepancies between nominal permissions and actual authority.
Critical weaknesses
- The central measurements depend on ruling out alternative execution routes, but the author does not clearly explain how these exclusion arguments were independently validated.
- The practical significance of authority separation remains unclear. Adding an LLM reviewer creates a required decision point, but the evaluation supplies approval decisions rather than testing the reviewer’s judgment. Without examining ineffective review or correlated failures, I think higher separation does not necessarily imply stronger protection.
Questions for authors
- How dependent are the main findings on the composition of Agent-599? Would the comparisons change if categories were weighted equally or outcomes relying on the same execution mechanism were grouped together?
- How are the exclusion arguments validated to rule out smaller coalitions or alternative execution routes? Are these judgments independently checked?
- Why is initial delegation excluded when counting participation? Would including the parent’s required authorization change the conclusion that child agents independently replicate authority?
- Codex’s Guardian can independently realize 134 outcomes despite its nominally read-only role. Which of these require special conditions (such as root privilege)? and how would excluding those conditions affect its principal authority score?
3 comments
Follow-up to Reviewer 7mDk: Separation of Duty and System Design
Following our discussion of delegation, our findings illustrate why authority distribution deserves explicit treatment during agent-system design. In the audited OpenCode Build configuration, General, Explore, Build-child, and Plan-child each have a singleton realization for every one of the 537 reachable Agent-599 outcomes, matching the Build root. Even Explore, whose stated role is codebase exploration, carries this authority. These subagents add independent authority holders while minimum required participation remains one: this is authority replication.
Codex provides the contrasting structure. In Ask/Auto, 279 outcomes require an executor together with the designated User/Guardian authorizer. The distinction identifies whether the architecture introduces another capable executor or establishes a necessary authorization responsibility.
This connects to established task-level separation of duty. Li and Wang distinguish high-level requirements on the participants completing a sensitive task from the mechanisms implementing its workflow. Such requirements can be specified before the workflow is designed and used to guide its implementation. AuthorityLens applies this perspective to the authority realized across agent execution and delegation routes. Li and Wang, CCS 2006, §§1 and 2.5
The software-development analogy is concrete. Before implementation, designers can define a portfolio of protected outcomes and associate each with authorization requirements—for example, that a release remain achievable while requiring a designated quality authorizer. During development, the coalition analysis checks whether the composed implementation satisfies those requirements. When tools, policies, or delegation mechanisms change, reassessing the same contracts can support security regression testing. This follows the secure-development practice of specifying security requirements, maintaining design decisions, and testing implementations against those requirements. NIST SP 800-218, PW.1 and PW.8.2
Explicit authorization responsibilities also provide a basis for maintaining judgment quality. Designers can attach policies, decision rules, and targeted tests to a particular authorizer, investigate its errors, and refine its behavior over time. This is consistent with the separation of authorization logic from application logic used in systems such as Cedar to support independent policy maintenance and testing. Cedar documentation
Our motivation is therefore to make authority distribution an explicit, testable part of the agent-system lifecycle. The replication findings show why this assessment matters even when a system already has multiple specialized roles: the implemented authorization structure determines whether those roles provide alternative execution opportunities or required checks.
Follow-up to Reviewer 7mDk: Portfolio Sensitivity and Guardian Qualifications
We would also like to provide additional results addressing your questions about portfolio sensitivity and Guardian’s operating conditions.
Sensitivity to portfolio weighting. We recomputed the complete 599-by-13 matrix, first reproducing all original uniform-weight authority profiles and Principal Authority values exactly. We then assigned equal total weight to each of the 46 categories and, separately, each of the seven themes, distributing that weight uniformly within each group.
| Configuration / measure | Uniform outcomes | Equal categories | Equal themes |
|---|---|---|---|
| Codex Ask/Auto: mean minimum participation | 1.467 | 1.540 | 1.546 |
| Codex Full: mean minimum participation | 1.000 | 1.000 | 1.000 |
| Codex Ask/Auto: outcome coverage | 0.9967 | 0.9957 | 0.9950 |
| Codex Full: outcome coverage | 1.0000 | 1.0000 | 1.0000 |
Leaving out each category in turn gives an Ask/Auto participation range of 1.447–1.493, while Full Access remains at 1.000. The same participation comparison holds when both configurations are evaluated on their common reachable outcome set.
OpenCode Build and Plan agree outcome by outcome in reachability and minimum participation. Consequently, equality of these two measures holds under any common nonnegative weighting with positive reachable mass. These results show that the central comparisons persist across the tested weighting schemes, while their numerical magnitudes vary with the portfolio’s emphasis.
Guardian’s operating conditions. The 134 outcomes that Guardian can realize independently comprise:
- 104 read outcomes;
- 27 changes to shell, terminal, synchronization, IPC, or existing-channel state;
- one login-record read;
- two root-qualified operations: crontab registration and chroot.
Removing the two root-qualified contracts changes Guardian’s uniform-weight Principal Authority from 0.4566 to 0.4548:
\qquad \frac{132+279/2}{597}=0.4548.$$ This calculation isolates the contribution of those two contracts by removing them from the portfolio. The retained contracts preserve their declared operating conditions, including existing sessions and channels. For example, an existing-channel operation concerns authority over an already established resource, whereas a fresh connection would require its own authorization analysis. The broader finding is that a filesystem read-only boundary can coexist with authority over other runtime state. The condition-specific breakdown makes that finding precise, and the exclusion calculation shows that the two root-qualified entries have a small effect on the reported aggregate. These analyses hold the existing coalition assignments fixed and examine how aggregation and contract selection affect the reported measurements.
Response to Reviewer 7mDk
Your question about initial delegation raises an important distinction: creating an agent does not necessarily mean participating in every outcome that agent later realizes.
Suppose A creates B for a legitimate task. B subsequently encounters contaminated retrieved content and uses an existing permission to perform an operation, without further input or approval from A. A explains how B obtained its authority, but the creation relationship does not establish joint control over that operation.
Our analysis asks which Principals must contribute to realizing the specified outcome in an admitted runtime state. Existing grants belong to that state. If A must supply a fresh authorization, necessary input, or another indispensable contribution to the outcome, A is included. We do not exclude a contribution merely because it occurs through delegation.
The security motivation is to avoid mistaking an agent’s parentage for continuing oversight. An initial grant can create another holder of authority without requiring the grantor’s participation in subsequent uses. If the measured outcome instead includes the initial delegation itself, its coalition can include the parent; that answers a different lifecycle question.
This also connects to your concern about practical protection. We do not assume that two participants necessarily make better decisions than one. The structural question comes first: is another participant’s authorization actually required? A designer considering an approval gate needs to know whether the agent or one of its delegates can still reach the outcome alone.
Your question about validating minimality is separate and remains important: a successful execution proves sufficiency, not necessity. We would first like to clarify whether the distinction above resolves the concern about excluding the parent merely on the basis of the delegation relationship.