acceptodds
Under review as a conference paper at ICLR 2027

AuthorityLens: Rethinking LLM-Based Agent Systems Through the Lens of Authority

Abstract

LLM-based agents are increasingly deployed with authority over consequential resources and decisions in real systems. These agents often operate alongside human and LLM-based participants who hold different forms of authority. As these systems become more powerful, controlling and distributing authority within them becomes an increasingly important design concern. Yet workflow roles, permission settings, and review mechanisms do not necessarily reflect the authority realized in practice. We introduce AuthorityLens, a framework for measuring a system's authority structure. Starting from an authority portfolio, we evaluate a system along three dimensions: what the system is authorized to do (System Authority), how much joint participation is required to exercise that authority (Authority Separation), and how much authority each participant holds (Principal Authority). We derive these measurements from the minimal combinations of participants sufficient to realize each outcome across admissible runtime states. We apply AuthorityLens to Codex, OpenCode, and Gemini CLI across 13 operating configurations over a common portfolio of agent operations. We find that nominal configurations do not map cleanly onto realized authority. In Codex, Full Access changes System Authority only marginally while substantially concentrating authority in the executing Assistant. OpenCode's Build and Plan configurations have the same System Authority and Authority Separation despite different workflows and root-level permissions. In Gemini CLI, model-based review increases Authority Separation without changing System Authority. Principal Authority further distinguishes authority replication from authority separation: spawned or delegated agents can become alternative holders of the same authority without increasing the required joint participation. Together, these results demonstrate that AuthorityLens provides a unified framework for measuring and comparing realized authority structures across agent systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, authority, separation of duty, agent security, multi-agent systems, agent harness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_7mDkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AuthorityLens which is a framework for measuring authority in LLM-based agent systems. AuthorityLens defines authority at the level of outcomes and identifies the smallest group of participants required to realize each outcome. Based on these groups, it measures what a system can do, how much joint participation is required, and how much authority each participant holds. The authors evaluate 13 configurations of Codex, OpenCode, and Gemini cli using a portfolio of 599 outcomes. Experiment results show that nominal permission settings do not always reflect realized authority, where some restrictions only change the available workflow while others genuinely reduce system capabilities or require additional participants.

Critical strengths
  1. The paper provides a useful framework for distinguishing what an agent system can do from who can do it independently. This makes authority concentration and separation measurable, which gives people a way to assess controls that capability benchmarks overlook.
  2. The study across three agent systems shows that the framework exposes meaningful discrepancies between nominal permissions and actual authority.
Critical weaknesses
  1. The central measurements depend on ruling out alternative execution routes, but the author does not clearly explain how these exclusion arguments were independently validated.
  2. The practical significance of authority separation remains unclear. Adding an LLM reviewer creates a required decision point, but the evaluation supplies approval decisions rather than testing the reviewer’s judgment. Without examining ineffective review or correlated failures, I think higher separation does not necessarily imply stronger protection.
Questions for authors
  1. How dependent are the main findings on the composition of Agent-599? Would the comparisons change if categories were weighted equally or outcomes relying on the same execution mechanism were grouped together?
  2. How are the exclusion arguments validated to rule out smaller coalitions or alternative execution routes? Are these judgments independently checked?
  3. Why is initial delegation excluded when counting participation? Would including the parent’s required authorization change the conclusion that child agents independently replicate authority?
  4. Codex’s Guardian can independently realize 134 outcomes despite its nominally read-only role. Which of these require special conditions (such as root privilege)? and how would excluding those conditions affect its principal authority score?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 7mDk

Your question about initial delegation raises an important distinction: creating an agent does not necessarily mean participating in every outcome that agent later realizes.

Suppose A creates B for a legitimate task. B subsequently encounters contaminated retrieved content and uses an existing permission to perform an operation, without further input or approval from A. A explains how B obtained its authority, but the creation relationship does not establish joint control over that operation.

Our analysis asks which Principals must contribute to realizing the specified outcome in an admitted runtime state. Existing grants belong to that state. If A must supply a fresh authorization, necessary input, or another indispensable contribution to the outcome, A is included. We do not exclude a contribution merely because it occurs through delegation.

The security motivation is to avoid mistaking an agent’s parentage for continuing oversight. An initial grant can create another holder of authority without requiring the grantor’s participation in subsequent uses. If the measured outcome instead includes the initial delegation itself, its coalition can include the parent; that answers a different lifecycle question.

This also connects to your concern about practical protection. We do not assume that two participants necessarily make better decisions than one. The structural question comes first: is another participant’s authorization actually required? A designer considering an approval gate needs to know whether the agent or one of its delegates can still reach the outcome alone.

Your question about validating minimality is separate and remains important: a successful execution proves sufficiency, not necessity. We would first like to clarify whether the distinction above resolves the concern about excluding the parent merely on the basis of the delegation relationship.

AuthorsOpenReview ↗

Follow-up to Reviewer 7mDk: Separation of Duty and System Design

Following our discussion of delegation, our findings illustrate why authority distribution deserves explicit treatment during agent-system design. In the audited OpenCode Build configuration, General, Explore, Build-child, and Plan-child each have a singleton realization for every one of the 537 reachable Agent-599 outcomes, matching the Build root. Even Explore, whose stated role is codebase exploration, carries this authority. These subagents add independent authority holders while minimum required participation remains one: this is authority replication.

Codex provides the contrasting structure. In Ask/Auto, 279 outcomes require an executor together with the designated User/Guardian authorizer. The distinction identifies whether the architecture introduces another capable executor or establishes a necessary authorization responsibility.

This connects to established task-level separation of duty. Li and Wang distinguish high-level requirements on the participants completing a sensitive task from the mechanisms implementing its workflow. Such requirements can be specified before the workflow is designed and used to guide its implementation. AuthorityLens applies this perspective to the authority realized across agent execution and delegation routes. Li and Wang, CCS 2006, §§1 and 2.5

The software-development analogy is concrete. Before implementation, designers can define a portfolio of protected outcomes and associate each with authorization requirements—for example, that a release remain achievable while requiring a designated quality authorizer. During development, the coalition analysis checks whether the composed implementation satisfies those requirements. When tools, policies, or delegation mechanisms change, reassessing the same contracts can support security regression testing. This follows the secure-development practice of specifying security requirements, maintaining design decisions, and testing implementations against those requirements. NIST SP 800-218, PW.1 and PW.8.2

Explicit authorization responsibilities also provide a basis for maintaining judgment quality. Designers can attach policies, decision rules, and targeted tests to a particular authorizer, investigate its errors, and refine its behavior over time. This is consistent with the separation of authorization logic from application logic used in systems such as Cedar to support independent policy maintenance and testing. Cedar documentation

Our motivation is therefore to make authority distribution an explicit, testable part of the agent-system lifecycle. The replication findings show why this assessment matters even when a system already has multiple specialized roles: the implemented authorization structure determines whether those roles provide alternative execution opportunities or required checks.

AuthorsOpenReview ↗

Follow-up to Reviewer 7mDk: Portfolio Sensitivity and Guardian Qualifications

We would also like to provide additional results addressing your questions about portfolio sensitivity and Guardian’s operating conditions.

Sensitivity to portfolio weighting. We recomputed the complete 599-by-13 matrix, first reproducing all original uniform-weight authority profiles and Principal Authority values exactly. We then assigned equal total weight to each of the 46 categories and, separately, each of the seven themes, distributing that weight uniformly within each group.

Configuration / measureUniform outcomesEqual categoriesEqual themes
Codex Ask/Auto: mean minimum participation1.4671.5401.546
Codex Full: mean minimum participation1.0001.0001.000
Codex Ask/Auto: outcome coverage0.99670.99570.9950
Codex Full: outcome coverage1.00001.00001.0000

Leaving out each category in turn gives an Ask/Auto participation range of 1.447–1.493, while Full Access remains at 1.000. The same participation comparison holds when both configurations are evaluated on their common reachable outcome set.

OpenCode Build and Plan agree outcome by outcome in reachability and minimum participation. Consequently, equality of these two measures holds under any common nonnegative weighting with positive reachable mass. These results show that the central comparisons persist across the tested weighting schemes, while their numerical magnitudes vary with the portfolio’s emphasis.

Guardian’s operating conditions. The 134 outcomes that Guardian can realize independently comprise:

  • 104 read outcomes;
  • 27 changes to shell, terminal, synchronization, IPC, or existing-channel state;
  • one login-record read;
  • two root-qualified operations: crontab registration and chroot.

Removing the two root-qualified contracts changes Guardian’s uniform-weight Principal Authority from 0.4566 to 0.4548:

\qquad \frac{132+279/2}{597}=0.4548.$$ This calculation isolates the contribution of those two contracts by removing them from the portfolio. The retained contracts preserve their declared operating conditions, including existing sessions and channels. For example, an existing-channel operation concerns authority over an already established resource, whereas a fresh connection would require its own authorization analysis. The broader finding is that a filesystem read-only boundary can coexist with authority over other runtime state. The condition-specific breakdown makes that finding precise, and the exclusion calculation shows that the two root-qualified entries have a small effect on the reported aggregate. These analyses hold the existing coalition assignments fixed and examine how aggregation and contract selection affect the reported measurements.
Reviewer_6GZmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces AuthorityLens, a framework for measuring the authority structure of LLM-based agent systems based on the minimal combinations of participants required to realize each outcome. Using three measures, namely System Authority, Authority Separation, and Principal Authority, the authors evaluate 13 configurations of Codex, OpenCode, and Gemini CLI on Agent-599, a portfolio of 599 authority-relevant outcomes covering files, execution environments, host and network operations, identity and access, software delivery, data systems, and applications. The results reveal that the nominal permission settings of these systems do not necessarily match the authority they realize in practice.

Critical strengths
  • The paper is well organized and its concepts are carefully formulated. The running banking example in Figure 1 makes these definitions intuitive.

  • The motivation is straightforward and practical. As LLM agents are increasingly granted authority over consequential resources, how authority is controlled and distributed is becoming an urgent and realistic problem for real-world applications.

  • Substantial empirical studies make the findings well grounded. The authors build Agent-599 and audit 13 configurations of three widely used agent systems against pinned source revisions, with operation witnesses for every outcome.

Critical weaknesses
  • The framework's practical scalability is unclear. It requires a detailed structural audit of each agent system, so onboarding a new system means conducting a complex audit from scratch. Since agent systems and their underlying models are updated rapidly, AuthorityLens can only provide a snapshot of a specific pinned version of each system.

  • AuthorityLens provides a framework for analyzing deployed agent systems, but it lacks quantitative guidelines for improving agent system design. Although Section 5 provides qualitative design insights, the paper does not demonstrate a framework-guided redesign.

  • All analyses are based on the authors' own Agent-599 portfolio. Does it cover a sufficient range of real-world scenarios? Are the results sensitive to the choice of outcomes in Agent-599? The manuscript lacks a sensitivity analysis, which makes the conclusions less convincing.

Questions for authors
  • Is the framework automated? If not, how much manual effort is required to evaluate one configuration or onboard a new agent system?

  • Given the strong reliance on shell-based realization routes in the evaluation, how sensitive are the reported authority profiles to the composition and weighting of Agent-599? Would the key comparisons remain stable under category-balanced weighting or alternative portfolios representing API-based or GUI-based agent workflows?

  • Can the authors provide a concrete example in which AuthorityLens guides a system redesign and yields measurable improvements?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6GZm

We would like to discuss your scalability concern, particularly whether the audit cost in our study should be taken as the cost of applying AuthorityLens to agent systems more generally.

General-purpose coding agents present a demanding case. A programmable execution interface can reach many resources, and the same outcome may be realized through native tools, shell commands, existing sessions, or delegation. Much of the difficulty is establishing those effective routes and boundaries.

A larger system can have a different structure. Consider agents supporting a laboratory: one analyzes data, another operates instruments, and another authorizes changes to an experimental procedure. If their tools and resource boundaries are explicitly constrained, the permitted routes to a particular outcome can be more clearly specified. Adding participants does not necessarily introduce the same uncertainty as adding general-purpose execution capabilities.

Our view is that audit difficulty depends substantially on the clarity of the authorization boundaries and the available realization routes, not just on agent count. Specialization creates an opportunity for more tractable analysis; we are not claiming that dedicated tool names alone guarantee it. A single tool that permits arbitrary execution can still expose broad authority.

This is also where we see the framework’s design value. If a designer wants a sensitive operation to require another Principal’s authorization, the question is whether every successful route enforces that requirement. Adding a reviewer to the usual workflow may leave an alternative executor’s authority unchanged. AuthorityLens provides a way to express that requirement and inspect whether the configuration realizes it.

Would a clearer distinction between auditing general-purpose execution environments and evaluating systems with explicitly constrained authorization interfaces help address your concern about practical applicability?

AuthorsOpenReview ↗

Follow-up to Reviewer 6GZm: Redesign, Automation, and Portfolio Sensitivity

Following our discussion of systems with explicitly constrained authorization interfaces, we would like to provide a concrete redesign example, clarify the automation workflow, and report additional portfolio-sensitivity results.

A fixed-portfolio redesign. We constructed an executable finite workflow with five protected outcomes: reading a batch record, recording an inspection, applying a standard revision, applying a high-risk revision, and releasing a batch. The same outcome contracts are evaluated across five configurations:

ConfigurationMean minimum participationOutcome coverageCritical outcomes admitting unilateral execution
Centralized executor1.0100%3
Multiple agents sharing the grants1.0100%3
Specialized roles with a maintenance bypass1.0100%3
Required authorization enforced across routes1.6100%0
Additional authorization for two selected outcomes2.0100%0

The design requirement is to preserve outcome reachability while making the designated authorization necessary for the critical operations. In the specialized configuration, the normal route requires approval, but a maintenance route supplies an executor-only coalition. Removing that bypass makes the approval requirement effective across the modeled routes. The final configuration additionally requires a third participant for the high-risk revision and batch release.

The coalition witnesses connect the measurement to a specific intervention: identify the route violating the authorization requirement, modify it, and reassess the same portfolio. The improvement is the elimination of unilateral realizations for the selected critical outcomes while preserving all five outcomes. Low-impact outcomes remain unilateral.

This constructed synthetic example uses explicit interfaces, fixed initial conditions, and a finite transition system, enabling exhaustive reachability analysis for every coalition. It illustrates how task-level authorization requirements can serve as acceptance criteria and regression checks during development.

Automation and maintenance. Our workflow separates source inspection, contract construction, and route-coverage reasoning from scripted execution checks, coalition aggregation, and metric calculation. In the finite example, route exploration is also automated because the transition system is explicit. Onboarding effort therefore centers on establishing the system’s authorization interfaces and admissible routes; the resulting contracts and evidence provide reusable inputs for subsequent assessments.

AI-assisted software engineering creates further opportunities to automate these stages. Big Sleep demonstrates agent-assisted source analysis and vulnerability discovery, while AI-powered OSS-Fuzz automates parts of test generation, build repair, execution, and failure analysis. These developments motivate extending automation to locating authorization paths, constructing test harnesses, and rerunning outcome checks after changes. A versioned portfolio supplies stable contracts for this process, with conclusions grounded in execution evidence and explicit coverage arguments. Big Sleep; AI-powered OSS-Fuzz.

Portfolio sensitivity. We also recomputed the original 599-by-13 matrix, first reproducing all original uniform-weight profiles and Principal Authority values exactly. Under equal weighting of the 46 categories and, separately, the seven themes, Codex Ask/Auto’s mean minimum participation changes from 1.467 to 1.540 and 1.546, while Full Access remains at 1.000. Their outcome coverage remains 0.9967, 0.9957, and 0.9950, respectively, versus 1.0000 for Full Access. Leaving out each category gives an Ask/Auto participation range of 1.447–1.493.

OpenCode Build and Plan agree pointwise in reachability and minimum participation, so equality of these measures holds under any common nonnegative weighting with positive reachable mass. These calculations establish the stability of those comparisons under reweighting; the finite workflow separately illustrates applying a deployment-specific portfolio to dedicated interfaces.

Together, these additions make the intended engineering use concrete: specify authorization requirements, inspect their realization across execution routes, revise the configuration, and reassess the same contracts as the system evolves.

Reviewer_qvoQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AuthorityLens, a framework for measuring the authority structure of LLM-based agent systems through minimal sufficient coalitions of participants. It defines three metrics: System Authority, Authority Separation, and Principal Authority, capturing which outcomes a system can realize, how many participants are required, and how authority is distributed among them.

Critical strengths
  1. This paper proposes to measure authority through achievable outcomes and necessary participants, rather than nominal permissions, which intuitively makes sense.

  2. The evaluation covers multiple popular agent systems and a broad set of operations. Some findings are interesting to know, such as how common agents like Codex's authority behavior change under different permission modes.

Critical weaknesses
  1. The paper primarily formalizes existing access-control concepts and applies them to current agent systems. While the resulting observations are useful, it is unclear what new understanding of LLM-based agents the framework provides beyond conventional software security analysis. In particular, the evaluation does not examine model behavior or the effectiveness of LLM-based authorization decisions.

  2. The proposed metrics characterize structural authority but do not account for the likelihood of exercising authority, the independence or reliability of reviewers, or the severity of different outcomes. As a result, it is unclear how these measurements can guide the design of safer LLM agents or whether higher Authority Separation actually corresponds to meaningful security improvements.

Questions for authors

What insights does AuthorityLens provide that would be difficult to obtain through conventional access-control analysis? Could the authors clarify the unique contributions of this framework to understanding LLM-based agents?

2 comments
AuthorsOpenReview ↗

Response to Reviewer qvoQ

We would like to explain the broader design question behind AuthorityLens, because we do not see the contribution as simply assigning new names to permission settings.

Consider an agent system supporting a biology laboratory or an industrial process. One agent may be able to change procedures, operate equipment, and access sensitive records. A designer has concrete choices to make: narrow its resource access, separate its responsibilities, or require authorization from another Principal before sensitive operations proceed. Making those choices requires understanding what authority the agent actually holds and whether the proposed controls change it.

These questions arise even without an attacker. An agent can misunderstand a request or act on incorrect information. Before deployment, we want to know what its decision could affect and whose authorization is required. After an incident, we need to identify which route allowed the outcome and why the expected boundary or approval step did not prevent it.

AuthorityLens provides an outcome-level language and measurements for examining that structure. Our study shows why capability, agent count, and authority distribution should be distinguished: configurations can realize almost the same outcomes while allocating unilateral authority differently, and additional agents can replicate authority without adding a necessary check.

This information can also inform where to concentrate auditing, isolation, maintenance, or more reliable models, when considered alongside the importance of the affected resources. We do not treat the authority score as a complete risk score or assume that another reviewer necessarily improves judgment.

The portfolio makes the analysis relevant to a deployment. A laboratory and a payment system can specify different protected outcomes, then compare candidate configurations against the same declared specification. Agent-599 is our study’s instance of that approach.

We build on access control and separation of duty. The contribution is a common representation and measurement framework for the authority realized across alternative execution routes and changing Principal populations. It complements behavioral evaluation by making another design question inspectable: what can each participant affect, and whose authorization must actually be obtained before that happens?

AuthorsOpenReview ↗

Follow-up to Reviewer qvoQ: Contribution and Development Lifecycle

Our first response described the deployment motivation. We would now like to make the contribution more concrete through the distinctions exposed by our audits and the design decisions they support.

From implementation findings to an outcome-level authority structure. Task-level separation of duty specifies requirements on the participants who collectively complete a sensitive task. AuthorityLens builds on this foundation by recovering the minimal sufficient Principal combinations realized by a concrete agent configuration across admitted execution routes and runtime states. The contribution is the common representation and its use in comparing outcome coverage, required participation, and the authority carried by individual Principals. Conventional source and permission analysis supplies evidence for this representation. Li and Wang, CCS 2006

Two findings illustrate the additional understanding. First, a command-mediation bypass has different authority consequences in Codex and OpenCode. Under the declared external-deletion conditions, Codex’s remaining resource boundary preserves the requirement for additional authorization. In OpenCode, an admitted alternative shell route leaves the executor sufficient on its own. The outcome-level analysis distinguishes a bypass that changes request admission from one that removes a necessary authorizer.

Second, in the audited OpenCode Build configuration, General, Explore, Build-child, and Plan-child each have a singleton realization for all 537 reachable Agent-599 outcomes, matching the Build root. Even a code-exploration role therefore supplies an alternative holder of the same authority. The resulting structure is replication: additional agents hold authority, while the minimum participation remains one. This separates the number and nominal specialization of agents from the joint authorization actually enforced.

These are system-level properties of LLM agents operating through programmable tools, review mechanisms, and delegated executors. The framework makes the relationship between these mechanisms comparable across configurations. Its finite-representation results further explain how the reported measures can be represented for changing Principal populations under the stated assumptions.

How this guides development and maintenance. These findings motivate treating authority requirements as part of the software development lifecycle. Before implementation, designers can define a portfolio of protected outcomes and specify the authorization conditions each must satisfy—for example, that a release remain achievable while requiring a designated quality authorizer. During development, AuthorityLens assesses whether the composed tools, executors, and delegation routes realize those requirements. A violating coalition identifies a concrete route whose grants or enforcement need attention.

Our constructed finite-workflow example illustrates this process. With the same five outcome contracts, specialized roles retaining a maintenance bypass have mean minimum participation 1.0. Removing that bypass raises it to 1.6, preserves all five outcomes, and makes joint participation necessary for the three critical operations. The intervention is guided by the violated authorization requirement and its witness route.

As tools, policies, and delegation mechanisms evolve, reassessing the same contracts can support security regression testing. This follows established secure-development practices of specifying security requirements, maintaining design decisions, and testing implementations against those requirements. NIST SP 800-218, PW.1 and PW.8.2

Explicit authorization responsibilities also provide a concrete place to maintain review policies, decision rules, and targeted behavioral tests. If an executor-only route remains, improving the reviewer’s judgment leaves that route’s authorization structure unchanged. Establishing the intended enforcement point makes subsequent improvements to reviewer quality relevant to that route.

AuthorityLens therefore connects implementation-level security analysis to authorization requirements that designers can specify, assess, and revisit throughout development. Its structural measurements complement behavioral evaluation by identifying which participants’ decisions actually govern each protected outcome.

Reviewer_Sb2mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper measures authority through the minimal combinations of participants sufficient to realize specified outcomes. It formulates system authority, authority separation, and principal authority, with finite-representation results for changing participant populations. The authors audit 13 configurations of Codex, OpenCode, and Gemini CLI over 599 outcome contracts (covering approval mechanisms, sandbox boundaries, and delegation).

Critical strengths
  • this paper aims to study timely, important problem.
  • I like the paper's outcome-level perspective and find it useful, as restricting one interface does not necessarily restrict authority when alternative execution or delegation routes remain. Distinguishing authority replication from required joint participation also provides a clear way to reason about multi-agent designs.
  • The concrete audits uncover meaningful differences between nominal vs realized. Particularly informative are the distinction between command mediation and resource boundaries, OpenCode’s delegated execution routes, and Guardian’s authority beyond filesystem reads.
Critical weaknesses
  • gap between the claimed positioning/contribution vs actual demonstration: Several headline findings follow naturally from configured approval and permission-inheritance rules. For example, Gemini Model Review increases participation without reducing outcome coverage, but the evaluation supplies approving review decisions, making this effect largely expected. The paper should distinguish these expected consequences from non-obvious audit discoveries and clarify what the framework adds beyond a conventional permissions audit.

  • The meaning of authority separation would need better justification. while LLM reviewer is considered as a separate Principal, why deterministic enforcement components (basically doing the same role as LLM reviewers) are not? It remains unclear when this distinction captures meaningful authority distribution rather than implementation choices or additional review stages.

  • The reported minimum coalitions are difficult to verify. Successful executions establish sufficient coalitions, but not the absence of smaller alternatives. The procedure for excluding alternative routes, including reviewer-failure paths, is insufficiently specified.

  • Quantitative conclusions depend on an insufficiently justified portfolio. The selection and granularity of the 599 outcomes are not clearly motivated. Without category-level or weighting sensitivity analyses, it is difficult to assess how representative the aggregate scores are and how they were constructed.

Questions for authors
  • Would replacing an LLM reviewer with a deterministic guard making identical authorization decisions change authority separation? Please what authority distinction would capture this?
  • How are Conseca’s error-to-ALLOW paths represented in the coalition analysis, and what is their impact on the claimed necessity of reviewer participation?
  • For one outcome classified as requiring a reviewer, could you provide a representative audit record showing the routes examined and the constraints excluding reviewer-free alternatives?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Sb2m

We would like to start with your question about deterministic guards, because it gets at what we are counting as a Principal.

Section 2.1 defines a Principal as “an entity to which authorizations are granted and to which their exercise is attributed.” We distinguish authority holders from the mechanisms that enforce authorization rules. This is consistent with the distinction between principals and the reference monitor in Lampson et al. (1992, Section 1).

Deterministic guards are already reflected in our analysis: they constrain which outcomes are reachable and which coalitions are sufficient. A rule may block an operation or route it to another authorizer, but executing that rule does not automatically introduce an additional authority holder. Otherwise, splitting one policy engine into several checking stages could increase measured separation without changing who holds authority.

For your replacement example, the relevant question is whether the replacement preserves the separately attributable authorization role. If it does, determinism alone is not a reason to exclude it. If the reviewer is removed and its behavior incorporated into the configuration’s fixed rules, the participants can change even when outcome coverage does not. This is a distinction in authority attribution, not a claim that the deterministic implementation is less secure.

Conseca illustrates why this boundary matters. Policy-generation failure can produce an empty policy while still leaving the enforcement model to make a decision. Certain enforcement failures instead produce ALLOW through Conseca’s configured fallback; an outer timeout produces DENY. Our intended unit is the complete Conseca authorization principal, including its fallback rules. A fallback authorization can therefore be attributed to Conseca without being described as a successful model-based assessment.

We agree that simply invoking a component would not establish necessary participation. What matters is whether completing the outcome requires an authorization result attributable to that Principal. Does this distinction between an authority holder and its enforcement machinery address your concern about the counting boundary?

AuthorsOpenReview ↗

Follow-up to Reviewer Sb2m: Representative Audit Record

Following our clarification of Principal attribution, we would also like to address your request for a representative audit record showing why reviewer participation is necessary. We use the external file-set deletion outcome in Codex Auto (SE-C05-RM-FORCE-SET-OUTSIDE), which illustrates how execution evidence and authorization constraints are connected.

Outcome and operating conditions. The task is to remove three specified files outside every initially writable root, including the workspace and /tmp, while preserving two designated control files. The executor has UID/GID 1000 and no additional capabilities or inherited privileged handles. The targets have ordinary ownership and permissions, with no symlinks or target submounts.

Runtime evidence. We executed the target operations in isolated Docker containers using controlled filesystem fixtures. Each run recorded the actual execution boundary, command exit status, and before/after filesystem state. Completion was verified against the removal of all designated targets and preservation of the control files’ contents and identity.

We tested shell continuation and variable expansion, both of which pass the command-entry check while retaining the configured execution boundary. Each route gives the following paired results:

Execution boundaryWorkspace-local targetsExternal targets
Workspace-writeAll three deletedAll three preserved; EROFS
Guardian read-onlyAll three preserved; EROFSAll three preserved; EROFS
Sandbox disabledAll three deletedAll three deleted

The controls remain preserved throughout. This paired case contains 12 boundary/route executions and two direct operation controls. Its 26 policy queries establish that the alternative command entries receive Skip with bypass_sandbox=false, whereas requesting unsandboxed execution in Auto receives NeedsApproval.

Connecting execution to authorization. The successful external-deletion operation supplies the completion witness. The pinned-source orchestration supplies the authorization link: a request requiring approval passes through request_approval before execution proceeds with the selected sandbox override. In Auto, that authorization is assigned to Guardian. The evidence for the reviewed realization therefore combines an operation witness with the source-checked approval path.

Establishing the minimum coalition. The exclusion argument rests on the shared resource constraint:

  • Shell continuation and alternative command expressions retain the read-only mount covering the external target directory.
  • Native patching has a separate writable-path check that also routes changes outside the permitted boundary through approval.
  • Eligible spawned Assistants inherit the active permission profile and selected reviewer.
  • Guardian’s own execution uses read-only permissions with approval policy Never.
  • The contract supplies no pre-existing privileged channel to the external targets.

Together, these constraints exclude a successful unilateral realization within the declared setting, while the approved execution path supplies a two-Principal realization. This supports the reported minimum mC(x)=2m_C(x)=2, with an executing Assistant and Guardian.

The corresponding workspace-local control shows why tracing alternative routes matters: there, command-entry bypass leaves a writable target, and the Assistant can complete the outcome alone. The same request-mediation observation therefore has different coalition consequences depending on the resource boundary. AuthorityLens records that distinction at the level of the protected outcome.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.