acceptodds
Under review as a conference paper at ICLR 2027

Multi-Principal Authority Resolution in Large Language Models

Abstract

AI agents increasingly act on behalf of multiple users whose authority rules depend on context. However, agents can fail to identify whose authority governs an action and execute instructions outside their authorized scope. We study multi-principal authority (MPA) resolution through a logic-based analysis grounded in individual rationality. We establish sufficient conditions under which the multi-principal problem reduces to a unique judgment from the governing principal. These conditions require applicable authority to exist and the governing principal to be uniquely selected with a consistent judgment. We derive MPAGuard from this analysis to structure LLM generation around identifying applicable principals and resolving whose judgment governs. We develop the MPA Dataset to systematically analyze how resolution errors change with the number of principals and contexts under conflicting authority rules. Across evaluated LLM families, separating context applicability from authority polarity improves authority-resolution accuracy. Combining this separation with priority-ordered evaluation achieves a Pareto-optimal trade-off between accuracy and token usage. Integration with AgentDojo further shows that structured resolution preserves authorized actions while reducing attack success. These findings highlight structured authority reasoning as a basis for reliable action control in multi-user environments.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, LLM Agents, Multi-Principal Authority Resolution, Agent Safety, Authorization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_d4F2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formalizes context-dependent authority resolution by selecting the highest-priority applicable principal and adopting that principal's judgment. MPAGuard compares direct selection with sequential or exhaustive principal evaluation, and joint versus separate applicability/polarity outputs. A synthetic benchmark shows benefits from explicit decomposition. On AgentDojo, PU-Two reduces average attack success from 0.31 to 0.10 while keeping average utility under attack near 0.42. The contribution is mainly an empirical study of structured authority prompting and its use as an action filter.

Critical strengths
  1. The separation of inapplicability, prohibition, and conflict is clear and useful. Proposition 1 is correct under its stated assumptions, and the paper appropriately distinguishes a unique formal decision from an LLM's ability to compute it. Appendix B also explains how selection errors accumulate without claiming that its simplified model proves PU superior to UP.

  2. The experiments compare several model families and expose specific errors rather than only reporting accuracy. The OneStep/TwoStep comparisons, all-matching control, and token/cost results show when explicit applicability helps. The paper also reports cases where CoT is competitive or better, which makes the practical comparison more balanced.

Critical weaknesses
  1. The practical action-control evidence is incomplete. AgentDojo uses one attack family and compares mainly variants of the proposed guard. Appendix E.3 says the acting agent is instructed to call the guard and selects the authority sources itself, but does not establish whether every side-effecting call is forcibly gated or how source identity and priority are protected. Moreover, PU-Two reduces benign utility from 59.8% to 36.1% for Llama3.3-70B (Table 12). Thus, preserved average utility under attack is not preserved utility generally. A trace audit of guard coverage and a matched established defense would make this claim much more convincing.

  2. The benchmark establishes a narrow prompting result, leaving the added value of learned authority resolution unclear. Each principal has one binary rule, contexts match by exact attribute equality, and the natural-language variant changes only the query surface. Local conflicts from the formal framework are therefore not tested. Proposition 1 is a straightforward consequence of the selection rule. A deterministic resolver, or extraction followed by deterministic resolution where language interpretation is needed, is an important reference point.

Questions for authors
  1. Is guard invocation enforced outside the acting model? Can you report the fraction of side-effecting calls checked, the treatment of missing or malformed guard calls, and how source attribution and priority are obtained? How does the guard compare with an established defense on the same attacked and benign tasks?

  2. Which parts of the evaluated authority task require learned interpretation rather than deterministic rule matching? A deterministic or extraction-plus-resolution baseline would clarify the practical role of MPAGuard and the source of its remaining errors.

3 comments

Thank you for taking the time to review our paper and for highlighting the need to clarify our evaluation and implementation.

Response to Questions

Q1. Guard enforcement, coverage, and authority-source inputs

Is guard invocation enforced outside the acting model? Can you report the fraction of side-effecting calls checked, the treatment of missing or malformed guard calls, and how source attribution and priority are obtained? How does the guard compare with an established defense on the same attacked and benign tasks?

Yes. Guard invocation is enforced by the executor rather than left to the acting model. Our wording “instruct the agent to invoke” was imprecise: MPAGuard is exposed through a tool-compatible interface, but the executor automatically intercepts non-read-only tool calls and requires a permit decision before execution.

The audit logs for Multi-turn + MPAGuard show the following action-level counts, including repeated attempts:

ModelConditionObserved target attemptsInvalid arguments blockedGuard checksPermittedDenied
Qwen3-32BBenign74173694
Qwen3-32BImportant Instructions6145609325284
Gemma3-27BBenign53053503
Gemma3-27BImportant Instructions4670467308159

All target calls with valid arguments that reached the executor were checked: 73/73 and 609/609 for Qwen, and 53/53 and 467/467 for Gemma. The remaining Qwen attempts were blocked before guard invocation by tool-schema validation—for example, passing {"content": []} when content must be a string, or omitting a required date argument. These calls were not executed.


Q2. Learned interpretation versus deterministic resolution

Which parts of the evaluated authority task require learned interpretation rather than deterministic rule matching? A deterministic or extraction-plus-resolution baseline would clarify the practical role of MPAGuard and the source of its remaining errors.

Extraction + Resolution achieves near-perfect accuracy across all three models on structured queries, outperforming the prompting baselines. On natural-language queries, it remains strong for Llama3.3-70B but reaches 89.3% for Gemma3-27B and falls four percentage points below PU-Two for Qwen3-32B. Thus, deterministic resolution is effective when the required information is reliably structured, while contextual interpretation remains a source of error.

Structured tool query

MethodQwen3-32BGemma3-27BLlama3.3-70B
Minimal76.3%69.7%97.4%
FindBest80.4%77.0%99.3%
PU-Two98.1%91.4%98.4%
Extraction + Resolution99.4%100.0%99.4%

Natural-language tool query

MethodQwen3-32BGemma3-27BLlama3.3-70B
Minimal72.9%66.7%97.7%
FindBest81.0%74.9%98.6%
PU-Two95.9%87.7%96.9%
Extraction + Resolution91.9%89.3%99.6%

Authorization is context-dependent: users delegate authority for particular situations, and the agent must determine whether a proposed action falls within that scope. Keywords or categorical representations must therefore preserve the relevant contextual boundaries.

For example, a user may authorize rm -rf * within a specific directory. Matching the command alone is insufficient; authorization also depends on the working directory and the scope of the delegation. If these conditions are precisely specified and reliably available to the execution system, deterministic checks can be accurate and efficient. If an LLM must first interpret the instruction and determine whether the current situation satisfies those conditions, however, that interpretation burden remains. Generating a structured representation adds cost and may introduce errors, so an end-to-end efficiency advantage is not guaranteed.

MPAGuard structures these contextual judgments by explicitly distinguishing whether delegated authority applies from whether it permits or prohibits the action. Its role is complementary to deterministic resolution, which can compute the final decision once those judgments are reliably established.

AuthorsOpenReview ↗

Response to Weaknesses: Action Control and Benchmark Scope

W1. Action-control evidence, defense comparisons, and utility

We have expanded AgentDojo evaluation to Important Instructions, Ignore Previous, and Tool Knowledge, with comparisons against Tool Filter, Spotlighting, and Repeat User Prompt. Please see our General Rebuttal Comment for the experimental settings, baseline comparisons, and detailed results.

MPAGuard consistently achieves lower ASR than Repeat User Prompt and Spotlighting, and generally lower ASR than Tool Filter. Utility under attack is slightly lower than Repeat User Prompt for Qwen but higher than all three defenses for Gemma. Compared with Tool Filter, benign utility is slightly lower for Qwen but substantially higher for Gemma.

We also clarify that authorization checks are enforced by the executor before non-read-only actions. In the audited conditions, every valid target call reaching the executor was checked; invalid tool arguments were blocked before execution. The runtime distinguishes original user requests from tool responses and fixes the original user’s priority over tool-provided instructions. Please see our response to Q1 for the audit counts and implementation details.

We agree that preserving utility under attack does not establish general utility preservation. The reported Llama3.3-70B benign-utility decrease is a genuine trade-off, and we will revise the manuscript to distinguish benign utility from utility under attack and qualify our claims accordingly.


W2. Benchmark scope and the role of learned interpretation

Our paper focuses on (1) formalizing the structure of authorization from explicitly specified multiple principals and (2) analyzing LLM authorization tendencies under these controlled conditions. The MPA Dataset provides verifiable labels to isolate errors in applicability, polarity, and governing-principal selection. Proposition 1 characterizes the conditions for a unique authority decision; it does not imply that computing that decision requires a learned resolver.

We agree that the structured setting admits deterministic resolution and therefore add an LLM-extraction-plus-deterministic-resolution baseline. It achieves near-perfect accuracy on structured queries, while its advantages are less consistent on natural-language queries, where contextual interpretation remains a source of error. Please see our response to Q2 for the detailed comparison and discussion.

We additionally evaluate MUSES-IF Selection and OrgAccess, where MPAGuard achieves the highest overall performance among the evaluated methods for both models. These results show that the approach also benefits authorization decisions beyond the controlled MPA setting. Please see our General Rebuttal Comment for the experimental settings, baseline comparisons, and detailed results.

We consider contextual ambiguity and within-principal conflicts important problems, but their systematic treatment lies beyond this paper’s scope. Overlapping rules require explicit conflict-resolution criteria, while ambiguous contexts require determining how uncertainty affects applicability and authorization. Please see our response to Reviewer Xb7a, under “Scope 1: Overlapping rules within a single principal” and “Scope 2: Ambiguous authority scopes,” for further discussion. Establishing clear LLM authorization decisions under such conditions is an important direction for future research.

We will revise the manuscript to explicitly state these limitations and opportunities for extending the framework.

AuthorsOpenReview ↗

Gentle Follow-Up Before the Discussion Deadline

Dear Reviewer d4F2,

We apologize for reaching out again as the discussion period draws to a close, and sincerely appreciate your time and thoughtful review.

We have updated the revised manuscript with additional experiments and clarifications addressing your concerns.
We would be grateful if you could review these updates and share any remaining feedback.

Thank you again for your time and consideration.

Reviewer_Xb7aOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how LLM agents resolve authority across multiple principals with context-dependent and potentially conflicting rules. It establishes conditions for a unique authority decision and proposes MPAGuard, which separates context applicability from authority polarity and resolves competing judgments through principal priority. The authors also construct the MPA dataset to evaluate different resolution procedures across model families and analyze their errors and efficiency. They also integrate MPAGuard into AgentDojo to evaluate its effects on task utility and resistance to prompt injection.

Critical strengths
  • The paper clearly formalizes how an LLM agent should determine whether an action is authorized when multiple principals provide context-dependent and potentially conflicting rules. This provides a clear basis for identifying applicable authority and resolving competing judgments through principal priority.

  • The MPA dataset is a useful contribution for studying multi-principal authority resolution. The evaluation is fairly comprehensive.

Critical weaknesses

It seems the AgentDojo evaluation seems to focus on a single attack type and mainly compares the proposed prompting variants. Are there additional results comparing MPAGuard with existing guard or prompt-injection defense baselines? Such comparisons would help clarify its advantages over existing approaches.

Questions for authors

May I know whether the framework can handle overlapping rules within a single principal or ambiguous authority scopes?

2 comments

Thank you for taking the time to review our paper and for your constructive feedback and thoughtful questions.


Weakness: Broader attacks and existing defense baselines

We have expanded the AgentDojo evaluation to three attack types—Important Instructions, Ignore Previous, and Tool Knowledge—and compared against Repeat User Prompt, Spotlighting, and Tool Filter, using Qwen3-32B and Gemma3-27B. Please see our General Rebuttal Comment for the experimental settings and full results. The main findings are:

  • Attack success rate: MPAGuard consistently achieves lower ASR than Repeat User Prompt and Spotlighting across both models and all three attack types, and generally achieves lower ASR than Tool Filter.
  • Utility: For Qwen, utility under attack is slightly lower than Repeat User Prompt; for Gemma, it is higher than all three existing defenses. Compared with Tool Filter, benign utility is slightly lower for Qwen but substantially higher for Gemma.

We additionally evaluate MUSES-IF Selection and OrgAccess, where MPAGuard achieves the highest overall performance among the evaluated methods for both models.

These findings suggest that identifying authority sources and explicitly distinguishing context applicability from authorization polarity can help prevent attacks while supporting authorized task completion. The benefits extend beyond rejecting unauthorized actions to preserving instructions that should be followed.


About Questions For Authors:

Q: May I know whether the framework can handle overlapping rules within a single principal or ambiguous authority scopes?

Scope 1: Overlapping rules within a single principal

Our theoretical development assumes that each user provides an explicit authority rule with a clearly specified context, without within-principal context overlap. The reviewer’s question extends this setting to cases where multiple contexts within the same principal apply simultaneously.

Such overlap requires a criterion for resolving potentially conflicting judgments. For example, “work on weekdays” and “rest on public holidays” both apply to a weekday that is also a public holiday. Deciding whether to work requires background knowledge or an explicit convention establishing which rule takes precedence. Context matching alone does not determine the answer.

A promising extension is to organize authority hierarchically by source and context, grouping related contexts and specifying precedence between general rules and more specific exceptions. This would allow our framework to represent overlapping authority through an explicit source–context hierarchy. It also connects authority resolution to default–exception reasoning, which we consider a realistic and important direction beyond the current controlled setting.

Separately, if the context itself is ambiguous, rather than clearly specified but overlapping, applicability—and potentially polarity—could be estimated probabilistically. This would require extending the current formulation with explicit uncertainty representations and decision criteria.

Scope 2: Ambiguous authority scopes

Our paper considers authority primarily as delegation from humans to an LLM agent: a delegator authorizes an agent to perform actions within a specified scope. We therefore assume identifiable authority providers and an explicit delegation relation. The agent evaluates whether its action is justified by that delegated authority; this evaluation does not itself create authorization. If the agent forgets who granted a permission, this concerns its belief or memory about authority rather than ambiguity in the delegation relation itself.

When multiple users delegate authority to the same agent, their scopes may overlap. If their contexts differ, the agent must distinguish those contexts when assessing applicability. If conflicting judgments apply in the same context, a priority or another explicit conflict-resolution criterion is necessary for a unique decision; otherwise, the conflict remains unresolved.

When authority is jointly delegated by multiple users, those users remain identifiable as its providers; their respective rights to revise or revoke it require separate specification. Thus, overlapping or joint delegation does not itself imply ambiguity about who grants authority. The remaining questions concern where that authority applies and how conflicting judgments are resolved.

Thank you for all the additional details which is helpful. I maintain my positive score.

Reviewer_Wd1ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MPAGuard, a structured prompting framework for determining whether an LLM agent's actions are authorized when multiple principals have potentially conflicting rules. The framework includes several variants for identifying applicable principals and resolving authorization decisions based on their priorities. The authors construct the MPA Dataset, a synthetic benchmark of 700 examples involving 1–7 principals, and evaluate eight open-source models. Results show that structured prompting generally outperforms direct prediction, especially for smaller models. TwoStep variants also reduce authorization errors and perform better as the number of principals increases. The authors further evaluate MPAGuard in AgentDojo against prompt injection attacks. PU-Two reduces the attack success rate from 0.31 to 0.10 while maintaining task utility at 0.42. In contrast, FindBest+CoT achieves a lower attack success rate but substantially reduces utility.

Critical strengths

This paper has the following major strengths:

  1. The paper is well organized and the research question is cleanly motivated and consistently carried through the analysis. This paper clearly presents the research problem and provides a well-structured framework for resolving authorization conflicts among multiple principals. In particular, the distinction between rule applicability and authorization decisions is clearly explained and consistently reflected in the method design and experimental analysis. This makes the proposed approach easy to follow and the results easy to interpret.

  2. Insightful experimental findings with practical implications. The experiments provide several useful findings. For example, separating applicability from authorization decisions reduces prediction errors and improves robustness as the number of principals increases. The results also show that priority-based early stopping can achieve comparable accuracy with lower token costs. Furthermore, the AgentDojo experiments demonstrate that MPAGuard can reduce prompt injection attack success rates while preserving task utility. These findings provide practical guidance for designing authorization mechanisms for LLM agents without additional training.

Critical weaknesses

This paper has the following major weaknesses:

  1. Limited benchmark complexity and lack of appropriate baselines. The MPA Dataset is highly simplified compared to the real-world authorization problems motivating the paper. Each principal specifies a single attribute-value context and an authorization decision, with an explicit priority order. The task can therefore be solved through deterministic rule matching without LLM reasoning. However, the paper lacks a baseline combining LLM-based rule extraction with deterministic resolution. Moreover, the benchmark only evaluates structured contexts rather than the complex natural-language rules discussed in the introduction, limiting the generalizability of the findings. Larger models also achieve near-perfect accuracy under minimal prompting, suggesting that the benefits may be concentrated in smaller models.

  2. Insufficient evaluation of effectiveness against prompt injection attacks. The AgentDojo experiments consider only one attack type and do not compare MPAGuard against existing defenses such as tool_filter, spotlighting, or repeat_user_prompt. Additionally, the setting primarily tests whether user instructions take priority over tool outputs, rather than complex multi-principal authorization. The paper also lacks discussion of the threat model, particularly the attacked agent's control over guard invocation and inputs. Stronger baselines and broader evaluations are needed to support the paper's claims.

  3. The paper contains several minor writing and presentation issues. For example, the model citations in Section 5 (lines 266–268) use textual citations where parenthetical citations would be more appropriate (e.g., “Gemma3-4B/12B/27B Gemma-Team et al. (2025)”). Additionally, the dataset is consistently referred to as the “MPA Dataset” throughout the paper but is called the “Authority Dataset” in Section 6.1.

Questions for authors
  1. How would a baseline combining LLM-based rule extraction with deterministic resolution compare in accuracy and token cost? If it achieves near-perfect accuracy at lower cost, what advantages does MPAGuard offer over deterministic resolution, and in what settings would it be preferable?

  2. Is MPAGuard invocation enforced by the evaluation harness or left to the agent? If it depends on the agent, how often does the agent bypass the guard or omit relevant sources under attack, and are these cases included in the reported ASR?

4 comments

Thank you for your time and careful review of our experiments, and for identifying points requiring clarification and corrections in the manuscript.


Response tot Questions For Authors:

Q1. LLM-based extraction with deterministic resolution

How would a baseline combining LLM-based rule extraction with deterministic resolution compare in accuracy and token cost? If it achieves near-perfect accuracy at lower cost, what advantages does MPAGuard offer over deterministic resolution, and in what settings would it be preferable?

We agree that deterministic resolution is appropriate when applicability, polarity, and principal priority are accurately represented in a structured form. We therefore add an extraction-plus-resolution baseline on the MPA Dataset, where an LLM extracts the relevant information and a deterministic procedure computes the final authorization decision.

Structured tool query

MethodQwen3-32BGemma3-27BLlama3.3-70B
Minimal76.3%69.7%97.4%
FindBest80.4%77.0%99.3%
PU-Two98.1%91.4%98.4%
Extraction + Resolution99.4%100.0%99.4%

Natural-language tool query

MethodQwen3-32BGemma3-27BLlama3.3-70B
Minimal72.9%66.7%97.7%
FindBest81.0%74.9%98.6%
PU-Two95.9%87.7%96.9%
Extraction + Resolution91.9%89.3%99.6%

This approach generally improves accuracy on structured inputs. However, its effectiveness depends on the preceding interpretation step. In the natural-language query setting, Qwen3-32B performs approximately four percentage points below PU-Two, while Gemma3-27B reaches approximately 89% accuracy. Deterministic resolution therefore does not eliminate errors in interpreting contexts and extracting authority information.

Regarding token cost, deterministic resolution can be more efficient once the required information is available in a structured form. However, producing that representation through autoregressive generation incurs its own cost and does not guarantee accurate extraction or a correct final decision.

The controlled structure of the MPA Dataset provides verifiable labels and isolates authority-resolution errors. It does not assume that natural-language authorization can always be converted into reliable categorical inputs. In our development experiments on AgentDojo and the additional benchmarks, some attempts to construct structured interfaces for deterministic resolution—including numerical applicability-confidence outputs—degraded performance and were not retained. These observations highlight the difficulty of connecting language interpretation to symbolic resolution, rather than establishing a general limitation of deterministic methods.

We thus view the two approaches as complementary. Deterministic resolution is suitable when reliable structured inputs are available; MPAGuard guides an LLM in identifying applicable authority and determining its polarity when these judgments require contextual interpretation.


Q2. Guard invocation and bypass

Is MPAGuard invocation enforced by the evaluation harness or left to the agent? If it depends on the agent, how often does the agent bypass the guard or omit relevant sources under attack, and are these cases included in the reported ASR?

Thank you for identifying an imprecision in our runtime description. The wording “instruct the agent to invoke” does not clearly distinguish the tool interface from execution enforcement. MPAGuard was exposed as a tool for compatibility with the agent interface, but the pre-execution check is mandatory for non-read-only actions: the execution mechanism requires a permit decision before allowing the corresponding action to proceed. We will revise the wording to make this distinction explicit.

Mandatory invocation is enforced by code rather than left to the acting agent’s discretion. The executor constructs the source catalog and fixes source identities and priorities. This prevents invocation bypass but does not guarantee correct authorization: the guard may overlook or misinterpret relevant evidence and incorrectly permit an unauthorized action or prohibit an authorized one. These are decision errors rather than invocation bypasses and can affect both ASR and task utility.

We will revise the manuscript to explicitly describe this code-enforced execution flow and distinguish mandatory invocation from the completeness of supplied authority sources and the correctness of guard decisions.

AuthorsOpenReview ↗

Response to Weaknesses: Benchmark Design and Evaluation

W1. Benchmark complexity and appropriate baselines

Our paper aims to theoretically characterize the conditions for a clear authority decision and empirically evaluate LLM permission/prohibition behavior by distinguishing context applicability from authority polarity. The controlled MPA Dataset provides verifiable labels for examining these components. We additionally introduce an extraction-plus-deterministic-resolution baseline and evaluate MUSES-IF Selection and OrgAccess, where MPAGuard achieves the highest overall performance among the evaluated methods for both models. Please see our General Rebuttal Comment for the experimental settings and detailed results, and our response to Q1 for the deterministic baseline discussion.

We agree that realistic authorization requires richer policy specifications. However, expanding rule complexity also requires specifying the intended authority semantics; complexity alone does not provide a clearer evaluation. For example, OrgAccess distinguishes accepted, rejected, and partial, but partial applies specifically when all other requirements are satisfied and collaboration or location permissions are partially satisfied. These distinctions must be communicated through the task instructions rather than inferred from the label names alone.

Authority encompasses many such policy-dependent conventions. Our objective is to establish a clear theoretical basis, examine fundamental authorization tendencies under controlled conditions, and assess their practical relevance through additional benchmarks. We therefore prioritize explicit, verifiable authority semantics over broader specifications whose resolution criteria remain underspecified. We discuss richer authority structures and their interpretation in our response to Reviewer Xb7a, under “Scope 1: Overlapping rules within a single principal” and “Scope 2: Ambiguous authority scopes.”

W2. Prompt-injection evaluation and threat model

We have expanded the AgentDojo evaluation to three attack types—Important Instructions, Ignore Previous, and Tool Knowledge—and added comparisons with Tool Filter, Spotlighting, and Repeat User Prompt. MPAGuard consistently achieves lower ASR than Repeat User Prompt and Spotlighting, and generally lower ASR than Tool Filter. Utility under attack is slightly lower than Repeat User Prompt for Qwen but higher than all three existing defenses for Gemma. Compared with Tool Filter, benign utility is slightly lower for Qwen but substantially higher for Gemma.

We additionally evaluate MUSES-IF Selection and OrgAccess to assess broader multi-user authorization beyond the user–tool distinction in AgentDojo. Please see our General Rebuttal Comment for the experimental settings and full results.

W3. Writing and presentation

Thank you for pointing out these presentation issues. We will change the model citations in Section 5 to parenthetical citations and replace “Authority Dataset” in Section 6.1 with “MPA Dataset” for consistency throughout the manuscript.


We will incorporate these additional benchmark and attack evaluations into the manuscript to clarify the applicability of MPAGuard and its trade-offs relative to existing defenses.

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns. The explanation of Guard invocation and bypass is helpful. The additional experiments and the discussions on the prompt injection evaluation/threat model largely addresses W2. However, my concern of W1 remains, as I'm still confused about the benefits of MPAGuard over other simpler alternatives. For example, current evidence still show that deterministic resolution baseline achieves strong results in both structured settings and natural language settings. These added results didn't necessarily show that MPAGuard is more competitive in natural language scenarios. My concern is still here: why do we need/prefer MPAGuard over other simple baselines?

To sum up, I acknowledge that the rebuttal and added experiments are valuable and appreciate authors' efforts, but my major concern of W1 still remains. Therefore, I would maintain my original score of 2.

AuthorsOpenReview ↗

Deterministic Resolution Is Part of Our Contribution, Not an Alternative

We thank the reviewer for their time, effort, and thoughtful follow-up. We appreciate their acknowledgment of our additional experiments and clarifications.

We believe there may be a misunderstanding regarding the role of deterministic resolution in our work, and would like to clarify this point.

To directly answer the reviewer's question:

Why do we need/prefer MPAGuard over other simple baselines? Because our paper systematically compares simple procedures implementing the same deterministic authority-resolution semantics and identifies which structures improve LLM accuracy and efficiency. Our results show that explicitly separating applicability from polarity and using priority-first resolution provides a strong accuracy–efficiency trade-off.

We clarify this through three points.

1. Deterministic resolution is already part of our proposed framework.

Our paper formalizes the conditions under which multi-principal authority admits a unique judgment and derives MPAGuard from this formulation.

We investigate two LLM-based procedures following the same deterministic resolution semantics:

  • UP (User-First): Evaluate all principals before priority-based resolution.
  • PU (Priority-First): Evaluate principals in priority order and stop at the first applicable principal.
  • (UP/PU) + OneStep / TwoStep: Compare joint versus separate applicability–polarity judgments, showing their impact on accuracy under identical resolution semantics.

Since these procedures already follow deterministic resolution rules, we interpreted the reviewer's request for a deterministic baseline as asking whether symbolic computation could provide additional benefits. We therefore introduced Extraction + Resolution, which combines LLM-based query fact extraction with symbolic UP-style resolution. After extraction, Python code deterministically computes applicability for all principals and selects the highest-priority applicable principal.

Thus, the additional baseline evaluates the benefits of symbolic computation within the same resolution framework, rather than introducing a fundamentally different authority-resolution principle.

2. Different implementations yield different accuracy, even under the same deterministic semantics.

Extraction + Resolution performs strongly but does not consistently outperform PU-Two. On natural-language queries, it outperforms PU-Two for Gemma3-27B but underperforms it for Qwen3-32B (91.9% vs. 95.9%).

These results show that deterministic computation alone does not guarantee accurate end-to-end resolution. Performance also depends on how contextual information is interpreted and how applicability and polarity judgments are established.

Our contribution is therefore not merely to specify a deterministic decision rule, but to investigate how LLMs can reliably perform the judgments required for authority resolution.

3. The benefits extend beyond the controlled MPA Dataset.

On MUSES-IF Selection and OrgAccess, Multi-turn + MPAGuard achieves the highest overall performance among the evaluated methods for both Qwen3-32B and Gemma3-27B. These results provide additional evidence that structured authority resolution benefits broader multi-user and organizational authorization tasks.


Finally, we would appreciate clarification on what specific "simpler baseline" the reviewer considers preferable to MPAGuard. We have already evaluated priority-first (PU), user-first (UP), direct governing-principal selection (FindBest), and symbolic UP-style resolution. Is the reviewer suggesting a fundamentally different deterministic algorithm? If so, how does it differ from these procedures, and what advantages is it expected to provide?


If the concern instead relates to the simplicity of our framework, we emphasize that authority resolution is not merely a matter of applying a deterministic priority rule. As Simon (1991) discusses, delegated authority operates within bounded scopes and often specifies decision premises rather than concrete actions. In multi-principal settings, determining whether an agent's action is justified therefore requires identifying applicable authority, its permission or prohibition, and the governing principal. Our contribution is to formalize these essential components and investigate how LLMs can reliably resolve them.

We hope this addresses the remaining concern, and appreciate the reviewer's continued engagement throughout the discussion period.

* Herbert A. Simon, 1991, Organizations and Markets

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Rebuttal Comment

We thank all reviewers for their valuable time, effort, and constructive feedback.

In this comment, we present additional experimental results and clarifications addressing concerns shared across the reviews.


Additional Experimental Settings

To address the reviewers’ shared concerns about evaluation breadth and baseline comparisons, we report additional results on MUSES-IF Selection [1], OrgAccess [2], and additional AgentDojo attack types, using Qwen3-32B and Gemma3-27B.

We compare the following settings:

  1. Benchmark baselines and existing methods. We evaluate each benchmark’s official prompts and available benchmark-provided methods or defenses.

  2. Explicit benchmark-specific label criteria. We clarify the criteria for positive, negative, and exception labels. For example, in OrgAccess, partial applies only when all other requirements are satisfied and collaboration or location permissions are partially satisfied; violations of other permission requirements lead to rejected. This clarification addresses overly broad predictions of partial and distinguishes unverified conditions from explicit violations according to the benchmark’s rules.

  3. MPAGuard with the same criteria. We add MPAGuard while retaining the same benchmark-specific label criteria, allowing comparison with setting (2).

Settings (2) and (3) are evaluated in both single-turn form, which directly returns a final decision, and multi-turn form, which produces a tentative assessment followed by verification against the original input.

[1] Yang et al. (2026). Multi-User Large Language Model Agents
[2] Sanyal et al. (2025). OrgAccess: A Benchmark for Role Based Access Control in Organization Scale LLMs

Benchmark 1. Agent Dojo

For AgentDojo, we expand the evaluation with the following attacks and defenses:

  • Attack types: Important Instructions, Ignore Previous, and Tool Knowledge.
  • Baselines and existing defenses: Default agent, Repeat User Prompt, Spotlighting, and Tool Filter.

To address the reviewers’ concerns about guard invocation, our additional AgentDojo evaluation enforces MPAGuard checks in the execution harness: every action subject to authorization is checked before execution, and the acting model cannot bypass this check. We compare this mechanism with Tool Filter, which selects a restricted tool set once at the beginning of each episode, before observing untrusted tool outputs, and maintains that restriction throughout the episode. Tool Filter therefore controls which tools remain available, whereas MPAGuard evaluates each proposed action and its arguments immediately before execution. Both restrictions are enforced by code.

Benign utility

MethodQwen3-32B: Benign utility ↑Gemma-27B-IT: Benign utility ↑
Default52.58%49.48%
Repeat user prompt56.70%52.58%
Spotlighting56.70%51.55%
Tool filter52.58%41.24%
Single-turn50.52%51.55%
Multi-turn51.55%51.55%
Single-turn + MPAGuard (ours)50.52%52.58%
Multi-turn + MPAGuard (ours)51.55%50.52%

Attack 1: Important Instructions

MethodQwen3-32B: Attack utility ↑Qwen3-32B: ASR ↓Gemma-27B-IT: Attack utility ↑Gemma-27B-IT: ASR ↓
Default37.36%23.85%42.61%22.58%
Repeat user prompt43.88%21.30%40.06%19.87%
Spotlighting39.43%23.37%41.02%25.60%
Tool filter42.29%6.84%38.31%6.20%
Single-turn35.93%5.09%47.38%3.50%
Multi-turn41.49%6.20%47.38%5.41%
Single-turn + MPAGuard (ours)36.57%5.88%47.38%3.34%
Multi-turn + MPAGuard (ours)43.08%5.25%48.17%4.29%

Attack 2: Ignore Previous

MethodQwen3-32B: Attack utility ↑Qwen3-32B: ASR ↓Gemma-27B-IT: Attack utility ↑Gemma-27B-IT: ASR ↓
Default38.79%11.13%44.99%5.09%
Repeat user prompt49.44%8.27%47.06%3.34%
Spotlighting42.13%9.54%48.97%4.61%
Tool filter43.24%4.29%40.86%1.43%
Single-turn42.93%3.02%48.81%1.59%
Multi-turn42.77%3.34%48.01%2.54%
Single-turn + MPAGuard (ours)42.77%2.70%48.33%0.95%
Multi-turn + MPAGuard (ours)44.04%3.18%49.28%1.91%

Attack 3: Tool Knowledge

MethodQwen3-32B: Attack utility ↑Qwen3-32B: ASR ↓Gemma-27B-IT: Attack utility ↑Gemma-27B-IT: ASR ↓
Default37.36%28.46%40.38%26.55%
Repeat user prompt44.83%23.21%39.90%26.39%
Spotlighting34.98%30.21%40.22%31.64%
Tool filter43.88%8.27%37.20%6.84%
Single-turn41.18%5.25%46.74%6.04%
Multi-turn42.45%5.09%47.69%7.15%
Single-turn + MPAGuard (ours)40.38%4.45%47.69%4.93%
Multi-turn + MPAGuard (ours)43.08%4.13%48.97%4.61%

Key Findings

  1. Low attack success with competitive utility. Across all three attack types and both models, MPAGuard achieves low ASR (0.95–5.88%) while retaining competitive task utility.

  2. Consistent improvements in the multi-turn setting. Adding MPAGuard to the multi-turn baseline reduces ASR and improves task utility under attack across all six model–attack combinations. Single-turn gains are less consistent.

  3. Model-dependent benign-utility trade-offs relative to Tool Filter. Multi-turn MPAGuard has 1.03 percentage points lower benign utility for Qwen and 9.28 points higher for Gemma, while achieving lower ASR than Tool Filter across all three attacks.

Benchmark 2. Muses IF Selection

MUSES-IF Selection evaluates which user instructions should be accepted or rejected based on a shared global objective and an authority hierarchy; we apply MPAGuard to identify applicable constraints and resolve each instruction’s acceptance or rejection under these rules.

  • “Original-style” applies the task-adapted baseline with the benchmark’s original decision rules, whereas “Shared rules” additionally supplies the same explicit, benchmark-specific decision criteria used by our Single-turn, Multi-turn, and MPAGuard settings.

Qwen/Qwen3-32B

MethodF1: 2–10 usersF1: 11–20 usersOverall F1
Policy Manifest (1 call) / Original-style68.11%52.84%60.07%
Policy Manifest (1 call) / Shared rules72.12%56.17%63.73%
Self-Review (2 calls) / Original-style72.12%52.85%61.98%
Self-Review (2 calls) / Shared rules76.25%58.20%66.75%
Single-turn74.40%55.84%64.63%
Multi-turn79.81%58.78%68.74%
Single-turn + MPAGuard (ours)74.98%58.06%66.08%
Multi-turn + MPAGuard (ours)85.22%73.69%79.15%

Multi-turn + MPAGuard (ours) vs Multi-turn: +10.41 percentage points.

google/gemma-3-27b-it

MethodF1: 2–10 usersF1: 11–20 usersOverall F1
Policy Manifest (1 call) / Original-style79.70%65.17%72.05%
Policy Manifest (1 call) / Shared rules85.24%69.65%77.03%
Self-Review (2 calls) / Original-style81.48%62.37%71.42%
Self-Review (2 calls) / Shared rules90.09%74.85%82.07%
Single-turn90.84%76.29%83.19%
Multi-turn90.04%85.80%87.81%
Single-turn + MPAGuard (ours)90.47%75.23%82.45%
Multi-turn + MPAGuard (ours)93.48%89.19%91.22%

Key Findings

  1. Best overall performance. Multi-turn MPAGuard achieves the highest F1 across both models, outperforming Policy Manifest, Self-Review, and the single-turn and multi-turn baselines.

  2. Benefits across user-group sizes. Improvements hold for both smaller and larger user groups, supporting MPAGuard’s effectiveness in resolving instructions from multiple users.


Benchmark 3. OrgAccess

OrgAccess evaluates whether an organizational request should be accepted, rejected, or partially accepted based on role-based access permissions; we apply MPAGuard to identify applicable permission constraints and resolve the final access decision under the benchmark-specific label criteria.

Qwen/Qwen3-32B

MethodEasy accuracyMedium accuracyHard accuracyOverall accuracy
Policy Manifest (1 call) / Original-style57.60%28.13%28.33%38.02%
Policy Manifest (1 call) / Shared rules55.37%31.97%31.53%39.62%
Self-Review (2 calls) / Original-style54.40%38.10%39.63%44.04%
Self-Review (2 calls) / Shared rules52.53%40.30%44.27%45.70%
Single-turn55.60%32.33%33.27%40.40%
Multi-turn54.37%39.30%41.13%44.93%
Single-turn + MPAGuard (ours)56.10%32.70%32.63%40.48%
Multi-turn + MPAGuard (ours)52.13%40.40%47.70%46.74%

google/gemma-3-27b-it

MethodEasy accuracyMedium accuracyHard accuracyOverall accuracy
Policy Manifest (1 call) / Original-style51.93%23.27%22.83%32.68%
Policy Manifest (1 call) / Shared rules50.43%26.20%27.97%34.87%
Self-Review (2 calls) / Original-style47.63%35.13%42.67%41.81%
Self-Review (2 calls) / Shared rules47.77%34.77%43.93%42.16%
Single-turn49.63%30.57%34.70%38.30%
Multi-turn48.83%33.70%40.80%41.11%
Single-turn + MPAGuard (ours)49.47%32.33%36.47%39.42%
Multi-turn + MPAGuard (ours)47.50%37.03%47.07%43.87%

Key Findings

  1. Best overall performance. Multi-turn MPAGuard achieves the highest overall accuracy for both models, including against Self-Review under the same shared rules and two-call budget.

  2. Greater benefits on difficult requests. MPAGuard improves multi-turn accuracy on medium and hard requests, with the largest gains on hard requests, although easy-request accuracy decreases.

AuthorsOpenReview ↗

Manuscript Revision: Summary of Updates

We thank all reviewers for their constructive feedback.

We have updated the revised manuscript to incorporate the additional experiments and clarifications discussed during the rebuttal.

The major updates are summarized below:

  • Broader evaluation (Sections 6.3–6.4): Expanded AgentDojo to three attack types with established defense baselines, and added MUSES-IF Selection and OrgAccess evaluations. MPAGuard substantially reduces attack success while maintaining competitive task utility and improves multi-user and organizational authorization.
  • Execution enforcement (Section 6.3; Appendix E.3): Clarified mandatory, executor-enforced authorization checks, and runtime audit results.
  • Symbolic resolution extension (Section 6.5): Added an LLM-extraction-plus-symbolic-resolution baseline as an extension of our authority-resolution framework, rather than an alternative to its underlying semantics. The results also clarify the remaining challenges in textual-to-symbolic interpretation.
  • Discussion and limitations (Section 7): Clarified the contribution of applicability–polarity decomposition, the distinction between deterministic resolution and LLM decision accuracy, and future directions involving overlapping rules and ambiguous authority scopes.

We hope these revisions address the reviewers' remaining concerns and clarify the contributions and scope of our work.

We sincerely appreciate the reviewers' time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.