acceptodds
Withdrawn from ICLR 2027

Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-agent systems derive their capabilities from sharing evidence, delegating tasks, and combining information across agents. The same process creates a safety problem: contributions that are admissible in isolation can jointly enable a prohibited use. Blocking every sensitive action avoids disclosure but defeats the purpose of collaboration. We introduce authorization-paired evaluation, which makes blocking prohibited uses and completing required authorized uses a joint success criterion, and FlowReview, a framework connecting object resolution, permission ranking, and deterministic enforcement. In controlled composition experiments, reviewing combined artifacts reduces the denied-commit rate from 86.0% to zero with no loss of authorized supply. Our findings show that preserving information and lineage alone does not ensure correct permission attribution. Object identity and permission must remain connected to execution through components whose outputs can be verified. Together, these findings establish a system-level requirement for multi-agent safety: govern composed information flows while preserving the capabilities that make collaboration useful, from combining distributed information to executing authorized actions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-agent System, Agent Safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_bUXVOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper tackles multi-agent composition vulnerabilities—where individually harmless fragments are combined downstream to enable prohibited actions—by introducing Authorization-Paired Evaluation, which couples prohibited and task-required authorized uses in the same scenario to measure Selective Correctness (C=(1−D)AC = (1 - D)A, requiring systems to block denied disclosures D=0D=0 while completing authorized supply A=1A=1). To enforce this "deny without disabling" target, the authors propose FlowReview, a three-stage control pipeline built on pre-registered object and action contracts: (1) Object Resolution, using a Global Reader over the union of workflow artifacts to identify reconstructed sensitive objects; (2) Permission Ranking, leveraging two-sided object binding and an isolated Permission Specialist LLM to verify candidate actions against trusted policy PP, object ID, and policy version while ignoring untrusted claims QQ; and (3) Deterministic Enforcement, where a runtime Commit Gate independently checks contract fields at the action boundary, supported by deterministic Runtime Lineage tracking and a Runtime Assembler to prevent context loss across multi-agent handoffs.

Critical strengths
  • Provided with abundant families of models. This work has carefully experimented on multiple settings with different models, including homogeneous model groups and heterogeneous model groups. Not only that, it involves both close-source models (e.g. Claude-sonnet-4-5) and open-source models (e.g. QwQ-32B). However this also potentially makes the disadvantage as the models are way too outdated, like QwQ was published in late 2024.
Critical weaknesses
  • Verbose and Meaningless pipelines. Firstly, it uses a LLM to serve for structured data validity as alleged "isolated permission specialist". The specialist's sole instruction is to select the unchanged candidate if and only if P == ALLOW and the object ID and policy version match. Further, this brings a lot of computation burdening such that it seems not that useful than just using hardcoded criteria. And yes, the author also provided Commit Gate served as re-verifier of rules, which makes it even more confusing to understand the intent.

  • Limited Novelty and lack of motivation. Authoritarian-paired Evaluation seems a trade-off between over-rejection and utility, yet it is just a simple and trivial approach that barely becomes contribution. The core issue is that multi-agent systems can fail when locally permissible steps combine into unauthorized actions. There are already works like: multi-CaMeL on cross-agent provenance; PCAS on declarative policies over causal dependency graphs with deterministic reference-monitor mediation; I have seen lots of limitation that this work is worth of getting accepted in this conference.

Questions for authors
  • What would be your differentiation between your concerns / motivation and other recent works?
Reviewer_x4i3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces the 'authorization-paired' evaluation, which measures whether a multi-agent system can block prohibited uses while preserving required authorized uses. It also proposes FlowReview, connecting object identification, permission decisions, and runtime enforcement. Experiments examine how composition and delegation affect these capabilities, with an additional tool-use evaluation.

Critical strengths

The paired evaluation captures an important practical requirement: safety controls should block prohibited uses while preserving the capabilities needed for authorized tasks. The controlled comparisons further distinguish failures in object identification, permission handling, and execution, providing a useful framework for diagnosing where safeguards break down in multi-agent workflows.

Critical weaknesses
  • The main read-scope comparison allows unresolved objects to pass, even when proposals contain object references. The improvement therefore depends partly on the baseline’s enforcement rule. A direct reference-based policy check maybe help clarify when joint artifact reconstruction is necessary.
  • The tool-use study blocks a predefined action that the executor is instructed to attempt. This supports execution control, but evidence that the complete framework can identify and govern unauthorized actions arising during ordinary collaboration remains limited.
Questions for authors

Could the authors explain why policy enforcement must wait for object reconstruction when the proposed action already contains an object reference? Comparing FlowReview with a simpler gate that checks this reference directly against trusted policy would help clarify when reviewing the combined artifacts is necessary to ensure safety while preserving authorized task completion.

Reviewer_1WGoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a new evaluation for data provenance in complex agentic flows and proposes a new mechanism FlowReview that judges action based on aggregated data as well as individual pieces.

Critical strengths

The paper adopts AgentDojo and evaluates the methods on a set of strong models. Overall, analysis and evaluation looks quite well done.

Critical weaknesses

The paper lacks clarity on the structure and would benefit from solid rewriting.

Specifically, it is not clear what is the main contribution of the paper given the provenance and lineage of the systems have been studied extensively over the last decades.

I understand that you combine the utility and security together into one metric, but I would say it can change the weight of the metric, i.e. when there 10 unimportant blocked calls and allowed sharing that is very sensitive?

It is not clear why we cannot track provenance outside of the LLM? See for example: "Efficient and Sound Probabilistic Verification for AI Agents" by Solko-Breslin et al (2026).

Questions for authors

Overall, improving and cleaning writing would help a lot for the clarity of the paper. I am not convinced that the proposed approach of LLM tracking provenance is relevant and cannot be done using deterministic methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.