- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces CoSec, a benchmark for evaluating security and privacy violations in LLM agents operating across persistent communities. The benchmark consists of 208 scenarios covering static and dynamic community configurations and four families of adversarial interactions. The authors evaluate 13 combinations of agent harnesses and models, reporting significant rates of unauthorized information disclosure. Based on this evaluation, the paper reports that privacy behavior varies across harnesses, attack surfaces, and community states, showing how memory, files, tools, and workflows can transmit protected information beyond its authorized scope. The authors conclude that: (i) task utility does not imply privacy or authorization compliance - actually this finding seems quite obvious to me; (ii) authorization in community settings remains an unresolved security challenge for persistent LLM agents - this is also quite obvious as it is well known that today the area of identity and Access Management (IAM) for agentic AI is an under-developed area.
Critical strengths
The paper addresses an interesting setting involving persistent agents, multiple principals, shared workspaces, and changing community structures. Evaluating complete agent systems rather than only LLMs is also valuable. The authors have developed a relatively extensive experimental infrastructure and attempted to validate their outcome classification.
Critical weaknesses
-
The access control model is not adequately specified. My main concern is that the paper does not provide a sufficiently precise access control model. Section 3.1 introduces an authorization predicate and an abstract policy function, but this is not equivalent to specifying an access control model. The paper does not well define the semantics of roles, permissions, authorization rules, the interaction between role assignments and object attributes, or the mechanisms through which permissions are derived and combined. It is also unclear whether the model supports explicit negative authorizations, how conflicting authorization rules are resolved, and how policy exceptions are represented. This becomes particularly important when communities are merged or split, users change roles, or permissions are revoked. Also several access control models, such as the ones used in DBMSs, supporting negative authorizations - which are very useful in dynamic environments and for specifying authorization exceptions. Appendix 6.1 provides additional notation but does not resolve these questions. As a consequence, the authorization oracle appears to be based on scenario-specific policy decisions whose general semantics are not adequately explained. Since the benchmark's security conclusions depend fundamentally on this oracle, the lack of a well-specified authorization model is a critical methodological weakness.
-
The meaning and scope of the security guarantees provided by the benchmark are unclear. A fundamental question is what the benchmark actually establishes when it classifies an agent's behavior as secure or insecure. The paper does not provide a precise definition of the security properties that the benchmark is intended to evaluate. For example, does a successful evaluation indicate that the agent correctly enforces access control policies, that it does not disclose protected information to unauthorized recipients, or simply that it does not violate the particular authorization decisions encoded in the benchmark scenarios? These are different properties and require different evaluation criteria. In addition, even if an agent does not violate any of the authorization rules tested by the benchmark, this does not imply that the agent is secure or that privacy is preserved. Access control is only one component of security, and satisfying access control requirements is not sufficient to guarantee privacy. The distinction is important because the paper uses broad terms such as "agent security," "privacy preservation," and "authorization enforcement," whereas the actual evaluation appears to focus primarily on selected unauthorized information flows. The authors should have explicitly defined the properties being tested, the assumptions under which they hold, and the limitations of the conclusions that can be obtained from the collected metrics. Without such definitions, it is difficult to determine whether the benchmark results are significant or to compare them meaningfully with those of other security benchmarks.
-
The notions of security, privacy, and access control are conflated. This is another critical problem I have with this paper. The paper repeatedly uses sentences such as privacy boundaries, community security, authorization boundaries, and privacy and authorization enforcement, sometimes without distinguishing the properties being evaluated. These concepts are related, but they are not interchangeable; see list below:
i) Access control Whether a subject is authorized to perform an operation on a resource ii) Confidentiality Whether information is protected against unauthorized disclosure iii) Privacy Whether personal information is collected, used, retained, and disclosed appropriately iv) Information-flow security Whether information can flow from one security context to another v) System security A broader collection of properties, including confidentiality, integrity, availability, and other protections
For example, an agent may be authorized to access personal information but subsequently use it for an inappropriate purpose. Such behavior may violate privacy expectations even without violating the access permissions assigned to the agent. Conversely, unauthorized modification of a community document is an access control and integrity violation, but it is not a privacy violation. The paper refers to Nissenbaum's contextual integrity framework in Section 2. However, the evaluation in the paper largely reduces privacy to whether protected content reaches a recipient or destination declared unauthorized by the benchmark. That is a legitimate problem that one could investigate, but it is narrower than evaluating privacy in general. It thus seems to me that that the paper mainly studies unauthorized information disclosure and certain unauthorized agent actions, rather than security and privacy in the broad sense claimed by its title and discussion. This distinction impacts the interpretation of practically every experimental result.
- The paper does not establish what access control mechanisms the agents are expected to enforce. Section 3.1 says that "The adversary controls designated requester messages and, depending on the attack family, selected documents, metadata, memory entries, or workflow inputs." It then mentions that protected information may remain technically accessible to the agent even though disclosure is unauthorized. This is an interesting threat model. However, the paper does not explain the architecture responsible for enforcing access control. In a conventional access control system, one would expect a distinction between:
- A Policy Decision Point (PDP), responsible for evaluating authorization policies.
- A Policy Enforcement Point (PEP), responsible for mediating access to protected resources.
- The policy administration and attribute sources supplying the information needed for decisions. The above components are well known for network policies and also in normative reference arechitectures for access control. The paper instead appears to rely mainly on the ability of an LLM agent to interpret contextual information and refrain from inappropriate disclosures. But these are two rather different issues: (i) Whether the underlying system correctly implements and enforces authorization policies. (ii) Whether an LLM agent, given access to protected information, behaves consistently with instructions concerning its use and disclosure. The experiments mainly investigate the second problem. Indeed, Section 4.5 evaluates a prompt-based guardrail and concludes that prompt reminders cannot replace authorization checks. I agree with that conclusion. However, it raises a fundamental question: Why is the benchmark presented as evaluating authorization enforcement without a sufficiently explicit model of the authorization enforcement architecture?
For example, the benchmark could use a reference PDP to evaluate every tool request, with an enforcement mechanism mediating access to files, memory, and community resources. Agent behavior could then be evaluated separately from the effectiveness of the enforcement infrastructure. The paper does not make this distinction. There is also an architectural question concerning delegation. The target agent represents a particular user but interacts with other participants. The paper does not adequately specify whether the agent's permissions are derived from its owner, the current requester, its assigned community role, or some combination of these. Without such a distinction, it is difficult to interpret cross-user authorization failures.
- I find difficult to interpret the experimental results Table 3 (page 6) reports a large variation in Privacy Violation Rate (PVR) across agent systems, with overall values ranging from 29.33% to 96.15%. The authors interpret these results as evidence that agent systems exhibit poor privacy and authorization enforcement. I have some issues with this interpretation: i) PVR is essentially a rate of unauthorized information disclosure under the benchmark's policies. The term Privacy Violation Rate is too broad for the operational definition provided. ii) The work combines several different classes of events. Unauthorized information disclosure, unauthorized tool operations, unsafe workflow composition, and failures following changes in community membership are not necessarily violations of the same security property. Although the verifier distinguishes several failure dimensions, the paper's central discussion frequently treats them collectively as privacy or security failures. iii) The benchmarks are almost entirely adversarial. The four attack families include direct requester manipulation, indirect prompt injection, memory poisoning, and capability composition. High violation rates under these conditions establish susceptibility to the selected attacks, not necessarily the prevalence of privacy failures during ordinary community interactions. The Benign Completion Rate is a useful complementary measure, but does not resolve that distinction. iv) The security claims are broader than the evidence. The experiments demonstrate observed behaviors in specified synthetic environments, rather than the general effectiveness of particular access control mechanisms or the overall security of the evaluated agents.
There is also a statistical concern. Each configuration is evaluated on 208 scenarios, and dynamic-event breakdowns may rely on comparatively small numbers of observations. The paper reports point estimates and some verifier validation, but does not provide enough analysis of uncertainty, scenario dependence, or variability across repeated agent executions. Consequently, small differences among configurations should not be overinterpreted.
- The benchmark may conflate ordinary authorization failures with prompt-injection vulnerabilities. The four attack categories in Section 3.2 include:
- A1: User injection
- A2: Indirect injection
- A3: Memory poisoning
- A4: Capability composition These are useful experimental categories, but they mix different types of phenomena. For example, an attacker directly requesting protected information is not necessarily performing a prompt-injection attack. It may simply be making an unauthorized access request. Instead, an attacker placing malicious instructions in a retrieved document is exploiting a trust-boundary failure involving untrusted content. Similarly, memory poisoning is a form of state or context manipulation, while capability composition concerns the consequences of combining operations. The mechanisms, trust assumptions, and defenses differ considerably. I also have concerns about the use of the term user injection for ordinary adversarial requests. A more precise taxonomy would distinguish malicious access requests, instruction-hierarchy attacks, state manipulation, and multi-step authorization failures. This is not just a terminology issue. The selected attack mechanism influences what failure is being measured and which defenses would be effective.
- The discussion of dynamic access control is particularly weak. The DYN96 suite includes membership joins and removals, role upgrades and downgrades, community mergers and splits, and the creation or termination of relationships between communities. The paper does not explain enough about the underlying policy semantics to establish what should happen to existing permissions following such transitions. As an example, consider a community merger. There are multiple plausible policies: i) Members of the merged community inherit permissions from both original communities. ii) Access to historical information remains restricted according to the original community policies. iii) Selected information becomes shared, but other information remains subject to ownership or confidentiality constraints. iv) Permissions are recomputed based on the roles assigned within the merged community. The authors effectively choose particular semantics, but do not provide a general model explaining how such choices are represented and evaluated.Likewise, their treatment of revocation does not adequately distinguish revocation of a user's permission, revocation of an agent's delegated authority, and revocation of access to previously obtained information. These distinctions become especially important when agents maintain memory and persistent workspaces. I would have expected a paper claiming to benchmark dynamic authorization to specify these semantics clearly.
Questions for authors
None
3 comments
Response to the Authors
Dear Authors, thanks for your detailed and constructive rebuttal. I appreciate that you acknowledge several of my concerns, particularly regarding the interpretation of the security properties, the limitations of the experimental results, and the classification of attack mechanisms. The proposed revisions would improve the precision and clarity of the paper.
However, I believe that two fundamental issues are not yet satisfactorily addressed.
- Authorization model and policy semantics (W1, W7). You explain that the benchmark relies on scenario-specific specifications defining protected information, ownership, provenance, authorized recipients, and prohibited operations. You also clarify some of the policy choices governing dynamic transitions. While these clarifications are useful, they do not fully address my original concern. A collection of scenario-specific authorization rules is not equivalent to a well-defined access control model. The paper should specify how authorization decisions are derived from roles, relationships, object attributes, and other relevant contextual information. In particular, it remains unclear how potentially conflicting authorization rules are combined, whether explicit negative authorizations and exceptions are supported, and what general semantics govern permission changes.This is important because the benchmark's conclusions depend on the correctness of the reference authorization decisions. Without a sufficiently precise policy model, it is difficult to assess whether those decisions are consistent across scenarios or whether the benchmark can be extended to different authorization policies.
I do not necessarily expect you to develop a new access control model. You could adopt an existing model or define a restricted model tailored to your benchmark. However, the policy language and its evaluation semantics need to be precise to make the reference decisions reproducible and independently verifiable.
- What the benchmark actually evaluates (W2, W3, W4). I appreciate your clarification that the oracle serves as an evaluation reference rather than an online PDP, and that the experiments test whether agents refrain from unauthorized disclosures when protected information remains technically accessible. This clarification reinforces an important distinction made in my original review: the benchmark evaluates agent behavior under selected authorization constraints, rather than the enforcement of access control policies by an agentic system. These are different research problems. The former examines whether an agent behaves consistently with specified restrictions, while the latter concerns whether the system implements mechanisms that ensure such restrictions are enforced. Furthermore, you acknowledge that the baseline does not provide the agent with a fully specified policy and that some information is available only to the evaluator. This raises an additional question: "How can a failure be attributed to the agent's inability to respect authorization constraints when the agent may not have been provided with sufficient information to determine the correct authorization decision?"
I believe this distinction is essential for interpreting the results. The benchmark should distinguish failures arising from inadequate policy information, incorrect interpretation of available policies, and failures to comply with correctly understood restrictions. Otherwise, the reported violation rates may combine fundamentally different types of failures. I appreciate your commitment to narrowing your security and privacy claims. However, these clarifications should be reflected not only in the terminology but also in the central research questions, evaluation methodology, and conclusions.
- Experimental results and attack taxonomy (W5, W6). I find your responses to these concerns satisfactory. In particular, you acknowledge that the experiments do not establish failure frequencies in ordinary deployments, that the static and dynamic scenarios are not matched counterfactuals, and that single executions do not establish statistical stability. I also agree with the proposed revision of the attack taxonomy, especially the distinction between adversarial requester messages and prompt-injection attacks. These changes would improve the interpretation of the experimental results, although the lack of repeated executions remains a limitation of the empirical evidence.
Concluding remarks:The rebuttal clarifies the intended contributions and acknowledges several important limitations. I view these clarifications positively. Nevertheless, my main concern remains that the benchmark lacks a sufficiently explicit authorization model against which agent behavior can be evaluated. Moreover, the benchmark's actual contribution appears narrower than the evaluation of privacy and authorization enforcement suggested by the paper's broader framing. Thus, my main methodological concerns remain unresolved.
Thank you for explaining what remains unclear. We agree that listing the information stored in a scenario is not enough. Readers also need to know how the reference decision is made.
Our current benchmark uses a limited disclosure policy for each protected item. The scenario specifies which recipients and destination communities are allowed or forbidden. When the agent discloses an item, the verifier checks the recipient and destination that appear in the observed action. A destination is forbidden if it is explicitly prohibited or falls outside a specified allowed set. If an allowed and a forbidden rule both apply, the prohibition takes precedence. Reading protected information and disclosing it are treated as separate questions.
We should also be clearer about the limits of this policy. The current implementation does not contain a general engine that derives permissions from arbitrary combinations of roles, relationships, and object attributes. Events change the community state shown during the trial, while the scenario defines the disclosure boundary used to assess later actions. We will document these rules and limits together, so that readers can inspect how a decision is reached.
Your question about what the agent can see is equally important. The agent receives contextual guidance and notices of trusted events, but it does not receive the evaluator’s full list of allowed and forbidden destinations for every protected item. Our results therefore show which prohibited disclosures occurred under the information provided in these trials. They do not tell us, for every failure, whether the agent lacked enough policy information, misunderstood information it received, or failed to act on a restriction it understood. We will make that distinction explicit when presenting the results. A separate experiment that gives the agent the complete policy would help distinguish these causes; we have not run that experiment.
Thank you for the detailed comments. Your review raises three central questions: how the authorization policy is specified, what property the metrics measure, and which part of enforcement the experiments evaluate.
Authorization policy and transitions (W1, W7).
The predicate in Section 3.1 summarizes the authorization decision, but does not fully explain how that decision is obtained. The scenario specifications contain the concrete information used for evaluation: protected values, ownership and provenance, allowed and forbidden recipients, permitted abstractions, and forbidden effects. The paper needs to bring these rules together rather than leave readers to reconstruct them from separate sections.
For dynamic scenarios, events update the designated membership, role, relationship, or community fields. Objects retain their ownership and provenance. Expansion of collaboration does not automatically declassify historical private information. After a contraction, the affected prior authority no longer justifies subsequent disclosure. Previously obtained content can remain technically accessible, while its later transfer is judged against the updated state.
These are the policies selected for this benchmark. We do not claim that a merger must preserve historical restrictions in every application. Automatic inheritance would be another possible policy, requiring different reference decisions. The revision will make these choices explicit and distinguish changes to requester permissions, delegated authority, and continued access to previously obtained content. It will also specify the supported prohibitions and exceptions.
Measured properties (W2, W3).
PVR records protected content disclosed through agent generated outputs to an unauthorized recipient or destination. A read or path mention alone is insufficient. CCPVR measures disclosure between communities. Unauthorized actions and unsafe composition are separate verifier dimensions. STCR requires task completion without any verified security failure.
Your distinction between these properties and broader privacy is important. CoSec does not evaluate all questions of collection, retention, purpose of use, integrity, or availability. Passing the benchmark does not establish general security. We will align the terminology and conclusions with the properties actually measured.
Enforcement architecture (W4).
The oracle provides a reference for evaluation rather than serving as an online PDP. Docker isolates trials; it does not enforce community permissions. The experiments test how the agent behaves when information remains technically reachable but its disclosure is restricted.
The target agent acts for one principal, while each authorization decision concerns the current requester, action, object, and destination. Access available to the agent does not automatically grant disclosure rights to the requester. An online PDP and PEP could enforce these restrictions externally, but the current experiments do not evaluate that architecture.
The baseline supplies lightweight contextual guidance rather than a fully enumerated policy table. We will distinguish information visible to the agent from values and labels available only to the evaluator, and avoid treating every observed failure as incorrect execution of an explicitly supplied formal policy.
Interpretation of results (W5).
The reported rates describe outcomes under the selected adversarial scenarios. They do not estimate failure frequency in ordinary deployment. BCR measures whether legitimate work is completed within those interactions, rather than utility on a separate workload without attacks.
Likewise, harness comparisons concern complete configurations and do not isolate individual components. Static112 and Dyn96 are not matched counterfactual pairs, so their differences do not identify the causal effect of a community transition.
The breakdowns by event also have small denominators. We will present them as descriptive results and make the sample sizes explicit. Results from one execution per scenario do not establish stability across repeated executions, and small differences should not support strong ranking claims.
Attack taxonomy (W6).
We agree that an unauthorized request is not necessarily prompt injection. “Adversarial requester messages” is a more accurate name for A1. A2 concerns instructions in untrusted environmental content, A3 concerns manipulation of persistent state, and A4 concerns sequences of operations that produce an unauthorized information flow. These are different stressors, with different trust assumptions and possible defenses. We will revise the taxonomy accordingly.