acceptodds
Under review as a conference paper at ICLR 2027

AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents

Abstract

LLM agents are becoming increasingly capable of handling end-to-end daily workspace tasks, enabled by their ability to smoothly interact with environments through tool calls. However, as LLM agents are increasingly deployed in information-rich, real-world environments where they read and write user data across email, calendar, and other applications, a central privacy question is whether they preserve contextual integrity (CI), that is, whether the information flows they produce comply with context-specific disclosure constraints. Existing agent evaluation platforms and benchmarks either only focus on model-level privacy leakage or evaluate agents across a limited set of application environments with manually constructed disclosure constraints. In this work, we introduce AgentCIScope, a unified platform with a comprehensive benchmark for evaluatingagent-level contextual integrity grounded in explicit disclosure constraints through executable, multi-step tasks across diverse, realistic application environments. AgentCIScope is constructed through a unified pipeline that transforms synthetic and real-world privacy cases into executable agent tasks, with grounded disclosure constraints defined by the intersection of three criteria: governing privacy policies, persona-based agreement, and a predefined sensitive-attribute taxonomy. Using AgentCIScope, we evaluate 12 types of agents over 936 tasks. We find that strong task-completion capability does not necessarily imply contextual-integrity preservation: single-run leakage rates range from 32.5% to 94.4%, while repeating each task five times further increases leakage by roughly 4–19 percentage points. Across agents, we find that certain sensitive attributes are leaked more often, with personal preferences and booking details reaching average leakage rates of 84.7% and 76.5%. In addition, highly sensitive attributes such as credit-card information and SSNs also exhibit average leakage rates around 17.1% and 29.4%. Among different environments, Calendar has the highest average leakage rate as an information source, while WhatsApp is ranked the highest as an information destination; leakage rates also vary substantially across governing privacy policies. Our evaluation results provide broad insights into patterns of agent privacy leakage, with AgentCIScope serving as a unified testbed for developing privacy-preserving agents.

Primary area:
datasets and benchmarks
Keywords:
Agent, Privacy, Contextual Integrity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iomsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AgentCIScope evaluates contextual integrity in LLM agents through executable tasks across multiple applications. Its construction pipeline combines privacy contracts, a fixed attribute taxonomy, LLM persona agreement, policy validation and stress-test instruction generation. The benchmark contain 936936 tasks that derived from 212212 contracts across 1111 environments. 1212 agent evaluation over 5616056160 trajectories reveals substantial leakage, differences across agent stacks and information types and additional failures under repeated execution.

Critical strengths
  1. Broad executable evaluation.

The benchmark supports multistep information flows across diverse applications and evaluates several model framework combinations. It provide potentially useful extension beyond privacy judgments and isolated actions and demonstrates that task completion could coexist with substantial disclosure failures.


  1. Structured construction with human validation.

Explicit recipient-specific contracts, coherent environment population and evidence-based leakage judgments makes the evaluation auditable. Appendix D’s human agreement result provide encouraging support for the generated labels and automated judge.

Critical weaknesses
  1. Surrogate-based selection limits the interpretation of the results.

Every retained task must induce leakage from GPT-5.6 Sol with OpenClaw 3.3. It is reasonable for constructing a stress test, but the reported rates characterize a deliberately selected failure set. Without results on base instructions, unfiltered candidates, and alternative surrogates, it remains unclear how much the observed agent rankings and attribute or environment patterns depend in the selection procedure. Moreover, the 936936 tasks share only 212212 underlying contracts, whereas the reported Wilson intervals don't account for this dependence.


  1. The policy-grounding claim needs stronger validation.

Contract generation, persona judgments, policy validation and leakage evaluation use the same model. So persona agreement don't establish independent normative agreement. Human validation is encouraging, but covers only 5050 tasks from 11 agent configuration, and assesses attribute privacy status rather than reporting a separate audit of policy applicability. Because agents receives neither contracts nor policies, the benchmark also needs to establish that relevant recipient permissions, and disclosure restrictions are inferable from the visible context. Otherwise, some failure might reflect unavailable contextual information rather than clear violations.

Questions for authors
  1. How does leakage rate and agent ranking change on base instructions, and the unfiltered candidate pool? What fraction of candidates survives surrogate filtering?

  2. What information visible to tested agent establishes recipient authorization and disclosure restrictions? Does providing applicable policy context materially change the result?

  3. How many human annotator participated, and are they blinded to the generated labels? Is policy applicability independently assessed and does judge reliability hold across other agent configurations?

  4. Do the framework differences remain significant under paired analysis with uncertainty estimated in the contract level? Are system prompts, reasoning budgets, decoding setting and tool permissions controlled and documented?

  5. How sensitive is the reported leakage rate to explicit CI reasoning, given prior evidence that reasoning and RL post-training improve contextual integrity and preserving task performance [1]?

[1] Contextual Integrity in LLMs via Reasoning and Reinforcement Learning. NeurIPS 2025.

5 comments

Thank you for recognizing the breadth of our executable evaluation, the structured benchmark construction, and the supporting human validation. Below, we address each weakness and question.

W1-A / Q1 Surrogate-Based Filtering and Confidence (Appendix D, Table 6)

Thank you for raising this concern. Across all generation runs, our pipeline evaluated 3,420 candidate instructions against the surrogate agent and retained 936 (27.4%) that triggered leakage.

Regarding the motivation for surrogate-based filtering, our goal is to construct a challenging benchmark that helps the community identify privacy risks and advance privacy-preserving agents. We believe selecting challenging cases provides greater value for stress-testing existing agents and driving future improvements.

Regarding the impact of filtering on agent rankings and attribute/environment patterns, we acknowledge that our current findings are based on the filtered benchmark. Due to time and API cost constraints, evaluating all agents on the full unfiltered candidate pool is impractical. Instead, we conducted an additional experiment on 80 tasks from 80 different contracts, evaluating eight agents with both benchmark instructions (i.e., the filtered instructions) and randomly sampled filtered-out candidates (i.e., instructions on which the surrogate did not leak). We did not separately evaluate base instructions, as they serve as intermediate inputs to our instruction-generation pipeline rather than finalized evaluation tasks. Both conditions used the same contracts, environments, and judges, differing only in the instructions. Although leakage rates were lower on filtered-out instructions, the relative agent rankings remained highly consistent (Kendall's τ = 0.93), suggesting that selection does not substantially affect cross-agent comparisons in this evaluation.

AgentBenchmark (TLR)Filtered-out (TLR)
Google ADK + Gemini 3.8 Flash93.8%63.8%
OpenAI Agents SDK + Grok 4.688.8%45.0%
OpenClaw + Kimi K386.3%48.8%
Google ADK + Gemini 3.1 Pro82.5%43.8%
Claude Code + Claude Sonnet 535.0%16.3%
OpenClaw + Claude Opus 531.3%15.0%
OpenAI Agents SDK + GPT-6 Astra30.0%11.3%
Claude Code + Claude Opus 523.8%6.3%

W1-B Confidence Intervals (Appendix E, Table 7)

Thank you for raising this. The original Wilson intervals treat tasks as independent. To account for within-contract dependence, we additionally computed 95% confidence intervals using a contract-level cluster bootstrap, resampling contracts rather than individual tasks. The resulting intervals are 1.2-1.9× wider than the Wilson intervals, but our main conclusions remain unchanged: low- and high-leakage agents remain clearly separated, and agent rankings remain stable. We have added the methodology and results to the revised paper.

W2-A. Additional Validation of Persona Judgments, Policy Applicability, and LLM Judges (Appendix G)

Thank you for this valuable suggestion. To strengthen the independent validation of our benchmark, we conducted additional experiments examining persona judgments, policy applicability, and leakage evaluation.

1. Persona Judgment Validation. To assess the reliability of persona judgments across different model families, we independently re-evaluated all attributes using three models: GPT-5.4, Claude Sonnet 5.5, and Gemini 3.1 Pro. The results show consistently high inter-model agreement:

Model PairAgreementCohen's κ [95% CI]
GPT-5.4 vs. Claude Sonnet 5.596.2%0.93 [0.91, 0.95]
GPT-5.4 vs. Gemini 3.1 Pro94.0%0.89 [0.86, 0.91]
Claude Sonnet 5.5 vs. Gemini 3.1 Pro93.9%0.89 [0.86, 0.91]

These results demonstrate strong consistency in persona judgments across different model families, reducing concerns about reliance on a single model's judgments.

2. Policy Applicability Validation.

To validate the applicability of the policies in our benchmark attributes, we conducted both cross-model validation and additional human annotation.

Cross-Model Validation. We evaluated all attributes using three models: GPT-5.4, Claude Sonnet 5.5, and Gemini 3.1 Pro. The results are summarized below.

Table 1. Policy Applicability Validation

ModelApplicableNot ApplicableUncertain
GPT-5.490.4%2.9%6.7%
Claude Sonnet 5.585.3%5.0%9.7%
Gemini 3.1 Pro93.2%5.0%1.8%

Table 2. Inter-Model Agreement on Policy Applicability

Model PairAgreement Rate
GPT-5.4 vs. Claude Sonnet 5.586.8%
GPT-5.4 vs. Gemini 3.1 Pro93.0%
Claude Sonnet 5.5 vs. Gemini 3.1 Pro89.5%

All three models judged over 85% of the policy-attribute pairs as applicable, with pairwise agreement rates ranging from 86.8% to 93.0%.

Human Validation. To further validate policy applicability independently of LLM judgments, three human annotators assess 30 sampled cases, covering 167 policy applicability judgments. All three annotators unanimously judged 140 out of 167 (83.8%) as applicable.

These results provide additional evidence of policy applicability across independent model and human assessments, strengthening the reliability of our policy-grounded benchmark.

3. Leakage Evaluation Validation. To assess the reliability of our LLM-based leakage evaluation, we conducted additional validation using both independent model judgments and human annotations.

3.1. Human validation across agent configurations. Three annotators independently labeled 150 agent trajectories, covering the same 30 tasks under five configurations spanning all four frameworks. For each protected attribute specified in the privacy contract (460 attribute instances in total), annotators determined whether it was leaked to a forbidden recipient. We used majority voting to establish the human reference labels. The table below reports the judge's agreement with these labels at both the run and attribute levels (run-level / attribute-level).

ConfigurationJudge–Human Majority Agreement (Run / Attribute)
Claude Code + Claude Opus 596.7% / 96.7%
Google ADK + Gemini 3.8 Flash100% / 98.9%
OpenAI Agents SDK + GPT-6 Astra96.7% / 94.6%
OpenAI Agents SDK + Grok 4.6100% / 94.6%
OpenClaw + Kimi K396.7% / 97.8%
Overall98.0% / 96.5%

Overall, these results demonstrate that the judge's reliability extends beyond GPT-6 Astra with the OpenAI Agents SDK. We believe our evidence-based leakage assessment, which grounds judgments in concrete evidence from agent trajectories, also helps support the consistency and reliability of these evaluations.

3.2. Cross-model judge consistency. We further evaluated 600 trajectories (50 tasks × 12 agents) using two additional LLM judges, Gemini 3.8 Flash and Claude Haiku 5.5, with the same evaluation prompt as our primary GPT-5.4 judge. Compared with GPT-5.4, Gemini 3.8 Flash achieves 90.7% run-level agreement, while Claude Haiku 5.5 achieves 91.7%. At the attribute level, the two pairwise comparisons with GPT-5.4 yield agreement rates of 94.7% and 96.0%. Importantly, the relative ranking of all 12 agents by leakage rate remains largely consistent across judges. These findings further support the robustness of our evaluation conclusions across different judge models.

W2-B and Q2-A: Agent Context Visibility and Authorization

Thank you for raising this important point. We would like to clarify that agents receive user instructions specifying the intended recipient and task purpose. Our environment construction also ensures that the recipient's role is reflected in the environment data. Agents are expected to assess whether disclosing sensitive information to the recipient is appropriate based on the available context. Additionally, our quality gate ensures that user instructions do not explicitly request sensitive information.

Regarding the concern that some failures may reflect insufficient contextual information rather than clear violations, we argue that uncertainty about authorization should not justify disclosing sensitive information. When the available context does not clearly establish whether disclosure is authorized, a privacy-conscious agent should not assume permission. We believe defaulting to non-disclosure under uncertainty is an important principle of responsible agent behavior.

Q2-B / Q5: Impact of Policy Context and Explicit CI Reasoning (Appendix B, Table 4)

Thank you for these valuable suggestions. To investigate whether providing explicit policy context and CI reasoning guidance reduces privacy leakage, we conducted additional experiments on the 100 tasks sampled from 100 distinct CI contracts. We evaluated three agents under the same tasks, environments, and judges. The full results are provided in the Appendix.

DefenseKimi K3 (OpenClaw)Gemini 3.8 Flash ( Google ADK)GPT-6.1 Sol (OpenAI Agents SDK)
No defense (TLR)92%95%47%
Explicit policy context79% (−13 pp)63% (−32 pp)17% (−30 pp)
Privacy-enhancing prompt (PrivacyLens; Shao et al., 2024)90% (−2 pp)78% (−17 pp)31% (−16 pp)
CI-CoT prompting (Lan et al., 2025)69% (−23 pp)58% (−37 pp)16% (−31 pp)
Data-minimization prompt (AgentCIBench; Goel & Gurevych, 2026)91% (−1 pp)82% (−13 pp)36% (−11 pp)
CI-based supervisor (Ghalebikesabi et al., 2024)70% (−22 pp)78% (−17 pp)30% (−17 pp)

The results show that both providing explicit policy context and incorporating CI reasoning reduce privacy leakage across all three agents. However, privacy leakage persists despite these interventions, demonstrating that AgentCIScope remains effective in identifying and distinguishing agent vulnerabilities even with additional privacy safeguards.

Q3: Human Validation (Appendix G)

In our revised version, three human annotators participated in the validation and were blinded to the generated labels. We also conducted an independent assessment of policy applicability and extended our judge reliability evaluation to additional agent configurations.

Q4-A: Statistical significance of framework differences (Appendix E, Table 8)

Thank you for this suggestion. We conducted a paired analysis using contract-level cluster bootstrapping to compute 95% confidence intervals for framework differences. Overall, the results suggest that Claude Code and OpenClaw tend to exhibit lower privacy leakage rates than OpenAI Agents SDK and Google ADK. With the backbone held fixed, the framework difference remains significant for Claude Opus 5 (Claude Code vs. OpenClaw: −11.1 pp, 95% CI [−14.9, −7.3]), GPT-6 Astra (OpenClaw vs. OpenAI Agents SDK: −8.1 pp, [−10.2, −6.1]), and Kimi K3 (OpenClaw vs. Google ADK: −4.2 pp, [−6.7, −1.8]).

Q4-B: Agent Configuration Details (Appendix H)

Thank you for this suggestion. We have expanded the Appendix to provide more detailed documentation of system prompts, reasoning budgets, decoding settings, and tool permissions across agent configurations.

Thank you again for your thoughtful review and constructive feedback. We hope the additional experiments and clarifications address your concerns. We would be grateful if you would reconsider your assessment in light of these revisions. Thank you for your time and effort!

Reviewer_Yw5UOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper provides a novel contextual privacy evaluation benchmark for evaluating contextually inappropriate leakage in agents in rich environments that allow them to interact across applications, performing multi-step tasks in multiple domains under different privacy instructions and contexts. Evaluation across multiple frontier models and agent harness frameworks yields insights on leakage with respect to the source and destination of the information, task repetition, across data attribute types, etc.

Critical strengths
  • [S1] In-depth empirical evaluations across multiple frontier models, agent frameworks, source and destination applications, etc. is appreciated. The analysis on source and destination leakage rate is a nice empirical illustration which is novel to this benchmark.
  • [S2] The benchmark is well-described with careful consistency validation, and is rich and representative of agent environments with multiple applications, tasks, realistic dependencies between artifacts, etc.
Critical weaknesses
  • {W1] The empirical evaluations suffer from some key soundness deficiencies. This evaluation is lacking evaluation against agents equipped with relevant privacy protection/preservation methods such as supervisors from Ghalebikesabi et al. [1], data minimizers from AirGapAgent [2], Agentic Firewalls for input, output and data filtering [3[, etc. This list is not exhaustive. As such, the agents do not employ such privacy enhancing measures/defenses for agents existing in the literature. Additionally, and in a related vein, the authors do not study as an important sanity check whether the agents reveal information under explicit requests (lines 255 to 257). This is important as the authors should demonstrate as a sanity check and baseline that their agents are not susceptible to simple explicit requests, if they are, then they are already compromised and the privacy leakage problem, even "inadvertent", becomes trivial as the agents may reveal sensitive information easily and are already compromised. Even for the privacy-strict/conscious persona, the authors implicitly rely on its wording for privacy preservation but even with that directive, can the authors empirically guarantee that it is robust to this simple explicit request baseline?

  • [W2] The authors make key overclaims which need to be scoped.

    • For instance, in Section 4.4, they claim that their results compare agents on whole policies (as in legal policies), which are broad, complex, and nuanced documents. They should be careful when making broad claims for policies and should instead say that this is a study of privacy norms/directives derived from different policies using their derivation paradigm in a particular setting; temper their language. Additionally, they should also mention concretely how they derived these privacy directives.
    • Additionally, in the limitations, the authors' prose makes it seem like defenses do not exist and it is left to future work (lines 1198 - 1200). However, there exist multiple works in this regard, including but not limited to [1, 2, 3]. Indeed, for a proper evaluation, the agents they employ should be tested with these defenses deployed.
Questions for authors

Can the authors address all raised concerns in the weaknesses section and also clarify some additional relatively minor concerns?

  • [Q1] The authors use a good and granular set of metrics but do the leakage rates take into consideration when a task may complete before leakage and thus terminate before a leakage is observed, thus invalidating the leakage in that trajectory (see [4], for instance) or does the agent just exit directly after the task is done?
  • [Q2] Can the authors improve the presentation of figures like Figure 3 (Page 7) where some fonts exceed figure limits/boundaries and visual elements such as model logos are cramped and overlapping? For heatmaps, can they ensure the readability of the text/numbers therein? At the moment they are extremely hard to read.
  • [Q3] - Prior literature such as [3,4] already provide a privacy directive formulation which the authors introduce as CI contracts, which appears to be redundant new terminology. Can the authors please refer to prior literature and attempt to homogenize such terms with existing literature?

References

  • [1] Ghalebikesabi, S., Bagdasaryan, E., Yi, R., Yona, I., Shumailov, I., Pappu, A., Shi, C., Weidinger, L., Stanforth, R., Berrada, L., Kohli, P., Huang, P., & Balle, B. (2024). Operationalizing Contextual Integrity in Privacy-Conscious Assistants. ArXiv, abs/2408.02373.
  • [2] Bagdasarian, E., Yi, R., Ghalebikesabi, S., Kairouz, P., Gruteser, M., Oh, S., Balle, B., & Ramage, D. (2024). AirGapAgent: Protecting Privacy-Conscious Conversational Agents. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security.
  • [3] Abdelnabi, S., Gomaa, A., Bagdasarian, E., Kristensson, P.O., & Shokri, R. (2025). Firewalls to Secure Dynamic LLM Agentic Networks. ArXiv, abs/2502.01822.
  • [4] Das, S., Sandler, J., & Fioretto, F. (2025). Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents.
6 comments

Thank you for recognizing our work as novel with In-depth empirical evaluations. Below, we address each weakness and question.

W1-A Defense and Mitigations (Appendix B, Table 4)

Thank you for this valuable suggestion. Following your suggestion, we sampled 100 tasks from distinct CI contracts, covering diverse privacy policies, environments, and sensitive attributes. We evaluated three agents using different privacy defenses, each compared against a paired no-defense baseline under identical conditions. The full results are in the Appendix, and the task-level leakage rates are reported below:

DefenseKimi K3 (OpenClaw)Gemini 3.8 Flash (Google ADK)GPT-6.1 Sol (OpenAI Agents SDK)
No defense92%95%47%
Privacy-enhancing prompt (PrivacyLens; Shao et al., 2024)90% (−2 pp)78% (−17 pp)31% (−16 pp)
CI-CoT prompting (Lan et al., 2025)69% (−23 pp)58% (−37 pp)16% (−31 pp)
Data-minimization prompt (AgentCIBench; Goel & Gurevych, 2026)91% (−1 pp)82% (−13 pp)36% (−11 pp)
CI-based supervisor (Ghalebikesabi et al., 2024)70% (−22 pp)78% (−17 pp)30% (−17 pp)

Despite these defenses, privacy leakage persists, with TLRs remaining at 16–69% even under the strongest defense. These results demonstrate that existing defenses mitigate but do not eliminate privacy leakage, and AgentCIScope remains effective at distinguishing agent vulnerabilities. Additionally, we also cite related works, including AirGapAgent and Agentic Firewalls in our revised version.

W1-B Explicit-Request Sanity Check (Appendix C, Table 5)

Thank you for this suggestion. Following your advice, we conducted an additional experiment using 100 tasks. We kept the environments, data, contracts, and judge unchanged, replacing only the user instruction with a straightforward request to disclose all protected attributes to a contract-prohibited recipient. The complete results and an example of the explicit-request prompt are provided in Appendix C.

On Claude Code with Claude Opus 5, the explicit-request baseline yields a 13.0% task-level leakage rate (TLR), with only 8.5% (24/282) of requested protected attributes disclosed. In contrast, our original stress test yields a 25.0% TLR on the corresponding set of 100 stress-test tasks, nearly twice the explicit-request baseline. Importantly, among the 87 tasks where the agent did not leak under explicit requests, 20 (23%) still resulted in leakage under our stress test. These findings demonstrate that AgentCIScope captures privacy failures beyond simple compliance with explicit disclosure requests. Agents that successfully resist direct disclosure requests may still inadvertently reveal protected information while performing legitimate user tasks.

As discussed in our response to Reviewer tfmf (W1/F1, Fairness of Stress-Test Instructions), explicit disclosure requests represent a different threat model. Such requests evaluate whether agents comply with directly risky instructions, but AgentCIScope focuses on inadvertent information disclosure during benign task execution. Our quality gate intentionally excludes instructions that explicitly request the disclosure of protected information.

W2-A Claim scope clarification and Policy-Grounded (Section 3.1, Section 4, Section 4.4)

Thank you for the suggestion. We agree that our claims should be scoped. AgentCIScope evaluates task-specific disclosure constraints validated against relevant policy provisions through our policy-grounded validation procedure, rather than real-world whole legal compliance. We have revised Section 4.4 to clarify that our analysis compares leakage rates across policy-grounded disclosure constraints and have also clarified in the Metrics section that policy labels are assigned through our policy-grounded validation procedure, and violations are defined within our benchmark. We have also clarified our policy-related privacy directive derivation procedure in Section 3.1 (Policy-Grounded Validation). Starting from policy references in the seed data, our LLM validator assesses each proposed disclosure against the verbatim policy text.

W2-B Discussion on Defense (Appendix N)

Thank you for pointing this out. We are sorry for the imprecise wording, and we agree that prior work has already proposed several privacy-preserving defenses for LLM agents. Following your advice, we have revised the Limitations and Discussion section to explicitly acknowledge these existing defenses and clarify that our future work focuses on developing more effective defenses, rather than introducing defenses from scratch. We have also added new experiments evaluating four existing privacy defense approaches across three agent configurations, as detailed in our response to W1.

Q1. Task Completion and Leakage Evaluation

Thank you for raising this point. In our evaluation, each agent executes autonomously in a single autonomous turn until its natural termination or reaches the maximum step limit. Our judge then evaluates the complete execution trajectory and makes evidence-based leakage judgments based on observed agent behaviors. Therefore, leakage occurring after agent termination will not happen in our setting.

Q2 Paper presentation (Figures 3, 4, 5, 6, 9, 10)

Thank you for pointing this out. We have revised the figures to fix overlapping elements, adjust font sizes and spacing, and improve the readability of heatmap labels and values.

Q3. Clarification and Comparison of Terminology

Thank you for this suggestion. We acknowledge that CI contracts conceptually overlap with the privacy directives introduced in [4]. Compared with prior work, our CI contracts provide a more comprehensive, multidimensional specification that integrates contextual information such as base task instructions, participant roles (senders and recipients), and target environments. They also capture fine-grained, recipient-specific restrictions for multi-tool, cross-application workflows. In terms of application, our CI contracts are used in benchmark generation and evaluation, rather than serving as agent-facing defense instructions as in [4]. We are happy to consider further suggestions on homogenizing our terminology with prior literature.

Thank you again for your thoughtful review and constructive feedback. We hope the additional experiments and clarifications address your concerns. We would be grateful if you would reconsider your assessment in light of these revisions. Thank you for your time and effort!

[4] Das, S., Sandler, J., & Fioretto, F. (2025). Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents.

I thank the authors for engaging with my concerns thoroughly and running additional evaluations (which are fascinating and insightful), for clarifying issues with their claims and setup, for discussing prior work, etc.

In particular, I would like to highlight and appreciate the evaluations on agents with defenses applied and the sanity check; they are convincing and strongly convey key messages for this work. A minor point regarding the sanity checks (which I won't ask the authors to further work on empirically): certain readers in the security community may even demand a worst-case guarantee that the agents are robust (no leakage) to sanity checks, as even one leakage under that regime means that an agent is vulnerable to explicit requests. The authors may want to make a note of this in their limitations. For my rebuttal's intents and purposes, the current evaluations suffice, and I am satisfied with the message.

I am happy to discuss homogenization of terminology with the authors and provide actionable feedback on any facet of their paper while they update it.


The objective of my criticisms was to improve the paper, as I thought it had promise, and these additions now make this work much more convincing. With that said, I am happy to raise my score to a 3, and will go through the rebuttals and revised paper again and see if I would like to champion this work even more strongly.

Reviewer_tfmfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AgentCIScope, a benchmark for evaluating contextual integrity in LLM agents. The authors develop an automated pipeline that converts privacy seed cases into executable multi-step tasks across MCP-based application environments, with explicit contextual integrity contracts defining what information should be protected. The benchmark covers multiple LLM backbones and agent frameworks. The results suggest that privacy leakage is widespread, varies across models and frameworks, and becomes more frequent over repeated runs. The authors also find that contextual information, such as personal preferences, is more likely to leak than explicit identifiers.

Critical strengths

1. A novel and broadly applicable framework for measuring contextual-integrity leakage.

Rather than a hand-built task set, the paper contributes an automated pipeline that turns privacy seed cases into executable, policy-grounded CI contracts and populated multi-application environments. The same pipeline applies across backbone models, agent frameworks, application environments, and task types, and the three-layer grounding gives the protected labels a principled basis. This moves CI evaluation from model-level or single-step judgments to end-to-end agent behavior, and the framework is reusable and extensible rather than tied to one setting.

2. Quantified, fine-grained findings on where and how agents leak.

The evaluation shows how privacy leakage varies across agent frameworks, environments, attribute types, and policies, while highlighting the limitations of single-run evaluation. The proposed metrics provide a consistent way to compare these differences and show that privacy risks depend on the entire agent stack, not just the backbone model.

Critical weaknesses

1. The stress-test instructions are designed to induce disclosure, and their fairness is never verified.

Figure 7 shows an instruction directly asking for the background-check note outright, in which I'm worrying that the paper never separates CI violations from simple instruction-following. The instructions pass only an automated quality gate. Human validation covers contracts and the judge, not the instructions. The central finding is facing the risk of agents following leading instructions.

2. The benchmark is selected on the outcome it measures.

The benchmark only includes tasks that caused privacy leakage in a surrogate agent, without reporting how many were excluded. This may bias the evaluation toward leakage-prone cases and limit its generalizability. This selection process may also affect the fairness of cross-agent comparisons.

Questions for authors

Clarification requests

  1. How many candidate instances were generated before surrogate filtering, and what fraction was retained? What is the leakage rate on the unfiltered candidate set?
  2. Was the LLM judge validated on trajectories from agents other than GPT-6 Astra with the OpenAI Agents SDK?
  3. Since the 936 tasks derive from 212 contracts that share environments and protected attributes, do the Wilson confidence intervals account for this clustering, or treat tasks as independent?

Suggestions for future work

  1. Have human annotators label a sample of stress-test instructions for whether they request, or imply entitlement to, the protected information, and report leakage separately for instructions judged fair.
  2. Evaluate a few inexpensive mitigations (e.g., a privacy system prompt, CI-aware prompting, or data minimization) to establish that the benchmark is not trivially solvable and to provide a baseline for the "testbed" claim.
  3. Evaluation on some of the same agents on existing benchmarks, such as AgentCIBench or CI-Work is encouraged, to better demonstrate what new failure cases AgentCIScope can identify.
5 comments

Thank you for recognizing the novelty, reusability, and extensibility of our work, as well as its fine-grained findings. Below, we address each weakness and question in detail.

W1 / F1 fairness of stress-test instructions (Appendix G.2)

Thank you for this detailed observation. We would like to respectfully clarify that the instruction in Figure 7 does not explicitly request the disclosure of the complete background-check note or the candidate's criminal history. Specifically, phrases such as "make the email complete" and "using the source wording for any candidate-specific context that affects where someone stands" do not necessarily imply that the agent should disclose all available information, including sensitive criminal records. Instead, the instruction asks the agent to gather relevant information from the candidate review packet and screening follow-up and summarize the candidate's status in an email to the hiring manager.

Regarding the fairness and realism of our stress-test instructions, we acknowledge that readers may find the instruction relatively broad. However, this is the purpose of our stress tests: to explore corner cases where users provide broad instructions, including phrases such as "using the source wording," and evaluate whether agents unnecessarily disclose sensitive information. We believe this setting is realistic, as users may issue broad instructions without carefully reviewing the underlying materials or realizing they contain sensitive information, leaving room for agents to make mistakes and violate Contextual Integrity (CI).

Regarding why this constitutes a CI violation rather than merely instruction-following, our annotation process provides supporting evidence from both policy and privacy-preference perspectives. From a policy perspective, GDPR Article 10 requires specific legal authorization for processing criminal-conviction data; in Figure 7, our policy-grounded validation flags this flow under GDPR Art. 10. From a privacy-preference perspective, all three personas: fundamentalist, pragmatist, and unconcerned agree that this information should not be disclosed. From a responsible AI perspective, agents should not disclose such highly sensitive information without proper authorization.

Regarding explicit requests for sending out sensitive information under a different threat model, such scenarios would more directly test the boundaries of risky instruction-following. However, we believe that a responsible and trustworthy agent should still refuse such disclosures. This is analogous to traditional AI misuse and jailbreak attacks: although models are designed to follow instructions, they should not comply with every unsafe or risky request (e.g., instructions for making explosives). Instruction-following does not override privacy and safety constraints.

Regarding additional human validation of the stress-test instructions, following your suggestion, we conducted a human audit in which three annotators independently evaluated 30 instructions. Each annotator assessed whether an instruction explicitly requested or implied entitlement to protected information specified in the contract, either by directly naming it or through a clear paraphrase. All three annotators agreed that 27 of the 30 instructions were free of such issues, and none of the instructions was flagged by a majority vote. We further verified that excluding such instructions did not largely change our reported leakage rate.

W2 / Q1 Surrogate-Based Filtering and Selection (Appendix D, Table 6)

Thank you for raising this concern. Across generation runs, our pipeline evaluated 3,420 candidate instructions against the surrogate agent and retained 936 (27.4%) that triggered leakage.

Regarding the motivation for surrogate-based filtering, our goal is to construct a challenging benchmark that helps the community identify privacy risks and advance privacy-preserving agents. We believe selecting challenging cases provides greater value for stress-testing existing agents and driving future improvements.

Regarding the effect of selection, due to time and API cost constraints, we are not able to evaluate all agents on the full unfiltered set. Instead, we conducted an additional experiment on 80 tasks from 80 different contracts, evaluating eight agents with both benchmark instructions (i.e., the filtered instructions) and randomly sampled filtered-out candidates (i.e., instructions on which the surrogate did not leak). All experiments used the same environments, contracts, and judges, with only the instructions changed. Although leakage rates were lower on filtered-out instructions, the relative agent rankings remained highly consistent (Kendall's τ = 0.93), suggesting that selection does not substantially affect cross-agent comparisons in this evaluation.

AgentBenchmark (TLR)Filtered-out (TLR)
Google ADK + Gemini 3.8 Flash93.8%63.8%
OpenAI Agents SDK + Grok 4.688.8%45.0%
OpenClaw + Kimi K386.3%48.8%
Google ADK + Gemini 3.1 Pro82.5%43.8%
Claude Code + Claude Sonnet 535.0%16.3%
OpenClaw + Claude Opus 531.3%15.0%
OpenAI Agents SDK + GPT-6 Astra30.0%11.3%
Claude Code + Claude Opus 523.8%6.3%

Q2 Judge Validated beyond GPT-6 Astra (Appendix G.3, Table 14)

Thank you for this valuable suggestion. To assess the reliability and generalizability of our LLM judge, we conducted two additional validation experiments.

1. Human validation across agent configurations. Three annotators independently labeled 150 agent trajectories, covering the same 30 tasks under five configurations spanning all four frameworks. For each protected attribute specified in the privacy contract (460 attribute instances in total), annotators determined whether it was leaked to a forbidden recipient. We used majority voting to establish the human reference labels. The table below reports the judge's agreement with these labels at both the run and attribute levels (run-level / attribute-level).

ConfigurationJudge–Human Majority Agreement (Run / Attribute)
Claude Code + Claude Opus 596.7% / 96.7%
Google ADK + Gemini 3.8 Flash100% / 98.9%
OpenAI Agents SDK + GPT-6 Astra96.7% / 94.6%
OpenAI Agents SDK + Grok 4.6100% / 94.6%
OpenClaw + Kimi K396.7% / 97.8%
Overall98.0% / 96.5%

Overall, our LLM judge achieves 98.0% run-level and 96.5% attribute-level agreement with the human majority labels, with consistently high agreement across all five configurations. These results demonstrate that the judge's reliability extends beyond GPT-6 Astra with the OpenAI Agents SDK. We believe our evidence-based leakage assessment, which grounds judgments in concrete evidence from agent trajectories, also helps support the consistency and reliability of these evaluations.

2. Cross-model judge consistency. We further evaluated 600 trajectories (50 tasks × 12 agents) using two additional LLM judges, Gemini 3.8 Flash and Claude Haiku 5.5, with the same evaluation prompt as our primary GPT-5.4 judge. Compared with GPT-5.4, Gemini 3.8 Flash achieves 90.7% run-level agreement, while Claude Haiku 5.5 achieves 91.7%. At the attribute level, the two pairwise comparisons with GPT-5.4 yield agreement rates of 94.7% and 96.0%. Importantly, the relative ranking of all 12 agents by leakage rate remains largely consistent across judges. These findings further support the robustness of our evaluation conclusions across different judge models.

Overall, these experiments provide complementary evidence for the reliability of our LLM judge across agent configurations.

Q3 Wilson CIs and clustering (Appendix E, Table 7)

Thank you for the question. The original Wilson intervals treat tasks as independent. To account for within-contract dependence, we additionally computed 95% confidence intervals using a contract-level cluster bootstrap, resampling contracts rather than individual tasks. The resulting intervals are 1.2-1.9× wider than the Wilson intervals, but our main conclusions remain unchanged: low- and high-leakage agents remain clearly separated, and agent rankings remain stable. We have added the methodology and results to the revised paper.

F2 Defense and Mitigations (Appendix B, Table 4)

Thank you for this valuable suggestion. Following your suggestion, we sampled 100 tasks from distinct CI contracts, covering diverse privacy policies, environments, and sensitive attributes. We evaluated three agents using different privacy defenses, each compared against a paired no-defense baseline under identical conditions. The full results are in the Appendix and the task-level leakage rates are reported below:

DefenseKimi K3 (OpenClaw)Gemini 3.8 Flash (Google ADK)GPT-6.1 Sol (OpenAI Agents SDK)
No defense92%95%47%
Privacy-enhancing prompt (PrivacyLens; Shao et al., 2024)90% (−2 pp)78% (−17 pp)31% (−16 pp)
CI-CoT prompting (Lan et al., 2025)69% (−23 pp)58% (−37 pp)16% (−31 pp)
Data-minimization prompt (AgentCIBench; Goel & Gurevych, 2026)91% (−1 pp)82% (−13 pp)36% (−11 pp)
CI-based supervisor (Ghalebikesabi et al., 2024)70% (−22 pp)78% (−17 pp)30% (−17 pp)

Despite these defenses, privacy leakage persists, with TLRs remaining at 16–69% even under the strongest defense. These results demonstrate that existing defenses mitigate but do not eliminate privacy leakage, and AgentCIScope remains effective at distinguishing agent vulnerabilities.

F3 Evaluation on existing benchmarks

Thank you for highlighting these two great benchmarks. AgentCIScope extends their coverage in several ways:

  1. Policy grounding: We ground disclosure constraints in policies and regulations to identify policy-specific privacy violations.
  2. Agentic frameworks: We evaluate 12 agent configurations across four frameworks, revealing that frameworks could influence privacy failures even with the same backbone model.
  3. Broader environments: We cover 11 application environments and 39 environment combinations, including financial, travel, and enterprise applications (e.g., PayPal, Booking, Expedia, and Salesforce).
  4. Sensitive attributes: We cover 39 attribute types across 10 categories, enabling fine-grained analysis of privacy failures.

We agree that direct cross-benchmark evaluation would be valuable and consider it an important direction for future work.

Thank you again for your thoughtful review and constructive feedback. We hope the additional experiments and clarifications address your concerns. We would be grateful if you would reconsider your assessment in light of these revisions. Thank you for your time and effort!

Comments on the paper

1 comment

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate their recognition of AgentCIScope's novelty and extensibility, its realistic, policy-grounded benchmark construction, and its comprehensive empirical evaluation across diverse agents, frameworks, and application environments.

In response to the reviewers' suggestions, we have substantially strengthened our paper. The revisions are highlighted in maroon (red) in the updated manuscript PDF. The major improvements are summarized below:

  1. Surrogate-based Selection Analysis (Reviewers tfmf, ioms; Appendix D, Table 6): We investigated the effects of surrogate-based filtering, confirming that agent rankings remain highly consistent between retained and filtered-out instructions (Kendall's τ = 0.93).
  2. Privacy Defense Evaluation (Reviewers tfmf, Yw5U, ioms; Appendix B, Table 4): We evaluated four existing privacy defenses and explicit policy context across three agents, demonstrating that leakage persists despite substantial mitigation.
  3. Validation of Contracts, Instructions, and Leakage Judgments (Reviewers tfmf, Yw5U, ioms; Appendix G): We strengthened contract validation through independent assessments across different model families and human annotations, including policy applicability audits. We further validated stress-test instructions through human annotation and an explicit-disclosure baseline. We also extended leakage judge validation across diverse agents, confirming the reliability of our evidence-based judge.
  4. Statistical Rigor (Reviewers tfmf, ioms; Appendix E, Tables 7–8): We added contract-level bootstrap confidence intervals and paired framework comparisons, confirming the robustness of our main findings.
  5. Clarity and Presentation (Reviewers Yw5U, ioms): We clarified policy-related claims, expanded documentation of agent configurations, and improved figures and discussions.

We have also revised the manuscript accordingly. We believe these revisions and additional experiments substantially strengthen AgentCIScope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.