- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces a benchmark PiSAs for measuring privacy leakage when multiple users share one agentic system. Each information item is labeled with two attributes: appropriateness (should it be used for this task?) and visibility (which users may legitimately know it?). The paper measures leakage in different channels of a multi-user system, such as gathered-information summary, inter-agent messages, and memory. Results across 3 topologies (Single, Centralized, Decentralized) and 4 memory regimes (None, Private, Shared, Hybrid) show that partitioning information into multiple agents reduces both privacy violation and completeness, while shared memory increases completeness at the cost of cross-user exposure. Overall, no studied configuration achieves acceptable privacy requirements, highlighting the need for a new approach for information sharing in multi-user agent system.
Critical strengths
-
The paper is well-written and easy to follow. The two-label design of each attribute is very natural and clean in multi-user system. Keeping the two dimensions separate lets the benchmark measures inappropriate information flows among multiple parties, which reveals potential privacy risks that can't be captured in output-only measurement.
-
The evaluation covers a broad range of agent topologies, memory configurations, privacy defenses, and model backbones,which is very comprehensive and produces informative results reveals interesting privacy-utility trade-offs.
Critical weaknesses
-
Inconsistency between the benchmark's normative labels and human judgment complicate the interpretation of violation rates. Human validation suggests that the benchmark is more conservative than human annotators, potentially overestimating appropriateness violations. Higher leakage of distractors and attached private details may therefore reflect failures to follow CI, disagreement with the benchmark’s normative assumptions, or both. Testing whether this pattern persists under human-majority relabeling would strengthen the attribution to incorrect privacy judgments.
-
The simplified simulation limits external validity. Users are represented by static information, and each scenario involves one executor pursuing a single task, with no memory accumulation across tasks. It remains unclear whether the findings generalize to natural, ongoing organizational interactions.
Questions for authors
None
4 comments
Thank you for your feedback
We thank the reviewer for their thoughtful feedback. We have addressed the concerns raised and provided additional analyses and clarifications in our responses and revised manuscript. We would appreciate it if you could take another look and let us know if any concerns remain. If our responses have addressed your reservations, we would be grateful if you could consider updating your scores accordingly.
Dear Authors,
Thank you for the detailed reply and the new analysis. I would like to keep my original positive score, and strongly encourage you to highlight these points in the revision.
We sincerely thank the reviewer for their encouraging feedback and for taking the time to consider our response and additional analyses. We have incorporated the points discussed into the revised manuscript and will make sure they are clearly highlighted.
We thank the reviewer for recognizing the paper’s clear presentation, natural two-label design that captures privacy risks beyond output-only measurements, and comprehensive evaluation revealing important privacy–utility trade-offs. We address the concerns below.
Q1: Validity of appropriateness labels
We analyzed the 80 disagreements in Appendix D of the revised manuscript. They fall into two categories:
Private details attached to appropriate facts (28 cases): For example, “Taylor has an HR-mediated conflict with Sasha” contains useful information for ticket allocation but also reveals a private HR-related reason. Human annotators, evaluating this attribute in isolation, consider it appropriate because it helps determine ticket assignments. However, the task only requires knowing that Taylor and Sasha cannot work together. PiSAs labels the clean constraint appropriate and the version with private details inappropriate.
Speculative evidence (52 cases): Human annotators may consider speculative evidence appropriate because it appears relevant to the task. PiSAs deliberately takes a normative approach, grounded in Contextual Integrity (CI) and carefully researched, task-specific decision rubrics reflecting organizational procedures (Appendix A). Only information required under these rubrics is labeled appropriate; additional information that appears relevant but does not satisfy the decision criteria is labeled inappropriate. For example, in Severity Classification, one attribute states, “This rule has been generating alerts that consistently fail to correlate with actual incidents.” This may suggest a benign alert, but the task rules require a confirmed finding about the current incident and is therefore labeled inappropriate. These labeling choices aim to reduce subjective human biases and speculative reasoning when evaluating whether agents preserve CI, and thus these observed disagreements do not indicate incorrect labels.
Following the reviewer's concern, we also added a sensitivity analysis in Appendix D, Table 7 of the revised manuscript. On the same model outputs and human-annotated attributes, human labels reduce average V_appr from 74.8% to 62.9% (Single), 52.4% to 44.9% (Centralized), and 31.4% to 27.4% (Decentralized). Thus, while absolute violation rates change, substantial violations remain and the ordering across architectures is unchanged.
Q2: Generalization to Long-running interactions
We thank the reviewer for highlighting this limitation. PiSAs provide a controlled first step towards evaluating CI in multi-users systems. Our results show that even in this simplified single-task setting, current models struggle with CI, while shared memory increases cross-user exposure. Long running interactions introduce further risks as information accumulates and its appropriateness changes across tasks. CIMemories[1] also reports increasing privacy violations across tasks (0.1% to 9.6% over 40 tasks). We see extending PiSAs to persistent memory and evolving privacy norms as important future work, with our current benchmark providing a baseline and groundwork for these more complex settings.
[1] Niloofar Mireshghallah, Neal Mangaokar, Narine Kokhlikyan, et al. “CIMemories: A Compositional Benchmark for Contextual Integrity in LLMs.” International Conference on Learning Representations (ICLR), 2026.