acceptodds
Withdrawn from ICLR 2027

MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM agents are increasingly deployed in multi-party environments, such as group chats, handling sensitive personal data on behalf of individual users. When an agent communicates, the message reaches every group member at once. The model must therefore judge whether disclosing private information is appropriate for every recipient simultaneously, a risk that is structurally harder to control than in one-to-one conversations. Yet existing contextual privacy benchmarks focus on one-to-one settings, so multi-party privacy risks remain largely unmeasured. We introduce MuPPET (Multi-Party Privacy Exposure Testing), a benchmark for contextual privacy in multi-party conversations. Our experiments show that multi-party conversations increase leakage most for the models that appear safest in one-to-one conversations, by up to a factor of 1.9. Small open-weight models leak the most in both settings, a concern since they are often preferred for local deployment with sensitive data. Existing contextual privacy defences offer only partial protection, degrade utility, and do not resolve the underlying party-tracking problem.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy, safety, agent, benchmark, evaluation, model alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1AvUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MuPPET, a benchmark of 562 synthetic workplace conversations for evaluating contextual privacy when an assistant speaks on behalf of one user to multiple recipients. It compares disclosure in one-to-one and group settings, evaluates eight models and two prompting defenses, and uses diagnostic probes to analyze failures in participant tracking and contextual privacy reasoning.

Critical strengths
  1. The paper examines a setting in which an assistant with access to one user’s private context communicates with a group. In this setting, the same response may be appropriate for some recipients but inappropriate for others. Explicitly separating the owner’s private memories from the shared conversational context makes this concern concrete.
  2. The one-to-one versus multi-party comparison keeps the conversation and memories fixed and changes only the intended audience, which provides a clean control. The increases for several models, including a change in model ranking are informative.
  3. The paper provides detailed generation and evaluation prompts, measures utility alongside privacy, and validates both judges against human annotations. This makes the pipeline inspectable.
Critical weaknesses
  1. The leakage label records disclosure, not a validated norm violation. Each seed's sensitive attribute is simply declared "not intended for disclosure" (L1605), and any response that reveals it counts as a leak (L212–214). The judge sees only the topic, the memories and the response (App. E.1), not the conversation or who already knows. The main-setting prompt also tells the model to "use these memories to personalize responses" and gives no privacy guidance (App. G). The human validation (App. F.1) only checks whether the attribute was revealed, not whether revealing it was inappropriate. Every reported number depends on this label, so the claim to test "implicit social norms" (Table 1, L149–150) is unsupported until humans validate the appropriateness labels.
  2. The central claim that multi-party settings are harder because of party tracking is not supported. Table 2 adds an explicit note about what not to share, worded differently in the two arms (L264–269). The headline 1.9× increase is therefore not measured in the benchmark's implicit setting, where leakage is much higher (GPT-5.5: 27.4% in Table 2 vs. 49.0% in Table 3). The test uses only 5 runs, so every "significant" raw p-value is 0.03125, the smallest possible, which becomes exactly 0.05 after BH correction (App. H). Three of eight models leak less in the multi-party setting. The Sec. 5.4 probes have no prompts, ground truth or scoring criteria, and about three-quarters of GPT-5.5's leaks fall in the unexplained residual category (Fig. 3). So neither the multi-party effect nor its party-tracking explanation is established, and these are the paper's main case for a separate benchmark.
Questions for authors
  1. Why should a workplace coordination assistant access and retain a coworker's health, immigration or pregnancy information, rather than only the resulting work constraint?

  2. How is "inappropriate disclosure" defined, and on what basis? If the non-disclosure labels (L1605) are stipulated in advance, how did scenario construction ensure that disclosure is actually inappropriate in each generated context, e.g., when several group members already know the attribute (App. B)? If appropriateness depends on context, what rules let the judge tell inappropriate from legitimate disclosure, given that it sees no recipient or context information (App. E.1) and F.1 only checks whether the attribute was revealed?

  3. Why was Gemini 2.5 Pro evaluated instead of the already available Gemini 3 Pro / 3.1 Pro? Does the headline claim, which relies on Gemini 2.5 Pro's ×1.9 increase (L301–308), still hold for Gemini 3.x?

  4. What are the exact prompts, ground-truth answers and scoring rules for each of the four probes (L420–427)? Without them, Figure 3 cannot be reproduced.

Reviewer_p6bjOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed MuPPET, a benchmark for evaluating contextual privacy in multi-party conversations. Unlike previous works, the benchmark focuses on the complex scenarios with many participants in the conversations.

Critical strengths

The paper is well-motived and very relevant as the LLMs are used in multi-user interactions. Data construction pipeline is explained clearly with all the prompts used in the Appendix. Privacy and utility metrics are evaluated using LLM-as-a-judges (with the prompts in the Appendix) and validated using human annotations.

Critical weaknesses

Major Weakness:

  1. The benchmark could have limited diversity with only 53 templates. And the conversations in the dataset seems artificial, sequential (only one turn per participant) and synthetic-like status updates than any real conversation (even though generated by the LLM), for example, in the Appendix B - Conversation transcript, after the speaker William asked a question, none of the following participants acknowledged the question from William “what's the best way to report urgent issues if they pop up over the weekend?” to Mark, there is no response from Mark and the others whereas, Patrick mentioned William but simply to say “good question” and just mentioned the “”weekend protocol” need to be followed.
  2. Impact of the group size couldn’t be studied specifically for the party tracking ability of the agents since all the 562 tasks uses 20 fixed set of participants. Did the authors explore NxK setting where N and K are not fixed? Based on my understanding, only one sensitive attribute is introduced per task so it will be harder to study real world privacy violation scenarios where there may be more than one sensitive attributes shared among the team members/participants.
  3. If the prompt includes (line 268) “the note specifies which participants should not receive that information”, why is the user tracking and knowledge attribution probes were worse for most of these models when the note already describes exactly this information.

Minor Weakness:

  1. LLM judge measures both the sensitive information leak as well as the inferred sensitive information. It is unclear how the “inferred secret” is included in the benchmark.
  2. How will the baseline similar to one vs one (pairwise - one vs 19 other participants and averaged results) and one vs all compares against the one vs the last one setting considered in the paper?
  3. Since the most powerful model Gemini and GPT in Table 2 dropped in privacy metrics, what does the action-knowledge gap imply here? Line 464 says “”as the model improves, fewer leaks attributed towards the failure in the user tracking and contextual failures”. Does the model improvement means the larger models? In fact, Gemini couldn’t be considered as an improved model since the GPT model has more action-knowledge gap than the Gemini model. This may imply that these simple probes are insufficient to capture the candidate models used in this paper.
  4. Would these probes on Gemini-Pro be biased because of the Gemini-flash used for data construction?
Questions for authors
  1. Why do the llama and Gemma models leak less in the multi-party setting? Even though same sized models leaked in the same setting. The size of the model (i.e., larger implies better at user tracking) may not be the reason for the privacy leak here since these llama and Gemma models scored better in user tracking probes in Fig 3.
  2. Are the results in Table 2 Multi-party and Table 3 undefended comparable?
  3. In line 266, what does the best-case scenario imply here? Is it that saying that you are handling sensitive information to the agent makes the agents more aware of the privacy nature of the query like in the CIMemories?
  4. What is the LLM-judge used for all the experiments?
Reviewer_kbtLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a benchmark for contextual privacy in LLM agents operating in multi-party conversations, addressing a gap in prior benchmarks that primarily focus on one-to-one interactions. In multi-party settings, information appropriate for one recipient may be inappropriate for others, creating additional privacy risks.

The benchmark is constructed through a multi-stage pipeline. Each seed consists of a high-level scenario, a sensitive attribute associated with an individual, and a legitimate question that could inadvertently elicit sensitive information. The pipeline generates synthetic workers with diverse demographic and occupational attributes, selects a target user whose AI agent responds on their behalf, and uses an LLM to generate scenario blueprints and multi-party conversations. Gemini 2.5 Flash is used throughout the generation and validation pipeline, with manual review supplementing flagged cases.

The benchmark evaluates privacy through sensitive attribute leakage to participants who did not previously know the information, treating these participants as unauthorized recipients. Utility is assessed using an LLM judge. Experiments show higher privacy leakage in multi-party conversations than in one-to-one settings. The paper also evaluates existing defenses and examines the privacy–utility trade-off.

Critical strengths
  • Meaningful benchmark gap: The paper identifies an important limitation of existing contextual integrity benchmarks by extending evaluation from one-to-one to multi-party conversations, which introduce more complex contextual privacy reasoning challenges.
  • Consideration of social context: The benchmark incorporates implicit social relationships and privacy norms, which are relevant to evaluating information-sharing appropriateness.
  • Empirical findings: The experiments demonstrate the benchmark's value by showing increased privacy leakage in multi-party settings and evaluating the effectiveness of existing defenses, including their privacy–utility trade-offs.
Critical weaknesses
  • Oversimplified privacy ground truth and construct validity. My primary concern is how the benchmark establishes ground truth for the contextual appropriateness of information flows. Although the paper is motivated by contextual integrity theory, its operationalization appears to simplify privacy violations to the disclosure of predefined sensitive attributes based on whether recipients have previously received the information.

    This does not fully capture contextual integrity's emphasis on social relationships, context-dependent norms, and transmission principles when determining appropriate information flows. While the generated conversations contain rich contextual information, the appropriateness of information flows within these contexts is not systematically established or evaluated. This raises a significant concern about the benchmark's construct validity.

  • Potential bias from using the same model for generation and evaluation. Gemini 2.5 Flash is used to generate both scenario blueprints and conversations, as well as to evaluate their consistency and coherence. Using the same model for generation and validation may introduce systematic biases into the benchmark construction and quality-control process.

  • Insufficient systematic validation of scenario realism. The benchmark appears to rely largely on LLM-based quality checks, supplemented by manual review of flagged cases, rather than a systematic manual validation process. The reported checks focus primarily on seed consistency and coherence, without adequately assessing whether the generated scenarios realistically represent workplace settings. In particular, the paper does not appear to systematically control or validate the realism of the questions asked, the information flows, or the information available to agents (e.g., personal notes and other accessible materials) when making information-sharing decisions.

Questions for authors
  1. How does the benchmark establish ground truth for appropriate information flows based on the five parameters of contextual integrity (sender, recipient, subject, attribute, and transmission principle) and the information-flow norms governing the relevant social context?

  2. How do the authors address potential biases arising from using Gemini 2.5 Flash for both scenario generation and validation?

  3. Is there a systematic manual review process beyond the inspection of flagged cases? How is the realism of the generated workplace scenarios validated, particularly regarding the questions asked, information flows, and materials accessible to agents?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.