- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DualMem, a privacy-aware memory architecture that separates generalized privacy norms and situation-specific information into public semantic and private episodic memories, represented as event-centric hypergraphs. It retrieves and grounds contextual norms to individual information items to guide LLM disclosure decisions. Experiments across three privacy benchmarks and eight LLMs demonstrate reduced privacy violations, with ablation studies examining the contributions of different components.
Critical strengths
-
Practical and Meaningful Research Problem. The paper addresses an important privacy challenge in persistent LLM assistants, with clear applications to email management, agent tool use, and personalized services. Applying Contextual Integrity (CI) to memory-based privacy reasoning is a valuable research direction.
-
Well-Structured Framework Design. The separation of generalized social norms into Public Semantic Memory and case-specific information into Private Episodic Memory is conceptually reasonable. In particular, distinguishing situation-level norm activation from item-level grounding enables more fine-grained privacy reasoning rather than applying the same privacy constraints indiscriminately to all information.
-
Broad Experimental Coverage. The paper evaluates eight LLMs of different scales and architectures across three privacy benchmarks. Table 1 provides a comprehensive comparison of privacy protection and task utility, demonstrating the effectiveness of the proposed framework across diverse settings.
Critical weaknesses
-
Unclear System-Level Privacy Threat Model. The paper does not sufficiently specify the trust boundaries of its memory architecture. In particular, it remains unclear who can access Private Episodic Memory, whether external extraction models can process sensitive user information, and how malicious or manipulated inputs might affect subsequent privacy decisions. A clearer threat model would help establish the security assumptions and practical limitations of the framework.
-
Insufficient Baselines to Isolate the Core Contributions. DualMem provides the evaluated LLM with additional contextual analysis, extracted knowledge states, and item-level normative evidence. The current baselines do not adequately distinguish the benefits of the proposed memory architecture from those of stronger preprocessing and enriched prompting. Comparisons with structured-record baselines and reasoning-based privacy approaches would strengthen the claimed contributions.
-
Insufficient Justification for the Hypergraph Representation. Although the paper emphasizes higher-order relationships in contextual privacy reasoning, the proposed method primarily relies on structured attributes and embedding-based similarity retrieval, without explicitly exploiting hypergraph topology or relational reasoning. It remains unclear whether the hypergraph representation offers advantages over simpler structured representations.
-
Oversimplified Normative Reasoning and Conflict Resolution. The method prioritizes restrictive judgments over permissive ones, even when the latter have higher semantic relevance. This may lead to overprotection or unnecessary refusal, which does not fully align with the principles of Contextual Integrity. Furthermore, Social-Chem-101 contains general social norms that may not directly correspond to information-disclosure policies. The distinction between semantic relevance and normative applicability requires further investigation.
-
Limited Evaluation of Grounding Reliability. The paper reports candidate norm coverage but does not sufficiently evaluate the precision or applicability of the final norm selected after status compression. This is particularly important when multiple conflicting norms are retrieved. Moreover, grounding quality is assessed using an LLM judge without independent human validation, raising concerns about evaluation reliability.
-
Limited Evaluation of Persistent Memory Scenarios. Although the paper is motivated by persistent LLM assistants, Private Episodic Memory is primarily constructed from static, case-specific descriptions. The evaluation does not examine memory updates, evolving information states, consent revocation, or changing access permissions across interactions. This limits the evidence supporting the framework's applicability to realistic persistent-memory systems.
Questions for authors
I appreciate the authors' effort to bring Contextual Integrity (CI) into practical LLM memory systems, and I find the proposed framework conceptually reasonable. However, as detailed in the Weaknesses section, I remain concerned that the work does not sufficiently explore the complexity of contextual privacy reasoning or the challenges of realistic privacy scenarios. In particular, the justification for the proposed architecture and its ability to handle nuanced privacy boundaries require stronger evidence. I would be happy to reconsider my rating if the authors can convincingly address these concerns.
8 comments
W3. Insufficient Justification for the Hypergraph Representation.
We agree that DualMem does not exploit hypergraph topology through message passing or graph-based reasoning. Hyperedges serve as relational units that jointly encode the attributes of a privacy event. To test whether joint encoding helps, we replaced each hyperedge with independently embedded relation-value pairs, aggregated by Norm ID before Top-5 retrieval.
| Variant | Mistral LR ↓ | Mistral LR$_h$ ↓ | Mistral Help. ↑ | Mistral Probe ↑ | Qwen LR ↓ | Qwen LR$_h$ ↓ | Qwen Help. ↑ | Qwen Probe ↑ |
|---|---|---|---|---|---|---|---|---|
| Full (DualMem) | 41.5 | 43.8 | 2.47 | 85.0 | 42.0 | 44.0 | 2.66 | 75.5 |
| w/o Joint Hyperedge Encoding | 50.5 | 52.8 | 2.42 | 74.5 | 51.0 | 53.0 | 2.52 | 60.5 |
Removing joint encoding increases LR by 9.0 pp for both models and lowers probing accuracy by 10.5 and 15.0 pp for Mistral-7B-Inst and Qwen-3-8B-Inst, respectively. Independent embedding and aggregation of relation-value pairs thus does not preserve the benefit of jointly encoding hyperedge attributes. We clarified the role of hyperedges in Sections 1-3 and added this ablation to Section 4.2 (RQ3).
W4. Oversimplified Normative Reasoning and Conflict Resolution.
We agree that the restrictive-priority rule is a conservative heuristic for selecting supporting evidence, not a principled mechanism for resolving contextual conflicts. It may cause overprotection when a less restrictive norm is more applicable. Social-Chem also provides general social norms rather than disclosure policies, so semantic similarity does not guarantee normative applicability. The selected status is contextual evidence, not a deterministic rule, and the final decision remains with the evaluated LLM. We have added these points to Appendix G.
W5. Limited Evaluation of Grounding Reliability.
We thank the reviewer for highlighting the need to evaluate the final selected norms beyond candidate coverage. We conducted an additional grounding quality evaluation using GPT-5.4-mini on 50 PrivacyLens cases covering 358 information items. We separately assessed the semantic relevance of the retrieved candidate norms and the normative applicability of the final selected supporting norms.
| Metric | Value |
|---|---|
| Number of evaluated items | 358 |
| Avg. # of Candidate Norms per Item | 4.40 |
| No-match rate | 3.07% |
| Item Recall@5 | 72.51% |
| Item Precision@5 | 82.54% |
| Supporting Norm Applicability | 89.6% |
The results show that 96.93% of information items have at least one candidate norm, with an Item Recall@5 of 72.51% and Item Precision@5 of 82.54%. Among items with an assigned supporting norm, 89.6% of the selected norms are judged applicable to their corresponding information flows. These results indicate that DualMem provides broad candidate coverage while generally selecting normatively applicable supporting evidence.
We have added these results to Section 4.2, RQ5 and provided the evaluation protocol and metric definitions to Appendix B.2 and C.2. This evaluation relies on an LLM judge without independent human validation, which remains a limitation.
W6. Limited Evaluation of Persistent Memory Scenarios.
We thank the reviewer for highlighting this important limitation. We agree that persistent-memory systems should support evolving information, memory updates, consent revocation, and changing access permissions, and our evaluation does not cover these. DualMem focuses on inference-time contextual privacy reasoning rather than the upstream construction and continuous maintenance of personal memory. These are complementary but distinct problems.
Since DualMem represents knowledge states as explicit hyperedges, updates to personal information or access permissions could in principle be reflected by modifying the affected nodes, attributes, and relations, without reconstructing the entire memory. Temporal knowledge graphs provide mechanisms for representing facts with temporal validity, while incremental KG construction supports updating knowledge graphs as underlying data sources change [1, 2].
However, reliable update mechanisms, particularly for consent revocation and evolving access permissions, require additional policies for temporal validity and consistency maintenance. We leave these to future work and state in Appendix G that dynamic memory construction and maintenance are beyond the scope of this study.
[1] Temporal Knowledge Graph Completion: A Survey, Cai et al., IJCAI, 2023,
[2] Incremental Knowledge Graph Construction from Heterogeneous Data Sources, Assche et al., Semantic Web, 2026.
Thank you for the detailed and thoughtful responses. I appreciate the authors' substantial efforts and recognize the challenges of incorporating Contextual Integrity (CI) into practical privacy mechanisms.
I believe W1, W2, and W5 have been reasonably addressed. However, I still have some concerns regarding W3 and W4.
For W3, the ablation supports joint encoding but does not fully establish the advantages of hypergraph representations over simpler structured alternatives.
For W4, since CI is central to this work, I believe its core principles deserve deeper investigation. CI concerns the appropriateness of information flows rather than simply restricting disclosure. The current restrictive-priority heuristic raises concerns about whether the framework can adequately handle conflicting norms and contextually appropriate sharing.
Regarding W6, I view dynamic memory management as a meaningful but challenging extension rather than a requirement for this work.
Overall, the rebuttal has strengthened the paper, and I appreciate the value of this research direction. However, given the remaining concerns, I will maintain my current rating.
We sincerely thank the reviewer for the constructive and detailed feedback. For clarity, all revisions in the manuscript are highlighted in blue text. We appreciate the recognition of the importance of contextual privacy, the separation of public semantic and private episodic memories, and the value of item-level grounding. The reviewer also highlights the breadth of our evaluation across eight LLMs and three privacy benchmarks. Below, we address the weaknesses (W) regarding the threat model, baseline comparisons, hypergraph representation, grounding reliability, and normative reasoning and persistent memory evaluation.
We agree that the trust boundaries were not stated explicitly. Public and Private describe the sharing scope of each memory (shared across cases vs. constructed per case), not a security mechanism, and we do not claim isolation or access-control guarantees. In our implementation, an external LLM (Claude-Sonnet-5) processes case information to construct Private Episodic Memory, so the extractor is assumed to be trusted.
DualMem targets inference-time disclosure decisions, not storage security of the extraction process. Unauthorized memory access and malicious input manipulation are not addressed. We expanded the Limitations (Appendix G), noting that practical deployments need complementary safeguards such as access control and secure memory management.
We thank the reviewer for raising this important concern. To separate the benefits of the memory architecture from those of stronger preprocessing and enriched prompting, we conducted additional experiments and expanded the ablation study.
First, we introduced CI-CoT (Contextual Integrity Chain-of-Thought), a reasoning-based baseline in which the evaluated LLM analyzes the actors, information, action, and context of the information flow using only its parametric knowledge, without external memory. The table reports averages across eight LLMs.
DualMem achieves lower average leakage than CI-CoT on all three benchmarks, with comparable or higher utility relative to CI-CoT. This suggests that explicit CI reasoning alone does not provide the same benefit as grounding reusable norms in situation-specific information.
Second, we report the extractor sensitivity experiment from the original submission (Appendix B.3, Table 10), now included in the main paper (Section 4.2, RQ4, Table 3), evaluating DualMem across different Private Episodic Memory extractors on 200 PrivacyLens cases.
Claude-Sonnet-5 uses explicit thinking in Stage A, whereas the other extractors do not. Relative to Claude-Sonnet-5, leakage and probing metrics change by at most 1.0 pp, suggesting that DualMem does not strongly depend on a particular preprocessing model.
Finally, the ablation study (Section 4.2, RQ3) isolates the contributions of Public Memory, Private Memory, structured episodic representation, and item-level grounding. Together, these experiments provide complementary evidence for the roles of explicit CI reasoning, memory construction, and the proposed architecture.