- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This research investigates whether LLMs encode context-dependent privacy information in their internal representations and whether such signals can be used to prevent inappropriate information disclosure and proposes the Contextual Linear Privacy Hypothesis and introduce a probe-guided attention masking framework. Specifically, a linear probe identifies restricted information spans from contextualized hidden representations, and the corresponding KV-cache positions are masked during generation. Experiments across five LLMs demonstrate promising results in contextual privacy classification and disclosure control.
Critical strengths
- This research addresses an important and practical problem by considering both training-time and inference-time privacy within a unified framework.
- The proposed framework is reasonable and relatively lightweight. Combining span-level linear probing with attention masking provides a straightforward approach to controlling contextual information disclosure without modifying the base model.
- The presentation and organization are generally strong.
Critical weaknesses
- Limited evidence supporting the Contextual Linear Privacy Hypothesis. Although the high accuracy of linear probes suggests that contextual privacy information is linearly accessible from hidden representations, this does not necessarily establish that LLMs intrinsically recognize privacy boundaries or just rely on such representations when generating responses. The proposed decomposition also does not uniquely identify a privacy-specific component. Therefore, the relationship between linear decodability and the claimed internal privacy awareness could be more carefully justified.
- Limited generalization to realistic and unstructured scenarios. The experiments rely on a single synthetic contextual integrity benchmark with structured information fields and predefined span boundaries. However, real-world LLM agents frequently process unstructured documents, conversational histories, and information whose privacy implications depend on multiple interacting spans. It remains unclear how the proposed method would generalize to such scenarios or how candidate spans would be reliably identified in practice.
- Limited evaluation of semantic information leakage. The Integrity metric may not fully capture semantic information leakage. For example, if the restricted information is "Annual salary: $120,000," the model might disclose "The employee earns approximately 120 thousand dollars per year" without reproducing the original value verbatim. Would such semantically equivalent disclosures be considered privacy violations under the current evaluation? Clarifying this would strengthen the reliability of the reported privacy protection results.
Questions for authors
- Could the authors further clarify whether the Contextual Linear Privacy Hypothesis implies only linear decodability or also suggests a causal role of privacy-related representations in generation?
- How would the proposed framework handle unstructured input, where sensitive spans are not explicitly identified or may overlap with other information?
- Does the Integrity evaluation account for semantic disclosures, including paraphrases or partial reconstruction of restricted information? as example shown in W3
2 comments
Thank you again for your valuable comments and constructive suggestions. We have carefully addressed your concerns in our rebuttal and provided additional clarifications where necessary. As the discussion period is limited, we would be most grateful if you could find a moment to look over our response and let us know whether our explanations have adequately addressed your questions. Should any concerns remain, we would be very happy to provide further clarification. Since our experiments are computationally intensive and take considerable time to complete, an early reply would be of great help in allowing us to prepare any additional results you may wish to see within the remaining discussion period. We sincerely appreciate your time and effort in reviewing our paper.
We thank the reviewer for the careful reading and the constructive comments.
W1/Q1. Limited evidence supporting the Contextual Linear Privacy Hypothesis.
Our hypothesis concerns the linear decodability of contextual disclosure labels from prefill span representations. Equation (2) provides a modeling decomposition. Whether the unmodified model uses this signal during generation is a separate causal question [1].
In our ablation (Fig. 2a and Appendix E.1), we estimated a steering direction from the difference between the allowed and restricted span centroids in prefill. We added it to one layer’s residual stream at every prompt position and decoding step. Across five models, steering achieved 12.6% and 14.8% mean Completeness at strengths of 0.5 and 1.0, respectively, close to Base (14.5%). Probe-guided attention masking achieved 32.3%.
Prefill–decode mismatch is a recurring challenge for activation steering because a direction estimated from prefill activations may transfer poorly to decode-time states [2]. Our attention-masking design sidesteps the prefill–decode transfer challenge by using the privacy signal read during prefill to guide a span-specific intervention during generation.
W2/Q2. Limited generalization to realistic and unstructured scenarios.
Please see our response to Reviewer 95oh (W2a) for how we interpret the 96.2% directional consistency and what the contextual-shift results support.
W3/Q3. Limited evaluation of semantic information leakage.
We thank the reviewer for suggesting that we check for semantic privacy leakage. Our original Table 3 followed the synthetic-benchmark evaluation pipeline of Lan et al. [3], which checks the final answer for annotated restricted keywords when scoring Integrity. Both the benchmark paper’s evaluation on this dataset and our original results left coverage of paraphrases and partial reconstructions unverified.
We re-evaluated the existing final answers underlying Table 3 using human review cross-checked with GPT-6.1 Sol. We checked whether each answer conveyed restricted information through rewording or made it recoverable in context.
The review identified previously uncounted semantic disclosures in 6 Base outputs, 14 CI-CoT outputs, and 3 Masking outputs, out of 365 outputs per method. For example, Mistral-7B disclosed a long-standing desire to buy a Tesla by saying the person had “have been eyeing this particular model for quite some time.”
We counted these outputs as Integrity violations and recomputed Completeness. Under the expanded evaluation, Masking achieved the highest Integrity and Completeness on all five models. We will document the semantic review procedure in the main text and update Table 3.
References
[1] Shao, Z., et al. (2026). DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions. Proceedings of the 43rd International Conference on Machine Learning. Paper
[2] Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021). Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. Transactions of the Association for Computational Linguistics, 9, 160–175. Paper
[3] Lan, G., et al. (2025). Contextual Integrity in LLMs via Reasoning and Reinforcement Learning. NeurIPS 2025. See §4 for the benchmark’s Integrity and Completeness definitions.