acceptodds
Under review as a conference paper at ICLR 2027

Utility, Sensitive-Content Exclusion and Privacy in Graph-Based Summarization

Abstract

Summarizing sensitive documents requires useful content, control over sensitive source spans, and a precise privacy guarantee. We develop a framework that separates these requirements and connects each to testable evidence in a source-backed graph summariser. We prove selection privacy under fixed public support and bounded score sensitivity, and construct counterexamples showing why stable filtering alone cannot extend this guarantee to private extractive outputs. We establish source-level exclusion under detector coverage and provenance conditions, and derive a tight objective-headroom bound on selection gains over uniform feasible summaries. Across eight corpora and 355,200 selection records, all 135 Holm-significant exponential-versus-frontier effects are positive. Controlled disease-label interventions isolate the effect of protection; evaluations on 100 biomedical abstracts and 48 clinical cases quantify the gap between excluding detected spans and covering every sensitive span. Larger protection radii reduce observed exposure at the cost of more empty releases. A 240-cell provider-adapted generation study complements these results with coverage gains and explicit bounds for its 11 failed cells. Together, the analysis and experiments identify which evidence supports each release claim and which design choices separate content protection from differential privacy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
differential privacy, text summarization, graph-based summarization, sensitive-content exclusion, output support, detector coverage, utility analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nbu2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the safety of graph-based extractive summarization in sensitive settings. It distinguishes three goals often conflated in prior work: summary utility, sensitive-content exclusion, and differential privacy. The paper presents a modular graph-based pipeline, theoretical results, and counterexamples, and experiments on general-text and medically annotated datasets. It evaluates a provider-adapted LLM generation setup.

Critical strengths

S1. The paper clarifies that filtering sensitive text alone does not provide differential privacy. The paper supports this point with theoretical analysis and experiments on medical data. S2. The counterexamples show that when the candidate-sentence set changes with the input, the exponential mechanism's differential-privacy guarantee may no longer hold. The proposed framework separates summary utility, sensitive-content exclusion, and differential privacy, providing a useful basis for evaluating privacy claims in summarization.

Critical weaknesses

W1. The main graph-based extraction pipeline builds its candidate set directly from the input documents. It does not satisfy the fixed, public candidate-universe assumption required by Theorem 1. Thus, Theorem 1 does not establish differential privacy for the main pipeline. The proposed public-vocabulary alternative changes the release from private source sentences to public-vocabulary items and is not evaluated experimentally. The authors should either prove privacy for a data-dependent candidate set or experimentally evaluate the fixed-public-universe approach in a relevant extractive-summarization setting. W2. The Aug-PE generation study has two key limitations. First, it lacks an ablation or matched control to separate gains from graph-based candidate filtering and LLM-based rewriting. Second, when failed runs are included through worst-case bounds, the scheduled-mean bounds for ai4privacy, OpenReview, and Yelp include zero, so robust positive gains are not established for these datasets. LLM-based rewriting is a heuristic privacy measure, not a formal differential-privacy guarantee. The paper should make this distinction clear. The authors should add a fixed-candidate-sentence control to isolate the effect of LLM rewriting. For datasets whose bounds include zero, they should run more replications or temper their performance claims.

Questions for authors

Q1. The candidate set in the main pipeline changes with the input, so it does not meet Theorem 1's assumptions. Can the authors provide a privacy proof or bound for a data-dependent candidate set? If not, is Theorem 1 intended only to illustrate the need for a fixed public candidate universe, rather than to guarantee privacy for the main pipeline? Q2. Could the Aug-PE study include an ablation with fixed candidate sentences to measure the contribution of LLM rewriting separately? For ai4privacy, OpenReview, and Yelp, the scheduled-mean bounds include zero when failed runs are accounted for using worst-case bounds. Could additional replications establish whether the gains are consistently positive?

Reviewer_c3QwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the summarisation of sensitive documents and argues that utility, exclusion of sensitive content, and differential privacy are distinct guarantees, therefore each of them requires different evidence. Using a source-backed sentence-graph summariser that filters detected sensitive spans and then selects sentences wit an exponential mechanism, the authors also prove conditional selection privacy for a fixed public candidate set, show via counterexamples that this guarantee does not extend to verbatim extractive outputs, establish source-level exclusion under detector-coverage and provenance conditions, and derive an upper bound on how much any selection rule can improve the objective over uniform selection. Experiments span eight corpora, controlled disease-label interventions, two biomedical detection benchmarks (NCBI Disease and E3C), and an adapted Aug-PE generation study.

Critical strengths

S1. The problem is important, summarising sensitive documents is a practically relevant setting, and the paper's central message is useful: utility, exclusion of sensitive content, and differential privacy are distinct guarantees that require different evidence. The distinction of concepts is valuable, and practitioners can easily conflate these points.

S2. Beyond the usual mention-level precision and recall, the medical evaluations also report complete-document coverage, i.e., whether or not every sensitive span in a document is detected. This distinction matters, Theorem 3 requires full coverage within a document, so a detector with high mention-level recall can still leave every document unprotected if each one document only contains a single miss. The E3C results illustrate this gap, with 64% recall but complete coverage on only 3 of 48 cases. So reporting doc-level coverage with mention-level metrics is a rigorous choice.

S3. The experiments use paired sign-flip tests with Holm correction, which is statistically sound, fix detector thresholds before test evaluation, and provide code for replication.

Critical weaknesses

W1. The paper is framed around utility, exclusion of sensitive content, and differential privacy. However, no single system satisfies all those three requirements. The main summariser is not differentially private, because the selection directly operates over sensitive documents, so any sentence contributed by an individual can appear verbatim in the release once selected. In addition, removing that individual makes such outputs impossible, so the output support depends on the private data and Theorem 1 does not apply. Consequently, ϵ\epsilon is only nominal parameter. Moreover, values as large as 512 would provide no meaningful protection even under a valid DP mechanism. The only DP valid component is in the public-label mechanism in Appendix A, which is just toy example.

W2. The graph construction is weakly motivated, in the reported experiments, effectively reduces to a linear chain of adjacent sentences. Edges come from hashing-embedding similarity at fixed τ = 0.6, plus edges between adjacent sentences. The no-disclosure ablation (Appendix J) shows that without adjacency edges, selection often stops after a single sentence. That means, the graph is effectively a linear chain, and the protection radius basically reduces to excluding ±r neighbouring sentences. The connectivity constraint over the graph is also not justified, and the unconstrained ablation shows that it has no measurable effect.

W3. About utility, the 135 significant positive effects come from comparison against frontier random (which is not a strong baseline). Non-private greedy dominates on every corpus, and uniformly feasible selection also exceeds the exponential mechanism on five corpora. As stated in §7 (Statistics) and Appendix G, empty releases are scored as zero, whereas the 9,050 failed uniform draws are excluded as missing rather than zero-valued. Could this systematically inflate the uniform baseline?

W4. About detection, Theorem 3 guarantees that the release excludes true sensitive spans only when the detector covers every such span. Therefore, detector coverage is very important to the exclusion guarantee. However, both detectors fall far short of the author's frozen acceptance targets (80% precision and 95% lower bound on complete-document coverage, in §8.2).

W5. The presentation, especially the abstract, is difficult to follow. Too many key terms (e.g., “fixed public support,” “objective headroom,” “provenance conditions,” “frontier,”) are used without definition, and the central message (like content protection and differential privacy are distinct) appears only in the final sentence of the abstract.

Questions for authors
  1. Since the main extractive summariser is not differentially private, what release guarantee does it actually provide? Can the public-label mechanism in Appendix A be applied to, and evaluated on, a real summarization task rather than a toy example?

  2. Why are values of ϵ\epsilon as large as 512? How should readers explain ϵ\epsilon given that it is not a privacy budget?

  3. What fraction of graph's edges are similarity edges rather than adjacency edges? And would a different threshold τ yield a graph whose structure goes beyond a linear chain?

  4. What motivates the connectivity constraint, given that the unconstrained ablation shows no measurable effect on utility?

  5. Could you recompute Table 1 with consistent treatment of failures, or explain why empty releases and failed uniform draws are handled differently (scored as zero versus excluded as missing)?

  6. Beyond frontier random, does the exponential mechanism provide any advantage over a stronger baseline at comparable settings?

Reviewer_cUUTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that three properties of a released summary of sensitive documents (objective utility, exclusion of sensitive source spans, and differential privacy) are distinct claims that require distinct evidence. It studies them in a sentence-graph extractive summariser that removes a radius-r neighbourhood around detected sensitive spans, selects sentences with an exponential-mechanism facility-location rule, and renders them with provenance. The theory gives a DP-guarantee for selection over a fixed public output universe (Thm 1), a counterexample showing that survivor stability does not imply common output support (Prop 2), source exclusion under complete detector coverage (Thm 3, Prop 4), and a headroom bound relative to uniform feasible selection (Prop 5). Experiments cover eight corpora, controlled disease-label interventions, detector evaluations on NCBI Disease and E3C, and an adapted Aug-PE study.

Critical strengths

The distinction between span-level exclusion and distributional privacy is useful, and the paper is explicit that passing a label-relative verifier does not imply protection of all true sensitive spans.

The authors state their scope clearly and report interesting results, including detector coverage gaps and the cost of larger radii.

The implementation is careful: it uses exact rational sampling. The authors present the failures, datasets, models, and protocols well.

Critical weaknesses

Limited Novelty: Theorem 1 is, as the authors state, an application of exponential-mechanism composition; Theorem 3, Prop 4 and Prop 5 follow in a few lines from their premises; Theorems 7 and 8 rest on standard greedy arguments. The output-support issue in Prop 2 is the problem studied by stability-based histograms, propose-test-release, and DP set union.

Limited Evaluation:

The public-label DP construction is tested only on a three-label toy (Table 5), and the Aug-PE study uses a single nominal ε = 4 (Sec 8.3). The ε sweep in Figure 1 therefore does not show a privacy-utility trade-off.

The headline result (all 135 significant effects positive) is against frontier random only; in Table 1 the statistical superiority of the proposed method is not mentioned. For the extractive pipeline, utility is measured only by the selection objective, with no summary-quality evaluation.

Complete coverage is reached on 0/100 NCBI abstracts and 3/48 E3C cases, and Appendix D notes that 48 cases cannot certify the 95% coverage target even with perfect detection. Larger radii are costly and do not help with missed labels.

Questions for authors
  1. How does Prop 2 relate to stability-based histograms, propose-test-release, and DP set union?

  2. Failed uniform draws are treated as missing, while empty releases count as zero (Sec. 7; 9,050 missing draws per Appendix G). Are the Table 1 comparisons on matched document-cells, and do they change under a consistent treatment?

  3. Given Tables 11 and 12, how should r be chosen when detector recall is low?

  4. Can you report any summary-quality measure (e.g., ROUGE or a small human evaluation) for the released outputs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.