- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies the safety of graph-based extractive summarization in sensitive settings. It distinguishes three goals often conflated in prior work: summary utility, sensitive-content exclusion, and differential privacy. The paper presents a modular graph-based pipeline, theoretical results, and counterexamples, and experiments on general-text and medically annotated datasets. It evaluates a provider-adapted LLM generation setup.
Critical strengths
S1. The paper clarifies that filtering sensitive text alone does not provide differential privacy. The paper supports this point with theoretical analysis and experiments on medical data. S2. The counterexamples show that when the candidate-sentence set changes with the input, the exponential mechanism's differential-privacy guarantee may no longer hold. The proposed framework separates summary utility, sensitive-content exclusion, and differential privacy, providing a useful basis for evaluating privacy claims in summarization.
Critical weaknesses
W1. The main graph-based extraction pipeline builds its candidate set directly from the input documents. It does not satisfy the fixed, public candidate-universe assumption required by Theorem 1. Thus, Theorem 1 does not establish differential privacy for the main pipeline. The proposed public-vocabulary alternative changes the release from private source sentences to public-vocabulary items and is not evaluated experimentally. The authors should either prove privacy for a data-dependent candidate set or experimentally evaluate the fixed-public-universe approach in a relevant extractive-summarization setting. W2. The Aug-PE generation study has two key limitations. First, it lacks an ablation or matched control to separate gains from graph-based candidate filtering and LLM-based rewriting. Second, when failed runs are included through worst-case bounds, the scheduled-mean bounds for ai4privacy, OpenReview, and Yelp include zero, so robust positive gains are not established for these datasets. LLM-based rewriting is a heuristic privacy measure, not a formal differential-privacy guarantee. The paper should make this distinction clear. The authors should add a fixed-candidate-sentence control to isolate the effect of LLM rewriting. For datasets whose bounds include zero, they should run more replications or temper their performance claims.
Questions for authors
Q1. The candidate set in the main pipeline changes with the input, so it does not meet Theorem 1's assumptions. Can the authors provide a privacy proof or bound for a data-dependent candidate set? If not, is Theorem 1 intended only to illustrate the need for a fixed public candidate universe, rather than to guarantee privacy for the main pipeline? Q2. Could the Aug-PE study include an ablation with fixed candidate sentences to measure the contribution of LLM rewriting separately? For ai4privacy, OpenReview, and Yelp, the scheduled-mean bounds include zero when failed runs are accounted for using worst-case bounds. Could additional replications establish whether the gains are consistently positive?