- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces PrivCert, a framework for differentially private reporting with explicit statement-support certificates. It emits only statements whose lower support bounds exceed a pre-selected threshold. The paper also reveals privacy–honesty trade-offs, and a worst-case lower bound for multi-statement certification. Experiments on synthetic tasks, other real datasets show reduced unsupported emission compared with DP text-generation baselines.
Critical strengths
- The topic the paper focuses on is very interesting and important, which separates privacy and evidential support.
- Theoretical proof are given in the paper to show that the proposed PrivCert method is near optimal. Although I did not check the proof myself, my GPT Pro said that it did not discover obvious mathematical errors within them.
Critical weaknesses
- To me, the novelty is somewhat limited. The main implementation is standard DP statistical-query answering followed by a one-sided confidence bound and thresholding. Its value is primarily the reporting interface and application.
- Further more, I do not see any case demonstration of what final supported claims are drawn with the proposed method PrivCert and what claims are not supported. Without these demonstrations, readers cannot grab an intuition on what final claims can be produced.
Questions for authors
- Some of the implementation details are missing in the paper, including the complete schemas, claim parser, some dataset construction details, baseline configurations, and exact SVT/Gaussian calibration.
- Is there any tighter privacy budget composition than sampling summing them up (line 260)?
Thank you for your constructive review and for recognizing the importance of distinguishing privacy guarantees from evidential support.
The central question of PRIVCERT is not simply how to generate differentially private text, but what a private report can actually substantiate about its underlying data. DP limits the influence of individual records, but does not automatically establish whether a reported statement is supported by those records. PRIVCERT addresses this evidence gap by formalizing explicit support certificates and emit-or-abstain decisions. Its contribution lies in characterizing the guarantees and fundamental trade-offs of this reporting contract, rather than introducing a new DP primitive.
W2: Concrete examples of emitted and abstained claims
Consider two illustrative candidates under a fixed support predicate, with reporting threshold θ=0.10 and certification margin τ=0.08.
These are schematic examples, not dataset-specific outputs. The essential distinction is that a statement is emitted only when its privacy-preserving lower certificate meets the reporting threshold.
The experiments also demonstrate nontrivial coverage. Across five runs, the tag-template proposer produces 404, 220, and 749 emission events on TAB, WildChat, and Yelp, respectively (Table 5), with FalseEmission of 0.010, 0.014, and 0.000 under the declared support semantics.
Q1: Implementation details
Appendix C.2 specifies the frozen DeBERTa-v3 NLI support model, premise–hypothesis construction, 384-token input limit, entailment-based support predicate, and privacy accounting. Appendix C.1 describes the experimental protocol.
We agree that complete proposal schemas, claim-parser details, and some baseline configurations could be documented more fully for exact reproducibility. These details concern the empirical implementation; the mathematical guarantees are stated in terms of explicit support and privacy assumptions.
Q2: Tighter privacy composition
We agree that basic sequential composition is conservative. However, Section 6.3 already evaluates alternative PRIVCERT implementations using contribution-bounded joint release (Hist), sparse-vector selection (SVT), and Gaussian certification.
More fundamentally, Theorem 5 shows that the dependence on m/(nεtot) is unavoidable in the specified worst-case overlapping-support regime. Thus, the privacy cost of certifying many statements is not merely an artifact of basic composition.
This is precisely why evidential support deserves separate treatment: making private reports trustworthy requires understanding not only their privacy guarantees, but also the attainable strength and coverage of their evidence.