acceptodds
Under review as a conference paper at ICLR 2027

Unifying Privacy Accounting: Information Equivalence and Information Loss

14 views

Abstract

Differential privacy (DP) admits several notions, but the choice among them may affect both privacy analysis and utility. In this paper, we consider four mainstream curve-based privacy notions within a unified information-theoretic framework. For a fixed ordered pair of output distributions, we establish information equivalence among the two directional privacy profiles of -DP, the hypothesis-testing trade-off function, and the extended privacy-loss distribution. The exact R\'enyi differential privacy (RDP) curve joins this equivalence class whenever it is finite at some order greater than one. Under this mild condition, choosing among these notions changes only their semantic interpretation and computational requirements. In contrast, taking the maximum of the directional privacy profiles or compressing the RDP curve into a single zero-concentrated differential privacy (zCDP) parameter can lose information. We quantify the information loss between the exact RDP curve and its zCDP bound for standard noise mechanisms. This gap is zero for Gaussian noise but positive for Gaussian-mixture, Laplace, discrete Gaussian, and Poisson-subsampled Gaussian mechanisms. Moreover, this gap grows linearly as the number of folds of composition increases. Our information-theoretic perspective has practical consequences. At the same certified privacy level, retaining the full RDP curve rather than using zCDP reduces the required noise variance by up to for Gaussian-mixture noise in workloads comparable in size to the American Community Survey. For DP-SGD on Fashion-MNIST under Poisson subsampling, an RDP-based privacy accountant improves test accuracy by up to percentage points compared to a zCDP-based accountant under the same guarantee obtained by converting from RDP and zCDP. These results clarify when different privacy notions have equivalent information (when the gap is zero or nearly zero) and when retaining more detailed privacy information improves utility (when the gap is large).

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Differential Privacy, Privacy Accountant, DP-SGD

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VNxMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates information equivalence among different differential privacy notion and the information lost when a full RDP curve is compressed into a single zCDP parameter. For a fixed ordered pair of output distributions, the authors establish connections among directional privacy profiles, hypothesis-testing trade-off functions, privacy-loss distributions, and exact RDP curves under a regularity condition. They further analyze the resulting information gap for several noise mechanisms and under composition.

Critical strengths
  1. The paper provides a unified view of several privacy notions and clarifies the conditions for their information equivalence. The counterexamples illustrate why preserving directional information matters.

  2. The proposed zCDP–RDP gap helps explain the different accounting losses associated with Gaussian, Gaussian-mixture, and Laplace mechanisms, including their behavior under independent composition.

Critical weaknesses

I do not fully undersantd the whole manuscript, while I have following concerns.

  1. The contribution may not significant enough. The relationships among privacy profiles, PLDs, hypothesis testing, and RDP have been studied. I suggest that the manuscript demonstrates more clearly the substantive advances beyond existing results such as "Optimal accounting of differential privacy via characteristic function" and "Laplace Transform Interpretation of Differential Privacy".

  2. The applicability of the information-equivalence result may limited. Theorem 2.1 establishes equivalence for a fixed ordered pair of output distributions induced by two neighboring datasets. However, practical privacy accounting typically requires mechanism-level guarantees obtained by taking the worst case over all neighboring dataset pairs. The authors acknowledge this limitation and it restricts the theorem’s practical implications.

Questions for authors

see Weaknesses

Reviewer_AWsmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

For a fixed ordered pair of output distributions, the paper relates directional privacy profiles, the hypothesis-testing trade-off function, and the extended privacy-loss distribution. It establishes equivalence with the exact RDP curve when finite at some order greater than one, quantifies zCDP envelope slack, and studies composition and utility through noise calibration and DP-SGD experiments.

Critical strengths

The treatment of directionality, singular components, and the RDP finiteness condition is careful and makes the unifying statement precise. I do not have a substantive correctness objection to the main arguments as presented.

Critical weaknesses
  1. The conceptual advance appears too incremental. The equivalence argument is closely connected to existing moment-generating-function and Laplace-transform treatments, including Wang et al. (2019) and Chourasia et al. (2025). The envelope slack and composition behavior also appear relatively direct in this representation. I can see the value of a careful consolidation, but I am not yet convinced that the paper provides a sufficiently substantial new insight. I would like a clearer account of the specific obstacle that the new results overcome.

  2. The experiments do not establish a substantial new capability. Improved utility from the full RDP curve over a coarser zCDP summary is consistent with retaining more privacy information, including subsampling effects. The direction of this improvement is therefore unsurprising to me. What I am missing is an example where the framework enables a new accounting task or improves on existing methods that already retain detailed privacy information. That would make the practical significance much more convincing.

Questions for authors
  1. Which precise result goes beyond the transform-based treatments of Wang et al. (2019) and Chourasia et al. (2025), and how does it change what can be understood or computed?
  2. Is there an example where this framework enables or substantially simplifies an accounting task not readily handled by established RDP or privacy-loss-distribution methods?
Reviewer_ykenOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

First, this paper studies the equivalence of several curve-based privacy notions within an information-theoretic framework. Second, it quantifies the information lost when the exact RDP curve is replaced by its zCDP bound. Third, it uses experiments to examine how the choice of privacy notion affects the required noise.

Critical strengths
  • The paper studies an important question: how different privacy notions relate to one another, and how this choice affects the noise required for a given privacy guarantee.
  • The theoretical analysis is thorough.
Critical weaknesses
  • The practical value of the problem appears limited. The accountants commonly used for DP-SGD with Poisson subsampling are not zCDP-based, so the practical implications of the conclusions remain unclear.
  • The contribution of Theorem 2.1 is unclear (see Questions).
Questions for authors
  • In practice, DP-SGD rarely relies on zCDP accounting. Widely used accountants such as PRV and RDP accountants do not go through zCDP. In which application scenarios is the paper's zCDP-based analysis relevant?
  • Compared with pior work, like Theorem 3.3 in Chourasia et al. (2025) [1], what does Theorem 2.1 add?
  • It would be helpful to include a proof sketch of the main results in the main text. Without one, it is difficult to assess the technical contribution.

[1] Chourasia, Rishav, Uzair Javaid, and Biplap Sikdar. "Laplace Transform Interpretation of Differential Privacy." arXiv preprint arXiv:2411.09142 (2024).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.