acceptodds
Under review as a conference paper at ICLR 2027

Auditing Information Disclosure During Large-Scale Gradient-Based Training via Gradient Uniqueness

Abstract

Auditing information disclosure across every datapoint during the training of LLMs is challenging. We propose a principled, attack-agnostic approach that uses mutual information to measure what the final model reveals about a datapoint's training membership. We show that this final-model disclosure is upper bounded by the sum of per-iteration gradient disclosures and that, under a reasonable set of assumptions, these gradient disclosures increase with Gradient Uniqueness (GNQ), which measures how distinguishable a datapoint's gradient is relative to other gradients in the batch. While naively computing GNQ requires forming and inverting a matrix for every datapoint (for a model with parameters), we introduce Batch-Space Ghost (BS-Ghost). This efficient algorithm performs all computations in a much smaller batch space and uses ghost kernels to compute GNQ "in-run" for every datapoint in the training corpus, with minimal computational and memory overhead. Our experiments show the following: (i) GNQ predicts MIA vulnerability without the need for shadow models. (ii) Beyond membership disclosure, GNQ predicts the success of reconstruction attacks. (iii) GNQ-guided removal and retraining identify datapoints that causally contribute to disclosure. (iv) GNQ outperforms counterfactual memorization in text extraction and common-knowledge discrimination without the need for additional model training. (v) For data attribution, GNQ-guided filtering reduces emergent misalignment in Qwen2.5-7B more than baselines. Further, GNQ attributes 1000 datapoints in 17 seconds—roughly faster than the baselines. (vi) GNQ explains how training choices affect training-set disclosure and captures how per-datapoint disclosure emerges during training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy auditing, information disclosure, membership inference, large language models, gradient descent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VycYOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Gradient Uniqueness (GNQ), a per-datapoint score for auditing how much a trained model reveals about its training data. GNQ measures how different a datapoint’s gradient is from the other gradients in the training batch and links to an information-theoretic measure of membership disclosure. An efficient algorithm, BS-Ghost, is given, which computes GNQ for every datapoint during training with low overhead. Experiments show that GNQ agrees with membership inference, reconstruction, and text extraction results, and that it can also be used for data attribution.

Critical strengths

The paper studies an important and practical problem: auditing how much a trained model discloses about each training datapoint. The proposed approach aims to do this during normal training at low cost, without shadow models or changes to the training data. If the claims hold, this could be a useful tool. The experiments also cover a wide range of settings and appear to point in a consistent direction.

Critical weaknesses

Section 3, which presents the core methodology, lacks sufficient clarity and justification, making the paper’s main claims difficult to assess. The key theoretical results and their assumptions are moved to the appendix, and the main text mostly states conclusions without explaining them. GNQ is defined in Eq. 1 with no intuition for why it should measure disclosure. The steps that follow, including the bound in Eq. (2), the Gaussian model of the background gradients, and the final link to GNQ, are given with little motivation. The reader therefore cannot see why each step holds or how strong the assumptions are. The 2D example in Section 4 comes too late. It also only shows that GNQ is an outlier score, not why that relates to information disclosure. As a result, it is hard to tell whether GNQ really bounds disclosure or is just a heuristic score that happens to correlate with attacks.

Questions for authors
  1. Can you give an intuitive explanation of the construction of GNQ in eq (1)?
  2. Can you state the key assumptions clearly in the main text and justify why they are reasonable and how they play a role in deriving the theorems?
2 comments

W1: “Section 3, which presents the core methodology, lacks sufficient clarity and justification, making the paper's main claims difficult to assess. The key theoretical results and their assumptions are moved to the appendix, and the main text mostly states conclusions without explaining them. GNQ is defined in Eq. 1 with no intuition for why it should measure disclosure. The steps that follow, including the bound in Eq. (2), the Gaussian model of the background gradients, and the final link to GNQ, are given with little motivation. The reader therefore cannot see why each step holds or how strong the assumptions are. The 2D example in Section 4 comes too late. It also only shows that GNQ is an outlier score, not why that relates to information disclosure. As a result, it is hard to tell whether GNQ really bounds disclosure or is just a heuristic score that happens to correlate with attacks.”

Q1: “Can you give an intuitive explanation of the construction of GNQ in eq (1)?”

Q2: “Can you state the key assumptions clearly in the main text and justify why they are reasonable and how they play a role in deriving the theorems?”

We thank the reviewer for these comments. To address them, we will make the six changes listed below. Note that none of these require changing or updating any technical results. Rather, they involve changing the order of statements, adding explanations, or clearly labeling the assumptions.

Q1: Intuition for the construction of GNQ

Change 1 (reorder Section 3 so GNQ will be derived). The new order will be: (i) give the threat model; (ii) give Theorem 1, which shows that final-model disclosure is bounded by the sum of per-iteration gradient disclosures; (iii) describe how to measure per-iteration gradient disclosure, in which the target (which the adversary is trying to detect) contributes a known shift gij/Bg_{ij}/B hidden in the unknown background WiW_i; (iv) give the observation that the detectability of this shift is gij⊤S−1gijg_{ij}^\top S^{-1} g_{ij}; (v) give Definition 1, followed by the requested intuition paragraph below; (vi) give Theorem 2.

Change 2 (intuition paragraph after Definition 1). We will add:

An adversary is trying to determine whether a target datapoint has been used during gradient descent. GNQij\mathit{GNQ}_{ij} is the squared length of the target datapoint's gradient, measured relative to the spread of the other datapoints' gradients in each direction. “Measured relative to the spread” is crucial, as directions in which the other datapoints' gradients vary widely tend to hide the target's contribution, whereas directions in which they have little or no component make it easy to find. Formally, given that djd_j is sampled into batch ii, the adversary must distinguish between two Gaussians: N(mi,Ci)\mathcal N(m_i, C_i) (the target is absent) and N(mi+gij/B,,Ci)\mathcal N(m_i + g_{ij}/B,\\, C_i) (the target is present), with Ci=qB2SC_i = \tfrac{q}{B^2}S. Two Gaussians with a common covariance are distinguishable only through their squared Mahalanobis distance,

so GNQij\mathit{GNQ}_{ij} is, up to the factor 1/q1/q, the signal-to-noise ratio of the membership test at iteration ii.

Change 3 (move Figure 1b and connect “outlier” to “disclosure”). Figure 1(b) and its description will be moved from Section 4 into Section 3, next to the intuition paragraph. We will explain that the ellipse shows the spread of the other gradients. This is the background distribution where the target datapoint may (or may not) be hidden. A gradient far outside the ellipse cannot be explained by that background, so its presence in the batch is easy to detect. This addresses the concern that the 2D example shows only an outlier score.

Q2: Assumptions in the main text

Change 4 (assumption block). Section 3 will contain the following labeled assumptions, each with its use and justification.

AssumptionUsed inJustification
A1Tj∼Bernoulli(π)T_j \sim \mathrm{Bernoulli}(\pi); the other Nt−1N_t-1 datapoints are i.i.d. from P\mathcal P, independent of TjT_j.Thm. 1, 2Problem setup using the standard membership-inference prior.
A2θi+1\theta_{i+1} is a deterministic function of (θi,g^i)(\theta_i, \widehat g_i), and θ0\theta_0 is independent of the data.Thm. 1Holds exactly for mini-batch SGD.
A3Batch masks MikM_{ik} are independent Bernoulli(q)\mathrm{Bernoulli}(q), q=B/Ntq=B/N_t, independent of the data and of earlier masks.Thm. 2Idealization (Poisson sampling) of shuffled fixed-size batches.
A4Given θi=θ\theta_i=\theta, the background WiW_i is Gaussian with mean mim_i and covariance Ci=qB2SC_i=\tfrac{q}{B^2}S.Thm. 2Modeling assumption. WiW_i is a sum of gradient contributions from many datapoints drawn independently from P\mathcal P. If these contributions remain approximately independent once θi\theta_i is fixed, the central limit theorem applies and supports this assumption.
A5Wi⊥(Tj,Mij)∣θi=θW_i \perp (T_j, M_{ij}) \mid \theta_i=\theta.Thm. 2Modeling assumption. Since θi\theta_i was trained on the same data set constituting the background, knowing θi\theta_i makes the background distribution weakly dependent on whether the target was used. We assume this dependence is negligible.
A6SS is the regularized uncentered second moment of the non-target gradients, estimated from the current batch.ComputationApproximation. The true covariance of the background is not available, so the auditor estimates it from the current batch.

Change 5 (precision regarding what is proved unconditionally and what requires modeling assumptions). Immediately after Eq. (3) we will state:

Theorem 1 is exact given A1 and A2. Theorem 2 is an equality given the predictive model implied by A3 through A5. Eq. (3) bounds disclosure given A3 through A5, but it may not hold if the model is unreasonable. The experiments in Section 6 test whether the model-based score tracks disclosure by actual attacks.

We believe that the assumptions are reasonable, but we will be clear about this possible limitation in the body of the paper.

Change 6 (closed-form bound in the main text). The derivation in Appendix A.3 will be stated in Section 3 as a corollary, so that the reader sees an explicit bound:

Corollary 1.

Under A1–A5, with information measured in nats,

I(Tj;θNr);≤;q2∑i=0Nr−1E![ln⁡!(1+GNQij4q)].I(T_j;\theta_{N_r}) \\;\le\\; \frac{q}{2}\sum_{i=0}^{N_r-1} \mathbb E\\!\left[\ln\\!\left(1+\frac{\mathit{GNQ}_{ij}}{4q}\right)\right].

We believe that with these changes, the paper can be significantly improved. We'd appreciate the reviewer's thoughts.

Reviewer_9ggVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

The paper proposes Gradient Uniqueness (GNQ), a per-example score for auditing information disclosure during training. It connects GNQ to membership information under modeling assumptions and introduces BS-Ghost for efficient computation without explicitly storing per-example gradients. Experiments examine membership inference, reconstruction, text extraction, data filtering, and behavior attribution.

Critical strengths
  1. The paper addresses a useful practical problem: identifying potentially vulnerable training examples during an existing training run. The connection between gradient geometry and membership disclosure offers an interesting motivation.

  2. BS-Ghost provides a practical way to compute the proposed score. The numerical consistency check, overhead measurements, and association with LiRA detection rates provide encouraging initial evidence.

Critical weaknesses
  1. The baseline comparisons are not yet sufficient to establish GNQ’s added value. For example, the membership and reconstruction experiments show correlations with attack outcomes, but do not compare GNQ with simpler scores such as loss or gradient norms, or related self-influence methods. The removal experiments also lack matched random-removal controls. These comparisons would help determine whether the proposed score offers a meaningful advantage.

  2. The primary research objective could be stated more clearly. The theory concerns membership information, while the experiments also address text extraction, common-knowledge discrimination, and behavior attribution. These are related but distinct objectives, and it is sometimes unclear which results validate the central claim and which are exploratory extensions. A more focused framing would make the contribution and its supporting evidence easier to assess.

Questions for authors
  1. Do you have comparisons with loss-based, gradient-norm, or self-influence scores under the same evaluation protocol, or matched random-removal results? Such evidence could materially strengthen the empirical case.

  2. Which task is intended as the primary validation of GNQ? Could you clarify how the other evaluations support that objective or serve as exploratory applications?

2 comments

W1: “The baseline comparisons are not yet sufficient to establish GNQ's added value. For example, the membership and reconstruction experiments show correlations with attack outcomes, but do not compare GNQ with simpler scores such as loss or gradient norms, or related self-influence methods. The removal experiments also lack matched random-removal controls. These comparisons would help determine whether the proposed score offers a meaningful advantage.”

Q1: “Do you have comparisons with loss-based, gradient-norm, or self-influence scores under the same evaluation protocol, or matched random-removal results? Such evidence could materially strengthen the empirical case.”

We thank the reviewer for requesting these comparisons. We did the requested comparisons and we observe that GNQ indeed offers a meaningful advantage. We demonstrate this using the Text Extraction experiment (Figure 3 in the paper is updated with these new results) and we are also representing the same plot here using the following table which is also added to the paper as Table 6:

Selection204080
GNQ203564
Counterfactual71941
Loss71632
Gradient Norm (Gradient Magnitude)71527
Random41027
CSG (token embeddings)4927
TracInCP (self-influence)246

Table 6: Number of exactly recovered sentences by selection method. Inspection budgets of 20, 40, and 80 correspond to 5%, 10%, and 20% of the dataset (n=400n=400). Random reports selection trial 1.

  • Please note that CSG uses gradient with respect to the input not the weights and they did not demonstrate their method on LLMs. We adapted the same implementation they published in their paper to text/LLMs by taking gradients with respect to input-token embeddings.

  • Why GNQ can provide additional value over other training-time signals: Section 4 and Figure 1 illustrate how GNQ accounts for a sample's gradient direction relative to the surrounding gradient distribution. Loss and gradient magnitude alone do not capture this directional relationship, so examples with similar scalar scores can have different GNQ values.

W2: “The primary research objective could be stated more clearly. The theory concerns membership information, while the experiments also address text extraction, common-knowledge discrimination, and behavior attribution. These are related but distinct objectives, and it is sometimes unclear which results validate the central claim and which are exploratory extensions. A more focused framing would make the contribution and its supporting evidence easier to assess.”

Q2: “Which task is intended as the primary validation of GNQ? Could you clarify how the other evaluations support that objective or serve as exploratory applications?”

The reviewer is correct that the theory behind GNQ concerns membership disclosure. Three sets of experiments test this directly. GNQ predicts the success of the LiRA membership inference attack (Figure 2), removing high-GNQ datapoints and retraining reduces membership inference to chance (Figure 5), and GNQ tracks membership inference success as training choices vary (Figures 10 and 13). These are the primary validation of GNQ.

However, GNQ may be more widely useful. For a fixed target djd_j, any attack is a function of the model, so its output can carry no more information about TjT_j than the model does. If that information is small, the attack’s output can change very little depending on whether djd_j was used, and so whatever the attack recovers cannot be attributed to djd_j’s inclusion in training. This motivates the experiments on image reconstruction (Figures 4 and 12), text extraction (Figure 3), and common-knowledge discrimination (Figures 6 and 7), which test whether GNQ is also informative for these attacks.

The data attribution experiments (Figures 8 and 9) are an exploratory application. We will revise the introduction and Section 6 to state this three-way distinction explicitly.

Thank you for the response and the additional experiments. I find GNQ an original approach to auditing, and the new results have eased my concerns about the evaluation.

My remaining concern is about how the paper introduces the problem. The introduction discusses prior auditing methods, but as a reader I had to piece together how those methods work, what they already do well, and which limitation motivates GNQ. The abstract and introduction devote considerable space to individual experimental findings and detailed design goals. For readers less familiar with the literature, these details arrive before the research gap is fully clear.

I would encourage the authors to give the research gap more prominence in the abstract and introduction. A brief explanation of what existing audits can do and what remains difficult would help readers less familiar with the area understand the motivation for GNQ.

Overall, I am raising my recommendation to 3.

Reviewer_pEVMOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how model trainers can audit potential information disclosure of individual training datapoints from the final model in an attack-agnostic way. The authors measure how distinguishable each datapoint’s gradient is from the gradients of other training samples, and refer to this quantity as Gradient Uniqueness (GNQ). They connect GNQ to information disclosure through a mutual-information-based analysis, motivating GNQ as a training-time signal for identifying datapoints that may be more vulnerable to leakage. To make GNQ practical at scale, they introduce BS-Ghost, which computes GNQ efficiently in batch space without explicitly computing per-example gradients. Experiments evaluate GNQ across membership inference, model inversion, removal and retraining, text extraction, common knowledge discrimination, and data attribution, while showing that it can be computed with low overhead during training.

Critical strengths
  1. The paper studies an important privacy auditing problem, and the idea of gradient uniqueness is novel and interesting.
  2. The paper is ambitious in targeting four practical requirements at once, including attack-agnostic auditing, coverage of all training datapoints, auditing without modifying the training protocol, and accounting for common knowledge.
  3. The evaluation is extensive, covering membership inference, reconstruction, data removal, text extraction, and attribution across different datasets and model scales.
Critical weaknesses
  1. The batch-level approximation to dataset-level GNQ is less convincing, and its sensitivity is unclear. While BS-Ghost makes the computation practical by replacing dataset-level gradient uniqueness with mini-batch-level uniqueness, it raises a concern that the score may be sensitive to batch composition, especially for small batches where typical datapoints may appear unique simply because similar gradients are absent. It would be useful to directly compare GNQ rankings across different random batch compositions and different batch sizes.
  2. The empirical advantage of GNQ over simpler or existing training-time signals is not sufficiently clear. As the model trainer, the auditor has access to cheap signals that may serve a similar purpose while being equally attack-agnostic and computable in-run, such as per-example loss, gradient magnitude, and CSG [A]. It is unclear how GNQ performs compared with these baselines, e.g., in the LiRA-based MIA and removal-and-retraining experiments, and what additional value GNQ provides.
  3. The practical meaning of high GNQ is unclear as an operational signal. GNQ is mainly used as a relative ranking metric, but a high rank does not by itself indicate that the absolute level of disclosure is large enough to warrant intervention. It is therefore unclear how an auditor should choose a threshold or removal budget, and whether GNQ values are comparable across datasets and training settings.

[A] Ravikumar et al. Memorization Through the Lens of Sample Gradients. ICLR 2026.

Questions for authors
  1. The Gaussian predictive model used in Theorem 2 is an approximation. How accurate is this approximation in practice? It would be useful to empirically compare the observed background-gradient distribution with the fitted Gaussian.
  2. The experiments aggregate per-iteration GNQ in different ways, including cumulative GNQ (line 403, MIA), maximum GNQ (line 424, Text Extraction), and average GNQ across checkpoints (line 478, disclosure differences in AG-News training). What motivates these different aggregation choices, and could different aggregation statistics change the ranking of datapoints?
6 comments

We thank the reviewer for recognizing the novelty of gradient uniqueness, and for acknowledging how our evaluation is extensive, covering membership inference, reconstruction, data removal, text extraction, and attribution across different datasets and model scales

W1: “The batch-level approximation to dataset-level GNQ is less convincing, and its sensitivity is unclear. While BS-Ghost makes the computation practical by replacing dataset-level gradient uniqueness with mini-batch-level uniqueness, it raises a concern that the score may be sensitive to batch composition, especially for small batches where typical datapoints may appear unique simply because similar gradients are absent. It would be useful to directly compare GNQ rankings across different random batch compositions and different batch sizes.”

We did the requested comparisons. We fix a model checkpoint, and we compare GNQ rankings across different random batch compositions, different batch sizes, and the dataset-level GNQ compared to the batch-level approximation. We found that GNQ rankings strongly agree across different batch compositions and different batch sizes, and that larger batches agree more closely with the full-dataset ranking. We tested batch sizes 8, 16, 32, and 64, while the dataset size is 400 datapoints. Below, all correlations are Spearman correlations across all 400 datapoints.

Batch Composition: For each batch size, we shuffled the datapoints five times. Each shuffle produced a ranking of all 400 datapoints. We averaged the correlations between the ten pairs of rankings.

Batch SizeCorrelation
80.8903
160.8639
320.8730
640.8805

Batch Size: We take the same shuffled list of 400 datapoints:

  • Divide it into batches of 8, calculate GNQ, and rank all 400.
  • Divide it into batches of 16, calculate GNQ, and rank all 400.
  • Compare those two rankings.

We repeated this for five shuffled lists and averaged the correlations.

Batch SizeCorrelation
8 vs 160.9375
8 vs 320.8931
8 vs 640.8639
16 vs 320.9316
16 vs 640.8992
32 vs 640.9580

Batch-level GNQ vs dataset-level GNQ: We calculated a reference ranking using all 400 datapoints together, then compared each mini-batch-derived ranking against it. We averages the correlations over five shuffles.

Batch SizeCorrelation
80.8243
160.8426
320.8561
640.8703

W2: “The empirical advantage of GNQ over simpler or existing training-time signals is not sufficiently clear. As the model trainer, the auditor has access to cheap signals that may serve a similar purpose while being equally attack-agnostic and computable in-run, such as per-example loss, gradient magnitude, and CSG [A]. It is unclear how GNQ performs compared with these baselines, e.g., in the LiRA-based MIA and removal-and-retraining experiments, and what additional value GNQ provides.”

We did the requested comparisons and we observe that GNQ indeed offers a meaningful advantage. We demonstrate this using the Text Extraction experiment (Figure 3 in the paper is updated with these new results) and we are also representing the same plot here using the following table which is also added to the paper as Table 6:

Selection204080
GNQ203564
Counterfactual71941
Loss71632
Gradient Norm (Gradient Magnitude)71527
Random41027
CSG (token embeddings)4927
TracInCP (self-influence)246

Table 6: Number of exactly recovered sentences by selection method. Inspection budgets of 20, 40, and 80 correspond to 5%, 10%, and 20% of the dataset (n=400n=400). Random reports selection trial 1.

  • Please note that CSG uses gradient with respect to the input not the weights and they did not demonstrate their method on LLMs. We adapted the same implementation they published in their paper to text/LLMs by taking gradients with respect to input-token embeddings.

  • Why GNQ can provide additional value over other training-time signals: Section 4 and Figure 1 illustrate how GNQ accounts for a sample's gradient direction relative to the surrounding gradient distribution. Loss and gradient magnitude alone do not capture this directional relationship, so examples with similar scalar scores can have different GNQ values.

Q1: “The Gaussian predictive model used in Theorem 2 is an approximation. How accurate is this approximation in practice? It would be useful to empirically compare the observed background-gradient distribution with the fitted Gaussian.”

In practice, LLM gradients are empirically observed to resemble a Gaussian distribution. For example, the study in [1] looked at the histogram of the gradient values for various weights throughout the trained network on dataset examples from common crawl, and they found that gradients are highly Gaussian with 0 mean and very few consistent outliers at low or high values.

[1] Beren and Eric Winsor “Basic Facts about Language Model Internals” https://www.alignmentforum.org/posts/PDLfpRwSynu73mxGw/basic-facts-about-language-model-internals-1

Q2: “The experiments aggregate per-iteration GNQ in different ways, including cumulative GNQ (line 403, MIA), maximum GNQ (line 424, Text Extraction), and average GNQ across checkpoints (line 478, disclosure differences in AG-News training). What motivates these different aggregation choices, and could different aggregation statistics change the ranking of datapoints?”

Cumulative GNQ tracks repeated contributions over training and is the aggregation most closely related to Theorem 1. Membership auditing is our primary validation experiments for the theory, that's why we used it with MIA as you pointed out.

Maximum GNQ highlights whether a datapoint was highly distinctive at any observed step. Our primary motivation to use it for Memorization auditing (i.e. Text Extraction as you pointed) is due to the seminal work of Feldman “Does Learning Require Memorization? A Short Tale about a Long Tail” which we cited in our paper. Feldman proved that datapoints that fall at the long tails of natural data distributions (i.e. more distinctive datapoints) are the ones that has higher chance being memorized. Furthermore, looking deeper at the functional form of GNQ, it looks very similar to Mahalanobis Distance (MD) but in the gradient space. MD is known to be a good outlier detection metric, and we believe this is a good reason why GNQ is able to do good in memorization auditing; because it captures this long tail.

We used average GNQ as just a summary statistic. Because we wanted to use one single number to compare between different datapoints and that's why we used the average as a typical summary statistic.

Could different aggregation statistic change the ranking of datapoints? Yes, and that's why we used Maximum GNQ for memorization auditing while using cumulative GNQ for membership auditing and we explained why we did so in the previous paragraphs. However, It is worth noting that in this experiment that you pointed (AG-News training), because every datapoint is evaluated at the same 11 checkpoints, the average is exactly a constant rescaling of the cumulative. These two statistics therefore produce identical rankings in this experiment.

W1: “The practical meaning of high GNQ is unclear as an operational signal. GNQ is mainly used as a relative ranking metric, but a high rank does not by itself indicate that the absolute level of disclosure is large enough to warrant intervention. It is therefore unclear how an auditor should choose a threshold or removal budget, and whether GNQ values are comparable across datasets and training settings.”

We strongly agree with your sentence “GNQ is mainly used as a relative ranking metric, but a high rank does not by itself indicate that the absolute level of disclosure is large enough to warrant intervention.”. That's why we mentioned something along your thoughts in the last paragraph of Section 3: “Note that the bound is one-sided, so small GNQ certifies small disclosure under the model. Large GNQ means that the data point might be at risk of disclosure (developing a two-sided bound would require taking into account possible forgetting, which is beyond the scope of this paper).”

Even though, as you stated and we agree, that we mainly used GNQ as a relative ranking metric, the practical meaning of high GNQ (i.e., absolute values of GNQ) can be seen by just substituting the GNQ values to compute the upper-bound in Equation 3. We will show that now:

We remind the reader that Equation 3 has the form:

I(Tj;θNr)≤∑i=0Nr−1E[f(GNQij)]I(T_j;\theta_{N_r}) \le \sum_{i=0}^{N_r-1}\mathbb{E}[f(GNQ_{ij})]

We also remind the reader that we stated in the paper that “The auditor observes one trajectory and its GNQ scores, so the sum along that trajectory, over the iterations at which Algorithm 1 produces a score for djd_j, is a single sample of the quantity being averaged over, and so it is a natural plug-in estimator.” That makes this bound has the form:

I(Tj;θNr)≤∑i=0Nr−1f(GNQij)I(T_j;\theta_{N_r}) \le \sum_{i=0}^{N_r-1} f(GNQ_{ij})

It can be shown that (derivation is added to the paper as Appendix A.3):

f(GNQij):=12Nrlog⁡(1+ρ(1−ρ)GNQijq)f(GNQ_{ij}):=\frac{1}{2N_r}\log\left(1+\frac{\rho(1-\rho)GNQ_{ij}}{q}\right)

Where: q=B/Ntq=B/N_t and 0<ρ<10 < \rho < 1.

So here is how an auditor would use GNQ as an operational signal:

The adversary is trying to know TjT_j. This is a one-bit of information—i.e., the maximum uncertainty the adversary has about the value of TjT_j (a.k.a the max entropy H(Tj)H(T_j)) is equal 1 bit. Hence, as the information disclosure I(Tj;θNr)I(T_j;\theta_{N_r}) approaches 1 as the adversary will be able to know the value of TjT_j with high certainty. This means as the upper-bound gets smaller, the auditor gets more confident that the absolute level of disclosure is minimal. GNQ controls this bound. Given that the auditor knows qq and NrN_r, and they can choose ρ=1/2\rho=1/2 which gives the maximum ρ(1−ρ)=1/4\rho(1-\rho)=1/4 which computes worst-case disclosure, plugging the GNQijGNQ_{ij} values computed at each iteration ii, the auditor now has

I(Tj;θNr)≤Kbits.I(T_j;\theta_{N_r}) \le K\quad \textit{bits}.

So an auditor can specify a threshold or removal budget on the maximum acceptable KK, let's call it KthresholdK_{threshold}. That informs them which datapoints they need to remove; the datapoints for which the values of GNQijGNQ_{ij} make the right hand side larger than KthresholdK_{threshold}.

We hope this addresses your raised comment about how high GNQ value would indicate about the absolute level of disclosure, and how the auditor would choose a threshold or removal budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.