acceptodds
Under review as a conference paper at ICLR 2027

From Erasure to Leakage: Nonresponse under Local Differential Privacy

Abstract

Nonresponse is often treated as a loss of data. Under local differential privacy, however, its effect depends on why users fail to respond. We study categorical distribution estimation with an explicit nonresponse symbol and distinguish two cases. When nonresponse is independent of the private category, the mechanism is exactly -ary randomized response followed by erasure. This gives a simple statistical interpretation: several information measures contract by , and the squared risk of the raw inverse estimator is inflated asymptotically by . The situation is different when response probabilities vary across private categories. The decision to respond then carries information about the private value and also changes the composition of the observed sample. We derive the exact privacy cost of this effect and characterize the resulting selection bias. If the true response propensities are known, the population distribution can be recovered with an unbiased estimator. If they are latent, the released distribution generally does not identify the population separately from the response process. Experiments on synthetic distributions and empirical categorical distributions from four UCI datasets closely match the theoretical predictions. They also show that two mechanisms with the same average nonresponse rate can have very different privacy and estimation costs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Categorical Data, Differential Privacy, Local Differential Privacy, Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aWGaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the non-response problems under local differential privacy: those independent of private categories and those dependent on private categories. For the former, the authors model the response as independent erasure following a random response and derive the information contraction and estimation risk; for the latter, the authors analyze additional privacy loss, sample selection bias, and the unidentifiability of the population distribution, and provide an unbiased corrected estimator when the response probability is known.

Critical strengths
  1. The paper studies a meaningful problem: how nonresponse affects privacy guarantees and statistical estimation under local differential privacy. Understanding these effects is important for reliable privacy-preserving data collection.

  2. The paper is well presented and easy to understand. Its clear organization helps readers follow the problem formulation and theoretical results.

Critical weaknesses
  1. The category-dependent analysis assumes that users’ response probabilities depend on their private categories. However, it is unclear how such dependence can be established or verified in practice. The authors should clarify the empirical justification for this assumption and its implications for the applicability of the analysis.

  2. The proposed unbiased correction requires knowing the true response probability for each category. These probabilities are difficult to obtain in practice, particularly. The authors should explain how these quantities can be obtained and evaluate how inaccurate estimates affect the correction.

Overall, the more refined results may stem from stronger assumptions about the response mechanism and the availability of category-specific response probabilities. This dependence should be made explicit when discussing the paper’s contributions and practical applicability.

Questions for authors

see Weaknesses

1 comment
AuthorsOpenReview ↗

Rebuttal for Reviewer aWGa

Thank you for your careful review. We agree that the applicability of category-dependent nonresponse should be demonstrated rather than assumed, and that knowing the true category-specific response probabilities is an idealized benchmark. The revision addresses both concerns.

1. (Evidence for heterogeneous participation.)

Section 6.3 uses observed examination participation from two NHANES cycles. In August 2021--August 2023, 11,93311{,}933 individuals completed the interview and 8,8608{,}860 completed the examination. Examination participation across four age groups ranged from 0.66750.6675 to 0.79110.7911. The ℓ1\ell_1 difference between the interviewed and examined age distributions was 0.06400.0640.

These data establish unequal second-stage participation, not refusal to submit an LDP report; kk-RR is simulated.

2. (How response propensities can be obtained.)

Section~7 now discusses two settings: response probabilities determined by a reporting protocol and probabilities estimated using a consented validation sample that observes both category and response status.

The NHANES experiment uses 1,0001{,}000 held-out records from the same survey cycle for calibration. This is a sample-splitting demonstration, not evidence that an independent external source exists. In a prospective application, validation would require a consented subset containing both category and response status, or a documented reporting protocol, with sufficient representation of rare categories. Without such additional information or assumptions on selection, the nonidentifiability result applies. In particular, population marginal totals alone do not generally determine all category-specific response probabilities.

3. (Effect of estimation error.)

We retain the known-propensity estimator as a theoretical benchmark but extend the analysis to estimated propensities. Eqs. (27)--(28) give the exact conditional bias and mean squared error, while Section 5.1 and Figure 1 quantify how estimation error can also affect the actual privacy guarantee when estimated propensities determine the informative channel.

A separate experiment in Appendix C.2 varies the calibration sample size from 100100 to 20,00020{,}000 while keeping the study channel fixed. At fixed study size, mean raw MSE decreases from 0.0097910.009791 to 0.0053400.005340. This experiment is distinct from the matched joint deployment in Figure 1. It also exposes the limits of plug-in calibration: at Ncal=100N_{\mathrm{cal}}=100, only 32.7%32.7\% of estimated designs are feasible under the estimated constraints; at Ncal=20,000N_{\mathrm{cal}}=20{,}000, the estimated feasibility fraction reaches 100%100\%, yet 49.5%49.5\% of those point-estimate designs exceed the target when evaluated at the true propensities. This is not evidence that larger samples worsen privacy. As sampling noise becomes smaller, a boundary-calibrated design can still be on either side of the target unless it has a validated safety margin.

4. (Practical performance.)

For NHANES 2021--2023, plug-in correction changes mean projected ℓ1\ell_1 error from 0.193620.19362 to 0.178830.17883 for age (paired difference −0.01479-0.01479, 95%95\% simulation-bootstrap interval [−0.02521,−0.00383][-0.02521,-0.00383]), but from 0.316720.31672 to 0.322720.32272 for race/ethnicity. We report both outcomes: finite-sample gains depend on the strength of selection and the quality of calibration.

Reviewer_xZtrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies categorical distribution estimation under local differential privacy when the analyst can observe nonresponse events. When response probabilities are the same across categories, the mechanism is equivalent to randomized response followed by independent erasure. The authors derive exact information-contraction identities and a finite-sample risk expression for the raw inverse estimator.

When response probabilities depend on the private category, the response decision itself can reveal information and change the composition of the respondent population. The paper derives the exact privacy cost and gives an unbiased linear estimator with its covariance when the true response probabilities are known and the stated invertibility conditions hold. It also shows that unknown response probabilities generally prevent separate identification of the population distribution and response behavior. Synthetic experiments check these results, while experiments using four UCI datasets examine simulated independent nonresponse.

Critical strengths
  1. Theorem 2 gives an exact expression for the privacy cost of category-dependent response. Under its assumptions, unequal response probabilities cause the total privacy loss to exceed the target budget if the informative channel already uses that full budget. This directly explains why privacy accounting must include the response decision.

  2. The paper clearly separates the cases of known and unknown response probabilities. For α>0\alpha>0 and known, strictly positive response probabilities, Eqs. (24)–(25) provide an unbiased linear estimator and its exact covariance. Theorem 4 shows why the population distribution is generally not identifiable when those probabilities are unknown. This makes clear what additional information is needed for the correction.

Critical weaknesses
  1. My main concern is the limited additional insight beyond the exact privacy formula. The independent-response analysis follows from the erasure representation. The information decompositions use the chain rule, and the nonidentifiability argument rests on the products wi=πiaiw_i=\pi_i a_i. These results give a consistent account of this channel, but I do not find the resulting contribution substantial enough to support acceptance.

  2. The experiments do not evaluate the correction when response probabilities must be estimated. The paper explicitly acknowledges that external estimates can be used in a plug-in correction and that its accuracy depends on errors in those estimates. However, the reported validation of the category-dependent correction uses the true response probabilities. Its performance with estimated probabilities therefore remains untested, which limits the evidence for its practical use.

Questions for authors
  1. Clarification: For applications in which category-specific response probabilities are not fixed by the protocol, what external information would make them estimable? A concrete example would help establish when the proposed correction can be used.

  2. Additional validation: How sensitive are privacy calibration and distribution estimation to errors in the response probabilities? Keeping the true probabilities fixed, could you perturb the estimates used in Eqs. (19) and (24) and report the actual privacy loss and estimation error? Please distinguish settings in which calibration is infeasible or the inverse estimator is undefined.

1 comment
AuthorsOpenReview ↗

Rebuttal for Reviewer xZtr

We appreciate the reviewer's concern about significance. The central contribution is not the erasure identity or any chain-rule calculation in isolation. For a single observable category-dependent response channel, the paper jointly determines (i) the exact LDP cost of the response decision, (ii) the change from population to respondent distribution, and (iii) the limit of identification without outside response information. These constraints matter together. The exact privacy expression has two competing terms, one for the nonresponse symbol and one for the informative report. The first cannot be reduced by lowering the kk-RR privacy parameter. Meanwhile, population correction requires information beyond the observed LDP channel, and inaccurate calibration can introduce a persistent estimation bias or cause the deployed mechanism to exceed its intended privacy budget. The revision now quantifies both consequences, rather than presenting the known-propensity estimator as an immediately deployable solution. We do not claim that the underlying missing-data principles are new.

1. (What external information is needed?)

Theorem 4 shows that the released reports alone generally identify only the products wi=πiaiw_i=\pi_i a_i. One possible design is a consented validation sample that records both the private category and response indicator, allowing category-specific response rates to be estimated when the sample adequately represents the target population. Another setting is a protocol with specified response probabilities.

The new NHANES case study (Section 6.3, Appendix C.3) illustrates a held-out calibration design using genuine interview-to-examination participation records. The held-out records come from the same NHANES cycle, not an external survey; NHANES did not deploy the LDP mechanism considered here.

2. (Sensitivity of estimation.)

Section 5.3 now gives, conditional on fixed independent propensity estimates, the exact bias of the raw plug-in estimator:

=\pi_i\frac{a_i-\widehat a_i}{\widehat a_i}.$$ We also derive its covariance and the bias--variance decomposition of mean squared error (Eqs. 27)--(28) and Appendix A.3). The variance decreases with the number of privatized reports, but misspecification bias need not vanish. We explicitly distinguish the raw-estimator result from projected estimation error. **3. (Sensitivity of privacy.)** Section 5.1 derives the actual privacy parameter after choosing the informative mechanism from estimated propensities (Eq. (20)), including the privacy loss associated with the observable nonresponse output. Figure 1 and Appendix C.1 report a matched joint-deployment experiment: the same estimated propensity vector determines both the informative channel and the estimator. In the controlled Zipf-5 setting, underestimating response spread with a maximum categorywise relative propensity error of $5\%$ gives actual privacy loss of $1.1001$ against the nominal target of $1$. At $10\%$, actual privacy loss reaches $1.1665$. The corresponding *exact raw squared* $\ell_2$ risks are $0.004637$ and $0.006059$, compared with $0.005314$ under correctly specified propensities. The Monte Carlo risks agree with these analytic values for feasible designs (Appendix C.1). Importantly, the lower risk at $5\%$ is observed under a weaker privacy guarantee and is therefore \emph{not} a same-budget utility improvement. At a larger error level, calibration bias and channel variance produce a higher risk. Thus estimation error need not increase monotonically with the calibration error magnitude in this joint experiment. | Calibration case | Actual $\varepsilon_Q$ | Exact raw $\ell_2^2$ risk | |---|---:|---:| Correct ($0\%$) | $1.0000$ | $0.005314$ | Underestimated spread ($5\%$) | $1.1001$ | $0.004637$ | Underestimated spread ($10\%$) | $1.1665$ | $0.006059$ | Designs that cannot satisfy the estimated nonresponse-output constraint, or have a nonpositive informative parameter, are explicitly marked infeasible; these are reported in Appendix C.1 rather than dropped silently. Appendix A.4 further characterizes when valid propensity bounds permit conservative privacy calibration. Point estimates alone do not provide a deterministic privacy certificate. **4. (Practical evidence and presentation.)** Section 6 has been reorganized to interpret the findings more directly. The new NHANES experiment demonstrates observable response heterogeneity and the mixed finite-sample effect of plug-in correction, while stating its limitations: observed examination participation, simulated LDP, held-out within-cycle calibration, and estimation for an unweighted interviewed cohort. These additions do not establish universal correction or solve identification with fully latent propensities. Instead, they clarify which additional information is required and quantify the consequences when that information is inaccurate.
Reviewer_eonkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how nonresponse affects local differential privacy and the estimation of population proportions. It distinguishes between cases where everyone has the same response probability and cases where it depends on the person’s private category.

Critical strengths

The topic of this paper is interesting and seems new: the intersection of sampling theory (specifically, estimation under non-response) and differential privacy. The paper provides sufficient introductions to relevant concepts and is relatively easy to follow.

Critical weaknesses

From a survey-sampling perspective, the paper's main finding is largely expected; for example, the effect of assuming a uniform non-response rate. I think the main contribution is the privacy calculation under unequal non-response rates across categories. This is also somewhat expected intuitively, but seeing written expressions is nice. However, since the category-dependent non-response rates are never known, the main message of this work does not offer a practical direction for how to proceed in such a case.

The experimental section is too fragmented and relies too heavily on the supplementary material, with not enough interpretation. I suggest keeping only the most meaningful experiments and discussing the results in depth.

Questions for authors

I think it would be interesting to consider local differential privacy when removing non-responses by imputing synthetic responses for those cases (e.g., using covariates).

1 comment
AuthorsOpenReview ↗

Rebuttal for reviewer eonk

We thank the reviewer for recognizing the relevance of the problem. The uniform-response case serves mainly as a reference point, corresponding to categorical randomized response followed by independent erasure. The substantive question is what changes when an observable response decision depends on the private category. In that case, the same response process affects both the privacy guarantee and the population represented by the reports.

1. Practical implications.

We have expanded the analysis beyond the known-propensity setting. Section 5.3 now derives the exact conditional bias and squared risk of a plug-in estimator that uses estimated response propensities (Eqs. (27)-(28)). Section 5.1 gives the actual privacy parameter when the informative channel is calibrated from misspecified rates (Eq. (20)). These results identify two distinct risks: inaccurate population estimates and a privacy budget that may be exceeded. The latter cannot be ruled out merely by using a point estimate of response probabilities.

2. Experimental presentation.

We have consolidated the experiments into Sections 6.1--6.3. The original frequency-estimation grids and supporting checks remain in Appendix D, rather than occupying the main narrative. Sections 6.2--6.3 instead examine the two limitations identified in the reviews: what changes when response probabilities are estimated inaccurately, and whether unequal participation can be observed in a real collection process. Figure 1 reports a matched sensitivity experiment that uses the same estimated propensities to configure the released channel and invert its reports; Table 1 presents the NHANES case study.

NHANES supplies observed participation in a second-stage health examination; LDP randomization is simulated, not deployed by the survey. For the 2021--2023 age groups, calibration reduces mean projected ℓ1\ell_1 error from 0.193620.19362 to 0.178830.17883, but for race/ethnicity it increases error from 0.316720.31672 to 0.322720.32272. The corresponding selection distances are 0.064010.06401 and 0.025950.02595, respectively. The difference between the two outcomes is informative: in the age experiment there is more selection to correct, whereas for race/ethnicity the smaller selection effect is not sufficient to offset calibration variance in the reported runs. We interpret these as results for the tested cohorts and randomization protocol, not as a general guarantee that propensity correction improves accuracy.

3. Covariate-based imputation.

This is a worthwhile extension. Covariates available for respondents and nonrespondents could support imputation or stratified correction under an additional conditional-response model, such as a missing-at-random assumption conditional on those covariates. Such information changes the identification assumptions; it does not by itself invalidate Theorem 4, which concerns the released LDP reports without additional information. We have clarified this distinction in Section 7 and treat imputation as a possible extension rather than claiming an additional empirical result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.