acceptodds
Under review as a conference paper at ICLR 2027

Value Before Disclosure: Data Valuation under Local Differential Privacy

Abstract

Data privacy is a critical concern in data valuation. Assessing a record's task-dependent value before purchase often requires access to potentially sensitive information. We propose PIVOTS, a local differential privacy (LDP) framework that lets providers estimate their contributions without first disclosing their raw records. The buyer uses one locally randomized report from each provider and validation data to construct a broadcast valuation function that each provider evaluates on its retained record to estimate its value. Theoretically, we establish mean-squared-error upper bounds and matching minimax lower bounds when validation data are sufficiently abundant. We also consider settings in which some non-sensitive features remain public, developing a corresponding valuation method and providing theoretical guarantees that quantify the resulting gain in valuation accuracy brought by partial feature disclosure. Experiments on synthetic and real-world data demonstrate accurate valuation under privacy constraints and substantial improvements in downstream task performance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Data Valuation, Local Differential Privacy, TKNN-Shapley, Nonparametric Regression

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XkKeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a data value evaluation algorithm based on LDP, with the core idea being for the seller to locally evaluate the relationship between its own data and that of its neighbors. The paper further considers the semi-trusted scenario.

Critical strengths

A large amount of theoretical analysis

multidimensional experiments

Critical weaknesses
  1. The solutions proposed in this paper are mostly permutations and combinations of existing solutions, and no essential differences from existing LDPs are observed.

  2. The scenario assumptions in this article are relatively idealized or overly strict

  3. limited experiments

Questions for authors

see w

7 comments

We sincerely thank the reviewer for the valuable comments and positive assessment of our theoretical analysis and experimental evaluation. We appreciate the concerns regarding methodological novelty, theoretical assumptions, and experimental limitations. We have revised the paper to address these issues and provide point-by-point responses below.

  • W1:The proposed framework builds on TKNN-Shapley, histogram-based estimation, and established local differential privacy mechanisms. Although combining these components for provider-side valuation is potentially useful, the paper does not sufficiently distinguish its fundamental methodological contributions from existing techniques for private statistical estimation and privacy-preserving data valuation.

    We agree that our work builds on established technical tools. Its novelty lies in the following three contributions.

    • A provider-side framework for private data valuation.. We develop a protocol in which each provider sends a single LDP report and locally estimates their record's value using aggregated information about the candidate pool, without accessing other providers' raw records or disclosing their own raw record before valuation.

    • A valuation-specific estimator. Because TKNN-Shapley depends nonlinearly on neighborhood counts, directly plugging in noisy counts can yield unstable valuation estimates. We address this issue through a tailored stabilization procedure, whose effectiveness is supported by our ablation studies.

    • Theoretical guarantees. We establish MSE upper bounds and matching minimax lower bounds. These results characterize how privacy and feature discretization affect valuation accuracy.

    We therefore believe that our work makes a significant contribution to privacy-preserving data valuation.

  • W2: The analysis assumes independent and identically distributed candidate records, bounded feature densities, and a known positive lower bound on the feature density. These assumptions may be difficult to verify in practical data marketplaces, where candidate records can be heterogeneous, dependent, or highly imbalanced.

    We acknowledge that our theoretical assumptions may be restrictive. We only consider an illustrative setting for deriving clean statistical guarantees, with possible adaption to broader settings. We clarify the scope as follows:

    • First, the LDP reporting protocol does not rely on the distributional assumptions used in our statistical analysis. Our upper-bound analysis assumes i.i.d. sampling and bounds on the feature densities, with the lower bounded density assumption alleviatable using Gyorfi and Kroll (2025). For the lower-bound analysis, we adopt assumptions such as i.i.d. and upper bounded feature density, which are also commonly used in nonprivate nonparametric estimation.

    • Second, these assumptions are primarily used for theoretical analysis and are common in nonparametric estimation (Ma and Yang, 2024; Gyorfi and Kroll, 2025). Under these conditions, we establish matching MSE upper and lower bounds.

  • W3:The paper is technically elaborate, but the experimental evaluation is relatively limited.

    We apologize for the limitation and have substantially expanded our experiments. The expansion including real-data ablation studies, comparisons with direct plug-in, unstabilized, and local baselines, non-private TKNN-Shapley results, and additional statistical significance tests. We have also added experiments across different privacy settings and budgets, together with computational cost measurements. These additional results provide stronger empirical evidence for the effectiveness of our estimator, particularly the benefits of feasible count fitting, stabilization, and using private candidate-pool statistics. The detailed results are provided in Appendices D and E.

We sincerely thank the reviewer again for the valuable comments. We hope our responses have addressed your concerns and clarified our work. We would greatly appreciate it if consider raising our score.

Yuheng Ma and Hanfang Yang. Optimal locally private nonparametric classification with public data. Journal of Machine Learning Research, 25(167):1–62, 2024.

László Györfi and Martin Kroll. On rate optimal private regression under local differential privacy. Statistica Sinica, 35:613–627, 2025. doi: 10.5705/ss.202022.0186.

Reviewer_XkKeOpenReview ↗

I have more concerns.

Thank you very much for the authors' clarification. I find the research problem addressed in this paper quite interesting. However, the current presentation still needs substantial improvement.

I would like to ask: How can data validation be performed in the absence of privacy constraints? What specific challenges arise when privacy protection is introduced? Are these challenges fundamentally different from those encountered in existing LDP settings?

Given the word limit for the rebuttal, please feel free to address these questions directly without any formalities. I am also quite familiar with LDP research, so a technically detailed and straightforward response would be appreciated.

Reviewer_XkKeOpenReview ↗

You can do better.

Based on your current response, I am increasingly inclined to view this work as primarily an engineering-oriented contribution, with relatively limited algorithmic novelty. Nevertheless, I personally believe that well-executed engineering research also deserves acceptance.

Reviewer_fSurOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how data providers can estimate the value of their records before disclosing the raw data. The proposed framework, PIVOTS, exploits the local-count representation of TKNN-Shapley values. Each provider submits one locally private report; the buyer aggregates these reports and broadcasts information that providers combine with their retained records to estimate their values locally. The framework covers full local differential privacy, partially public features, and label-only privacy. The authors establish mean-squared-error upper bounds and matching lower bounds for prescribed reporting mechanisms at selected grid resolutions. Experiments include synthetic scaling studies, stabilization ablations, and evaluations of ranking recovery, corruption detection, and downstream prediction on nine binary-classification datasets.

Critical strengths
  1. A meaningful formulation of valuation before disclosure. Allowing providers to combine their retained records with private candidate-pool summaries is a useful design. It incorporates information about other candidates, including redundancy, while avoiding raw-record disclosure during the valuation stage. Reusing a single randomized report across validation queries also avoids accumulating privacy loss separately for each query.

  2. Substantive analysis of a nonlinear valuation estimator. The theoretical treatment addresses count sensitivity, instability in small neighborhoods, discretization error, and dependence induced by reusing reports across queries. The lower-bound argument explicitly handles a realized-dataset target rather than replacing it with its population expectation. The synthetic ablations additionally support the finite-sample benefit of stabilization in the tested low-dimensional settings.

Critical weaknesses
  1. The experiments do not sufficiently isolate the proposed estimator’s contribution or establish consistent practical gains over simple selection strategies. PIVOTS and LDP-KNN differ in their valuation targets, reporting mechanisms, and use of the provider’s retained record. Consequently, their comparison does not isolate the benefit of the proposed private count estimation procedure. The authors acknowledge the difference between TKNN-Shapley and KNN-Shapley in Section 5.2, but controlled real-data comparisons remain necessary. Moreover, outperforming LDP-KNN does not establish improvement over random selection. In Table 8, under full LDP with (α=1\alpha=1) and 25% label corruption, PIVOTS has lower mean downstream balanced accuracy than random selection on six of the nine datasets. Examples include MAGIC (69.23 versus 72.76), CALIF (78.26 versus 80.63), and HOUSE8L (74.34 versus 76.64). These mean differences do not establish statistical significance, but they qualify the practical improvement claim. Particularly useful controls would include a common TKNN-Shapley target, non-private TKNN selection, and a local baseline that uses the same public/validation information and retained record but no private candidate-pool reports. The last comparison would establish how much benefit actually comes from the privately estimated pool statistics.

  2. The connection between the restricted theoretical guarantees and the intended application remains incomplete. Theorem C.3 fixes the reporting mechanism and grid, and the appendix explicitly states that the lower bound does not optimize over alternative mechanisms or grids. This supports optimal estimation within the prescribed experiment at the selected resolution, rather than a general minimax characterization of non-interactive LDP valuation. The main theorem statement should make this distinction equally explicit. Practical feasibility also needs better documentation. A single report contains 2h−s2h^{-s} noisy coordinates, so one reporting round does not imply low communication cost. The real-data experiments do not sufficiently document the actual grid sizes, communication requirements, runtime, and memory consumption. Finally, Appendix A.4 correctly notes that releasing a locally computed valuation does not automatically protect the provider’s own record. Yet global top-50% selection requires information about relative valuations across providers. The paper should clarify whether these experiments are an offline utility evaluation or an implementation of a private acquisition procedure, and specify what additional information the selection stage reveals. This does not invalidate the report-level privacy guarantee, but it limits the application conclusions currently supported.

Questions for authors
  1. Under a common TKNN-Shapley target and identical access to each provider’s retained record, how does PIVOTS compare with direct count plug-in estimation, feasible fitting without stabilization, and a local baseline that does not use candidate-pool reports? Do the synthetic ablation conclusions carry over to the real datasets?

  2. Which downstream improvements over random selection are statistically supported by paired comparisons? What are the corresponding results for non-private TKNN-Shapley selection?

  3. Who performs global top-50% selection, and what does the buyer observe during this stage: local scores, rankings, or participation decisions? What privacy assumptions apply to these additional observations?

  4. Which full-LDP neighborhood definition is used in the proofs and experiments: exact validation locations as in Section 3.2, or validation cell centers as in Appendix A.1? Please also specify the mechanism and grid restrictions directly in Theorem 4.3.

  5. What are the actual feature dimensions, grid resolutions, per-provider report sizes, and computational costs in the real-data experiments? If the implementation simulates aggregate noise or uses an implicit representation, how does this correspond to the stated distributed protocol?

4 comments

We sincerely thank the reviewer for the valuable comments and the intriguing questions. We have significantly revised our paper, especially the experiment section, to address the comments. Please see the point-to-point responses blow.

  • Q1: Under a common TKNN-Shapley target and identical access to each provider’s retained record, how does PIVOTS compare with direct count plug-in estimation, feasible fitting without stabilization, and a local baseline that does not use candidate-pool reports? Do the synthetic ablation conclusions carry over to the real datasets?

    We have added comprehensive ablation studies comparing PIVOTS with all three baselines. We evaluated these methods on nine real datasets under full LDP, semi-LDP, and label LDP, considering both label and feature corruption and privacy budgets α∈{1,2,3,4}\alpha \in \{1,2,3,4\}. Here, we summarize the results for α=1\alpha=1 under full LDP. The complete results are provided in Appendix E and show trends consistent with those reported here.

    MetricDirect plug-inUnstabilizedLocal
    Recovery16/18 (9)16/18 (10)14/18 (5)
    Detection AUROC16/18 (12)17/18 (14)15/18 (11)
    Downstream Acc.18/18 (16)18/18 (16)13/18 (10)

    Each entry reports [number of settings PIVOTS wins] / [number of settings] (number of statistically significant wins). The statistical significance is tested using Wilcoxon tests with Holm correction under 50 runs. Consistent with the synthetic findings, PIVOTS wins most comparisons across all three metrics and outperforms direct plug-in and unstabilized PIVOTS in every downstream comparison.

  • Q2: Which downstream improvements over random selection are statistically supported by paired comparisons? What are the corresponding results for non-private TKNN-Shapley selection?

    We have illustrated comparisons with random selection and conducted additional experiments for NP-TKNN. The detailed results are provided in Appendix E (e.g., Table 16 - 30).

    • Across 216 comparisons (9 datasets ×\times (label/semi/full LDP) ×\times (feature/label corruption) ×\times (α=1/2/3/4)(\alpha = 1/2/3/4)) with random selection, PIVOTS achieves higher balanced accuracy in 136 cases, including 44 statistically significant improvements. Random selection performs better only in 79 cases with 26 significance.
    • We have also compared non-private TKNN-Shapley (NP-TKNN). NP-TKNN achieves higher downstream balanced accuracy than Random in 43 out of 54 comparisons, compared with 34/54 for PIVOTS against random at α=1\alpha=1. Despite privacy constraints, PIVOTS remains close to NP-TKNN, particularly under Label LDP, where their mean balanced accuracy differs by at most 0.27 percentage points across settings.
  • Q3: Who performs global top-50% selection, and what does the buyer observe during this stage: local scores, rankings, or participation decisions? What privacy assumptions apply to these additional observations?

    We have replaced global top-50% selection with a local threshold rule and updated the numerical results.

    • We agree that global top-50% selection requires a threshold that is not directly available under our current protocol. Determining this threshold requires information about sellers' local scores, and LDP estimation errors may also affect their relative rankings. Following Wang et al. (2024), we used this criterion solely for numerical evaluation. A practical implementation would require an additional privacy-preserving mechanism, such as LDP quantile estimation or a cryptographic protocol.
    • Following the reviewer's suggestion, we have replaced the selection rule with the local criterion Φ~i>0\widetilde{\Phi}_i>0. This rule selects sellers whose estimated Shapley values indicate a positive contribution to the buyer's task, allowing each seller to make the decision locally without disclosing their score or ranking. Other publicly specified thresholds could be used in the same way. All numerical results have been updated accordingly (see Section 5), and the empirical conclusions remain unchanged.
  • Q4: Which full-LDP neighborhood definition is used in the proofs and experiments: exact validation locations as in Section 3.2, or validation cell centers as in Appendix A.1? Please also specify the mechanism and grid restrictions directly in Theorem 4.3.

    We have revised Sections 3.2 and 4.2 to clarify the methodology and the privacy protocol. First, we revised Section 3.2 to explicitly state that full LDP uses validation cell centers when defining neighborhoods, consistent with Appendix A.1. In contrast, semi/label-LDP uses exact validation locations. Second, we clarified the reporting mechanisms and grid restrictions following Theorem 4.3. For s>0s>0, fix the cell-label reports with independent Lap⁡(2/α)\operatorname{Lap}(2/\alpha) noise and the regular grid h−1=⌈τ−1(Nα2τd+s)1/(s+2)⌉h^{-1}=\lceil\tau^{-1}(N\alpha^2\tau^{d+s})^{1/(s+2)}\rceil, subject to h≤τh\le\tau for full LDP and h≤τ/sh\le\tau/\sqrt{s} for semi-LDP. For s=0s=0, fix binary randomized response without a grid. The infimum is over estimators, with the reporting mechanism and grid held fixed.

  • Q5: What are the actual feature dimensions, grid resolutions, per-provider report sizes, and computational costs in the real-data experiments? If the implementation simulates aggregate noise or uses an implicit representation, how does this correspond to the stated distributed protocol?

    We have added the suggested details for the real data experiment in Appendix D.5 (Table 15). The results indicate that PIVOTS has modest runtime and memory requirements under our simulation setup. In these experiments, the distributed reporting protocol was simulated on a single machine. Evaluating PIVOTS in a real distributed deployment is an important direction for future work.

We sincerely thank the reviewer again for the valuable comments. We hope our responses have addressed your concerns and clarified our work. We would greatly appreciate it if consider raising our score.

Jiachen T. Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data Shapley for data selection tasks: Misleads and merits. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 52033–52063. PMLR, 2024. URL https://proceedings.mlr.press/v235/wang24cg.html.

Dear Reviewer fSur,

Thank you again for your time and comments. We have posted our responses and uploaded the revised paper. Before the discussion closes, we would appreciate hearing whether our responses address your concerns and which points still need clarification. We would be happy to discuss further.

Best,

Authors

Thanks for the response and clarifications. The author's response solved my major concerns, I'd like to keep my rating.

Reviewer_1SUqOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces PIVOTS, a framework for provider-side data valuation before raw data are disclosed. Each provider submits a locally private report, which the buyer aggregates and broadcasts back to providers. Providers then combine this summary with their locally retained records to estimate their own data values. The framework also accommodates varying numbers of private feature coordinates. The authors validate PIVOTS through both theoretical analysis and empirical experiments.

Critical strengths
  1. Significance. The paper addresses privacy-preserving data valuation, an important and practical problem in real-world data markets.

  2. Novelty. The authors propose a TKNN-Shapley-based method that enables privacy-preserving data valuation on the seller side, offering a novel approach to balancing data privacy and valuation utility.

  3. Theoretical and empirical validation. The authors substantiate the effectiveness of the proposed method through both theoretical analysis and empirical experiments.

  4. Presentation quality. The paper is well-written and clearly organized, making the proposed approach easy to follow and understand.

Critical weaknesses

Although the seller-side data valuation strategy avoids direct data exposure, the framework appears to rely heavily on the seller's trustworthiness. How does PIVOTS prevent or detect sellers who manipulate or forge evaluation results to inflate the value of their data?

Questions for authors

See in Weaknesses.

3 comments

We sincerely thank the reviewer for the valuable comments and the intriguing question. We agree that potential manipulation by sellers is an important issue in privacy-preserving data valuation. Below, we provide possible solutions to address this problem. We have incorporated these discussions into the revised version of the paper.

  • The current PIVOTS framework can be adapted to solve this issue.

    Preventing or detecting dishonest reporting is challenging under LDP because the randomization of users' data makes it difficult to distinguish honest reports from manipulated ones. Nevertheless, PIVOTS could be extended with existing mechanisms for both prevention and detection.

    • For prevention, one possible approach is to design a reward mechanism that encourages sellers to report truthfully. For example, Chen et al. (2020) proposed a peer-prediction mechanism that uses payments to incentivize truthful data reporting.

    • For detection, we could consider methods that identify suspicious reports or abnormal patterns in the collected data. For example, Li et al. (2025) studied how to detect malicious users and poisoning attacks in LDP protocols.

  • The problem could be alleviated by system design.

    We believe PIVOTS can be deployed in settings where the application controls the reporting process. For example, Apple has deployed LDP on users' devices (Apple, 2017). Users choose whether to participate, while the software automatically generates and transmits LDP reports. Adopting a similar deployment model for PIVOTS (or other LDP protocols) could therefore limit sellers' opportunities to manipulate their reports.

  • The design of LDP data valuation framework that is robust to malicious seller is an important direction for future work.

    This is a promising research direction, and several studies have already highlighted its importance. For example, Cheu et al. (2021) and Cao et al. (2021) investigated manipulation attacks under LDP and demonstrated how malicious users can distort aggregate estimates. In future work, we would like to explore general mechanisms for preventing and mitigating dishonest reporting in LDP protocols and investigate how these mechanisms can be incorporated into PIVOTS to improve its robustness.

  • We have expanded the discussion of related work in Appendix G.5 of the revised manuscript. We would welcome any suggestions for relevant studies we may have overlooked.

We sincerely thank the reviewer again for the valuable comments. We hope our responses have addressed your concerns and clarified our work. We would greatly appreciate it if consider raise our score.

Yiling Chen, Yiheng Shen, and Shuran Zheng. Truthful data acquisition via peer prediction. In Advances in Neural Information Processing Systems, volume 33, 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/d35b05a832e2bb91f110d54e34e2da79-Abstract.html.

Xiaolin Li, Ninghui Li, Boyang Wang, and Wenhai Sun. Mitigating data poisoning attacks to local differential privacy. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, pp. 1230–1244. Association for Computing Machinery, 2025. doi: 10.1145/3719027.3744839. URL https://doi.org/10.1145/3719027.3744839.

Albert Cheu, Adam Smith, and Jonathan Ullman. Manipulation attacks in local differential privacy. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 883–900. IEEE, 2021. doi: 10.1109/SP40001.2021.00001. URL https://doi.org/10.1109/SP40001.2021.00001.

Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Data poisoning attacks to local differential privacy protocols. In 30th USENIX Security Symposium (USENIX Security 21), pp. 947–964. USENIX Association, 2021. ISBN 978-1-939133-24-3. URL https://www.usenix.org/conference/usenixsecurity21/presentation/cao-xiaoyu.

Thank you for the authors' clarifications. My concerns have been addressed, and I am willing to raise my score accordingly. However, as I am not an expert in this field, I remain somewhat uncertain about the novelty of the proposed method, according to the other reviewers' comments, the methodological novelty appears to be limited.

We sincerely thank you for carefully considering our rebuttal and raising your score. We truly appreciate your time and thoughtful feedback. We also understand your remaining uncertainty regarding methodological novelty. We will further list the discussion details and both our and the reviewer's perspectives for the Area Chair. Thank you again for your constructive feedback and reconsideration of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.