- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors of the paper study joint quantile estimation and confidence sets under sequential, local differential privacy (specifically, one-report-per-user local differential privacy). Comparisons with all current thresholds are encoded as one categorical variable and decoded through a private channel. At the true quantiles, category probabilities depend only on the requested quantile levels, making each candidate channel’s reference score distribution public. This enables selection by calibrated ellipsoid volume without knowing quantile densities; maximum-width selection additionally uses a private density probe. A disjoint pilot/probe/main split supports conditional asymptotic inference after selection. A fixed-threshold correction separates score noise from localization and density-estimation errors. Experimental validation includes simulated distributions, comparisons among five channels, calibration checks, and an illustration using public BMI records.
Critical strengths
(1) The inference-specific mechanism-selection formulation is useful: The public category distribution at the target quantiles allows computation of the score covariance despite the unknown population distribution. (A similar technique was used in [1]). Equations (8)–(9) identify why unknown density factors cancel when comparing ellipsoid volumes, while the maximum-width objective retains dependence on individual quantile densities. The common optimal decoder puts heterogeneous channels on the same CDF workload. This explains when the desired uncertainty summary should affect mechanism choice.
(2) The analysis addresses selection and the actual online recursion: Theorem 3.2 analyzes moving thresholds and does not seem to assume that online scores already follow the frozen truth distribution. Conditioning on the pilot and probe handles the selected channel, including selection near ties. Equation (19) and Proposition 3.3 provide a decomposition for fixed-threshold correction. The experiments differentiate channel improvements from localization costs and report independent calibration checks and coverage shortcomings. This makes the statistical contribution more convincing than a numerical comparison of channel covariances alone!
Critical weaknesses
(1) I believe the authors do not sufficiently compare to related works in the literature on quantile Inference. Can the authors comment on how techniques from [1] apply to their results?
(2) Limits of reliable finite-sample coverage: Appendix B.3 reports coverage around 0.850–0.865 under a narrower density-probe bandwidth despite using >= 500,000 users! The improved fixed-threshold procedure uses 175,000 final inference reports out of 500,000 total users and increases confidence volumes by approximately 5.3–6.0 times relative to the recursive branch. This limits any claims about practical efficiency. The asymptotic results do not (yet) give an operationally and reliable way to choose the bandwidth and localization allocation for an unknown/unfamiliar distribution. Also, there are limits of the demonstrated incremental benefit of the selector: Section 4.2 explicitly states that the headline 42%–43% Wald-volume reductions are the gains of fixed OLH over fixed RR under the same protocol: both determinant-only and calibrated Wald selection choose OLH in those designs. The current evidence does not imply a meaningful practical gain of finite-score selection over a simpler determinant-based selector with Gaussian calibration. The confidence-geometry formulation is certainly useful! But I believe the headline improvements cannot be interpreted as demonstrating an additional benefit from adaptive selection.
Questions for authors
(1) I believe the authors do not sufficiently compare to related works in the literature on quantile Inference. Can the authors comment on how techniques from [1] apply to their results?
(2) Contribution-isolating evaluation: Across the existing configurations, how much improvement in coverage and confidence-set size comes from finite-score mechanism selection relative to determinant-only Wald selection with Gaussian calibration and to the best fixed channel? Please compare these using the same user, privacy, and declared communication budgets. This would clarify the added value of the proposed selection/calibration layer.
(3) Tuning robustness: What rule should a practitioner use to choose the density bandwidth and localization allocation before seeing the evaluation distribution? Please quantify coverage, volume/width, and RMSE sensitivity around the reported bandwidth factors and the allocation. The narrow-bandwidth coverage shortfall and the cost of fixed-threshold correction make this central to practical usefulness.
3 comments
Response to Reviewer VtQ5 (2/3)
We ran a comparison across contrasting density profiles using the same feasible width selector. The populations are and the equal mixture of and . Both use the central five probabilities , public bounds , , and the same ten-channel library with an eight-bit report allowance. Every method receives users, split 10%/20%/70%; localization uses of the last stage, leaving final reports. The bandwidth multiplier 1.05 and localization fraction were chosen after separate model-specific development runs and frozen before 500 fresh confirmation repetitions per population; the confirmation uses a disjoint seed family. The larger localization fraction supplies a longer anchor recursion, while the smaller multiplier reduces smoothing across the mixture valley. This is a development-selected configuration, not a data-driven tuning rule for an unknown distribution. The other recursion settings are unchanged. Channel choice uses only the independent private density probe and public score-law cutoffs. The nominal simultaneous coverage is 0.95:
| Population | Method | Joint coverage | Mean maximum width | RMSE |
|---|---|---|---|---|
| Normal | Fixed Subset2 | 0.948 | 0.1705 | 0.0255 |
| Normal | Fixed PairStair | 0.948 | 0.1883 | 0.0265 |
| Normal | Feasible selector | 0.948 | 0.1705 | 0.0255 |
| Bimodal mixture | Fixed Subset2 | 0.942 | 0.3595 | 0.0415 |
| Bimodal mixture | Fixed PairStair | 0.946 | 0.3017 | 0.0349 |
| Bimodal mixture | Feasible selector | 0.946 | 0.3017 | 0.0349 |
The private selector chooses Subset2 (100.0%) for normal data and PairStair (100.0%) for the bimodal mixture. Its mean maximum width is 9.5% smaller than fixed PairStair in the normal population and 16.1% smaller than fixed Subset2 in the mixture. Within each population, the selector exactly matches the better of these two fixed channels in width, coverage and RMSE; the percentage gains compare it with the other fixed channel. Neither Subset2 nor PairStair is the better choice in both populations, and the same private selection rule follows this change. The table gives coverage and RMSE alongside width; this larger-budget comparison is separate from the original 500000-user results.
The dominant Wald-volume gains in these studies come from the reporting channel (OLH or Subset2). Selection adds no gain over the fixed Wald winner here; the incremental practical benefit of finite-score calibration is small, and a width reduction accompanied by lower coverage is not an equal-coverage advantage.
Response to Reviewer VtQ5 (3/3)
Q3. The reference implementation fixes public bounds and levels before evaluation, a 10%/20%/70% pilot/probe/main split, step exponent , bandwidth and , with pilot/main step constants 0.50/0.30, a bandwidth cap at half the smaller pilot boundary margin and densities truncated to . For locally Lipschitz densities, balances the stated density-error bound, and requires .
The submitted narrow-bandwidth recursive comparison has RR/OLH coverage 0.865/0.850 at and . Replacing only its final density standardization by true densities raises both to 0.908. The fixed-threshold expanded-library results below use a different estimator and independent repetitions. The broader rerun retains multiplier 0.70 to reproduce the recursive design; the profile comparison uses separately developed, frozen tuning as stated above. We currently have no data-driven bandwidth/localization rule with a finite-sample coverage guarantee for an unknown distribution.
The prespecified sensitivity multipliers are 0.70, 1.05, 1.40, 1.75, 2.10. Small bandwidth amplifies private probe noise: two density-floor events account for about 86.1% of Subset2 squared error in the normal 0.70 cell. For untruncated standardized with bounds , the reference multiplier 1.40 itself gives : the population symmetric difference quotient is 0.42325 against , and Wald/Max coverage is 0.880/0.928 with no density-floor activation. Thus the reference rule also has a material coverage shortfall in this design.
Fixed-threshold inference uses reports; halving increases leading width by , and the correction has the exact decomposition
=N^{-1/2}\sum_{s=b+1}^{n_3}V_s +\sqrt N(\widehat D-D)e+\sqrt N r(e).$$ Here $e=\overline{\boldsymbol q}_b-\boldsymbol Q$, $D=\operatorname{diag}\\{f(Q_j)\\}$, $h(q)=\boldsymbol\tau-F(q)$ componentwise, $r(e)=h(\boldsymbol Q+e)+De$, and $V_s=\boldsymbol\tau-a_{\widehat r,Y_s}-h(\overline{\boldsymbol q}_b)$. Conditional on the localization reports, $V_s$ is centered. The remaining terms are the density-estimation/localization product and the local curvature remainder. The submitted inference result assumes $b,N\to\infty$ with $N/b$ bounded. Under its local smoothness and uniform-library conditions, $e=O_P(b^{-1/2})$ and consistent density estimation make both remainders vanish. A sublinear localization allocation would require additional uniform rate arguments. At $n=500000$, $\epsilon=0.5$, five quantiles and the expanded library (volume and width entries are ratios of medians, relative to the same law's multiplier-1.40, $b/n_3=0.5$ cell): | Law | Multiplier | $b/n_3$ | Coverage W/Max | RMSE W | Median volume/width ratio | |---|---:|---:|---:|---:|---:| | Normal | 1.40 | 0.500 | 0.932/0.942 | 0.0396 | 1.000/1.000 | | Normal | 0.70 | 0.500 | 0.912/0.938 | 0.1590 | 1.035/1.092 | | Normal | 1.40 | 0.250 | 0.918/0.916 | 0.0339 | 0.357/0.799 | | Normal | 1.40 | 0.750 | 0.960/0.954 | 0.0560 | 6.101/1.423 | | Normal | 1.40 | Recursive | 0.928/0.940 | 0.0266 | 0.172/0.700 | | Mixture | 1.40 | Recursive | 0.922/0.928 | 0.0292 | 0.180/0.698 | At $b/n_3=1/2$, the complete bandwidth comparison is: | Law | Multiplier | Coverage W/Max | RMSE W | Median volume/width ratio | |---|---:|---:|---:|---:| | Normal | 0.70 | 0.912/0.938 | 0.1590 | 1.035/1.092 | | Normal | 1.05 | 0.952/0.942 | 0.0605 | 1.015/1.061 | | Normal | 1.40 | 0.932/0.942 | 0.0396 | 1.000/1.000 | | Normal | 1.75 | 0.946/0.944 | 0.0387 | 0.998/0.986 | | Normal | 2.10 | 0.924/0.930 | 0.0396 | 1.006/0.948 | | Mixture | 0.70 | 0.918/0.950 | 0.0722 | 0.941/1.078 | | Mixture | 1.05 | 0.922/0.940 | 0.0450 | 0.932/1.025 | | Mixture | 1.40 | 0.930/0.930 | 0.0442 | 1.000/1.000 | | Mixture | 1.75 | 0.942/0.940 | 0.0440 | 1.124/0.994 | | Mixture | 2.10 | 0.944/0.948 | 0.0434 | 1.126/0.980 | Each setting has 500 independent repetitions; the mixture is $0.5N(-1,0.75^2)+0.5N(1,0.75^2)$ with the quantile bounds given above. Revised Tables 10 and 11 give every predefined setting, including density-floor events. Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.
Response to Reviewer VtQ5 (1/3)
Thank you for your time and valuable comments.
W1 / Q1. Alabi, Ben-Eliezer and Chaturvedi (2022) include Gaussian population-quantile accuracy as well as central-DP sketches and continual release. Their server accesses raw stream items; ours receives one locally private report per user and targets joint confidence coverage after channel selection. A valid simultaneous CDF band yields joint quantile intervals by generalized inversion: on its coverage event, F←(τj)≤F←(τj)≤F←(τj)for every j. Simultaneous band coverage, nondecreasing bands and conservative endpoint conventions yield this statement, and an inverse-modulus condition, such as a positive local density lower bound, controls interval width. In the one-report LDP setting, this route requires a reporting design for a simultaneous CDF band that accounts for sampling, privacy and approximation errors.
W2 / Q2. In the submitted strong-privacy comparison, both determinant-only and calibrated Wald selection choose OLH; the 42–43% volume gain is therefore a fixed-OLH versus fixed-RR gain. In a separate near-tie design at ϵ=1, the determinant-only OLH/RR factor is 0.955, whereas the estimated calibrated factor is 1.011 and selects RR. In the central-five privacy sweep, with matched user counts, privacy budgets, splits and an eight-bit Stage-3 ceiling, selected-method finite/Gaussian coverage differences are at most 0.004 and mean paired size ratios are 0.986–1.006. A separate rerun of the broader configurations retains the recursive estimator, multiplier 0.70 and 10%/20%/70% split, but uses 500 fresh online repetitions, 20000 calibration draws plus 20000 independent holdout draws, and common Stage-3 ceilings of 7, 8 and 10 bits for 3, 5 and 9 quantiles. The submission used separate repetitions, 500 calibration draws and its stated communication accounting. In this rerun, finite-score and determinant-only/Gaussian Wald selection choose the same channel in both libraries. Their coverage differences are between -0.002 and 0.002; their mean paired volume ratios are 0.9503–1.0329. These differences change calibration, not the selected channel. Across both geometries in this rerun, finite/Gaussian coverage differences range from -0.008 to 0.014, the largest from width-selector switching, and mean paired size ratios from 0.9503 to 1.0329.
Comparing the five-channel and expanded libraries within this same rerun gives:
For these two settings, the submission's separate five-channel results were 0.940/0.0135 and 0.924/0.0165, respectively; they are not the rerun values in the table. In that mixture, holding the Subset2 estimates fixed and replacing only the final Wald density standardization by true densities raises coverage from 0.906 to 0.930, with whitened covariance trace 5.01 against reference 5. For width selection, the ratio of mean selected width to the smallest mean fixed width is 0.9621–1.0000 (five-channel library) and 0.9692–1.0000 (expanded); at n=100000 in the mixture, the expanded ratio 0.9692 comes with selected/fixed coverage 0.924/0.938.