- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies conditional quantile learning under differential privacy by representing the quantile function as a shared input-dependent location plus a level-specific offset, , so that multiple quantile levels can be learned without repeatedly splitting the privacy budget. The authors propose two methods: Joint, which learns the shared location and offsets jointly while exploiting quantile ordering to reduce privacy noise, and CDF, which first privately learns the location and then privately estimates the residual distribution to obtain quantile offsets. They also derive prediction-error bounds that separate model-sharing approximation error from private learning error, and show empirical gains over separate private quantile fits.
Critical strengths
S1. The paper addresses a clear and privacy-utility trade-off in conditional quantile learning.
S2. The shared-location formulation is simple and interpretable.
S3. The empirical study is extensive and includes several useful ablations.
Critical weaknesses
W1. The main structural assumption is very restrictive for a paper framed as learning conditional quantile functions.
DP-ShareQ assumes that all conditional quantiles share the same input effect:
Equivalently, quantile gaps do not depend on . This rules out heteroskedasticity and more general changes in conditional shape, which are precisely situations where conditional quantiles are most useful. When this assumption fails, the learned curve is not the true conditional quantile function.
W2. The methodological novelty is somewhat incremental at the component level.
The common-slope idea is closely related to composite quantile regression, while the two proposed learners combine established tools such as private noisy-gradient optimization, Huber regression, private distribution estimation, Gaussian covariance design, and isotonic projection. The integration of these ideas and the ordered-gradient analysis are useful, but the paper should more clearly articulate which methodological ingredient is fundamentally new beyond combining existing components for this particular shared quantile model.
W3. The theoretical guarantees rely on strong distributional assumptions, especially for CDF.
The optimal shared-curve result assumes conditionally symmetric residuals, while the Huber-to-quantile transfer additionally requires conditional residual densities to be unimodal/nonincreasing away from zero and imposes a bound on the pooled density. These assumptions are difficult to verify in applications and exclude many common asymmetric or multimodal conditional distributions.
W4. The real-data uncertainty assessment is based on one fixed train/validation/test split.
For each real dataset, the paper fixes a single partition and repeats only the private-learning noise 100 times. The reported standard deviations therefore quantify privacy randomness but not sampling or split variability. This is particularly concerning for small datasets such as Real Estate. Repeated data splits or cross-validation would provide a much more reliable assessment of the empirical gains.
Questions for authors
See weaknesses.
6 comments
C3/W3. Distributional assumptions. Revised Section 4 removes conditional symmetry, unimodality, and density assumptions while retaining the original big-O learning-error rates for bounded i.i.d. records and fixed learning settings. Proposition 3 is reformulated without conditional symmetry, and Theorem 4 retains its original bound. The substantive change concerns the Huber-to-quantile risk comparison in the original Theorem 5. We now bound this discrepancy without the original distributional shape assumptions.
Proposition 3. For every choice of location coefficients, pooled residual quantiles give the optimal shared offsets. Minimizing this population risk over the location defines the optimal shared curve used in both learning bounds. The original result identified this curve through the symmetry-defined location .
Theorem 4. The bound is unchanged. Conditional symmetry was used only to identify the optimal shared reference, which we now define directly by population risk minimization.
Proposition 5, replacing the original Theorem 5. Equation (14) links Huber loss to quantile risk without the original shape assumptions. Huber loss equals quantile risk with fixed offsets up to a constant, and fitting residual quantiles reduces this risk. Appendix A.6 then bounds the gap from the optimal shared curve.
Theorem 6. The original contained ridge cost , with , and finite-CDF approximation. Equation (44) now gives
where is the nonnegative gap between the two quantile risks in Equation (14). Equation (42) retains the difference between the gains from fitting residual quantiles at the optimal shared location and at the learned location. Appendix A.10 gives the full proof.
C4/W4. Split variability. We have rerun the real-data comparisons throughout the main text and appendix on five data splits, with ten noise repetitions per split. The updated results report means and standard deviations across the five splits. At ε=1, both methods have lower mean nIPL than all three reference methods in all 12 dataset–map settings (Table 1). On Real Estate with linear features, nIPL ×100 is 5.159 ± 0.381 for Joint and 5.307 ± 0.603 for CDF, compared with 7.562 ± 0.330 for Full and 7.181 ± 0.846 for Sep-Out.
We welcome any further comments or questions. If our responses resolve your concerns, we would be grateful if you would consider reassessing your score.
Thank you. I find that the authors have substantially addressed most of my previous concerns, and the revisions largely improve the original manuscript.
I have two more questions. I appreciate the additional experiments in Appendix B.6, but after accounting for the privacy cost of tuning parameter selection, the average nIPL have largely increased. Does this evidence indicate that this cost is non-negligible? Besides, Appendix B.6 uses 60% of the data for training and does not retrain the selected model, whereas the main experiments retrain the models on 75% of the data after tuning parameter selection. Please consider to clarify this distinction and the reason.
Thank you for your careful review and comments on our model and methodological contribution. We believe that addressing all reviewers' comments has substantially improved the paper. We have uploaded a revised manuscript incorporating our responses to all comments. Equation, section, table, and figure numbers below refer to this revision. Changes are marked in blue, and updated numbers in green.
C1/W1. Input-dependent quantile gaps. Sharing introduces approximation error when quantile spacing varies with the input. However, under privacy constraints, this cost is traded against the gain from reduced DP noise. As Section 4 explains, sharing reduces the number of privately learned coefficients and the resulting DP noise. We assess this trade-off by comparing with Full and an input-dependent scale extension under the same privacy budget.
DP-ShareQ can also capture input-dependent uncertainty by adding scale coefficients. Appendix G.2 evaluates LS-CDF, q(x,τ)=l(x)+s(x)d(τ), under the same total privacy budget. In Figure 8(d), at γ=0.75, nIPL ×100 is 3.685 for LS-CDF, 3.882 for CDF, and 3.800 for Full. The extension improves the shared model under input-dependent scale and outperforms Full. On the real datasets, however, CDF has 2.0%, 6.2%, and 1.9% lower mean nIPL than LS-CDF at ε=0.1,0.5,4 (Table 18). This comparison supports retaining CDF as the main model at the evaluated budgets.
C2/W2. Methodological contribution. DP-ShareQ is a new differentially private quantile regression algorithm. We have clarified the contributions of Joint and CDF in the Introduction.
The first challenge is to learn many quantiles under one privacy budget. Separate fits divide the budget across levels. Fitting all levels together still requires many noisy coefficients when the model allows different input effects at each level. Joint and CDF both learn q(x,τ)=ϕ(x)⊤β+d(τ), sharing one input effect across levels. Joint fits the location and offsets together. CDF fits the location and obtains all offsets from one private residual distribution.
The second challenge is to reduce privacy noise beyond coefficient sharing, which alone does not ensure better prediction (Appendix F.4). We derive tighter sensitivity bounds by ruling out changes that cannot occur together when one record is replaced.
In Joint, a record's residual sign changes at most once across ordered quantile levels. We use the resulting gradient set (Equation (3)) to calibrate noise and allocate its variance across location and offset directions. Table 2 shows the prediction gains from both the ordered bound and noise allocation.
In CDF, all Fourier coefficients are functions of the same residual. Bounding their changes together requires less noise than bounding each coefficient separately. For Fourier orders 16, 32, and 64, noise standard deviations fall by 22–24% (Section 3.2). Table 2 reports the corresponding reduction in prediction loss.