- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper compares two strategies for improving conditional coverage given a fixed initial score and an additional labeling budget: global score adaptation (via learning conditional quantiles) and local calibration (via RLCP). The authors derive bounds for high-probability conditional coverage and oracle-relative prediction-set length error, establish a lower bound on global propagation of localized estimation errors, weaken the covariate regularity assumption required for existing RLCP bounds, and evaluate both approaches through finite-sample convergence-rate bounds and empirical experiments.
Critical strengths
- Good experimental control. By fixing the initial score, referenced Oracle, and a common additional labeled-data budget, the study reduces potential confounding factors arising from differences in base models.
- Meaningful theoretical refinements. Proposition 3 demonstrates how the failure of local estimation can contaminate other regions through global threshold, and Theorem 4 relaxes the regularity requirements of RLCP to depend on local variations of the target quantile, making it more applicable to broader distribution classes.
- Comprehensive experimental analysis. The study employs synthetic data with known conditional CDFs and report useful metrics, (e.g., lower-tail coverage, length errors, and infinite prediction sets). The cases where localization is preferred are also included in the low-dim and label allocation experiments.
Critical weaknesses
- The significance of the incremental contributions is unclear. The two key approaches or routes (score adaption and RLCP) are both existing methods and some of the corresponding theoretial results have been established. While the authors identify several technical differences from prior work, it is still unclear whether these refinements lead to qualitatively new statistical insights or practically meaningful consequences. The authors should more clearly explain the significance of the genuinely new conclusions and explain why these conclusions represent a substantive advancement, rather than just serving as a specialization of existing error analyses or a weakening of assumptions. Otherwise, the contributions that are merely marginally incremental is unlikely to be convincing.
- The comparison of convergence rates does not separate the two approaches clearly. The neural network controls the L2 error, local polynomial controls uniformly over , whereas RLCP focus on a neighborhood of the realized centre. The comparable results are mainly for particular functional classes and methods. The paper does not establish general matching bounds or a minimax separation between the two approaches. The authors should clarify the scope of the comparative conclusions and explain what new statistical insights (in particular, beyond those suggested by classical nonparametric estimation theory) can be drawn from these rate comparisons.
- Limited empirical conclusions. The main high-dim synthetic experiment uses smooth conditional quantile with low-dim structure. While neural estimator can make use of such structure, the RLCP baseline employs Euclidean localization in the ambient feature space. The resulting advantages align with the intuition behind classical non-parametric estimation, but it fails to distinguish the effects of "global vs local" from the ability to leveraging structured representations. A controlled comparison with learned-metric or representation-based localization, with any additional fitting cost accounted for, would help clarify the scope of the conclusions. The interpretation of coverage and efficiency should also be clarified, because infinite prediction sets are counted as covered while the reported length summaries are restricted to finite sets.
- Clarity and organization of the paper could be improved. I found the presentation of the mathematical statements unnecessarily difficult to follow. The main text presents complex technical bounds without emphasizing what is fundamentally new over existing methods, and the detailed distinctions from prior work are pushed to the appendix. Moreover, the readability of the paper is affected by slightly heavy notation (e.g., the meaning of changes across sections) and comparison of different error norms and guarantee domains that lacks intuitive clarity.
Questions for authors
See Weaknesses.
5 comments
Revised manuscript: sharp coverage comparisons and experimental controls
Thank you for the detailed review. The revised PDF addresses the four points as follows.
W1. Significance (abstract, introduction, Proposition 3 and Theorem 4). The paper analyzes existing procedures. Proposition 3 shows that an error in one region can cause overcoverage elsewhere, where the fit is exact, of the same order as the average fitting error. The effect persists with a population threshold. Theorem 4 shows a different obstruction for local calibration: the bias depends on variation of the target quantile alone. It can be zero when the conditional score CDF jumps (Remark 20); there is still a finite-sample cost from the local calibration size.
W2. A common loss and its limits (Section 4, Table 1 and Appendices G–H). Propositions 7–8 control the same supremum loss on ambient RLCP's admissible realized neighborhood. Proposition 6 instead concerns root mean square error over covariates. Appendix G proves sharp full-domain supremum rates for quantile estimation and realized coverage of threshold sets with on its stated classes. In particular, on its Lipschitz composition class with , the neural rate up to logs cannot hold uniformly; the supremum lower rate is . Appendix H (Proposition 28) concerns the supremum coverage error for over a fixed interior center's neighborhood, for fixed-kernel ambient RLCP with deterministic and . For , that worst-case error saturates at the rate , while the local-polynomial route attains a faster rate under the same loss. Neither appendix supplies a lower bound for length or a separation from learned localization.
W3. Experimental control and efficiency (Section 5, Table 2; Appendix F.4, Table 5). Both routes now use the same score quantile fitted on half the labels and the same remaining labels for calibration. At , rank-localized RLCP gives RMS error 0.0535 versus 0.0551 for global calibration; the absolute paired mean differences are below 0.002 across four synthetic cells, compared with 0.018–0.035 for ambient RLCP. This shows that giving RLCP access to the learned structure recovers most of the synthetic gain. On Blog, the separate-fold forest has the lowest group error; that is a group diagnostic, not pointwise coverage. Section 5 and Appendix F.1 state that infinite sets count as covered; Figure 3 and Table 4 report finite-only length or half-width summaries. If infinite lengths occur with positive probability, the unconditional mean length is infinite.
W4. Presentation. The abstract and introduction now lead with the two calibration mechanisms. The introduction distinguishes the initial score , the oracle transformation , and its fitted approximation . Table 1 states the loss, domain and conditioning for each guarantee.
Follow-up: A common-loss comparison and the benefit of learned localization
Thank you again for your review. The revised manuscript adds three results since our first response.
W2. Section 4.2 and Corollary 32 compare both routes under one loss, the integrated squared conditional-coverage error, on the same Lipschitz composition class, score and budget . Neural fitting followed by global calibration attains , minimax among threshold rules up to logarithms. Fixed-kernel ambient RLCP with deterministic bandwidths has optimal rate .
Then, Proposition 33 shows that RLCP localized on the rescaled fitted quantile attains the same rate. The rate gap therefore concerns RLCP localized in the original covariates. Under integrated error, this reconciles score adaptation, representation learning and RLCP: the same fitted quantile, used as a correction or as a localization coordinate, reaches the minimax exponent on this class.
W3. Table 2 now runs this construction as stated. Its RMS coverage error is within of the global route in the four synthetic cells, where ambient RLCP is – higher.
Together, these results give a far more complete answer to the question in our title: on this class, the integrated rate is set by learning the conditional quantile with a neural network and by where RLCP localizes.
W1. Significance of the contributions.
Proposition 3 shows that estimation errors in one region can cause overcoverage elsewhere, even where the fitted quantile is exact and calibration uses the population distribution. The average-error term therefore captures an unavoidable effect of global calibration, which additional calibration data alone cannot remove.
The recent preprint by Conrad et al. (2026) establishes RLCP guarantees under continuity of the conditional score CDF in the covariates. Our Theorem 4 replaces this requirement with regularity of the target quantile. This has a meaningful consequence: localization bias can vanish even when the conditional score distribution is discontinuous, while retaining high-probability coverage and length guarantees. Finally, the new Appendix G shows that, under the same compositional assumptions, the integrated neural rate cannot be guaranteed uniformly over covariates by any estimator. This explains why the distinction between integrated and pointwise guarantees reflects a statistical limitation rather than merely different proof techniques.
These results have practical consequences. If the application needs a guarantee over the whole covariate space, local polynomial fitting followed by global calibration has a sharp coverage rate on the classes of Appendix G. If average accuracy is enough, a neural fit can use the structure of the quantile. Direct local calibration is useful when the quantile is hard to fit but changes little across neighborhoods with enough calibration points.
W2. Rate comparison: matching bounds and scope of the conclusions.
The new Appendix G establishes matching uniform minimax bounds for conditional coverage errors, on isotropic Hölder classes and on a Lipschitz composition class. Globally calibrated local-polynomial fitting attains these bounds. The direct comparison is in Propositions 7–8. The upper-bound exponents agree for isotropic Hölder smoothness β≤1; above one, the local-polynomial fit uses higher-order smoothness and has the faster exponent. The new Appendix H turns this into a separation: for a fixed center and deterministic bandwidths, RLCP's worst-case coverage error over the isotropic Hölder class is at least of order N−1/(d+2) for every β>1, with probability bounded away from zero, whereas local-polynomial fitting followed by global calibration attains N−β/(2β+d) up to logarithms. See also our response to Q2 of Reviewer 5rux.
W3. Learned localization, and the reading of coverage and efficiency.
We added learned localization under the same label budget (table in our response to Reviewer 5rux). Once localization uses the learned structure, it can match global calibration in high dimension. As our theory supports, the advantage of adapting the score and calibrating globally is most pronounced when the conditional quantile has smoothness of order higher than one, which RLCP cannot exploit (Appendix H). Note also that RLCP cannot use higher-order kernels or local polynomials, because its conformal validity rests on a covariate-shift argument.
Infinite sets count as covered, and the paper reports their frequency wherever length is summarized: at most 1.1% (synthetic) and 1.3% (real data) for the selected RLCP, with the full shares in the appendix. Length summaries are over finite sets and are stated as such. We can also report median lengths if you think this would help in understanding our results.
W4. Clarity and notation.
We have aligned the notation in the revised manuscript. We will use the remaining space in the main text to present the minimax results and to explain the distinction between pointwise and integrated guarantees.