acceptodds
Under review as a conference paper at ICLR 2027

Adapt the Score or Localize the Calibration? A Finite-Sample Comparison for Conditional Conformal Prediction

Abstract

Given a fixed nonconformity score and target coverage , should extra labels transform the score before global calibration, or calibrate it locally? Under score regularity, we bound both routes' errors in conditional coverage and oracle-relative length, with high probability over calibration, given the training data. Subtracting an estimated conditional score quantile before global calibration gives a three-term error bound, valid simultaneously wherever the score assumptions hold: the pointwise quantile-estimation error, its average over covariates, and a calibration fluctuation. We prove that this average term is needed: an error confined to one region causes over-coverage elsewhere, even with population calibration. For randomly localized conformal prediction (RLCP), we bound localization bias by the target quantile's variation alone; the conditional score distribution need not be continuous in the covariate. These bounds hold over the realized centre's neighbourhood and account for the smaller effective calibration sample. We apply the global bounds to local-polynomial and deep-network quantile estimators. Neural estimators can exploit additive or compositional structure, whereas our RLCP bound depends on neighbourhood data mass in the full covariate space. The bounds favour quantile learning when the fitting data can capture this structure; local calibration remains useful when the fit is poor but nearby data are plentiful. Simulations and eight real datasets illustrate the global route's gains; low-dimensional simulations show when localization is preferable.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Conformal prediction, conditional coverage, score adaptation, localized calibration, quantile regression, finite-sample guarantees.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ybLYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper compares two strategies for improving conditional coverage given a fixed initial score and an additional labeling budget: global score adaptation (via learning conditional quantiles) and local calibration (via RLCP). The authors derive bounds for high-probability conditional coverage and oracle-relative prediction-set length error, establish a lower bound on global propagation of localized estimation errors, weaken the covariate regularity assumption required for existing RLCP bounds, and evaluate both approaches through finite-sample convergence-rate bounds and empirical experiments.

Critical strengths
  1. Good experimental control. By fixing the initial score, referenced Oracle, and a common additional labeled-data budget, the study reduces potential confounding factors arising from differences in base models.
  2. Meaningful theoretical refinements. Proposition 3 demonstrates how the failure of local estimation can contaminate other regions through global threshold, and Theorem 4 relaxes the regularity requirements of RLCP to depend on local variations of the target quantile, making it more applicable to broader distribution classes.
  3. Comprehensive experimental analysis. The study employs synthetic data with known conditional CDFs and report useful metrics, (e.g., lower-tail coverage, length errors, and infinite prediction sets). The cases where localization is preferred are also included in the low-dim and label allocation experiments.
Critical weaknesses
  1. The significance of the incremental contributions is unclear. The two key approaches or routes (score adaption and RLCP) are both existing methods and some of the corresponding theoretial results have been established. While the authors identify several technical differences from prior work, it is still unclear whether these refinements lead to qualitatively new statistical insights or practically meaningful consequences. The authors should more clearly explain the significance of the genuinely new conclusions and explain why these conclusions represent a substantive advancement, rather than just serving as a specialization of existing error analyses or a weakening of assumptions. Otherwise, the contributions that are merely marginally incremental is unlikely to be convincing.
  2. The comparison of convergence rates does not separate the two approaches clearly. The neural network controls the L2 error, local polynomial controls uniformly over XX, whereas RLCP focus on a neighborhood of the realized centre. The comparable results are mainly for particular functional classes and methods. The paper does not establish general matching bounds or a minimax separation between the two approaches. The authors should clarify the scope of the comparative conclusions and explain what new statistical insights (in particular, beyond those suggested by classical nonparametric estimation theory) can be drawn from these rate comparisons.
  3. Limited empirical conclusions. The main high-dim synthetic experiment uses smooth conditional quantile with low-dim structure. While neural estimator can make use of such structure, the RLCP baseline employs Euclidean localization in the ambient feature space. The resulting advantages align with the intuition behind classical non-parametric estimation, but it fails to distinguish the effects of "global vs local" from the ability to leveraging structured representations. A controlled comparison with learned-metric or representation-based localization, with any additional fitting cost accounted for, would help clarify the scope of the conclusions. The interpretation of coverage and efficiency should also be clarified, because infinite prediction sets are counted as covered while the reported length summaries are restricted to finite sets.
  4. Clarity and organization of the paper could be improved. I found the presentation of the mathematical statements unnecessarily difficult to follow. The main text presents complex technical bounds without emphasizing what is fundamentally new over existing methods, and the detailed distinctions from prior work are pushed to the appendix. Moreover, the readability of the paper is affected by slightly heavy notation (e.g., the meaning of SS changes across sections) and comparison of different error norms and guarantee domains that lacks intuitive clarity.
Questions for authors

See Weaknesses.

5 comments

W1. Significance of the contributions.

Proposition 3 shows that estimation errors in one region can cause overcoverage elsewhere, even where the fitted quantile is exact and calibration uses the population distribution. The average-error term therefore captures an unavoidable effect of global calibration, which additional calibration data alone cannot remove.

The recent preprint by Conrad et al. (2026) establishes RLCP guarantees under continuity of the conditional score CDF in the covariates. Our Theorem 4 replaces this requirement with regularity of the target quantile. This has a meaningful consequence: localization bias can vanish even when the conditional score distribution is discontinuous, while retaining high-probability coverage and length guarantees. Finally, the new Appendix G shows that, under the same compositional assumptions, the integrated neural rate cannot be guaranteed uniformly over covariates by any estimator. This explains why the distinction between integrated and pointwise guarantees reflects a statistical limitation rather than merely different proof techniques.

These results have practical consequences. If the application needs a guarantee over the whole covariate space, local polynomial fitting followed by global calibration has a sharp coverage rate on the classes of Appendix G. If average accuracy is enough, a neural fit can use the structure of the quantile. Direct local calibration is useful when the quantile is hard to fit but changes little across neighborhoods with enough calibration points.

W2. Rate comparison: matching bounds and scope of the conclusions.

The new Appendix G establishes matching uniform minimax bounds for conditional coverage errors, on isotropic Hölder classes and on a Lipschitz composition class. Globally calibrated local-polynomial fitting attains these bounds. The direct comparison is in Propositions 7–8. The upper-bound exponents agree for isotropic Hölder smoothness β≤1\beta\leq1; above one, the local-polynomial fit uses higher-order smoothness and has the faster exponent. The new Appendix H turns this into a separation: for a fixed center and deterministic bandwidths, RLCP's worst-case coverage error over the isotropic Hölder class is at least of order N−1/(d+2)N^{-1/(d+2)} for every β>1\beta>1, with probability bounded away from zero, whereas local-polynomial fitting followed by global calibration attains N−β/(2β+d)N^{-\beta/(2\beta+d)} up to logarithms. See also our response to Q2 of Reviewer 5rux.

W3. Learned localization, and the reading of coverage and efficiency.

We added learned localization under the same label budget (table in our response to Reviewer 5rux). Once localization uses the learned structure, it can match global calibration in high dimension. As our theory supports, the advantage of adapting the score and calibrating globally is most pronounced when the conditional quantile has smoothness of order higher than one, which RLCP cannot exploit (Appendix H). Note also that RLCP cannot use higher-order kernels or local polynomials, because its conformal validity rests on a covariate-shift argument.

Infinite sets count as covered, and the paper reports their frequency wherever length is summarized: at most 1.1% (synthetic) and 1.3% (real data) for the selected RLCP, with the full shares in the appendix. Length summaries are over finite sets and are stated as such. We can also report median lengths if you think this would help in understanding our results.

W4. Clarity and notation.

We have aligned the notation in the revised manuscript. We will use the remaining space in the main text to present the minimax results and to explain the distinction between pointwise and integrated guarantees.

AuthorsOpenReview ↗

Revised manuscript: sharp coverage comparisons and experimental controls

Thank you for the detailed review. The revised PDF addresses the four points as follows.

W1. Significance (abstract, introduction, Proposition 3 and Theorem 4). The paper analyzes existing procedures. Proposition 3 shows that an error in one region can cause overcoverage elsewhere, where the fit is exact, of the same order as the average fitting error. The effect persists with a population threshold. Theorem 4 shows a different obstruction for local calibration: the bias depends on variation of the target quantile alone. It can be zero when the conditional score CDF jumps (Remark 20); there is still a finite-sample cost from the local calibration size.

W2. A common loss and its limits (Section 4, Table 1 and Appendices G–H). Propositions 7–8 control the same supremum loss on ambient RLCP's admissible realized neighborhood. Proposition 6 instead concerns root mean square error over covariates. Appendix G proves sharp full-domain supremum rates for quantile estimation and realized coverage of threshold sets with S(x,y)=yS(x,y)=y on its stated classes. In particular, on its Lipschitz composition class with d=2k≥4d=2^k\ge4, the neural L2(PX)L^2(P_X) rate N−1/4N^{-1/4} up to logs cannot hold uniformly; the supremum lower rate is (log⁡N/N)1/(d+2)(\log N/N)^{1/(d+2)}. Appendix H (Proposition 28) concerns the supremum coverage error for S(x,y)=yS(x,y)=y over a fixed interior center's neighborhood, for fixed-kernel ambient RLCP with deterministic hN→0h_N\to0 and NhNd→∞Nh_N^d\to\infty. For β>1\beta>1, that worst-case error saturates at the rate N−1/(d+2)N^{-1/(d+2)}, while the local-polynomial route attains a faster rate under the same loss. Neither appendix supplies a lower bound for length or a separation from learned localization.

W3. Experimental control and efficiency (Section 5, Table 2; Appendix F.4, Table 5). Both routes now use the same score quantile fitted on half the labels and the same remaining labels for calibration. At (d,N)=(100,32000)(d,N)=(100,32000), rank-localized RLCP gives RMS error 0.0535 versus 0.0551 for global calibration; the absolute paired mean differences are below 0.002 across four synthetic cells, compared with 0.018–0.035 for ambient RLCP. This shows that giving RLCP access to the learned structure recovers most of the synthetic gain. On Blog, the separate-fold forest has the lowest group error; that is a group diagnostic, not pointwise coverage. Section 5 and Appendix F.1 state that infinite sets count as covered; Figure 3 and Table 4 report finite-only length or half-width summaries. If infinite lengths occur with positive probability, the unconditional mean length is infinite.

W4. Presentation. The abstract and introduction now lead with the two calibration mechanisms. The introduction distinguishes the initial score SS, the oracle transformation S~\widetilde S, and its fitted approximation S^\widehat S. Table 1 states the loss, domain and conditioning for each guarantee.

AuthorsOpenReview ↗

Follow-up: A common-loss comparison and the benefit of learned localization

Thank you again for your review. The revised manuscript adds three results since our first response.

W2. Section 4.2 and Corollary 32 compare both routes under one loss, the integrated squared conditional-coverage error, on the same Lipschitz composition class, score and budget NN. Neural fitting followed by global calibration attains N−1/2(log⁡N)3N^{-1/2}(\log N)^3, minimax among threshold rules up to logarithms. Fixed-kernel ambient RLCP with deterministic bandwidths has optimal rate N−2/(d+2)N^{-2/(d+2)}.

Then, Proposition 33 shows that RLCP localized on the rescaled fitted quantile attains the same N−1/2(log⁡N)3N^{-1/2}(\log N)^3 rate. The rate gap therefore concerns RLCP localized in the original covariates. Under integrated error, this reconciles score adaptation, representation learning and RLCP: the same fitted quantile, used as a correction or as a localization coordinate, reaches the minimax exponent on this class.

W3. Table 2 now runs this construction as stated. Its RMS coverage error is within 0.00150.0015 of the global route in the four synthetic cells, where ambient RLCP is 0.0180.018–0.0350.035 higher.

Together, these results give a far more complete answer to the question in our title: on this class, the integrated rate is set by learning the conditional quantile with a neural network and by where RLCP localizes.

Reviewer_WYu5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper compares two approaches to improving conditional coverage under a fixed base score and additional label budget: learning a score transformation followed by global calibration, and randomly localized conformal prediction (RLCP). For the first approach, the authors derive finite-sample bounds involving pointwise estimation error, average estimation error, and calibration variability, and show why the average error term is necessary. For RLCP, they control localization bias through variation in the target conditional quantile, without requiring the full conditional score distribution to be continuous in the covariates. Simulations and experiments on eight regression datasets illustrate the strengths and limitations of both approaches.

Critical strengths

The analysis explains how global calibration propagates estimation errors across regions. Beyond providing an upper bound, the paper shows that estimation errors in one region can affect coverage elsewhere through the shared calibration threshold, even where the score is estimated correctly. This establishes the necessity of the average error term and provides a useful explanation of the method’s behavior.

The RLCP analysis requires weaker regularity in the covariates. The localization bias depends on variation in the target conditional quantile rather than continuity of the entire conditional score distribution. A concrete example illustrates this distinction. The analysis controls both conditional coverage error and length error relative to the fixed-score oracle.

The theoretical comparison is supported by a reasonably thorough empirical study. The experiments use a common base score and label budget, cover multiple dimensions and sample sizes, and include repeated runs and eight real datasets. They also report settings where localization performs better or the chosen learner struggles, supporting a balanced discussion of the tradeoffs.

Critical weaknesses

The scope of the comparison limits its broader practical implications. The main local baseline uses a fixed distance in the original feature space, without learning a representation or metric. The results therefore establish a comparison with ambient-space RLCP, but do not show whether the same conclusions hold for local calibration methods that can exploit structure. The authors acknowledge this limitation, but it still narrows the practical guidance.

The theoretical bounds do not directly establish a practical performance ordering. The results involve different norms and domains of validity, and some rates rely on idealized estimator conditions. The analysis clarifies the sources of error, but it remains unclear how well these quantities predict which approach will perform better at a given finite sample size.

The experiments do not fully separate the sources of the observed gains. The main synthetic setting combines smoothness and compositional structure, while some low-dimensional results depend on the learner and its hyperparameter range. RLCP bandwidth selection also uses small validation sets on some datasets. Further analysis would help distinguish gains from structural learning from differences caused by the particular learners or tuning procedures.

Questions for authors
  1. Could you clarify the precise theoretical advances over the closest score-transformation and RLCP analyses? In particular, which settings are newly covered by the necessity result for average estimation error and the quantile-based localization bound, and what do these results add to our understanding of the finite-sample tradeoff?

  2. How sensitive are the main empirical conclusions to the learner configuration and RLCP bandwidth selection?

3 comments

Q1. Theoretical advances over the closest score-transformation and RLCP analyses.

Proposition 3 shows that an average error term, as in Laplante (2026), is necessary. Theorem 4 replaces the regularity of the full conditional score CDF required by Conrad et al. (2026) with that of the target quantile, and holds with high probability, where Min et al. (2026) control averaged coverage. See also our response to W1 of Reviewer ybLY.

Two new appendices sharpen the trade-off: the uniform rates of Proposition 7 are minimax optimal (Appendix G), and RLCP saturates at first-order smoothness (Appendix H). See our responses to W2 of Reviewer ybLY and Q2 of Reviewer 5rux.

Q2. Sensitivity to the learner configuration and to RLCP bandwidth selection.

Whether the conclusions extend to localization that exploits structure is addressed by the new learned-localization baselines (table in our response to Q1 of Reviewer 5rux).

  • Network. Halving the hidden width barely changes the coverage error. Without early stopping, the error rises markedly, above the raw-score level in the synthetic cell tested.
  • RLCP bandwidth. Appendix F already varies it: neither the selector (Figure 3) nor the kernel changes any ordering, and on real data the best bandwidth in hindsight reaches the network's group error on four of eight datasets (Figure 4).
AuthorsOpenReview ↗

Revised manuscript: theoretical advances and sensitivity checks

Thank you for the careful review. The revision addresses both questions in the main text and adds the supporting checks in Appendix F.4.

Q1. What the theory adds (Introduction, Sections 2–3). Proposition 3 (p. 4) shows that an average fitting-error term is necessary: an error confined to one region can cause overcoverage of that order elsewhere, where the fitted score is exact, even at the population calibration threshold. An accurate local fit and a larger calibration sample cannot remove that effect on their own. This is a necessity result at a single coverage level, rather than another pointwise-plus-averaged upper bound. Theorem 4 (p. 5) controls realized RLCP coverage and oracle-relative length with high probability given the center, using variation of the target quantile to bound the bias. Unlike a condition on the entire conditional score CDF, this allows that CDF to jump across covariates. In Remark 20 the quantile is constant, the CDF jumps, and the localization bias is zero. The related-work discussion distinguishes this result from the corresponding full-CDF regularity bound and from coverage bounds averaged over calibration and localization.

Q2. Learner and bandwidth sensitivity (Section 5 and Appendix F.4). At (d,N)=(100,8000)(d,N)=(100,8000), halving the network width changes RMS coverage error from 0.0678 to 0.0690; taking epoch 200 rather than the validation-selected checkpoint raises it to 0.1064. The corresponding group errors on Blog are 0.0463, 0.0485 and 0.0823. The ambient mass selector uses calibration covariates, which is outside the fixed-bandwidth validity result. Computing the radii on the separate selection fold instead changes the paired mean error by −0.0001±0.0002-0.0001\pm0.0002 in the synthetic cell and +0.005±0.010+0.005\pm0.010 on Blog. Increasing that selection fold from N/8N/8 to N/4N/4 changes the Blog error by −0.002±0.009-0.002\pm0.009.

Structure versus calibration. Table 2 (p. 10) and the paired intervals in Table 5 show that RLCP using the same fitted quantile and calibration fold comes within 0.002 in mean RMS error of the global route in all four synthetic cells, compared with a gap of 0.018–0.035 for ambient RLCP. On Blog, the separate-fold forest has the lowest group error. Figure 8 gives a different regime in low dimension, where changing how many labels go to fitting can change the comparison. Section 4 and Table 1 distinguish the common neighborhood supremum comparison (Propositions 7–8) from the neural RMS guarantee. These bounds identify the costs of each route; they do not rank the fitted procedures at a given finite budget.

Reviewer_WYu5OpenReview ↗

Response to Author

Thank author for the clarification and the additional experiments. The learned-localization comparison directly addresses my main empirical concern. Overall, I am maintaining my initial assessment.

Reviewer_5ruxOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a finite-sample comparison of score adaptation followed by global calibration and localized calibration for improving conditional coverage in conformal prediction. Under a shared score and labeled-data budget, the authors derive high-probability bounds on conditional coverage and oracle-relative prediction-set length. They show that global calibration propagates local estimation errors through a necessary average-error term, while RLCP's localization bias depends only on variation in the target conditional quantile. Theoretical rate comparisons and experiments suggest that score adaptation benefits from smoothness and low-dimensional structure, whereas localized calibration can be preferable when quantile estimation is inaccurate and sufficient local data are available.

Critical strengths
  1. This paper provides some interesting theoretical insights. The necessity of the average-error term in global calibration and the weaker quantile-smoothness condition for RLCP are particularly useful. The experiments also cover settings where each approach performs better.
Critical weaknesses
  1. The methodological novelty is quite limited. Both approaches are well established, and several convergence results build on existing quantile estimation theory. The main contributions are refinements of theoretical guarantees rather than fundamentally new methods.

  2. The examined experimental baselines are very limited. The experiments mainly compare learned score adaptation against RLCP operating in the original covariate space. This makes it difficult to tell whether the gains come from score adaptation itself or from the limitations of ambient-space localization. Stronger baselines such as PLCP (ICML 2024) or forest-based adaptive calibration would help clarify this. It would also be useful to compare against CPCP (ICML 2026), which directly targets conditional coverage error through density-weighted quantile regression. While the paper already covers a special case of RCP, these additional comparisons would make the empirical conclusions more convincing.

  3. The theoretical bounds for the two approaches are stated in different norms and domains, making direct comparisons less straightforward.

Questions for authors
  1. Could the authors include comparisons with learned-localization methods such as PLCP or forest-based adaptive calibration, as well as CPCP for score adaptation? This would help clarify whether the observed gains come from global calibration itself or from the ability to exploit learned structure.

  2. The neural score-adaptation bounds are stated in L2(PX)L^2(P_X), while the RLCP guarantees hold within a realized neighborhood. Could the authors provide a more directly comparable theoretical analysis, or further explain how these different guarantees support the claimed advantages of score adaptation?

6 comments

W1. Limited methodological novelty.

The paper is theoretical and introduces no new method. It addresses a question we believe is central to the theory of CP: how to achieve approximate conditional coverage given a black-box score SS and a fresh batch of labeled data. The oracle threshold is the conditional quantile qS1−α(x)q_S^{1-\alpha}(x). One route adapts the score so that this quantile becomes approximately constant and can be estimated by global calibration; the other estimates it directly through local calibration. This dichotomy is the conceptual cornerstone of our work. Our results make precise the obstacles and guarantees of each route. Ambient RLCP does suffer from the curse of dimensionality, and the references you point out are ways to overcome it, but their theoretical analysis requires handling a learned representation and is left to future work.

Q1. Do the gains come from global calibration or from learned structure? (forest-based, CPCP baselines)

We added LCP-RF, CPCP, and two learned localizations of RLCP, each fitted on N/2N/2 labels and calibrated on the other N/2N/2: on the rank of RCP's own fitted quantile (same fit, same calibration fold), and on the rank of a separately learned regression of the score. All use the same score, NN labels and 20 paired replications. In the synthetic model, the population mean and quantile of the score are proportional, so both learned ranks target the same ordering. Once localization uses a learned representation, the two routes perform comparably in these experiments.

MethodSynthetic (20,8000)(20, 8000)(50,8000)(50, 8000)(100,8000)(100, 8000)(100,32000)(100, 32000)blogMEPS 19
RCP (the paper's route)0.05170.06080.06780.05510.0463 ± 0.00730.0371 ± 0.0037
RLCP, localized on RCP's fitted quantile0.05220.06060.06710.05350.0510 ± 0.00860.0388 ± 0.0037
CPCP0.06160.07460.07950.07620.0799 ± 0.02190.0502 ± 0.0113
RLCP, ambient covariates0.08710.08580.08590.08630.1030 ± 0.00850.0800 ± 0.0098
RLCP, localized on a learned regression of the score0.05210.06060.06950.05420.0415 ± 0.00700.0388 ± 0.0038
Forest-localized calibration (LCP-RF, independent forest)0.07620.08160.08340.08310.0352 ± 0.00430.0392 ± 0.0043
Raw score, split conformal0.08770.08590.08590.08630.1442 ± 0.01380.0929 ± 0.0085

Synthetic cells (d,N)(d, N) of Eq. (20): RMS error of the exact conditional coverage. Real data: RMS group-coverage error on a common kk-means partition, ± 95% half-width. Bold: lowest mean, and entries not separated from it by the paired 95% interval.

One caveat on the learned-localization rows: a guarantee conditional on a learned representation, ϕ(X)=ϕ(x)\phi(X)=\phi(x), does not imply coverage conditional on X=xX=x. The transfer requires controlling the discrepancy between the two conditional score CDFs at the relevant thresholds. The synthetic columns measure coverage conditional on X=xX=x, so the transfer holds empirically here; establishing it is left to future work.

Q2. Comparability of the L2(PX)L^2(P_X) neural bounds and the neighborhood RLCP bounds.

The directly comparable analysis requested is already in the submission: Propositions 7–8 bound both routes in the same norm, on the same domain (RLCP's admissible neighborhood), for the same score, oracle and label budget. Our calibration theorems are themselves pointwise; the L2(PX)L^2(P_X) norm enters only through the neural estimation guarantee used to instantiate them. The neural guarantee addresses how learning the structure of the conditional score quantile can improve integrated conditional-coverage error. The new Appendix G shows that, under the same compositional assumptions, the integrated neural rate cannot be guaranteed uniformly over covariates by any estimator. Appendix G also proves that the uniform rates of Proposition 7 are minimax optimal, including logarithmic factors, on isotropic Hölder classes and on a Lipschitz composition class. The new Appendix H gives a matching lower bound for Proposition 8: for Hölder smoothness β>1\beta>1 and deterministic bandwidths, RLCP's worst-case coverage error saturates at N−1/(d+2)N^{-1/(d+2)}, where Proposition 7 attains N−β/(2β+d)N^{-\beta/(2\beta+d)} up to logarithms.

I sincerely thank the authors for the detailed rebuttal and substantial additional results. I appreciate the new learned-localization baselines and the theoretical developments in Appendices G and H. In particular, the minimax lower bounds help clarify the distinction between integrated and uniform guarantees, while the new experiments provide a more balanced comparison.

However, two main concerns remain:

  1. Empirical interpretation. The new results show that learned-localization methods can perform comparably to RCP, suggesting that the original gains largely arise from exploiting learned structure rather than global calibration itself. I encourage the authors to explicitly acknowledge this distinction and incorporate these results into the manuscript.

  2. Scope of theoretical contributions. While Appendix H establishes a meaningful separation, it concerns fixed-kernel RLCP under specific bandwidth and loss assumptions, rather than a general separation between score adaptation and learned localization. The neural L2L^2 guarantees also remain not directly comparable to the RLCP guarantees.

Overall, I recognize the meaningful improvements and have decided to raise my rating to 2. Nevertheless, my concerns about the scope and significance of the theoretical contributions remain, preventing me from recommending acceptance.

AuthorsOpenReview ↗

Revised manuscript: shared-quantile ablation and scope of the guarantees

Thank you for the follow-up. We incorporated the new comparisons into Section 5 and Table 2 (p. 10) and added the protocols and paired intervals in Appendix F.4 and Table 5.

Does the gain come from learning or from global calibration? In the four synthetic cells, both routes use the same fitted score quantile and the same calibration observations. One subtracts the fitted values and calibrates globally; the other keeps the original score and runs RLCP on the rank of that fit. The paired RMS differences (rank RLCP minus global) are +0.0005±0.0007+0.0005\pm0.0007, −0.0002±0.0005-0.0002\pm0.0005, −0.0007±0.0006-0.0007\pm0.0006, and −0.0016±0.0004-0.0016\pm0.0004 (95% half-widths). The gap between ambient RLCP and the global route is 0.018–0.035. Giving RLCP the same fit therefore recovers most of the gain in these cells, although its mean minimum evaluated coverage is lower. Synthetic coverage is evaluated at the original covariates; it is not inferred from coverage conditional on the learned rank.

On Blog, shared-fit RLCP has higher group error by 0.0048±0.00390.0048\pm0.0039; the separate-fold forest has the lowest mean group error. On MEPS the paired interval for shared-fit RLCP includes zero. Table 2 also reports RLCP on the rank of a separately fitted mean score and our common-score CPCP implementation. The rank-based methods use the half-and-half split, while CPCP uses a 40/40/20 split. We did not evaluate PLCP. The updated supplementary archive includes the code and records that regenerate Table 2.

Which guarantees can be compared? Section 4 and Table 1 (p. 7) show that Propositions 7–8 bound the same supremum loss on ambient RLCP's admissible realized neighborhood. The neural guarantee in Proposition 6 instead controls L2(PX)L^2(P_X) error. Appendix G gives full-domain supremum lower bounds for quantile estimation and coverage of threshold sets with S(x,y)=yS(x,y)=y; on its Lipschitz composition class the faster integrated neural rate cannot hold uniformly. It does not rank calibration methods. Appendix H, Proposition 28 treats supremum coverage error for S(x,y)=yS(x,y)=y on a fixed interior center's neighborhood, for fixed-kernel ambient RLCP and deterministic hN→0h_N\to0 with NhNd→∞Nh_N^d\to\infty. For β>1\beta>1, the worst-case rate saturates at N−1/(d+2)N^{-1/(d+2)} in this setting, while the local-polynomial route has a faster upper rate for the same loss. This does not establish a separation from learned localization or cover data-driven bandwidths.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.