acceptodds
Under review as a conference paper at ICLR 2027

Scale Adaptation in Conformal Prediction: Efficiency and Coverage Allocation

Abstract

Conformal prediction provides prediction intervals with finite-sample marginal coverage under exchangeability. However, valid intervals can still be unnecessarily wide when prediction errors are asymmetric or prediction difficulty varies across inputs. We study two related procedures to improve interval efficiency. Efficient Conformal Prediction with Flexible Quantile Allocation (ECP-FQ) chooses how to divide the error budget between the lower and upper residual tails. This adjusts interval placement, but gives every input the same width. ECP-FQ+ extends this approach by using estimated prediction-error magnitude to adjust interval centers and widths across inputs. Our main procedure keeps the center at the point prediction and selects one of five exponents to control how strongly width responds to this estimated error magnitude. Both methods select their interval shape on held-out data and use a separate calibration sample to preserve marginal coverage. Under an oracle location–scale model with varying scale and suitable regularity and integrability conditions, we show that slightly weakening proportional width adjustment reduces expected length even after finite-sample calibration. We also analyze imperfect scale estimates and characterize how scale adaptation affects interval length and conditional coverage. Empirically, compared with choosing between constant-width intervals and widths proportional to estimated prediction-error magnitude, allowing intermediate adjustments yields clear width reductions in 18 of 24 simulation settings with learned predictors. At 90% nominal coverage, the five-exponent selector reduces width by 1.8–12.5% in six of ten real dataset–learner settings, with pointwise paired 95% confidence intervals excluding zero; the remaining four settings show small, statistically inconclusive differences.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Conformal prediction, uncertainty quantification, prediction interval efficiency, flexible quantile allocation, scale adaptation, finite-sample coverage.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_c6W9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper suggests two approaches to improve the efficiency of conformal prediction in terms of interval length. While conformal prediction guarantees valid marginal coverage under the exchangeability assumption between calibration and test data, its prediction intervals may be unnecessarily large. The first approach they suggest is Efficient Conformal Prediction with Flexible Quantile Allocation (ECP-FQ). This method chooses how to divide the error budget between the lower and upper residual tails. The second approach they suggest is ECP-FQ+, which uses estimated prediction error magnitude to adjust interval centers and widths across inputs. The parameters that control the interval shape are chosen using held-out data, disjoint from the calibration set used to ensure valid marginal coverage. The authors show empirically and theoretically that interval length reduction is achieved when allowing intermediate adjustments between constant-width intervals and widths proportional to estimated error.

Critical strengths
  1. The paper addresses the important problem of improving the efficiency of conformal prediction in terms of interval length. Indeed, standard conformal prediction methods encounter the limitation of unnecessarily large intervals and adapting interval center and width can lead to more informative intervals.

  2. The proposed methods are straightforward to understand and implement and do not involve substantial computational complexity. The parameters that control the interval shape have a clear interpretation. In particular, the scale exponent γ\gamma intuitively controls the amount of adaptation to estimated error.

  3. The paper provides an extensive and rigorous theoretical analysis of the proposed methods. The paper also provides a comprehensive empirical evaluation, including comparisons with many conformal prediction baselines. The experiments cover both synthetic and real-world datasets, different prediction models, and multiple coverage levels. The authors also provide meaningful ablation studies.

  4. The paper is written concisely and is well organized. The motivation, proposed methods, and theoretical and empirical results are well structured.

Critical weaknesses
  1. The methodological novelty of the paper is quite limited. The proposed methods build on existing ideas in conformal prediction, such as residual normalization and selecting conformity scores using held-out data. In particular, the main approach introduces a scale exponent into the normalized conformity scores and selects it using held-out data. While the theoretical analysis is very interesting and thorough, the proposed method itself does not seem very novel.

  2. The paper includes an extensive evaluation against several conformal prediction baselines. However, comparisons with additional methods specifically designed to optimize prediction interval length, such as CPL and Ad-EffOrt, would further strengthen the empirical evaluation. The authors explicitly note that these methods were not evaluated.

  3. Although the paper introduces both ECP-FQ and ECP-FQ++, the empirical results suggest that most of the improvements arise from scale-exponent selection. In particular, ECP-FQ does not demonstrate clear improvements over fixed signed-tail allocations on real datasets, and jointly optimizing interval centers and widths seems to provide little additional benefit over scale-only adaptation. Thus, I question the practical value of the additional components.

Questions for authors

None

Reviewer_dGMbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how to improve the efficiency of regression prediction intervals while maintaining finite-sample marginal coverage in conformal prediction. The authors propose ECP-FQ, which optimizes interval placement under asymmetric residuals via flexible two-sided miscoverage allocation, and ECP-FQ+, which controls interval centers and input-dependent widths using center-shift and scale-exponent parameters fitted via data splitting. They prove that under location-scale models, partially relaxing full scale normalization strictly reduces expected interval length, and this local improvement persists in expectation over finite calibration samples. They further analyze the impact of scale estimation, parameter tuning errors, and candidate grids. Experiments validate the benefits of intermediate scale exponents and reveal the trade-off between average interval length and conditional reliability.

Critical strengths
  1. The theoretical analysis is quite in-depth, particularly regarding finite-sample efficiency results. Beyond demonstrating at the population level that reducing the scale exponent can enhance efficiency, the authors derive an exact derivative for finite calibration samples (Eq. 9). This shows that, under a correctly specified location-scale model, the local improvement in expected interval length persists after averaging over the calibration sample. Furthermore, the authors analyze the impact of scale estimation error, characterize the conditions for efficiency gains via the residual density at the interval boundaries, and discuss the trade-off between marginal efficiency and conditional coverage.

  2. The experimental design is well-structured, effectively highlighting the contributions of the proposed method. By comparing Γ2\Gamma_2 and Γ5\Gamma_5 within a matched setting, the authors directly evaluate the value of introducing intermediate scale exponents. They also examine center adjustment, scale adjustment, and their joint tuning.

  3. The experiments are comprehensive, with thorough reporting of results. The study encompasses synthetic experiments covering various noise distributions, scale ratios, and sample sizes, as well as applications to five real-world datasets using two different base predictive models, accompanied by a sound analysis of the findings.

Critical weaknesses
  1. The methodological innovation is somewhat limited, as it is closely linked to existing research on prediction interval efficiency optimization. From an algorithmic perspective, the primary "scale-only" approach involves selecting an appropriate scale exponent γ\gamma within the expression Sγ(x,y)=∣y−μ^(x)∣/σ^(x)γS_\gamma(x,y)=|y-\hat\mu(x)|/\hat\sigma(x)^\gamma. Concepts such as balancing the density of residuals at the endpoints of the shortest interval and optimizing interval length subject to marginal coverage constraints have already been explored in the literature. Consequently, the novelty of this paper lies primarily in the theoretical characterization of a specific family of scale exponents, rather than in the development of a general-purpose method for prediction interval optimization. Given existing works such as CPL, EffOrt, Ad-EffOrt, and BoostedCP, the authors should further clarify the independent value of their theoretical findings.

  2. There remains a gap between the core theoretical results and the empirical performance of the actual algorithm. Theorem 3 relies on idealized assumptions, including an exact location-scale structure, known location and scale functions, and a pre-specified exponent, whereas the practical algorithm utilizes estimated parameters μ^,σ^\hat{\mu}, \hat{\sigma} and selects γ\gamma using a finite allocation set. Therefore, local theoretical results do not directly guarantee efficiency improvements when using practical grids or data-driven selection. Although Theorem 7, Theorem 8, and Corollary 9 account for fitting and selection errors, these guarantees rely on additional regularity conditions and, in some experimental settings, fail to provide meaningful assurances of efficiency gains.

  3. Efficiency improvements may come at the cost of reduced conditional reliability. While the method maintains marginal coverage, it can lead to a significant drop in coverage for subpopulations with high noise, which is observed in the experiments. For applications where group fairness or conditional reliability is a priority, such efficiency gains may not be worthwhile. Although Mondrian CP can provide group-specific marginal coverage guarantees for predefined subpopulations, it would be more practically valuable to optimize the scale exponent while controlling subgroup coverage degradation.

  4. Experimental advantages are not universal. In experiments on real-world data with a 90% coverage target, only 6 out of 10 settings showed a clear (significant) reduction in interval width when using Γ5\Gamma_5 compared to Γ2\Gamma_2; furthermore, ECP-FQ did not demonstrate clear benefits from flexible tail allocation. Moreover, existing experiments have yet to show that this method offers a consistent advantage over more flexible interval construction approaches, such as BoostedCP.

Questions for authors
  1. Conformal prediction is usually employed as a post-hoc uncertainty quantification tool for existing predictive models. However, ECP-FQ+ not only requires an additional estimate of the scale (derived from out-of-fold residuals on the training set) but also, when b≠0b \neq 0, results in intervals that are not centered at the original point prediction μ^\hat\mu and may not even contain μ^\hat\mu. While this does not compromise the validity of the CP framework, it indicates that the method is effectively optimizing the interval structure itself, rather than merely calibrating a fixed predictor. If the objective is to enhance interval efficiency while preserving marginal coverage, what is the necessity of restricting the center to the form cb(x)=μ^(x)+bσ^(x)c_b(x) = \hat\mu(x) + b\hat\sigma(x)? A global scalar bb may struggle to accommodate variations in residual distributions across different input locations. Could this partly explain why joint tuning of λ=(b,γ)\lambda=(b,\gamma) does not yield consistent additional gains over scale-only tuning? Could you explain the motivation behind this constrained parameterization and its advantages over more flexible interval construction methods?

See Weaknesses for other concerns.

Reviewer_JGhKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose two conformal prediction methods with marginal coverage guarantees, ECP-FQ and ECP-FQ+. Despite their names, the two methods consider different interval families that are not nested. ECP-FQ constructs asymmetric prediction intervals of the form [μ^(x)+qL,μ^(x)+qU][\widehat{\mu}(x)+q_L,\widehat{\mu}(x)+q_U], where the lower and upper residual quantiles are selected to minimize interval width qU−qLq_U-q_L subject to a marginal coverage constraint. ECP-FQ+ instead constructs intervals of the form μ^(x)+bσ^(x)±qσ^(x)γ\widehat{\mu}(x)+b\widehat{\sigma}(x)\pm q\widehat{\sigma}(x)^\gamma, where σ^\widehat{\sigma} is an estimated conditional error scale. The method selects the center shift bb and scale exponent γ\gamma to minimize estimated average interval width. The main implementation fixes b=0b=0 and selects γ\gamma, interpolating between standard split conformal prediction based on absolute residuals (γ=0\gamma=0) and normalized absolute residuals (γ=1\gamma=1). Both methods select interval parameters on a held-out dataset, followed by independent conformal calibration to guarantee marginal coverage. The authors show that partially normalizing interval widths can reduce expected length under an oracle location–scale model and provide bounds accounting for estimation and selection errors.

Critical strengths

The authors consider two standard split conformal methods, based on absolute residuals and normalized absolute residuals, and propose a simple extension that minimizes estimated interval width over a parameterized family containing both methods as special cases.

Critical weaknesses

The paper is poorly written and organized. The authors introduce ECP-FQ and ECP-FQ+ as closely related methods, even though they consider different interval families that are not nested, as the authors acknowledge. The main contribution appears to be the relatively simple scale-exponent selection procedure in ECP-FQ+, but this is obscured by the presentation of multiple additional procedures, theoretical results, and extensive auxiliary analyses. The paper would benefit from a more focused presentation that clearly identifies the main contribution and distinguishes it from secondary results.

The novelty of these methods is unclear. The main methodological contribution appears to be the proposal of two low-dimensional parameterized families of conformal prediction intervals, with parameters selected by minimizing estimated interval width on a held-out dataset, followed by independent conformal calibration. However, selecting conformity scores to minimize prediction interval length is already well studied, including by Yang and Kuchibhotla (2025). Moreover, the quantile-width optimization underlying ECP-FQ is closely related to the shortest-interval construction studied by Chernozhukov, Wüthrich, and Zhu (2021), although their method uses conditional outcome quantiles rather than marginal residual quantiles. ECP-FQ+ essentially interpolates between standard constant-width and normalized conformal prediction intervals by selecting a scale exponent. It is unclear what is methodologically new beyond the particular parameterizations considered here.

The theoretical contribution also appears limited. The marginal coverage guarantee in Theorem 1 follows directly from standard split conformal prediction, since the score function is selected using an independent sample. Theorem 3 shows that, under an oracle location–scale model, slightly weakening scale normalization reduces expected interval length. However, this improvement comes at the expense of conditional coverage in higher-noise regions. The tradeoff between minimizing expected interval length under marginal coverage and imposing stronger conditional coverage requirements has already been studied by Kiyani et al. (2024) and Xie et al. (2024). The authors should clarify what new theoretical insight their result provides beyond this literature. The oracle inequalities in Section 4.4 rely on standard finite-class selection arguments, closely related to Yang and Kuchibhotla (2025). I am not convinced that these results constitute a sufficiently substantial theoretical contribution for ICLR.

The empirical results do not convincingly establish an advantage over existing methods. The clearest improvements are relative to standard and normalized split conformal prediction, while comparisons with more competitive methods, including BoostedCP (Xie et al., 2024), are mixed. Important baselines, including CPL (Kiyani et al., 2024) and Ad-EffOrt (Le Bars and Humbert, 2025), are missing. Given that the main methodological contribution appears to be the optimization of conformity scores to reduce prediction interval length, comprehensive comparisons with existing length-optimization methods are essential.

References:

  • Yang and Kuchibhotla (2025), Selection and Aggregation of Conformal Prediction Sets, JASA.
  • Chernozhukov, Wüthrich, and Zhu (2021), Distributional Conformal Prediction, PNAS.
  • Kiyani, Pappas, and Hassani (2024), Length Optimization in Conformal Prediction, NeurIPS.
  • Xie, Barber, and Candès (2024), Boosted Conformal Prediction Intervals, NeurIPS.
  • Le Bars and Humbert (2025), On Volume Minimization in Conformal Regression, ICML.
Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.