acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Quantification in Non-Stationary Time Series via Adaptive Conformal Regression and Reinforcement Learning

Abstract

Conformal prediction (CP) provides rigorous distribution-free uncertainty quantification, yet its effectiveness in time-series forecasting is hindered by non-stationarity and data dependencies. Recent adaptive CP methods mitigate distributional shifts by dynamically calibrating intervals based on recent residuals and adaptive weighting strategies. However, they remain limited by outlier sensitivity, undetected systematic biases, and the decoupling of calibration from model learning. In this work, we introduce CORE that establishes a mutual feedback loop between reinforcement learning (RL) and conformal prediction. By reformulating conformal regression as a sequential decision-making process, CORE utilizes RL exploration to better cover uncertain or outlier regions, adapts calibration through exploration feedback, and designs uncertainty-guided rewards, enabling dynamically improved interval quality through policy interaction. We provide theoretical analysis and conduct extensive empirical experiments to validate its effectiveness across 9 time-series standard datasets. The results demonstrate that CORE achieves superior validity and adaptivity, consistently outperforming 6 state-of-the-art baselines, with solid coverage rates and averaged 2.34% improvement in interval length.

Primary area:
learning on time series and dynamical systems
Keywords:
uncertainty quantification, conformal prediction, time-series, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8NS5OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper formulate the adaptive conformal prediction for non-stationary time series procedure as a RL problem. It incorporates prediction accuracy, interval width and coverage into reward function. And then use a PPO-based actor-critic model to generate quantile predictions. They also provide theoretical coverage and efficiency guarantees, which is still questionable.

Critical strengths
  1. This is an important problem in uncertainty quantification for non-stationary time series. It is interesting to integrate prediction and adaptive conformal calibration through a feedback mechanism.
  2. The experiments are relatively comprehensive, covering nine time-series datasets and six adaptive conformal baselines.
Critical weaknesses
  1. The necessity of RL is not sufficiently justified. Actions are quantile predictions and do not influence the underlying dynamics, that is the MDP state. It seems like a contextual bandit problem. It is unclear why this problem requires PPO rather than supervised or online quantile regression.
  2. The abstracts claims that RL exploration improves coverage in uncertain or outlier regions and helps address systematic prediction biases. However, the paper does not clearly explain how exploring different quantile predictions achieves these goals. Moreover, the experiments do not directly evaluate outlier robustness or systematic bias correction.
  3. Algorithm 1 states that the actor is updated using quantile regression and behavior cloning losses. But it does not specify how the PPO objective is incorporated. They also should clarify how the proposed rewards and the full action vector are used in policy optimization.
  4. In Algorithm 1, the conformity-score buffer is updated using the current observation $y_t$ (Step 7) before the prediction interval for the same time step is constructed (Steps 12–13). Then, the reported coverage rates may be affected by data leakage.
  5. The proof of Proposition 1 appears to infer convergence of the prediction error from a gradient-norm convergence bound, but this inference is not adequately justified. In addition, the step-size choice in Appendix D.2 is inconsistent with the summability condition assumed in Proposition 1. The coverage proof in Appendix D.1 also relies on a questionable bound for the empirical CDF increment. Taken together, these issues raise concerns about the claimed coverage and interval-efficiency guarantees.
Questions for authors

See the weeknesses.

7 comments

W1: Motivation for RL and Distinction from Contextual Bandits and Quantile Regression

We thank the reviewer for raising important questions about the necessity of RL and the validity of our problem formulation. We agree that CORE's quantile predictions do not alter the underlying time-series dynamics. Our motivation for RL is therefore not to control the external environment, but to extend adaptation beyond conventional conformal calibration to the predictor itself. While traditional adaptive CP methods primarily adjust conformity thresholds or miscoverage levels, CORE incorporates prediction accuracy, interval length, and coverage reliability into policy optimization, establishing a feedback loop between prediction learning and adaptive calibration.

MDP Formulation vs. Contextual Bandits. The key distinction lies between the external time-series dynamics and the internal calibration state. In CORE, the actor observes a historical feature window ot=xt−w+1o_t=x_{t-w+1} and produces a quantile action vector at=at(q)q∈Q\mathbf a_t=\\{a_t^{(q)}\\}_{q\in \mathcal{Q}}. Although the external data dynamics are independent of these actions, the prediction–calibration system maintains a conformity-score buffer, recent miscoverage history, and an adaptive significance level αt\alpha_t. Once yty_t is observed, the current action determines the conformity score and coverage outcome, which update the calibration buffer and αt+1\alpha_{t+1} (Eq. 4). These updates subsequently affect conformal thresholds, prediction intervals, and future rewards. Thus, the sequential dependence arises not from controlling the underlying time series, but from how current predictions influence future interval quality through the internal calibration mechanism. We will clarify this distinction in Section 3.1, explicitly separating the actor's observations from the full internal state of the prediction and calibration system.

RL vs. Supervised or Online Quantile Regression. The main difference concerns the optimization objective. Supervised quantile regression, including its online variants, typically minimizes prediction errors through the pinball loss at individual time steps. CORE retains quantile regression as an auxiliary objective while additionally incorporating accuracy, interval efficiency, and realized coverage into a unified reward. In particular, coverage involves a non-differentiable interval-inclusion event, while the influence of current predictions on subsequent calibration states introduces temporal dependencies in interval quality. RL therefore provides a natural framework for optimizing predictions using both immediate performance and downstream calibration feedback, rather than relying solely on pointwise prediction losses.

Our ablation results provide empirical support for this design. In Section 5.4, CORE-w/o-RL replaces RL-based policy optimization with supervised MLP quantile regression. As shown in Table 1, this variant generally exhibits a less favorable coverage and efficiency trade-off than CORE, suggesting that supervised quantile fitting alone does not fully reproduce the benefits of reward-driven optimization. Nevertheless, this comparison does not establish that alternative online optimization methods could not achieve similar performance.

Why PPO? We do not claim that PPO is the only suitable RL algorithm. We adopt PPO as a practical optimizer for continuous quantile actions, benefiting from its clipped surrogate objective for stable policy updates. Other RL methods, such as SAC or DDPG, could in principle optimize the same reward-driven prediction–calibration objective with appropriate changes to policy parameterization and training. Accordingly, CORE's primary contribution lies in the feedback-driven integration of prediction learning and adaptive calibration, rather than in the choice of a particular RL optimizer.

In summary, CORE does not require prediction actions to influence the external data-generating process. Instead, current predictions affect both immediate interval quality and future calibration through the system's internal state. RL provides a means of incorporating this sequential feedback into predictor optimization. We will make this motivation more explicit in the revised manuscript and avoid presenting external non-stationarity alone as sufficient justification for RL.

W2: Quantile-guided Exploration, Outlier Robustness, and Systematic Bias Correction

We thank the reviewer for the suggestions about the mechanism and empirical validation of quantile-guided exploration. We would like to clarify that CORE does not directly identify outliers or systematic prediction biases by randomly exploring different quantile levels. It combines multi-quantile prediction, adaptive calibration, and reward feedback to improve the representation of and adaptation to evolving predictive uncertainty.

Mechanism of Quantile-guided Exploration. CORE predicts multiple quantiles to characterize different regions of the conditional distribution, allowing it to capture heteroscedasticity and distributional asymmetry in time series. Compared with using only a median prediction, this enables the model to adapt the lower and upper interval bounds to changing uncertainty rather than relying solely on uniform expansion through the conformal threshold. Furthermore, incorporating accuracy, interval length, and coverage into the reward allows interval-quality feedback to guide policy optimization. Consequently, under distributional changes, CORE can adjust prediction intervals through both conformal calibration and updates to the learned quantile predictions.

Empirical Validation of Quantile Granularity. Figure 3 in Section 5.3 examines the effect of quantile-set granularity ∣Q∣|\mathcal Q|. We further summarize quantitative results on three datasets at (α=0.1\alpha=0.1) in the table below. Compared with median-only prediction, quantile configurations improve interval quality across datasets. However, improvements are not always monotonic. Some configurations achieve higher coverage at the cost of wider intervals. While finer quantile granularity provides greater flexibility in learning interval boundaries, it also increases computational complexity, with diminishing improvements as ∣Q∣|\mathcal Q| grows. Our sensitivity analysis therefore supports ∣Q∣=7|\mathcal Q|=7 as a practical trade-off between interval quality and computational cost.

dataset/Quantile listQ=[0.5], len(Q)=1Q=[0.05, 0.5, 0.95], len(Q)=3Q=[0.05, 0.35, 0.65, 0.95], len(Q)=4Q=[0.05, 0.25, 0.5, 0.75, 0.95], len(Q)=5Q=[0.05, 0.2, 0.4, 0.6, 0.8, 0.95], len(Q)=6Q=[0.05, 0.2, 0.35, 0.5, 0.65, 0.8, 0.95], len(Q)=7
LEN(Electricity) ×103\times 10^32.421± 0.0861.757± 0.1141.650± 0.1341.695± 0.1051.630± 0.1531.509± 0.091
LEN(Illness) ×106\times 10^61.905± 0.1121.712± 0.0831.601± 0.0951.625± 0.1241.581± 0.0881.520± 0.138
LEN(Traffic)×10−2\times 10^{-2}7.526 ± 0.8077.152 ± 0.6017.442 ± 1.0757.155 ± 0.8826.935 ± 0.7216.863 ± 0.938
COV(Electricity)0.911 ± 0.0040.908 ± 0.0040.907 ± 0.0030.912 ± 0.0020.908 ± 0.0060.905 ± 0.003
COV(Illness)0.904 ± 0.0020.913 ± 0.0080.904 ± 0.0030.906 ± 0.0020.904 ± 0.0020.906± 0.001
COV(Traffic)0.899 ± 0.0150.907 ± 0.0100.904 ± 0.0040.906 ± 0.0020.905 ± 0.0030.9097± 0.003

Outlier Robustness and Systematic Bias Correction. CORE addresses high-uncertainty regions through two complementary mechanisms. Multi-quantile prediction represents uncertainty patterns identifiable from historical observations, while the conformity-score buffer and adaptive significance level αt\alpha_t adjust subsequent calibration thresholds based on observed prediction errors and coverage outcomes. Unlike methods that only adjust calibration thresholds, CORE also updates the prediction policy through quantile loss, accuracy reward, and coverage reward, enabling it to correct systematic prediction biases by jointly optimizing prediction and adaptive calibration.

We appreciate the reviewer's suggestion for more direct empirical validation. Our existing experiments provide supporting evidence from several aspects. In Figure 2, CORE's predictions track local variations in the ground-truth series more closely than competing methods, while its prediction intervals adapt around some regions of sharp fluctuations, including potentially outlying observations, rather than uniformly widening to include extreme values. Moreover, our empirical evaluation covers 9 real-world time-series datasets, including TSLA prices, which exhibit substantial volatility and non-stationarity. Across these diverse settings, CORE maintains near-target coverage with competitive interval lengths, showing its adaptability under complex temporal dynamics.

Overall, CORE does not rely on quantile exploration to explicitly detect outliers. It captures evolving predictive uncertainty and combines adaptive calibration with reward feedback to improve interval quality. Our experiments provide quantitative support for this design, and we appreciate the reviewer's suggestion to further clarify its relationship to outlier robustness and systematic bias correction.

W3 & W4: Algorithm Clarifications

PPO Objective and Actor Optimization. We thank the reviewer for highlighting the need to clarify the PPO optimization procedure and the role of the full action vector. CORE's actor is not optimized solely through the quantile regression and behavior cloning losses. Instead, PPO maximizes the expected discounted return, where the three reward components (Eqs. 5–7) are incorporated into value estimation by the critic and generalized advantage estimation (GAE). The resulting advantage estimates guide actor updates through the PPO clipped surrogate objective. Meanwhile, the quantile regression and behavior cloning losses in Section 3.2.2 serve as auxiliary supervision, encouraging statistically meaningful quantile predictions and accurate central predictions, respectively.

About the full action vector at=at(q)q∈Q\mathbf a_t=\\{a_t^{(q)}\\}_{q\in\mathcal Q}, the median prediction is used to compute the accuracy reward and conformity score, while the lower and upper quantiles determine the prediction interval and influence the interval-length and coverage rewards. In addition, the quantile regression loss provides supervision across all predefined quantile levels. Thus, the full quantile vector contributes to interval construction and the associated optimization signals. We acknowledge that Line 17 of Algorithm 1 does not adequately describe the PPO objective or its relationship with the auxiliary losses. We will revise the algorithm to explicitly include the PPO clipped objective, the reward-to-advantage computation, and the role of the full action vector in policy updates.

Predict-Calibrate-Update Order and Data Leakage. We thank the Reviewer for identifying the ordering issue in Algorithm 1. The current observation yty_t is intended to compute the conformity score and update the calibration buffer for subsequent predictions, rather than to calibrate the interval for the same time step. Specifically, CORE follows a prequential prediction protocol: at time tt, the conformal threshold and prediction interval C^t\hat C_t are first constructed using the historical calibration buffer St−1\mathcal{S}_{t-1} and the current significance level αt\alpha_t. Only after constructing the interval is yty_t observed and used to compute the conformity score, coverage feedback, and reward. The calibration buffer and significance level are then updated to St\mathcal{S}_t and αt+1\alpha_{t+1} for subsequent predictions, consistent with the adaptive update rule in Eq. 4 of Section 3.3.

We acknowledge that Algorithm 1 incorrectly places the buffer update before the construction of the current prediction interval, failing to reflect this intended temporal ordering. We will correct the pseudocode to clearly distinguish the historical calibration information used for the current prediction from the updates performed after observing yty_t, and verify that the experimental implementation follows this protocol.

Reviewer_wxRbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CORE, a framework combining reinforcement learning and adaptive conformal prediction for uncertainty quantification in non-stationary time series. A quantile-based actor–critic model is trained using prediction losses and rewards balancing accuracy, coverage, and interval length. Its outputs are calibrated using sliding-window residuals and an adaptive miscoverage-level update. The paper presents theoretical analyses of coverage and interval efficiency under temporal dependence and bounded drift. Evaluation on nine time-series datasets against six baselines, supplemented by ablations, sensitivity analyses, and computational cost measurements, reports near-nominal coverage and shorter intervals in many settings.

Critical strengths
  1. CORE uses coverage- and width-aware rewards to feed interval quality back into predictor training. This addresses a relevant limitation of post-hoc calibration and offers a potentially useful approach to uncertainty estimation under distribution shift.

  2. Across nine datasets and two target coverage levels, CORE often achieves shorter intervals with near-nominal coverage against six baselines, including PID, ECI, and NCC. The component ablations and computational-cost measurements strengthen the empirical case for the framework’s practical utility.

Critical weaknesses
  1. Appendix D.2 treats a bound on average gradient norms as a bound on prediction error, without a valid argument connecting them. Absorbing auxiliary rewards into the objective also changes the quantity being optimized and does not establish convergence of prediction error alone.

  2. The compared baselines are somewhat outdated. Adding recent methods such as COP [1] would enrich the comparison.

[1] Hu D, Wu J, Xia S T, et al. Distribution-informed online conformal prediction[C]//International Conference on Learning Representations. 2026, 2026: 58230-58255.

Questions for authors
  1. Are the actor outputs intended to remain estimates of the stated conditional quantiles after reward optimization? If so, how does CORE avoid counting uncertainty twice when adding a median-residual correction to the predicted quantile interval? If not, what statistical interpretation should be assigned to these outputs?

  2. Equation (6) normalizes interval width by the recent target median, which may be negative or near zero. How is this handled in the experiments, and what preprocessing ensures that the reward consistently penalizes wider intervals?

4 comments

W1: Prediction-error convergence and composite reward optimization (Appendix D.2)

We thank you for the careful examination of the theoretical derivations in Appendix D.2. We agree with the two key concerns: the current proof does not sufficiently establish the connection between gradient-norm convergence and prediction-error convergence, nor does it rigorously distinguish optimization of the composite reward objective from reduction of prediction error itself. These issues primarily concern the proof of Proposition 1 and its subsequent use in the coverage bound. We will clarify the relevant theoretical assumptions and revise the overly strong convergence claims.

First, about the relationship between gradient norm and prediction error, the original derivation in Appendix D.2 uses stochastic gradient optimization analysis to establish an upper bound on the average gradient norm. We agree that such a stationarity guarantee does not directly imply that prediction error converges at the same rate. Establishing this connection requires additional assumptions on the objective function's geometry, such as the PL condition, as well as consideration of the predictor's irreducible error. We will therefore explicitly distinguish among optimization stationarity, excess prediction risk, and absolute prediction error, rather than directly interpreting the convergence rate of the gradient norm as that of prediction error. Under suitable optimization conditions, the convergence of prediction risk toward its optimal value may be further analyzed, whereas guarantees on absolute prediction error require additional realizability or approximation-error assumptions.

Second, about composite reward optimization, CORE adopts a reward-shaping strategy to provide complementary learning signals for prediction accuracy, interval length, and coverage consistency. Rather than optimizing prediction error alone, these reward components guide the policy toward a better balance between predictive accuracy and uncertainty quantification quality. Importantly, we fix the weights of all three reward components to 1 across all datasets, without dataset-specific tuning. The consistently strong empirical results suggest that this reward design is effective and reasonably robust across different time-series settings.

We will further clarify the relationship between the optimization properties established in Appendix D.2 and the coverage analysis in Theorem 1.


W2: Additional comparison baseline

We thank the reviewer for suggesting the inclusion of more recent baselines. We agree that COP is a highly relevant recent method. Following this suggestion, we conducted additional experiments comparing CORE with COP at two target miscoverage levels, α=0.1\alpha=0.1 and α=0.05\alpha=0.05.

Weather (COV)Weather (LEN)Traffic (COV)Traffic (LEN ×10−2\times 10^{-2})Electricity (COV)Electricity (LEN)Illness (COV)Illness (LEN ×106\times 10^6)TSLA (COV)TSLA (LEN)
α=0.1\alpha=0.1 CORE0.8997.4510.8996.8630.9021509.2090.9061.5200.89811.416
α=0.1\alpha=0.1 COP0.9017.5320.9026.7340.9011517.9800.9041.6740.90411.671
α=0.05\alpha=0.05 CORE0.9517.8720.9497.7710.9511778.5660.9521.6920.94714.560
α=0.05\alpha=0.05 COP0.9598.0190.9518.1270.9511883.450.9512.1270.96015.181

The results show that COP achieves performance comparable to CORE at the 90% target coverage level, with both methods maintaining near-nominal coverage and competitive interval lengths. At the higher 95% confidence level, however, COP requires substantially wider prediction intervals to maintain the target coverage. These results suggest that CORE maintains better interval efficiency under all coverage requirements. We will extend these experiments to the remaining 4 datasets and incorporate the complete results into the revised manuscript, along with an expanded discussion in the related work section.

Q1: Quantile Interpretation and Double-counting of Uncertainty

We thank the reviewer for raising this important question regarding the relationship between quantile-based actor outputs and conformal calibration. We would like to clarify their statistical interpretations and explain how CORE uses adaptive calibration to mitigate potential double-counting of uncertainty.

In CORE, the actor learns predictions at different quantile levels through the quantile regression loss, capturing input-dependent distributional characteristics and predictive uncertainty. However, since the actor is also jointly optimized using prediction accuracy, interval length, and coverage consistency rewards, its outputs are not assumed to remain perfectly calibrated conditional quantiles. Instead, they are interpreted as quantile-guided predictive estimates. Consequently, the initial quantile-based interval does not necessarily achieve the target coverage and still requires calibration based on recent observations.

About the potential double-counting, we emphasize that CORE does not treat the quantile-based interval width and median-residual correction as two independent sources of uncertainty. The former constructs an initial interval that reflects input-dependent variability and distributional asymmetry, while the latter uses a sliding calibration window to estimate recent prediction errors and adjust the interval accordingly. Rather than applying a fixed residual correction, CORE further incorporates an adaptive confidence level that adjusts the residual quantile threshold based on recent miscoverage feedback. When empirical coverage exceeds the target level, adaptive calibration can reduce the additional interval expansion and increase the correction to address under-coverage. Therefore, CORE is not designed to simply add two independent uncertainty estimates, but to integrate quantile-guided prediction with feedback-driven calibration to balance coverage validity and interval efficiency in dynamic environments.


Q2: Interval Length Reward Normalization

We thank the reviewer for highlighting the normalization issue in Eq. 6. The interval length reward is designed to normalize prediction interval widths by the scale of recent target values, ensuring consistent optimization across datasets and time periods while encouraging more compact intervals. We agree that the target median can be negative or close to zero, potentially reversing the intended penalty or causing numerical instability. To address this, we adopt an absolute-scale normalization with a positive stabilizing term rlen=1−∣C^t∣∣y~t∣+ϵ,ϵ>0r_{\mathrm{len}}=1-\frac{|\widehat C_t|}{|\widetilde y_t|+\epsilon}, \epsilon>0, where y~t\widetilde y_t denotes the median of recent target values and ϵ\epsilon prevents division by zero. This ensures that the reward consistently decreases as the interval width increases, regardless of the sign of the target median.

We appreciate the reviewer for identifying this important detail and will correct the formulation in Eq. 6 in the revision to improve its mathematical precision.

Thank you for your comment, I will keep my positive score. Good luck

Dear Reviewer wxRb,

Thank you very much for your feedback and for keeping positive to our paper. We sincerely appreciate your time and consideration, and we are grateful for your kind wishes.

With regards, Authors of paper 3473

Reviewer_BBtCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers adaptive conformal prediction for time series forecasting under distribution shifts and proposes CORE, a framework that combines reinforcement learning with online conformal calibration. The authors introduce a quantile-guided actor-critic approach in which an RL policy generates quantile-based predictions, while a sliding window of conformity scores and an adaptive miscoverage level construct prediction intervals. The policy is trained using PPO with a reward function that incorporates prediction accuracy, interval length, and coverage. Theoretically, the authors derive coverage and interval-efficiency bounds, decomposing the coverage error into finite-window calibration error, prediction-induced error, and adaptive tracking error. Finally, numerical experiments on nine time series datasets demonstrate the proposed method's performance compared with several existing conformal prediction approaches.

Critical strengths
  1. The paper is generally well written.

  2. The idea of using reinforcement learning to update the prediction interval under distribution shift is intersting.

Critical weaknesses
  1. I am quite confused about the optimization problem pertaining to the actor update. In Equation (2), the authors introduce the quantile regression loss Lquantile=∑q∈Qρq(yt−at(q)).\mathcal L_{\rm quantile} = \sum_{q\in\mathcal Q} \rho_q(y_t-a_t^{(q)}). At a first read, this loss is separate across at(q)a_t^{(q)}, so we may as well estimate them separately. Also, the empirical version of this loss is not clearly written. However, if we use a single network that outputs a vector {at(q)}q∈Q\{a_t^{(q)}\}_{q \in \mathcal Q}, then this is not separable, and we need to optimize them jointly. So it would be good if the authors could make this clear.

  2. I am also not sure whether the RL framework is absolutely necessary here. In Section 3.1, the state is defined as a window of observed features, whose evolution does not appear to depend on the chosen action, namely the predicted quantiles. I agree that current predictions affect subsequent calibration through the residual and adaptive level αt\alpha_t, so there is a sequential feedback mechanism. . However, it is not clear what advantage RL provides over directly minimizing a rolling or sequence-level loss that accounts for this feedback, including coverage error and interval length. Although Section 5.4 includes an ablation without RL, it only considers a supervised quantile regressor, which does not seem sufficient to address this question. I think it would be helpful if the authors could compare against a non-RL approach that directly optimizes a comparable objective while retaining the same adaptive calibration procedure. This would help clarify whether the improvement comes from the RL formulation itself or simply from incorporating calibration and interval efficiency into the training objective.

Questions for authors

Kindly see the Weakness section.

5 comments

W1: Clarification of the Joint Actor Optimization and Empirical Quantile Regression Objective

We thank the reviewer for highlighting the ambiguity in the optimization objective presented in Eq.2. Although the current formulation defines the pinball loss for each quantile, it does not sufficiently clarify the parameter sharing across quantile predictions or the corresponding empirical optimization objective.

Specifically, we agree that the loss in Eq.2 is additive across individual quantile predictions. However, this does not imply that they are optimized independently in practice. Similar to the shared-parameter multi-quantile prediction approach adopted in Conformalized Quantile Regression [1], CORE employs a single shared actor network to simultaneously output KK conditional quantile predictions. Since all quantile predictions depend on the same network parameters, their individual loss terms are jointly optimized through a unified objective and backpropagation, rather than by training separate quantile regressors.

To eliminate this ambiguity, we will explicitly describe the vector-valued output of the actor and provide the empirical quantile regression objective $\frac{1}{N}\sum_{t=1}^{N} \sum_{q\in\mathcal{Q}} \rho_q\left(y_t-a_t^{(q)}\right)$ in revision. Here, NN denotes the number of training samples, and at=at(q)q∈Q\mathbf{a}_t=\\{a_t^{(q)}\\}_{q\in\mathcal Q} denotes the vector of quantile predictions generated by the shared actor policy πθ\pi_\theta. For each training sample, we compute and sum the pinball losses across all quantiles, then average them over the training samples to obtain a unified empirical objective. As all quantile predictions share the same parameters θ\theta, their losses jointly contribute to updating the actor through backpropagation, rather than independently updating separate prediction models. This implementation is also consistent with the shared-parameter multi-quantile prediction approach adopted in [1].

We will further clarify how the quantile regression loss interacts with the PPO policy optimization objective during actor updates, providing a more complete description of CORE's actor optimization mechanism.

[1] Romano, Yaniv, Evan Patterson, and Emmanuel Candes. "Conformalized quantile regression." Advances in neural information processing systems 32 (2019).

W2: Necessity of RL and Comparison with non-RL Optimization.

We thank the reviewer for the valuable suggestions about the necessity of RL and the comparison with non-RL alternatives. We address these concerns by clarifying our motivation for adopting RL and presenting additional experimental results.

RL Formulation and Its Motivation. We agree with the reviewer that prediction actions do not directly affect the evolution of the underlying time series. However, CORE introduces RL not to control the external data-generating process, but to extend adaptation from the calibration module to the predictive model itself. Traditional adaptive CP primarily adjusts prediction intervals by updating the conformal threshold or miscoverage level, whereas CORE further leverages calibration feedback to optimize the actor responsible for generating quantile predictions. Specifically, current predictions affect the conformity score buffer and adaptive miscoverage level αt\alpha_t through residuals and coverage feedback, thereby influencing subsequent prediction intervals and their associated rewards. Thus, although the evolution of external observations is independent of prediction actions, the internal calibration state exhibits action-dependent dynamics.

Building on this feedback mechanism, RL provides a means of optimizing the long-term performance of the closed-loop prediction-calibration process. Unlike supervised quantile regression, which primarily minimizes prediction errors at individual time steps, CORE employs PPO to incorporate prediction accuracy, coverage consistency, and interval efficiency into policy learning through cumulative rewards and advantage estimation. This enables the actor to learn from subsequent calibration feedback when updating its predictions. We do not claim that RL is the only way to capture such feedback, but it provides a natural sequential decision-making framework for jointly optimizing prediction and adaptive calibration. We will clarify this motivation in the revised manuscript and explicitly distinguish between external observations and internal calibration states.

Comparison with Non-RL Optimization. The existing CORE-w/o-RL variant in Section 5.4 replaces RL-based policy learning with supervised quantile regression to evaluate the difference between conventional quantile regression and feedback-driven policy learning. To further distinguish the contribution of RL optimization from that of calibration-aware training signals, we introduce an additional variant, CORE-w/o-PPO, following the reviewer's suggestion. This variant retains the same quantile prediction architecture and adaptive calibration procedure as CORE but replaces PPO with direct gradient-based optimization of a differentiable loss incorporating prediction accuracy, interval length, and coverage.

Traffic (COV)Traffic (LEN ×10−2\times 10^{-2})Electricity (COV)Electricity (LEN)Illness (COV)Illness (LEN ×106\times 10^6)
CORE0.8996.8630.9021509.2090.9061.520
CORE-w/o-RL0.8759.6550.8811820.8480.8911.686
CORE-w/o-PPO0.8848.7550.8921792.3510.8851.651

Due to time constraints, we have currently conducted additional experiments on three datasets under the (\alpha=0.1) setting. Compared with CORE-w/o-RL, CORE-w/o-PPO achieves shorter prediction intervals across all three datasets, suggesting that incorporating calibration-aware training signals can improve interval efficiency even without RL. Nevertheless, the full CORE consistently produces the shortest prediction intervals while maintaining coverage closer to the target level. These results suggest that, beyond the benefits of calibration-aware training objectives, PPO-based policy optimization provides additional empirical gains. We will extend these experiments to the remaining datasets and settings and provide a more comprehensive analysis in the revised manuscript.

Dear Reviewer BBtC,

As the discussion period is drawing to a close, we would greatly appreciate hearing from you. We hope our rebuttal has addressed your concerns, and if there are any points that remain unclear, we would be more than happy to provide further clarification or respond again.

Authors of paper 3473

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

We sincerely thank the reviewers and the AC for your constructive feedback. In the revised manuscript, we explicitly describe how the PPO objective, reward-based advantage estimation, and auxiliary quantile and behavior-cloning losses jointly optimize the actor, and clarify how the full quantile action vector contributes to prediction and interval construction. We clarified and emphasized the calibration procedure to follow a strict predict-calibrate-update order, ensuring that the current observation is not used to construct its own prediction interval and thereby eliminating the potential data-leakage concern. The theoretical analysis now clearly separates optimization stationarity from prediction-error convergence, with the latter established only under explicit additional conditions. We also aligned the step-size assumptions with the finite-horizon analysis, corrected the empirical-CDF argument, and stated the conditions required for the coverage and efficiency guarantees, addressing the theoretical concerns raised by reviewers 8NS5 and wxRb. Finally, we included COP as an additional SOTA baseline and updated the corresponding experimental comparisons. These changes directly address the reviewers’ concerns regarding optimization clarity, temporal correctness, and the validity of the theoretical guarantees, without altering CORE’s methodological design or experimental protocol.

AuthorsOpenReview ↗

Summary of Our Rebuttal and Revision

We sincerely thank all reviewers for their thoughtful feedback and constructive suggestions. As the discussion period draws to a close, we would like to provide a consolidated summary of how we have addressed the common concerns raised by the reviewers, as well as the key improvements incorporated into our revised manuscript. We would also like to express our special appreciation to reviewers BBtc and wxRb for their continued positive feedback and encouragement throughout the discussion.

First, we would like to revisit several common questions raised during the discussion. 1. The necessity of introducing RL (reviewers BBtc and 8NS5). We have further clarified how RL enables exploration and leverages calibration feedback to guide policy optimization, establishing a closed-loop interaction between exploration and adaptive calibration. Furthermore, we have conducted additional experiments comparing CORE with non-RL alternatives, including the newly introduced CORE-w/o-PPO variant (suggested by reviewer BBtc), demonstrating the importance of both calibration-aware optimization and RL-based policy learning in improving coverage reliability and interval efficiency. 2. Details and effectiveness of quantile-guided exploration (reviewers wxRb and 8NS5). Building upon Figure 3 and the parameter sensitivity analysis in Section 5.3, we further provided additional experimental results and more detailed analyses to clarify the underlying trade-off mechanisms and empirically demonstrate the effectiveness of quantile-guided exploration.

In addition, we have made important revisions to improve the clarity, rigor, and completeness of our manuscript. In response to reviewer 8NS5's suggestions about the algorithm pseudocode, we have carefully refined the mathematical notation and formulations. In particular, we have introduced explicit time indices for key components, including the calibration buffer S\mathcal{S}, to clarify the temporal dependencies and eliminate potential concerns about data leakage. We have also substantially refined our theoretical analysis to improve its rigor and clarity. Furthermore, following reviewer wxRb's suggestion, we have incorporated an additional comparison with a recent method introduced this year.

Although we have not yet received a follow-up response from reviewer 8NS5, we believe that our clarifications about several shared concerns have been positively acknowledged by reviewers BBtC and wxRb. Once again, we sincerely appreciate all reviewers' time, constructive feedback, and support. We remain open to further discussion and would be happy to address any additional questions or concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.