- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper formulate the adaptive conformal prediction for non-stationary time series procedure as a RL problem. It incorporates prediction accuracy, interval width and coverage into reward function. And then use a PPO-based actor-critic model to generate quantile predictions. They also provide theoretical coverage and efficiency guarantees, which is still questionable.
Critical strengths
- This is an important problem in uncertainty quantification for non-stationary time series. It is interesting to integrate prediction and adaptive conformal calibration through a feedback mechanism.
- The experiments are relatively comprehensive, covering nine time-series datasets and six adaptive conformal baselines.
Critical weaknesses
- The necessity of RL is not sufficiently justified. Actions are quantile predictions and do not influence the underlying dynamics, that is the MDP state. It seems like a contextual bandit problem. It is unclear why this problem requires PPO rather than supervised or online quantile regression.
- The abstracts claims that RL exploration improves coverage in uncertain or outlier regions and helps address systematic prediction biases. However, the paper does not clearly explain how exploring different quantile predictions achieves these goals. Moreover, the experiments do not directly evaluate outlier robustness or systematic bias correction.
- Algorithm 1 states that the actor is updated using quantile regression and behavior cloning losses. But it does not specify how the PPO objective is incorporated. They also should clarify how the proposed rewards and the full action vector are used in policy optimization.
- In Algorithm 1, the conformity-score buffer is updated using the current observation $y_t$ (Step 7) before the prediction interval for the same time step is constructed (Steps 12–13). Then, the reported coverage rates may be affected by data leakage.
- The proof of Proposition 1 appears to infer convergence of the prediction error from a gradient-norm convergence bound, but this inference is not adequately justified. In addition, the step-size choice in Appendix D.2 is inconsistent with the summability condition assumed in Proposition 1. The coverage proof in Appendix D.1 also relies on a questionable bound for the empirical CDF increment. Taken together, these issues raise concerns about the claimed coverage and interval-efficiency guarantees.
Questions for authors
See the weeknesses.
7 comments
W2: Quantile-guided Exploration, Outlier Robustness, and Systematic Bias Correction
We thank the reviewer for the suggestions about the mechanism and empirical validation of quantile-guided exploration. We would like to clarify that CORE does not directly identify outliers or systematic prediction biases by randomly exploring different quantile levels. It combines multi-quantile prediction, adaptive calibration, and reward feedback to improve the representation of and adaptation to evolving predictive uncertainty.
Mechanism of Quantile-guided Exploration. CORE predicts multiple quantiles to characterize different regions of the conditional distribution, allowing it to capture heteroscedasticity and distributional asymmetry in time series. Compared with using only a median prediction, this enables the model to adapt the lower and upper interval bounds to changing uncertainty rather than relying solely on uniform expansion through the conformal threshold. Furthermore, incorporating accuracy, interval length, and coverage into the reward allows interval-quality feedback to guide policy optimization. Consequently, under distributional changes, CORE can adjust prediction intervals through both conformal calibration and updates to the learned quantile predictions.
Empirical Validation of Quantile Granularity. Figure 3 in Section 5.3 examines the effect of quantile-set granularity . We further summarize quantitative results on three datasets at () in the table below. Compared with median-only prediction, quantile configurations improve interval quality across datasets. However, improvements are not always monotonic. Some configurations achieve higher coverage at the cost of wider intervals. While finer quantile granularity provides greater flexibility in learning interval boundaries, it also increases computational complexity, with diminishing improvements as grows. Our sensitivity analysis therefore supports as a practical trade-off between interval quality and computational cost.
| dataset/Quantile list | Q=[0.5], len(Q)=1 | Q=[0.05, 0.5, 0.95], len(Q)=3 | Q=[0.05, 0.35, 0.65, 0.95], len(Q)=4 | Q=[0.05, 0.25, 0.5, 0.75, 0.95], len(Q)=5 | Q=[0.05, 0.2, 0.4, 0.6, 0.8, 0.95], len(Q)=6 | Q=[0.05, 0.2, 0.35, 0.5, 0.65, 0.8, 0.95], len(Q)=7 |
|---|---|---|---|---|---|---|
| LEN(Electricity) | 2.421± 0.086 | 1.757± 0.114 | 1.650± 0.134 | 1.695± 0.105 | 1.630± 0.153 | 1.509± 0.091 |
| LEN(Illness) | 1.905± 0.112 | 1.712± 0.083 | 1.601± 0.095 | 1.625± 0.124 | 1.581± 0.088 | 1.520± 0.138 |
| LEN(Traffic) | 7.526 ± 0.807 | 7.152 ± 0.601 | 7.442 ± 1.075 | 7.155 ± 0.882 | 6.935 ± 0.721 | 6.863 ± 0.938 |
| COV(Electricity) | 0.911 ± 0.004 | 0.908 ± 0.004 | 0.907 ± 0.003 | 0.912 ± 0.002 | 0.908 ± 0.006 | 0.905 ± 0.003 |
| COV(Illness) | 0.904 ± 0.002 | 0.913 ± 0.008 | 0.904 ± 0.003 | 0.906 ± 0.002 | 0.904 ± 0.002 | 0.906± 0.001 |
| COV(Traffic) | 0.899 ± 0.015 | 0.907 ± 0.010 | 0.904 ± 0.004 | 0.906 ± 0.002 | 0.905 ± 0.003 | 0.9097± 0.003 |
Outlier Robustness and Systematic Bias Correction. CORE addresses high-uncertainty regions through two complementary mechanisms. Multi-quantile prediction represents uncertainty patterns identifiable from historical observations, while the conformity-score buffer and adaptive significance level adjust subsequent calibration thresholds based on observed prediction errors and coverage outcomes. Unlike methods that only adjust calibration thresholds, CORE also updates the prediction policy through quantile loss, accuracy reward, and coverage reward, enabling it to correct systematic prediction biases by jointly optimizing prediction and adaptive calibration.
We appreciate the reviewer's suggestion for more direct empirical validation. Our existing experiments provide supporting evidence from several aspects. In Figure 2, CORE's predictions track local variations in the ground-truth series more closely than competing methods, while its prediction intervals adapt around some regions of sharp fluctuations, including potentially outlying observations, rather than uniformly widening to include extreme values. Moreover, our empirical evaluation covers 9 real-world time-series datasets, including TSLA prices, which exhibit substantial volatility and non-stationarity. Across these diverse settings, CORE maintains near-target coverage with competitive interval lengths, showing its adaptability under complex temporal dynamics.
Overall, CORE does not rely on quantile exploration to explicitly detect outliers. It captures evolving predictive uncertainty and combines adaptive calibration with reward feedback to improve interval quality. Our experiments provide quantitative support for this design, and we appreciate the reviewer's suggestion to further clarify its relationship to outlier robustness and systematic bias correction.
W3 & W4: Algorithm Clarifications
PPO Objective and Actor Optimization. We thank the reviewer for highlighting the need to clarify the PPO optimization procedure and the role of the full action vector. CORE's actor is not optimized solely through the quantile regression and behavior cloning losses. Instead, PPO maximizes the expected discounted return, where the three reward components (Eqs. 5–7) are incorporated into value estimation by the critic and generalized advantage estimation (GAE). The resulting advantage estimates guide actor updates through the PPO clipped surrogate objective. Meanwhile, the quantile regression and behavior cloning losses in Section 3.2.2 serve as auxiliary supervision, encouraging statistically meaningful quantile predictions and accurate central predictions, respectively.
About the full action vector , the median prediction is used to compute the accuracy reward and conformity score, while the lower and upper quantiles determine the prediction interval and influence the interval-length and coverage rewards. In addition, the quantile regression loss provides supervision across all predefined quantile levels. Thus, the full quantile vector contributes to interval construction and the associated optimization signals. We acknowledge that Line 17 of Algorithm 1 does not adequately describe the PPO objective or its relationship with the auxiliary losses. We will revise the algorithm to explicitly include the PPO clipped objective, the reward-to-advantage computation, and the role of the full action vector in policy updates.
Predict-Calibrate-Update Order and Data Leakage. We thank the Reviewer for identifying the ordering issue in Algorithm 1. The current observation is intended to compute the conformity score and update the calibration buffer for subsequent predictions, rather than to calibrate the interval for the same time step. Specifically, CORE follows a prequential prediction protocol: at time , the conformal threshold and prediction interval are first constructed using the historical calibration buffer and the current significance level . Only after constructing the interval is observed and used to compute the conformity score, coverage feedback, and reward. The calibration buffer and significance level are then updated to and for subsequent predictions, consistent with the adaptive update rule in Eq. 4 of Section 3.3.
We acknowledge that Algorithm 1 incorrectly places the buffer update before the construction of the current prediction interval, failing to reflect this intended temporal ordering. We will correct the pseudocode to clearly distinguish the historical calibration information used for the current prediction from the updates performed after observing , and verify that the experimental implementation follows this protocol.
W1: Motivation for RL and Distinction from Contextual Bandits and Quantile Regression
We thank the reviewer for raising important questions about the necessity of RL and the validity of our problem formulation. We agree that CORE's quantile predictions do not alter the underlying time-series dynamics. Our motivation for RL is therefore not to control the external environment, but to extend adaptation beyond conventional conformal calibration to the predictor itself. While traditional adaptive CP methods primarily adjust conformity thresholds or miscoverage levels, CORE incorporates prediction accuracy, interval length, and coverage reliability into policy optimization, establishing a feedback loop between prediction learning and adaptive calibration.
MDP Formulation vs. Contextual Bandits. The key distinction lies between the external time-series dynamics and the internal calibration state. In CORE, the actor observes a historical feature window ot=xt−w+1 and produces a quantile action vector at=at(q)q∈Q. Although the external data dynamics are independent of these actions, the prediction–calibration system maintains a conformity-score buffer, recent miscoverage history, and an adaptive significance level αt. Once yt is observed, the current action determines the conformity score and coverage outcome, which update the calibration buffer and αt+1 (Eq. 4). These updates subsequently affect conformal thresholds, prediction intervals, and future rewards. Thus, the sequential dependence arises not from controlling the underlying time series, but from how current predictions influence future interval quality through the internal calibration mechanism. We will clarify this distinction in Section 3.1, explicitly separating the actor's observations from the full internal state of the prediction and calibration system.
RL vs. Supervised or Online Quantile Regression. The main difference concerns the optimization objective. Supervised quantile regression, including its online variants, typically minimizes prediction errors through the pinball loss at individual time steps. CORE retains quantile regression as an auxiliary objective while additionally incorporating accuracy, interval efficiency, and realized coverage into a unified reward. In particular, coverage involves a non-differentiable interval-inclusion event, while the influence of current predictions on subsequent calibration states introduces temporal dependencies in interval quality. RL therefore provides a natural framework for optimizing predictions using both immediate performance and downstream calibration feedback, rather than relying solely on pointwise prediction losses.
Our ablation results provide empirical support for this design. In Section 5.4,
CORE-w/o-RLreplaces RL-based policy optimization with supervised MLP quantile regression. As shown in Table 1, this variant generally exhibits a less favorable coverage and efficiency trade-off than CORE, suggesting that supervised quantile fitting alone does not fully reproduce the benefits of reward-driven optimization. Nevertheless, this comparison does not establish that alternative online optimization methods could not achieve similar performance.Why PPO? We do not claim that PPO is the only suitable RL algorithm. We adopt PPO as a practical optimizer for continuous quantile actions, benefiting from its clipped surrogate objective for stable policy updates. Other RL methods, such as SAC or DDPG, could in principle optimize the same reward-driven prediction–calibration objective with appropriate changes to policy parameterization and training. Accordingly, CORE's primary contribution lies in the feedback-driven integration of prediction learning and adaptive calibration, rather than in the choice of a particular RL optimizer.
In summary, CORE does not require prediction actions to influence the external data-generating process. Instead, current predictions affect both immediate interval quality and future calibration through the system's internal state. RL provides a means of incorporating this sequential feedback into predictor optimization. We will make this motivation more explicit in the revised manuscript and avoid presenting external non-stationarity alone as sufficient justification for RL.