acceptodds
Under review as a conference paper at ICLR 2027

Conformal Prediction for Time Series with Deep Sequence Models

Abstract

Recent advances in deep learning for time series prediction have amplified the need for reliable uncertainty quantification. Conformal prediction has gained attention as a distribution-free framework for constructing prediction intervals with coverage guarantees. However, its coverage guarantees rely on data exchangeability, an assumption generally violated in time series. Active research has focused on developing conformal prediction methods for time series that overcome this limitation. While deep sequence models, such as recurrent neural networks and Transformers, have often been used in conformal prediction for time series, limited work has systematically studied how deep sequence models can be utilized in conformal prediction for time series. In this work, we systematically investigate the use of deep sequence models in conformal prediction for time series through three approaches: conditional quantile regression, conditional quantile function estimation, and localized conformal prediction. We provide a theoretical analysis establishing asymptotic conditional coverage guarantees for all three approaches under suitable assumptions. Through comprehensive experiments on real-world datasets, we demonstrate the effectiveness of leveraging deep sequence models into conformal prediction for time series.

Primary area:
learning on time series and dynamical systems
Keywords:
Conformal Prediction for Time Series, Time Series Prediction, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4S4QOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how deep sequence models can be incorporated into conformal prediction for time series through three approaches: conditional quantile regression (DSCP-QR), conditional quantile function estimation (DSCP-QFE), and localized conformal prediction (DSCP-Local). The authors establish asymptotic conditional coverage guarantees under suitable assumptions and evaluate the proposed methods on three real-world datasets using different base predictors and sequence encoders. The experimental results demonstrate competitive prediction interval efficiency for DSCP-QR and relatively stable coverage for DSCP-Local, while also revealing limitations of DSCP-QFE.

Critical strengths
  1. The proposed framework is clearly organized, and the distinctions between quantile-specific prediction heads, continuous quantile function estimation, and representation-based localization are well explained.
  2. The ablation studies on quantile crossing constraints and quantile levels provide useful observations regarding the trade-off between architectural constraints and predictive performance.
Critical weaknesses
  1. DSCP-QR and DSCP-QFE appear to construct prediction intervals directly from estimated conditional residual quantiles. They do not provide finite-sample distribution-free coverage guarantees, and their validity relies instead on the consistency of the estimated conditional quantiles under additional statistical assumptions.
  2. The theoretical guarantees rely on restrictive assumptions and these assumptions may be difficult to justify for highly overparameterized deep sequence models. The theoretical results primarily follow from the consistency of conditional quantile or CDF estimation. The paper could better clarify what new theoretical insights are obtained specifically from incorporating deep sequence models.
  3. The theoretical contribution emphasizes asymptotic conditional coverage, but the experiments primarily report marginal empirical coverage.
  4. The experimental comparison omits many recent online/adaptive conformal baselines, including:

[1]: Areces F, Mohri C, Hashimoto T, et al. Online conformal prediction via online optimization[C]//Forty-second International Conference on Machine Learning. 2025.

[2]: Hu D, Wu J, Xia S T, et al. Distribution-informed online conformal prediction[C]//International Conference on Learning Representations. 2026.

Questions for authors
  1. Can the theoretical guarantees for DSCP-Local be extended to the actual experimental implementation? The theoretical analysis assumes a compactly supported kernel with a shrinking bandwidth and a fixed encoder, whereas the experiments use softmax similarity weights and finite calibration windows. Can the authors provide theoretical justification for these practical choices, or alternatively evaluate an implementation that directly satisfies Assumptions A.21–A.24? How would the analysis change when the encoder is learned from temporally dependent observations?
  2. Can the authors provide a more direct empirical assessment of conditional coverage? Since conditional coverage is a central theoretical motivation, it would be valuable to evaluate coverage across context-dependent groups, different uncertainty levels, and different temporal regimes.
7 comments
AuthorsOpenReview ↗

Rebuttal 1

DSCP-QR and DSCP-QFE appear to construct prediction intervals directly from estimated conditional residual quantiles. They do not provide finite-sample distribution-free coverage guarantees, and their validity relies instead on the consistency of the estimated conditional quantiles under additional statistical assumptions.

We appreciate the reviewer’s request for clarification. Our formulation follows the sequential residual-quantile approach. Within this setting, our analysis connects joint quantile training for QR and training over quantile levels for QFE to consistent endpoint estimation and coverage within temporally dependent data.

Finite-sample distribution-free coverage guarantee is inherently challenging in this setting since temporal dependence prevents us from directly invoking the standard exchangeability-based guarantees of conformal calibration. In the related approach of CQR [1], for instance, the finite-sample marginal coverage guarantee is obtained through a conformal calibration step under exchangeability. However, validity for nonexchangeable time-series data requires appropriate dependence-aware argument.

DSCP-QR and DSCP-QFE directly estimate conditional residual quantiles without such additional conformal calibration step. Our stated guarantee is asymptotic coverage conditional on the forecasting context, established through consistent estimation, which is a common target in conformal prediction literature [2, 3, 4]. We follow this approach by stating explicit sufficient conditions on dependence structure and regularity on encoders, and show how they connect the proposed training objectives to conditional coverage.

[1] Romano, Patterson & Candes. Conformalized Quantile Regression. NeurIPS, 2019.

[2] Xu & Xie. Sequential Predictive Conformal Inference for Time Series. ICML, 2023.

[3] Lee, Xu & Xie. Kernel-based Optimally Weighted Conformal Time-Series Prediction. ICLR, 2025.

[4] Chernozhukov, Wüthrich & Zhu. Distributional conformal prediction. Proceedings of the National Academy of Sciences, 2021.

AuthorsOpenReview ↗

Rebuttal 2

The theoretical guarantees rely on restrictive assumptions and these assumptions may be difficult to justify for highly overparameterized deep sequence models. The theoretical results primarily follow from the consistency of conditional quantile or CDF estimation. The paper could better clarify what new theoretical insights are obtained specifically from incorporating deep sequence models.

Thank you for raising this point. We introduced deep sequence models to benefit from their expressive representations of temporal context. For QR and QFE, we acknowledge that the approximation and optimization conditions are difficult to verify for general deep networks. Here we provide an illustrative example where joint approximation can be satisfied for QR and QFE.

Consider ϵ^=m(Z)+ψ(U)\widehat \epsilon = m(Z) + \psi(U), where U∼Unif⁡(0,1)U \sim \operatorname{Unif}(0,1) is independent of ZZ, m(z)m(z) is an affine transformation of an encoder output, and ψ\psi is a feasible strictly increasing QFE head. Then, q⋆(z,τ)=m(z)+ψ(τ)q^\star(z, \tau) = m(z) + \psi(\tau), and QFE represents this directly while QR uses positive constant increments between quantile levels. In this setting, both joint approximation errors vanish.

Still, joint approximation remains a substantive condition for general data distributions. Our theoretical contribution is to specify sufficient conditions that the deep sequence encoder and associated estimators must satisfy to obtain consistency, and then to establish how this consistency yields conditional coverage under temporal dependence. We have added the discussion about this justification and limitation in Appendix A.1.

AuthorsOpenReview ↗

Rebuttal 3

The theoretical contribution emphasizes asymptotic conditional coverage, but the experiments primarily report marginal empirical coverage.

Although evaluating conditional coverage remains challenging, particularly in time series settings, we agree that the suggested approach provides a useful proxy for assessing conditional coverage. Accordingly, we designed an additional experiment with the following procedure:

  1. Train a sequence encoder: We train an independent sequence encoder on the calibration set of each time series. The encoder is expected to map samples with similar temporal contexts or regimes to nearby representations in the latent space.

  2. Cluster test samples: We extract context representations for the test samples using the trained encoder and partition them into groups using k-means clustering.

  3. Evaluate group-wise coverage: We compute the coverage gap within each cluster to assess deviations from the target coverage level across different groups.

Air

MethodLR (k=3k=3)LR (k=5k=5)LSTM k=3k=3LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.003-0.005-0.025-0.028-0.005-0.005
DSCP-Local-0.021-0.022-0.021-0.020-0.003-0.005

Solar

MethodLR (k=3k=3)LR (k=5k=5)LSTM (k=3k=3)LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.025-0.026-0.033-0.034-0.011-0.011
DSCP-Local-0.009-0.014-0.007-0.017-0.008-0.012

Sapflow

MethodLR (k=3k=3)LR (k=5k=5)LSTM (k=3k=3)LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.020-0.025-0.042-0.052-0.067-0.079
DSCP-Local-0.015-0.017-0.010-0.018-0.020-0.022

The results show that the group-wise coverage gaps for DSCP-Local consistently remain above the threshold of ΔCov=−0.25α\Delta\mathrm{Cov}=-0.25\alpha. DSCP-QR exhibits larger coverage gaps, (3 out of all 9 experiments across 3 datasets and 3 base predictors under k=3 groups; and 5 out of all 9 experiments under k=3 groups), where this observation is consistent with the overall coverage trends reported in Table 1, which means conditional coverage gaps in those cases partially due to the undercoverage in marginal coverage. We have updated the main results section of the manuscript to include these results and added the experimental setup and implementation details to the appendix.

Reviewer_QV8fOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies three approaches to prediction intervals for time series using deep sequence models: quantile regression, quantile-function estimation, and local calibration. It derives asymptotic conditional coverage guarantees under method-specific assumptions. Experiments on three datasets show competitive performance for quantile regression and local calibration, while quantile-function estimation exhibits undercoverage.

Critical strengths
  • The paper provides a careful account of how estimation consistency yields coverage under temporal dependence. The analysis handles the joint feasibility of quantile heads sharing an encoder and, for QFE, explains how an objective integrated over quantile levels can support guarantees at fixed interval endpoints. The Local proof also explicitly handles a dependent test representation through a mixing argument, rather than assuming that fixed-query consistency automatically transfers to the future observation. These details make the theoretical treatment useful and help clarify the assumptions behind applying conditional distribution estimation to sequential prediction.
  • The comparative experiments provide useful evidence about practical design choices. Evaluating multiple interval-construction strategies with different encoders and point predictors is informative. In particular, DSCP-Local maintains dataset-averaged coverage relatively close to the target across the nine dataset–predictor combinations in Table 1, while DSCP-QR achieves competitive interval scores in several settings. The head-design ablations also reveal substantial sensitivity in QFE. This supports the value of the paper as a comparative study, although the evidence is stronger for selected configurations than for the three approaches collectively.
Critical weaknesses
  • The methodological and theoretical advances over the closest prior work need clearer differentiation. DSCP-QR substantially overlaps with existing residual-quantile predictors, including the Transformer-based SPCI-T [1] and CoRNN, the recurrent ablation introduced in the CoRel paper [2]. Its nonnegative-increment construction for ordered quantile heads also has a precedent in forecasting work [4]. Representation-based weighting for conformal prediction likewise has substantial precedents, including HopCPT [5], CT-SSF [6] and ResCP [7]. The paper should identify the precise methodological advances of DSCP-QR and DSCP-Local over these existing approaches. The theoretical contribution also needs clearer differentiation. Asymptotic conditional coverage obtained through conditional-quantile or conditional-CDF consistency is already established in SPCI [8] and KOWCPI [9]. The present QR/QFE results may provide additional sufficient conditions for neural function classes, but they rely on a common compact model class, vanishing approximation error, and optimization approaching the global empirical infimum. The paper does not establish these properties for the implemented architectures or explain clearly which previous limitations its results resolve.
  • The theoretical guarantees are insufficiently connected to the implemented methods and the empirical claims. The most concrete issue concerns DSCP-Local. Appendix A.3 analyzes a compactly supported radial kernel with a shrinking bandwidth and an increasing effective calibration sample. Appendix B.3 instead evaluates softmax similarity weights, and Table 4 uses calibration windows of 100 or 500 observations and a fixed temperature search space. The authors explicitly acknowledge that these experimental variants are not directly covered by the theorem. Thus, the strongest empirical method does not currently have the theoretical support that the overall framing suggests. Furthermore, the extension from representation-conditional to context-conditional coverage assumes that the encoder preserves the full conditional residual distribution (Corollary 3.4 and Assumption A.28). The implemented DSCP-Local encoder is trained at the quantile levels 0.1, 0.5, and 0.9, but fitting these quantiles alone does not establish this sufficiency assumption. In particular, distributions can agree at the training quantiles while differing at the endpoints used for the 90% and 95% intervals. The authors explicitly state the assumption, so the concern is its justification for the learned representation and the resulting scope of the context-conditional guarantee.

The experiments report coverage averaged over time and sequences, which cannot validate the central conditional-coverage claim. A more direct evaluation would report coverage conditional on the representation or context, for example by binning the test sequences according to the learned representation and reporting coverage within each bin.

DSCP-QFE exhibits substantial undercoverage in Table 1, with dataset-averaged coverage of approximately 76.3% to 85.4% at a nominal level of 90%. This also shows that the asymptotic results alone provide limited guidance about the reliability of the implemented procedures. These concerns do not demonstrate that the stated theorems are false, but they materially limit the conclusions supported in the manuscript.

References:

[1] Junghwan Lee, Chen Xu, and Yao Xie. Transformer Conformal Prediction for Time Series. arXiv:2406.05332, 2024.

[2] Andrea Cini, Alexander Jenkins, Danilo Mandic, Cesare Alippi, and Filippo Maria Bianchi. Relational Conformal Prediction for Correlated Time Series. ICML, 2025.

[3] Adèle Gouttes, Kashif Rasul, Mateusz Koren, Johannes Stephan, and Tofigh Naghibi. Probabilistic Time Series Forecasting with Implicit Quantile Networks. arXiv:2107.03743, 2021.

[4] Youngsuk Park, Danielle Maddix, François-Xavier Aubet, Kelvin Kan, Jan Gasthaus, and Yuyang Wang. Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting. AISTATS, 2022.

[5] Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal Prediction for Time Series with Modern Hopfield Networks. NeurIPS, 2023.

[6] Baiting Chen, Zhimei Ren, and Lu Cheng. Conformalized Time Series with Semantic Features. NeurIPS, 2024.

[7] Roberto Neglia, Andrea Cini, Michael M. Bronstein and Filippo Maria Bianchi. ResCP: Reservoir Conformal Prediction for Time Series forecasting. ICLR, 2026.

[8] Chen Xu and Yao Xie. Sequential Predictive Conformal Inference for Time Series. ICML, 2023.

[9] Jonghyeok Lee, Chen Xu, and Yao Xie. Kernel-based Optimally Weighted Conformal Time-Series Prediction. ICLR, 2025.

Questions for authors
  • Methodological contribution of DSCP-QR: Beyond the noncrossing output head described in Section 2.1, what substantive changes to the conditioning information, training objective, or interval construction distinguish DSCP-QR from SPCI-T [1] and CoRNN [2]? Is the contribution primarily the coverage analysis of this approach, or is there an additional methodological difference that I have overlooked?
  • Interpretation of QFE’s undercoverage: For the DSCP-QFE configurations reported in Table 1, was substantial undercoverage already present on the validation data, or did it emerge primarily on the test data? Section 4.2 suggests that the monotonic prediction head limits expressiveness. How does the validation behavior support that interpretation? This would help distinguish a persistent fitting limitation from a deterioration in coverage between validation and test.

References

[1] Junghwan Lee, Chen Xu, and Yao Xie. Transformer Conformal Prediction for Time Series. arXiv:2406.05332, 2024.

[2] Andrea Cini, Alexander Jenkins, Danilo Mandic, Cesare Alippi, and Filippo Maria Bianchi. Relational Conformal Prediction for Correlated Time Series. ICML, 2025.

8 comments
AuthorsOpenReview ↗

Rebuttal 1

Methodological novelty, theoretical analysis compared to the existing works, and validity of the assumptions.

Thank you for the suggestion. We agree that residual-quantile prediction, representation-based weighting, and nonnegative-increment constructions have explored in the existing studies. Our contribution is not proposing a new architecture or a new general principle linking consistency to conditional coverage, but a systematic comparison of three ways to use sequence representations, with sufficient conditions for their consistency and coverage.

  1. Methodological distinction. DSCP-QR shares the architecture formulation of SPCI-T [1] and CoRNN [2]. We did not claim this formulation with the nonnegative-increment construction as novel. For DSCP-Local, the specific construction is to train a sequence encoder separately from the base forecaster using quantile loss at multiple quantile levels, and use the representations to weight calibration residuals. This is different from HopCPT’s weighting and training objective [5], CT-SSF’s construction of nonconformity scores in the base forecaster’s latent space [6], and ResCP’s use of untrained reservoir states [7]. We consider these as specific differences within an established representation-weighting framework, rather than claiming representation-based approach itself as new.

  2. Theoretical distinction. We agree that SPCI [8] and KOWCPI [9] established asymptotic conditional coverage through those consistencies. Their estimator-specific analyses focus on quantile regression forests and reweighted Nadaraya–Watson estimation, respectively. Our QR/QFE results instead analyze joint empirical-risk minimization over encoder–head function classes and establish uniform quantile consistency from uniform risk convergence, joint approximation, and vanishing optimization error, explicitly accounting for shared parameters across quantile levels. QFE additionally treats an objective integrated over quantile levels. For DSCP-Local, we analyze conditional-CDF estimation for a fixed representation under explicit local-mass, CDF-regularity, mixing, and bandwidth conditions. We formalize context-conditional coverage through an additional residual-sufficiency assumption. We did not claim that encoder training automatically establishes this assumption

  3. Scope of the assumptions. For QR and QFE, the model-class conditions in Assumptions A.3 and A.14 are satisfied by a fixed finite encoder–head architecture with jointly continuous component maps, such as an LSTM or Transformer, provided that the context support is compact and the parameters are restricted to a common compact set. These conditions do not establish vanishing approximation or optimization error. Joint approximation remains a separate requirement on the target conditional quantile functions and the jointly feasible encoder–head class, while the optimization assumption requires the achieved empirical risk to approach the global empirical infimum. We do not establish vanishing approximation error for the experimental data distributions or vanishing optimization error for the implemented training procedure. Accordingly, our results provide asymptotic consistency and conditional-coverage guarantees under the stated assumptions, rather than finite-sample coverage guarantees.

  4. Empirical contribution. With the experiment results, we complement these theoretical analyses by comparing coverage, interval efficiency, encoder choices, and failure cases across the three methods on real world datasets and different base predictors.

[1] Junghwan Lee, Chen Xu, and Yao Xie. Transformer Conformal Prediction for Time Series. arXiv:2406.05332, 2024.

[2] Andrea Cini, Alexander Jenkins, Danilo Mandic, Cesare Alippi, and Filippo Maria Bianchi. Relational Conformal Prediction for Correlated Time Series. ICML, 2025.

[3] Adèle Gouttes, Kashif Rasul, Mateusz Koren, Johannes Stephan, and Tofigh Naghibi. Probabilistic Time Series Forecasting with Implicit Quantile Networks. arXiv:2107.03743, 2021.

[4] Youngsuk Park, Danielle Maddix, François-Xavier Aubet, Kelvin Kan, Jan Gasthaus, and Yuyang Wang. Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting. AISTATS, 2022.

[5] Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal Prediction for Time Series with Modern Hopfield Networks. NeurIPS, 2023.

[6] Baiting Chen, Zhimei Ren, and Lu Cheng. Conformalized Time Series with Semantic Features. NeurIPS, 2024.

[7] Roberto Neglia, Andrea Cini, Michael M. Bronstein and Filippo Maria Bianchi. ResCP: Reservoir Conformal Prediction for Time Series forecasting. ICLR, 2026.

[8] Chen Xu and Yao Xie. Sequential Predictive Conformal Inference for Time Series. ICML, 2023.

[9] Jonghyeok Lee, Chen Xu, and Yao Xie. Kernel-based Optimally Weighted Conformal Time-Series Prediction. ICLR, 2025.

AuthorsOpenReview ↗

Rebuttal 2: Bridge between theoretical analysis and actual implementation of the methods

  1. About kernel assumptions. We appreciate the reviewer’s valuable feedback. We agree that the original assumptions did not directly cover the softmax weights. Correspondingly, in the revision, we have successfully extended our analysis to cover the softmax weight families based on cosine similarity and squared Euclidean distance. In the revised Appendix A.3.1, we remove the compact-support requirement and establish the same conditional-CDF consistency and coverage guarantee results. The revised assumptions now only require bounded nonnegative kernels with sufficient expected kernel mass and vanishing population CDF bias. We note that these are strict relaxations, i.e., the original assumptions imply the revised ones.

Following, we have also added discussion of familiar kernel function families that are covered by revised analysis, including Gaussian kernels in particular. For squared Euclidean distance, weights proportional to exp⁡(−∥H−h∥2/ϑ)\exp(-\lVert H-h \rVert^2 / \vartheta) are the Gaussian kernel weights. For cosine weighting, as we are considering normalized representations, it reduces to exp⁡(ϑH⊤h)=exp⁡(ϑ)exp⁡(−ϑ∥H−h∥2/2)\exp(\vartheta H^\top h) = \exp(\vartheta) \exp(-\vartheta \lVert H - h \rVert^2/2). These correspondences place the Euclidean and cosine softmax weights within the revised analysis. Raw dot-product weights are not generally covered by this identification. We have accordingly revised Appendix B.3 to clarify these connections.

Consistent with this, Table 22 provides empirical support for these choices. Cosine and Euclidean achieves better performance in general compared to dot product, while avoiding the large undercoverage observed for dot product in several settings.

Regarding calibration window, we select window length and temperature parameter using validation data. Combining asymptotic analysis with finite-sample evaluation and data-driven bandwidth is an established practice in classical kernel estimation [1-3]. Combining asymptotic coverage analysis with finite-window implementation also commonly appears in time-series conformal literature [4, 5]. This is because direct empirical verification of the asymptotic guarantee is necessarily limited. The analysis considers increasing calibration sizes with shrinking bandwidths, whereas experiments can examine only a finite range of these settings. Therefore our theorem characterizes conditional coverage as the calibration sample size increases under the stated bandwidth conditions, while our experiments assess performance at the selected finite sizes.

  1. About residual sufficiency. We agree that fitting the quantiles at 0.1,0.5,0.9\\{0.1, 0.5, 0.9\\} alone does not establish Assumption A.28 or determine the more extreme quantiles required for the 90% or 95% intervals. In DSCP-Local, these quantile heads provide supervision for learning the representation, while the final interval endpoints are estimated from the weighted calibration residuals. Our analysis distinguishes these roles. For a fixed encoder, the HH-conditional result requires the conditional CDF estimation assumptions, while recovering ZZ-conditional coverage additionally requires residual sufficiency. This condition is about preserving the conditional residual distribution, and is not established by the choice of training quantiles alone.

More generally, DSCP-Local extends direct context-based residual estimation to calibration in an encoder representation space. It allows flexible choice of representation, even with identity encoder which recovers direct context-based calibration and hence ZZ-conditional coverage. For a reduced representation, Assumption A.28 specifies what must be preserved to recover the full ZZ-conditional residual distribution.

To investigate the reviewer’s specific concern, in revised Table 20, we compared encoder training at 0.1,0.5,0.9\\{0.1, 0.5, 0.9\\} with training at 0.05,0.5,0.95\\{0.05, 0.5, 0.95\\}, matching the endpoints of the 90% intervals. It also includes the results with training at median only and at 0.1,0.2,…,0.9\\{0.1, 0.2, \dots, 0.9\\}. Endpoint matched training produced broadly comparable coverage and interval width, without a consistent improvement across settings. Training at the endpoints more directly aligns the encoder objective with the evaluated intervals, but neither this alignment nor the observed performance verifies full residual sufficiency. Accordingly, Assumption A.28 remains an explicit additional condition for the ZZ-conditional guarantee.

[1] Optimal Bandwidth Selection for Nonparametric Conditional Distribution and Quantile Functions. Journal of Business & Economic Statistics, 2013.

[2] Cross-Validation and the Estimation of Conditional Probability Densities. Journal of the American Statistical Association, 2004.

[3] Methods for Estimating a Conditional Distribution Function. Journal of the American Statistical Association, 1999.

[4] Conformal Prediction for Time Series. IEEE TPAMI, 2023.

[5] ResCP: Reservoir Conformal Prediction for Time Series Forecasting. ICLR, 2026.

AuthorsOpenReview ↗

Rebuttal 3

Conditional coverage evaluation

Although evaluating conditional coverage remains challenging, particularly in time series settings, we agree that the suggested approach provides a useful proxy for assessing conditional coverage. Accordingly, we designed an additional experiment with the following procedure:

  1. Train a sequence encoder: We train an independent sequence encoder on the calibration set of each time series. The encoder is expected to map samples with similar temporal contexts or regimes to nearby representations in the latent space.

  2. Cluster test samples: We extract context representations for the test samples using the trained encoder and partition them into groups using k-means clustering.

  3. Evaluate group-wise coverage: We compute the coverage gap within each cluster to assess deviations from the target coverage level across different groups.

Air

MethodLR (k=3k=3)LR (k=5k=5)LSTM (k=3k=3)LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.003-0.005-0.025-0.028-0.005-0.005
DSCP-Local-0.021-0.022-0.021-0.020-0.003-0.005

Solar

MethodLR (k=3k=3)LR (k=5k=5)LSTM (k=3k=3)LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.025-0.026-0.033-0.034-0.011-0.011
DSCP-Local-0.009-0.014-0.007-0.017-0.008-0.012

Sapflow

MethodLR (k=3k=3)LR (k=5k=5)LSTM (k=3k=3)LSTM (k=5k=5)Chronos (k=3k=3)Chronos (k=5k=5)
DSCP-QR-0.020-0.025-0.042-0.052-0.067-0.079
DSCP-Local-0.015-0.017-0.010-0.018-0.020-0.022

The results show that the group-wise coverage gaps for DSCP-Local consistently remain above the threshold of ΔCov=−0.25α\Delta\mathrm{Cov}=-0.25\alpha. DSCP-QR exhibits larger coverage gaps, (3 out of all 9 experiments across 3 datasets and 3 base predictors under k=3 groups; and 5 out of all 9 experiments under k=3 groups), where this observation is consistent with the overall coverage trends reported in Table 1, which means conditional coverage gaps in those cases partially due to the undercoverage in marginal coverage. We have updated the main results section of the manuscript to include these results and added the experimental setup and implementation details to the appendix.

Reviewer_kWmJOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies three approaches to uncertainty quantification for the time series forecasting using sequence model, including conditional quantile regression (DSCP-QR), conditional quantile-function estimation (DSCP-QFE), and similarity-weighted local calibration (DSCP-Local). The authors establish asymptotic coverage under assumptions and empirically evaluate the proposed methods using LSTM and Transformer encoders on three real world datasets. The experiments show that DSCP-Local obtains competitive interval efficiency with coverage close to the target in most cases.

Critical strengths
  1. The theorem is clearly stated, the assumptions are explicitly listed, and the proofs are sound. The distinction between representation- and context-conditional coverage is potentially useful.
  2. The experiment test the proposed quantification methods across datasets, base predictors, encoder architectures, and coverage levels. The practical evidence supports the efficacy of DSCP-Local and limitations of the DSCP-QFE are identified.
Critical weaknesses
  1. The DSCP-Local theorem does not directly cover the softmax similarity weights and the windows used in the experiments. It would be better if the authors could extend the analysis to cover the methods used in the practical implementation, or reason for why the theorem could cover this.
  2. The related literature cites CT-SSF and Transformer conformal prediction, but the baselines do not include these two. It would be helpful for direct comparison if these two methods can be included.
  3. The connection between the assumptions and the actual architectures needs further justification. For DSCP-QR and DSCP-QFE, the assumptions are vanishing approximation error, but it is not clear whether their LSTM/Transformer encodes and quantile heads satisfy this. Similarly, for DSCP-Local, it requires the learned representative to preserve the conditional residual distribution, which is a more or less strong requirement. It would be better if the authors can explain how the architectures satisfy these assumptions and, if cannot, if these assumptions can be relaxed.
Questions for authors
  1. How should neural architectures be chosen to satisfy the assumptions?
  2. Can the DSCP-Local analysis be extended to the implemented softmax weights?
3 comments
AuthorsOpenReview ↗

Rebuttal 1

Bridge between theoretical analysis and actual implementation of the methods

We thank the reviewer for constructive suggestions. We agree that the original assumptions did not directly cover the softmax weights. Correspondingly, in the revision, we have successfully extended our analysis to cover the softmax weight families based on cosine similarity and squared Euclidean distance. In the revised Appendix A.3.1, we remove the compact-support requirement and establish the same conditional-CDF consistency and coverage guarantee results. The revised assumptions now only require bounded nonnegative kernels with sufficient expected kernel mass and vanishing population CDF bias. We note that these are strict relaxations, i.e., the original assumptions imply the revised ones.

Following, we have also added discussion of familiar kernel function families that are covered by revised analysis, including Gaussian kernels in particular. For squared Euclidean distance, weights proportional to exp⁡(−∥H−h∥2/ϑ)\exp(-\lVert H-h \rVert^2 / \vartheta) are the Gaussian kernel weights. For cosine weighting, as we are considering normalized representations, it reduces to exp⁡(ϑH⊤h)=exp⁡(ϑ)exp⁡(−ϑ∥H−h∥2/2)\exp(\vartheta H^\top h) = \exp(\vartheta) \exp(-\vartheta \lVert H - h \rVert^2/2). These correspondences place the Euclidean and cosine softmax weights within the revised analysis. Raw dot-product weights are not generally covered by this identification. We have accordingly revised Appendix B.3 to clarify these connections.

Consistent with this, Table 22 provides empirical support for these choices. Cosine and Euclidean achieves better performance in general compared to dot product, while avoiding the large undercoverage observed for dot product in several settings.

Regarding calibration window, we select window length and temperature parameter using validation data. Combining asymptotic analysis with finite-sample evaluation and data-driven bandwidth is an established practice in classical kernel estimation [1-3]. Combining asymptotic coverage analysis with finite-window implementation also commonly appears in time-series conformal literature [4, 5]. This is because direct empirical verification of the asymptotic guarantee is necessarily limited. The analysis considers increasing calibration sizes with shrinking bandwidths, whereas experiments can examine only a finite range of these settings. Therefore our theorem characterizes conditional coverage as the calibration sample size increases under the stated bandwidth conditions, while our experiments assess performance at the selected finite sizes.

[1] Li, Lin & Racine. Optimal Bandwidth Selection for Nonparametric Conditional Distribution and Quantile Functions. Journal of Business & Economic Statistics, 2013.

[2] Hall, Racine & Li. Cross-Validation and the Estimation of Conditional Probability Densities. Journal of the American Statistical Association, 2004.

[3] Hall, Wolff & Yao. Methods for Estimating a Conditional Distribution Function. Journal of the American Statistical Association, 1999.

[4] Xu & Xie. Conformal Prediction for Time Series. IEEE TPAMI, 2023.

[5] Neglia, Cini, Bronstein & Bianchi. ResCP: Reservoir Conformal Prediction for Time Series Forecasting. ICLR, 2026.

AuthorsOpenReview ↗

Rebuttal 2

CT-SSF and Transformer Conformal Prediction as baselines

Thank you for the suggestion. Transformer-based conformal prediction (SPCI-T) shares the same architecture as our DSCP-QR with a Transformer encoder, except that SPCI-T does not incorporate our proposed architectural constraints on the quantile prediction head designed to prevent quantile crossing. The experimental results for DSCP-QR with a Transformer encoder and the constrained prediction head are reported in Tables 12–14 of the appendix. We additionally report the results for DSCP-QR with a Transformer encoder but without the quantile prediction head constraints in Table 17 of the appendix. In this setting, we observe that quantile crossing occurs.

CT-SSF relies on the feature encoder of the base predictor to obtain representations for conformal prediction. Consequently, it requires the base predictor to have an accessible, learnable feature encoder, making it not fully model-agnostic. In our experimental setup, CT-SSF is therefore not directly applicable when using linear regression (LR) as the base predictor. Also, when using Chronos as the base predictor, we generate forecasts for 10 steps ahead, with the same latent representation used across all 10 forecast horizons. Therefore, CT-SSF cannot be directly applied in the manner originally intended by its authors, as it relies on the feature representations provided by the base predictor for each forecast.

Although CT-SSF is applicable to our LSTM base predictor setup, it requires access to both of the feature encoder and prediction head of the LSTM base predictor. Therefore, a separate experimental setup involving retraining the base predictor is necessary. The table below presents additional experiments comparing CT-SSF with DSCP-QR-RNN and DSCP-Local-RNN using an LSTM base predictor across three datasets at a target coverage level of 0.9. For CT-SSF, we adopted a hyperparameter tuning procedure similar to that used for DSCP-Local based on the choice of the original publication. Specifically, we used the Adam optimizer with a learning rate of 0.01 and a maximum of 200 inversion steps. The attention mechanism was configured with a dimension of 32 and a temperature of 0.5. The calibration dataset was partitioned into 60% for training, 15% for validation, and 25% for calibration.

While CT-SSF obtain stable coverage at target coverage level, it output larger prediction intervals compared to DSCP-QR and DSCP-Local.

MethodAirSolarSapflow
ΔCovPI-WWSΔCovPI-WWSΔCovPI-WWS
DSCP-QR-RNN−0.021 ± 0.01962.3 ± 9.0103.7 ± 13.0−0.015 ± 0.10658.2 ± 10.689.5 ± 104.0−0.016 ± 0.138779.8 ± 441.01177.5 ± 1014.4
DSCP-Local-RNN+0.000 ± 0.00369.1 ± 9.4115.0 ± 14.3+0.023 ± 0.01066.0 ± 15.387.7 ± 17.7+0.018 ± 0.013611.2 ± 423.2856.5 ± 629.4
CT-SSF−0.004 ± 0.00274.1 ± 8.3131.1 ± 14.1+0.020 ± 0.00871.2 ± 18.2119.2 ± 21.6+0.019 ± 0.011911.1 ± 732.71320.9 ± 1017.1
AuthorsOpenReview ↗

Rebuttal 3

Architectures satisfying the assumptions

We thank the reviewer for the valuable feedback. For model class assumptions (Assumption A.3 and A.14) of QR and QFE, it suffices to have a fixed finite encoder-head architecture, such as LSTM or Transformer, with compact context support and parameters in a common compact set. However, it is true that choosing an LSTM or Transformer alone does not establish vanishing approximation error. Joint approximation remains a separate requirement about the target distribution and class of encoder heads.

As an illustrative example, consider ϵ^=m(Z)+ψ(U)\widehat \epsilon = m(Z) + \psi(U), where U∼Unif⁡(0,1)U \sim \operatorname{Unif}(0,1) is independent of ZZ, m(z)m(z) is an affine transformation of an encoder output, and ψ\psi is a feasible strictly increasing QFE head. Then, q⋆(z,τ)=m(z)+ψ(τ)q^\star(z, \tau) = m(z) + \psi(\tau), and QFE represents this directly while QR uses positive constant increments between quantile levels. In this setting, both joint approximation errors vanish. Still, we acknowledge that joint approximation remains a substantive condition for general data distributions. We have added discussion regarding this justification and limitation in revised Appendix A.1.

For Local, we first emphasize that this framework extends direct context-based residual estimation to calibration in an encoder representation space, which generalizes the direct residual history conditioning in, e.g., [1, 2]. Local allows flexible choice of representation, even with identity encoder which recovers direct context-based calibration and hence ZZ-conditional coverage. Residual sufficiency is a general condition that specifies what a representation must preserve to recover the original ZZ-conditional distribution. In our fixed encoder setting, this is precisely the condition needed to recover the full ZZ-conditional CDF and hence coverage for any choice of interval endpoints.

We here note that, for coverage at specified endpoints αl,αh\\{\alpha_l, \alpha_h\\}, the requirement can be weakened. Using the population pinball risk Rτ(q)R_\tau(q), suppose that heads gT,τg_{T, \tau} approach the true ZZ-conditional oracle risks at each τ∈αl,αh\tau \in \\{\alpha_l, \alpha_h\\}. Then, since the population HH-conditional quantile minimizes the risk among functions of HH, 0≤Rτ(qτH,⋆∘Enc⁡)−Rτ(qτ⋆)≤Rτ(gT,τ∘Enc⁡)−Rτ(qτ⋆)→0.0 \le R_\tau(q_\tau^{H, \star} \circ \operatorname{Enc}) - R_\tau(q_\tau^\star) \le R_\tau(g_{T, \tau} \circ \operatorname{Enc}) - R_\tau(q_\tau^\star) \to 0. Thus, the population HH-conditional endpoints attain the corresponding ZZ-conditional oracle risks. That is, with continuous ZZ-conditional residual CDF, we have the correct conditional probability levels at both endpoints and therefore context-conditional coverage.

[1] Xu & Xie. Sequential Predictive Conformal Inference for Time Series. ICML, 2023.

[2] Areces, Mohri, Hashimoto & Duchi. Online Conformal Prediction via Online Optimization. ICML, 2025.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

We sincerely appreciate the time and effort for reviewing the manuscript. Your comments have helped us substantially improve the manuscript. Below, we summarize the major changes made in response to your suggestions. Detailed responses to each point are provided under the corresponding reviewer comments. All changes in the revised manuscript are highlighted in blue.

  • Extended theoretical analysis: relaxed assumptions for DSCP-Local to cover more general kernel functions, including softmax weights used in experiments, with empirically supporting experiments (Appendix A.3.1, Table 21, Table 22).

  • Added three online adaptive conformal prediction methods to the baselines: ACI [1], LQT [2], COP [3] (Table 1, 15, 16).

  • Added group conditional coverage experiment as a proxy to conditional coverage measure (Section 4, Table 17).

  • Conducted ablation experiments to investigate miscoverage of DSCP-QFE (Section 4, Table 18, Table 19).

  • Conducted additional ablation experiments to investigate the effect of quantile level in training encoder for DSCP-Local (Section 4, Table 20).

[1] Adaptive conformal inference under distribution shift

[2] Online conformal prediction via online optimization

[3] Distribution-informed online conformal prediction

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.