- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how deep sequence models can be incorporated into conformal prediction for time series through three approaches: conditional quantile regression (DSCP-QR), conditional quantile function estimation (DSCP-QFE), and localized conformal prediction (DSCP-Local). The authors establish asymptotic conditional coverage guarantees under suitable assumptions and evaluate the proposed methods on three real-world datasets using different base predictors and sequence encoders. The experimental results demonstrate competitive prediction interval efficiency for DSCP-QR and relatively stable coverage for DSCP-Local, while also revealing limitations of DSCP-QFE.
Critical strengths
- The proposed framework is clearly organized, and the distinctions between quantile-specific prediction heads, continuous quantile function estimation, and representation-based localization are well explained.
- The ablation studies on quantile crossing constraints and quantile levels provide useful observations regarding the trade-off between architectural constraints and predictive performance.
Critical weaknesses
- DSCP-QR and DSCP-QFE appear to construct prediction intervals directly from estimated conditional residual quantiles. They do not provide finite-sample distribution-free coverage guarantees, and their validity relies instead on the consistency of the estimated conditional quantiles under additional statistical assumptions.
- The theoretical guarantees rely on restrictive assumptions and these assumptions may be difficult to justify for highly overparameterized deep sequence models. The theoretical results primarily follow from the consistency of conditional quantile or CDF estimation. The paper could better clarify what new theoretical insights are obtained specifically from incorporating deep sequence models.
- The theoretical contribution emphasizes asymptotic conditional coverage, but the experiments primarily report marginal empirical coverage.
- The experimental comparison omits many recent online/adaptive conformal baselines, including:
[1]: Areces F, Mohri C, Hashimoto T, et al. Online conformal prediction via online optimization[C]//Forty-second International Conference on Machine Learning. 2025.
[2]: Hu D, Wu J, Xia S T, et al. Distribution-informed online conformal prediction[C]//International Conference on Learning Representations. 2026.
Questions for authors
- Can the theoretical guarantees for DSCP-Local be extended to the actual experimental implementation? The theoretical analysis assumes a compactly supported kernel with a shrinking bandwidth and a fixed encoder, whereas the experiments use softmax similarity weights and finite calibration windows. Can the authors provide theoretical justification for these practical choices, or alternatively evaluate an implementation that directly satisfies Assumptions A.21–A.24? How would the analysis change when the encoder is learned from temporally dependent observations?
- Can the authors provide a more direct empirical assessment of conditional coverage? Since conditional coverage is a central theoretical motivation, it would be valuable to evaluate coverage across context-dependent groups, different uncertainty levels, and different temporal regimes.
7 comments
Rebuttal 2
The theoretical guarantees rely on restrictive assumptions and these assumptions may be difficult to justify for highly overparameterized deep sequence models. The theoretical results primarily follow from the consistency of conditional quantile or CDF estimation. The paper could better clarify what new theoretical insights are obtained specifically from incorporating deep sequence models.
Thank you for raising this point. We introduced deep sequence models to benefit from their expressive representations of temporal context. For QR and QFE, we acknowledge that the approximation and optimization conditions are difficult to verify for general deep networks. Here we provide an illustrative example where joint approximation can be satisfied for QR and QFE.
Consider , where is independent of , is an affine transformation of an encoder output, and is a feasible strictly increasing QFE head. Then, , and QFE represents this directly while QR uses positive constant increments between quantile levels. In this setting, both joint approximation errors vanish.
Still, joint approximation remains a substantive condition for general data distributions. Our theoretical contribution is to specify sufficient conditions that the deep sequence encoder and associated estimators must satisfy to obtain consistency, and then to establish how this consistency yields conditional coverage under temporal dependence. We have added the discussion about this justification and limitation in Appendix A.1.
Rebuttal 3
The theoretical contribution emphasizes asymptotic conditional coverage, but the experiments primarily report marginal empirical coverage.
Although evaluating conditional coverage remains challenging, particularly in time series settings, we agree that the suggested approach provides a useful proxy for assessing conditional coverage. Accordingly, we designed an additional experiment with the following procedure:
-
Train a sequence encoder: We train an independent sequence encoder on the calibration set of each time series. The encoder is expected to map samples with similar temporal contexts or regimes to nearby representations in the latent space.
-
Cluster test samples: We extract context representations for the test samples using the trained encoder and partition them into groups using k-means clustering.
-
Evaluate group-wise coverage: We compute the coverage gap within each cluster to assess deviations from the target coverage level across different groups.
Air
| Method | LR () | LR () | LSTM | LSTM () | Chronos () | Chronos () |
|---|---|---|---|---|---|---|
| DSCP-QR | -0.003 | -0.005 | -0.025 | -0.028 | -0.005 | -0.005 |
| DSCP-Local | -0.021 | -0.022 | -0.021 | -0.020 | -0.003 | -0.005 |
Solar
| Method | LR () | LR () | LSTM () | LSTM () | Chronos () | Chronos () |
|---|---|---|---|---|---|---|
| DSCP-QR | -0.025 | -0.026 | -0.033 | -0.034 | -0.011 | -0.011 |
| DSCP-Local | -0.009 | -0.014 | -0.007 | -0.017 | -0.008 | -0.012 |
Sapflow
| Method | LR () | LR () | LSTM () | LSTM () | Chronos () | Chronos () |
|---|---|---|---|---|---|---|
| DSCP-QR | -0.020 | -0.025 | -0.042 | -0.052 | -0.067 | -0.079 |
| DSCP-Local | -0.015 | -0.017 | -0.010 | -0.018 | -0.020 | -0.022 |
The results show that the group-wise coverage gaps for DSCP-Local consistently remain above the threshold of . DSCP-QR exhibits larger coverage gaps, (3 out of all 9 experiments across 3 datasets and 3 base predictors under k=3 groups; and 5 out of all 9 experiments under k=3 groups), where this observation is consistent with the overall coverage trends reported in Table 1, which means conditional coverage gaps in those cases partially due to the undercoverage in marginal coverage. We have updated the main results section of the manuscript to include these results and added the experimental setup and implementation details to the appendix.
Rebuttal 1
DSCP-QR and DSCP-QFE appear to construct prediction intervals directly from estimated conditional residual quantiles. They do not provide finite-sample distribution-free coverage guarantees, and their validity relies instead on the consistency of the estimated conditional quantiles under additional statistical assumptions.
We appreciate the reviewer’s request for clarification. Our formulation follows the sequential residual-quantile approach. Within this setting, our analysis connects joint quantile training for QR and training over quantile levels for QFE to consistent endpoint estimation and coverage within temporally dependent data.
Finite-sample distribution-free coverage guarantee is inherently challenging in this setting since temporal dependence prevents us from directly invoking the standard exchangeability-based guarantees of conformal calibration. In the related approach of CQR [1], for instance, the finite-sample marginal coverage guarantee is obtained through a conformal calibration step under exchangeability. However, validity for nonexchangeable time-series data requires appropriate dependence-aware argument.
DSCP-QR and DSCP-QFE directly estimate conditional residual quantiles without such additional conformal calibration step. Our stated guarantee is asymptotic coverage conditional on the forecasting context, established through consistent estimation, which is a common target in conformal prediction literature [2, 3, 4]. We follow this approach by stating explicit sufficient conditions on dependence structure and regularity on encoders, and show how they connect the proposed training objectives to conditional coverage.
[1] Romano, Patterson & Candes. Conformalized Quantile Regression. NeurIPS, 2019.
[2] Xu & Xie. Sequential Predictive Conformal Inference for Time Series. ICML, 2023.
[3] Lee, Xu & Xie. Kernel-based Optimally Weighted Conformal Time-Series Prediction. ICLR, 2025.
[4] Chernozhukov, Wüthrich & Zhu. Distributional conformal prediction. Proceedings of the National Academy of Sciences, 2021.