acceptodds
Under review as a conference paper at ICLR 2027

Valid and Efficient Split Conformal Regression for Time Series

Abstract

We study conformalized quantile regression and conformalized median regression that fit a model on one block of a time series and calibrate the conformal interval on the adjacent block. The existing theory of conformal prediction for time series rests largely on mixing conditions, which are hard to verify from a time-series model and fail for many standard processes, including simple ones with short memory. We replace this theoretical toolbox with the functional dependence measure, which in principle accommodates long-memory observations. The accuracy of the conformal interval length for time series has been understudied. To the best of our knowledge, this paper is the first work that establishes non-asymptotic coverage guarantees and accuracy of interval length simultaneously for split conformal regression on time series. Furthermore, for Gaussian linear processes with long memory, where both the estimation of the center and its calibration converge slowly, we establish a sharper rate for the length error. We show that the calibrated length converges faster than the estimated center itself, and provide a matching lower bound for the usual centers when the calibration block is sufficiently large relative to the training block. To our knowledge, this is the first theoretical analysis of conformal interval length dedicated to long memory.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Time series, Functional dependence measure, Conformalized quantile regression, Long memory, Prediction interval length

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zmGkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the validity and efficiency of split conformal prediction for time series, focusing on Conformalized Quantile Regression (CQR) and Conformalized Median Regression (CMR). Unlike the usual routes relying on conventional mixing conditions, the authors use functional dependence measures to establish non-asymptotic bounds on marginal coverage errors and interval-length deviations from their respective oracles. They further investigate Gaussian long-memory processes and derive sharper convergence rates for interval length, along with a matching lower bound under additional conditions. The theoretical results are illustrated through simulation studies and an application to electricity load forecasting.

Critical strengths
  1. The use of functional dependence measures extends existing theoretical guarantees to a broader class of time series, including non-mixing and long-memory processes that are not covered by conventional mixing-based analyses.

  2. Theorems 1–3 provide non-asymptotic guarantees for both marginal coverage and interval-length accuracy under temporal dependence, addressing an important gap in the existing literature.

  3. The results in Section 4 are particularly interesting. Theorem 4 establishes sharper convergence rates for interval length under Gaussian long memory, while Proposition 5 provides a matching lower bound under additional conditions. The simulation studies also provide supporting evidence for these theoretical findings.

Critical weaknesses
  1. Theorem 4 is limited to Gaussian linear processes with constant covariates, making it unclear how broadly the sharper rates apply to general time-series regression. Appendix D.4 provides some encouraging results for symmetric non-Gaussian innovations, but the theoretical extension remains open. The matching lower bound also relies on additional restrictions.

  2. Assumption 1 requires uniform consistency of the fitted regression functions, which can be difficult to establish for flexible models such as neural networks. Although Appendices B.1–B.2 discuss consistency and misspecified learners, it is unclear what finite-sample rates can be expected for more general learning methods.

  3. The connection between the theory and the NYISO application could be clearer. The theoretical analysis assumes stationarity and a fixed data split, while the experiments use rolling refitting. The annual CMR coverage also varies considerably (0.833–0.934). Given that the evidence for long memory is mainly based on estimated Hurst exponents, further discussion of the underlying assumptions would be helpful.

  4. The efficiency results are relative to specific oracle intervals, which are not necessarily minimum-length intervals. The experiments also focus largely on verifying the theoretical rates, with few comparisons against existing time-series conformal prediction methods.

Questions for authors
  1. Section 6 briefly discusses extending Theorem 4 beyond Gaussian processes. What are the main technical obstacles to obtaining similar rates for symmetric non-Gaussian innovations or models with nonconstant covariates?

  2. For a fixed sample size T=n+mT=n+m, can the theoretical bounds offer any guidance on choosing the training-to-calibration ratio, especially under long memory?

  3. Could the authors clarify how the stationarity assumption relates to the NYISO application, and what the theoretical results imply for coverage under repeated rolling refitting?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zmGk (1/2)

We thank the reviewer for the careful reading and the constructive comments. All pointers refer to the revised version, in which the changes are marked in blue.

W1 and Q1

A more general theory of Section 4. The reviewer is right that Theorem 4 was stated for a Gaussian location model and that its extension was open. The revision adds the extension as Theorem 6, with a complete proof in Appendix C.8. Take iid symmetric innovations with a finite fourth moment and a Lipschitz density that is nonincreasing on [0,∞)[0,\infty). The bound of Theorem 4 then holds in the form (17) for every 1/2<β<11/2 < \beta < 1. Theorem 6 also covers nonconstant covariates. The constant location is replaced by a regression function μ(x)\mu(x) fitted by any learner, with covariates independent of the innovations. The fitted function enters the length through its mean squared error at the calibration covariates, that is, at second order.

Technical obstacles (Q1). There were two. The proof of Theorem 4 used tools that are specific to Gaussian laws. With a covariate, the error of the fitted function varies from one calibration point to the next. The revision replaces the Gaussian tools by a bound on the functional dependence measure that needs only the symmetry of the innovations (Lemma 22 and Step 3 of the proof). It handles the fitted function by conditioning on the training block and on the covariates.

Why a linear process in Section 4. Theorems 1–3 already apply to long-memory processes without a model (Appendix B.5). Section 4 asks for more, a sharper rate with a matching lower bound. Such a rate depends on the form of the dependence, so it is stated for a specified class of processes. This is common practice in the long-memory literature (Dehling & Taqqu, 1989; Wu, 2003). The linear process is the standard class, and its form is representative. By Wold's decomposition, every centered, purely nondeterministic stationary process has a linear representation with uncorrelated innovations (Brockwell & Davis, 1991, Section 5.7).

Lower bound. The revision removes the condition that related mm to nn. The lower bound (16) now holds for all n≥n0n\ge n_0 and m≥m0′m\ge m_0'. Its proof also gives a lower bound of order rm(β)r_m(\beta) for the calibration term, the known center included (Step 3 in Appendix C.6). These lower bounds are for the Gaussian location model.

References. Dehling & Taqqu (1989), Wu (2003) and Brockwell & Davis (1991) are already cited in the paper.

W2

Assumption 1 was stronger than necessary, and we have weakened it in the revision (proofs in Appendix A.4). For the coverage bounds (4) and (8) and for the CMR length bound (10), a small fitting error at the calibration and test covariates suffices. Uniform consistency is not needed. When these covariates, given the training block, have a bounded density with respect to the stationary covariate law, this error is controlled by the L1L^1 or L2L^2 error of the learner. An L1L^1 or L2L^2 rate of the learner is then sufficient (Appendix B.1).

AuthorsOpenReview ↗

Response to Reviewer zmGk (2/2)

W3 and Q3

Stationarity. We agree that real series are stationary only approximately. The raw data are the hourly loads, which have strong daily and annual cycles. The operator's day-ahead forecast carries these cycles. We use what remains after the forecast is subtracted, the percentage forecast error, with a separate daily series for each hour of the day. Such a series is much closer to a stationary one. We also checked the homogeneity of the errors over time, through the yearly median and the yearly interquartile range of the daily mean error. The reference level is 2.60% of the forecast, taken from 2005–2013, the years used to fix the design. In the twelve complete evaluation years, 2014–2025, the yearly median lies between 1.87 and 3.04. It is always within one percentage point of the reference, a band fixed before the evaluation. The interquartile range lies between 1.37 and 1.91, against 1.35 to 1.74 in 2005–2013. Only 2026, for which eight months are available, has a median outside the band.

Rolling refitting. The revision closes this gap. The end of Appendix A.4 now shows that, on a stationary series, the guarantees of Theorems 1–3 hold at every origin of a rolling scheme. It also shows that the fraction of covered targets converges almost surely to the common marginal coverage. This concerns the long-run fraction, not the coverage of a single year. The variation of the yearly coverage between .833 and .934 is therefore outside the guarantee. Under long memory, the coverage of a single year is expected to vary more than with independent errors.

Evidence for long memory. The Hurst exponent is a common diagnostic of long memory in applied work, and a series with a Hurst exponent above .5 is regarded as long-range dependent (Leland et al., 1994; Zhai et al., 2026). Long memory is not an assumption of Theorems 1–3, which cover short and long memory alike.

References. Leland et al. (1994) and Zhai et al. (2026) are already cited in the paper.

W4

Oracle intervals. Each procedure has its own target. CQR targets the interval between two conditional quantiles, and CMR targets the conditional median plus or minus a fixed radius. The oracle interval of a procedure is this target. Our theorems show that the calibrated interval attains its length at an explicit rate. In Section 4 and in Studies 2 and 3, the law of the response is symmetric and unimodal. The target is then also the shortest set with coverage 1−α1-\alpha among those that do not use the past. On real data no oracle is known, and the hindsight width serves as a proxy for the oracle width.

Comparison. The revision adds Table 3 in Appendix E. It compares CMR and CQR with NexCP (Barber et al., 2023), ACI (Gibbs & Candès, 2021) and a gapped split on the same targets, with the same forecast and the same information. The overall coverage of all rules lies between 89.5% and 90.2%. CMR and CQR have narrower intervals on average than NexCP and ACI, which have steadier yearly coverage. A gap between the blocks changes almost nothing.

Purpose of the experiments. This paper does not intend to propose a new interval. It provides coverage and length guarantees for the standard split CQR and CMR intervals, and the experiments are designed around these guarantees. They are not meant as a benchmark of methods. The simulations test the guarantees where the oracle interval is known. The real data then illustrate them on more than twelve years of targets that were not used to design the procedures. Both intervals cover close to the nominal level, and the mean width of CMR is within 2% of the hindsight width, our proxy for the oracle width.

References. Barber et al. (2023) and Gibbs & Candès (2021) are already cited in the paper.

Q2

Our bounds give orders, so the guidance concerns the orders of nn and mm, not their exact ratio. In the model of Section 4, the length error is of exact order n−(2β−1)+rm(β)n^{-(2\beta-1)}+r_m(\beta) for all sufficiently large nn and mm, for the centers of Proposition 5 (Theorem 4 and the revised Proposition 5). For 1/2<β<3/41/2 < \beta < 3/4, the two terms have the same exponent. The best order, T−(2β−1)T^{-(2\beta-1)}, is then attained when nn and mm are both of order TT, at any fixed ratio. For 3/4<β<13/4 < \beta < 1, the calibration term m−1/2m^{-1/2} dominates. The best order, T−1/2T^{-1/2}, then needs mm of order TT, whereas nn may be of smaller order, down to T1/(4β−2)T^{1/(4\beta-2)}. In both cases the ratio changes the constant only, and the theory does not determine the constant.

Thank you for the detailed response and additional experiments, which have addressed most of my concerns. I appreciate the authors' efforts in strengthening the paper and maintain my positive assessment.

Reviewer_uDawOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a technique from prior research, a “functional dependence measure”, to address stationary time series where the standard “mixing” assumption fails. It establishes both non-asymptotic “coverage” guarantees and bounds on the “interval length” for Conformal Quantile Regression (CQR) and Conformal Mean Regression (CMR). Furthermore, in a Gaussian long-memory setting, the authors prove that the calibrated interval length converges at a faster rate than the estimated center, and they provide a matching lower bound for this result.

Critical strengths
  1. While much of the prior literature either guarantees only coverage or analyzes interval length only asymptotically, this paper guarantees them simultaneously and non-asymptotically for the same calibrated interval.
  2. The paper shows strong theoretical material, including the following: It analytically shows that Assumption 3 (functional dependence) holds for a diverse range of examples, including Bernoulli autoregressions, Gaussian long memory, long memory in the scale of a volatility model, and contractive nonlinear recursions, and it provides a concrete case, the Bernoulli autoregression, in which existing mixing-based bounds fail.
Critical weaknesses
  • One concern is that the paper assumes strictly stationary data (which the authors themselves acknowledge). Since real-world time series frequently exhibit non-stationary phenomena such as distributional shift, the practical implications of this assumption are not sufficiently discussed. It would be valuable to examine whether this assumption holds, even approximately, in real applications.
  • The real-data validation is limited to a single electricity load forecasting dataset, and there is no comparison with baseline methods (e.g., mixing-based approaches). As a result, the paper does not provide quantitative evidence of how much better the proposed intervals are compared to existing mixing-based approaches.
Questions for authors
  • The sharper rates in Theorem 4 / Proposition 5 rely on the symmetry of the innovations. In the Study 2 experiment with asymmetric exponential innovations, the observed rate at certain memory strengths was closer to the slower rate of Theorem 3 rather than Theorem 4. How do the authors explain this phenomenon, and can they provide even a partial proof for the case of asymmetric innovations?
  • Can the authors clarify why the stationary long-memory assumption is meaningful in practice?
  • In the real-data analysis, the lengths of the CQR and CMR intervals are compared only against a "proxy for the oracle length." Could the authors provide a direct comparison with existing mixing-based methods on the same dataset (e.g., SPCI [1], NexCP [2])?

[1] Xu, Chen, and Yao Xie. "Sequential predictive conformal inference for time series." International Conference on Machine Learning. PMLR, 2023. [2] Barber, Rina Foygel, et al. "Conformal prediction beyond exchangeability." The Annals of Statistics 51.2 (2023): 816-845.

1 comment
AuthorsOpenReview ↗

Response to Reviewer uDaw

We thank the reviewer for the careful reading and the constructive comments. All pointers refer to the revised version, in which the changes are marked in blue.

W1

We agree that real series are stationary only approximately.

Design. The raw data are the hourly loads, which have strong daily and annual cycles. The operator's day-ahead forecast carries these cycles. We use what remains after the forecast is subtracted, the percentage forecast error, with a separate daily series for each hour of the day. Such a series is much closer to a stationary one.

Check. We also checked the homogeneity of the errors over time, through the yearly median and the yearly interquartile range of the daily mean error. The reference level is 2.60% of the forecast, taken from 2005–2013, the years used to fix the design. In the twelve complete evaluation years, 2014–2025, the yearly median lies between 1.87 and 3.04. It is always within one percentage point of the reference, a band fixed before the evaluation. The interquartile range lies between 1.37 and 1.91, against 1.35 to 1.74 in 2005–2013. Only 2026, for which eight months are available, has a median outside the band.

W2 and Q3

Comparison. The revision adds Table 3 in Appendix E. It compares CMR and CQR with NexCP (Barber et al., 2023), ACI (Gibbs & Candès, 2021) and a gapped split on the same targets, with the same forecast and the same information. The overall coverage of all rules lies between 89.5% and 90.2%. CMR and CQR have narrower intervals on average than NexCP and ACI, which have steadier yearly coverage. A gap between the blocks changes almost nothing. SPCI (Xu & Xie, 2023) is not in the table. It fits conditional quantiles of the residuals given their lags, and thus conditions on more information.

Purpose of the experiments. This paper does not intend to propose a new interval. It provides coverage and length guarantees for the standard split CQR and CMR intervals, and the experiments are designed around these guarantees. They are not meant as a benchmark of methods. The simulations test the guarantees where the oracle interval is known. The real data then illustrate them on more than twelve years of targets that were not used to design the procedures. Both intervals cover close to the nominal level, and the mean width of CMR is within 2% of the hindsight width, our proxy for the oracle width. The revision also reports the ten zone areas, 240 hourly series, where the coverage of CMR has median .894 and lies between .846 and .910.

References. Barber et al. (2023) and Gibbs & Candès (2021) are already cited in the paper. Xu & Xie (2023), the reviewer's reference [1], is cited in the revision.

Q1

The reviewer is right that asymmetric innovations can give a slower rate. This is consistent with our theory. The paper has two sets of results. Theorems 1–3 of Section 3 need no symmetry, so they apply to asymmetric innovations and give the calibration rate m1/2−βm^{1/2-\beta}. Section 4 asks a different question: for a specific class of long-memory series, can the rate be sharper? Its answer is yes, and it assumes symmetric innovations. Without symmetry, the proven rate is that of Theorem 3. The simulation follows it at β=.75\beta=.75 and .9. Appendix D.4 gives the reason. With an asymmetric law, the error of the calibrated radius has a first-order term that is proportional to the difference of the density at the two endpoints of the oracle interval. At β=.6\beta=.6 the simulated rate stays close to that of Theorem 4, because long memory makes the law of the series more symmetric.

Q2

Long memory is common in real data. The volatility of stock returns clusters, and absolute returns have long memory (Ding et al., 1993). Network traffic, language and music show it as well (Leland et al., 1994; Greaves-Tunnell & Harchaoui, 2019). Our application is another example. The estimated Hurst exponents of all 264 forecast-error series exceed .5, with median .75, and a Hurst exponent above .5 indicates long memory (Appendix E). Under long memory the autocovariances are not summable, so the long-run variance of the classical central limit theorem does not exist. Many standard time-series methods rest on it and cannot handle long memory. Our theorems allow it, and Section 4 gives a sharper length rate for long-memory linear processes.

References. Ding et al. (1993), Leland et al. (1994) and Greaves-Tunnell & Harchaoui (2019) are already cited in the paper.

Reviewer_jPwFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper analyzes conformalized quantile regression (CQR) and conformalized median regression (CMR) on a stationary time series. The model is fit on one block and calibrated on the adjacent block. Instead of mixing, dependence is measured by Wu’s functional dependence measure, applied to thresholded oracle scores.

Theorems 1–3 give non-asymptotic marginal coverage and sup-norm length bounds. For Gaussian long memory with a constant covariate, Theorem 4 shows the center error enters the length only at second order. Proposition 5 gives a matching lower bound when the calibration block is much larger than the training block. Simulations confirm the rates, and an electricity-load application shows near-nominal coverage.

Critical strengths
  1. Theorems 1–3 bound coverage and length together, for adjacent blocks with no gap, on processes where mixing and switch-coefficient bounds are vacuous. This widens the class of series that split-conformal theory covers.
  2. The proof of the long-memory result handles the dependence between the center and the adjacent calibration block by conditioning on the training innovations. The simulations go beyond confirming rates: the skewed-innovation control (Table 1, App. D.4) shows the improvement really comes from symmetry.
Critical weaknesses
  1. Incremental general theory, with a gap to practice.
  • Theorems 1–3 combine known tools: the pathwise comparison, projection bound, and a Chebyshev quantile bound. Appendix A acknowledges this.
  • The Chebyshev bound gives only polynomial tails.
  • Guarantees are marginal, for one split and one test point. Assumption 1 needs sup-norm consistency of the learner, which flexible models rarely have.
  • The application uses rolling refits, which the theory does not cover.
  • There is no comparison with ACI and EnbPI/SPCI.
  1. Narrow long-memory setting; the “exact rate” is partly cosmetic.
  • Section 4 assumes a Gaussian process with a constant covariate, i.e., a location model. The authors’ own example in Appendix C.5 shows that covariate-dependent width errors enter at first order.
  • Proposition 5 needs m≥nmax(1,4β−2)log⁡nm \ge n^{max(1,4 \beta−2)} \log n, which is about $n^{1.6} $ at β=.9\beta = .9.
  • There is no lower bound for the calibration term, and none at n≃mn \simeq m, the regime Theorem 4 emphasizes.
Questions for authors
  1. Can the lower bound be shown at n≃mn \simeq m, or for the calibration term rm(β)r_m(\beta) with a known center?
  2. Do the guarantees extend to the rolling-origin refits used in Appendix E?
  3. On the NYISO data, how do coverage and width compare with ACI or a gapped split?
  4. Can Assumption 1 be weakened from sup-norm to L1/L2 error, so the theorems cover flexible learners?
  5. Does the second-order improvement survive for CMR with a fitted μ(x)\mu(x) and symmetric noise independent of X? The Appendix C.5 counterexample concerns CQR width errors.
3 comments
AuthorsOpenReview ↗

Response to Reviewer jPwF (1/2)

We thank the reviewer for the careful reading and the constructive comments. All pointers refer to the revised version, in which the changes are marked in blue.

W1

Known tools. We agree that the steps of the proof are known. The results are new in two respects. First, to our knowledge no earlier result compares the calibrated length with the oracle length at a non-asymptotic rate under temporal dependence. Second, the existing coverage theory rests largely on mixing conditions. These are hard to verify from a model and fail for many standard processes, including simple ones with short memory (examples in Appendices B.5 and B.6). Theorems 1–3 replace them with a condition on the functional dependence measure, which is verified from the model (Proposition 13).

Polynomial tails. Theorems 1–3 are stated without a model on purpose. Assumption 3 asks only for a polynomial decay of the functional dependence measure in L2L^2, with any exponent γ>1/2\gamma > 1/2. This controls second moments, and Chebyshev's inequality is the tool for second moments.

Marginal guarantee, one split, rolling refits. The guarantee is marginal, as it is for split conformal prediction with exchangeable data. On a stationary series, it holds at every origin of a rolling scheme. The fraction of covered targets then converges almost surely to the common marginal coverage (Appendix A.4, response to Q2).

Sup norm. Assumption 1 was stronger than necessary, and we have weakened it in the revision. For the coverage bounds (4) and (8) and for the CMR length bound (10), a small fitting error at the calibration and test covariates suffices (proofs in Appendix A.4). Under a density condition on the covariates, this error is bounded by the L1L^1 or L2L^2 error of the learner up to a constant (response to Q4).

ACI, EnbPI and SPCI. Table 3 in Appendix E compares CMR and CQR with ACI (Gibbs & Candès, 2021), NexCP (Barber et al., 2023) and a gapped split on the same targets (response to Q3). EnbPI (Xu & Xie, 2021) and SPCI (Xu & Xie, 2023) are not in the table. EnbPI resamples a fitted predictor, and the predictor here is the operator's forecast. SPCI fits conditional quantiles of the residuals given their lags, and thus conditions on more information.

References. Gibbs & Candès (2021), Barber et al. (2023) and Xu & Xie (2021) are already cited in the paper. Xu & Xie (2023) is cited in the revision: C. Xu and Y. Xie, Sequential predictive conformal inference for time series, ICML 2023, PMLR 202:38707–38727.

W2

Why a linear process in Section 4. Theorems 1–3 already apply to long-memory processes without a model (Appendix B.5). Section 4 asks for more, a sharper rate with a matching lower bound. Such a rate depends on the form of the dependence, so it is stated for a specified class of processes. This is common practice in the long-memory literature (Dehling & Taqqu, 1989; Wu, 2003). The linear process is the standard class, and its form is representative. By Wold's decomposition, every centered, purely nondeterministic stationary process has a linear representation with uncorrelated innovations (Brockwell & Davis, 1991, Section 5.7).

A more general theory of Section 4. The reviewer is right that Theorem 4 was stated for a Gaussian location model. The revision adds Theorem 6, with a complete proof in Appendix C.8. Take iid symmetric innovations with a finite fourth moment and a Lipschitz density that is nonincreasing on [0,∞)[0,\infty). The bound of Theorem 4 then holds in the form (17) for every 1/2<β<11/2 < \beta < 1. Theorem 6 also replaces the constant location by a regression function μ(x)\mu(x) fitted by any learner, with covariates independent of the innovations. The fitted function enters the length through its mean squared error at the calibration covariates, that is, at second order. The example of Appendix C.5 concerns CQR, whose fitted width is part of the length. CMR fits a center only (response to Q5).

Proposition 5. The revision removes the condition on mm. The lower bound (16) now holds for all n≥n0n\ge n_0 and m≥m0′m\ge m_0', with no condition relating the two block sizes. This includes n≃mn\simeq m, where the exact rate is rn(β)r_n(\beta). Its proof also gives a lower bound of order rm(β)r_m(\beta) for the calibration term, the known center included (Step 3 in Appendix C.6, response to Q1). These lower bounds are for the Gaussian location model.

References. Dehling & Taqqu (1989), Wu (2003) and Brockwell & Davis (1991) are already cited in the paper.

AuthorsOpenReview ↗

Response to Reviewer jPwF (2/2)

Q1

Yes to both. We have added new theory and revised Proposition 5. Its proof gives a lower bound of order rm(β)r_m(\beta) for the calibration term, which holds with a known center (Step 3 in Appendix C.6). Its bound (16) now holds with no condition relating nn and mm, which covers n≃mn\simeq m.

Q2

Yes, on a stationary series. We have revised the paper on this point. The end of Appendix A.4 now shows that the guarantees of Theorems 1–3 hold at every origin of a rolling scheme. It also shows that the fraction of covered targets converges almost surely to the common marginal coverage. In Appendix E this applies to CMR. The CQR fit there uses functions of the date as covariates, which the stationary setting does not cover.

Q3

Table 3 in Appendix E of the revision makes this comparison. It uses the same 111,010 targets, the same forecast and the same information, with n=m=730n=m=730 and nominal level 90%. The other rules were added after the evaluation of CMR and CQR, and their parameters were fixed on earlier data.

A gap of 7 or 30 days between the blocks changes the coverage of CMR by .02 to .03 percentage points and its mean width by less than 1 MW. The overall coverage of ACI (Gibbs & Candès, 2021) and NexCP (Barber et al., 2023) is 89.83% and 90.23%, against 89.54% for CMR. Their yearly coverage varies less, 87.7–92.3% and 87.9–92.4% against 83.3–93.4%. Their intervals are wider on average, 5.92% and 5.97% of the forecast against 5.81%. ACI returned the whole real line at three targets, and its width is computed without them.

Purpose of the experiments. This paper does not intend to propose a new interval. It provides coverage and length guarantees for the standard split CQR and CMR intervals, and the experiments are designed around these guarantees. They are not meant as a benchmark of methods. The simulations test the guarantees where the oracle interval is known. The real data then illustrate them on more than twelve years of targets that were not used to design the procedures. Both intervals cover close to the nominal level, and the mean width of CMR is within 2% of the hindsight width, our proxy for the oracle width.

References. Gibbs & Candès (2021) and Barber et al. (2023) are already cited in the paper.

Q4

Yes, under a condition on the covariates. The revision replaces the sup norm by the fitting error at the calibration and test covariates (Assumption 1). When these covariates, given the training block, have a bounded density with respect to the stationary covariate law, this error is controlled by the L1L^1 or L2L^2 error of the learner (Appendix B.1). This applies to the coverage bounds (4) and (8) and to the CMR length bound (10). The other bounds still use the sup norm.

Q5

Yes. This is Theorem 6 of the revision. The fitted function enters the length through its mean squared error at the calibration covariates, that is, at second order (proof in Appendix C.8). We also added a simulation in which the center is a linear regression fitted by least squares. Its length error decreases at the same rate as in the location model (Appendix D.6).

Reviewer_jPwFOpenReview ↗

Thanks for your asnwers

I would like to thank the authors for the substantial rebuttal that partially addresses my concerns. However, I tend to keep my already positive score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thank You – General Comments and Summary of Main Changes

We thank the three reviewers for their careful reading and their constructive comments. We have uploaded a revised version of the paper. All changes are marked in blue. This comment lists the main changes. The responses to the individual reviewers give the details.

1. A more general theory of Section 4 (Reviewers jPwF and zmGk). Theorem 4 was stated for a Gaussian location model. The new Theorem 6 extends its upper bound to symmetric innovations that are not Gaussian, under the conditions stated in the theorem. It also replaces the constant location by a regression function fitted by any learner, with covariates independent of the innovations. The proof is in Appendix C.8. In a new simulation with a least-squares fit, the length error decreases at the same rate as in the location model (Appendix D.6).

2. Lower bound of Proposition 5 (Reviewers jPwF and zmGk). The revision removes the condition that related the two block sizes. The lower bound (16) now holds for all sufficiently large nn and mm, which includes blocks of comparable size. The proof is in Appendix C.6.

3. Assumption 1 (Reviewers jPwF and zmGk). Assumption 1 was stronger than necessary, and we have weakened it in the revision. It now bounds the fitting error at the calibration and test covariates only. Uniform consistency of the learner is not needed for the coverage bounds (4) and (8), which are restated under the weaker assumption (proofs in Appendix A.4). Under a density condition on the covariates, an L1L^1 or L2L^2 rate of the learner is sufficient (Appendix B.1). The submitted bounds remain valid under the original assumption.

4. Rolling origins (Reviewers jPwF and zmGk). The theorems were stated for one split. The end of Appendix A.4 now shows that, on a stationary series, the guarantees of Theorems 1–3 hold at every origin of a rolling scheme. The fraction of covered targets then converges almost surely to the common marginal coverage.

5. Real data analysis (all reviewers). Section 5 now presents the real data analysis in the main text, with its main figure (Figure 4). Table 3 in Appendix E is new. It compares CMR and CQR with NexCP (Barber et al., 2023), ACI (Gibbs & Candès, 2021) and a gapped split on the same targets, with the same forecast and the same information. This paper does not intend to propose a new interval. It provides coverage and length guarantees for the standard split CQR and CMR intervals, and the experiments are designed around these guarantees.

Numbering. Theorems 1–4, Proposition 5, equations (1)–(16) and the section numbers are unchanged. Theorem 6 and equation (17) are new, so each numbered result of the appendices moves up by one. All pointers in our responses refer to the revised version.

References. Barber et al. (2023) and Gibbs & Candès (2021) are already cited in the paper.

Dear reviewers and AC,

As the discussion period is coming to an end soon, we would love to hear your feedback and answer any followup questions. Again, we appreciate your time and efforts in this revision process.

Best, 54770 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.