- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses heterogeneous distribution shifts across components in conformal prediction for multivariate time-series forecasting. It proposes Boundary-aware Conformal Calibration (BaCR), which combines a unified recalibrator with component-wise trackers. Its main contributions include the calibration framework, theoretical analysis, and extensive experiments on datasets of varying scales using coverage and interval-quality metrics. Overall, I consider this a strong paper with clear methodological contributions and solid empirical validation.
Critical strengths
- The challenges are clear, and the methods to address the challenges are novel and easy but simple to understand and follow. Meanwhile, the experiments are solid. They cover both simulation and real-world datasets, and the evaluation metrics are not merely coverage but also the Winkler score. There are performance evaluations, ablation studies, and parameter sensitivity analysis to fully verify the effectiveness of the proposed method.
Critical weaknesses
- The paper mentions the heterogeneity of different components, but there are no tests on this part, and also I am wondering if there is heterogeneity in the dataset used in the paper. Also, the evaluation metrics include coverage and the Winkler score, but I think the interval width is more important to see if the method really improves the coverage instead of making a sacrifice to efficiency of the interval width to get better coverage. A lower Winkler score does not necessarily mean narrower intervals.
Questions for authors
- What is the performance when there is no heterogeneity in the component signal? Also, when a less severe or more severe distribution shift occurs, what will the performance of the proposed method be?
- Could you provide the interval-width measurements across different baselines and the proposed method?
4 comments
Rebuttal (2/2)
Q2 (Width):
We agree that lower Winkler scores do not necessarily imply narrower intervals. We report widths with ΔCov below. Among methods with |ΔCov| ≤ 1 pp, BaCR has the lowest width in 6 of 8 settings: AIR, LA, and BAY overall, and AIR/LA/SD (). On BAY , BaCR exceeds the coverage range (−1.2 pp), while HopCPT is inside and narrower. The BAY gap is from five of 32 components (−5.7 pp, versus −0.4 pp for the other 27). Their historical records show repeated weekly profiles or a newly emerging congestion pattern, suggesting that limited historical representativeness may contribute to BaCR’s remaining under-coverage.
| Dataset | Metric | SCP | NexCP | HopCPT | SPCI | CoRel | ACI | PID | BaCR |
|---|---|---|---|---|---|---|---|---|---|
| AIR | ΔCov(%) | +3.3 | +0.8 | +2.3 | -1.4 | -2.2 | -1.2 | -0.5 | +0.3 |
| Width | 75.4 | 66.7 | 73.7 | 67.3 | 54.3 | 75.0 | 73.6 | 62.7 | |
| ΔCov$_{S^+}$(%) | -1.4 | +0.7 | -0.1 | -4.3 | -5.7 | -1.6 | -0.7 | -0.6 | |
| Width$_{S^+}$ | 53.0 | 61.5 | 68.7 | 54.8 | 44.7 | 69.9 | 67.6 | 54.4 | |
| LA | ΔCov(%) | -2.3 | -0.0 | -0.1 | -5.4 | -1.1 | -1.9 | -0.0 | +0.4 |
| Width | 14.7 | 18.0 | 15.6 | 12.6 | 14.4 | 16.7 | 29.7 | 15.0 | |
| ΔCov$_{S^+}$(%) | -6.0 | -0.3 | -1.5 | -6.2 | -1.5 | -2.4 | -0.6 | -0.1 | |
| Width$_{S^+}$ | 12.6 | 21.1 | 16.4 | 13.1 | 15.3 | 18.9 | 52.3 | 15.7 | |
| BAY | ΔCov(%) | -0.6 | +0.3 | +0.1 | -0.5 | -1.1 | -1.3 | -2.1 | +0.4 |
| Width | 7.1 | 7.4 | 6.7 | 6.9 | 6.3 | 7.1 | 19.7 | 6.6 | |
| ΔCov$_{S^+}$(%) | -2.9 | +0.3 | -0.8 | -2.7 | -4.5 | -1.3 | -2.0 | -1.2 | |
| Width$_{S^+}$ | 7.7 | 10.2 | 8.3 | 7.8 | 7.4 | 9.4 | 23.4 | 8.6 | |
| SD | ΔCov(%) | -2.4 | +0.6 | -0.1 | -4.4 | -2.8 | -1.2 | -0.1 | +0.2 |
| Width | 96.3 | 106.9 | 90.1 | 86.9 | 96.1 | 103.9 | 114.0 | 97.4 | |
| ΔCov$_{S^+}$(%) | -4.7 | +0.6 | -0.4 | -5.3 | -4.7 | -1.3 | -0.1 | -0.5 | |
| Width$_{S^+}$ | 98.4 | 122.3 | 111.7 | 91.6 | 98.6 | 116.8 | 131.7 | 105.0 |
Thank you to the authors for the rebuttal, which addresses my concerns. I will maintain my positive assessment and current score.
Thank you for acknowledging our contribution and maintaining this positive assessment. We sincerely appreciate it!
Rebuttal (1/2)
We sincerely appreciate your efforts to reviewing our work and thanks for pointing out these issues. We have addressed each of your concerns below and will add all results into the revised manuscript. We believe that the revised version has been significantly improved as a result of your valuable suggestions. Please let us know if there are any remaining issues where further improvement is needed.
Heterogeneity is common in spatiotemporal data. Prior work treats heterogeneity across locations as a central property of traffic data, including LA, BAY and SD [1,2], and the source paper of the AIR [3] shows that air quality differs across stations.
We also measured this on our data: Components shift differently between calibration and test, some towards under-coverage and others towards over-coverage. For each component, we built a 90% SCP interval from its own calibration residuals and measured its coverage on the test set. Without shift, every component would cover about 90%. The table shows the distribution of these component-level coverage rates: the gap between the 5th and 95th percentiles is 7–12 percentage points (pp).
[1]Xu, Jiyuan, et al. "Cpiri: Channel permutation-invariant relational interaction for multivariate time series forecasting." ICLR 2026; [2]Dong, Zheng, et al. "Heterogeneity-informed meta-parameter learning for spatiotemporal time series forecasting." KDD 2024; [3]Zheng, Yu, et al. "Forecasting fine-grained air quality based on big data." KDD 2015.
Both experiments use the paper's synthetic settings in Section 6.2. Bold marks the lowest width and Winkler among methods with |ΔCov| ≤ 1 pp.
Heterogeneity. In None, components share AR coefficients, scale, volatility-cycle phase, regime change points/levels and tail parameters. Static adds component-specific fixed scales Ai, so components differ in level but shift identically. Hetero is the original paper setting.
BaCR keeps coverage error within the band in all settings, with the lowest width and Winkler score among methods with |ΔCov| ≤ 1 pp.
Shift severity. We vary regime strength d∈{0,0.8,1.6,2.4} with levels eζ, ζ∼N(0,(0.6d)2) (d=1.6 in the paper). At d=0, periodic volatility remains but regime shifts disappear.
Across severities, BaCR's coverage error stays within 0.4 pp. Its width rises from 2.31 to 3.14, tracking the oracle (2.19 to 2.87), and it achieves the lowest coverage-valid Winkler score. The only exception is width at d=0, where CoRel is marginally narrower but has a higher Winkler score.