- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DSSER addresses multivariate forecasting when historical observations are missing. It combines an STID-based forecasting model with a sparse evidence module that retrieves historical observations and uses them to make a bounded correction to the base forecast. During training, a teacher with access to the original history supervises student views with different missing rates. Teacher uncertainty adjusts the training losses, while a learned gate controls the evidence correction during inference.
Critical strengths
-
The paper is in good structure and well organized
-
Recovered information can hurt a forecast when it is unreliable. Maintaining a base prediction and limiting the size of the evidence-based correction is a sensible design. The teacher and its Monte Carlo uncertainty estimation are required only during training, which keeps the inference procedure relatively lightweight.
-
DSSER achieves the best results in all 36 main dataset–rate–metric comparisons. At 90% missingness, its MAE improvements over the strongest reported competing baseline are approximately 5.1% on PEMS04, 5.0% on METR-LA, and 4.7% on China-AQI.
Critical weaknesses
-
The paper adds considerable terminology and machinery to forecasting with missing observations. It is unclear what additional insight comes from framing prediction differences as “forecast drift,” particularly since lower drift does not necessarily correspond to better forecasting accuracy. The framework then combines multiple student views, Monte Carlo teacher uncertainty, several auxiliary losses, three evidence anchors, and a learned routing mechanism. Although the ablations show benefits from individual components, they do not establish whether this full framework is necessary compared with a simpler missingness-aware forecaster with a bounded residual correction. This makes the main contribution harder to identify and increases the training and tuning burden. A comparison against such a simpler alternative would help justify the complexity.
-
The meaning and evaluation of “uncertainty calibration” need clarification. As described, CSUC uses teacher uncertainty and missingness to adjust supervision weights, while RGER produces a learned gating score. It is unclear what statistical calibration property these quantities are intended to satisfy. The reported Spearman correlation and high-/low-risk MAE comparisons demonstrate risk discrimination, but do not establish calibration. The authors should first specify the calibration target and then provide an appropriate evaluation. For regression uncertainty, this could include predicted-error-scale versus empirical-RMSE reliability diagrams and ENCE, or coverage calibration and interval width if predictive intervals are available. If routing confidence represents the probability that a candidate correction improves the base forecast, an event-based reliability diagram and ECE would be appropriate. Otherwise, it should be described and evaluated as a gating weight rather than a calibrated probability. Although the paper cites calibration-related distillation work, its relationship to established regression calibration methods, such as Kuleshov et al. (2018) and Levi et al. (2022), remains insufficiently discussed. This distinction matters because the current evidence supports uncertainty-aware weighting more directly than statistical uncertainty calibration.
Questions for authors
See weekness above, the paper lacks the deep dive into the uncertainty calibration and complicates the whole process.
Thank you for the additional experiments. The results across different history lengths and forecasting horizons address my first concern. In particular, the tests with contiguous missing blocks and spatially correlated missingness address my main concern about robustness beyond random masking. I have no further questions and will raise my score.