- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies post-hoc scenario selection from probabilistic forecasts, where a large pool of possible futures must be compressed into only K representative scenarios. The authors identify a limitation of conventional average-optimal selection, which may omit low-probability but important modes even when they are present in the generated pool. To address this, they propose ModeRisker, which optimizes a mode-level Conditional Value-at-Risk (CVaR) objective over coverage losses and uses a supergradient-linearized greedy procedure to select scenarios based on CVaR-weighted marginal gains, together with an outlier-rescue mode-inference mechanism and a modality screen for determining when risk-aware selection is appropriate. Experiments on controlled synthetic data and three real-world domains: electricity demand, traffic occupancy, and M5 retail demand, show that ModeRisker can reduce rare-mode omission while also clarifying its limits, including unimodal forecasts, generator miscalibration, and rare events that are absent from the predictive support.
Critical strengths
The strongest aspect of the paper is the clean formulation of rare-mode omission as a selection objective problem rather than a sampling problem. Mode-level CVaR provides an interpretable bridge between average coverage and worst-mode protection, while the supergradient-linearized greedy method retains roughly the computational structure of conventional greedy selection. The controlled synthetic experiments make the failure mode particularly visible: average-greedy delivery can collapse even while the rare mode remains present in the sampled pool.
A second strength is the unusually careful treatment of scope and failure cases. The paper explicitly shows that ModeRisker cannot repair a miscalibrated generator or recover an event absent from the generator's support, and that in unimodal or high-mass-regime settings the selector should effectively revert to average-greedy behavior. This makes the real-data evaluation more convincing than a study reporting only positive cases and provides a practically useful taxonomy of when scenario selection can and cannot help.
Critical weaknesses
The first concern is that ModeRisker's practical success depends strongly on mode inference, while the deployment-time mode-discovery procedure is relatively heuristic. The paper itself shows that overclustering can divert the CVaR budget toward artificial fragments and substantially reduce rare-mode delivery, and the sensitivity bound becomes less favorable as decreases. Although the proposed outlier-rescue procedure performs well in the reported experiments, the resulting system still depends on choices such as the number of main clusters, minimum rescue-cluster size, residual threshold, and modality-screen threshold. This limits how generally the mode-level risk guarantee transfers from oracle modes to deployment.
The second concern is that the real-data evidence for the main positive use case is statistically narrower than the synthetic evidence. The electricity results rely on only a small number of genuinely deep holiday events; although the authors replicate across clients, years, and generator families, clients on a shared event date are not independent realizations of the rare-event mechanism. A hierarchical analysis that treats event date as an independent level, or substantially more rare-event dates/domains, would increase confidence that the reported advantage is not specific to a few calendar shocks.
Questions for authors
Could the authors clarify the theoretical status of the supergradient-linearized greedy algorithm? The paper convincingly establishes why the naïve CVaR greedy rule can stall, and empirical swap-search results suggest that ModeRisker is close to the optimum in the tested settings. Is there any approximation guarantee, monotonic improvement result, or identifiable class of mode geometries for which the proposed linearized greedy method can be shown to remain close to the global CVaR optimum? If not, I suggest making the empirical nature of the “near-optimal” claim especially explicit.
Could the authors also comment on uncertainty at the rare-event/date level in the electricity experiments? Since many client series share the same holiday event, client-level confidence intervals may overstate the effective number of independent rare-event replications. Would a hierarchical bootstrap over dates and clients, or an analysis aggregating each rare date into a single independent unit across the two years, preserve the observed advantage?