acceptodds
Under review as a conference paper at ICLR 2027

ModeRisker: Mode-Level Risk Minimization for Scenario Delivery in Probabilistic Forecasting

Abstract

Consumers of probabilistic forecasts receive representative scenarios rather than the full predictive distribution, and these scenarios are almost always chosen to optimize average coverage. We show that average-optimal selection undergoes a sharp phase transition: once a mode's probability falls below a threshold set by , the mode is omitted entirely, even though the sample pool still contains it — and rare futures are the reason scenario sets exist. We therefore formalize scenario delivery as mode-level risk minimization: over per-mode coverage losses. This objective is not submodular, and the natural greedy provably stalls; linearizing at the CVaR supergradient restores near-optimal selection at the same cost. We evaluate with Realized Tail Coverage (), the CVaR over test instances of the best-of- distance, under a protocol that stratifies by multimodality and matches to the mode mass, on bifurcation processes and on three real domains with three generator families. Average-greedy delivery collapses from to as drops from to while pool containment stays at ; ModeRisker tracks the containment bound down to and improves stratified by at no average cost. The bottleneck is mode discovery, not selection: cluster-quality metrics mislead, and an outlier-rescue layer restores near-oracle delivery. Real data delimits the scope: selection pays only where a rare mode is in the pool and the generator is calibrated, and a modality screen keeps the machinery inert elsewhere, which makes the layer safe to deploy by default. Code is available at https://anonymous.4open.science/r/moderisker-CD82.

Primary area:
learning on time series and dynamical systems
Keywords:
probabilistic forecasting, scenario selection, scenario reduction, conditional value-at-risk, rare modes, multimodal predictive distributions, submodular optimization, tail risk evaluation, time series

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_E5fHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-hoc scenario selection from probabilistic forecasts, where a large pool of possible futures must be compressed into only K representative scenarios. The authors identify a limitation of conventional average-optimal selection, which may omit low-probability but important modes even when they are present in the generated pool. To address this, they propose ModeRisker, which optimizes a mode-level Conditional Value-at-Risk (CVaR) objective over coverage losses and uses a supergradient-linearized greedy procedure to select scenarios based on CVaR-weighted marginal gains, together with an outlier-rescue mode-inference mechanism and a modality screen for determining when risk-aware selection is appropriate. Experiments on controlled synthetic data and three real-world domains: electricity demand, traffic occupancy, and M5 retail demand, show that ModeRisker can reduce rare-mode omission while also clarifying its limits, including unimodal forecasts, generator miscalibration, and rare events that are absent from the predictive support.

Critical strengths

The strongest aspect of the paper is the clean formulation of rare-mode omission as a selection objective problem rather than a sampling problem. Mode-level CVaR provides an interpretable bridge between average coverage and worst-mode protection, while the supergradient-linearized greedy method retains roughly the computational structure of conventional greedy selection. The controlled synthetic experiments make the failure mode particularly visible: average-greedy delivery can collapse even while the rare mode remains present in the sampled pool.

A second strength is the unusually careful treatment of scope and failure cases. The paper explicitly shows that ModeRisker cannot repair a miscalibrated generator or recover an event absent from the generator's support, and that in unimodal or high-mass-regime settings the selector should effectively revert to average-greedy behavior. This makes the real-data evaluation more convincing than a study reporting only positive cases and provides a practically useful taxonomy of when scenario selection can and cannot help.

Critical weaknesses

The first concern is that ModeRisker's practical success depends strongly on mode inference, while the deployment-time mode-discovery procedure is relatively heuristic. The paper itself shows that overclustering can divert the CVaR budget toward artificial fragments and substantially reduce rare-mode delivery, and the sensitivity bound becomes less favorable as β\beta decreases. Although the proposed outlier-rescue procedure performs well in the reported experiments, the resulting system still depends on choices such as the number of main clusters, minimum rescue-cluster size, residual threshold, and modality-screen threshold. This limits how generally the mode-level risk guarantee transfers from oracle modes to deployment.

The second concern is that the real-data evidence for the main positive use case is statistically narrower than the synthetic evidence. The electricity results rely on only a small number of genuinely deep holiday events; although the authors replicate across clients, years, and generator families, clients on a shared event date are not independent realizations of the rare-event mechanism. A hierarchical analysis that treats event date as an independent level, or substantially more rare-event dates/domains, would increase confidence that the reported advantage is not specific to a few calendar shocks.

Questions for authors

Could the authors clarify the theoretical status of the supergradient-linearized greedy algorithm? The paper convincingly establishes why the naïve CVaR greedy rule can stall, and empirical swap-search results suggest that ModeRisker is close to the optimum in the tested settings. Is there any approximation guarantee, monotonic improvement result, or identifiable class of mode geometries for which the proposed linearized greedy method can be shown to remain close to the global CVaR optimum? If not, I suggest making the empirical nature of the “near-optimal” claim especially explicit.

Could the authors also comment on uncertainty at the rare-event/date level in the electricity experiments? Since many client series share the same holiday event, client-level confidence intervals may overstate the effective number of independent rare-event replications. Would a hierarchical bootstrap over dates and clients, or an analysis aggregating each rare date into a single independent unit across the two years, preserve the observed advantage?

Reviewer_oYupOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ModeRisker, a method for selecting representative scenarios from a pool of predictive samples. It uses a linearized greedy algorithm to minimize CVaR over per-mode coverage losses. It also introduces an outlier-rescue inference layer for mode discovery. Experiments on synthetic and real data evaluate rare-mode delivery and Realized Tail Coverage (RTC).

Critical strengths

Introducing CVaR into scenario selection is a useful idea that can improve the representation of rare scenarios.

Critical weaknesses

However, the overall method is relatively simple. Its practical performance also depends substantially on whether the modes and their probabilities are correctly identified.

Questions for authors
  1. Proposition 2 states that the linearized rule assigns qa⋆,qb⋆>0q_a^\star,q_b^\star>0. However, the sorted filling rule in Proposition 1 appears to assign positive weight to only one of the tied worst modes, rather than both. Is the statement in Proposition 2 not fully rigorous as written?
  2. Since ModeRisker aims to minimize mode-level CVaR, Section 4 should report the achieved values of this objective alongside RTC and delivery rates. RTC aggregates instance-level distances, whereas the proposed objective aggregates per-mode mean losses; the two are not equivalent.
  3. Could the authors include a baseline that targets RTC more directly by minimizing the empirical CVaR of instance-level distances, without grouping samples into modes, using a comparable greedy or linearized greedy algorithm? Would ModeRisker still outperform this baseline on RTC? Such a comparison would help distinguish the benefit of mode-level aggregation from that of replacing an average-loss objective with a tail-risk objective.
Reviewer_K9vSOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how to select K representative scenarios from a probabilistic forecaster’s sample pool. It argues that the standard average-optimal selection rule abruptly drops a rare mode once its probability falls below a threshold set by K. Therefore, the authors propose minimizing CVaR over per-mode coverage losses.

Critical strengths

S1. The paper makes an interesting connection between risk-averse optimization and probabilistic time-series forecasting. By introducing mode-level CVaR optimization for scenario selection, the authors address an important limitation of conventional average-based approaches, which may overlook rare but potentially consequential future outcomes. This provides a compelling motivation for the proposed framework.

Critical weaknesses

W1. The paper is very hard to follow. It is very condensed and gives the reader little background before the technical content. Section 3.1 defines the setting in three sentences, and Section 3.2 then moves straight to the dual feasible set, the supergradient, and waterfilling. It never states the CVaR definition or its dual representation, and it never explains why the supergradient is the right quantity to linearize. Several central quantities are defined only in passing or not at all. The paper also has no conceptual illustration. It is a disservice to any reader.

W2. The paper’s premise is that consumers of probabilistic forecasts receive K scenarios rather than the full predictive distribution, and that these scenarios are “almost always” chosen by average-optimal selection. This premise is not supported by evidence. The cited works concern scenario generation for stochastic programs, and the trajectory-prediction citations reflect an evaluation convention (best-of-K).

Many operational settings instead request full predictive distributions or quantile sets. Examples include epidemic forecasting hubs, which collect forecasts as sets of quantiles scored with the weighted interval score [Ref1]; energy forecasting competitions [Ref2]; and large-scale retail forecasting [Ref3].

Therefore, the authors should narrow the claimed setting to the applications where K-scenario delivery actually happens, and cite evidence that average-optimal selection is the prevailing practice there.

W3. While the synthetic experiments demonstrate improvements in rare-mode coverage, the real-world results are considerably more mixed. Improvements in electricity forecasting are concentrated in particular rare-event conditions and depend on generator calibration, while the traffic and retail experiments show limited or no benefits. This raises questions about how frequently this method will provide meaningful advantages in practical forecasting pipelines.

References

[Ref1] JBracher, J., Ray, E.L., Gneiting, T. and Reich, N.G., 2021. Evaluating epidemic forecasts in an interval format. PLoS computational biology, 17(2), p.e1008618.

[Ref2] Hong, T., Pinson, P., Fan, S., Zareipour, H., Troccoli, A. and Hyndman, R.J., 2016. Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond. International Journal of forecasting, 32(3), pp.896-913.

[Ref3] Makridakis, S., Spiliotis, E., Assimakopoulos, V., Chen, Z., Gaba, A., Tsetlin, I. and Winkler, R.L., 2022. The M5 uncertainty competition: Results, findings and conclusions. International Journal of Forecasting, 38(4), pp.1365-1385.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.