- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors consider optimization problems where a decision must be made, a random variable is realized (drawn from a distribution in an ambiguity set), and then an action can be taken in response to . The authors consider the ambiguity set to be those that are close in KL divergence to a nominal distribution . Connections/interpretations are drawn to when the is learned from a generative model, such as a diffusion model or normalizing flow. The authors motivate the use of the KL divergence by viewing it as requiring absolute continuity of relative to (as 's support must be a subset of 's), but allowing reweighting of elements in the support. They provide a sample, tilt, resample algorithm to deploy this idea, and demonstrate its effectiveness in a multi-item newsvendor problem and a power capacity expansion problem.
Critical strengths
- The paper considers a very interesting problem where a 2-stage stochastic optimization takes place: one decision occurs before the realization of noise and the other decision takes place after. It is cool to consider how these ideas can connect with generative models providing the nominal distribution or center of ambiguity sets.
Critical weaknesses
- The paper lacks a precise technical formulation of many of its claims. For example, a 'record', which intuitively is a subset of a larger dataset currently used for some type of learning, is never formally defined. Additionally, how the generative model is mathematically represented in the problem is never presented, and, as such, much of the claims of the paper are more 'intuitive' rather than formal.
- Building on this, the paper has no theoretical contribution; the numerical results are of a caliber which would be good for validating a theoretical contribution, but not strong enough if the entire contribution of the paper is empirical, in my opinion.
- The paper itself is a bit difficult to read in its organization. The discussion of generative models and scenarios feels very disconnected from Equation (1), and even the numerical experiments are primarily informal descriptions of the problem, rather than very explicit statements of the form "in this problem, is this function". One unified problem formulation would be nice, as right now the notion of a KL divergence ambiguity set, the sequential aspect of the two-stage problem, the relationship to normalizing flows and generative models, the notion of scenario, are very disjoint and not connected, making the overall focus of this paper very unclear.
Questions for authors
- Is there any theoretical insight to be gained from using the KL divergence for ambiguity sets? How does this part of the formulation differ from (Hu and Hong 2013)?
- What mathematically changes about the problem when is generated from a diffusion process as opposed to the empirical distribution, or any other learning method?
- Is there a way to formalize the notion of record and scenario more explicitly? How does the latent layer actually affect the optimization problem?
- How does the sequential nature of the optimization problem (before and after the realization of uncertainty) relate to the discussion in the later parts of the paper? I had trouble seeing the connection.
- Can you share more details regarding the computation time for the numerical results? Can this handle 'real world' scale problems?
4 comments
Q4-Q5
Q4: How does the sequential nature of the optimization problem (before and after the realization of uncertainty) relate to the discussion in the later parts of the paper? I had trouble seeing the connection.
The two-stage structure is fundamentally what makes the number of scenarios the computational bottleneck. Because the first-stage decision is fixed before is realized, each scenario enters the master as its own copy of the second-stage problem, so the cost of solving the master grows quickly with the number of scenarios and only a few can be held in practice. Which scenarios are included largely determines , which is why we generate them adaptively at the current decision through the tilt. The same structure defines the evaluation under distribution shifts, where the plan is fixed from the record and the second-stage problem is re-solved on each day of the shifted distribution, so robustness is needed for the first-stage decision which cannot adapt to the realized .
Q5: Can you share more details regarding the computation time for the numerical results? Can this handle 'real world' scale problems?
Regarding computation time, a full run on the paper's capacity expansion model (20 zones, 1,440-dimensional daily scenarios) takes about 10–13 minutes on 8 CPU threads, of which 75–90% is spent re-solving the master in each of the 12 rounds as its scenario set grows to 48, while generating scenarios adds negligible time (KL-Gen takes 9.7–10.9 minutes against 10.4–10.7 for KL-Rec). Solving a single master on every record day instead (i.e., SAA over the full record) takes 15.5 minutes with half a year of record, 51 minutes with one year, and 6.1 hours with five years. KL-Gen's plans cost less than this SAA on 2020 at every record length, by 2.7–17.8 bn$/yr, although the SAA is cheaper in distribution with one or five years of record.
Regarding scale, the newsvendor study is a deliberately simple synthetic problem, which lets us know the true distributions and run each configuration on 300 records. The capacity expansion model, however, uses hourly demand and wind and solar capacity factors simulated from historical weather at a zonal resolution comparable to planning studies, and as such we consider this problem to be close to "real world" scale. To assess whether the approach extends to an even larger problem, we also considered a 214-bus version of the same system with 72-hour scenarios (46,224 dimensions), whose results we did not include because the full study was not complete in time for the manuscript. On this model, a run takes 3.2–3.6 hours, of which 91–92% is again the master, so the method scales as the scenario-based baselines do.
I appreciate your reply. Thank you. My review remains unchanged: in the absence of a strong theoretical contribution, the empirical portion of the results should be very comprehensive.
Thank you for the reply. We agree that a paper without a strong theoretical contribution needs a comprehensive empirical evaluation, and we acknowledge that our paper belongs in that category. With that in mind, we would like to explain why we believe the empirical part meets the bar you describe, particularly after the additional work completed during the rebuttal phase.
The original manuscript included two studies.
- A newsvendor study with a known distribution, four controlled shifts, 300 records at each record size and five learned nominals, compared against SAA, KL-Rec, W-DRO, KL-OT and CSSC.
- A capacity expansion study of the Texas system built on twenty years of hourly demand, wind, and solar data, considering a range of record lengths, compared against the same baselines.
During the rebuttal phase, we added the following analyses.
- All 6,900 capacity expansion planning solutions were fixed and re-evaluated on five held-out years and pre-specified shifts.
- The whole newsvendor study was re-run with bounded demand (for well-posedness), which did not change any verdict.
- We ran ablations on the number of rounds, the initialization, and the risk measure.
- We ran two convergence studies, which are reported in our reply to Reviewer KDsN.
Regarding your question about real-world scale, we also ran the method on a 214-county model with 72-hour scenarios and unit commitment, where a run takes 3.2 to 3.6 hours and the main comparison continues to hold, with KL-Gen attaining a lower out-of-distribution cost than KL-Rec in all 10 records (one-sided ).
We also view the capacity expansion study as a contribution in its own right, since methods of this kind are often evaluated only on small synthetic problems, and we believe that a test on a planning problem of realistic size with weather-driven data and a held-out evaluation provides useful evidence for readers who would consider using such a method. In the same spirit, we have tried to honestly report negative results, such as the narrower advantage over KL-OT in the held-out evaluation.
We understand that you may weigh this evidence differently, but if it changes your view of the empirical part, we would be grateful if you would consider reflecting that in your evaluation.
Q1-Q3
We thank the reviewer for their insights. The three identified weaknesses are closely related, and we plan to address them in the revision. We will prioritize more clearly defining the record, the learned nominal Qθ, the KL ball and its dual, the finite master problem, and the latent problem, and we will state the decision x, the uncertainty ξ, and the recourse cost Q explicitly at the start of each experiment. We agree that our contribution is primarily methodological and empirical, and we do not claim new theory for the KL-DRO problem itself, but the revision will state as formal results the equivalence of the latent and original problems and the condition, adopted from (Hu and Hong 2013), under which the problem centered at a learned nominal is well posed (https://openreview.net/forum?id=nNV05EIQf1¬eId=QFwiWI7cqX). We have also strengthened the empirical evidence with a held-out evaluation under pre-specified shifts (https://openreview.net/forum?id=nNV05EIQf1¬eId=eVFUivqQ8s) and a bounded-demand version of the newsvendor study (https://openreview.net/forum?id=nNV05EIQf1¬eId=mNHJQe1DTL). We address the reviewer's questions below.
We do not claim new theoretical insight from the KL divergence itself, but we do rely on two of its standard properties. Every member of the KL ambiguity set assigns zero probability wherever its nominal does, so the nominal determines which realizations the adversary can reach, and the dual of (Hu and Hong 2013) turns the worst case over the ball into a single convex program on any finite scenario set (5). Our formulation otherwise follows (Hu and Hong 2013), except that we use a mean–CVaR objective instead of the expectation (which has many precedents and is not a contribution of ours) and center the ball at a learned nominal Qθ rather than at the empirical distribution. This definition of the nominal distribution/center of the ambiguity set is what lets the adversary reach realizations beyond the record, and it is also what requires the finite scenario approximation that our sampler builds.
Centered at the empirical distribution Q^N, the KL ball only reweights the N recorded realizations, and the dual (4) of (Hu and Hong 2013) yields a finite convex program (5). Centered at a continuous Qθ, the ball also contains distributions that place mass on realizations absent from the record, in proportion to the probability Qθ assigns them, so the learned model determines what the adversary can reach. Since the expectation in (4) is then no longer a finite sum, problem (1) must in general be approximated with a finite scenario set, which is what our framework builds. The learning method matters only through Qθ, so diffusion models with deterministic samplers, normalizing flows, and copulas are all compatible with the proposed framework.
What we call the "record" is the finite sample of past realizations available to the decision-maker, ξ1,…,ξN, with empirical distribution Q^N. The record is the only data used to fit Qθ, and it is the nominal of the record-based baselines. Its realizations are not necessarily independent or identically distributed (they may exhibit seasonality and drift over time, as in the capacity expansion dataset), and none of our methods or statements rely on independence. A "scenario" is an element of the finite set on which the master (5) is solved, a realization ξi with weight wi, drawn from the record or from Qθ depending on the method. We will make both definitions explicit in the revised manuscript.
The latent layer does not change the optimization problem we solve. For any decoder, the latent problem (7) and the original problem (1) have the same value and minimizers, by the argument in Sec. 4.2, which we will state as a formal theoretical result in the revision. The latent layer only changes how scenarios are drawn and weighted, since importance weights computed in the latent space are exact and in closed form.