- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PROSE, a framework for learning policies for multistage stochastic optimization with continuous decisions. At each decision, a NN observes the current state (), known optimization parameters, and future scenarios (). A state-dependent repair maps its prediction onto the feasible set of actions. Training objective minimizes realized trajectory cost by differentiating through the policy, repair, and known state transitions, without using optimal actions as supervision. The paper evaluates PROSE on stochastic energy dispatch and multi-echelon inventory replenishment. Compared to rolling two-stage stochastic optimization, it reports approximately 4.3 % lower dispatch cost, 1.89% higher inventory profit, and median per-decision speedups of roughly 1600-2100x.
Critical strengths
- The combination of trajectory optimization and structured repair is practically motivated. The chain rule accounts for the repair's dependence on the exogenous state. 2, The reported results are encouraging, PROSE reduces total dispatch cost and outperforms the baselines.
Critical weaknesses
- Comparison to learning-based methods. The main baseline comparison is with rolling two-stage stochastic optimization. Differentiable predictive control already optimizes neural policies through closed-loop rollouts without any expert labels.
- The feasibility guarantee seems to be a slight overclaim. App. B describes an iterative thermal dispatch procedure with a finite transfer cap and an independent feasibility check. The repair might stall or reach its cap with nonzero violations. The current feasibility approach used might not support that more general claim made in the text.
Questions for authors
- Can the authors add any learning-based method as a baseline? Just rolling two-stage optimization does not create a sufficient class of baselines. Some examples could be [1], [2]. If the time is insufficient to produce other baseline results, the authors would need to justify why PROSE is better than other learning-based control methods?
- What happens if thermal repair finds no improving transfer or reaches the 512-transfer cap with residual overload? Is successful completion guaranteed under stated conditions, or is there a fallback? It would help in understanding how many times the NN policy predicts an action that does not require repair.
[1] Drgoňa et al. (2022), “Learning Stochastic Parametric Differentiable Predictive Control Policies.” [2] Heess et al. (2015), “Learning Continuous Control Policies by Stochastic Value Gradients.”
3 comments
Thank you for the clarifications and updated results. Can you clarify, for a day t, how do you choose the most similar 10 past days? Do you take 32 days following each of the ten similar past days? Do you hard stop at day t?
Thank you for the follow-up.
Scenario selection in inventory. For a decision on day , a past day is eligible if its demand path, the 32 days through , was fully observed before a cutoff. Each eligible day gets a score that adds two terms:
-
How much demand on day differs from demand on day , the last day observed before the decision. This is the mean squared difference over SKUs, normalized per SKU. The normalization keeps high-volume SKUs from dominating the score.
-
How far apart and fall in the calendar year, normalized and squared. The calendar wraps at year-end, so late December is close to early January.
Lower scores mean more similar days, and we keep the ten lowest. Appendix C.5 gives the exact scoring rule.
The scenario from a selected day is its 32-day demand path, so demand on days through is used as possible demand for days through . Each of the ten scenarios has probability 0.1.
Hard stop. Yes. The data are split chronologically into training, validation, and test, and the cutoff depends on the split:
-
In training, the cutoff is day , so every path ends by day .
-
In validation and test, the cutoff is the first day of the split, so validation scenarios use only demand from before the validation period, and test scenarios only demand from before the test period.
No scenario ever contains demand from day onward, and no validation or test scenario contains demand from its own split. PROSE and the two-stage program use the same scenarios.
Response to Reviewer DoeG
We thank the reviewer for the thoughtful comments and the references to Drgoňa et al. [1] (DPC) and Heess et al. [2] (SVG). W1, W2, ... and Q1, Q2, ... number the Critical Weaknesses and Questions For Authors in order, and G1 to G4 refer to the general response.
[W1, Q1] Missing learning-based baselines such as DPC and SVG. G1 adds a controlled learned baseline, an imitation proxy with PROSE's architecture and repairs, and PROSE beats it in both case studies (Appendix E.2). DPC and SVG are not directly comparable: neither enforces the hard constraints our problems require.
PROSE's constraints are hard, couple the components of each action, and move with the policy's earlier actions, such as an exact power balance within ramp windows set by the previous dispatch. Its closed-form repair enforces them, so every deployed action is certified feasible. DPC penalizes constraint violations and gives only probabilistic guarantees, so its actions can be infeasible (a penalty makes a balance violation small but not zero). DPC has also not been shown to scale to problems of our size: its examples have at most four control inputs and 20-step horizons, against 153 generators and 276 decisions per day in our dispatch case study. SVG has no mechanism for constraint satisfaction and learns a Q-function or dynamics model that PROSE takes as known.
[W2, Q2] Return value if the repair fails. Every implemented action is certified feasible, and the fallback has never been needed (G4). The repair runs at every decision by design, since no raw network output meets an exact power balance. The repair is included in PROSE's reported latency, a median of 17.5 ms per decision against 29.0 s for the two-stage program.