- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper proposes Feasible Pathwise Gradient (FeasPG), an optimization method for multistage stochastic programs (MSPs) that learns policies satisfying constraints at every stage while accounting for how earlier decisions change the feasible sets of later decisions. It also develops a finite-sample certificate for bounding the selected policy’s optimality gap. The paper addresses two major questions, how can one differatinate through projected decisions when constraints themselves depend on a state altered by earlier decisions and after training how can we quantify with statistical confidence hoe far the learned feasible policy is from the optimal one.
Critical strengths
- The most important finding is the gradient error which deserves attention
- The feasibility presevtation without huge training is good.
Critical weaknesses
-
The practical novelty relative to existing differentiable optimization methods needs to be sharper The paper combines differentiable projection, pathwise gradients through sequential dynamics, proposal-distance regularization, and information-relaxation certification. Each component has relevant prior work, including OptNet, differentiable optimization layers, PROF, and information-relaxation methods. The authors discuss related work and a concurrent paper on differentiating through projections for inventory policies. However, the manuscript should make the precise boundary of novelty unmistakable.
-
How frequently do the experiments encounter nondifferentiable projection points, and how sensitive are the empirical results to these events? Can the authors validate the practical relevance of the assumptions beyond gradient checks on selected examples?
-
What are the computational costs of constructing the inner lower bounds and calibration bands as the horizon, decision dimension, and history dimension increase? How often does the method return an uninformative certificate?
-
How robust are the learned policies to errors in the fitted exogenous data model? Can the authors quantify the distribution shift between generated and historical trajectories and explain why the method loses its advantage in this case?
Questions for authors
- Is the main contribution the moving-recourse sensitivity formula, its theoretical analysis, the certificate, or their combination? The paper needs to demonstrate clearly what could not already be achieved by applying existing differentiable optimization and stochastic-programming tools together.
Also check weaknesses
5 comments
Response to Reviewer nW5N (2/2): Certificate Cost and Informativeness, Robustness
Cost of the certificate. We tested scalability using the multi-item inventory model in Appendix E, with shared storage and serially dependent demand. Each inner problem is one LP per evaluation path. It took 2.5 milliseconds at the base size of 12 periods and four items, and 28.5 milliseconds at 96 periods. Measured solve times grew approximately linearly across the horizon and item-count sweeps.
Calibration took less than a second. A full certificate over 4,096 evaluation paths took a median of 15 seconds on one core, and at most 171 seconds in the 32-item case. On the power-flow benchmark, each per-path QP took 0.14 core-seconds. Once the policy and calibration are fixed, each evaluation path can be processed independently, so the per-path optimization problems can be solved in parallel.
How often is the certificate uninformative? Judging this requires a reference value for , so we computed one from the true model, which the certificate does not use.
In the reservoir experiment, the model is small enough that bounds on can be computed exactly over all paths. They place the actual optimality gap between 3.3% and 4.4% of policy cost. The certificate reported 5.4% on average, a close assessment obtained without or the true probabilities.
In the larger inventory tests, we bounded from below by information relaxation with the exact conditional means, and from above by a base-stock policy computed from the true model. We observed no coverage failures in any of the 17 configurations, each tested over 100 repetitions. Under serially dependent demand, however, the certificate was conservative. At the base size (12 periods, four items, one history feature, calibration paths), the true gap was at most 1.3% of policy cost, while was 78%. The information-relaxation part was accurate: the uncorrected gap averaged 3.83, against 3.75 with the exact conditional means. Nearly all of was the finite-sample correction for estimating conditional demand distributions from data.
Relative to policy cost, this correction did not depend on problem size: with calibration paths, stayed between 0.76 and 0.81 as the horizon varied from 6 to 96 and the number of items from one to 32. It decreased with more data: increasing the sample to paths reduced from 0.78 to 0.36. The number of history features that the correction conditions on mattered most. With one feature, paths did not bring below ; with two and three features, approximately and paths were needed. These rates agree with Corollary C.1. Our calibration does not assume a parametric model for the uncertainty, and its rate attains the exponent of a minimax lower bound for uniformly valid one-sided corrections over Hölder classes (Theorem C.2), so corrections of this kind cannot remove this dependence without further assumptions. Additional knowledge of the stochastic process may yield tighter certificates. For example, a linear autoregressive model with a specified noise family could support model-specific confidence bounds instead of nonparametric estimation. We leave such certificates to future work.
Robustness to errors in the uncertainty model. We trained under a perturbed inflow level, noise, autocorrelation, or load, then tested on the nominal model. Costs rose by 1.5–29% on ONS and 0.3–3.8% on CAMELS-BR. FeasPG still beat Seasonal in every tested case and seed, retaining 88–111% of its original cost advantage. Its costs were more sensitive to the perturbations than Seasonal’s, which uses little information from the uncertainty model.
Why the advantage was lost on the ONS history. Realized load during 2022–2025 was 56% higher than the seasonal average assumed in training. For both FeasPG and Seasonal, this historical record was costlier than all 4,096 model-generated test paths.
To investigate this mismatch, we refitted the uncertainty model on pre-2022 data with a load trend. After retraining, FeasPG beat Seasonal by approximately 5%, with an improvement in all five training runs. The historical cost also fell near the median of the revised model’s generated-path costs. Updating only the load level was not enough: FeasPG remained 1.2% worse than Seasonal. This follow-up experiment points to the demand trend as an important cause of the original performance loss. On CAMELS-BR, the DeepSets policy beat Seasonal in all five historical windows.
We thank the reviewer for the questions and hope these results clarify the contributions and address the practical concerns.
I thank the authors for the detailed response. However, there are some additional doubts.
i) The additional kink diagnostics are encouraging. However, how are nondifferentiable projection points identified, and how is their proximity to active-set transitions measured?
One major point is the presentation and portrayal of the contributions which is very hard to follow. Despite reading through the manuscript it is hard to find the contributions clearly as given in the rebuttal. I suggest the authors to keep this comparison table which can summarize the differences clearly. Further, in my opinion this paper will benefit from a revisit in terms of portrayal of contrbutions and writing.
Thus, I maintain my current score.
Response to Reviewer nW5N (1/2): Contribution, Nondifferentiable Points, and Gradient Checks
Thank you for the questions. We first state precisely what is new, then address the practical questions.
Main contribution and novelty boundary.
Our main contribution is the analysis of the gradient and of the penalty, and the certificate, not the gradient formula alone. Away from kinks, a differentiable QP layer such as OptNet inside a differentiable simulator computes the same gradient as FeasPG. Existing tools do not establish when this gradient is unbiased despite the projection's kinks, how much bias arises when the dependence of later constraints on earlier decisions is ignored, or what a proposal-distance penalty does over multiple stages. The certificate cannot be assembled from existing tools: information relaxation evaluates its penalty with conditional expectations from a known model, and when these must be estimated from a fixed set of sample paths, the bound is no longer guaranteed without our correction.
The table below summarizes what is new and what we inherit from prior work.
How often do nondifferentiable points occur? Across 150 selected checkpoints and all test paths, our diagnostics detected kinks in at most 0.0002% of stage projections. Under a 10−5 parameter perturbation, at most one in approximately 1.2 million stage projections changed its binding constraints.
Flat gradients are a separate issue. On the new inventory benchmark, we initialized the policy to propose orders exceeding warehouse capacity. Projection cuts these orders back to the limit, so a small reduction in the proposed order leaves the actual order unchanged. Without the proposal-distance penalty, training stayed in this flat region in every tested seed. With the penalty, training left the region and operating cost fell by 4.4–4.7%.
How sensitive are the results to the derivative convention? We retrained the policies with three alternatives: the derivative from the other side of a kink, zero within 10−9 of a kink, and smoothing, which computes gradients at small random perturbations of the parameters (Appendix B.10). We used five seeds for each treatment and selected checkpoints by validation cost.
On the same 4,096 test paths, costs changed by at most 0.34% on the synthetic system and 0.88% on ONS, less than the seed-to-seed standard deviation in each case. On ONS, the first two alternatives gave identical policies. Even widening the zero-derivative region enough to change gradients in 9–15% of updates changed test cost by at most 0.9%.
Do the checks extend beyond selected examples? Yes. We checked gradients every 25 updates throughout training, in all five seeds, using fresh paths. Autodiff and finite differences agreed closely, with median relative errors between 6×10−10 and 10−6. Every large discrepancy occurred when the finite-difference step crossed a kink.
The initialization needed separate attention. The synthetic MLP’s zero-initialized output layer placed 8.3% of projections at detected kinks, which the almost-everywhere result does not automatically cover. After one update, the largest relative error was 7×10−5. Across horizons from 12 to 96, measured gradient norms grew approximately as T1.2, much more slowly than the general exponential worst-case bound.