acceptodds
Under review as a conference paper at ICLR 2027

When Does Uncertainty Quantification Matter in Predict-then-Optimize?

Abstract

Predict-then-optimize (PtO) pipelines first predict uncertain problem parameters and then solve an optimization problem using the resulting predictive distribution. A common intuition is that better uncertainty quantification (UQ) should improve downstream decisions, but the extent to which this is true depends on how distributional errors interact with the geometry of the optimization problem. This paper formalizes that dependence and characterizes when predictive uncertainty materially affects end-to-end decision quality. We model a PtO pipeline in which a predictor outputs a predictive law that is consumed directly by a downstream stochastic, risk-averse, chance-constrained, or precision-weighted optimization problem. We derive bounds showing that excess decision risk is controlled by the discrepancy between the predictive and true laws, scaled by the sensitivity of the downstream optimization problem. When the predictive law is restricted to a structured low-dimensional family, this error further decomposes into estimation error and structural bias. Moment-based coupling bounds relate decision degradation to errors in predictive means and covariances, while structured uncertainty models can substantially reduce estimation complexity. For scale-invariant decision rules, only relative uncertainty accuracy matters. In precision-weighted quadratic problems, the resulting efficiency loss admits a fully explicit characterization. Numerical experiments on newsvendor, strongly convex, chance-constrained, and precision-weighted problems confirm the theoretical bounds and sensitivity predictions. Overall, the results show that the importance of UQ is not intrinsic to the prediction problem alone, but is jointly determined by the difficulty of estimating the predictive distribution and the sensitivity of the downstream optimizer to decision-relevant distributional errors.

Primary area:
optimization
Keywords:
Predict-then-Optimize, Uncertainty Quantification, Decision Risk

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_tT2WOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the effect of uncertainty quantification in contextual optimization/predict and optimize setting. It particularly treats the setting where the prediction model relies on a distributional predictor and feeds it to the optimization problem to solve a downstream task. The paper claims that better UQ improves decisions only when a distributional feature is both imperfectly recovered and visible to the optimizer. In this setting, the paper claims that the uncertainty quantification has 2 sources : a statistical error and a decision error. They particularly look at this error by bounding the regret/excess risk of a projected distribution onto a structured family (location–scatter laws) by a finite sample estimation, a population misspecification and a decision sensitivity term. Experiments are run on retail inventory and industry portfolios and electricity procurement.

Critical strengths

The main strength of this paper is that it addresses an important problem in the DFL / contextual optimization community from a distributional point of view with respect to uncertainty quantification. It presents 2 theorems that deal with decision-risk stability and excess risk sensitivity.

Critical weaknesses

One of the major weaknesses is that the paper's results are very limited and can be summarized into a single main theorem that bound the excess risk of the methods. The main theorem presented only treats the projection of a learnt distribution into a structured family and states the results on that particular structure which limit the applicability of the theorem. The estimation term is only instantiated for location–scatter laws, and the misspecification term is never bounded for any family. The proof of the results rely on very classical results requiring the projection and standard W1 bounds for the location-scatter laws.

The results also rely on a triangular inequality which makes the clean separation between the terms loose. The decomposition in equation (10) is an upper bound and nothing proves its tightness. Additionally since the decision task is defined with respect to the learnt distribution, I would expect the two sources of uncertainty to interact which the paper does not explore. The latter also brings me to my next point about the paper not having proper positioning relative to prior work particularly with the predict and optimize (DFL) community. In the DFL community, most of the existing methods tend to rely on the point prediction algorithm and only a small number of methods rely on the distributional approach presented by the paper. Therefore a more systematic approach and a clearer presentation of the framework is needed to make the results more aligned with the DFL/predict and optimize literature.

Another weakness is related to empirical evaluation. As presented, the experiments are confusing and do not test the theory. The experiment claims to address the 2 sources of uncertainty presented in theorem 4.3 however they only present expected effects which are not representative of the presented theory. Most of the work in the experimental section focuses on the distinction between risk-neutral and risk-averse(CVaR) which concerns Proposition 4.4 rather than the main theorem.

Questions for authors

1- Beyond combining a projection step, standard W1 bounds for location–scatter laws, and the triangle inequality, what is technically new in Theorem 4.3? What are the main contributions beyond this theorem?

2- Can the misspecification term be bounded for at least one concrete family? Can the estimation term be extended beyond location–scatter laws?

3- Is the decomposition in Eq. (10) tight? Do the estimation and optimization errors interact, since the decision is made under the learned distribution? Also, the claim that UQ helps only when a feature is imperfectly recovered and visible to the optimizer is a necessity claim. What result supports it, given that Theorem 4.3 is only an upper bound?

4- How does this work relate to DFL and predict-then-optimize methods? Do decision-focused losses control δ_f? When should a distributional predictor outperform a decision-focused point predictor?

5- Which experiment tests which term of Theorem 4.3? Most experiments compare risk-neutral and CVaR objectives, which relates to Proposition 4.4 rather than the main theorem.

6- What is the purpose of Sections 4.3 and 4.4? Why consider moment-based decisions, and why introduce CVaR?

Reviewer_NzUjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how errors in predictive distributions affect downstream optimization decisions. It introduces task-dependent discrepancies to bound decision regret and separates distributional error into estimation error and model misspecification.

Examples and experiments in inventory, portfolio allocation, and electricity procurement illustrate how the effects of distributional errors depend on the downstream objective.

Critical strengths
  • The paper addresses an important question on prediction uncertainty and decision making.
  • It provides a useful discussion of why distributional accuracy does not necessarily translate into better decisions.
Critical weaknesses
  • The main results provide limited new insight into when UQ matters. The key observation is straightforward: the continuity and tail conditions in Assumption 3.1 control the error in expected objectives, and decision regret is then bounded by twice this error. Thus, Theorem 4.2 largely follows a standard objective-perturbation argument, while Theorem 4.3 separates estimation error and misspecification through the triangle inequality. These bounds can be loose for individual problems, and it is unclear what additional insight they provide. Overall, it is very unclear to me what new phenomenon or practical guidance the proposed framework offers beyond existing stability results.

  • The numerical results mainly illustrate familiar observations: dependence can matter for tail-risk objectives, and covariance estimates can affect portfolio decisions. They do not directly examine the estimation and misspecification terms or show that the proposed discrepancies help explain performance or select an uncertainty model. There is also a specific mismatch in the portfolio discussion. The scale-invariance result concerns precision-weighted least squares, whereas the experiment uses mean–variance portfolio optimization.

Questions for authors

What is the main new insight beyond existing stochastic-programming stability results? In particular, what conclusion about the value of UQ follows specifically from the directed discrepancy or the proposed decomposition?

Reviewer_61mzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies stability/control aspects of decision making problems; whose feasibility is designated by a cost function. The main idea is to bound the excess risk caused due to perturbations in distributional assumptions. Effects of introducing specific structure for characterizing the probability spaces (e.g. parametrization) are also discussed. Further, population/sample estimates are also considered.

Critical strengths
  • The paper has a good progression of ideas deriving estimates by adding a little complexity at each step.
  • The theory produces results very akin to bias-variance type decomposition where the excess risk is characterized by two axes; one quantifying where having more data would help (sample complexity) and the other quantifying the effect of model misspecification.
  • The parameter estimation type example is a good demonstration.
  • The paper also demonstrates with experiments how their theory can be used.
Critical weaknesses
  • There are a lot of issues with notation, there is often some new notation or terminology that seems to pop up, it would be great to have all notation at one place (like a notation section).
  • If in the problem setup, a simple measure theoretic idea is provided -- I think that would be of great help to put things into perspective: something like defining the probability space (Θ,F,q)(\Theta, \mathcal{F}, q) and I⊂F\mathcal{I} \subset \mathcal{F} -- from what I understood.
  • It would be great if the presentation could be improved further.
  • A discussion on how the three questions asked (distributional perturbation affecting excess risk, the probability metric used or identified for statistical analysis of decision relevant perturbation and the effect of structural assumptions on excess risk) have been answered must be added -- I am not sure if it is clearly already evident.
Questions for authors
  • I see that the Wasserstein-1 distance falls out naturally into the proofs and therefore a good candidate to work with, is there any other similar distance that can work?
  • Are the definitions in task-induced distribution discrepancies newly being introduced in this paper or are they well studied already?
  • Can you make this sentence "The four quantities retain progressively less task information." in lines 143-145 clear by giving a little more explanation (is theorem 4.2 talking about that)?
Reviewer_qQZPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper discusses when improving the accuracy of a predicted distribution actually leads to better downstream optimization decisions by showing some theoretical decompositions of the optimality gap with uncertainty quantification. The authors also used serveral applications to demonstrate it.

Critical strengths
  1. The paper explicitly shows the decomposition of the decision and estimation accuracy.
  2. The paper also discusses the implications in different applications.
Critical weaknesses

My main concern is the novelty of the first two claimed contributions, namely, "(i) a task-induced hierarchy from directed decision discrepancy to oscillation, integral probability, and locally controlled Wasserstein distances; (ii) an end-to-end theorem combining estimability, structural misspecification, and downstream sensitivity."

Specifically, my understanding is that contribution (i) is mainly reflected in Lemma 4.1 and Theorem 4.2 under Assumption 3.1, while contribution (ii) is mainly reflected in Theorem 4.3. However, Lemma 4.1 and Theorems 4.2 and 4.3 appear to follow from straightforward inequalities under Assumption 3.1. Similar arguments are commonly used as direct proof techniques in the existing literature [2, 3], so I do not see these results as fundamentally new observations.

Given the limited technical novelty of these results, I think the authors need to further justify the significance of the metrics in Equation (3), particularly by demonstrating their use in practice. Without such justification, it is unclear to me whether the proposed hierarchy provides substantial new insights beyond the current literature.

[1] Smart Predict-then-Optimize (SPO) [2] Distribution-Free Model-Agnostic Regression Calibration via Nonparametric Methods [3] Predict-then-Calibrate: A New Perspective of Robust Contextual LP

Questions for authors
  1. Are the authors aware of the existing literature on [1], the Smart Predict-then-Optimize (SPO) framework introduced by Elmachtoub and Grigas? This line of work seems highly relevant to the central question studied in this paper. Could the authors clarify how their contributions differ from that line of work?

I would also be interested in a comparison with [2,3], which also studies how uncertainty quantification affects downstream optimization performance under certain continuity assumptions.

[1] Smart Predict-then-Optimize (SPO) [2] Distribution-Free Model-Agnostic Regression Calibration via Nonparametric Methods [3] Predict-then-Calibrate: A New Perspective of Robust Contextual LP

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.