- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents a pretrain-then-calibrate methodology to address contextual optimization problems under chance constraint satisfaction. The authors propose an interpolation method where the new predictor is formed as a convex combination of a pretrained decision rule and a fixed feasible decision rule. To determine the appropriate convex combination parameter, they introduce four calibration methods that select the parameter over a validation set. The paper provides calibration guarantees for these methods and presents experimental results demonstrating the effectiveness of their approach in mitigating the feasibility-optimality tradeoff.
Critical strengths
The paper addresses an important challenge currently faced by the contextual optimization community: designing contextual optimization methods with strong feasibility guarantees. The paper proposes a clear and intuitive approach that achieves a favorable trade-off between feasibility and optimality. To calibrate the decision rule, the paper introduces four distinct calibration methods and provides theoretical feasibility guarantees for each resulting rule. The experimental results were run on both synthetic and real data, and indicate that performance depends on the specific calibration rule selected which requires empirical testing to choose the best rule for a given context; the method overall offers a superior compromise on the feasibility-optimality trade-off. Furthermore, the approach is agnostic to the choice of the pretrained decision rule, requiring only the calibration step in the validation phase making the method compatible with most of the existing DFL methods proposed in the literature.
Critical weaknesses
While the paper's strength is that the proposed framework is agnostic, the paper criticizes existing methodologies that deal with contextual optimization with chance-constraints and views them as overly-conservative methods. However, the paper doesn’t offer a substitute approach, it rather proposed method that relies on pre-trained decision rules existing in the literature to build an interpolation rule to mitigate the feasibility-optimality tradeoff. Therefore, the proposed method is a calibration layer rather than a totally new approach to deal with the underlying problem. This should be clarified in the positioning of the paper.
The decision policy introduced by the paper relies heavily on the pretrained decision rule and the fixed feasible rule. However, the paper does not detail these 2 policies. From the pretrained decision side, although agnostic, the paper would greatly benefit from theoretical results that benchmarks the optimality and feasibility guarantees of existing methods compared to the interpolated policy to show the improvement that this method provided. From the fixed feasibility rule w0, the choice of such a rule and how to obtain such a rule for our application has not been discussed by the paper. Additionally, the quality of such a rule with respect to both metrics and how the choice of such policy affects the method are not discussed theoretically.
In the experimental section, only a single decision rule is used (although the model varies from linear to NN), which limits the empirical demonstrating that the method is model-agnostic. Although additional methods are introduced in the appendix, the main text would benefit significantly from an ablation study across diverse DFL approaches to highlight this advantage.
Minor comments :
The main idea in the paper claims to offer a rule with rigorous feasibility guarantees (page 2 line 90), yet the text also suggests that strict feasibility is not explicitly sought (page 5 line 242). The framing and terminology should be revised for consistency and precision.
Lemma 1 does not appear to be utilized or referenced anywhere in the main analysis; its necessity should be clarified .
In most applications, being able to control the tradeoff between feasibility and optimality might be preferred.Beyond \alpha, can this method be adapted so that the decision maker has more control over this tradeoff and can set a threshold for either optimality or feasibility to fit their particular applications?
The title safe contextual optimization might be a bit misleading
Questions for authors
1- Could the authors elaborate on how practitioners should construct the fixed feasible rule w₀ in practice? How sensitive is the empirical optimality to the choice of w₀, for instance when comparing an overly conservative rule against a nearly optimal choice?
2- How does the quality of w₀ across feasibility and optimality metrics impact the performance of the overall interpolated policy, and can this relationship be formally characterized theoretically?
3- To strengthen the model-agnostic claims, could the main text include an ablation study benchmarking diverse DFL frameworks as the base pretrained rule, rather than evaluating a single decision rule across different network architectures?
4- The paper characterizes existing chance-constrained contextual optimization techniques as overly conservative. Could you clarify this point further? Also could you discuss whether those baseline methods can be integrated into your framework, and address if applying this calibration layer helps alleviate their inherent conservatism?
5- Could the authors reconcile the claim of rigorous feasibility guarantees (page 2, line 90) with the subsequent remark indicating that strict feasibility is not explicitly sought (page 5, line 242)?
6- What specific role does Lemma 1 play within the theoretical framework?
7- Beyond adjusting \alpha, can this calibration framework be extended to grant decision-makers explicit control by enforcing targeted thresholds on either feasibility or optimality?
8- Given that the framework relies on existing pretrained decision rules rather than replacing them, would it be more accurate to position this work as a modular calibration layer? Could the framing in the introduction be updated accordingly?
2 comments
Response to Reviewer 8KpK (Part 2)
Regarding the minor comments, we list them as M1-M4 respectively.
Response to M1:
The two statements refer to different stages of our framework: during pretraining, we do not require rigorous feasibility guarantees, allowing us to prioritize objective performance; the subsequent calibration stage then establishes statistical feasibility guarantees. We have revised the wording on page 5 using blue color to clarify this distinction and ensure consistency.
Response to M2:
We thank the reviewer for pointing this out. Lemma 1 establishes the nesting property, which is essential to the conformal calibration guarantee (Theorem 1) and the DKW calibration guarantee (Theorem 2). We have added explicit references to Lemma 1 in the revised manuscript to clarify its role in these results.
Response to M3:
Our framework follows the standard chance-constrained programming formulation, where the risk tolerance serves as a natural and interpretable parameter for controlling the optimality–feasibility tradeoff. Decision makers can specify their desired feasibility level through , and our calibration procedure seeks a low-cost decision rule satisfying the corresponding statistical feasibility requirement. Different prespecified values of allow practitioners to explore different tradeoffs within the same framework.
We also appreciate the suggestion of specifying an optimality threshold instead. This leads to a related satisficing-style formulation, where one seeks to maximize feasibility subject to an acceptable cost level (Chen & Sim, 2009; Long et al., 2022) . While this is a meaningful alternative, it requires a different calibration objective and corresponding statistical analysis. We view it as an interesting direction for future research, while the present work focuses on the standard chance-constrained formulation.
Response to M4:
We thank the reviewer for this suggestion. We have revised the title to “Pretrain-Then-Calibrate: Enhancing the Optimality–Feasibility Tradeoff in Chance-Constrained Contextual Optimization” to better reflect the probabilistic nature of our feasibility guarantees.
Response to Q1:
Yes, we have added related discussion. Please see our general response on discussion, especially on how to choose them using domain knowledge and in a data-driven way, its sensitivity, and its impact on objective performance.
Response to Q2:
The impact of on the objective performance depends on the interpolation path. Theorem 2(ii) provides a formal guarantee relative to the best feasible candidate along this path, while Appendix D.7 empirically examines sensitivity to different reference rules. Please see our general response on for further discussion.
Response to Q3:
Please see our previous response on weakness 3.
Response to Q4:
Existing (distributionally) robust optimization approaches often enforce feasibility through learned uncertainty sets, followed by worst-case optimization. While these approaches provide feasibility protection, they can be overly conservative because protecting against all realizations within an uncertainty set may impose stronger requirements than the original chance constraint. This can lead to unnecessary sacrifices in objective performance, as illustrated by our empirical comparisons in Section 4.3 and Appendix E.
Our framework takes a fundamentally different approach. Instead of starting from an overly conservative decision and attempting to improve its optimality, we start from an objective-driven pretrained decision rule with potentially strong optimality performance but without feasibility guarantees. We calibrate this rule toward a safe reference decision, adjusting it sufficiently to achieve statistical feasibility guarantees while preserving objective performance. Our approach directly calibrates constraint-satisfaction probabilities, without requiring learned uncertainty sets or downstream robust optimization. This is the key mechanism through which we aim to mitigate excessive conservatism.
Regarding integration, our model-agnostic framework can in principle accommodate decision rules produced by existing RO/DRO or contextual optimization methods. However, applying safety-oriented calibration to an already conservative rule does not necessarily improve its objective performance, since our interpolation moves toward greater feasibility instead of reversing existing conservatism. Our primary contribution is therefore an optimization pipeline that avoids introducing unnecessary conservatism during pretraining.
Response to Q5:
We have fixed this issue in our revised paper and in our previous response.
Response to Q6:
Please see our previous response on minor comments.
Response to Q7:
Please see our previous response on minor comments.
Response to Q8:
Please see our previous response on weakness 1.
Response to Reviewer 8KPK (part 1)
We sincerely appreciate your detailed review and thoughtful comments on our paper.
Response to W1 (methodology contribution):
We have clarified our contribution in the revised introduction. We politely mention that our whole pretrain-then-calibrate pipeline is exactly the substitute (and indeed better) approach to deal with this problem. Please read our general response on method contribution on the novelty, strength and practicality of our framework for more details.
Response to W2 (dependence of decision policy on the pretrained decision rule / feasible rule):
We provide a practical guideline for choosing the two rules at the end of Section 3. We recommend selecting wpre for strong downstream objective performance, using either task-only training (with a feasibility heuristic if needed); rigorous feasibility is not required at this stage. In contrast, w0 should provide readily verifiable safety through problem-specific knowledge, without good objective performance. These complementary roles allow calibration to exploit the pretrained rule’s objective performance as well as provide feasibility.
We emphasize that the flexibility in choosing pretrained decision rules is a key advantage of our framework. Our method does not prescribe a particular pretraining algorithm or decision-rule class, allowing practitioners to leverage expressive models such as neural networks, to get an objective-driven decision rule. Importantly, our calibration guarantees are independent of the complexity of the pretrained decision-rule class. This enables us to obtain statistical feasibility guarantees for flexible, objective-driven decision rules without requiring model-specific feasibility analysis, model retraining, or downstream robust optimization. Our experiments with linear, neural, and RF-SAA rules further demonstrate this flexibility.
Regarding theoretical comparisons, our Theorems 1 and 2 already establish statistical feasibility guarantees, with Theorem 2(ii) additionally providing a pathwise optimality guarantee. Rather than claiming universal theoretical dominance over existing optimization approaches, which would require method-specific assumptions, we evaluate their relative performance through extensive empirical comparisons in Section 4.3 and Appendix E. These results demonstrate favorable optimality–feasibility tradeoffs across different applications, highlighting the simplicity, generality, and practical effectiveness of our framework.
Regarding the construction of the safe reference rule w0, and its impact on objective performance, we refer the reviewer to our general response.
Response to W3 (limited use of a single decision rule):
We understand this concern as involving two complementary aspects: (1) whether our framework can accommodate decision rules obtained from different pretraining methodologies, and (2) how our complete pipeline compares with existing contextual and decision-focused optimization approaches.
Regarding the first aspect, our framework can accommodate decision rules from different pretrained methodologies. We have added a new ablation study in Appendix D.5. Specifically, we consider task-only neural training, prediction-first learning (MSE followed by optimization), and decision-focused learning (task-based learning/SPO+). We use the resulting decision rules as pretrained inputs to the same calibration framework, without retraining or modifying the pretrained models. The results demonstrate effective feasibility calibration across these distinct pretraining methodologies, providing direct empirical support for our model-agnostic claim.
Regarding the second aspect, our framework outperforms existing contextual and decision-focused optimization approaches empirically. Section 4.3 compares our complete pipeline against CPO, LRO, CPP, and Trim-DRO, while Appendix E provides additional comparisons with SPO-RC+ and ICRC-based robust optimization. These experiments demonstrate the practical competitiveness of our framework against alternative contextual optimization pipelines.
Finally, we emphasize that our theoretical calibration guarantees are independent of the specific pretraining algorithm and the complexity of its hypothesis class. Together with the new pretraining-pipeline ablation and existing benchmark comparisons, these results support both the generality and practical effectiveness of our approach.