- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies the parametric Average Percentile Upper Bound (APUB) optimization, which uses the conditional value at risk (CVaR) as the objective function. The paper considers two bootstrap methods, generative parametric bootstrap and weighted likelihood bootstrap, for constructing the objective function. Under correct model specification, the paper establishes the convergence of loss functions in probability, the asymptotic coverage at the ideal APUB optimizer, and a uniform error bound for the loss function conditional on the data. The paper demonstrates the excellent performance of the proposed two bootstrap APUB in comparison with the nonparametric bootstrap APUB, Bayesian DRO and moment DRO methods.
Critical strengths
The paper is generally well written. The theory developed in this paper studies the APUB optimization previously proposed in Liu et al. (2026), for two extended versions of parametric bootstrap. The theory is generally solid and technically substantial, covering the convergence of loss function under parametric bootstrap estimators, the coverage of selected decisions, and uniform error bound, though they depend on strong high-level assumptions. The proofs seem mostly correct. The empirical results demonstrate that the APUB with parametric bootstrap methods perform well and better than the Bayesian DRO under misspecified priors and the moment DRO methods.
Reference
Shixin Liu, Ming Gao, and Jian Hu. (2026). Minimizing upper confidence bounds: A data-driven framework for stochastic programming. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research.
Critical weaknesses
- The current APUB with parametric bootstrap is a natural extension of the nonparametric bootstrap version of APUB recently proposed in Liu et al. (2026). The objective functions using CVaR are the same in the two papers. The main difference is that Liu et al. (2026) uses a nonparametric bootstrap that resamples the original data directly, while the current paper uses a parametric bootstrap that samples from the parametric model and a weighted likelihood bootstrap.
Regarding the theory, I feel that the theory established in the paper is standard and not surprising, as the results on convergence of loss in probability, asymptotic coverage of the ideal optimizer, and uniform error bound are generally expected based on the theoretical properties of bootstrap estimation.
More importantly, some theorems rely on strong high-level assumptions, which probably should be verified using low-level assumptions on the model and the data generation process. Such assumptions include
-
Assumption (A4) in Proposition 1, regarding the uniform convergence of . This generally consists of two convergence results: the uniform convergence of to , and the convergence of the function. Both convergence results should be proved under low-level assumptions.
-
Assumption (P3) in Proposition 2, for a similar reason to Assumption (A4) above.
-
Assumptions (9), (10), and (11) in Theorem 2: The three convergence results should be proved, rather than being assumed. The form of the function is also not given.
In addition, the propositions and theorems rely on some strong and unconventional assumption, such as Assumption (P2) in Proposition 2. I hope the authors can clarify when such assumptions can hold, and it will be best if all the assumptions can be directly verified for the models in Section 5.
- I feel that the paper has not adequately demonstrated the advantage of APUB over the competitors. The paper starts with a discussion on the distributionally robust optimization literature. However, the main text has not investigated the robustness of APUB optimization to model misspecification and/or data contamination. For all the theory and numerical experiments in the paper, the model is always correctly specified, with a true parameter. What if the model specification is incorrect, or the data contains some outliers? I think one potential advantage of using bootstrap methods is that they can possess some robustness in such scenarios, but this robustness issue has been completely neglected in the current paper.
Questions for authors
-
The theory only concerns the convergence of the loss function , but it does not seem to establish the convergence for the estimators of and the APUB optimal solution of . It seems some of these convergence results have been proved en route. Perhaps the authors can state the results more clearly. For example, in Theorem 2, the convergence of to should be a proved result, but it reads like an assumption.
-
The definition of in Theorem 3 needs to be clearer.
-
The numerical experiments do not take into account the computation time. The bootstrap-based APUB is already computationally intensive. I wonder if the nonparametric bootstrap APUB may be computationally slightly more efficient than the parametric version. The authors should report the computation time for all the competing methods under consideration.
-
In practice, how do one choose the value of ? From the numerical results, it seems that the regret does not change much, but only the observed coverage gets better when increases. However, due to the limited sample size, we cannot increase arbitrarily close to 1, as fewer data points in the tail may increase random error. What is a suggested method to choose the best value for in practice?
Authors’ Second-Round Response
We thank the reviewer for the constructive comments. We have submitted the revised manuscript, with major changes marked in blue. The changes addressing your concerns are summarized below.
Response to Weakness 1 and Question 1: Literature and attribution — Section 2, p. 2; Section 3.1, p. 3; Appendix F.1, pp. 38–39
We expanded the review to include 14 complementary studies covering empirical confidence penalties, statistically calibrated DRO, bootstrap optimization, parametric confidence regions, conformal optimization, and sequential UCB methods, including direct UCB minimization.
The revision distinguishes their statistical targets and decision settings. The cited conformal methods calibrate predictive outcome regions, whereas APUB addresses estimation uncertainty in expected cost. We acknowledge that confidence-based optimization, parametric uncertainty protection, and bootstrap fitting predate our work.
Section 3.1 attributes PB and WLB to their sources. Section 3.2 attributes upper-percentile aggregation to nonparametric APUB and its positive-part representation to established CVaR theory. Propositions 1–2 state sufficient conditions for the PB/WLB fitted-cost laws used in our framework.
Response to Weakness 2 and Question 2: Contribution and proof methods — Section 4, pp. 4–6; Appendix F.2 and Table 7, p. 39
We added a proof roadmap and a table distinguishing established techniques from what each result verifies here.
Theorem 1 uses upper-average stability and approximate-minimizer arguments, related to existing nonparametric APUB and Bayesian risk analyses. The construction-specific task is establishing uniform fitted-cost contraction after nonlinear parameter refitting. Proposition 1 handles weighted likelihood fitting; Proposition 2 additionally controls PB’s random fitted generating distribution.
The revised Theorem 2 derives the statistical limits needed for selected-decision coverage under explicit regularity conditions. Its proof combines likelihood expansions, moment control, a uniform delta method, and local equicontinuity. Theorem 3 combines DKW concentration, a finite-net union bound, and Lipschitz continuity.
This clarifies the contribution without claiming that bootstrap fitting, upper-tail aggregation, or the general proof techniques are new.
Response to Weakness 3: Equation (5) — Section 3.2, pp. 3–4
The paragraph following Equation (5) explains its purpose and origin. APUB protects against decisions that appear attractive under one fitted model but become costly under estimation perturbations. The upper tail is over fitted expected costs, not individual realized market losses. Zero-level aggregation gives bootstrap averaging; positive levels emphasize unfavorable fits.
The extension concerns the PB/WLB fitted-cost laws and their statistical justification, rather than a new risk functional.
Response to Weakness 4: Beyond consistency — p. 6; Appendix F.3, pp. 39–40; Appendix G, pp. 41–44
We explain three additional properties: preservation of convexity when fitted costs are convex; an estimation-standard-error adjustment in the regular normal limit; and coverage at the ideal data-selected optimizer under the stated assumptions.
Theorem 3 is a finite-ensemble probability bound, conditional on the observed data, rather than an asymptotic result. It controls bootstrap simulation error, not population coverage.
The new Appendix G compares APUB against two matched normal-approximation penalties. Decision quality is similar, while APUB is approximately three times faster under the reported eight-start penalty protocol and provides numerical global optimality-gap certificates. We do not claim universal regret superiority.
Response to Weakness 5 and Question 3: Practical goals and results — Section 5, pp. 8–9; Appendix G.2–G.3, pp. 41–42
We clarified the practical question: how costly is it to act on an unlucky training sample? The text distinguishes mean regret, variability across training samples, and coverage of the reported cost at the recommended decision.
We explain each comparison: SAA versus FMO assesses family information; BAG versus APUB assesses upper-tail adjustment; NP-APUB assesses aggregation without parametric fitting; and the two Bayesian priors assess prior sensitivity.
At N=100 and level 0.5, mixed-normal mean regrets are 7.14 for WLB-APUB and 7.28 for PB-APUB, versus 7.51 for FMO and 8.81 for SAA. The aligned Bayesian prior achieves 6.30, while the misspecified prior gives 15.50. These results support a useful prior-free alternative, not dominance over reliable prior knowledge.
Appendix G additionally reports average regret among the worst 50% and 20% of training replications. We distinguish these tail metrics from central percentile bands and avoid interpreting improved coverage as proof of improved tail regret.