acceptodds
Under review as a conference paper at ICLR 2027

Robust Optimization with Random Empirical Distributions

Abstract

We propose learning objectives that apply a risk functional to the random expected loss produced by a distribution over empirical reweightings. Under an unpenalized risk envelope representation, the objective equals a robust optimization problem over mixtures of the candidate measures. The construction inherits properties of the outer risk functional. A density specified through inverse KL divergence gives a Dirichlet family interpolating between the empirical expectation and the risk of a uniformly sampled observation. At the latter endpoint, the uniform vertex law is maximal in convex order among probability weight laws with the same mean. Gaussian perturbations continue beyond this boundary with weights that may be negative and still sum to one, allowing integrated losses outside the observed range for nonconstant losses. For a fixed outer risk respecting convex order, the induced risks are ordered along the path, with nested ambiguity sets under the envelope representation. For losses linear in the decision, a prescribed weight covariance recovers spherical linear shrinkage in a mean–standard deviation objective when the centered feature matrix has full column rank. For Dirichlet weights and conditional value-at-risk (CVaR), we develop a stochastic subgradient algorithm. On Mammographic Mass, selecting the weight distribution by cross-validation gives slightly higher mean AUC than -SVM at all five tested CVaR levels and lower mean classification error at the three higher levels.

Primary area:
optimization
Keywords:
Robust optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1U3gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a robust optimization framework that applies an outer risk functional to the expected loss under randomly reweighted empirical measures. It establishes an equivalent DRO representation, studies a Dirichlet–Gaussian family, and connects the framework to covariance shrinkage. For CVaR as the outer risk functional, the authors develop a stochastic subgradient algorithm for Dirichlet weights.

Critical strengths

The paper provides a unified framework with several interesting theoretical properties. In particular, the Dirichlet–Gaussian construction and the discussion in Section 3 (including Figure 1) help explain how the choice of weight distribution affects the resulting risk.

Critical weaknesses
  1. The paper is difficult to follow due to dense notation and insufficient explanation of the motivation for introducing a distribution over empirical weights and an outer risk functional. Theorem 1 appears to follow directly from the standard risk-envelope representation and linearity of expectation.
  2. Although Appendix E discusses standard convergence conditions, no convergence rate or finite-iteration guarantee is provided for Algorithm 1.
  3. Experiments are limited to a single small dataset, with marginal and metric-dependent improvements over ν\nu-SVM. The empirical evidence is insufficient to establish the practical advantages of the proposed framework.
Questions for authors
  1. What are the concrete advantages of specifying an outer risk functional and a distribution over empirical weights rather than directly defining a DRO ambiguity set?
  2. While negative weights increase variability in convex order, could the authors provide empirical evidence or an intuitive explanation of how this might improve robustness under distribution shifts?
  3. Algorithm 1 relies on Monte Carlo sampling, and the proposed approach introduces an additional hyperparameter selected through cross-validation. How does the overall computational cost compare with standard DRO/CVaR methods? A discussion of computational complexity would be helpful.
  4. Given the marginal improvements on a single dataset, can the authors provide stronger empirical evidence to demonstrate the practical advantages of the proposed framework? Additionally, can the authors clarify the respective contributions of Dirichlet and Gaussian weights through ablation studies?
2 comments

Response to Reviewer 3

We greatly thank the reviewer for the constructive review. Below are our responses to each point.

W1.

The paper is difficult to follow due to dense notation and insufficient explanation of the motivation for introducing a distribution over empirical weights and an outer risk functional. Theorem 1 appears to follow directly from the standard risk-envelope representation and linearity of expectation.

We have simplified the notation throughout the paper and clarified the motivation. We kindly refer the reviewer to the updated paper.

Let Xθ(ξi)X_\theta(\xi_i) be the loss of model θ\theta on observation ξi\xi_i. Classic Distributionally robust optimization (DRO) accounts for uncertainty in the data distribution by minimizing the largest expected loss over an ambiguity set [1]. A candidate with nonnegative weights wiw_i summing to one has expected loss ∑iwiXθ(ξi)\sum_iw_iX_\theta(\xi_i). With ambiguity set W\mathcal{W}, the classical DRO problem is

min⁡θsup⁡w∈W∑i=1nwiXθ(ξi).\min_\theta\\ \sup_{w\in\mathcal{W}}\\ \sum_{i=1}^n w_i X_\theta(\xi_i).

A classical DRO selects the most adverse candidate in W\mathcal{W} with the inner maximization, even when we would only assign tiny probability to it.

To account for other candidates, we propose to assign a probability distribution to the candidates and aggregate their expected losses through a risk functional. This lets the learning objective account for how candidates are distributed and how strongly their adverse losses should matter.

Let W=(W1,…,Wn)∼πW=(W_1,\ldots,W_n)\sim\pi be random weights with ∑iWi=1\sum_iW_i=1. For a risk functional R\mathcal{R}, the proposed learning objective is

\qquad W\sim\pi.$$ While the math tools in Theorem 1 is standard, it leads to an important DRO interpretation of the proposed learning objective, which has not appeared in the literature. ## W2. > Although Appendix E discusses standard convergence conditions, no convergence rate or finite-iteration guarantee is provided for Algorithm 1. We have added a **finite-iteration guarantee (Theorem 17, Appendix E.6)**. Assume convex losses and regularization on a closed convex feasible set, an attained finite minimum, and bounded subgradients. Run Algorithm 1 with fresh independent Dirichlet batches, step sizes $\eta_t=(t+10)^{-3/5}$, and average the final half of the updated iterates. For any fixed batch size $M\geq1$, $$\mathbb{E}[F(\bar z_T)]-F^* \leq\frac{2(D_0+5G_F^2)}{T^{2/5}},\qquad T\geq10.$$ Here $F$ is the joint decision–threshold objective, $\bar z_T$ the returned average, $D_0$ the initial squared distance to a minimizer, and $G_F$ a subgradient bound, using the scaled norms defined in the appendix. Informally, the expected optimization error decreases at rate $O(T^{-2/5})$, even with a fixed Monte Carlo batch size. Corollary 18 gives $O(\varepsilon^{-5/2})$ iterations to reach expected error at most $\varepsilon$. The expectation is over the weight draws, with the observed data fixed. ## Q1. > What are the concrete advantages of specifying an outer risk functional and a distribution over empirical weights rather than directly defining a DRO ambiguity set? For any given risk functional $\mathcal{R}$ with envelope representation, the Dirichlet–Gaussian family provides objectives ranging from the empirical average $\frac1n\sum_{i=1}^n X_\theta(\xi_i)$, through $\mathcal{R}(X_\theta)$, to more conservative objectives. Dirichlet weights provide the interpolation and Gaussian perturbations the extrapolation. The weighted losses are ordered in convex order, giving this risk ordering under Theorem 6’s assumptions. **The classical DRO formulation does not have the interpolation-extrapolation scheme for a given risk functional.** Furthermore, we tune the parameter $\tau$ by cross-validation, so the data determine whether the objective should be more or less conservative than the original $\mathcal{R}(X_\theta)$.

Q2.

While negative weights increase variability in convex order, could the authors provide empirical evidence or an intuitive explanation of how this might improve robustness under distribution shifts?

For fixed sample losses, ∑i=1nWiX(ξi)\sum_{i=1}^n W_iX(\xi_i) is linear in WW. Gaussian perturbations to weights therefore give a Gaussian centered at that observation’s loss. Intuitively, This adds Gaussian noise to the sample losses X(ξi)i=1n\\{X(\xi_i)\\}_{i=1}^n (not the observations ξi\xi_i), letting the risk functional consider losses beyond the observed range. See Figure 1 in the paper for visual illustration.

Specifically, For −1<τ<0-1 < \tau < 0, Section 3.3, Eq. (11), gives

\sqrt{-\frac{\tau}{1+\tau}}\\,\operatorname{sd}(X)G, \qquad G\sim\mathcal N(0,1),\qquad G\perp X. \tag{11}$$ Here $X$ is a uniformly sampled loss from $\\{X(\xi_i)\\}_{i=1}^n$. For empirical evidence, the additional experiment on QSAR selects Gaussian weights in all folds at $\nu=0.50$ and $0.40$. DRO can also improve out-of-sample performance without a change in the data-generating distribution by addressing sampling uncertainty \[2\]. Our experiments concern this setting. Controlled distribution-shift experiments remain important future work. ## Q3. > Algorithm 1 relies on Monte Carlo sampling, and the proposed approach introduces an additional hyperparameter selected through cross-validation. How does the overall computational cost compare with standard DRO/CVaR methods? A discussion of computational complexity would be helpful. We compare the arithmetic cost of one joint objective/subgradient evaluation, using $n$ observations and dense linear losses with $d$ features. For empirical CVaR \[3\] and the standard dual formulations of KL- and $\chi^2$-DRO \[4, Eq. (5)\], evaluating the losses and forming the weighted model gradient costs $O(nd)$. The remaining scalar terms cost $O(n)$, giving $O(nd+n)$ per full-data evaluation. For Algorithm 1, evaluating the sample losses once costs $O(nd)$. Drawing $M$ Dirichlet weight vectors and accumulating $\widehat q$ costs $O(Mn)$. The model gradient $\sum_i\widehat q_i\nabla_\theta X_\theta(\xi_i)$ is then computed once in $O(nd)$, giving $\mathbf{O(nd+Mn)}$ per evaluation. The $M$ draws therefore share one computation of the model gradient. For Gaussian weights, the analytic Gaussian mixture objective and gradient in Appendix C require $O(n)$ scalar operations, giving $O(nd+n)$ without Monte Carlo sampling. Thus the Gaussian branch has the same per-evaluation order as these classical reformulations; the Dirichlet branch adds $O(Mn)$ sampling and aggregation work. Selecting $\tau$ from $K$ candidates with $J$ inner folds requires $JK$ fits plus a final refit at each fixed $\nu$. ## Q4. > Given the marginal improvements on a single dataset, can the authors provide stronger empirical evidence to demonstrate the practical advantages of the proposed framework? We have expanded the evaluation to five datasets by adding Credit Approval, QSAR Biodegradation, WDBC, and Raisin. Across 24 comparisons at fixed $\nu$, selecting $\tau$ gives **lower mean error in 18 comparisons, higher mean AUC in 16, and higher mean F1 in 18**. Tables 1–5 in the Official Comment report the full results. For example, at $\nu=0.30$ on WDBC, error decreases from $4.92\\%$ to $3.34\\%$ and F1 increases from $92.90\\%$ to $95.32\\%$. ## Q5. > Additionally, can the authors clarify the respective contributions of Dirichlet and Gaussian weights through ablation studies? Table 6 in the Official Comment reports the weight laws selected across the five outer folds at each $\nu$. WDBC selects Dirichlet weights in all folds, whereas QSAR selects Gaussian weights in all folds at $\nu=0.50$ and $0.40$. Both cases show outperformance. The data decide via cross validation whether a more or less conservative objective is preferred. Both Dirichlet interpolation and Gaussian extrapolation contribute to this choice, with their use depending on $\nu$ and the dataset. Removing either one restricts selection to one side of the whole path and is therefore not ideal. ## References [1] Aharon Ben-Tal, Dick den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust Solutions of Optimization Problems Affected by Uncertain Probabilities. *Management Science*, 59(2):341–357, 2013. [2] John Duchi and Hongseok Namkoong. Variance-based Regularization with Convex Objectives. *Journal of Machine Learning Research*, 20(68):1–55, 2019. [3] R. Tyrrell Rockafellar and Stanislav Uryasev. Optimization of Conditional Value-at-Risk. *The Journal of Risk*, 2(3):21–41, 2000. [4] Hongseok Namkoong and John C. Duchi. Stochastic Gradient Methods for Distributionally Robust Optimization with $f$-divergences. In *Advances in Neural Information Processing Systems*, volume 29, pages 2208–2216, 2016.
Reviewer_577JOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a new approach to distributionally robust optimization based on random weights assigned to training observations. It introduces a family of weight distributions that connects empirical risk minimization to more conservative objectives, including an extension using negative weights. The authors study the mathematical properties of these objectives, propose an optimization algorithm, and test their approach on a classification problem.

Critical strengths

The authors propose a coherent framework connecting empirical risk minimization (ERM), CVaR, and distributionally robust optimization (DRO) through random empirical weights The results appear mathematically correct, with reasonably rigorous proofs.

Critical weaknesses

The article is hard to read with a lot of notations, overly ambiguous phrasing and high level inferences. It does not look suited to this king of conference. It should be in a specialized journal, not at ICLR.

Most theoretical results are consequences of established mathematics:

  • Theorem 1 (DRO representation) follows directly from the dual representation of risk measures and linearity.
  • Theorem 6 (ordering of Dirichlet weights) relies on the existing result of Letac and Piccioni.
  • The stochastic subgradient method uses standard CVaR optimization techniques.
  • The covariance-shrinkage construction is essentially a linear algebra identity. The most original element appears to be the Gaussian extension with signed weights. However, even its main properties follow from standard Gaussian and convex-order arguments.

The experiments involve only one dataset with 961 observations. The reported AUC improvements over ν-SVM range from 0.001 to 0.007, while classification error improves in only three of the five configurations. More importantly, the comparison is not sufficiently strong:

  • The baseline ν-SVM is not independently tuned over its risk parameter.
  • There is no separate ablation demonstrating the contribution of Gaussian extrapolation.
  • No experiments assess robustness under distribution shift or contamination.
  • No broader benchmark establishes whether the small gains generalize.
Questions for authors

What is the main theoretical contribution of the paper? Several results appear to follow directly from established properties of risk measures, Dirichlet distributions, and convex order. Could the authors clarify which results are genuinely new? Why is the Gaussian extension particularly interesting? Could a simpler distribution of signed weights achieve similar properties? What makes the Gaussian choice preferable? What is the statistical interpretation of the parameter \tau? Is there a principled way to select it, beyond cross-validation? Is the comparison with ν-SVM sufficient? In particular, how does the proposed method compare with a ν-SVM whose parameter \nu is also selected by cross-validation? Which part of the proposed method produces the improvement? Could the authors compare Dirichlet interpolation and Gaussian extrapolation separately to identify their respective contributions? Is there convincing evidence that the proposed method performs better than existing techniques?

3 comments

We thank the reviewer for the careful review that helps us clarify contributions and improve readability. We have simplified the writing and notation, rewritten the introduction around the main contributions.

Q1.

What is the main theoretical contribution of the paper?

Our first contribution is a family of new learning objectives. They assign a probability distribution π\pi to the weights W=(W1,…,Wn)W=(W_1,\ldots,W_n) on observations, and apply a risk functional R\mathcal{R} to the resulting weighted losses. Write Xθ(ξi)X_\theta(\xi_i) for the loss of a model with parameters θ\theta on observation ξi\xi_i. Our minimization problem is

\qquad W\sim\pi.$$ Our main theoretical contributions are as follows. - **DRO representation and functional properties (Section 2).** We express the proposed objective as an equivalent Distributionally Robust Optimization (DRO) for a large class of risk functionals $\mathcal{R}$. The DRO has a new type of ambiguity set defined by mixture distributions. We show that the objective preserves convexity and other properties of the chosen risk functional. - **Interpolation and extrapolation of risk functional (Section 3).** When distribution $\pi$ over weights is Dirichlet distribution, its parameter builds a spectrum of objective functions, where the two ends are expected loss $n^{-1}\sum_{i=1}^n X_\theta(\xi_i)$, and the risk functional $\mathcal{R}(\cdot)$ itself. Gaussian mixture distributions as $\pi$ further extend the risk functional to more conservative objectives. - **Recovery of spectral covariance shrinkage (Section 4).** For linear losses and mean–standard deviation risk, we recover a classic spectral covariance shrinkage estimator by specifying the weight covariance. The recovery requires negative weights. - **Optimization for CVaR with random weights (Section 5).** We develop a stochastic subgradient algorithm for the case where $\mathcal{R}$ is CVaR and $\pi$ is Dirichlet distribution. ## Q2. > Several results appear to follow directly from established properties of risk measures, Dirichlet distributions, and convex order. Could the authors clarify which results are genuinely new? Compared with the literature reviewed in the paper, our learning objective has two main differences: $(i)$ The weight distribution $\pi$ is specified directly, with its parameter tuned by cross-validation. It need not be a Bayesian posterior. For example, a density proportional to $\exp\\{-\beta D_{\mathrm{KL}}(u\\|w)\\}$ on the simplex gives a symmetric Dirichlet distribution (Proposition 3). $(ii)$ A Gaussian mixture distribution for $\pi$ allows negative observation weights $W$. The weighted losses extend beyond the empirical loss in convex order, giving more conservative learning objectives. The four contributions listed in Q1 are genuinely new. While the derivations use math tools from convex analysis, probability, and linear algebra, **the resulting family of risk functional, its properties, and optimization with negative random weights have not been studied in the literature**. ## Q3. > Why is the Gaussian extension particularly interesting? The Gaussian extension, like other distributions on the hyperplane, allows negative weights on observations, and can produce objective that is more conservative than the risk functional $\mathcal{R}$. It is particularly interesting for the following reasons: - For fixed sample losses $X(\xi_i)$, the loss $\sum_{i=1}^n W_iX(\xi_i)$ follows a univariate Gaussian mixture distribution. **It adds Gaussian noise to the sample losses $\\{X(\xi_i)\\}_{i=1}^n$** (Section 3.3, Eq. (11)). See Figure 1 in the paper for visual illustration. Gaussian mixture makes risk functionals such as mean-standard deviation easy to compute, and makes algorithm design convenient. - As component variances tend to zero, Gaussian mixture distribution **converge to the vertex law (the uniform distribution on the simplex vertices)**, which is also the limit of Dirichlet distribution. The connection point builds a family of distributions with convex order. The ambiguity set has a subset relation (Theorem 6). ## Q4. > Could a simpler distribution of signed weights achieve similar properties? No, to the best our knowledge. Other distributions on the hyperplane of weights may also increase variability in convex order, but do not have other properties and do not have simple and interpretable form as Gaussian mixtures. For example, a single Gaussian distribution does not have the vertex law as limit. ## Q5. > What makes the Gaussian choice preferable? It is simple and have desired properties discussed in Q4 and Q5. Theorem 7 also gives an explicit interpretation of the induced worst-case weights. The Gaussian mixture distribution adds weight to above-mean losses and subtracts weight from below-mean losses.

Q6.

What is the statistical interpretation of the parameter τ\tau?

For τ>0\tau > 0, it is the total concentration parameter of Dirichlet distribution. The density of π\pi is proportional to exp⁡−βDKL(u∣w)\exp\\{-\beta D_{\mathrm{KL}}(u\\|w)\\}, where DKLD_{\mathrm{KL}} is the KL divergence and uu is the uniform weight vector. Since τ=β+n\tau=\beta+n, increasing τ\tau places more weight near uu.

For −1<τ<0-1<\tau<0, The loss random variable follows Gaussian mixture, and τ\tau controls the variance of mixture components. Specifically, Section 3.3 shows that

\sqrt{-\frac{\tau}{1+\tau}}\\,\operatorname{sd}(X)G, \qquad G\sim\mathcal N(0,1),\qquad G\perp X.$$ Across the full family, decreasing $\tau$ increases the variability of weighted losses. This makes the objective more conservative (Theorem 6). Positive $\tau$ interpolates between expected loss and the chosen risk functional $\mathcal{R}$, while negative $\tau$ extrapolates beyond $\mathcal{R}$. ## Q7. > Is there a principled way to select it, beyond cross-validation? Bayesian inference is used in related work to obtain the parameter \[1, 2\]. However, their formulations do not cover our signed extension with $\tau < 0$. Cross-validation has the advantage that predictive performance determines whether the objective should be more or less conservative than the risk functional $\mathcal{R}$. ## Q8. > Is the comparison with ν-SVM sufficient? Yes, for the comparison studied here. Linear $\nu$-SVM can be written as $\operatorname{CVaR}_\alpha$ minimization with $\alpha=1-\nu$, see \[3\]. **We choose CVaR as $\mathcal{R}$ and test whether random weights improve the risk functional.** If the CVaR classifier with a cross-validated weight distribution outperforms raw CVaR ($\nu$-SVM), it directly demonstrates the benefit of selecting the weight distribution. Other classifiers concerns different risk functional, and thus are not needed to make the point here. ## Q9. > In particular, how does the proposed method compare with a ν-SVM whose parameter $\nu$ is also selected by cross-validation? **The proposed method can further improve the performance when $\nu$ is obtained by cross-validation.** The current experiments compare both methods over the same $\nu$ grid, with $\tau$ selected at each fixed $\nu$. The grid already contains result of hyperparameter tuning by cross validation for $\nu$. For example, on WDBC dataset, $\nu$-SVM with $\nu=0.30$ has the least mean error among all $\nu$. With fixed $\nu=0.30$, selecting $\tau$ reduces error from $4.92\\%$ to $3.34\\%$. We have expanded the evaluation to five datasets. Such improvement is persistent across datasets and metrics, as can be seen from Tables 1–5 in the Official Comment. ## Q10. > Which part of the proposed method produces the improvement? Our formulation has two design choices: the risk functional $\mathcal{R}$ and the weight distribution $\pi$. The case study fixes $\mathcal{R}$ as CVaR. Thus the improvement is obtained by **selecting $\pi$ through cross-validation of $\tau$**. ## Q11. > Could the authors compare Dirichlet interpolation and Gaussian extrapolation separately to identify their respective contributions? Table 6 in the Official Comment reports how often each weight law is selected across the five outer folds at each $\nu$. Table 7 provides paired evidence of improvement over the $\nu$-SVM benchmark. On QSAR at $\nu=0.50$, all five folds select **Gaussian** extrapolation, and the error difference is **−2.09 percentage points [−3.32, −0.85]**. On WDBC at $\nu=0.30$, all five folds select **Dirichlet** interpolation, and the error difference is **−1.58 points [−2.64, −0.70]**. These comparisons illustrate the practical benefit of allowing the data to select a more or less conservative objective at the same $\nu$. The data decide whether Dirichlet interpolation or Gaussian extrapolation is preferred via cross validation. These results support retaining both directions of risk adjustment: **their contribution depend on the dataset and $\nu$, and the selected method leads to outperformance over the benchmark in most reported comparisons.** Removing either segment restricts this choice and is not ideal. ## Q12. > Is there convincing evidence that the proposed method performs better than existing techniques? We have expanded the evaluation to five datasets by adding Credit Approval, QSAR Biodegradation, WDBC, and Raisin. Across 24 comparisons at fixed $\nu$, selecting $\tau$ gives **lower mean error in 18 comparisons, higher mean AUC in 16, and higher mean F1 in 18**. Section 6 reports the principal comparisons, with full tables and variability in Appendix F. These results support the benefit over the empirical CVaR/$\nu$-SVM baseline across several datasets, with improvements varying by setting.

References

[1] Di Wu, Helin Zhu, and Enlu Zhou. A Bayesian Risk Approach to Data-driven Stochastic Optimization: Formulations and Asymptotics. SIAM Journal on Optimization, 28(2):1588–1612, 2018.

[2] Nicola Bariletto and Nhat Ho. Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization. In Advances in Neural Information Processing Systems, volume 37, pages 36438–36470. Curran Associates, Inc., 2024.

[3] Jun-ya Gotoh and Akiko Takeda. A Linear Classification Model Based on Conditional Geometric Score. Pacific Journal of Optimization, 1(2):277–296, 2005.

Reviewer_M9kuOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a distributionally robust optimization (DRO) framework by introducing a probability distribution over empirical sample weights and applying an outer risk functional to the resulting random weighted loss. Using a risk-envelope representation, the authors establish in Theorem 1 that the proposed objective is equivalent to a DRO problem over an induced ambiguity set. This formulation provides additional flexibility in specifying the weight distribution and the risk functional.

Specifically, in Section 3, the authors construct a family of Dirichlet weight distributions through an inverse KL divergence formulation. By varying the concentration parameter (τ\tau), the framework interpolates between deterministic uniform weights, corresponding to empirical risk minimization (ERM), and a uniformly sampled simplex vertex, corresponding to the empirical-loss risk under the chosen outer functional. The authors further extend this family through Gaussian perturbations with a specifically designed covariance structure, resulting in a Gaussian mixture that allows signed sample weights and weighted losses beyond the observed loss range.

The paper also establishes a connection to classical covariance shrinkage, showing that an appropriately chosen weight covariance can recover spherical linear shrinkage under a mean–standard-deviation risk functional and linear loss functions, subject to a full-column-rank condition.

Finally, the authors derive a stochastic subgradient algorithm for optimizing the proposed Dirichlet-weighted CVaR objective. They apply the framework to a classification task, where the empirical CVaR endpoint is connected to the classical (ν\nu)-SVM formulation. Experiments on the Mammographic Mass dataset compare the proposed approach with a (ν\nu)-SVM baseline and report modest performance improvements on several evaluation metrics.

Critical strengths
  • The proposed risk-envelope formulation provides an interesting alternative perspective on distributionally robust optimization (DRO), allowing flexibility in the choice of empirical weight distributions and outer risk functionals.

  • The Gaussian mixture construction, which allows signed empirical weights, offers an interesting extension beyond conventional probability-based reweighting schemes and provides a different perspective on robustness.

  • The theoretical results are generally well structured, with detailed proofs provided for the main claims.

Critical weaknesses
  • Although the authors cite several closely related studies, including [1] and [2], the novelty and advantages of the proposed framework over existing approaches are not sufficiently established. A more explicit discussion of the technical distinctions and additional benefits of the proposed approach would help clarify its contribution.

  • The theoretical framework relies on several structural assumptions on the outer risk functional, including the existence of an unpenalized risk-envelope representation. These assumptions limit the applicability of some of the main results to certain commonly used risk measures. For example, Value-at-Risk (VaR), which is generally nonconvex, does not admit the stated risk-envelope representation. The scope of the framework beyond these assumptions remains unclear.

  • The theoretical analysis primarily focuses on the empirical loss distribution and the induced ambiguity sets constructed from the training samples, without establishing corresponding out-of-sample generalization guarantees. In particular, it remains unclear how the theoretically demonstrated expansion of the ambiguity sets translates into improved generalization or robustness under realistic distribution shifts. Although the paper includes held-out evaluations, it does not directly investigate robustness under distribution shift.

  • The empirical improvements appear relatively modest compared with the reported variability across evaluation folds. In particular, the AUC improvements range from 0.001 to 0.007, while the reported standard deviations are substantially larger. Without an appropriate paired statistical analysis, it is difficult to assess whether these improvements are statistically meaningful. Furthermore, the empirical evaluation is limited to a single dataset.

[1]. Wu, D., Zhu, H., & Zhou, E. (2018). A Bayesian Risk Approach to Data-driven Stochastic Optimization: Formulations and Asymptotics. SIAM Journal on Optimization, 28(2), 1588–1612.

[2]. Bariletto, N., & Ho, N. (2024). Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization. Advances in Neural Information Processing Systems, 37, 36438–36470.

Questions for authors
  • Could the authors clarify how the proposed framework differs from existing approaches, particularly [1] and [2] mentioned above? What are the key theoretical or practical advantages that cannot be obtained from these earlier formulations?

  • Is it possible to extend the proposed framework to a broader class of risk functionals, such as VaR, that do not admit the unpenalized risk-envelope representation used in Equation (3)? If so, which of the main theoretical properties would continue to hold?

  • Could the authors clarify whether the proposed framework provides any theoretical guarantees on out-of-sample generalization or robustness under distribution shifts?

  • Could the authors provide additional empirical evidence to support the effectiveness of the proposed approach? In particular, paired confidence intervals or other appropriate statistical analyses would help assess the significance of the reported improvements. Controlled synthetic experiments under different distribution shifts would also help illustrate the practical implications of the theoretical results and demonstrate whether the proposed formulation improves robustness in meaningful settings.

5 comments

We greatly thank the reviewer for the careful review, which helps us clarify our contribution and motivates extensions of the work.

Q1.

Could the authors clarify how the proposed framework differs from existing approaches, particularly [1] and [2] mentioned above? What are the key theoretical or practical advantages that cannot be obtained from these earlier formulations?

For convenience of illustration, we restate our problem formulation. The notations are simplified as suggested by other reviewers. Let Xθ(ξi)X_\theta(\xi_i) be the loss of model θ\theta on observation ξi\xi_i, and let W=(W1,…,Wn)∼πW=(W_1,\ldots,W_n)\sim\pi be random weights with ∑iWi=1\sum_iW_i=1. For a risk functional R\mathcal{R}, such as R(Y)=E[Y]+sd⁡(Y)\mathcal{R}(Y)=\mathbb{E}[Y]+\operatorname{sd}(Y), the learning objective is

\qquad W\sim\pi.$$ There are two main differences from \[1\] and \[2\]: 1. **The weight distribution $\pi$ is specified directly.** It need not be a Bayesian posterior. For example, a density proportional to $\exp\\{-\beta D_{\mathrm{KL}}(u\\|w)\\}$ relative to the uniform law on the simplex gives a symmetric Dirichlet distribution for $\beta > -n$, where $u=(1/n,\ldots,1/n)$ (Proposition 3). 2. **The weights $W$ can be negative.** The Gaussian mixture distribution as $\pi$ allows negative weights, leading to learning objectives that are more conservative than the risk functional $\mathcal{R}$ in convex order (Section 3.3). This can only be achieved by negative weights. These choices give an important advantage: - Cross-validation can select distribution parameter to improve out-of-sample performance. THe performance on data decides if the objective function should be less or more conservative. Furthermore, we establish the following properties and computational results for this construction: - **DRO representation and functional properties (Section 2).** We express the proposed objective as an equivalent Distributionally Robust Optimization (DRO) for a large class of risk functionals $\mathcal{R}$. The DRO has a new type of ambiguity set defined by mixture distributions. We show that the objective preserves convexity and other properties of the chosen risk functional. - **Recovery of spectral covariance shrinkage (Section 4).** For linear losses and mean–standard deviation risk, we recover a classic spectral covariance shrinkage estimator by specifying the weight covariance. The recovery requires negative weights. - **Optimization for CVaR with random weights (Section 5).** We develop a stochastic subgradient algorithm for the case where $\mathcal{R}$ is CVaR and $\pi$ is Dirichlet distribution. ## Q2. > Is it possible to extend the proposed framework to a broader class of risk functionals, such as VaR, that do not admit the unpenalized risk-envelope representation used in Equation (3)? If so, which of the main theoretical properties would continue to hold? The objective above can also use nonconvex functionals such as VaR without an envelope representation. Several main properties continue to hold: 1. **Inheritance of risk properties (Theorem 2).** Each property transfers whenever $\mathcal{R}$ possesses it. For VaR, translation equivariance, positive homogeneity, and constant preservation survive for weights summing to one; monotonicity in the sample losses also transfers for nonnegative weights. 2. **Interpolation endpoints (Theorem 4).** The weighted loss still approaches the empirical loss distribution as $\tau\downarrow0$ and the empirical mean as $\tau\to\infty$, independently of $\mathcal{R}$. These give the corresponding VaR limits when the quantile functional is continuous at the limiting law. 3. **Convex ordering (Proposition 5 and Theorem 6).** The simplex boundary and the ordering of weighted losses along the Dirichlet/Gaussian path depend only on the weight laws, so both remain valid. The ordering of risk values additionally requires $\mathcal{R}$ to respect convex order, which VaR generally does not. The DRO representation (Theorem 1), ambiguity-set inclusion (Theorem 6), and risk-identifier characterization (Theorem 7) rely on the envelope and do not extend directly to VaR. Convexity of the objective is also generally lost. Existing VaR optimization methods can be applied to a sampled approximation when their assumptions hold. Specifically, draw $W^{(1)},\ldots,W^{(M)}\sim\pi$, form $Y_m(\theta)=\sum_iW_i^{(m)}X_\theta(\xi_i)$, and minimize the empirical quantile of these $M$ weighted losses. For linear losses, this yields a finite-scenario VaR problem of the type studied by \[3\], subject to their feasible-set assumptions. Developing algorithms and guarantees for nonconvex choices is an interesting extension beyond the current scope.

Q3.

Could the authors clarify whether the proposed framework provides any theoretical guarantees on out-of-sample generalization or robustness under distribution shifts?

We have added a generalization bound in Appendix G (Theorem 20). We combine the uniform CDF estimation result of [6, Theorem 4.1] with explicit comparisons for our Dirichlet and Gaussian weight laws. Informally, under i.i.d. sampling, bounded losses, and regularity of the risk functional, an approximately optimized model θ^\widehat\theta with τ^\widehat\tau selected from the data satisfies, with probability at least 1−δ1-\delta,

\leq 2A_{n,\delta}+B_n(\widehat\tau)+\eta_n.$$ Here $R(\theta)=\mathcal{R}(X_\theta(\xi))$ is the population risk and $R^*=\inf_{\theta\in\Theta}R(\theta)$. The three terms quantify **statistical estimation error, the effect of the weight law, and optimization error**, respectively. The estimation term is controlled by sample size and model complexity; the appendix gives explicit expressions for $B_n$, including for CVaR and mean–standard deviation risk. The guarantee covers both Dirichlet interpolation and signed Gaussian extrapolation, uniformly over $\tau$. As the selected $\tau$ approaches zero from either side and the estimation and optimization errors vanish, the learned model is consistent for the population risk. DRO can improve out-of-sample performance when the data-generating distribution does not shift, by guarding against sampling uncertainty, see \[7\]. Our new guarantee concerns this setting. Robustness under distribution shift remains an important direction for future theoretical work. ## Q4. > Could the authors provide additional empirical evidence to support the effectiveness of the proposed approach? In particular, paired confidence intervals or other appropriate statistical analyses would help assess the significance of the reported improvements. Controlled synthetic experiments under different distribution shifts would also help illustrate the practical implications of the theoretical results and demonstrate whether the proposed formulation improves robustness in meaningful settings. We have added four datasets: Credit Approval, QSAR Biodegradation, Wisconsin Diagnostic Breast Cancer (WDBC), and Raisin, alongside Mammographic Mass. Across the five datasets and 24 comparisons at fixed $\nu$, selecting $\tau$ gives **lower mean error in 18 comparisons, higher mean AUC in 16, and higher mean F1 in 18**. Table 1 in the paper and Tables 2–5 in the Official Comment report the full results and variability. **Table 7 provides paired evidence of improvement over the $\nu$-SVM benchmark.** Among the **19** settings in the table, **eight** error intervals exclude zero in our favor and none favor the baseline; these error gains occur on all four datasets. For F1, **nine** intervals favor our method and one favors the baseline. For AUC, **four** favor our method and none favor the baseline. The remaining intervals include or touch zero. The gains occur with both directions of risk adjustment. On QSAR at $\nu=0.50$, all five folds select **Gaussian** extrapolation, and the error difference is **−2.09 percentage points [−3.32, −0.85]**. On WDBC at $\nu=0.30$, all five folds select **Dirichlet** interpolation, and the error difference is **−1.58 points [−2.64, −0.70]** (selection counts in Table 6). These comparisons illustrate the practical benefit of allowing the data to select a more or less conservative objective at the same $\nu$. DRO can improve out-of-sample performance when the data-generating distribution does not shift, by guarding against sampling uncertainty, see [7]. Controlled experiments under distribution shift remain for future research.

Table 7. Paired confidence intervals

Entries are CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau minus ν\nu-SVM: point difference [95% paired bootstrap interval]. Error and F1 differences are in percentage points (pp); AUC differences are in AUC units. Negative error differences and positive AUC/F1 differences favor selecting τ\tau.

We use 20,000 paired bootstrap resamples, sampling the same observations for both methods within each test fold and class, then averaging the five within-fold metric differences. These pointwise intervals condition on the saved models, partitions, selected τ\tau, and class counts. They exclude retraining and selection variability, do not account for overlapping training sets, and have no multiplicity adjustment. Differences use unrounded means.

Datasetν\nuError difference (pp)AUC differenceF1 difference (pp)
Credit Approval0.80-2.17 [-3.48, -0.87]+0.0048 [+0.0016, +0.0085]+3.76 [+1.90, +5.93]
0.70-0.58 [-2.03, +0.87]+0.0009 [-0.0023, +0.0044]+1.22 [-0.43, +2.99]
0.60-0.43 [-1.16, +0.29]+0.0001 [-0.0024, +0.0027]+0.45 [-0.31, +1.28]
0.50-0.43 [-1.16, +0.14]+0.0014 [-0.0027, +0.0059]+0.34 [-0.25, +0.96]
0.40+0.87 [-0.14, +2.03]+0.0020 [-0.0039, +0.0083]-1.25 [-2.57, -0.07]
QSAR0.60-0.95 [-1.90, 0.00]+0.0062 [+0.0023, +0.0103]+1.98 [+0.36, +3.70]
0.50-2.09 [-3.32, -0.85]+0.0038 [-0.0034, +0.0111]+3.39 [+1.34, +5.50]
0.40+0.47 [-0.47, +1.42]-0.0037 [-0.0082, +0.0008]-0.37 [-1.90, +1.23]
0.30+0.66 [0.00, +1.42]-0.0016 [-0.0045, +0.0013]-0.89 [-2.09, +0.31]
WDBC0.70-10.54 [-13.01, -8.08]+0.0022 [+0.0007, +0.0042]+20.16 [+15.36, +25.82]
0.60-2.99 [-4.76, -1.23]+0.0010 [+0.0001, +0.0023]+5.27 [+2.56, +8.47]
0.50-0.36 [-1.76, +1.05]-0.0005 [-0.0022, +0.0010]+0.90 [-0.97, +2.97]
0.400.00 [-1.06, +1.05]-0.0003 [-0.0016, +0.0005]+0.23 [-1.22, +1.85]
0.30-1.58 [-2.64, -0.70]-0.0002 [-0.0010, +0.0004]+2.42 [+1.03, +4.15]
Raisin0.90-3.67 [-5.56, -1.78]+0.0003 [-0.0003, +0.0010]+6.61 [+4.21, +9.25]
0.80-2.56 [-4.11, -1.00]-0.0005 [-0.0013, +0.0002]+4.31 [+2.40, +6.41]
0.70-1.67 [-2.78, -0.56]-0.0005 [-0.0017, +0.0005]+2.43 [+1.10, +3.88]
0.60-0.22 [-0.78, +0.33]+0.0002 [-0.0008, +0.0011]+0.29 [-0.31, +0.97]
0.50+0.22 [0.00, +0.56]-0.0001 [-0.0006, +0.0004]-0.25 [-0.65, 0.00]

References

[1] Di Wu, Helin Zhu, and Enlu Zhou. A Bayesian Risk Approach to Data-driven Stochastic Optimization: Formulations and Asymptotics. SIAM Journal on Optimization, 28(2):1588–1612, 2018.

[2] Nicola Bariletto and Nhat Ho. Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization. In Advances in Neural Information Processing Systems, volume 37, pages 36438–36470. Curran Associates, Inc., 2024.

[3] Jong-Shi Pang and Sven Leyffer. On the Global Minimization of the Value-at-Risk. arXiv preprint math/0401063, 2004.

[4] R. Tyrrell Rockafellar. Convex Analysis. Princeton Mathematical Series, volume 28. Princeton University Press, Princeton, NJ, 1970.

[5] R. Tyrrell Rockafellar and Stan Uryasev. The Fundamental Risk Quadrangle in Risk Management, Optimization and Statistical Estimation. Surveys in Operations Research and Management Science, 18(1–2):33–53, 2013.

[6] Liu Leqi, Audrey Huang, Zachary C. Lipton, and Kamyar Azizzadenesheli. Supervised Learning with General Risk Functionals. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, volume 162, pages 12570–12592. PMLR, 2022.

[7] John Duchi and Hongseok Namkoong. Variance-based Regularization with Convex Objectives. Journal of Machine Learning Research, 20(68):1–55, 2019.

Comments on the paper

3 comments

We thank all reviewers for their careful review. We collect the tables here because they address questions raised in more than one review.

Linear ν\nu-SVM can be equivalently written as a CVaR⁡α\operatorname{CVaR}_{\alpha} minimization with ν=1−α\nu=1-\alpha, see [1]. We therefore use ν\nu-SVM as the benchmark when we apply our random weight scheme on CVaR⁡α\operatorname{CVaR}_{\alpha} functional. We test whether selecting τ\tau by cross-validation improves classification under various ν\nu values.

Tables 2–5 report predictive performance on four additional datasets. Together with Mammographic Mass (Table 1 in the paper), the evaluation on five datasets shows improvement on the ν\nu-SVM benchmark in the majority of comparisons at fixed ν\nu. Among the 24 settings across datasets and ν\nu values, our method gives lower mean classification error in 18, higher mean AUC in 16, and higher mean F1 in 18.

Five stratified outer folds evaluate performance. Three stratified inner folds select τ\tau by mean balanced accuracy at each fixed ν=1−α\nu=1-\alpha. The candidates include Dirichlet weights, empirical CVaR, and Gaussian weights. Entries are means ±\pm sample standard deviations across the five outer folds; error and F1 are percentages. Bold marks the better unrounded mean at each fixed ν\nu.

Table 6 reports the parameter τ\tau of the weight distributions selected by inner cross-validation. Both Dirichlet interpolation (τ>0\tau>0) and Gaussian extrapolation (−1<τ<0-1<\tau<0) are selected in various datasets, and the selection frequencies varies substantially across datasets and ν\nu value. This supports retaining both segments to provide the flexibility, so cross-validation can select a less or more conservative objective according to the data and the parameter ν\nu. Each row counts the selections in the five outer folds, including the raw CVaR (τ=0\tau=0), and sums to five.


Table 2. Credit Approval

ν\nuMethodError ↓\downarrowAUC ↑\uparrowF1 ↑\uparrow
0.80ν\nu-SVM17.97±4.2717.97\pm4.270.913±0.0220.913\pm0.02276.79±6.9076.79\pm6.90
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau15.80±2.83\mathbf{15.80}\pm2.830.917±0.019\mathbf{0.917}\pm0.01980.55±4.36\mathbf{80.55}\pm4.36
0.70ν\nu-SVM14.20±1.9614.20\pm1.960.921±0.0150.921\pm0.01583.42±3.1583.42\pm3.15
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau13.62±2.92\mathbf{13.62}\pm2.920.922±0.015\mathbf{0.922}\pm0.01584.63±3.60\mathbf{84.63}\pm3.60
0.60ν\nu-SVM14.49±1.9814.49\pm1.980.923±0.0150.923\pm0.01584.92±2.1884.92\pm2.18
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau14.06±2.09\mathbf{14.06}\pm2.090.923±0.013\mathbf{0.923}\pm0.01385.37±2.08\mathbf{85.37}\pm2.08
0.50ν\nu-SVM14.35±1.8814.35\pm1.880.925±0.0140.925\pm0.01485.20±1.8585.20\pm1.85
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau13.91±1.88\mathbf{13.91}\pm1.880.926±0.016\mathbf{0.926}\pm0.01685.55±1.86\mathbf{85.55}\pm1.86
0.40ν\nu-SVM14.35±1.88\mathbf{14.35}\pm1.880.922±0.0170.922\pm0.01785.20±1.85\mathbf{85.20}\pm1.85
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau15.22±2.0515.22\pm2.050.924±0.017\mathbf{0.924}\pm0.01783.95±2.6483.95\pm2.64

Table 3. QSAR Biodegradation

ν\nuMethodError ↓\downarrowAUC ↑\uparrowF1 ↑\uparrow
0.60ν\nu-SVM18.58±2.2118.58\pm2.210.898±0.0230.898\pm0.02369.71±5.3069.71\pm5.30
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau17.63±2.49\mathbf{17.63}\pm2.490.904±0.027\mathbf{0.904}\pm0.02771.68±4.71\mathbf{71.68}\pm4.71
0.50ν\nu-SVM16.11±1.9316.11\pm1.930.914±0.0240.914\pm0.02474.87±3.4074.87\pm3.40
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau14.03±2.52\mathbf{14.03}\pm2.520.918±0.030\mathbf{0.918}\pm0.03078.25±4.08\mathbf{78.25}\pm4.08
0.40ν\nu-SVM13.27±2.15\mathbf{13.27}\pm2.150.927±0.022\mathbf{0.927}\pm0.02279.62±3.37\mathbf{79.62}\pm3.37
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau13.74±2.4913.74\pm2.490.924±0.0250.924\pm0.02579.24±3.7979.24\pm3.79
0.30ν\nu-SVM12.23±1.88\mathbf{12.23}\pm1.880.927±0.018\mathbf{0.927}\pm0.01881.55±3.04\mathbf{81.55}\pm3.04
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau12.89±2.7312.89\pm2.730.926±0.0200.926\pm0.02080.66±4.0380.66\pm4.03

[1] Jun-ya Gotoh and Akiko Takeda. A Linear Classification Model Based on Conditional Geometric Score. Pacific Journal of Optimization, 1(2):277–296, 2005.

Table 4. Wisconsin Diagnostic Breast Cancer

ν\nuMethodError ↓\downarrowAUC ↑\uparrowF1 ↑\uparrow
0.70ν\nu-SVM16.69±1.8316.69\pm1.830.988±0.0070.988\pm0.00771.20±4.0771.20\pm4.07
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau6.15±2.55\mathbf{6.15}\pm2.550.990±0.006\mathbf{0.990}\pm0.00691.37±3.71\mathbf{91.37}\pm3.71
0.60ν\nu-SVM8.79±2.258.79\pm2.250.989±0.0070.989\pm0.00786.60±3.8486.60\pm3.84
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau5.80±2.88\mathbf{5.80}\pm2.880.990±0.006\mathbf{0.990}\pm0.00691.87±4.15\mathbf{91.87}\pm4.15
0.50ν\nu-SVM6.15±1.666.15\pm1.660.990±0.007\mathbf{0.990}\pm0.00790.95±2.6290.95\pm2.62
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau5.80±2.52\mathbf{5.80}\pm2.520.990±0.0080.990\pm0.00891.85±3.56\mathbf{91.85}\pm3.56
0.40ν\nu-SVM5.27±1.765.27\pm1.760.992±0.007\mathbf{0.992}\pm0.00792.33±2.7292.33\pm2.72
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau5.27±2.40\mathbf{5.27}\pm2.400.992±0.0070.992\pm0.00792.56±3.43\mathbf{92.56}\pm3.43
0.30ν\nu-SVM4.92±2.124.92\pm2.120.994±0.006\mathbf{0.994}\pm0.00692.90±3.1692.90\pm3.16
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau3.34±0.75\mathbf{3.34}\pm0.750.994±0.0060.994\pm0.00695.32±1.11\mathbf{95.32}\pm1.11

Table 5. Raisin

ν\nuMethodError ↓\downarrowAUC ↑\uparrowF1 ↑\uparrow
0.90ν\nu-SVM19.56±3.5219.56\pm3.520.919±0.0130.919\pm0.01375.98±5.2275.98\pm5.22
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau15.89±2.56\mathbf{15.89}\pm2.560.919±0.013\mathbf{0.919}\pm0.01382.59±3.04\mathbf{82.59}\pm3.04
0.80ν\nu-SVM17.78±3.4517.78\pm3.450.921±0.013\mathbf{0.921}\pm0.01379.11±4.9379.11\pm4.93
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau15.22±3.01\mathbf{15.22}\pm3.010.920±0.0130.920\pm0.01383.42±3.52\mathbf{83.42}\pm3.52
0.70ν\nu-SVM16.33±3.3216.33\pm3.320.921±0.012\mathbf{0.921}\pm0.01281.66±4.1281.66\pm4.12
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau14.67±3.48\mathbf{14.67}\pm3.480.921±0.0120.921\pm0.01284.09±4.14\mathbf{84.09}\pm4.14
0.60ν\nu-SVM14.11±2.9614.11\pm2.960.921±0.0130.921\pm0.01384.85±3.3984.85\pm3.39
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau13.89±3.36\mathbf{13.89}\pm3.360.922±0.012\mathbf{0.922}\pm0.01285.15±3.83\mathbf{85.15}\pm3.83
0.50ν\nu-SVM13.33±3.24\mathbf{13.33}\pm3.240.922±0.013\mathbf{0.922}\pm0.01386.21±3.40\mathbf{86.21}\pm3.40
CVaR⁡α\operatorname{CVaR}_{\alpha} with selected τ\tau13.56±3.1113.56\pm3.110.922±0.0130.922\pm0.01385.96±3.2685.96\pm3.26

The choice of ν\nu grid in Tables 2-5 follow the intercept feasibility conditions in Appendix F.


Table 6. Weight laws selected by inner cross-validation

Each dataset–ν\nu row counts the selections in the five outer folds.

Datasetν\nuDirichlet (τ>0\tau > 0)Empirical CVaR⁡α\operatorname{CVaR}_{\alpha} (τ=0\tau=0)Gaussian (−1<τ<0-1 < \tau < 0)
Mammographic Mass0.90500
0.80122
0.70203
0.60302
0.50401
Credit Approval0.80104
0.70104
0.60113
0.50401
0.40311
QSAR Biodegradation0.60203
0.50005
0.40005
0.30221
WDBC0.70500
0.60500
0.50500
0.40500
0.30500
Raisin0.90500
0.80500
0.70500
0.60203
0.50221
Total73839

We thank the reviewers for careful reading and the ACs for overseeing the review process. We have revised the manuscript and provided point-by-point responses. We summarize the main updates below.

  1. Contribution and presentation. We have simplified the notation and clarified the motivation and main contributions. Our responses explain how the new family of learning objectives and its properties differ from classical DRO and Bayesian formulations, while acknowledging the established mathematical tools used in the derivations. [M9ku Q1; 577J Q1–Q2; 1U3g W1, Q1]

  2. Gaussian construction. We explain the role of negative weights and why the weighted loss follows a Gaussian mixture distribution. We also discuss the Gaussian construction’s tractability, the induced worst-case weights, and the possibility of other signed-weight distributions. [577J Q3–Q5; 1U3g Q2]

  3. Parameter interpretation and selection. We clarify the statistical interpretation of τ\tau along the Dirichlet–Gaussian family, which connects expected loss, the chosen empirical risk, and more conservative objectives. Cross-validation lets the data determine the preferred direction of risk adjustment. [577J Q6–Q7; 1U3g Q1]

  4. Scope of risk functionals. We clarify the scope of the risk-envelope representation and explain which theoretical properties continue to hold for nonconvex risk functionals such as VaR. We also discuss how existing VaR optimization methods may apply. [M9ku Q2]

  5. Out-of-sample guarantees. We have added a population-risk bound in Appendix G covering both weight families. We explain how DRO can improve out-of-sample performance under sampling uncertainty and distinguish this setting from distribution shift, which remains future work. [M9ku Q3–Q4; 1U3g Q2]

  6. Optimization and computational cost. We have added a finite-iteration guarantee for Algorithm 1 in Appendix E.6. We also compare per-evaluation computational costs with standard DRO/CVaR methods and discuss the additional cost of cross-validation. [1U3g W2, Q3]

  7. Empirical and statistical evidence. We have expanded the evaluation to five datasets and added paired bootstrap confidence intervals. We explain the CVaR-SVM connection and clarify how the comparisons at fixed ν\nu assess the benefit of selecting the weight distribution. The results appear in Section 6, Appendix F, and Tables 2–5 and 7 in the responses. [M9ku Q4; 577J Q8–Q10, Q12; 1U3g Q4]

  8. Contributions of both weight families. Table 6 reports how often Dirichlet interpolation and Gaussian extrapolation are selected. Both contribute, with their use depending on the dataset and ν\nu, supporting the flexibility to select a less or more conservative objective. [577J Q11; 1U3g Q5]

Please see the individual responses for details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.