Robust Optimization with Random Empirical Distributions

We propose learning objectives that apply a risk functional to the random expected loss produced by a distribution over empirical reweightings. Under an unpenalized risk envelope representation, the objective equals a robust optimization problem over mixtures of the candidate measures. The construction inherits properties of the outer risk functional. A density specified through inverse KL divergence gives a Dirichlet family interpolating between the empirical expectation and the risk of a uniformly sampled observation. At the latter endpoint, the uniform vertex law is maximal in convex order among probability weight laws with the same mean. Gaussian perturbations continue beyond this boundary with weights that may be negative and still sum to one, allowing integrated losses outside the observed range for nonconstant losses. For a fixed outer risk respecting convex order, the induced risks are ordered along the path, with nested ambiguity sets under the envelope representation. For losses linear in the decision, a prescribed weight covariance recovers spherical linear shrinkage in a mean–standard deviation objective when the centered feature matrix has full column rank. For Dirichlet weights and conditional value-at-risk (CVaR), we develop a stochastic subgradient algorithm. On Mammographic Mass, selecting the weight distribution by cross-validation gives slightly higher mean AUC than $\nu$-SVM at all five tested CVaR levels and lower mean classification error at the three higher levels.