An Asymptotic One-Sample Surrogate for Non-Gaussian ERM with General Convex Regularization

We study convex empirical risk minimization (ERM) with non-Gaussian features and nonsmooth penalties through a surrogate driven by a single Gaussian vector. The surrogate preserves the penalty and calibrates its coefficients using the actual joint law of the feature–label model. We prove that this population calibration exists and is unique. We conjecture a conditional Gaussian prediction principle for fresh scores when the data satisfy a Poincaré inequality. Conditional on this conjecture, standard concentration conditions on the data and regularity conditions on the losses imply agreement between the ERM minimizer and its surrogate in normalized means, prediction variances, and conditional fresh-score laws averaged over the fresh feature. The result covers general uniformly strongly convex penalties and, under projection anti-concentration, pure Lasso with regularization strengths bounded away from zero and infinity. Transfer of the surrogate's support statistics to the trained estimator remains open, although it is strongly supported by simulations. The core proof relies on a cavity Hamilton–Jacobi argument that compares the main statistics of the two minimizers without differentiating the penalty.