LENS: Level-set ENSemble for post-hoc uncertainty quantification

To exploit the nonlinear geometry of the empirical loss landscape in parameter space, we introduce Level-set ENSemble (LENS), an ensemble-based post-hoc uncertainty quantification method. Rather than imposing a parametric approximation to the posterior or restricting uncertainty exploration to a linear subspace, LENS constructs an ensemble of parameter configurations constrained to lie near a fixed loss level set. This set contains models with empirical loss comparable to the trained model but potentially different predictions. The resulting model ensemble yields uncertainty estimates derived from structured, non-linear variation in parameter space. The proposed method is architecture-agnostic and does not require retraining. Across diverse prediction tasks spanning text, image, and tabular data, LENS improves uncertainty estimation and calibration over strong post-hoc methods, while maintaining predictive accuracy. Our results establish loss level sets as a principled and practical route to exploring non-linear uncertainty structure in deep learning models.