$\nu$-Bayes: Robust Belief Updating Under Subpopulation Shift

Subpopulation shift changes the prevalence of latent groups between training and test distributions. Without group annotations or an explicit model of this shift, ordinary Bayesian updating treats the training mixture as representative, potentially favoring patterns supported by overrepresented groups. We introduce $\nu$-Bayes, a group-annotation-free Bayesian method that learns the influence of each observation on the posterior, which in practice upweights informative minority examples while downweighting overrepresented ones. Starting from the principle of minimum discrimination information, we represent each observation through a soft constraint on its posterior expected loss. The associated Lagrange multipliers become learned likelihood powers, with ordinary Bayes recovered under a uniform allocation. Rather than specifying a separate tolerance for every observation, $\nu$-Bayes learns a shared loss threshold, while $\nu$ controls the degree to which the likelihood powers may concentrate. We show that the resulting dual is an empirical distributionally robust optimization (DRO) problem over an order-$\infty$ R\'enyi ambiguity set. Consequently, without using group annotations during optimization, its robust objective controls the empirical risk of every latent group containing at least a $\nu$ fraction of the training sample and provides population worst-group guarantees under subpopulation mixture shift. We combine primal–dual optimization with IVON to make the posterior update practical for deep neural networks. Experiments on controlled and benchmark shifts demonstrate improved worst-group performance over ordinary IVON and show how the learned likelihood powers redistribute influence across observations.