Empirical Priors for Bayesian Neural Networks via Weight Pruning Sensitivity

Bayesian neural networks provide a principled framework for uncertainty estimation, yet their performance is highly dependent on the choice of prior over network parameters. Designing informative priors for modern neural networks remains challenging because their large parameter spaces make it difficult to characterize the functional importance of individual parameters in a computationally tractable way. In this work, we propose an empirical prior for Bayesian neural networks that uses continuous pruning sensitivity to allocate Gaussian prior precision across individual parameters. Through extensive experiments across architectures, datasets, and prior configurations, we demonstrate that our approach achieves consistent performance gains across model families and dataset scales, suggesting that network pruning can be used not only for model compression, but also as a useful tool for designing informative priors in Bayesian neural networks.