DSS-LVR Flow: A Unified Method for Structural Sparsity in Bayesian Neural Networks

Bayesian neural networks (BNNs) provide uncertainty-aware modeling of complex nonlinear relationships in scientific applications. Their flexibility, however, comes with rapidly expanding and redundant parameterizations, making it important to identify which predictors, hidden units, and larger computational structures are actually supported by the data. Discrete spike-and-slab (DSS) priors provide a natural Bayesian mechanism for this purpose: the point-mass spike enables exact exclusion, while the continuous slab retains posterior uncertainty for active parameters. We introduce DSS-LVR Flow, a flow-based method built on a DSS latent-variable representation (DSS-LVR), in which a group-specific activation latent variable and a shared continuous threshold jointly determine each inclusion decision. By changing which parameters share an inclusion state, the same construction supports predictor- and hidden-unit-level selection, while combinations of these primitive states induce edge and path activity without additional structural latent variables. DSS-LVR Flow uses a role-aware inverse autoregressive flow (IAF) to approximate the joint posterior. It also introduces a smooth, bounded, exact-zero structural activation function with a controllable transition region for gradient-based optimization. Experiments show that DSS-LVR Flow efficiently approximates parameter posteriors with accuracy close to that of long-run MCMC. In multilayer perceptrons, it also supports multiple structural sparsity targets, producing highly sparse structures while preserving accurate nonlinear function recovery.