Phase-Free Bidirectional Structural Adaptation in Bayesian Neural Networks

Biological neural systems continuously adapt their structure through the formation and elimination of neural connections, whereas artificial neural networks typically retain a fixed architecture throughout training. Amongst existing structural adaptation approaches, most rely on pruning rather than growth, with the combination of the two being the exception, typically relying on pre-defined adaptation phases. We propose, to the best of our knowledge, the first phase-free bidirectional structural adaptation algorithm that leverages parameter statistics in a Bayesian neural network framework to control optional growth or pruning steps through a single unified criterion. Concretely, candidates for growth and pruning are proposed based on weight-level uncertainty and signal-to-noise-ratio, while candidate selection follows a common penalised evidence lower bound that explicitly accounts for model complexity. This enables network capacity to continuously increase, decrease, or remain unchanged throughout training. We evaluate the proposed framework on fully connected and convolutional architectures using Fashion-MNIST and CIFAR-10. Across our experiments, the method achieves favourable performance–complexity trade-offs compared with static architectures and dynamic baselines that rely on predefined growth and pruning schedules or separate structural criteria.