PBNNs: Plastic Bayesian Neural Networks for Bio-Inspired Bayesian Learning

The Bayesian neural network (BNN) represents model parameters with probability distributions to introduce uncertainty during learning. The variational Bayesian learning approach prevents overfitting and naturally introduces regularization, which is especially useful for applications where data is scarce. However, traditional BNNs use global learning rates to adjust posterior means and variances, which fails to take full advantage of uncertainty and leads to suboptimal distribution updates. In this work, we overcome this challenge by using Bayesian plasticity to adjust individual synaptic learning rates that depend on uncertainty. We demonstrate the effectiveness of Bayesian plasticity through MLP and CNN implementations that yield superior solutions and robustness comparable to traditional BNNs initialized in the same manner.