Energy-Efficient Probabilistic Bayesian Neural Networks for Resource-Constrained Environments

In the realm of cyber-physical systems, where tight integration between computational and physical elements is paramount, the demand for efficient and scalable machine learning models is ever-present. Bayesian Neural Networks (BNNs) present a promising avenue for tackling complex tasks in such systems, yet their widespread adoption is hindered by computational complexity and resource constraints. To address these challenges, we propose an energy-efficient FPGA implementation of Probabilistic Bayesian Neural Networks (ProbBNNs), a novel architecture that leverages probabilistic computing principles to streamline inference in BNNs. In ProbBNN, instead of representing each parameter as a random variable with a mean and variance, a set of parameters for each neuron is represented by a Probability Density Function (PDF) that characterizes the distribution of these random variables, with random streams following the PDFs processed through probabilistic computing principles to propagate uncertainty throughout the network. The adoption of probabilistic computing eliminates the need for expensive Multiply-Accumulate (MAC) operations, further enhancing the scalability and cost-effectiveness of our proposed implementation of ProbBNN in real-world applications. Furthermore, Gaussian Mixture Models (GMMs) are employed to efficiently capture the underlying distributions of weights, thereby reducing the number of BNN parameters. Through evaluation and comparison with traditional BNN architectures, this implementation demonstrates significant improvements in computational efficiency, memory utilization, and energy efficiency, making it well suited for deployment in resource-constrained cyber-physical systems, where efficient and reliable decision-making is crucial.