Bessel-type Activation Functions in Neural Networks

This paper introduces a novel family of activation functions using hypergeometric functions with trainable parameters. Hypergeo-metric functions possess a complex series structure that may create numerical issues. Therefore, we focus on Bessel-type functions of the first kind. The Bessel functions are a subfamily of hypergeometric functions, with characteristics similar to ReLU and sinusoidal activation functions. We investigate the performance of Bessel functions as activation functions within Convolutional Neural Networks (CNNs). Comparative analyses are conducted against established activation functions such as ReLU and sigmoid on benchmark datasets, including MNIST, Happy House, and CIFAR-10. This research aims to explore the potential of Bessel functions as a general class of parametric activation functions adaptable to diverse network architectures.