Linear Irrational Unit (LIU): An Activation Function For Convolutional Neural Network

In a convolutional neural network (CNN), the output is produced by passing the input through an activation function (AF). The traditional AFs like sigmoid and tanh suffer from disappearing gradient problems. The disappearing gradient issue was solved by the ReLU AF, but the negative region issue still exists. This study introduces a non-monotonic, nonlinear AF dubbed the Linear Irrational Unit (LIU). The function is continuously differentiable in both the positive and negative regions, which overcomes the disappearing gradient problem. As the proposed function has a non-zero value in the negative region, it does not suffer from the negative region problem. The effectiveness of the proposed LIU activation function was assessed using MNIST, CIFAR-10 and Chest X-ray datasets. With our proposed CNN model, the LIU activation function can reach an accuracy of 85.26% on the Chest X-ray dataset, 99.52% on the MNIST dataset, and 83.15% on the CIFAR dataset, which is better than the performance of other modern activation functions.