Modified Satlin activation function for Deep Neural Networks

Deep Neural Networks are widely used to solve a variety of practical problems: language processing, image and video recognition, and others. Such architectures include millions of elementary neurons, used activation functions. Their properties significantly affect the entire network results. The paper examines the problem of increasing Deep Neural Networks learning rate and accuracy through the use of variable shape activation functions, which parameters can be adjusted during network training. As a solution, a modified Satlin activation function is proposed. It has compressive properties, but does not contain asymptotic branches, protecting Deep Neural Network from a vanishing gradient.