SELU Activation Function for Precise CTR Prediction

We examine the application of the scaled exponential linear unit (SELU) activation function in click-through rate (CTR) prediction models. When using the SELU activation function, the neuron activations automatically converge towards zero mean and unit variance, thus removing the need to use batch normalization. We compare the use of SELU together with batch and layer normalization, residual networks, and PReLU, the parametric ReLU. We demonstrate the improvements obtained with SELU on a large-scale dataset containing over 700 million instances on CTR prediction using the prediction models DeepFM and Masknet. The improvements are significant in prediction accuracy, especially in terms of logloss and F1 score. The percentage improvements reach 9.22 % in logloss and 1.76 % in F1 for DeepFM and 14.57 % in logloss and 4.06 % in F1 for Masknet without changing the AUC significantly. The SELU activation function is easy to integrate in any neural network model and offers an alternative to traditional activation functions, optimizing model precision in real-world CTR applications.