On the Effect of Activation Functions in Real Valued Time Delay Neural Networks Based Behavioral Modeling of 5G Power Amplifiers

This paper focuses on the behavioral modeling of nonlinear power amplifiers using real-valued time delay neural network models. A study of the effects of the activation function on the performance of these neural network-based models is presented. Two devices under test with different nonlinearity profiles are used. Real-valued time delay neural network, and augmented real-valued time delay neural network models were derived using the sigmoid, the rectified linear unit, and the tanh activation functions. For each model, the performance was assessed in terms of the normalized mean square error for various settings of the model parameters (number of layers and number of neurons per layer). The results showed that the activation function impact may be device dependent but is model independent. Furthermore, it was observed that the ReLU is more suitable for large size models while the tanh and sigmoid activation functions outperform the ReLU function at models with reduced size.