On Transfer Learning and Autoencoder-Assisted End-to-End Cooperative Communication Systems

Relay-assisted cooperative communication is essential for improving spectrum efficiency and coverage expansion. Recently, deep learning has been adopted in a cooperative communication system using an autoencoder (AE) thanks to its noise and error reduction ability. However, the existing AE-based method for cooperative communication systems has limitations in model scalability owing to the unsatisfied model architecture. The data rate is limited because it can only deal with codewords of short length. Therefore, we first propose a convolutional neural network-based autoencoder to make the AE process input data of any length, which can be integrated with outer channel code of long or short lengths for data rate improvement without modifying model architecture. Then, transfer learning is developed to make the AE satisfy a dynamic environment by leveraging the information learned from the past task. Simulation results show that the proposed method outperforms the traditional amplify-and-forward (AF), decode-and-forward (DF) protocols adopting the maximum ratio combine strategy. Moreover, the proposed method achieves a 2 dB gain compared to the conventional AE over fading channels in coded transmissions.