Automatic Modulation Classification with Multi-modal Data Fusion Located at The Feature Layer

Since the widespread adoption of 5G technology, wireless communication systems have exhibited significant data throughput. Accordingly, the union of deep learning and wireless communication technology has become a major mainstream development direction. Automatic modulation classification (AMC) has the ability to aid 5G wireless communications by reallocating spectrum resources with low usage. As a result, it will play an increasingly important role in dynamic spectrum access. To improve the classification performance, a multitude of AMC techniques rooted in convolutional neural network prototypes and their variations have emerged in this field. However, many approaches rely solely on single-modal data from either sequence or image signals, disregarding the potential benefits of combining multimodal data for enhanced processing. To address this issue, we propose a multimodal fusion network (AMFN) for AMC. This network combines a convolutional neural network (CNN), a long and short-term memory network (LSTM), and two types of attention-based mechanism modules. Moreover, the network combines the features of sequence and image signal modes, enhancing the recognition ability of modulation signals by complementing each other. Experiment results demonstrate that the algorithm outperforms other related methods on the widely utilized public datasets RML2016.10A.