Open-Set Automatic Modulation Classification Using Deep Metric Learning and Openmax

Automatic modulation classification (AMC) is a key technique for identifying the modulation schemes of wireless signals, enabling improved performance and security in communication systems by accurately classifying signal types. However, most existing AMC research assumes modulation classes are part of a closed set, which can cause classifiers to misidentify unknown modulation schemes as known ones, undermining both the security and reliability of communication systems. To address this, we propose a novel open set AMC (OS-AMC) method based on deep metric learning and OpenMax (M-OpenMax). The proposed M-OpenMax-based OS-AMC method utilizes crossentropy loss and center loss to extract separable and discriminative signal features and uses OpenMax to adjust the nonnormalized score output of the model to achieve the classification of known signals and removal of unknown signals. Experimental results demonstrate that the proposed M-OpenMax-based OSAMC method outperforms other open-set AMC techniques, particularly in its ability to handle unknown modulation types.