Effects of Composite Cross-Entropy Loss on Adversarial Robustness

Convolutional neural networks (CNNs) can efficiently extract image features and perform corresponding classification. Typically, the CNN architecture uses the softmax layer to map the extracted features to classification probabilities, and the cost function used for training is the cross-entropy loss. In this paper, we evaluate the influence of a number of representative composite cross-entropy loss functions on the learned feature space at the fully connected layer, when a target classification is introduced into a multi-class classification task. In addition, the accuracy and robustness of CNN models trained with different composite cross-entropy loss functions are investigated. Improved robustness is achieved by changing the loss between the input and the target classification. Preliminary experiments were conducted using ResNet-50 on the Cholec80 dataset for surgical tool recognition. Furthermore, the model trained with the proposed composite cross-entropy loss incorporating another target all-one classification demonstrates a 31% peak improvement in adversarial robustness. Adversarial training with target adversarial samples yields 80% robustness against PGD attack. This investigation shows that the careful choice of the loss function can improve the robustness of CNN models.