Controlling Neural Network Training via Modification of Cross-Entropy Loss Function

There have been various loss functions proposed to improve the training of neural networks with sigmoid activation output nodes. For neural networks with softmax activation output nodes, the cross-entropy loss function is commonly used for training and several attempts have been made to improve the performance of such networks by modifying the standard cross-entropy loss. However, rather than simply aiming to improve overall classification accuracy, it is often necessary to address misclassification costs differently depending on their real-world importance. In practice, the cost of errors can vary greatly across domains such as finance, security, insurance, and healthcare. From this perspective, this paper proposes a modified cross-entropy loss function designed to control the training of neural networks with softmax output nodes. The effectiveness of the proposed loss function is demonstrated through simulations on the CEDAR handwritten digit recognition task, showing that classification performance can be adjusted according to the order of the modified loss function. This approach will serve as the basis for designing a novel cost-sensitive learning method tailored to neural networks with softmax outputs.