Improving Transformer for Enterprise Bankruptcy Prediction Based on Swarm Intelligence Optimization Algorithm

This article explores the use of swarm intelligence optimization algorithms to optimize Transformer models and applies them to the study of enterprise bankruptcy prediction. Through experimental observation, we found that the predictive performance of the model gradually improves with the increase of iteration times. Specifically, the fitness value decreased from the initial 0.0368 to 0.0337 after 15 iterations, indicating the continuous optimization of the model during the training process. In the analysis of key factors in predicting corporate bankruptcy, retained earnings and total accounts receivable were identified as the two factors with the highest correlation with corporate bankruptcy, providing important basis for the selection of input features for the model. In terms of model performance evaluation, the prediction accuracy on the training set reached 100%, demonstrating the model's high fitting ability on known data. However, what is more important is the performance of the model on the test set, where 728 instances were correctly predicted, only 1 instance was incorrectly predicted as' bankrupt ', and 13 instances were incorrectly predicted as' alive'. These results resulted in a prediction accuracy of 98.11% on the test set, with only a difference of 1.89% compared to the training set. This indicates that the model has good generalization ability and can be effectively applied to predict unknown data. The significance of this study lies in optimizing the Transformer model through swarm intelligence optimization algorithms, which not only improves the accuracy of enterprise bankruptcy prediction, but also enhances the model's generalization ability, providing more reliable risk assessment tools for financial institutions and investors.