Different Performance of Transformer and Logistic Regression Models in Credit Risk Prediction

Nowadays, people’s use of credit cards is increasing significantly, and the importance of predicting credit default for banks has become higher. This paper aims to compare the different performances of two binary classification methods--logistic regression and transformer--in credit risk performance to determine which one bank should use wider when making loans. In the experiment, we selected 600,000 pieces of data randomly and then applied them in both logistic regression and transformer models. As a result, we evaluated the performance of each model in various aspects, and we eventually found that logistic regression is more accurate than transformer. So we can conclude that although there are diverse novel credit scoring models appearing in the world, logistic regression is still one of the most practical, useful and precise ones, which not only saves time but also performs well. However, in the future, if the data features get more complicated, people might discover more uses of transformers in the economic field, especially in credit risk prediction.