TabTransformer-Based Credit Default Prediction for Structured Economic Data

In this paper, we investigate the application of Transformer-based deep learning models to structured tabular data, with a focus on credit default prediction. Traditional machine learning methods often struggle to capture complex feature interactions and require extensive feature engineering when dealing with heterogeneous categorical and numerical variables. To address this challenge, we adopt the TabTransformer architecture, which combines column embeddings and self-attention mechanisms to enable end-to-end representation learning on mixed-type economic data. Extensive experiments on a benchmark credit dataset demonstrate that TabTransformer outperforms baseline models—including logistic regression, random forests, and multilayer perceptrons—in terms of classification performance. In addition to predictive accuracy, we integrate SHAP-based interpretability, categorical embedding visualization, and attention heatmaps to provide transparent insights into the model's decision-making process. Our findings confirm the efficacy of deep Transformer models in structured data modeling and reinforce their potential for deployment in real-world financial risk assessment systems.