AutoCross: Automatic Feature Interaction Learning with Self-Attention Neural Networks Integrating Feature Graph Information

Credit default prediction is a crucial task for forecasting whether borrowers will default, providing accurate and efficient decision-making information for bank credit departments. However, this task is challenging for two main reasons. First, the input features are often sparse and high-dimensional, requiring effective methods to extract meaningful information from them. Second, the high-order combinatorial features on which default prediction depends are typically handcrafted by experts, a process that is time-consuming, labor-intensive, and difficult to enumerate all possible feature combinations. Therefore, automatically learning high-order combinatorial features can reduce the cost of feature engineering and improve the predictive performance of the model. In this study, we propose a module called Feature Relationship Fusion Attention (FRFA), which learns the relationships between features and integrates these relationships into the attention calculation to enhance the model's ability to capture high-order combinatorial features. Based on the proposed FRFA module, we introduce a model specifically designed for credit default prediction, called AutoCross, which efficiently learns high-order combinatorial features automatically by integrating the node position information and node importance information from the fused feature relationship graph into the attention mechanism. In summary, this model fits large-scale data through an end-to-end approach, leveraging the AutoCross model to automatically learn high-order feature interactions, thereby improving the accuracy and reliability of credit default prediction.