Advancing enterprise credit rating through multi-head self-attention and transfer learning

This study introduces the multi-head self-attention rotary-positional encoding Informer (MHSA-RPEI), a novel enterprise credit rating model that leverages self-correlation weights and positional information of financial indicators to enhance rating accuracy. Corporate credit rating presents challenges such as temporal dynamics, complex non-linear relationships, and cross-regional variations, which traditional and deep learning models struggle to address. The MHSA-RPEI model overcomes these limitations through four innovations: (1) objective MHSA-RPEI model, which is specifically designed for credit rating, achieving high precision using only financial data and serving as an objective and validated tool; (2) revolutionary rotary positional encoding (RoPE), which enhances temporal modeling, improving dynamic risk assessment; (3) strategic transfer learning (TL), which facilitates cross-regional adaptations, mitigating data scarcity and domain variations; and (4) optimized informer architecture, which improves long-term risk forecasting based on extensive financial time series. Experiments on the U.S., Chinese, and Japanese financial data showed that MHSA-RPEI achieved 92.16% accuracy (U.S.), 90.32% (China), and 92.50% (Japan), with area under the receiver operating characteristic curve scores ranging from 88.47% to 91.95%, surpassing traditional and state-of-the-art models. Furthermore, TL experiments highlighted the exceptional adaptability of the MHSA-RPEI for cross-regional applications, yielding performance gains of 5–6%. Ablation studies confirmed that the RoPE and self-correlation weight modules were crucial to the effectiveness of the model.