Distributed Transformer Framework for Financial Anomaly Detection

Financial fraud is a significant concern for investors and financial institutions, leading to substantial economic losses. Traditional fraud detection techniques often struggle with challenges such as low accuracy, slow processing times, and limited adaptability across different financial sectors. To address these issues, this paper introduces a distributed knowledge distillation framework utilizing Transformer models. The approach employs a multi-attention mechanism to highlight important features, followed by a feed-forward neural network for extracting high-level representations. A final neural network classifier then determines fraudulent activity. Additionally, to tackle inconsistencies in financial data and imbalanced distributions across industries, a distributed knowledge distillation algorithm is proposed. Financial fraud cases causing serious damage to the interests of investors are not uncommon. As a result, a wide range of intelligent detection techniques are put forth to support financial institutions’ decision-making. Currently, existing methods have problems such as poor detection accuracy, slow inference speed, and weak generalization ability. Therefore, we suggest a distributed knowledge distillation architecture for financial fraud detection based on Transformer. Firstly, the multi-attention mechanism is used to give weights to the features, followed by feed-forward neural networks to extract high-level features that include relevant information, and finally neural networks are used to categorize financial fraud. Secondly, for the problem of inconsistent financial data indicators and unbalanced data distribution focused on different industries, a distributed knowledge distillation algorithm is proposed. This algorithm combines the detection knowledge of the multi-teacher network and migrates the knowledge to the student network, which detects the financial data of different industries. This method integrates insights from multiple teacher models and transfers their knowledge to a student network, enhancing fraud detection capabilities across diverse industries. Experimental evaluations demonstrate that the proposed approach surpasses traditional methods, achieving an F1 score of 92.87%, accuracy of 98.98%, precision of 81.48%, recall of 95.45%, and an AUC score of 96.73%.