Financial fraud prediction leveraging knowledge graphs and multimodal features

Aiming at the problem that existing financial fraud prediction methods have single feature extraction and do not fully utilise multimodal financial features, this paper first constructs a knowledge graph based on multimodal financial data, and uses BERT and ResNet to extract features from text entities and image entities respectively. A bidirectional cross-modal attention mechanism is designed, and a bidirectional text and image element matching method is proposed. The consistency of all element pairs is comprehensively used to represent the global consistency features of text and images. Finally, a hybrid fusion method is designed to fuse text features, image features, and text-image consistency features. The final prediction result is obtained through a fully connected layer. Experimental results show that the proposed model has a prediction accuracy of 92.4%, which is better than the comparative methods and can effectively improve the accuracy of financial fraud prediction.