Data Engineering and ML for Real-Time Fraud Detection in Financial Transactions

Fraud detection remains a critical challenge for financial institutions, especially in high-frequency transaction environments. This study proposes a real-time, AI-driven fraud detection system utilizing XGBoost and optimized data engineering to process up to 10,000 transactions per second with minimal latency of 7ms. The system addresses the issue of data imbalance in fraudulent transactions while ensuring transparency through Explainable AI (XAI), utilizing SHAP values to make model predictions interpretable. Experimental results on real-world financial transaction datasets show that the proposed system achieves 99.2% accuracy, with precision and recall scores exceeding 95% and 93%, respectively. When compared to baseline models and existing literature. our approach demonstrates superior scalability, speed, and accuracy. This solution offers a robust and efficient framework for real-time fraud detection, ensuring both performance and transparency, while maintaining low prediction latency, making it ideal for modern financial services.