Financial Fraud Detection for Credit Card Transactions Using Apache Spark

The financial sector's rapid adoption of digital transformation has made identifying fraud a progressively complex challenge, since modern fraudulent activities present significant risks to institutions globally. This paper presents a novel methodology for detecting financial fraud that integrates big data analytics and machine learning techniques within a distributed environment. With an extensive number of actual financial transactions, the system uses Apache Spark for real-time data processing and machine learning algorithms to accurately classify fraudulent transactions. This method effectively addresses two major issues: efficiently processing large data sets and managing the severe class imbalance between fraudulent and non-fraudulent transactions. Collectively, these solutions provide a scalable and resilient framework for improving fraud detection in the digital era.