Explainable AI for Online Transaction Fraud Detection Using Machine Learning and PySpark

As digital transactions grow exponentially, so do concerns about online financial fraud, which continues to threaten both consumers and financial institutions. A highly scalable, data-lean approach of the machine-learning solution for fraud detection against online attacks is proposed herein. For this purpose, this paper adopts a diverse set of financial transaction data. Using Logistic Regression, Decision Tree, KNN, and Random Forest to classify whether a transaction was legitimate or fraudulent, Explainable AI was used to enable transparency to the features causing these fraud classifications. Compared to traditional Scikit-Learn, the PySpark-based model achieved 168% higher recall and 38% faster training time. The new paradigm approach utilizes PySpark to solve data in scalable ways, while explaining this complexity so that a real-time adaptation can be made.