Enhancing Healthcare Services through Machine Learning-Driven Anomaly Detection

The healthcare industry’s expanding data presents opportunities for artificial intelligence (AI) and machine learning (ML). Despite advancements in medical technology, medical errors remain a significant issue. Anomaly detection, identifying data points that deviate from normal patterns, is crucial for addressing these errors and detecting healthcare fraud. This study uses the “Healthcare Providers Data for Anomaly Detection” dataset to identify fraudulent activities. Clustering and two-step clustering, two unsupervised anomaly detection methods, are employed to uncover anomalies without prior knowledge. These techniques are applied to detect unusual events, such as anomalous patient health states and time-series data deviations. The study focuses on building a robust framework for anomaly detection by preprocessing data, selecting key features, and training models. Principal Component Analysis (PCA) and Lasso regression are used for feature selection in two-step clustering and K-means clustering, respectively. Models are evaluated using metrics like F1 score, precision, and recall. Results indicate that two-step clustering achieves higher accuracy, while K-means excels in efficiency. Both methods are effective for detecting time-series anomalies. Optimal feature selection further enhances model performance. This research highlights the significance of using clustering-based unsupervised learning techniques in anomaly detection. By leveraging these methodologies, healthcare fraud and other data anomalies can be effectively identified, improving anomaly detection systems across diverse data types.