Data Wrangling and Generation for Machine Learning Models in Medication Adherence Analytics: A practical Standpoint using Patient-Level and Medical Claims Data

Even though data wrangling (DW) accounts for more than half of the machine learning (ML) process, there is a dearth of research on data wrangling and dataset preparation. Consequently, we present a valuable state-of-the-art dataset prepared based on medical and pharmaceutical claims, as well as patient-level data collected from the transactional processing system of a Zimbabwean health insurance company. We were given two initial raw sub-datasets (one for the diabetics and one for the hypertensive patients) that had been retrieved and divided based on the International Classification of Diseases 10th Revision Code (ICD-10). Since adherence metrics measure the percentage of patients covered by prescription claims for the same medication in the same therapeutic class, within the measurement year, the created dataset only comprised data for 2022, which spanned from January 1 until December 31, 2022. We acknowledge, however, that there is no universally approved compendium of DW stages; yet certain building blocks and functionalities with considerable overlaps are commonly accepted and can be considered typical of DW. In light of this, we take a pragmatic approach to generating a dataset, incorporating nine essential DW building blocks and tasks, employed iteratively and incrementally, akin to an agile approach to dataset generation. Thus, we used some of the DW tasks iteratively and incrementally throughout the DW process without restricting them to a single stage. Following a rigorous DW procedure, the created dataset was then used to build an ML model, which then demonstrates a generally good degree of performance, as evidenced by the 81% accuracy. The outcomes, techniques, and insights proffered in this article inform future researchers, data scientists, and analysts on how to conduct data mining, dataset enrichment, and DW. The created dataset can stimulate further study in areas such as evaluating medication adherence (MA) and performing ML tasks such as classifying, predicting, and clustering MA behaviours.