A Quantitative Framework for Evaluating the Performance of Algorithm-Directed Whole-Population Remote Patient Monitoring: Tutorial for Type 1 Diabetes Care

Clinics continue to adopt care models shaped by the algorithmic analysis of continuous glucose monitoring (CGM) data, such as remote patient monitoring for type 1 diabetes. No clinic-facing quantitative framework currently exists to track the impact of such algorithm-directed care on patient outcomes and clinical workload. We used CGM data from the Teamwork, Targets, Technology, and Tight Control (4T) Study (Pilot n=135 and Study 1 n=133), in which algorithms enable precision, whole-population care by directing clinician attention to patients with deteriorating glucose management. Youth meeting criteria for clinical review are then contacted by Certified Diabetes Care and Education Specialists. Through iterative data analysis and meetings with a variety of stakeholders, we identified metrics for reviewing and revising clinical workloads, glucose management, and timeliness of care. For each metric, we developed an interactive dashboard to provide clinical and administrative leaders with an overview of the program. The metrics to track clinical workload were the total number of youths (1) in the program, (2) in each study, and (3) cared for by each clinician. The metrics to track glucose management were the number of youths meeting each criterion for review, including (4) total, (5) for each clinician, and (6) for each study. The metric to track timeliness of care was (7) the number of days since meeting criteria for clinical review. When presented at regular program leadership meetings, the metrics facilitated data-driven decision-making about clinical and operational components of the program. In this paper, we describe the process of developing and operationalizing this reproducible, clinician-facing key performance indicator tool to monitor an algorithm-enabled remote patient monitoring program. As the role of algorithms grows in directing clinical effort and prioritizing patients for care, this framework may help clinics track clinical workload, patient outcomes, and the timeliness of care.