Beyond the Hype: A Critical Evaluation of Predictive Analytics Accuracy in High-Stakes Business Decisions

The dawning digital era has brought expectations of massive improvements in the accuracy and value of corporate decision-making relating to strategic direction, business execution, performance management, and workforce management. With the advent of analytics, people-related and other data from disparate systems within a company are being used to predict important organizational outcomes or events such as customer satisfaction, future workforce attrition, expense overruns, project completion problems, etc. This predictive process is typically referred to as advanced, predictive, or prescriptive analytics. In this study, we take a critical look at the current organizational practice of predictive analytics using employee data residing in organizational HR systems. Focusing on high-stakes business outcomes, we consider the data and use of predictive models, and we review what HR professionals are telling their CEOs. We also look to ensure that the self-reported accuracy ratings are realistic and that the business benefits derived from these predictions or recommendations are indeed realized. Addressing this question across six large corporations in India using more than 10 regularly used applications, the results of this study are based on data from 816 HR professionals and 333 business leaders. All use predictive scores from related applications or in their decision-making roles. We find that there are a number of serious model accuracy problems with the applications used. We also find that both HR professionals and business leaders in India suffer from significant accuracy overestimation.