Improving the aggregation and evaluation of NBA mock drafts

Abstract If professional teams can accurately predict the order of their league’s draft, they would have a competitive advantage when using or trading their draft picks. Many experts and enthusiasts publish forecasts of the order players are drafted into professional sports leagues, known as mock drafts. Using a novel dataset of mock drafts for the National Basketball Association (NBA), we explore mock drafts’ ability to forecast the actual draft. We analyze authors’ mock draft accuracy over time and ask how we can reasonably aggregate information from multiple authors. For both tasks, mock drafts are usually analyzed as ranked lists, and in this paper, we propose ways to improve on these methods. We propose that rank-biased distance is the appropriate error metric for measuring accuracy of mock drafts as ranked lists. To best combine information from multiple mock drafts into a single consensus mock draft, we also propose a combination method based on the ideas of ranked-choice voting. We show that this method provides improved forecasts over the standard Borda count combination method used for most similar analyses in sports, and that either combination method provides a more accurate forecast across seasons than any single author.