Oriented data-generating processes: a categorization of ROC curves

Decision makers attempting to classify a binary state of the world may commit two types of errors. Even when the two alternative states have equal prior probabilities and when the two types of errors are equally costly, a classification criterion may be chosen which leads to one type of error being committed more frequently than the other, because of asymmetries in the data that informs their decisions. We formalize this possibility through a categorization of data-generating processes (DGPs), which may be ‘oriented’ towards evidence favoring one of the two alternatives, or which may be ‘unoriented’. We identify the shape properties of the receiver operating characteristic (ROC) curves associated with DGPs in these three categories. We also identify the orientation of DGPs obtained from common distribution families. Then, we illustrate the usefulness of our categorization with several applications, e.g., the standard decision making problem, ranking intersecting ROC curves for particular classes of decision makers, interpreting Bayesian persuasion strategies, and burden of proof assignments in simple litigation settings.