Best Practices for Applying and Interpreting the Total Operating Characteristic

The Total Operating Characteristic (TOC) is an improvement on the quantitative method called the Relative Operating Characteristic (ROC), both of which plot the association between a binary variable and a rank variable. TOC curves reveal the sizes of the four entries in the confusion matrix at each threshold, which make TOC curves more easily interpretable than ROC curves. The TOC has become popular, especially to assess the fit of simulation models to predict land change. However, the literature has shown variation in how authors apply and interpret the TOC, creating some misleading conclusions. Our manuscript lists best practices when applying and interpreting the TOC to help scientists learn from TOC curves. An example illustrates these practices by applying the TOC to measure the ability to predict the gain of crop in Western Bahia, Brazil. The application compares four ways to design the rank variable based on the distance to either pixels or patches of either the presence or change of crop. The results show that the gain of crop during the validation time interval is more strongly associated with the distance to patches rather than pixels of crop. The Discussion Section reveals that if authors show the TOC curves, then readers can interpret the results in ways that the authors might have missed. The Conclusion encourages scientists to follow best practices to learn the wealth of information that the TOC reveals.