A hitchhiker’s guide to information theoretical measures in psychology

In psychology, as in other sciences, information theory can be used as a tool to complement more standard regression-based methods of data analysis. It is important to see the potential of information theoretical measures as statistical tools without implying a connection to their origins in communication theory and engineering. The use of these measures may provide us with additional insights due to their sensitivity to non-linear relationships, their flexibility to the mixing of data types, and their more straightforward generalization towards investigating higher-order interactions. We briefly reintroduce information theory and compare several measures such as mutual information and co-information with correlation and regression-based methods for the investigation of variable dependence. • Information theory and psychology have a rich history • Information theoretical measures can be disconnected from information theory • These measures complement variance-based measures of variability and association • They are more general with respect to interpretation and possible data types • There are many extensions towards the investigation of higher-order interactions.