A New Index for Quantifying the Statistical Difference Between Two Probability Distributions

In many scientific fields (e.g., statistics, data science, machine learning, and image processing), effectively quantifying the statistical difference between two probability distributions is an important task. Although a wide variety of measures have been proposed in the literature, some of them (such as the chi-square divergence and the Kullback–Leibler divergence) do not satisfy one or both of two key axioms: normalization and symmetry. This paper proposes a new index for quantifying the statistical difference between two probability distributions, called the distribution discrepancy index (DDI). The proposed DDI is based on the recently developed concepts of informity and cross-informity in informity theory. Its value ranges from 0 to 1, with values close to 1 indicating a large discrepancy and values close to 0 indicating minimal discrepancy. The DDI satisfies the two key axioms and is applicable to both discrete and continuous distributions. This paper also proposes the distribution similarity index (DSI) as a complement to the DDI. Three examples are presented to compare the DDI with three existing discrepancy measures (the Hellinger distance, total variation distance, and Jensen–Shannon divergence) and the DSI with two existing similarity measures (the Bhattacharyya coefficient and overlapping index).