Research on the Characteristics of Joint Distribution Based on Minimum Entropy

This paper focuses on the extreme-value issue of Shannon entropy for joint distributions with specified marginals, a subject of growing interest. It introduces a theorem showing that the coupling with minimal entropy must be essentially order-preserving, whereas the coupling with maximal entropy aligns with independence. This means that the minimum-entropy coupling in a two-dimensional system forms an upper triangular discrete joint distribution by exchanging the rows and columns of the joint distribution matrix. Consequently, entropy is interpreted as a measure of system disorder. This manuscript’s key academic contribution is in clarifying the physical meaning behind optimal-entropy coupling, where a special ordinal relationship is pinpointed and methodically outlined. Furthermore, it offers a computational approach for order-preserving coupling as a practical illustration.