iSIM-Sigma: Efficient Standard Deviation Calculation for Molecular Similarity

The average and variance of the molecular similarities in a set are of high value and useful for cheminformatics tasks such as chemical space exploration and subset selection. However, the calculation of the variance of the complete similarity matrix has a quadratic complexity, <i>O</i>(<i>N</i>²). As the sizes of molecular libraries constantly increase, this pairwise approach is unfeasible. In this work, we present an approach to calculate the exact standard deviation of molecular similarities in a set (with <i>N</i> molecules and <i>M</i> features) for the Russell-Rao (RR) and Sokal-Michener (SM) similarity indexes in <i>O</i>(<i>NM</i>²) complexity. Furthermore, we present a highly accurate linear complexity approximation, <i>O</i>(<i>N</i>), based on sampling representative molecules from the set. The proposed approximation can be extended to other similarity indices, including the popular Jaccard-Tanimoto (JT). With only the sampling of 50 molecules, the proposed method can estimate the standard deviation of similarities in a set with an RMSE lower than 0.01 for sets of up to 50,000 molecules. In comparison, random sampling does not warrant a good approximation with the same number of selected molecules as shown in our results.