Pre-Determining the Optimal Number of Clusters for k-Means Clustering Using the Parameters Package in R and Distance Metrics

A drawback of the popular k-means clustering algorithm is that the number of clusters must be specified in advance, which can be challenging for users who may not have extensive experience with the methods involved (such as Elbow, Silhouette, and Hartigan, among others). To address this issue, we employed the n_clusters() function from the parameters package in R, which applies 27 clustering methods to determine the optimal number of clusters more easily. In addition, the Canberra, Euclidean, and Manhattan distance metrics were employed to identify similar data objects in the k-means clusters. These metrics were measured and compared in terms of accuracy, F1-score, precision, and recall via numerical studies employing 10 simulated and two real datasets. The results indicate that the n_clusters() function from the parameters package could accurately identify the correct number of clusters in all of the simulated and real datasets. In particular, the Hartigan and Tracew methods identified the correct number of clusters with an accuracy of 100%. These findings suggest that using the Hartigan or Tracew method can accurately pre-determine the correct number of clusters for the k-means clustering algorithm. Also, the findings show that the performances in terms of accuracy, F1-score, precision, and recall of both the Euclidean and Manhattan distance metrics were excellent, with both of them consistently outperforming the Canberra distance across all datasets.