Identifying Hierarchical Community Structures in Content-Based Scholarly Social Networks

Community detection plays a pivotal role in social network analysis by partitioning networks into cohesive groups of vertices with dense intra-group connections and sparse inter-group connections. In this paper, we utilized a scholarly social network based on researchers' topic similarity derived from their publication metadata to identify interdisciplinary research communities. As topics often form a hierarchy, we hypothesize that the constructed scholarly network will exhibit hierarchical community structures. Therefore, we explore the efficacy of two prominent community detection algorithms, Louvain and Spectral clustering, known for their capacity to detect hierarchical community structures within networks. While both algorithms demonstrate this capability, the original Louvain algorithm is susceptible to the resolution limit problem due to its reliance on the modularity measure. To address this limitation, we propose the nested hierarchical Louvain algorithm, which iteratively partitions the network based on previously identified subgraphs, and we find that the bias towards large communities is mitigated. To evaluate the hierarchy produced by each of the algorithms, we employ the Cophenetic Correlation Coefficient (CPCC), a metric commonly used in hierarchical clustering evaluations but less frequently utilized in hierarchical community analysis. We argue that CPCC can be a useful measure to identify the presence of implicit hierarchical community structure in social networks when it is not explicitly available from domain knowledge while also further mitigating the inherent bias present in using modularity as a metric. Experimental results, conducted on both synthetic networks and the scholarly social network, demonstrate that the nested hierarchical Louvain algorithm, as well as Spectral Clustering, successfully identifies more finely structured hierarchical communities, offering greater depth in the dendrogram compared to the basic Louvain algorithm.