A machine learning framework for interdisciplinary research recommendation via researcher clustering

Interdisciplinary research (IDR) plays a pivotal role in addressing complex global challenges, yet forming productive cross-disciplinary teams remains a persistent obstacle. This study proposes a reproducible machine-learning framework to identify potential interdisciplinary collaborators by clustering researchers using semantic features from abstracts, author keywords, and reference lists. We assembled a Scopus corpus (2019–2023) of 9,160 publications (16,110 authors), preprocessed text with standard tokenization and stop-wording, and derived topic/semantic vectors via LDA and LSA (optimal k = 11). We compared three feature configurations—(A) abstracts, (B) abstracts+keywords, (C) abstracts+keywords+references—and two clustering algorithms: K-means (k via elbow) and DBSCAN (ε via k-distance). Evaluation combined internal validity (Silhouette = 0.559 reported for best configuration) with domain-expert assessment (expert agreement = 29%) and robustness checks. Results show systematic improvement in cluster cohesion as feature sets were enriched: K-means on the fused three-feature representation produced 11 coherent clusters, while DBSCAN revealed numerous fine-grained niche communities aligned with emerging interdisciplinary themes. The fused feature approach yields practical gains for institutional researcher profiling, collaboration recommendation, HR planning, and policy design. Code and processed data will be made available to support reproducibility; limitations and future testing with contextual embeddings (e.g., SPECTER family) are discussed.