Scalability and Efficiency of Deep Learning Models on High-Performance Computing Clusters: Bibliometric Analysis

Purpose: This systematic review identifies the main advancements, core papers, voids, and outlooks regarding utilising high-performance computing (HPC) clusters for large-scale deep learning models. Moreover, it aims to assess research trends, significant approaches, and methods to improve the effectiveness and adaptability of these models. Design/Methodology/Approach: This paper involves a systematic literature review and bibliometric analysis of published articles from 2018-2023. To limit the results, some specific keywords have been introduced: Web of Science, deep learning efficiency, scalability, and HPC clusters. Studies were screened according to the PRISMA flowchart, covering 364 articles, 19 of which were included in the systematic review based on further criteria. Findings: The bibliometric analysis revealed that the most globally cited articles were from IEEE Transactions on Emerging Topics in Computing and Joule. However, the most relevant sources identified were the Journal of Supercomputing, Concurrency and Computation: Practice and Experience, and IEEE Access. Researchers from the USA, China, Korea, and the UK authored the most significant contributions. Research Limitation: The study examined only works from the Web of Science database from 2018 to 2023. Practical Implication: The proposed research results contribute crucial information to enhance the effectiveness of deep predictive models in large-scale HPC environments, which are essential for enterprises adopting artificial intelligence (AI) and machine learning (ML) methodologies in colossal data analysis applications. Social Implication: Propelling deep learning models with the help of HPC clusters can create more vital AI solutions that can respond to society's needs. Originality/ Value: The novelty of the research stems from the bibliometric assessment and the question of which sources and authors in this field are the most important.