Swarm intelligence in handling of big data

Abstract Recently, the growing demand for handling the analysis process of huge data, which involves vast volumes of data with speed, has garnered increasing interest. The quality and scale of the data significantly influence proficiency in problem-solving. This study area denotes a recent addition to the spectrum of information-processing techniques research. This area faces significant difficulties attributed to the enormity of data, its multidimensional complexity, and the ongoing changes in data patterns. However, there is a potential for addressing these challenges with support from related research areas, like swarm-based intelligence (SI), encompassing nature-inspired search methodologies. This paper focuses on examining how swarm intelligence algorithms are related to big data analytics with the basic aim of analyzing the usefulness and significance of SI in the analytics of big data. A thorough examination of the correlations and associations between swarm intelligence approaches and their efficacy in evaluating huge quantities of data is presented. The study of data mining and analytics here includes the classification and clustering of large data-sets proposed in our previous works published in (Banchhor and Srinivasu in Data & Knowledge Engineering 127:101788, 2020 ) and (Kulkarni and Jena in Int Rev Comput Softw (IRECOS) 11:997, 2016) with a detailed analysis of existing approaches for satisfactory coverage. The classification framework (Banchhor and Srinivasu in Data & Knowledge Engineering 127:101788, 2020 ) has a naïve Bayes classifier along with gray wolf Optimization and the clustering framework is based on Particle Swarm Optimization (PSO) (Kulkarni and Jena in Int Rev Comput Softw (IRECOS) 11:997, 2016). Swarm-based optimization and the plain analytical process for big data are compared through three key performance metrics: accuracy, sensitivity, and specificity. Analysis shows that applying swarm intelligence to big data handling, using localization and skin segmentation datasets, leads to significant performance improvements. This performance gain is a result of applying swarm-based intelligence within classification and clustering approaches, measured by accuracy, sensitivity, and specificity.