Secure and Efficient Big Data Collection with Differential Confidentiality and Machine Learning

In the era of Big Data, where data are continuously and exponentially generated, secure collection of massive volumes of data is essential for enabling effective analysis using advanced techniques such as machine learning. These methods facilitate extracting meaningful insights from structured, semi-structured, or unstructured data generated by websites, devices, and sensors at high velocities. Securing this data, analyzed by modern machine learning techniques (e.g., linear regression) and subjected to continuous streams, presents a significant challenge due to its volume and diverse formats. Traditional security and privacy methods prove inadequate in addressing these complex challenges. As a result, the security and efficiency issue for big data collection still deserves research. This paper proposes a secure mechanism for big data collection offering an unprecedented opportunity for predictive analysis and decision-making for improved security performance and efficiency. Differential confidentiality emerges as a promising solution for business data and confidential data collection. The collected big data is stored securely using DiffPrivLib. The discussion and performance evaluation results show the security and efficiency of the proposed secure mechanism. With the implementation of differential confidentiality on the collected data, the results of the data analysis presented an accuracy of 0.91 which shows a very important efficiency of the system to classify correctly the instances.