METHODS OF PROCESSING AND ANALYZING BIG DATA IN MACHINE LEARNING TASKS: APPROACHES AND PROSPECTS

This article explores the methods of processing and analyzing big data in order to improve the accuracy and efficiency of machine learning (MO) models. The main focus is on classification problems, the effectiveness of algorithms such as XGBoost, the support vector machine (SVM), ensemble methods, as well as systems for working with big data, including Hadoop and Apache Spark. The key stages of working with data are described: cleaning, normalization, selection of features, which is critically important for building stable models on large amounts of data. Accuracy, completeness, F-measure, and AUC-ROC metrics were used to evaluate the effectiveness of the algorithms, which made it possible to conduct a comparative analysis and identify the most productive approaches. Special attention is paid to the application of MO in the context of organizational innovations, including the tasks of classification, forecasting the success of innovations and innovation portfolio management. Recommendations on the choice of technologies and algorithms for various data types and scales are presented, and prospects for integrating distributed computing platforms with MO algorithms to achieve scalable and efficient solutions are discussed.