STREAM PROCESSING ALGORITHMS FOR UNSTRUCTURED DATA ANALYSIS

Relevance of the selected researchThe rapid growth of unstructured streaming data and the need for real-time decision-making necessitate thedevelopment of effective algorithms tailored for dynamic processing, semantic enrichment, and adaptiveanalytics.PurposeThe purpose of the study is to develop an optimized mathematical model for the processing of unstructureddata streams to support real-time decisions.MethodsThe study used the following methods: decomposition analysis, mathematical formalism, optimizationmodeling, reinforcement learning, simulation modeling.The research employed a diverse array ofmethodologies, including decomposition analysis, mathematical formalism, optimization modeling,reinforcement learning, and simulation modeling.Results obtainedThis study presents an optimized algorithm for real-time processing of unstructured data streams, derivedfrom the decomposition of tested models and the formalization of a generalized mathematical framework.By integrating Kafka with Zero-Copy I/O, CEEMDAN-based preprocessing, ONNX-inference models,knowledge graph enrichment, and QoS-driven orchestration (Kubernetes, KEDA), the proposed solutionachieved a 3.8× reduction in Latency (from 250.0 to 65.0 ms), increased Accuracy to 91.3%, and doubledThroughput (1100 to 2200 events/sec). Additionally, CPU Load was reduced by 27%, Memory Usage by45%, while the Adaptability Score and Semantic Alignment Score improved from 0.52 to 0.88 and from0.64 to 0.91, respectively. These results confirm the algorithm’s efficiency, scalability, and applicability inintelligent Decision Support Systems under real-time constraints.Scientific novelty of the studyThe scientific novelty of the research lies in the development of a mathematical model for streamingprocessing, characterized by dynamic ingesting, semantic enrichment, and optimization throughreinforcement learning, all aimed at enhancing real-time productivity. Prospects for future researchFurther research involves validating the optimized algorithm within high-entropy streams, the scaling tomultimodal datasets, and the enhancement through adaptive retraining and dynamic orchestration.