THE STUDY OF OPTIMIZATION OF THE SEGMENTATION PROCESS FOR BUILDING DISTRIBUTED GENERALIZED SUFFIX TREES

Suffix trees are important data structures used for substring search and analysis of large text sequences, butbuilding them on large data volumes remains a challenging task. Existing methods for constructingdistributed generalised suffix trees (GSTDs) have significant limitations in terms of segmentation efficiencyunder resource constraints. Previous studies have focused on static memory allocation strategies that do notprovide an optimal balance between performance and resource consumption. Therefore, the aim of thestudy was to develop new segmentation methods for efficient data distribution between nodes of distributedsystems. The methods with the following algorithms were compared for this purpose: hybrid division withload balancing, fractal division and dynamic adaptive division, as well as their integration with the MasterWorker architecture. The research employed the methods of modelling, experimental method, comparativemethod, and statistical analysis. The results showed that Master-Worker integration significantly improvedperformance, including reduced execution time, improved speedup and efficiency, and improved segmentmerging accuracy. The highest indicators were achieved when applying dynamic adaptive division, whichturned out to be the most effective for data sets of different structures. Experimental results demonstrate a30% reduction in computational costs compared to traditional methods. Thus, this study makes a significantcontribution to improving algorithms for building distributed suffix trees and their application in big dataanalysis. Further research may focus on optimizing segmentation methods using machine learning (ML) toautomatically select a strategy based on the data type. Besides, it is promising to extend the proposedalgorithms to other types of data, such as multimedia streams and graphs.