Multimodal technology for searching and clustering weakly structured text-graphic documents

The problem of searching and clustering poorly structured text-graphic documents (TGD) using neural network technologies is addressed. A multimodal approach to building a system for searching and analyzing TGD is proposed, utilizing a hybrid criterion to compare text and graphic fragments within the analyzed documents. Integrating processes for identifying text-graphic fragments in electronic documents based on image characteristics and text signatures (keywords) is described. A method for clustering and indexing TGD is introduced, leveraging the SOINN algorithm, linguistic descriptors, and a modular file array processing system. This method enables the formation of a dynamic cluster structure for TGD and the creation of unified indexes. The testing results and practical recommendations for implementing neural network models in TGD search and clustering are presented.