Розроблення підходу для аналізу та автоматичної перебудови структури вебсайтів

This study explores websites such as online stores, which are considered to be a set of interconnected web pages. The task addressed relates to the high computational complexity of manual analysis of the topology of modern websites, as well as the lack of formalized mechanisms that could make it possible to integrate the semantic features of web pages into the process of automated hyperlink reconstruction. Within the framework of this study, a website is crawled in order to obtain complete HTML documents, from which the structural features of pages are extracted (the number of headings, depth of embedding, presence of <article>, number of incoming links, etc.). The resulting vectors make it possible to construct cosine similarity matrices to assess the mutual proximity of pages. An approach has been proposed to rebuilding the link structure of the website taking into account this similarity; a comparison of the initial and transformed website was carried out using the metric characteristics of modularity, clustering, diameter, and similarity distribution. The results demonstrate that taking into account the DOM structure allows for the formation of a logical, reasonable distribution of pages between clusters. And the subsequent automatic procedure for setting hyperlinks makes it possible to improve structural integrity by establishing effective relationships between thematically close pages. The practical significance of this work involves the possibility of using the proposed approach for automated optimization of internal links of static websites. As a result, the architecture of the web resource is improved, website navigation becomes transparent, and website indexing by search engines is increased