Automated Collection and Processing of Spatiotemporal Data for the Analysis of Sustainable Development in Industrial Systems

A software-analytical platform has been developed for the automated collection, structuring, and normalization of spatiotemporal data obtained from national statistical service web platforms. Algorithms for dynamic adaptation to changes in the DOM structure of web pages have been implemented, ensuring parsing resilience against modifications in HTML markup. Methods for bypassing anti-bot mechanisms have been introduced, including navigator.webdriver modification, User-Agent customization, and management of Chrome DevTools Protocol parameters. A data processing pipeline has been developed, incorporating semantic integrity verification, outlier removal, and transformation into a relational structure. Algorithm accuracy has been verified through manual validation of a randomly selected sample of 50 records, demonstrating a 98% match rate. The average data extraction speed reached 12.4 records per second. The results confirm the reproducibility of the methods, high adaptability to changes in web resources, and applicability in tasks related to the collection and processing of statistical data for analyzing the sustainable development of industrial systems. Future research is aimed at integrating machine learning for predictive analytics and enhancing computational efficiency in processing large-scale datasets.