Full-text Russian-language search in a distributed repository of patent information

A significant increase in the number of patent publications in recent years has created difficulties in conducting classical manual analysis and searching for patent analogues. Automation of the search for patent analogues is a key tool for reducing time and financial costs at the stages of patent application formation and patent examination. The use of Big Data technologies and distributed systems makes it possible to build an effective architecture of a system of patent analogues and improve the quality of patent search results. The theoretical significance of the work lies in the development of the architecture and concept of a full-text patent search system based on a comparative analysis of the effectiveness of various distributed systems for searching and processing textual Russian-language information, taking into account its morphological and syntactic features. The practical significance of the work lies in the implemented software, which includes tools for parsing patent documents into a distributed file system, searching taking into account the features of the natural Russian language, as well as a web interface for visualizing search results. Modern frameworks and technologies are used in the process of work: Apache Hadoop, Spark, Hive, Elasticsearch, PostgreSQL, ClickHouse. Elasticsearch showed the best results in both response time and search quality (accuracy – 0.87, completeness – 0.82, F-measure – 0.84) for complex queries reflecting the specifics of the Russian language.