Deep Learning for Patent Image Analysis

The article describes the novel method for analyzing images of dependency graphs to expand the described physical effects and consists of using deep learning models to determine the types of dependencies of a physical quantity. The software for analyzing patent images has been developed that allows extracting images from patent descriptions in Yandex.Patents and Google Patents. By means of software implementation of the developed algorithm for thematic filtering of patent images (which mainly contain drawings of devices, mathematical and chemical formulas), dataset of dependency graphs of physical quantities is formed. Testing the effectiveness of thematic filtering on a sample based on manually observed images of dependency graphs showed high AUC ROC values of 0.97, accuracy of 0.79. The process of forming a training set of dependency graph images consist 3 stages: a) based on a system request to Google Images; b) using the Matplotlib library to plot graphs of various types of dependencies; c) manually labeled more than 500 images of dependency graphs from patent documents. An algorithm for segmentation of images of dependence graphs has been developed (which allows you to get rid of noisy parts of the image, such as the coordinates of points, their designation, etc.), according to which two cases were tested: (a) using the methods of color spaces, watershed, contour extraction based on the OpenCV library and (b) using the scikit-image library. An algorithm for classifying dependence graphs into 9 classes has been developed using the LSTM, CNN, ViT neural network models.