Construction of hierarchical vector representations of OKVED codes using fully connected adapters

Industry classification is a crucial characteristic of a company, influencing its business environment, investment risks, and other key operational aspects. In Russia, the OKVED codes are used to describe industry affiliation, but they possess a hierarchical structure and an uneven distribution, which makes their representation challenging. Traditional machine learning methods often struggle to analyze and predict outcomes based on these hierarchical attributes. By leveraging modern neural network techniques, it is possible to construct compact vector representations of OKVED codes that capture their hierarchical nature and serve as informative features for economic analysis tasks. The proposed approach creates a vector for an industry code in an iterative manner, in which vector representations for increasingly detailed levels of the hierarchy are developed. At each step of the process, a new vector corresponding to the next level of depth is combined with the vector from the previous level. This iterative layering significantly enhances the expressiveness of the vectors, particularly for underrepresented codes. To facilitate the transition between hierarchical levels, fully connected adapter layers are introduced. Additionally, the study explores the integration of attention mechanisms during the transition between levels. This allows the model to dynamically focus on the most meaningful representations of the levels, thereby theoretically improving the quality of the generated vectors. The effectiveness of the obtained embeddings is demonstrated by integrating the developed representations into a default prediction model and observing an improvement in the model's performance. Experimental results indicate a notable improvement in the Gini score when the embeddings are utilized, compared to the model that does not incorporate industry affiliation information at all, and to models that employ simpler embedding methods such as one-hot encoding or plain lookup tables. We also compared the proposed hierarchical representations with embeddings obtained by concatenating independent vectors of different levels of the classifier and demonstrated that our method surpasses such approaches as well. The results of this research can be applied to various economic tasks, including financial risk assessment, market segmentation, and industry analysis. The proposed models and methods for generating OKVED code embeddings can be validated using publicly available datasets and extended to other applied scenarios. By incorporating hierarchical structures into vector representations, this approach enhances the interpretability and usability of industry classification data in machine learning applications.