IMPROVED INTEROPERABILITY WITH TRANSFORMER-BASED PANEL STRUCTURES (VASWANI ET AL)

This article proposes a rethinking of the Transformer architecture in the context of the formation of a convenient characteristic space. The application of this model in this case is universal and can be extended to any data with a panel structure. According to the results of model training, vector images (embeddings) of related objects described by panel data are located closer to each other in the new space, while observations corresponding to other unrelated objects are located further away. The use of such an innovation is aimed at improving the convenience of interaction with panel structures at all stages of the study, since often, due to the complexity of the initial distribution of information, laborious pre-processing is required before fitting the selected model. Moreover, the whole procedure for creating a new feature space implies simplifying the task associated with identifying hidden relationships within the initial data for the subsequent application of statistical learning algorithms.