Overcoming data sparsity in eXtreme Multi-Label classification using word embedding-based models

The transition from the analog to the digital world generates a massive volume of data every day. Millions of user’s upload, download, and search data on and from social media, Wikipedia, YouTube, Amazon, etc. It is critical to analyze and retrieve meaningful information from millions of users with millions of categories/labels. Hence, it is a challenging task to build a multi-label classifier that can classify a large volume of data into its relevant categories/labels. Large-scale Recommendation, Tagging, and Categorization applications typically operate in high-dimensional feature spaces with millions of labels. Due to the high dimensionality of the data, these applications suffer from data scalability issues, label correlations, and computational time constraints. Traditional Multi-Label Classification approaches have failed to handle the high dimensionality of the input space in such applications. The eXtreme Multi-Label Classification (XMLC) is the adaptation of Multi-Label Classification for high-dimensional input space. Word embedding techniques play a crucial role in enhancing XMLC’s performance by capturing semantic relationships between features and labels. This research focuses on integrating word embeddings with deep learning models to improve classification performance in high-dimensional input spaces. The proposed “Word embedding based Deep eXtreme Multi-Label Classifier (WDXMLC)” approach leverages word embeddings to effectively handle data sparsity and capture feature–label correlations in XMLC problems. Experiments are performed on Eurlex-4 K, Wiki10-31 K, and AmazonCat-13 K high-dimensional data sets. These experiments conclude that word embedding-based models play an important role in enhancing the performance of the Extreme Multi-Label Classifier. The proposed WDXMLC approach achieves relatively 9% ~ 12% improvement compared to FastText, BOW-CNN, and CNN-Kim for AmazonCat-13 K sparse data set using precision@5 and nDCG@5 measures.