Bag-of-Word approach is not dead: A performance analysis on a myriad of text classification challenges

The Bag-of-Words (BoW) representation, enhanced with a classifier, was a pioneering approach to solving text classification problems. However, with the advent of transformers and, in general, deep learning architectures, the field has dynamically shifted its focus towards customizing these architectures for various natural language processing tasks, including text classification problems. For a newcomer, it might be impossible to realize that for some text classification problems, the traditional approach is still competitive. This research analyzes the competitiveness of BoW-based representations in different text-classification competitions run in English, Spanish, and Italian. To analyze the performance of these BoW-based representations, we participated in 12 text classification international competitions, summing up 24 tasks comprising five English tasks, seven in Italian, and twelve in Spanish. The results show that the proposed BoW representations have a difference of just 10% w.r.t. the competition winner and less than 2% in three tasks corresponding to author profiling. BoW outperforms BERT solutions and dominates in author profiling tasks. • We characterize a number of tasks where Bag-of-Words (BoW) models are still competitive w.r.t. neural approaches. • BoW models can be both lexical and semantic, with transparent models and explainable predictions. • BoW models dominate author profiling tasks. • We develop a fully documented open-source implementation ready to use and test.