Impact of Different Embeddings on Text Classification

Text representation has consistently posed significant challenges in the field of natural language processing. Features such as frequency, position in the text, and co-occurrence of words are used extensively to design various text representation techniques. There are different techniques currently used for text representation. Some are the Bag of words, TF-IDF, Word2vec, GloVe, and FastText. Normally, frequency-based text representations, which convert each word to a single number, are paired with machine learning algorithms, while context-based word embeddings that convert each word to n-dimensional vectors are used with neural network classifiers. This paper examines both frequency and context-based word embeddings on multiple machine learning and neural network-based classifiers and compares the results. The experimental results conclude that Bag of words and TF-IDF (frequency-based) have higher accuracy with ML classifiers such as Logistic Regression and decision tree; Word2vec, GloVe, and Fast Text (context-based) have performed extremely well for neural network-based classifiers such as LSTM, DNN, and GRU.