Empirical Insight into Question Similarity with Embedding Approaches and Learning Ecosystems

Question similarity estimation is crucial in various natural language processing (NLP) applications, predominately in community QA platforms and information retrieval, for building context-aware NLP tasks and steering improved search accuracy and personalizing. The state-of-the-art approaches adapted to numerous methods range from traditional feature-based methods to state-of-the-art deep learning models, with linguistic features, syntactic structures, and semantic representations capturing the similarity between pairs of questions. With this intent, in this paper, we are conducting an empirical study to empirically investigate the proposed similarity estimation pipeline, including dataset selection, feature engineering techniques, and evaluation metrics. Each pipeline is uniquely constructed with ML and DL-based classification techniques in connection with NLP-based word embedding schemes. The datasets are adapted to training and testing Our empirical results highlight the impact of training dataset size, domain specificity, and feature representations on the performance of question similarity models. Our analysis found that GloVe Twitter 200 (GVTW200) embeddings consistently outperformed others in question similarity tasks, with higher Accuracy and AUC compared to TF-IDF and GloVe Wiki 100 (GVWI100). GVTW200 was the most reliable across models. By combining semantic features from pre-trained embeddings with syntactic features like question length and token overlaps, we achieved robust results. KNN, EXTR, and SVCR models showed the highest accuracy and consistency, with advanced metrics like fuzz ratios further enhancing performance for precise similarity estimation.