Efficient Duplicate Question Detection

Identifying duplicate questions in large online forums is crucial for improving user experience and operational efficiency in customer support. Traditional approaches, such as LSTM-based models, have been effective but are now outperformed by newer Transformer-based architectures. This paper examines the performance of Sentence-BERT models on the Quora Question Pairs dataset, comparing it to the winning Siamese LSTM model from the 2017 Kaggle competition. Our results show that Sentence-BERT models achieve superior accuracy and F1 scores, highlighting their efficiency and reduced computational overhead. This research underscores the advancements in NLP and the potential of Transformer models to enhance duplicate question detection in various applications.