Detecting AI-Generated Social Media Comments Using BERT-Extracted Semantic Style Features

The rapid rise of artificial intelligence (AI) in generating synthetic content has raised significant concerns regarding the spread of fake news and political propaganda, especially in the context of India's 2024 elections. Identifying human-written comments and tweets amidst AI-generated ones is critical for mitigating such challenges. This paper proposes a method for detecting AI-generated comment by fine-tuning a BERT (Bidirectional Encoder Representations from Transformers) model for classification tasks. The BERT model is fine-tuned on a custom AI text classification dataset, incorporating a neural network classification head to distinguish between machine-generated text (MGT) and human-written text (HWT). A custom dataset of YouTube comments, consisting of both human-written and AI-generated comments, is constructed to evaluate the model's performance in real-world social media scenarios. The model is fine-tuned on this dataset to detect AI-generated comments in social media discussions. To further analyze the discriminative capabilities of the model, Principal Component Analysis (PCA) is applied to visualize the embeddings. The results indicate that the fine-tuned BERT model effectively captures semantic patterns and separates AI comments and human comments into distinct regions in the embedding space, with a clear decision boundary between the two classes. This approach demonstrates the potential of fine-tuned transformer-based models for detecting AI-generated text and offers a robust solution to countering AI-driven misinformation on social media platforms.