1299-GRM Decoding Health Prompts in AI: A Comparative Machine Learning Study
Automated detection of health-related content in conversational AI datasets is essential for population health surveillance and clinical intelligence. This study evaluates six machine learning classifiers: Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Random Forest (RF), Logistic Regression (LR), Decision Tree (DT), and Naïve Bayes (NB) on a corpus of 16,523 annotated chat prompts. Features include TF-IDF n-grams (unigrams through trigrams), Latent Dirichlet Allocation (LDA) topics, and BERTopic clusters. Chi-square feature selection (SelectKBest) is applied to optimize the feature space. Using 3-fold stratified cross-validation, the optimal configuration — Uni+Bi+Tri+LDA with chi-square selection (k=3,000, N=4,000) yields F1=0.775 and AUC=0.941, demonstrating that principled feature selection substantially elevates classification performance.
Authors
- Olorunsogo Bidemi Adeoye (ORCID: https://orcid.org/0000-0002-9239-4515)
- Amir Karami (ORCID: https://orcid.org/0000-0003-1936-7497)
Institutions
- Kennesaw State University (US)
Publication Details
- Journal
- DigitalCommons - Kennesaw State University (Kennesaw State University)
- Published
- 2026-11-20
- Primary Topic
- Text and Document Classification Technologies
- Type
- article
- Field-Weighted Citation Impact
- 0.00