1299-GRM Decoding Health Prompts in AI: A Comparative Machine Learning Study

Automated detection of health-related content in conversational AI datasets is essential for population health surveillance and clinical intelligence. This study evaluates six machine learning classifiers: Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Random Forest (RF), Logistic Regression (LR), Decision Tree (DT), and Naïve Bayes (NB) on a corpus of 16,523 annotated chat prompts. Features include TF-IDF n-grams (unigrams through trigrams), Latent Dirichlet Allocation (LDA) topics, and BERTopic clusters. Chi-square feature selection (SelectKBest) is applied to optimize the feature space. Using 3-fold stratified cross-validation, the optimal configuration — Uni+Bi+Tri+LDA with chi-square selection (k=3,000, N=4,000) yields F1=0.775 and AUC=0.941, demonstrating that principled feature selection substantially elevates classification performance.

Authors

Institutions

Publication Details

Journal
DigitalCommons - Kennesaw State University (Kennesaw State University)
Published
2026-11-20
Primary Topic
Text and Document Classification Technologies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

1299-GRM Decoding Health Prompts in AI: A Comparative Machine Learning Study

Olorunsogo Bidemi Adeoye, Amir Karami
DigitalCommons - Kennesaw State University (Kennesaw State University)
Text and Document Classification Technologies
article

1299-GRM Decoding Health Prompts in AI: A Comparative Machine Learning Study

Olorunsogo Bidemi Adeoye, Amir Karami
article en

Abstract

Automated detection of health-related content in conversational AI datasets is essential for population health surveillance and clinical intelligence. This study evaluates six machine learning classifiers: Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Random Forest (RF), Logistic Regression (LR), Decision Tree (DT), and Naïve Bayes (NB) on a corpus of 16,523 annotated chat prompts. Features include TF-IDF n-grams (unigrams through trigrams), Latent Dirichlet Allocation (LDA) topics, and BERTopic clusters. Chi-square feature selection (SelectKBest) is applied to optimize the feature space. Using 3-fold stratified cross-validation, the optimal configuration — Uni+Bi+Tri+LDA with chi-square selection (k=3,000, N=4,000) yields F1=0.775 and AUC=0.941, demonstrating that principled feature selection substantially elevates classification performance.

DigitalCommons - Kennesaw State University (Kennesaw State University)
Kennesaw State University (US)
Text and Document Classification Technologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon