Machine Learning–Based Detection of Prophylactic Indication Mentions for Stress-Related Mucosal Disease in Chinese Clinical Progress Notes: Text Classification Study

Abstract Background Machine learning has been extensively applied in medical prediction; however, existing machine-assisted pharmacotherapy review systems predominantly rely on structured, codifiable data while neglecting unstructured natural language clinical narratives, thereby allowing latent medication risks to remain undetected. Objective To address the limitations of structured data–based review, this study proposes and validates a machine learning framework grounded in text classification for the automated identification of “prophylactic” vs “nonprophylactic” medication indications within clinical narratives. As a result, it aims to assist in the automated detection of prophylactic indication mentions for stress-related mucosal disease in clinical narratives as a preliminary step toward comprehensive medication-use review. Methods A total of 3695 progress notes were independently annotated by 2 clinical pharmacists to construct a binary dataset for classifying prophylactic medication indication for stress-related mucosal disease, yielding an interannotator agreement κ of 0.8934. Comparative experiments were conducted using traditional machine learning models, including logistic regression, support vector machine, random forest, CatBoost, and LightGBM, alongside TextCNN and a Chinese pretrained BERT model. The dataset was split into training, validation, and test sets on a patient-wise basis, and model evaluation was conducted on the independent test set. SHAP (Shapley additive explanations) analysis was used to achieve model interpretability and elucidate decision-making processes, ultimately enabling automated indication detection of prophylactic proton pump inhibitor administration. Results All models and strategies demonstrated efficacy in medical text classification tasks. In a single-center cohort of 66 patients (3695 notes, 11.0% positive), the CatBoost model with balanced class weights achieved a note-level F 1 -score of 0.727 (precision=0.693; recall=0.765) and a patient-level F 1 -score of 0.889 (precision=0.800; recall=1.000) on the independent test set. At the note level, CatBoost achieved a higher F 1 -score than TextCNN ( F 1 -score=0.718) and BERT ( F 1 -score=0.628). Conclusions In summary, the text classifier constructed through machine learning and natural language processing effectively extracts prophylactic medication indications for stress-related mucosal disease from unstructured progress notes. Combined with patient-level data partitioning, standardized annotation protocols, and an ethics-compliant framework, this approach provides an interpretable preliminary screening tool for identifying potential prophylactic indications to support subsequent pharmacist adjudication.