Construction of an Artificial Intelligence Model and Application for an Automatic Recognition of Traditional Chinese Medicine Herbals Based on Convolutional Neural Networks

Background Conventional methods for identifying traditional Chinese medicine (TCM) herbals mainly rely on subjective experiences, making it difficult to meet the needs for accurate classification and identification. Objective This study aims to develop an artificial intelligence model and a desktop application capable of automatically recognizing 163 types of TCM herbals based on convolutional neural networks (CNN) . Methods From January 2020 to June 2024, data from two datasets of 163 TCM herbals were collected for training, validation, and testing of the deep learning model. The performance of the CNN model was evaluated for the accuracy, sensitivity, specificity, precision, area under the receiver operating characteristic (ROC) curve (AUC), and F1 score. After model training, an application was developed using PyQt5 technology for convenient clinical use. Results A total of 276 767 images were included in this study. Five models, including EfficientNetB0, ResNet50, MobileNetV3, VGG19, and ResNet18, were developed. After comparing their performance, the EfficientNetB0 model achieved the highest accuracy (99.0%) and AUC (0.994 2) in the validation dataset, and it was selected as the optimal model. In the test dataset, the EfficientNetB0 model achieved an accuracy of 99.0%, sensitivity of 99.0%, specificity of 100.0%, and an AUC of 1.0 across all categories, demonstrating an excellent performance. Conclusion The deep learning model developed based on CNN can quickly and accurately recognize 163 types of TCM herbals with high sensitivity and recognition capability, thus providing a robust support for physicians to accurately identify TCM herbals.