Tongue Image Segmentation and Constitution Identification with Deep Learning

Traditional Chinese medicine (TCM) gathers patient information through inspection, olfaction, inquiry, and palpation, analyzing and interpreting the data to make a diagnosis and offer appropriate treatment. Traditionally, the interpretation of this information relies heavily on the physician’s personal knowledge and experience. However, diagnostic outcomes can vary depending on the physician’s clinical experience and subjective judgment. This study employs AI methods to focus on localized tongue assessment, developing an automatic tongue body segmentation using the deep learning network “U-Net” through a series of optimization processes applied to tongue surface images. Furthermore, “ResNet34” is utilized for the identification of “cold”, “neutral”, and “hot” constitutions, creating a system that enhances the consistency and reliability of diagnostic results related to the tongue. The final results demonstrate that the AI interpretation accuracy of this system reaches the diagnostic level of junior TCM practitioners (those who have passed the TCM practitioner assessment with ≤5 years of experience). The framework and findings of this study can serve as (1) a foundational step for the future integration of pulse information and electronic medical records, (2) a tool for personalized preventive medicine, and (3) a training resource for TCM students learning to diagnose tongue constitutions such as “cold”, “neutral”, and “hot”.